This HOWTO describes how to _non_ live-migrate a docker container from one
docker host to another.

** This is an experimental implementation of docker migration, which may affect
your running containers.

0. Install CRIU, p.haul, docker on both nodes

   Besides the packages that are needed to compile and run CRIU and p.haul,
   the specific docker binary that supports checkpoint/restore should be used.

   Refer to step 0 in test/mtouch/HOWTO about the pacekages for CRIU and p.haul.

   The docker version that supports checkpoint and restore can be obtained by

   # git clone https://github.com/huikang/docker.git
   # cd docker.git
   # git checkout cr-combined-1006

   On both nodes, compile and store the the docker binary as

   /usr/bin/docker-1.9.0-dev

   Note that the path above is for now hard-coded in p_haul_docker.py

1. Prepare criu and p.haul on both nodes (adapted from test/mtouch/HOWTO)

   a) CRIU

   * Clone CRIU repository from git://github.com/xemul/criu
     and compile it with 'make'

   * Make _local_ directory for service ($csdir)
   * Start CRIU service by root user
     Note that this step is mandatory. Although runC will start criu in swrk
     mode, p.haul needs to connect to criu service for validating CPU.

   # criu service -o $csdir/log -v4 --daemon

   b) On destination node start the p.haul service

   [dst]# ./p.haul-wrap service
   Starting p.haul rpyc service

   It will go daemon and will print logs on the terminals

3. Run the test container on source node

   a) Start the docker daemon

   # /usr/bin/docker-1.9.0-dev daemon -s aufs

   b) Start the container

   # /usr/bin/docker-1.9.0-dev run -d busybox:latest /bin/sh -c 'i=0; while true; do echo $i >> /foo; i=$(expr $i + 1); sleep 1; done'

   This command will return the container's ID, e.g.,  d78.
   (borrowed from https://criu.org/Docker)

4. Migrate container from source node

   [src]# ./p.haul-wrap client to docker [container ID, e.g., d78]

   to is the ip address of the dst node

   For example:

   [src]# ./p.haul-wrap client 192.168.11.106 docker d78
   192.168.11.106 is the destination node IP and d78 is the container ID


   When the command returns, on the destination node run

   [dst]# /usr/bin/docker-1.9.0-dev ps
   [dst]# /usr/bin/docker-1.9.0-dev exec d78 cat /foo

   to verify the counter is continuously being incremented.

Known limitations.

1. No support from docker python binding
2. Docker daemon has to be restarted on the destination node
