Use a local supervisor image for the dind supervisor

We change the Makefile and dinctl so that instead of having to deploy and then pull a supervisor image, we save it from the local
docker images and load it inside the dind container.

We also fix dind so that it uses the dind hack script from the docker repo as official dind images do (this avoids breaking the
permissions of /dev/pts/ptmx in the host as in https://github.com/kubernetes/kubernetes/issues/18230).

Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>

.gitignore

@@ -20,3 +20,4 @@ Dockerfile.runtime.*
 /build/
 /dist/
 tools/dind/config/services/docker.service.d/proxy.conf
+tools/dind/supervisor-image.tar

Makefile

@@ -9,7 +9,7 @@
 # * base - builds the "base" component (a yocto builder with the output rootfs at /dest)
 # * gosuper - builds the "gosuper" component (a golang image with the Go supervisor component at /go/bin/gosuper and /build/gosuper)
 # * nodesuper - builds the node component, with the node_modules and src at /usr/src/app and /build (also includes a rootfs-overlay there)
-# * supervisor-dind: build the development docker-in-docker supervisor that run-supervisor uses
+# * supervisor-dind: build the development docker-in-docker supervisor that run-supervisor uses (requires a SUPERVISOR_IMAGE to be available locally)
 #
 # Variables for build targets:
 # * ARCH: amd64/rpi/i386/armv7hf/armel/aarch64 architecture for which to build the supervisor - default: amd64
@@ -19,13 +19,13 @@
 # * DOCKER_BUILD_OPTIONS: Additional options for docker build, like --cache-from parameters
 #
 # Test/development targets:
-# * run-supervisor, stop-supervisor - build and start or stop a docker-in-docker resin-supervisor (requires aufs and ability to run privileged containers)
+# * run-supervisor, stop-supervisor - build and start or stop a docker-in-docker resin-supervisor (requires aufs, ability to run privileged containers, and a SUPERVISOR_IMAGE to be available locally)
 # * format-gosuper, test-gosuper - build a gosuper image and run formatting or unit tests
 # * test-integration - run an integration test (see gosuper/supertest). Requires a docker-in-docker supervisor to be running
 #
 # Variables for test/dev targets:
 # * IMAGE: image to build and run (either for run-supervisor or test-gosuper/integration)
-# * SUPERVISOR_IMAGE: In run-supervisor, the supervisor image to run inside the docker-in-docker image
+# * SUPERVISOR_IMAGE: In run-supervisor and supervisor-dind, the supervisor image to run inside the docker-in-docker image
 # * PRELOADED_IMAGE: If true, will preload user app image from tools/dev/apps.json and bind mount apps.json into the docker-in-docker supervisor
 # * SUPERVISOR_EXTRA_MOUNTS: Additional bind mount flags for the docker-in-docker supervisor
 # * PASSWORDLESS_DROPBEAR: For run-supervisor - start a passwordless ssh daemon in the docker-in-docker supervisor
@@ -120,7 +120,10 @@ ${DOCKERD_PROXY}:
 		touch ${DOCKERD_PROXY}; \
 	fi
 
-supervisor-dind: ${DOCKERD_PROXY}
+supervisor-tar:
+	docker save --output tools/dind/supervisor-image.tar $(SUPERVISOR_IMAGE)
+
+supervisor-dind: ${DOCKERD_PROXY} supervisor-tar
 	cd tools/dind \
 	&& docker build \
 		$(DOCKER_HTTP_PROXY) \

README.md

@@ -10,23 +10,15 @@ We are using [waffle.io](https://waffle.io) to manage our tickets / issues, so i
 
 ## Running supervisor locally
 
-### Deploy your local version to a Docker registry
+### Build a local supervisor image
 
-We'll show how to use the DockerHub registry, but any other can be specified as part of the `IMAGE` variable.
-
-If you haven't done so yet, login to the registry:
+Build the supervisor with a specific repo and tag, e.g.
 ```bash
-docker login
-```
-Use your username and password as required.
-
-Then build the supervisor and deploy it to a specific repo and tag, e.g.
-```bash
-./tools/dev/dindctl deploy --image username/resin-supervisor:master --arch amd64
+./tools/dev/dindctl build --image username/resin-supervisor:master --arch amd64
 ```
 
-This will build the Supervisor docker image and upload it to dockerhub. You can use other registries by adding to the
-image name e.g. `myregistry.com/username/resin-supervisor:master`.
+This will build the Supervisor docker image locally. If you then run `docker images` you should see the repo/tag you
+set there.
 
 ### Set up config.json
 
@@ -74,7 +66,8 @@ Ensure your kernel supports aufs (in Ubuntu, install `linux-image-extra-$(uname
 ./tools/dev/dindctl run --image username/resin-supervisor:master
 ```
 
-This will setup a docker-in-docker instance with an image that runs the supervisor image.
+This will setup a docker-in-docker instance with an image that runs the supervisor image. The image has to be available
+locally, either because you built it as described above, or because you pulled it before running `dindctl run`.
 
 If you want to develop and test your changes, you can run:
 

tools/dev/dindctl

@@ -4,22 +4,22 @@
 # faster development iterations by bind-mounting the local './dist' directly into the running
 # supervisor container.
 #
-# Setting the '--mount-nm' flag in either 'run' or 'deployrun' action will bind-mount
+# Setting the '--mount-nm' flag in either 'run' or 'buildrun' action will bind-mount
 # './node_modules/' into the running supervisor. In this case, it's up to the developer
 # to make sure that the correct dependencies are installed.
 #
 # Usage: dindctl action [options]
 #
 # Actions:
-# 	deploy					build and deploy local supervisor image - you can override registry/image name with --image.
-# 	run [options]			build dind host container, run it (with name resin_supervisor_1), then pull the configured supervisor image into the dind host and run it.
-# 	deployrun [options]		run 'deploy' and then immediately 'run' the deployed container.
+# 	build					build local supervisor image - you can override image name with --image.
+# 	run [options]			build dind host container, run it (with name resin_supervisor_1), which will include the specified supervisor image and run it.
+# 	buildrun [options]		run 'build' and then immediately 'run' the built container.
 # 	refresh					recompile sources in './src' and restart supervisor container on dind host - requires --mount-dist in order to work properly.
 # 	logs [-f]				print out supervisor log files - use '-f' to follow instead, or any other arguments you'd send to journalctl.
 # 	stop					stop dind supervisor host container.
 # Options:
 # 	--arch | -a  [arch]		architecture of the supervisor to build (default: amd64 )
-# 	--image | -i [image]	image name for supervisor image to build/deploy/use ( default: resin/$ARCH-supervisor:master )
+# 	--image | -i [image]	image name for supervisor image to build/use ( default: resin/$ARCH-supervisor:master )
 # 	--dind-image [image]	image name for the dind host container
 # 	--mount-dist			bind-mount './dist/' (where webpack stores the built js) from local development environment into supervisor container.
 # 	--mount-nm				bind-mount './node_modules/' from local development environment into supervisor container.
@@ -88,15 +88,12 @@ function parseOptions {
 	done
 }
 
-function deploySupervisor {
+function buildSupervisor {
 	echo "Building and deploying for architecture $ARCH and tagging as $IMAGE"
 	make -C "$SUPERVISOR_BASE_DIR" \
 		ARCH="$ARCH" \
 		IMAGE="$SUPERVISOR_IMAGE" \
-		supervisor \
-	&& make -C "$SUPERVISOR_BASE_DIR" \
-		IMAGE="$SUPERVISOR_IMAGE" \
-		deploy
+		supervisor
 }
 
 function buildSupervisorSrc {
@@ -124,7 +121,7 @@ function runDind {
 		SUPERVISOR_IMAGE="$SUPERVISOR_IMAGE" \
 		PASSWORDLESS_DROPBEAR="$PASSWORDLESS_DROPBEAR" \
 		SUPERVISOR_EXTRA_MOUNTS="$SUPERVISOR_EXTRA_MOUNTS" \
-		IMAGE="$DIND_IMAGE"
+		IMAGE="$DIND_IMAGE" \
 		run-supervisor
 }
 
@@ -145,14 +142,14 @@ if [ "$action" = "logs" ]; then
 else
 	parseOptions "$@"
 	case $action in
-		deploy)
-			deploySupervisor
+		build)
+			buildSupervisor
 			;;
 		run)
 			runDind
 			;;
-		deployrun)
-			deploySupervisor && runDind
+		buildrun)
+			buildSupervisor && runDind
 			;;
 		refresh)
 			refreshSupervisorSrc

tools/dind/Dockerfile

@@ -26,6 +26,10 @@ RUN apt-key adv --keyserver hkp://pgp.mit.edu:80 --recv-keys 58118E89F3A912897C0
 	&& apt-get --force-yes install docker-engine=${DOCKER_VERSION}-0~${RELEASE_NAME} \
 	&& rm -rf /var/lib/apt/lists/*
 
+ENV DIND_COMMIT 3b5fac462d21ca164b3778647420016315289034
+RUN curl -sL https://raw.githubusercontent.com/docker/docker/${DIND_COMMIT}/hack/dind > /usr/bin/dind \
+	&& chmod +x /usr/bin/dind
+
 RUN passwd -d root
 
 # Change os release to a resin-sync compatible one
@@ -40,3 +44,5 @@ COPY resin-vars vpn-init /usr/src/app/
 RUN if [ "$PASSWORDLESS_DROPBEAR" = "true" ]; then sed -i 's/\(DROPBEAR_EXTRA_ARGS=\).*/\1"-B"/' /etc/default/dropbear; fi
 
 RUN systemctl enable resin-supervisor-dind
+
+COPY supervisor-image.tar /usr/src/

tools/dind/config/services/docker.service.d/aufs.conf

@@ -1,3 +1,3 @@
 [Service]
 ExecStart=
-ExecStart=/usr/bin/docker daemon --storage-driver aufs --host=fd://
+ExecStart=/usr/bin/dind /usr/bin/docker daemon --storage-driver aufs --host=fd://

tools/dind/config/services/resin-supervisor-dind.service

@@ -9,7 +9,7 @@ WorkingDirectory=/usr/src/app
 EnvironmentFile=/usr/src/app/config/env
 EnvironmentFile=/usr/src/app/config/localenv
 ExecStartPre=/bin/bash -c 'if [ "${PRELOADED_IMAGE}" == "true" ]; then /usr/bin/docker pull $(jq ".[0].imageId" ${APPS_PATH}); fi'
-ExecStartPre=/usr/bin/docker pull ${SUPERVISOR_IMAGE}
+ExecStartPre=/usr/bin/docker load --input /usr/src/supervisor-image.tar
 ExecStartPre=-/usr/bin/docker kill resin_supervisor
 ExecStartPre=-/usr/bin/docker rm resin_supervisor
 ExecStartPre=-/bin/touch /etc/resolv.conf