ExternalVendorCode/balena-supervisor
1
0
Fork 0
mirror of https://github.com/balena-os/balena-supervisor.git synced 2025-01-31 00:23:57 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Use a local supervisor image for the dind supervisor

Browse Source 
We change the Makefile and dinctl so that instead of having to deploy and then pull a supervisor image, we save it from the local
docker images and load it inside the dind container.

We also fix dind so that it uses the dind hack script from the docker repo as official dind images do (this avoids breaking the
permissions of /dev/pts/ptmx in the host as in https://github.com/kubernetes/kubernetes/issues/18230).

Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
This commit is contained in:
Pablo Carranza Velez 2017-07-10 09:37:36 -07:00
parent 5bfeccccef
commit 392d963348
7 changed files with 35 additions and 35 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.gitignore vendored
View File

@ -20,3 +20,4 @@ Dockerfile.runtime.*
/build/ /build/
/dist/ /dist/
tools/dind/config/services/docker.service.d/proxy.conf tools/dind/config/services/docker.service.d/proxy.conf
tools/dind/supervisor-image.tar

11
Makefile
View File

@ -9,7 +9,7 @@
# * base - builds the "base" component (a yocto builder with the output rootfs at /dest) # * base - builds the "base" component (a yocto builder with the output rootfs at /dest)
# * gosuper - builds the "gosuper" component (a golang image with the Go supervisor component at /go/bin/gosuper and /build/gosuper) # * gosuper - builds the "gosuper" component (a golang image with the Go supervisor component at /go/bin/gosuper and /build/gosuper)
# * nodesuper - builds the node component, with the node_modules and src at /usr/src/app and /build (also includes a rootfs-overlay there) # * nodesuper - builds the node component, with the node_modules and src at /usr/src/app and /build (also includes a rootfs-overlay there)
# * supervisor-dind: build the development docker-in-docker supervisor that run-supervisor uses # * supervisor-dind: build the development docker-in-docker supervisor that run-supervisor uses (requires a SUPERVISOR_IMAGE to be available locally)
# #
# Variables for build targets: # Variables for build targets:
# * ARCH: amd64/rpi/i386/armv7hf/armel/aarch64 architecture for which to build the supervisor - default: amd64 # * ARCH: amd64/rpi/i386/armv7hf/armel/aarch64 architecture for which to build the supervisor - default: amd64
@ -19,13 +19,13 @@
# * DOCKER_BUILD_OPTIONS: Additional options for docker build, like --cache-from parameters # * DOCKER_BUILD_OPTIONS: Additional options for docker build, like --cache-from parameters
# #
# Test/development targets: # Test/development targets:
# * run-supervisor, stop-supervisor - build and start or stop a docker-in-docker resin-supervisor (requires aufs and ability to run privileged containers) # * run-supervisor, stop-supervisor - build and start or stop a docker-in-docker resin-supervisor (requires aufs, ability to run privileged containers, and a SUPERVISOR_IMAGE to be available locally)
# * format-gosuper, test-gosuper - build a gosuper image and run formatting or unit tests # * format-gosuper, test-gosuper - build a gosuper image and run formatting or unit tests
# * test-integration - run an integration test (see gosuper/supertest). Requires a docker-in-docker supervisor to be running # * test-integration - run an integration test (see gosuper/supertest). Requires a docker-in-docker supervisor to be running
# #
# Variables for test/dev targets: # Variables for test/dev targets:
# * IMAGE: image to build and run (either for run-supervisor or test-gosuper/integration) # * IMAGE: image to build and run (either for run-supervisor or test-gosuper/integration)
# * SUPERVISOR_IMAGE: In run-supervisor, the supervisor image to run inside the docker-in-docker image # * SUPERVISOR_IMAGE: In run-supervisor and supervisor-dind, the supervisor image to run inside the docker-in-docker image
# * PRELOADED_IMAGE: If true, will preload user app image from tools/dev/apps.json and bind mount apps.json into the docker-in-docker supervisor # * PRELOADED_IMAGE: If true, will preload user app image from tools/dev/apps.json and bind mount apps.json into the docker-in-docker supervisor
# * SUPERVISOR_EXTRA_MOUNTS: Additional bind mount flags for the docker-in-docker supervisor # * SUPERVISOR_EXTRA_MOUNTS: Additional bind mount flags for the docker-in-docker supervisor
# * PASSWORDLESS_DROPBEAR: For run-supervisor - start a passwordless ssh daemon in the docker-in-docker supervisor # * PASSWORDLESS_DROPBEAR: For run-supervisor - start a passwordless ssh daemon in the docker-in-docker supervisor
@ -120,7 +120,10 @@ ${DOCKERD_PROXY}:
touch ${DOCKERD_PROXY}; \ touch ${DOCKERD_PROXY}; \
fi fi
supervisor-dind: ${DOCKERD_PROXY} supervisor-tar:
docker save --output tools/dind/supervisor-image.tar $(SUPERVISOR_IMAGE)
supervisor-dind: ${DOCKERD_PROXY} supervisor-tar
cd tools/dind \ cd tools/dind \
&& docker build \ && docker build \
$(DOCKER_HTTP_PROXY) \ $(DOCKER_HTTP_PROXY) \

21
README.md
View File

@ -10,23 +10,15 @@ We are using [waffle.io](https://waffle.io) to manage our tickets / issues, so i
## Running supervisor locally ## Running supervisor locally
### Deploy your local version to a Docker registry ### Build a local supervisor image
We'll show how to use the DockerHub registry, but any other can be specified as part of the `IMAGE` variable. Build the supervisor with a specific repo and tag, e.g.
If you haven't done so yet, login to the registry:
```bash ```bash
docker login ./tools/dev/dindctl build --image username/resin-supervisor:master --arch amd64
```
Use your username and password as required.
Then build the supervisor and deploy it to a specific repo and tag, e.g.
```bash
./tools/dev/dindctl deploy --image username/resin-supervisor:master --arch amd64
``` ```
This will build the Supervisor docker image and upload it to dockerhub. You can use other registries by adding to the This will build the Supervisor docker image locally. If you then run `docker images` you should see the repo/tag you
image name e.g. `myregistry.com/username/resin-supervisor:master`. set there.
### Set up config.json ### Set up config.json
@ -74,7 +66,8 @@ Ensure your kernel supports aufs (in Ubuntu, install `linux-image-extra-$(uname
./tools/dev/dindctl run --image username/resin-supervisor:master ./tools/dev/dindctl run --image username/resin-supervisor:master
``` ```
This will setup a docker-in-docker instance with an image that runs the supervisor image. This will setup a docker-in-docker instance with an image that runs the supervisor image. The image has to be available
locally, either because you built it as described above, or because you pulled it before running `dindctl run`.
If you want to develop and test your changes, you can run: If you want to develop and test your changes, you can run:

27
tools/dev/dindctl
View File

@ -4,22 +4,22 @@
# faster development iterations by bind-mounting the local './dist' directly into the running # faster development iterations by bind-mounting the local './dist' directly into the running
# supervisor container. # supervisor container.
# #
# Setting the '--mount-nm' flag in either 'run' or 'deployrun' action will bind-mount # Setting the '--mount-nm' flag in either 'run' or 'buildrun' action will bind-mount
# './node_modules/' into the running supervisor. In this case, it's up to the developer # './node_modules/' into the running supervisor. In this case, it's up to the developer
# to make sure that the correct dependencies are installed. # to make sure that the correct dependencies are installed.
# #
# Usage: dindctl action [options] # Usage: dindctl action [options]
# #
# Actions: # Actions:
# deploy build and deploy local supervisor image - you can override registry/image name with --image. # build build local supervisor image - you can override image name with --image.
# run [options] build dind host container, run it (with name resin_supervisor_1), then pull the configured supervisor image into the dind host and run it. # run [options] build dind host container, run it (with name resin_supervisor_1), which will include the specified supervisor image and run it.
# deployrun [options] run 'deploy' and then immediately 'run' the deployed container. # buildrun [options] run 'build' and then immediately 'run' the built container.
# refresh recompile sources in './src' and restart supervisor container on dind host - requires --mount-dist in order to work properly. # refresh recompile sources in './src' and restart supervisor container on dind host - requires --mount-dist in order to work properly.
# logs [-f] print out supervisor log files - use '-f' to follow instead, or any other arguments you'd send to journalctl. # logs [-f] print out supervisor log files - use '-f' to follow instead, or any other arguments you'd send to journalctl.
# stop stop dind supervisor host container. # stop stop dind supervisor host container.
# Options: # Options:
# --arch | -a [arch] architecture of the supervisor to build (default: amd64 ) # --arch | -a [arch] architecture of the supervisor to build (default: amd64 )
# --image | -i [image] image name for supervisor image to build/deploy/use ( default: resin/$ARCH-supervisor:master ) # --image | -i [image] image name for supervisor image to build/use ( default: resin/$ARCH-supervisor:master )
# --dind-image [image] image name for the dind host container # --dind-image [image] image name for the dind host container
# --mount-dist bind-mount './dist/' (where webpack stores the built js) from local development environment into supervisor container. # --mount-dist bind-mount './dist/' (where webpack stores the built js) from local development environment into supervisor container.
# --mount-nm bind-mount './node_modules/' from local development environment into supervisor container. # --mount-nm bind-mount './node_modules/' from local development environment into supervisor container.
@ -88,15 +88,12 @@ function parseOptions {
done done
} }
function deploySupervisor { function buildSupervisor {
echo "Building and deploying for architecture $ARCH and tagging as $IMAGE" echo "Building and deploying for architecture $ARCH and tagging as $IMAGE"
make -C "$SUPERVISOR_BASE_DIR" \ make -C "$SUPERVISOR_BASE_DIR" \
ARCH="$ARCH" \ ARCH="$ARCH" \
IMAGE="$SUPERVISOR_IMAGE" \ IMAGE="$SUPERVISOR_IMAGE" \
supervisor \ supervisor
&& make -C "$SUPERVISOR_BASE_DIR" \
IMAGE="$SUPERVISOR_IMAGE" \
deploy
} }
function buildSupervisorSrc { function buildSupervisorSrc {
@ -124,7 +121,7 @@ function runDind {
SUPERVISOR_IMAGE="$SUPERVISOR_IMAGE" \ SUPERVISOR_IMAGE="$SUPERVISOR_IMAGE" \
PASSWORDLESS_DROPBEAR="$PASSWORDLESS_DROPBEAR" \ PASSWORDLESS_DROPBEAR="$PASSWORDLESS_DROPBEAR" \
SUPERVISOR_EXTRA_MOUNTS="$SUPERVISOR_EXTRA_MOUNTS" \ SUPERVISOR_EXTRA_MOUNTS="$SUPERVISOR_EXTRA_MOUNTS" \
IMAGE="$DIND_IMAGE" IMAGE="$DIND_IMAGE" \
run-supervisor run-supervisor
} }
@ -145,14 +142,14 @@ if [ "$action" = "logs" ]; then
else else
parseOptions "$@" parseOptions "$@"
case $action in case $action in
deploy) build)
deploySupervisor buildSupervisor
;; ;;
run) run)
runDind runDind
;; ;;
deployrun) buildrun)
deploySupervisor && runDind buildSupervisor && runDind
;; ;;
refresh) refresh)
refreshSupervisorSrc refreshSupervisorSrc

6
tools/dind/Dockerfile
View File

@ -26,6 +26,10 @@ RUN apt-key adv --keyserver hkp://pgp.mit.edu:80 --recv-keys 58118E89F3A912897C0
&& apt-get --force-yes install docker-engine=${DOCKER_VERSION}-0~${RELEASE_NAME} \ && apt-get --force-yes install docker-engine=${DOCKER_VERSION}-0~${RELEASE_NAME} \
&& rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/apt/lists/*
ENV DIND_COMMIT 3b5fac462d21ca164b3778647420016315289034
RUN curl -sL https://raw.githubusercontent.com/docker/docker/${DIND_COMMIT}/hack/dind > /usr/bin/dind \
&& chmod +x /usr/bin/dind
RUN passwd -d root RUN passwd -d root
# Change os release to a resin-sync compatible one # Change os release to a resin-sync compatible one
@ -40,3 +44,5 @@ COPY resin-vars vpn-init /usr/src/app/
RUN if [ "$PASSWORDLESS_DROPBEAR" = "true" ]; then sed -i 's/\(DROPBEAR_EXTRA_ARGS=\).*/\1"-B"/' /etc/default/dropbear; fi RUN if [ "$PASSWORDLESS_DROPBEAR" = "true" ]; then sed -i 's/\(DROPBEAR_EXTRA_ARGS=\).*/\1"-B"/' /etc/default/dropbear; fi
RUN systemctl enable resin-supervisor-dind RUN systemctl enable resin-supervisor-dind
COPY supervisor-image.tar /usr/src/

2
tools/dind/config/services/docker.service.d/aufs.conf
View File

@ -1,3 +1,3 @@
[Service] [Service]
ExecStart= ExecStart=
ExecStart=/usr/bin/docker daemon --storage-driver aufs --host=fd:// ExecStart=/usr/bin/dind /usr/bin/docker daemon --storage-driver aufs --host=fd://

2
tools/dind/config/services/resin-supervisor-dind.service
View File

@ -9,7 +9,7 @@ WorkingDirectory=/usr/src/app
EnvironmentFile=/usr/src/app/config/env EnvironmentFile=/usr/src/app/config/env
EnvironmentFile=/usr/src/app/config/localenv EnvironmentFile=/usr/src/app/config/localenv
ExecStartPre=/bin/bash -c 'if [ "${PRELOADED_IMAGE}" == "true" ]; then /usr/bin/docker pull $(jq ".[0].imageId" ${APPS_PATH}); fi' ExecStartPre=/bin/bash -c 'if [ "${PRELOADED_IMAGE}" == "true" ]; then /usr/bin/docker pull $(jq ".[0].imageId" ${APPS_PATH}); fi'
ExecStartPre=/usr/bin/docker pull ${SUPERVISOR_IMAGE} ExecStartPre=/usr/bin/docker load --input /usr/src/supervisor-image.tar
ExecStartPre=-/usr/bin/docker kill resin_supervisor ExecStartPre=-/usr/bin/docker kill resin_supervisor
ExecStartPre=-/usr/bin/docker rm resin_supervisor ExecStartPre=-/usr/bin/docker rm resin_supervisor
ExecStartPre=-/bin/touch /etc/resolv.conf ExecStartPre=-/bin/touch /etc/resolv.conf