Applying a delta update consists of two parts:
1. The request to the delta server for the delta payload (an rsync batch file, plus some prepended Docker metadata). The response is a redirect to a URL that contains the delta (currently S3).
2. The request for the actual download of the delta. The response is streamed directly to rsync, which applies it onto the mounted root filesystem of the final image.
The first step may take a while as it may trigger the generation of the delta if the request is the first one for this combination of src/dest image and the images are large. If the request times out, either because of the delta server taking too long to respond or bad network, the Supervisor automatically schedules a retry to be performed after a while.
Currently, similar behaviour applies to the second step as well -- if the request fails, we immediately bail out and the Supervisor schedules a retry of the whole process (i.e. from step 1). But in this case it means we might have downloaded and applied some or most of the delta when a socket timeout occurs causing us to start all over again, wasting time and bandwidth.
This commit splits the process into the two discreet steps and improves the behaviour on the second step. Specifically:
- makes the Supervisor try to resume the delta download request several times before it bails out and starts the process all over again.
- removes arbitrary timeout which applied over the whole process and meant some deltas would never manage to be applied (because of large delta size and low network bandwidth).
- makes sure any launched rsync processes always exit and any opened streams consumed and closed.
Most of the improvements are in the two dependencies linked below -- `resumable-request` and `node-docker-delta` -- and this commit merely combines the updated versions of these modules.
Change-Type: minor
Connects-To: #140
Depends-On: https://github.com/resin-io/node-docker-delta/pull/19
Depends-On: https://github.com/resin-io-modules/resumable-request/pull/2
This makes the Async suffix for docker functions unnecessary. It also allows us to remove dockerode as an
explicit dependency.
Change-Type: minor
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
This allows us to also remove a few npm dependencies and the docker compose binary.
Change-Type: major
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
This avoids problems when updating the supervisor on an older OS, where the VPN and other
host services still require config.json to have an apiKey field to authenticate.
Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
Hand over authentication credentials to the docker engine
Fetch an access token from the API if possible and hand it over to the delta server
Change-Type: minor
Signed-off-by: Andreas Fitzek <andreas@resin.io>
Also remove versionist.conf.js as it's no longer necessary (versionist now
defaults to its internal versionist.conf).
Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
Resin OS 2.X removes the use of compressed modules, which was the initial
motivation for us to bind mount kmod into user containers (as Debian distros
don't include support for compressed modules).
This is a breaking change, but we still keep bind mounting on devices that are
on 1.X to ensure we don't break apps currently relying on the feature.
Implementation note: some functions in device.coffee have been refactored to
extract (DRY) a memoization procedure for Promise-returning functions.
`device.getOSVersion()` now also memoizes its result.
Change-Type: major
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
We use the default versionist configuration.
Changelog has been edited to match the versionist format as much as possible.
Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
* Use appId in dependent app assets tar path, and only create the tar if it doesn't exist
* Support AUFS by upgrading node-docker-delta to 1.0.0 and docker-toolbelt to 1.3.0
busybox's init uses /etc/inittab for configuration just like sysvinit,
however it doesn't use any runlevels.
the tty part of inittab is appended to "/dev/", and it becomes connected
to the stdout of the spawned process
Signed-off-by: Petros Angelatos <petrosagg@gmail.com>
This build strategy lends itself to how Rockerfiles work. In the build
Dockerfile all the build utilities (e.g gcc, python) are installed and
run the build process to produce some build artifacts. There are two
build Dockerfiles, one for the nodejs part and one for the golang part.
The build artifacts of these are combined into the runtime Dockerfile.
For all this to work there is some minimal glue implemented in the
Makefile.
Part of this commit is a switch of the base image the runtime is based
on to the minimal OpenEmbedded one produced by #198
Signed-off-by: Petros Angelatos <petrosagg@gmail.com>
- Validate the options in the YAML file
- Define bind mounts for each service as in Resin apps
- Keep the modified compose file inside the supervisor's /data folder
- Fix error reporting in the first stage of "up"
this version fixes the issue where failed attempts to apply the delta
result in a lot of docker images on the disk, which could also
potentially lead to out of space issues
Signed-off-by: Petros Angelatos <petrosagg@gmail.com>
