Fixes issues on systems where mount files if the target doesn't exist (seems to happen on
boards that run docker on overlayfs).
Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
Applying a delta update consists of two parts:
1. The request to the delta server for the delta payload (an rsync batch file, plus some prepended Docker metadata). The response is a redirect to a URL that contains the delta (currently S3).
2. The request for the actual download of the delta. The response is streamed directly to rsync, which applies it onto the mounted root filesystem of the final image.
The first step may take a while as it may trigger the generation of the delta if the request is the first one for this combination of src/dest image and the images are large. If the request times out, either because of the delta server taking too long to respond or bad network, the Supervisor automatically schedules a retry to be performed after a while.
Currently, similar behaviour applies to the second step as well -- if the request fails, we immediately bail out and the Supervisor schedules a retry of the whole process (i.e. from step 1). But in this case it means we might have downloaded and applied some or most of the delta when a socket timeout occurs causing us to start all over again, wasting time and bandwidth.
This commit splits the process into the two discreet steps and improves the behaviour on the second step. Specifically:
- makes the Supervisor try to resume the delta download request several times before it bails out and starts the process all over again.
- removes arbitrary timeout which applied over the whole process and meant some deltas would never manage to be applied (because of large delta size and low network bandwidth).
- makes sure any launched rsync processes always exit and any opened streams consumed and closed.
Most of the improvements are in the two dependencies linked below -- `resumable-request` and `node-docker-delta` -- and this commit merely combines the updated versions of these modules.
Change-Type: minor
Connects-To: #140
Depends-On: https://github.com/resin-io/node-docker-delta/pull/19
Depends-On: https://github.com/resin-io-modules/resumable-request/pull/2
deploy-to-resin.js had a few references to variables that are no longer defined.
In pr-to-meta-resin.sh, the passing of GitHub credentials was wrongly placed and messy.
As per @Page-'s advice, we use the credentials as part of the remote url when pushing (we don't do it when
cloning to avoid the github password being visible with a `git remote -v`).
We change GITHUB_USERNAME to GITHUB_USER as hub will automatically pick it up, avoiding the need
to pass the credentials when creating the PR.
Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
We mark when the device is rebooting and avoid some steps in the update cycle that change the device
state, similarly to when the device is in local mode, to avoid problems with non-atomic operations.
This doesn't solve *all* the potential scenarios of a reboot happening in the middle of an update, but at least
should prevent the case where we start an app container and reboot the device before saving the containerId, potentially
causing a duplicated container issue.
We also correct the API docs to reflect the 202 response when reboot or shutdown are successful.
Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
We used to store the uuid which would cause the supervisor to not attempt a provisioning even if offline mode
was turned off. This was to avoid preloaded apps being reloaded constantly leaving multiple containers.
We now avoid persisting the uuid, so that when the supervisor goes out of offline mode it can provision
without the need to wipe out the db. We avoid the problem with preloaded apps by not loading them
if there's apps already stored on the db.
(In the future, apps in the db will only represent target state and we can make preloaded apps be reloaded on every
start, but for now we can't do it as long as we store the containerId on the db - deleting an app on the db
means losing track of its containerId and therefore leaving an orphaned container)
Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
This makes the Async suffix for docker functions unnecessary. It also allows us to remove dockerode as an
explicit dependency.
Change-Type: minor
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
We now remove the quotes from the image name in the ExecStartPre of the supervisor dind service,
so that the images are properly pulled.
We also fix the parsing of arguments in dindctl so that it correctly parses all of the arguments (the last one was being missed if it didn't have a parameter).
Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
The test for an exec format error caused a `err.json.trim` is not a function
error so the message shown didn't relate to what the problem actually was.
This makes the test for the exec format error safer.
Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
Up to now, a new debian:jessie build would invalidate the cache for the base and gosuper images, slowing down Circle builds.
Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
We should revisit this if we want to later copy those folders into the host and use Circle's caching system with them, or if https://github.com/moby/moby/issues/32507 gets implemented in which case we can use a bind mounted folder instead.
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
Otherwise the step where we build them doesn't reuse what was built when building the supervisor,
because we're explicitly setting `--cache-from` options.
We also push the images as soon as they're built.
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
Also, avoid building the dind builder twice, and try to get a reusable docker instance.
Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
This makes it easier to use, as it's meant to be used constantly when developing. No need to change PATH
or type `./tools/dev/dindctl`, just run `./dindctl` now.
Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
We change the Makefile and dinctl so that instead of having to deploy and then pull a supervisor image, we save it from the local
docker images and load it inside the dind container.
We also fix dind so that it uses the dind hack script from the docker repo as official dind images do (this avoids breaking the
permissions of /dev/pts/ptmx in the host as in https://github.com/kubernetes/kubernetes/issues/18230).
Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
