Turned out that disk I/O can be the bottleneck when applying deltas on some devices. When the disk can’t keep up and consume the downloaded delta, there’s memory bloat due to buffering.
The updated version provides far better reliability when the device is under load and pretty much constant memory consumption with any number of concurrent deltas.
Change-Type: patch
The problem was caused by the build picking up 3.2.0 which had a bug, causing an `TypeError: l is not a function`.
Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
We've been using UglifyJS 0.4.6 (the webpack default) so far, but this doesn't support ES6 and some dependency
updates are starting to cause builds to break (e.g. https://github.com/request/request/issues/2772, which also happens to break
my builds in the multicontainer branch).
Here we switch to the latest uglifyjs-webpack-plugin which is designed for ES2015 support.
Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
When sending events to mixpanel, we now use an explicit whitelist for the properties sent with the event, to avoid accidental leakage of any sensitive information.
Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
- Updates resumable-request to 1.0.1
- Updates docker-progress to 2.0.3
- Removes `DEFAULT_DELTA_APPLY_TIMEOUT`; it’s not needed anymore, docker-delta reliably tracks rsync.
- Properly end the update when applying the delta results in an error.
Change-Type: patch
This handy tool uses the resin-sync module to rsync javascript changes into the running container
on a device in the local network. It allows rapid iterations when testing the supervisor on a real device.
Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
2.6.2 has a serious bug which causes the js file to not be included in the published package.
Webpack gave us an unexpected workaround because it will add the .coffee file anyways, but we should
still update to the fixed version.
Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
Applying a delta update consists of two parts:
1. The request to the delta server for the delta payload (an rsync batch file, plus some prepended Docker metadata). The response is a redirect to a URL that contains the delta (currently S3).
2. The request for the actual download of the delta. The response is streamed directly to rsync, which applies it onto the mounted root filesystem of the final image.
The first step may take a while as it may trigger the generation of the delta if the request is the first one for this combination of src/dest image and the images are large. If the request times out, either because of the delta server taking too long to respond or bad network, the Supervisor automatically schedules a retry to be performed after a while.
Currently, similar behaviour applies to the second step as well -- if the request fails, we immediately bail out and the Supervisor schedules a retry of the whole process (i.e. from step 1). But in this case it means we might have downloaded and applied some or most of the delta when a socket timeout occurs causing us to start all over again, wasting time and bandwidth.
This commit splits the process into the two discreet steps and improves the behaviour on the second step. Specifically:
- makes the Supervisor try to resume the delta download request several times before it bails out and starts the process all over again.
- removes arbitrary timeout which applied over the whole process and meant some deltas would never manage to be applied (because of large delta size and low network bandwidth).
- makes sure any launched rsync processes always exit and any opened streams consumed and closed.
Most of the improvements are in the two dependencies linked below -- `resumable-request` and `node-docker-delta` -- and this commit merely combines the updated versions of these modules.
Change-Type: minor
Connects-To: #140
Depends-On: https://github.com/resin-io/node-docker-delta/pull/19
Depends-On: https://github.com/resin-io-modules/resumable-request/pull/2
This makes the Async suffix for docker functions unnecessary. It also allows us to remove dockerode as an
explicit dependency.
Change-Type: minor
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
This allows us to also remove a few npm dependencies and the docker compose binary.
Change-Type: major
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
This avoids problems when updating the supervisor on an older OS, where the VPN and other
host services still require config.json to have an apiKey field to authenticate.
Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
Hand over authentication credentials to the docker engine
Fetch an access token from the API if possible and hand it over to the delta server
Change-Type: minor
Signed-off-by: Andreas Fitzek <andreas@resin.io>
