With the move to v3 target state and the move forward to remove
database ids from the supervisor, we want to ensure the ids are only
used for legacy support (such as within the API). This change renames
the method and sets it as deprecated
It seems that in some cases the supervisor can report
an image without a `status` field leading to a cloud side 401 response.
See #1905 for more details.
Change-type: patch
The check for the docker network supervisor0 assumed that if the
interface supervisor0 existed, then the network would exist too. However this is not
true on the case of docker directory corruption, which would lead to a
loop with `Error: (HTTP code 404) no such network - network supervisor0 not found`.
Change-type: patch
Closes: #1806
The functionality is pretty much the same, so we don't need the two
functions in two different places.
Signed-off-by: Christina Wang <christina@balena.io>
PR #1824 changed app update behavior to test that all images are there
before moving between releases. This check always fails in local mode
since local mode images are handled differently.
This PR fixes local mode again by skipping the check when `localMode` is
set.
Change-type: patch
The strategy has been broken for a while but it was not clear how to
fix it before the changes to image management. This PR fixes application
manager to remove images before downloading the new image. This will
only have an effect on changing images.
Closes: #1233
Change-type: patch
For download-then-kill strategy, this waits for all changing images on the target
release to be available on device before killing the old services. This
will prevent that multicontainer applications get to a state where some
services of the new release start runnning much before others have been
downloaded.
When adding new services to a multicontainer app, the supervisor will
now wait for other changing services to be downloaded before starting
the new service.
Closes: #1812
Change-type: patch
This removes the need for the app module to know about the naming
conventions for networks and volumes since those exist now within the
service itself. This also fixes a small bug where the volume would be
removed before the service itself had been successfully stopped.
Change-type: patch
PR #1749 introduced a bug when pushing local target state. An update to
the [image name normalization](f1bd4b8d9b/src/lib/docker-utils.ts (L81))
failed to consider the local image name format. This results in mangling
of image names in the database, i.e. the image `ubuntu:latest` is stored
as `/ubuntu:latest`. This causes an exception to be returned by the
dockerode `getImage('/ubuntu:latest').inspect()` call.
This sends the supervisor into a crash loop and is shown on the supervisor
journal logs as
```
getaddrinfo ENOTFOUND images
at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:64:26)
```
Unfortunately if this happens on a user device, since the mangled image
name is already on the database, the easiest way to fix is to remove the
supervisor database and let the supervisor recreate it. Deleting the
database should be side effect free.
Change-type: patch
Preparing for the new v3 target state, where the supervisor will make environment
dependent ids optional and rely on using general UUIDs and user known identifiers
for comparison. This PR moves forward in that direction by removing some of those
comparisons for v2 target state.
- imageId to be replaced with imageName
- serviceId to be replace by serviceName
- releaseId to be replaced by commit (future release_uuid)
This is a backwards compatible change, meaning it doesn't completely get rid of
these identifiers (which are still being used by supervisor API and for state
patch), but will not depend on those identifiers for calculating steps to target state.
Change-type: minor
This replaces stored `volatileState` with a more declarative ImageTask API.
An ImageTask stores volatile image state for operations that cannot be
obtained through an engine query, such as fetching and removing an
image, state that can be updated while the task is running.
Image controller methods can now use the `reportEvent` method to create
and update the state of a longer running task.
The image manager module now uses tags instead of docker IDs as the main
way to identify docker images on the engine. That is, if the target
state image has a name `imageName:tag@digest`, the supervisor will always use
the given `imageName` and `tag` (which may be empty) to tag the image on
the engine after fetching. This PR also adds checkups to ensure
consistency is maintained between the database and the engine.
Using tags allows to simplify query and removal operations, since now
removing the image now means removing tags matching the image name.
Before this change the supervisor relied only on information in the
supervisor database, and used that to remove images by docker ID. However, the docker
id is not a reliable identifier, since images retain the same id between
releases or between services in the same release.
List of squashed commits
- Remove custom type NormalizedImageInfo
- Remove dependency on docker-toolbelt
- Use tags to traack supervised images in docker
- Ensure tag removal occurs in sequence
- Only save database image after download confirmed
Relates-to: #1616#1579
Change-type: patch
This PR cleans up testing for supervisor compose modules. It also fixes broken
tests for application manager and removes a lot of dependencies for those tests
on DB and other unnecessary mocks. There are probably a lot of cases that tests
are missing but this should make writing new tests a lot easier.
This PR also creates a new mock dockerode (mockerode) module that should make it
easier to test operations that interact with the engine. All references
to the old mock-dockerode have not yet been removed but that should come
soon in another PR
List of squashed commits:
- Add tests for network create/remove
- Move compose service tests to test/src/compose and reorganize test descriptions
- Add support for image creation to mockerode
- Add additional tests for compose volumes
- Update mockerode so unimplemented fake methods throw. This is to ensure
tests using mockerode fail if an unimplemented method is used
- Update tests for volume-manager with mockerode
- Update tests for compose/images
- Simplify tests using mockerode
- Clean up compose/app tests
- Create application manager tests
Change-type: minor
This fixes a specific issue when the supervisor cannot find the right
source for deltas (e.g. after the DB gets deleted), where legacy
behavior was to look for any image in the app.
Change-type: patch
Relates-to: #1729
A previous PR (#1656) fixed validation for network ipam config,
checking that both network and subnet are defined for each ipam config entry
(as described in the docker documentation).
After that PR, the validations throws an exception if the network target state is incorrect,
but this turns out to be the wrong approach, because that exception is also triggered
when querying target state.
This isn't a problem in normal operation, but it is in local mode, because local
mode queries the old target state before sending a new one. Since the query fails,
the CLI can never push the new target state.
This PR replaces the exception with a warning on the logs, since a
misconfigured network won't cause any engine failures, it will just
prevent containers to communicate through the provided network.
A future improvement should move this validation to an earlier point in the process,
so the target state can get rejected before it even gets to a point it
can be used.
Relates-to: #1693
Change-type: patch
'mz' can be safely replaced with fs.promises
and util.promisify for faster native methods.
'mkdirp' after Node v8 uses native fs.mkdir, thus
is redundant. 'body-parser' is deprecated and
contained within express v4.x.
Closes: #1567
Change-type: patch
Signed-off-by: Christina Wang <christina@balena.io>
This commit updates dockerode types to the latest 2.x version, removing the need
for custom composer types for network.
This commit also modifies network tests to use the new types
Change-type: minor
Using safeStateClone within doPurge to applyIntermediateTarget after
successful volume purge has led to various type deficiencies being revealed
in common.js. Add several inline types in common.js to satisfy
the type checker (credit: Page <page@balena.io>). Delete common.d.ts
since it's not required and might mistakenly mask true I/O types of
functions in common.js.
Closes: #1611
Change-type: patch
Signed-off-by: Christina Wang <christina@balena.io>
The device request object was created with untouched fields left unset. When
comparing state to determine if a transition is required this would
result in a mismatch between:
{
Driver: '',
Count: 1,
DeviceIDs: null,
Capabilities: [Array],
Options: null
}
and
{
Count: 1,
Capabilities: [Array],
}
Which in turn resulted in the target service being continously restarted.
The fix is to instantiate the object in full.
Connects-to: https://github.com/balena-io/balena-supervisor/issues/1449
Connects-to: ae646a07ec6a6c96f7cb91f1d37898a94dbab47a
Change-type: patch
Signed-off-by: Robert Günzler <robertg@balena.io>
This paves the way for running multiple applications and storing
information related to the application against the application itself. A
couple of hacks have been added to v1 and v2 endpoints to maintain
compatability but these should eventually be removed with the addition
of a v3 api.
Change-type: minor
Signed-off-by: Cameron Diver <cameron@balena.io>
Currently, when the label `io.balena.features.balena-socket` is set,
the balena engine socket is mounted under `/run/balena-engine.sock`.
This causes a problem when using systemd inside the container, since
this service remounts `/run` and `/run/lock` as tmpfs, causing the
socket to become unavailable.
Making a mount of the socket into `/host/run` solves this issue. This is
the same approach taken with DBUS.
Change-type: patch
Signed-off-by: Felipe Lalanne <felipe@balena.io>
Connects-to: #1494
A docker-compose.yml with the following structure
```
version: '2.1'
services:
app_1:
build: ./noisy-1
image: noisy1
app_2:
build: ./noisy-1
image: noisy1
app_3:
build: ./noisy-1
image: noisy1
```
Will lead to the supervisor creating multiple image database entries
with the same dockerId (this is because of how the engine handles this
particular case). This case is not handled by the removal process
leading to image pile up and increased disk usage.
Change-type: patch
Signed-off-by: Felipe Lalanne <felipe@balena.io>
Connects-to: #1434
In order to make supervisor upgrades more transparent, lets move away
from this env var since it requires a container restart any time the supervisor
is upgraded. We should ultimately move towards providing the supervisors
set of capabilities, but that can come later
Connects-to: #1447
Change-type: major
Signed-off-by: Matthew McGinn <matthew@balena.io>
Each service, when requesting access to the Supervisor API, will
now get an individual key which can be scoped to specific resources.
In this iteration the default scope will be to the application that
the service belongs to.
We also have a `global` scope which is used by the cloud API when in
managed mode.
Change-type: patch
Signed-off-by: Rich Bayliss <rich@balena.io>
