41 Commits

Author SHA1 Message Date
Felipe Lalanne
e9a52e6786 Store rejected apps in the database
This moves from throwing an error when an app is rejected due to unmet
requirements (because of contracts) to storing the target with a
`rejected` flag on the database.

The application manager filters rejected apps when calculating steps to
prevent them from affecting the current state. The state engine uses the
rejection info to generate the state report.

Change-type: minor
2024-08-30 10:52:11 -04:00
Felipe Lalanne
dbacca977a
Do not use DB to store container logs info
This removes the dependence of the supervisor on the containerLogs
database for remembering the last sent timestamp. This commit instead
uses the supervisor startup time as the initial time for log retrieval.
This might result in some logs missing for services that may start
before the supervisor after a boot, or if the supervisor restarts.
However this seems like an acceptable trade-off as the current
implementation seems to make things worst in resource contrained
environments.

We'll move storing the last sent timestamp to a better storage medium in
a future commit.

Change-type: minor
2024-07-30 10:51:18 -04:00
Felipe Lalanne
6217546894 Update typescript to v5
This also updates code to use the default import syntax instead of
`import * as` when the imported module exposes a default. This is needed
with the latest typescript version.

Change-type: patch
2024-03-05 15:33:56 -03:00
Felipe Lalanne
988a1c9e9a Update @balena/lint to v7
This updates balena lint to the latest version to enable eslint support
and unblock Typescript updates. This is a huge number of changes as the
linting rules are much more strict now, requiring modifiying most files
in the codebase. This commit also bumps the test dependency `rewire` as
that was interfering with the update of balena-lint

Change-type: patch
2024-03-01 18:27:30 -03:00
Christina Ying Wang
c4f9d72172 Remove dependent devices content in codebase
This includes:
- proxyvisor.js
- references in docs
- references device-state, api-binder, compose modules, API
- references in tests

The commit also adds a migration to remove the 4 dependent device tables from the DB.

Change-type: minor
Signed-off-by: Christina Ying Wang <christina@balena.io>
2023-02-06 19:34:02 -08:00
Pagan Gazzard
a4c13aa2e9 Update to typescript 4.8.2
Change-type: patch
2022-09-19 16:36:17 +01:00
Felipe Lalanne
eee2460445 Fix database migration for legacyApps
Migration `M00008` had a bug with the check for legacy apps, which
resulted in devices that had at some point been updated from a single
container supervisor to get the error

```
Undefined binding(s) detected when compiling UPDATE. Undefined column(s): [appUuid] query
```

This adds a new migration with the fix to ensure broken fix the
inconsistent database state.

Change-type: patch
Closes: #1913
2022-04-01 17:58:20 -03:00
Felipe Lalanne
7425d1110b Add support for GET v3 target state
This change updates types and database format in order to allow
receiving the new format of the target state from the cloud and allow
applications to keep working.

This change also updates metadata in the containers, meaning services
will need to be restarted on supervisor update

Change-type: major
2022-03-22 19:08:02 -03:00
Felipe Lalanne
104a8006fb Update apiSecret table to id services by name
It adds a migration replacing the serviceId column by serviceName and
populates serviceNames from services in the target state.
2021-07-28 09:57:38 -04:00
Felipe Lalanne
fd06c06092 Update supervisor to typescript 4
Change-type: patch
2021-04-19 15:18:21 +00:00
Miguel Casqueira
277d984af2 Prevent inserting null commit during DB migration
Change-type: patch
Closes: #1581
Signed-off-by: Miguel Casqueira <miguel@balena.io>
2021-02-03 10:44:11 -05:00
Cameron Diver
f08316dc57 Allow storing commits against their appIds
This paves the way for running multiple applications and storing
information related to the application against the application itself. A
couple of hacks have been added to v1 and v2 endpoints to maintain
compatability but these should eventually be removed with the addition
of a v3 api.

Change-type: minor
Signed-off-by: Cameron Diver <cameron@balena.io>
2020-11-10 10:50:08 +00:00
Felipe Lalanne
adffde932e Fix supervisor deadlock during migration
Due to the singleton work, when performing migration M00005 and there
are apps with services created in the database, a deadlock occurs
during database initialization due to a circular
dependency for generating scoped keys.

Change-type: patch
Signed-off-by: Felipe Lalanne <felipe@balena.io>
Connects-to: #1468
2020-09-28 23:52:36 -03:00
Rich Bayliss
c08de8701e api: Implement scoped Supervisor API keys
Each service, when requesting access to the Supervisor API, will
now get an individual key which can be scoped to specific resources.
In this iteration the default scope will be to the application that
the service belongs to.

We also have a `global` scope which is used by the cloud API when in
managed mode.

Change-type: patch
Signed-off-by: Rich Bayliss <rich@balena.io>
2020-09-17 11:25:56 +00:00
Miguel Casqueira
466c7999db Remove CoffeeScript tests and all CoffeeScript tools
Closes: #1318
Change-Type: patch
Signed-off-by: Miguel Casqueira <miguel@balena.io>
2020-05-15 13:01:51 -04:00
Pagan Gazzard
758f3caa48 Update to @balena/lint 5.x
Change-type: patch
2020-05-15 12:08:42 +01:00
Cameron Diver
f791424829 ⤴️ Upgrade migrations to work with knex
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2020-04-15 13:37:50 +01:00
Pagan Gazzard
9c3295c912 Add type checking for javascript files
Change-type: patch
2020-03-24 16:37:28 +00:00
Pagan Gazzard
81966446d9 Update to balena-lint and enable javascript linting
Change-type: patch
2020-03-24 09:28:12 +00:00
Roman Mazur
4974c9200c
Clean up local mode engine objects using snapshots
Snapshot is collected to compare with engine state when local mode is turned off.

Change-type: minor
Signed-off-by: Roman Mazur <roman@balena.io>
2019-06-26 17:40:01 +03:00
Cameron Diver
e148ce0529
Report all logs from a container's runtime
We add a database table, which holds information about the last
timestamp of a log successfully reported to a backend (local or remote).
We then use this value to calculate from which point in time to start
reporting logs from the container. If this is the first time we've seen
a container, we get all logs, and for every log reported we save the
timestamp. If it is not the first time we've seen a container, we
request all logs since the last reported time, ensuring no interruption
of service.

Change-type: minor
Closes: #937
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-04-05 10:06:27 +01:00
Cameron Diver
f9626a3ee4 device-config: Add migration for SUPERVISOR_DELTA_APPLY_TIMEOUT
The default value for the delta apply timeout was changed from `''` to
`'0'` (note strings as these are database values) - but if the value
existed in the database already, this would fail validation. We add a
migration which will look explcitily for the failing value and switch it
to the new default.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-02-25 13:22:02 +01:00
Cameron Diver
91a6340563
Move config.json flag back to the database
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-12-14 15:01:40 +00:00
Pablo Carranza Velez
d5b2fcd4dd Update knex to 0.15.2 and sqlite3 to 4.0.4
We also replace a createTableIfNotExists in the migrations with hasTable then createTable, to
avoid a warning message about it being not recommended.

Change-type: patch
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-12-12 13:42:29 -03:00
Cameron Diver
a2e2948a4e
Allow local mode to be controlled via config.json
Change-type: minor
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-11-28 14:53:08 +00:00
Cameron Diver
96ecef8052
Fix m00001 migration by using targetValues
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-11-13 22:14:06 +00:00
Pablo Carranza Velez
b3860b2b70 fix: Store and retrieve device config without namespaces
This avoids issues on provisioning where the current state
(esp. config.txt) that we want to save is retrieved without
a RESIN_ or BALENA_ prefix, causing those values to be lost.

Change-type: patch
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-10-20 04:40:55 +02:00
Cameron Diver
39d8ac0133
Change config function providers to be mutable
Also change logsChannelSecret value to be queried with the api backend,
so that logs are not shared between instances. This has been implemented
as the first config function provider with mutability.

Change-type: minor
Closes: #675
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-06-26 14:02:25 +01:00
Cameron Diver
343989f487
Add migration for source field to be added to app table
This field will represent the apiEndpoint that the application came
from, (or an empty string for local apps). This means that when
configuring an application to work on a different environment, as long
as the endpoint is different, the supervisor can know not to start the
old application.

Change-type: minor
Connects-to: #658
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-05-22 12:11:54 +01:00
Pablo Carranza Velez
9dfb76d2c8 When migrating from legacy supervisors, mark the temporary composition as legacy
We migrate to a default composition because we need to avoid deleting existing docker images, but
we need to use the legacy-container label to avoid potentially creating a duplicated container when a target state comes in.

(Just like we do for preloaded apps)

Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
2018-03-13 08:17:25 -07:00
Pablo Carranza Velez
f009e34293 migrations: Use the correct service name for legacy apps
Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
2018-03-09 06:54:16 -08:00
Pablo Carranza Velez
dc62418db4 Some fixes in current state reporting, error handling for "container not found", plus more style improvements
Also, ensure the properties argument to eventTracker.track is an object

Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
2018-03-06 10:32:29 -08:00
Pablo Carranza Velez
bb57bcc37c Fix default volume name in migrations, and do not use the legacy-container flag on containers from older supervisors (they'll be restarted anyways)
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
2018-03-06 10:32:29 -08:00
Pablo Carranza Velez
58b167b43d Various bugfixes and sytlistic improvements
* Use the correct defaults for the delta config variables that have them

* Only mount /lib/firmware and /lib/modules if they exist on the host

* hardcode-migrations.js: Nicer line separation

* APIBinder: switch to using a header for authentication, and keep credentials saved in the API clients

* Fix hrtime measurements in milliseconds

* Do not uses classes for routers

* compose: properly initialize networkMode to the first entry in networks if there is one

* Fix some details regarding defaults in validation and service

Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
2018-03-06 10:32:29 -08:00
Pablo Carranza Velez
ba829412e1 Various bugfixes
* Get imageId when normalising a dep. app from the DB

* Fix the appId in migrations when updating the supervisor

* Use the update lock to update a service's metadata

* Restart clears volatile target state

* Fix function definition for updateMetadata

* Improve backwards compatibility of /v1/apps/:appId endpoint

* Fix multicontainer deltas to work with resumable-request 2.0

* Fix dependent target normalisation logic

Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
2018-03-06 10:32:29 -08:00
Pablo Carranza Velez
f653fa4961 Add support for service hostname
Plus several small bug fixes:

* Allow target states with apps with no release

* Fix lock override and a TypeError in compareServicesForUpdate

* Lowercase service names when doing migrations and legacy preload

* Fix deltas from scratch

Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
2018-03-06 10:32:28 -08:00
Pablo Carranza Velez
839ebf8688 Fix preloaded apps and support legacy preloading, and fix some details in the default service when migrating
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
2018-03-06 10:32:28 -08:00
Pablo Carranza Velez
60f0cd2fcb Change all labels to use hyphens instead of underscores, and fix some instances of using split instead of a regex
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
2018-03-06 10:32:28 -08:00
Pablo Carranza Velez
3a710506a6 Switch to a new image management system keeping the docker image ID in the database, allowing deltas and proper comparison for images that have a digest.
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
2018-03-06 10:32:28 -08:00
Pablo Carranza Velez
25695aade5 Add support for init, mem_reservation, shm_size, read_only and sysctls.
Also several bugfixes:

* Fix VPN control, logging in deviceConfig, and action executors in proxyvisor

* Fix bug in calculation of dependencies due to fields still using snake_case

* Fix snake_case in a migration, and remove unused lib/migration.coffee

* In healthcheck, count deviceState as healthy when a fetch is in progress (as in the non-multicontainer supervisor)

* Set always as default restart policy

* Fix healthcheck, stop_grace_period and mem_limit

* Lint and reduce some cyclomatic complexities

* Namespace volumes and networks by appId, switch default network name to 'default', fix dependencies in networks and volumes, fix duplicated kill steps, fix fat arrow on provisioning

* Check that supervisor network is okay every time we're applying target state

Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
2018-03-06 10:32:28 -08:00
Pablo Carranza Velez
b003f48d7b Switch to using knex migrations to set up the database, and change the database format to use integers for ids instead of strings.
Also includes various improvements and bugfixes to services and the migration from legacy /data to volumes.

The switch ti migrations involves a dirty hack for webpack to properly resolve the paths to the migrations js files - it uses an expression
that webpack can't resolve, so we hardcode it to a value and use the ContextReplacementPlugin to make that value resolve to the migrations folder.

The downsides to this approach are:
- a change in knex code would break this
- the migration code is added twice to the supervisor image: once in the migrations folder (because knex needs to loop through the directory to find the files),
and once inside app.js (because I can't make webpack treat them as external)

Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
2018-03-06 10:32:28 -08:00