1632 Commits

Author SHA1 Message Date
Christina Ying Wang
ed1c18e369
Add support for init field from compose
Init supports boolean values, and is not included in the config when
not defined.

Change-type: patch
Signed-off-by: Christina Ying Wang <christina@balena.io>
2024-09-26 10:39:59 -03:00
Felipe Lalanne
e9a52e6786 Store rejected apps in the database
This moves from throwing an error when an app is rejected due to unmet
requirements (because of contracts) to storing the target with a
`rejected` flag on the database.

The application manager filters rejected apps when calculating steps to
prevent them from affecting the current state. The state engine uses the
rejection info to generate the state report.

Change-type: minor
2024-08-30 10:52:11 -04:00
Felipe Lalanne
227fee9941 Set the app update status when reporting state
Change-type: minor
2024-08-30 10:52:11 -04:00
Felipe Lalanne
48e526ec43 Refactor contracts validation code
This updates the interfaces on lib/contracts and the validation in
the application-manager module.
2024-08-30 10:52:11 -04:00
Felipe Lalanne
e9f460fd75 Add update status to types
Change-type: minor
2024-08-30 10:52:11 -04:00
Felipe Lalanne
788afee9a1
Remove unused patchDevice function
This function was a remainder of the dependent devices code that no
was removed on #2105

Change-type: patch
2024-08-29 10:34:43 -04:00
Christina Ying Wang
eaa07e97a9 Add support for redsocks dnsu2t config
Users may specify dnsu2t config by including a `dns` field
in the `proxy` section of PATCH /v1/device/host-config's body:
```
{
  network: {
    proxy: {
      dns: '1.1.1.1:53',
    }
  }
}
```

If `dns` is a string, ADDRESS and PORT are required and should be
in the format `ADDRESS:PORT`. The endpoint with error with
code 400 if either ADDRESS or PORT are missing.

`dns` may also be a boolean. If true, defaults will be configured.
If false, the dns configuration will be removed.

If `proxy` is patched to empty, `dns` will be removed regardless
of its current or input configs, as `dns` depends on an active
redsocks proxy to function.

Change-type: minor
Signed-off-by: Christina Ying Wang <christina@balena.io>
2024-08-28 14:01:51 -07:00
Christina Ying Wang
8bf346a6fd Parse dnsu2t block to dns config
Signed-off-by: Christina Ying Wang <christina@balena.io>
2024-08-28 13:51:46 -07:00
Christina Ying Wang
b775f8f14d Stringify dns subsection of redsocks input config to dnsu2t
Signed-off-by: Christina Ying Wang <christina@balena.io>
2024-08-28 13:51:46 -07:00
Christina Ying Wang
e724f60beb Strip additional fields from HostConfiguration type
Signed-off-by: Christina Ying Wang <christina@balena.io>
2024-08-28 13:51:46 -07:00
Christina Ying Wang
51e59725f8 Add unit test for usingInferStepsLock
Change-type: patch
Signed-off-by: Christina Ying Wang <christina@balena.io>
2024-08-26 13:44:51 -07:00
Christina Ying Wang
3cebfa9f78 Revert PR #2364
Change-type: patch
Signed-off-by: Christina Ying Wang <christina@balena.io>
2024-08-22 14:31:35 -07:00
Christina Ying Wang
fc6927e53d Avoid unnecessary config calls during Supervisor init
Change-type: patch
Signed-off-by: Christina Ying Wang <christina@balena.io>
2024-08-20 19:11:14 -07:00
Felipe Lalanne
b088b78a3e
Do not write noProxy to redsocks.conf
This fixes a regression introduced by the refactor in #2329 where
`noProxy` was being included in the data added to redsocks.conf.

Change-type: patch
2024-08-08 11:59:20 -04:00
Felipe Lalanne
a255001c2e
Verify that LED_FILE exists on blinking setup
Before v1, the blinking module would not throw when the passed led file
does not exist. This change checks for file existence and defaults to
`/dev/null` otherwise

Change-type: patch
2024-08-07 15:33:07 -04:00
Felipe Lalanne
d789e5bb77
Avoid leaking memory on deep promise recursions
The following pattern
```ts
async function longRunning() {
   // do something
   await setTimeout(delay);
   await longRunning();
}
```

Is regularly used for long running operations on the supervisor (e.g.
polling target state). We have
recently discovered that this pattern can slowly leak memory as it
essentially creates an infinite promise chain. Using `void longRunning()` breaks
the chain and avoids the issue.

This commit fixes all those instances where the pattern was used.

Change-type: patch
2024-07-31 18:39:29 -04:00
Felipe Lalanne
8bc08750e9
Use promises for setup/writing for logging backend
The balena logging backend now uses async functions to setup the
connection and write messages to the request stream. This adds some
backpressure on `log` calls by by the log monitor module, to prevent a
very agressive container causing the supervisor to waste CPU cycles just
dropping messages.

Change-type: patch
2024-07-30 10:51:19 -04:00
Felipe Lalanne
f3fcb0db7a
Improve the LogBackend interface
This make the LogBackend `log` method into an async method in
preparation for upcoming changes that will use backpressure from the
connection to delay logging coming from containers.

This also removes unnecessary imageId from the LogMessage type

Change-type: patch
2024-07-30 10:51:19 -04:00
Felipe Lalanne
5af948483a
Use stream pipeline instead of pipe
This also removes the use of JSONStream from the monitor module

Change-type: patch
2024-07-30 10:51:19 -04:00
Felipe Lalanne
dbacca977a
Do not use DB to store container logs info
This removes the dependence of the supervisor on the containerLogs
database for remembering the last sent timestamp. This commit instead
uses the supervisor startup time as the initial time for log retrieval.
This might result in some logs missing for services that may start
before the supervisor after a boot, or if the supervisor restarts.
However this seems like an acceptable trade-off as the current
implementation seems to make things worst in resource contrained
environments.

We'll move storing the last sent timestamp to a better storage medium in
a future commit.

Change-type: minor
2024-07-30 10:51:18 -04:00
Pagan Gazzard
4976578a83 Improve log message typing
Change-type: patch
2024-07-17 11:14:17 +01:00
Pagan Gazzard
c5d0eafea9 Logs: only truncate the message if it's possible it will need it
Change-type: patch
2024-07-16 18:09:12 -04:00
Pagan Gazzard
34cf8b6708 Logs: skip setting default values if we're dropping as an invalid log
Change-type: patch
2024-07-16 18:09:12 -04:00
Pagan Gazzard
0a817af10c Logs: only call Date.now() if a timestamp is not already present
Change-type: patch
2024-07-16 18:09:12 -04:00
Felipe Lalanne
28706510da
Remove custom typings for docker-delta
This are no longer needed since docker-delta migrated to typescript in
v4 and the custom typings were actually masking some bugs

Change-type: patch
2024-07-04 18:41:33 -04:00
Christina Ying Wang
f99ccb58c6 Remove unnecessary exports from host-config
This limits the host-config interface to necessary methods
only

Signed-off-by: Christina Ying Wang <christina@balena.io>
2024-07-03 16:47:51 -07:00
Christina Ying Wang
53f5641ef1 Refactor host-config to be its own module
The host-config module exposes the following interfaces: get,
patch, and parse.

`get` gets host configuration such as redsocks proxy configuration
and hostname and returns it in an object of type HostConfiguration.

`patch` takes an object of type HostConfiguration or LegacyHostConfiguration
and updates the hostname and redsocks proxy configuration, optionally
forcing the patch through update locks.

`parse` takes a user input of unknown type and parses it into type
HostConfiguration or LegacyHostConfiguration for patching, erroring if
parse was unsuccessful.

LegacyHostConfiguration is a looser typing of the user input which does
not validate values of the five known proxy fields of type, ip, port,
username, and password. We should stop supporting it in the next
major Supervisor API release.

Change-type: minor
Signed-off-by: Christina Ying Wang <christina@balena.io>
2024-07-03 16:47:51 -07:00
Christina Ying Wang
be986a62a5 Add HostConfig.parse method
Parses input from PATCH /v1/device/host-config into either
type HostConfiguration, or if LegacyHostConfiguration if
input is of an acceptable shape (for backwards compatibility).

Once input has been determined to be of type HostConfiguration,
we can easily extract ProxyConfig from the object for patching,
stringifying, and writing to redsocks.conf.

Change-type: minor
Signed-off-by: Christina Ying Wang <christina@balena.io>
2024-07-03 16:47:51 -07:00
Christina Ying Wang
f17f7efe60 Add HostConfig.patchProxy method
Signed-off-by: Christina Ying Wang <christina@balena.io>
2024-07-03 16:47:49 -07:00
Christina Ying Wang
9c6681bb23 Add RedsocksConf.stringify method
`stringify` takes a RedsocksConfig, an internal object
representation of the redsocks.conf file, and transforms
it into a valid string that can be written to redsocks.conf.

Signed-off-by: Christina Ying Wang <christina@balena.io>
2024-07-03 16:45:52 -07:00
Christina Ying Wang
1e224be0cd Add RedsocksConf.parse method
This is part of the host-config refactor which
enables easier encoding to / decoding from `redsocks.conf`.

Signed-off-by: Christina Ying Wang <christina@balena.io>
2024-07-03 16:45:06 -07:00
Christina Ying Wang
725d7790fb Move noProxy handling to separate module
Signed-off-by: Christina Ying Wang <christina@balena.io>
2024-06-28 11:34:27 -07:00
Christina Ying Wang
0cf5a4bf18 Move hostname get/set to separate "module" (directory)
Signed-off-by: Christina Ying Wang <christina@balena.io>
2024-06-28 11:34:21 -07:00
Felipe Lalanne
ede27b63ce
Fix engine deadlock on network+service change
This fixes a regression on the supervisor state engine computation
(added on v16.2.0) when
the target state removes a network at the same time that a service
referencing that network is changed. Example going from

```
services:
   one:
      image: alpine: 3.18
      networks: ['balena']

networks:
   balena:
```

to

```
services:
   one:
      image: alpine: latest
```

Would never reach the target state as killing the service in order to
remove the network is prioritized, but one of the invariants in the target state calculation is
to not kill any services until all images have been downloaded. These
two instructions were in contradiction leading to a deadlock.

The fix involves only adding removal steps for services depending on a
changing network or volume if the service container is not being removed
already.

Change-type: patch
2024-06-24 18:12:12 -04:00
Felipe Lalanne
45d1f1f964
Split target state set/get into separate module
This removes the circular dependency between device state index and
preloading and cleans up the device state module code

Change-type: patch
2024-06-03 11:40:47 -04:00
Felipe Lalanne
fedeb518e5
Move device-state.ts into the device-state/index.ts
Change-type: patch
2024-06-03 11:40:47 -04:00
Felipe Lalanne
9497eed380
Move device-state/target state to api-binder/poll
This goes in the direction of grouping modules by responsibility. The
api-binder module is the middleware between the device and the backend,
thus the target state polling code makes more sense there.

Change-type: patch
2024-06-03 11:40:46 -04:00
Felipe Lalanne
027c2575b1 Move OS variant retrieval to config module
This also deprecates the `getOSVariant` function of the `os-release`
module, as the OS variant are no longer defined in `/etc/os-release`.

Change-type: patch
2024-05-27 14:36:03 -04:00
Felipe Lalanne
59689b2789 Do not export balenaApi on api-binder
Instead, add `getBalenaApi` function to api-helper so other modules can
access a balena API instance.

Further reduces circular dependencies to 5

Change-type: patch
2024-05-27 14:36:03 -04:00
Felipe Lalanne
ac2db38742 Move api-keys module to src/lib
This removes circular dependencies between the device-api module and
the compose module, reducing total circular dependencies to 15

Change-type: patch
2024-05-27 14:36:03 -04:00
Felipe Lalanne
48f75e61b8 Do not re-export ContractObject on lib/contracts
Further reduce circular dependencies to 10

Change-type: patch
2024-05-27 14:36:03 -04:00
Felipe Lalanne
bef5e78440 Move Compose(Network|Volume)Config to top level types
This reduces dependencies from 80 to 47

Change-type: patch
2024-05-27 14:36:03 -04:00
Felipe Lalanne
234e0de075 Move composition types to compose/types
This reduces circular dependencies from 250 to 80 by ensuring that
modules that only require types do not import the full module with all
its dependencies.

Change-type: patch
2024-05-27 14:36:03 -04:00
Felipe Lalanne
94de4006a0 Split compose types into interface and implementation
This splits `App`, `Network`, `Service` and `Volume` which used to be
defined as classes into an interface and a class implementation that is
not exported. This will allow to work with just the types in some cases
and prevent circular dependencies when importing.

Change-type: patch
2024-05-27 14:36:03 -04:00
Christina Ying Wang
1dcd156fc8 Update @balena/contrato to 0.9.4
Change-type: patch
Signed-off-by: Christina Ying Wang <christina@balena.io>
2024-04-30 16:39:49 -07:00
Pagan Gazzard
4adf710520 Update @types dependencies
Change-type: patch
2024-04-29 16:29:07 +01:00
Felipe Lalanne
ae823fea18 Update docker related dependencies
This bumps dockerode, removes resin-docker-build in favor of
@balena/compose, and updates docker-delta and docker-progress packages.

Change-type: patch
2024-04-26 12:03:04 -04:00
Felipe Lalanne
6f02b17968 Refactor MDNS resolver into a module
Also add integration tests for the resolver functionality to prevent
regressions.

Change-type: patch
2024-04-23 19:23:32 -04:00
Felipe Lalanne
ad52561de5 Fix mdnsResolver import
The `mdns-resolver` module does not provide a default export. Trying to
use a default import notation, causes the `resolve` function to not be
found, breaking MDNS resolution.

Change-type: patch
2024-04-23 19:23:32 -04:00
Christina Ying Wang
14bdc522c1 Gracefully handle multiple reboot/shutdown requests
Since HTTP's server.close() is async, there is a slim chance
for two instances of /v1/reboot or /v1/shutdown to be processed.
If the server is already closed when server.close() is called,
the call throws ERR_SERVER_NOT_RUNNING
which doesn't need to be surfaced to the user. This change
only allows one server.close() attempt to occur at a time.

Change-type: patch
Signed-off-by: Christina Ying Wang <christina@balena.io>
2024-04-23 12:59:44 -07:00