Commit Graph

107 Commits

Author SHA1 Message Date
Felipe Lalanne
e9a52e6786 Store rejected apps in the database
This moves from throwing an error when an app is rejected due to unmet
requirements (because of contracts) to storing the target with a
`rejected` flag on the database.

The application manager filters rejected apps when calculating steps to
prevent them from affecting the current state. The state engine uses the
rejection info to generate the state report.

Change-type: minor
2024-08-30 10:52:11 -04:00
Felipe Lalanne
48e526ec43 Refactor contracts validation code
This updates the interfaces on lib/contracts and the validation in
the application-manager module.
2024-08-30 10:52:11 -04:00
Felipe Lalanne
ac2db38742 Move api-keys module to src/lib
This removes circular dependencies between the device-api module and
the compose module, reducing total circular dependencies to 15

Change-type: patch
2024-05-27 14:36:03 -04:00
Felipe Lalanne
234e0de075 Move composition types to compose/types
This reduces circular dependencies from 250 to 80 by ensuring that
modules that only require types do not import the full module with all
its dependencies.

Change-type: patch
2024-05-27 14:36:03 -04:00
Felipe Lalanne
94de4006a0 Split compose types into interface and implementation
This splits `App`, `Network`, `Service` and `Volume` which used to be
defined as classes into an interface and a class implementation that is
not exported. This will allow to work with just the types in some cases
and prevent circular dependencies when importing.

Change-type: patch
2024-05-27 14:36:03 -04:00
Felipe Lalanne
ae823fea18 Update docker related dependencies
This bumps dockerode, removes resin-docker-build in favor of
@balena/compose, and updates docker-delta and docker-progress packages.

Change-type: patch
2024-04-26 12:03:04 -04:00
Christina Ying Wang
fb1bd33ab6 Refine update locking interface
* Remove Supervisor lockfile cleanup SIGTERM listener
* Modify lockfile.getLocksTaken to read files from the filesystem
* Remove in-memory tracking of locks taken in favor of filesystem
* Require both `(resin-)updates.lock` to be locked with `nobody` UID
  for service to count as locked by the Supervisor

Signed-off-by: Christina Ying Wang <christina@balena.io>
2024-04-04 14:07:47 -07:00
Felipe Lalanne
6217546894 Update typescript to v5
This also updates code to use the default import syntax instead of
`import * as` when the imported module exposes a default. This is needed
with the latest typescript version.

Change-type: patch
2024-03-05 15:33:56 -03:00
Felipe Lalanne
988a1c9e9a Update @balena/lint to v7
This updates balena lint to the latest version to enable eslint support
and unblock Typescript updates. This is a huge number of changes as the
linting rules are much more strict now, requiring modifiying most files
in the codebase. This commit also bumps the test dependency `rewire` as
that was interfering with the update of balena-lint

Change-type: patch
2024-03-01 18:27:30 -03:00
Felipe Lalanne
87b195685a Use the state-helper functions in app module tests 2024-01-29 12:25:55 -08:00
Pagan Gazzard
c9f032e13a Switch _.isFunction usage to native versions
Change-type: patch
2023-10-16 14:30:25 -03:00
Pagan Gazzard
20df54668c Switch _.isArray usage to native versions
Change-type: patch
2023-10-16 14:30:25 -03:00
Felipe Lalanne
8f17c30de6 Replace dbus test service with mock-systemd-bus
This avoids unnecessary mocking and tests against the real systemd API

Change-type: patch
2023-08-16 14:46:58 -04:00
Christina Ying Wang
84a9e7e9ac Replace BALENA-FIREWALL rule in INPUT chain instead of flushing
The issue with the original Supervisor implementation of the firewall is that
on Supervisor start, the Supervisor flushes the INPUT chain of the filter table.
This doesn't play well with services that add to the INPUT chain on startup that
may start up before the Supervisor, such as certain NetworkManager connection
profiles. This change only replaces the BALENA-FIREWALL rule in the INPUT chain,
preserving the other rules as well as their order.

Closes: #1482
Change-type: patch
Signed-off-by: Christina Ying Wang <christina@balena.io>
2023-03-01 13:42:07 -08:00
Felipe Lalanne
89175432af Find and remove duplicate networks
We have seen a few times devices with duplicated network names for some
reason. While we don't know the cause the networks get duplicates, this
can be disruptive for updates as trying to create a container referencing a duplicate
network results in a 400 error from the engine.

This commit finds and removes duplicate networks via the state engine,
this means that even if somehow a container could be referencing a
network that has been duplicated later somehow, this will remove the
container first.

While thies doesn't solve the problem of duplicate networks being
created in the first place, it will fix the state of the system to
correct the inconsistency.

Change-type: minor
Closes: #590
2023-02-10 20:24:36 -05:00
Christina Ying Wang
c4f9d72172 Remove dependent devices content in codebase
This includes:
- proxyvisor.js
- references in docs
- references device-state, api-binder, compose modules, API
- references in tests

The commit also adds a migration to remove the 4 dependent device tables from the DB.

Change-type: minor
Signed-off-by: Christina Ying Wang <christina@balena.io>
2023-02-06 19:34:02 -08:00
Felipe Lalanne
4d74505087 Fix wait-for-it script to work with external signals
The wait-for-it script used during tests would setup a timer
that would send SIGUSR2 to the parent process after the timer ends.
Since node was ignoring additional signals, the timer ending would have
no effect after the node process had replaced the start script. However
when node has pid != 1, SIGUSR2 default behavior is to terminate the
process, meaning the tests would fail after 30 seconds.

The script is now updated so the timer is killed once the services are
ready for the tests.
2023-01-31 10:43:19 -03:00
Christina Ying Wang
e1bacda580 Update host-config, route, and action tests for host config endpoints
Change-type: minor
Signed-off-by: Christina Ying Wang <christina@balena.io>
2023-01-11 15:48:13 -08:00
Christina Ying Wang
fcd28591c6 Add tests for doPurge action and v1/v2 app purge routes
Signed-off-by: Christina Ying Wang <christina@balena.io>
2023-01-09 16:25:27 -08:00
Christina Ying Wang
a24d5acf7f Add tests for doRestart action and v1/v2 app restart routes
Signed-off-by: Christina Ying Wang <christina@balena.io>
2023-01-09 16:25:27 -08:00
Pagan Gazzard
b8891ebb08 Use timers/promises for promisified setTimeout
Change-type: patch
2022-11-21 18:17:34 -05:00
Christina Ying Wang
f558be0a16 Create default network as config-only when services have host networking
This eliminates chances of host-Docker address collision for apps such
as the Supervisor where all services have host networking.

Closes: #2062
Change-type: patch
Signed-off-by: Christina Ying Wang <christina@balena.io>
2022-11-16 10:19:36 -08:00
pipex
827f892c13 Migrate all device config tests to integration.
This means that configuration backend tests no longer use stubs and
(mostly) avoid internal dependencies in the tests. Instead of stubs and
mock-fs, the tests use [testfs](https://github.com/balena-io-modules/mocha-pod#working-with-the-filesystem)
which allows working with a real filesystem and ensuring everything is
re-set between tests.

This is the last change needed in order to be able to merge #1971. Here is the list of changes

- [x] Migrate splash image backend tests
- [x] Migrate extlinux backend tests
- [x] Migrate config.txt backend tests
- [x] Migrate extra-uenv config tests
- [x] Migrate odmdata config tests
- [x] Migrate config utils tests
- [x] Migrate device-config tests

Change-type: patch
2022-11-14 11:12:52 -03:00
Christina Ying Wang
75bf2aa3b4 Improve NotFoundError handling
Use isNotFoundError which converts an error of the default type
`unknown` into NotFoundError if the error is an instance of NotFoundError.
Thrown errors are of type `unknown` by default so we should use methods
with type guards for better type narrowing.

Signed-off-by: Christina Ying Wang <christina@balena.io>
2022-11-08 15:41:52 -08:00
Felipe Lalanne
46fa7321c0 Run the built supervisor as part of docker-compose tests
This allows to test that the supervisor build actually runs and opens up the
possibility of running more exhaustive API tests against a working supervisor.

Change-type: patch
2022-11-03 15:45:39 -03:00
Christina Ying Wang
532e75a77e Migrate API tests to unit/integration
This excludes route tests or refactoring. Also, created tests
for API middleware.

Change-type: patch
Signed-off-by: Christina Ying Wang <christina@balena.io>
2022-10-25 19:06:39 +00:00
pipex
0befb30018 Migrate firewall tests to integration 2022-10-19 14:09:45 -03:00
pipex
aa3002f909 Migrate docker-util tests
Merged docker-utils and delta tests into a single test suite. They are
now ran as part of the integration tests using the real engine.

Change-type: patch
2022-10-19 12:05:52 -03:00
Christina Ying Wang
f586b7c9a8 Make dbus module side-effect free to not interfere with unit tests
When code that is unit tested is part of a file that imports modules which
depend on the dbus module, this breaks the unit test environment because there
is no system socket set up, as the unit test mocha config doesn't import fixtures.ts.

For example, if we change src/compose/utils to import device-config or api-binder, both
of those modules import lib/dbus which invokes a dbus.getBus call at the root level. This
is problematic for unit testing.

We can get around the root-level dbus.getBus call by initializing dbus only when it's first
needed. The mocked-dbus test setup code can also be removed in favor of legacy mocha
hooks, which makes the dbus stubbing in the legacy test environment more clear.
We can remove these legacy hooks when all the legacy tests are migrated to unit/integration.

Signed-off-by: Christina Ying Wang <christina@balena.io>
2022-10-18 14:27:19 -07:00
Christina Ying Wang
71b2aea0fe Use v2 router directly instead of through application manager
Signed-off-by: Christina Ying Wang <christina@balena.io>
2022-10-18 14:16:53 -07:00
Christina Ying Wang
ce5bf89dfc Move /v1 routes in deviceState.router to v1.ts
Signed-off-by: Christina Ying Wang <christina@balena.io>
2022-10-18 14:16:53 -07:00
Christina Ying Wang
a2d9af2407 Move /v1 routes in apiBinder.router to v1.ts
Signed-off-by: Christina Ying Wang <christina@balena.io>
2022-10-18 14:16:53 -07:00
Christina Ying Wang
d08f25f0a3 Consolidate API middlewares, move api-keys to device-api
Signed-off-by: Christina Ying Wang <christina@balena.io>
2022-10-18 14:16:53 -07:00
Christina Ying Wang
5af146ec4e Move supervisor-api.ts to device-api/index.ts
Signed-off-by: Christina Ying Wang <christina@balena.io>
2022-10-18 14:16:53 -07:00
Felipe Lalanne
819e184095 Setup environment for dbus tests
Change-type: patch
2022-10-17 11:24:15 -03:00
Pagan Gazzard
0237bd7cf4 Update type dependencies
Change-type: patch
2022-10-03 14:38:42 -05:00
Felipe Lalanne
0fb1de2a1a Migrate tests for image manager 2022-09-28 10:37:41 -03:00
Felipe Lalanne
a69fbf6eac Migrate volume-manager tests to integration
Now the tests are ran against the actual docker engine instead of
against mockerode.

The new tests actually caught a bug in
`volumeManager.removeOrphanedVolumes`, where that function would try to
remove volumes for stopped containers, causing an exception.
This commit also fixes that bug.
2022-09-28 10:37:41 -03:00
Felipe Lalanne
18c2f8cec9 Migrate lib/legacy tests to integration tests
This also sets up a dbus container as part of the test environment. This
container exposes a socket that the sut container can use
2022-09-28 10:37:41 -03:00
Felipe Lalanne
a4da25c1ef Disable logs globally using mocha hooks 2022-09-28 10:37:41 -03:00
Felipe Lalanne
460659429d Update dependencies to fix NPM build
Change-type: patch
2022-09-26 15:26:48 -03:00
Pagan Gazzard
5518eb17bd Update to nodejs 16
Change-type: minor
2022-09-19 17:51:48 +01:00
Pagan Gazzard
96418d55b5 Update @balena/lint to 6.2.0
Change-type: patch
2022-09-19 16:41:28 +01:00
Pagan Gazzard
a4c13aa2e9 Update to typescript 4.8.2
Change-type: patch
2022-09-19 16:36:17 +01:00
Felipe Lalanne
48e0733c7e Remove side effects for module imports
The supervisor uses the following pattern for async module
initialization

```typescript
// module.ts

export const initialised = (async () => {
    // do some async initialization
})();

// somewhere else
import * as module from 'module';

async function setup() {
  await module.initialise;
}
```

The above pattern means that whenever the module is imported, the
initialisation procedure will be ran, which is an anti-pattern.

This converts any instance of this pattern into a function

```typescript
export const initialised = _.once(async () => {
    // do some async initialization
});
```

And anywhere else on the code it replaces the call with a

```typescript
await module.initialised();
```

Change-type: patch
2022-09-06 15:48:18 -04:00
Felipe Lalanne
c1e6dadeb4 Create test/unit and test/integration folders
This sets up the new `test/unit` and `test/integration` folders
and starts classification of some of the test files.

Note that unit tests include, `fs-utils` and `system-info` tests.

While these tests interact with the filesystem, the implementation
of these modules is simple enough, and the tests are fast enough to
allow these tests to fall under the `unit` test category (according to
test/README)

Change-type: patch
2022-08-24 14:28:36 -04:00
Felipe Lalanne
382cdb0cfa Refactor test suite to use tsconfig paths
This replaces all relative paths in the test suite  (e.g
`../src/compose/service.ts`) with the aliased path configured through
tsconfig.

This is a big change but it doesn't affect any functionality
2022-08-18 13:05:10 -04:00
Christina Wang
ffa1c73418 Better document mocked-dbus, add missing dbus interface methods
Change-type: patch
Signed-off-by: Christina Wang <christina@balena.io>
2022-06-09 11:41:50 -07:00
Felipe Lalanne
c04955354a Use write + sync when writing configs to /mnt/boot
This commit updates all backends that write to /mnt/boot to do it
through a new `lib/host-utils` module. Writes are now done using write +
sync as rename is not an atomic operation in vfat.

The change also applies for writes through the `/v1/host-config`
endpoint.

Finally this change includes some improvements on tests.

Change-type: patch
2022-05-03 11:23:00 -04:00
Felipe Lalanne
f1cd3d367c Cleanup unused methods and dependencies on db ids 2022-03-22 19:28:43 -03:00