Commit Graph

176 Commits

Author SHA1 Message Date
Pablo Carranza Velez
8f07bf62de Add a random jitter to target state polls, and a config var to ignore update notifications and not poll immediately after startup
This commit does two related things:

* We make the poll interval a random time between 0.5 and 1.5 times the configured interval.
* We introduce the BALENA_SUPERVISOR_INSTANT_UPDATE_TRIGGER configuration variable, that defaults to true. If this variable is set
to false, then calls to /v1/update are ignored, and on startup the supervisor waits for a poll interval before getting the target state.

This will help especially on cases where there's a large number of devices on a single network. By disabling instant updates and setting a large
poll interval, we can now achieve a sitation where not all devices apply an update at the same time, which can help avoid
overwhelming the network.

Change-type: minor
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2019-04-03 09:15:49 -07:00
Pablo Carranza Velez
ea6ebf8223 test: Add a test case for deviceConfig.getDefaults
This would help avoid bugs like the one fixed in 9.11.1.

Change-type: patch
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2019-03-28 16:01:00 -07:00
Cameron Diver
b3192679c7
fix: Correctly compare and generate network membership aliases
Before this change, service name resolution would only occur in the
default network. This was because we were not explicitly adding aliases
of the service names to the aliases fields.

We also fix the comparison, which would do funny things based on
container IDs, which was correct but unnecessary.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-03-28 10:27:21 +00:00
Cameron Diver
84356b82b8
state-engine: Return a noop when waiting for a dependency
We run the risk of the state engine exiting early when a dependency is
not ready, especially in local mode. This changes forces a noop to be
returned when we are waiting on another service, which is the process
used elsewhere in the state engine.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-03-13 10:34:15 +00:00
Cameron Diver
911ee7f009
Run iptables rules synchronous to avoid locking errors
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-02-19 17:43:04 +00:00
Cameron Diver
5f82f6fd3f
Apply iptables rules to ipv6
Change-type: patch
Closes: #867
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-02-13 20:57:05 +00:00
Cameron Diver
49dbaaba12
Allow newlines to be part of environment variables
We were not allowing newlines previously by virtue of the regex not
allowing them. The docker daemon and supervisor handling code both
support them, so we allow them in the parsing code too.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-02-12 11:19:55 +00:00
Cameron Diver
6bf008cc85
Remove environment variable whitespace trimming
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-02-11 17:13:35 +00:00
Cameron Diver
88f19b4147
Set default delta apply timeout of 0
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-02-11 10:21:04 +00:00
Cameron Diver
f3264862ca
fix: Normalize ports from compose file on instantiation
Adjacent ports are always grouped together by docker when reporting the
container state (from an inspect), so adjacent ports defined in the
compose file would not match as they would not have been normalized.

We make sure to always normalize the input port configuration, so that
it will match the docker output (if it should).

We also don't sort in the fromComposePorts function anymore as that is
handled by the normalize function.

Closes: #897
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-02-08 15:14:07 +00:00
Cameron Diver
2e09ed071c
Ensure the balena-fin always has it's overlay assigned
Change-type: patch
Closes: #884
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-02-05 16:19:42 +00:00
Cameron Diver
db74e748a1
refactor: Fully type and validate config module set and get
We define the type for each config value, and validate the data when
retrieving and setting it.

Change-type: minor
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-11 10:37:25 +00:00
Cameron Diver
01ed7bb103
refactor: Convert ApiBinder module to typescript
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-08 11:59:56 +00:00
Cameron Diver
3a130f4f9c
refactor: Change config modules export type to new-style
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-02 18:12:22 +00:00
Cameron Diver
dc34025545
refactor: Generate and normalise PortMaps from compose ports in-class
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-02 12:19:30 +00:00
Cameron Diver
e82749e63a
tests: Add test case for ascending ports issue
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-02 11:31:25 +00:00
Cameron Diver
b32fba43e1
refactor: Convert DeviceConfig module to typescript
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-12-21 17:18:35 +00:00
Cameron Diver
de5157c04c
refactor: Change export style of db to named
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-12-21 17:18:33 +00:00
Cameron Diver
96b614ad6a
Replace offline -> unmanaged in logs and tests
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-12-17 10:53:24 +00:00
Cameron Diver
82602abf8d
config: Replace supervisorOfflineMode and offlineMode with unmanaged
Change-type: major
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-12-14 15:01:41 +00:00
Cameron Diver
19f8b75766
Revert "localMode: Remove local mode from device-config"
This reverts commit eda477ff86.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-12-14 11:05:57 +00:00
Cameron Diver
64a8c03eba
unmanged: Don't require a device name when setting a target state
Also set a default device name of 'local', to avoid an undefined value.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-11-29 11:35:08 +00:00
Cameron Diver
eda477ff86
localMode: Remove local mode from device-config
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-11-29 11:35:07 +00:00
Cameron Diver
5c04df2cb8
compose: Fix network option field names and update dockerode types
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-11-28 17:43:31 +00:00
Cameron Diver
b37ce5e3d9
config: Set default apiKey of empty string to avoid undefined keys
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-11-28 14:53:09 +00:00
Cameron Diver
f48e85095b
events: Correctly proxy mixpanel events
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-11-21 13:09:06 +00:00
Cameron Diver
0ff52751d8
tests: Increase timeout for test suites
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-11-06 15:27:58 +00:00
Cameron Diver
e270652122
Remove ts-node from test process
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-11-06 12:05:35 +00:00
Pagan Gazzard
d6e9283a15 Fix coffee-script lint failures 2018-11-02 14:50:12 +00:00
Pablo Carranza Velez
8298487a88 Rename most of the documentation and variable names from resin to balena
Change-type: minor
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-11-01 17:00:13 -07:00
Akis Kesoglou
b71c59c002 api-binder: Use API v5
Change-type: major
2018-11-01 13:17:30 +02:00
Pablo Carranza Velez
1e47d0c434 test: Use longer delays in logger tests
We've seen at least one case where the 10ms delay
wasn't enough, causing the tests to fail where they shouldn't.

(Ideally we should find a better way to detect when the streams
have finished propagating, but this will avoid failures in the meantime).

Change-type: patch
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-10-25 16:48:06 -07:00
Pablo Carranza Velez
b3860b2b70 fix: Store and retrieve device config without namespaces
This avoids issues on provisioning where the current state
(esp. config.txt) that we want to save is retrieved without
a RESIN_ or BALENA_ prefix, causing those values to be lost.

Change-type: patch
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-10-20 04:40:55 +02:00
Pablo Carranza Velez
24cbfbb860 deviceConfig: allow BALENA_ config variables
They will take precedence over any existing RESIN_ variables. We strip both namespaces now
whenever we get the target values.

This also fixes preloading with a legacy config (the interface to get the config keys from
the legacy apps.json was broken).

Change-type: minor
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-10-18 17:20:53 +02:00
Pablo Carranza Velez
220f4b5810 service: Fix device name env vars by injecting and filtering in the interface with docker
Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-10-18 14:49:00 +02:00
Pablo Carranza Velez
ed3f5522ae Replace io.resin labels (and their env vars) with io.balena equivalents
But we keep backwards compatibility by normalizing existing io.resin labels
into io.balena ones, and adding both RESIN_ and BALENA_ env vars for these features.

Change-Type: minor
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-10-18 13:21:31 +02:00
Pablo Carranza Velez
8003f25c3d Add /tmp/balena lock and handover paths and BALENA_ env vars
We change the lockfile to /tmp/balena/updates.lock, and the resin-kill-me file to /tmp/balena/handover-complete.
In the host, we change to use /tmp/balena-supervisor instead of /tmp/resin-supervisor.

We add BALENA_ env vars in addition to the RESIN_ env vars.

We keep backwards compatibility by using both paths for the lockfile and handover, and keeping the RESIN_ env vars.

Changelog-entry: Move the handover and lock files to /tmp/balena, rename them, and add BALENA_ env vars
Change-type: minor
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
2018-10-16 15:12:04 +02:00
Cameron Diver
6e2801380b
Support local logging via standard logging interface
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-10-10 13:02:40 +01:00
Cameron Diver
5537ae4e2e
fix: Correctly handle array based network definitions for service
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-10-08 11:29:32 +01:00
Cameron Diver
06bbf9751a
compose: Support changing the tty option for compose services
This enables the switch to be added to the compose, and the handling of
docker messages has been changed to ensure that the multiplexed logs
which result are handled properly.

Change-type: minor
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-09-24 12:04:51 +01:00
Cameron Diver
892d227cc2
compose/service: Convert module to typescript and add network config
Stability improvements;
* Printing of unsupported compose fields
* Added a lot of tests
* All compose configuration has a default value, enabling better
comparison

Change-type: minor
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-09-21 10:30:27 +01:00
Cameron Diver
e0231f15e9
Compose: Support more network creation options
Change-type: minor
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-09-21 10:30:27 +01:00
Cameron Diver
08b1c90ea1
fix: Fix event-tracker being passed no options object
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-09-04 16:02:06 -07:00
Cameron Diver
c1748436d8
fix: Import logger properly and add startup test
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-09-04 15:53:49 -07:00
Cameron Diver
8ca42062a3
events: Report supervisor version with any mixpanel events
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-09-04 13:17:20 -07:00
Cameron Diver
607c722b0b
events: Add rate limiting based on event name when reporting to mixpanel
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-09-04 13:17:17 -07:00
Cameron Diver
67b5503ac4
Fix: Change import of ports module to be extensionless in tests
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-09-03 09:39:06 -07:00
Cameron Diver
d3a18da573
Refactor: Convert logging module to typescript
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-09-03 09:39:06 -07:00
Cameron Diver
8ee26adbbe
Refactor: Convert event tracker module to typescript
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-09-03 09:36:21 -07:00
Cameron Diver
f7958e6c6b
Dont restart service on device name change
Change-type: patch
Closes: #659
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-07-25 10:45:05 +01:00
Petros Angelatos
202362241c
logger: Add unit tests
Change-type: minor
Signed-off-by: Petros Angelatos <petrosagg@gmail.com>
2018-07-18 12:31:14 -07:00
Petros Angelatos
bba1da370b
logger: Remove pubnub leftovers
Change-type: minor
Signed-off-by: Petros Angelatos <petrosagg@gmail.com>
2018-07-18 12:31:10 -07:00
Cameron Diver
1604fdf276
Allow the enabling and disabling of persistent logging via env var
Closes: #698
Change-type: minor
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-07-17 19:43:54 +01:00
Cameron Diver
0e29e5f6f7
Refactor config code to be consistent in location
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-07-16 17:48:03 +01:00
Cameron Diver
5e5ff05b4f
Re-enable majority of tests
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-07-16 16:43:50 +01:00
Cameron Diver
39d8ac0133
Change config function providers to be mutable
Also change logsChannelSecret value to be queried with the api backend,
so that logs are not shared between instances. This has been implemented
as the first config function provider with mutability.

Change-type: minor
Closes: #675
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-06-26 14:02:25 +01:00
Cameron Diver
4027d286d1
Add tests for new port module
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-06-25 09:46:07 +01:00
Cameron Diver
7b77e45f69
Store port ranges as ranges, to reduce memory usage
Before this change, port ranges were iterated and stored as an object
per port mapping. Now the port ranges are stored as ranges until they
need to be converted to objects. The need to convert to objects still
exists as this is the format which the docker remote API expects, but
hopefully this should alleviate bugs like #644 by making the memory more
shorter-lived.

Also added more tests.

Change-type: patch
Closes: #644
Signed-of-by: Cameron Diver <cameron@resin.io>
2018-06-25 09:46:01 +01:00
Cameron Diver
809bc87b88
Convert config module to typescript
Extract config.json handling code out to separate modules too.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-06-18 13:41:18 +01:00
Cameron Diver
089f31cb5d
Pin a device to a commit when preload has a pinDevice field
Change-type: minor
Closes: #668
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-06-11 20:27:16 +01:00
Cameron Diver
5622ddce94
Convert iptables module to typescript
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-06-11 10:20:17 +01:00
Cameron Diver
1e48d02781
Move config backend code out to classes which implement a common base
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-06-06 14:50:21 +01:00
Cameron Diver
89627a3392
Add support for extlinux configuration files
Currently the only supported variable is RESIN_HOST_EXTLINUX_isolcpus

Change-type: minor
Closes: #671
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-06-05 17:00:45 +01:00
Cameron Diver
dc59c83409
Move boot config related code to config-utils module
This commit abstracts all of the boot config code out of the
device-config module, ready to extend with different config backends.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-06-05 17:00:45 +01:00
Akis Kesoglou
8479801674 Add support for Balena deltas
Resin’s delta server supports Balena deltas as version 3 deltas. This commit adds support for triggering delta generation for Balena deltas, and applying them locally to the device via a simple pull.

The delta version to use when updating has been abstracted away as an env var that is user-defined. The default value is still instructing use of rsync deltas (v2).

Change-Type: minor
2018-05-23 20:59:56 +03:00
Cameron Diver
bc37ee56e4
Check against application source for target applications
The supervisor will now check that a source of an application matches
the current source, and only start it if so.

Change-type: patch
Closes: #658
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-05-22 12:11:57 +01:00
Cameron Diver
bea0b00804
Ignore leading and trailing whitespace when parsing env vars
Change-type: patch
Closes: #644
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-05-22 11:32:31 +01:00
Cameron Diver
ea8e8d2f5f
Don't generate config fields in offline mode
Change-type: patch
Closes: #648
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-05-21 14:45:32 +01:00
Cameron Diver
96d7bbefd0
Fix handling of systemd errors when polling for log to display
Change-type: patch
Closes: #610
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-05-08 15:12:13 +01:00
Cameron Diver
9150c3fdbc
Convert conversions module to typescript
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-05-02 09:36:21 +01:00
Pablo Carranza Velez
652b596c80 Add some more unit tests to the multicontainer supervisor
We add a bunch of additional unit tests, and also a coverage report using istanbul.

The tests are not meant to cover everything, but they're a first attempt at having *some* unit testing
on the supervisor. There's much to improve but hopefully it helps catch obvious errors.

Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
2018-04-30 19:39:01 -07:00
Cameron Diver
ddbf3418de
Remove trailing slashes from working directories of services
This is to combat when a working directory is in the compose file for a
service with a trailing slash. Docker will strip this slash and that
means service comparisons will fail going forward - even if they are the
same.

Change-type: patch
Closes: #635
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-04-27 18:03:36 +01:00
Cameron Diver
5b8f1ce334
Add tests for validation code
Change-type: minor
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-04-27 16:09:50 +01:00
Pablo Carranza Velez
1ce9560427 Add additional tests to compose Service, and add linting to tests
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
2018-04-11 15:15:30 -07:00
Cameron Diver
1c27ebc354
Fix parsing of non-unit memory numbers and add tests
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-04-04 14:03:24 +01:00
Lucian Buzzo
9bca2557ea Add mocha to enable unit testing
Implements a simple test for `parseMemoryNumber`

change-type: minor
2018-04-04 13:22:17 +01:00