1039 Commits

Author SHA1 Message Date
Cameron Diver
37945b4aa5 Don't attempt to setup a log stream to the cloud before provision
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-07-09 13:21:05 +01:00
Cameron Diver
20a83e8e0a Add more typescript conversions and export utilities from existing
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-07-09 13:21:05 +01:00
Michel Wohlert
39c9810bbc Merge remote-tracking branch 'balena/master' into instantly-apply-update-on-api-request 2019-07-09 13:10:15 +02:00
Michel Wohlert
7a42b6719a Apply targetState immediately after update request
Cancel delayed promise if exists and schedule a new one without delay, when /v1/update is called

Change-type: patch
2019-07-06 00:06:42 +02:00
Cameron Diver
89807c21fa
Fix network composition parsing, and correctly report compose config
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-07-04 11:47:39 +01:00
Cameron Diver
e5d7379b74
Keep the network and volume models consistent across usage
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-07-03 10:42:42 +01:00
Cameron Diver
eaff3a2ee5
Pass around instantiated Network objects when comparing state
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-07-03 10:36:38 +01:00
Cameron Diver
23e564389d
Seperate volume handling into manager and object classes
This change also makes sure that in the application-manager workflow we
pass around instances of the Volume class, rather than just the config.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-07-03 10:36:37 +01:00
Pablo Carranza Velez
743c8406c9 fix: When reporting initial config, avoid creating a "VPN enabled" unnecessary override
Since we were comparing the VPN's value before adding the explicit "true", there were cases
were the VPN is off, and therefore "value" didn't match the default, so the supervisor would
create a device specific SUPERVISOR_VPN_CONTROL = true, which is unnecessary and causes issues if
users don't expect this and move the device to an app that has VPN disabled. The correct behavior
is to compare "varValue" and only create a device config var if this value differs from the default.

(This was the behavior before the TS conversion in 01ed7bb103b4df8fb0679cf858220db42d4a0b92 )

Change-type: patch
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2019-07-01 12:28:46 -07:00
Roman Mazur
645bc6c185
Ensure we get input on parsing errors
We wrap JSON and date parsing code to ensure input data is logged in case of an error.

Change-type: minor
Signed-off-by: Roman Mazur <roman@balena.io>
2019-06-26 17:40:02 +03:00
Roman Mazur
7c4d8d7653
Ensure local mode switch runs before target state
This change makes DeviceState to wait until local mode switch is definitely
completed before actually applying the state, which avoids races in state cleanup.

Change-type: patch
Signed-off-by: Roman Mazur <roman@balena.io>
2019-06-26 17:40:02 +03:00
Roman Mazur
4974c9200c
Clean up local mode engine objects using snapshots
Snapshot is collected to compare with engine state when local mode is turned off.

Change-type: minor
Signed-off-by: Roman Mazur <roman@balena.io>
2019-06-26 17:40:01 +03:00
Roman Mazur
024b9c45f4
Report device state in local mode
In local mode, we now update device status on the backend,
but omit applications info in our updates.

Closes: #959
Change-type: minor
Signed-off-by: Roman Mazur <roman@balena.io>
2019-06-20 15:38:33 +03:00
Cameron Diver
8c69ae7645 Don't drop failed api patch data
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-06-20 12:56:10 +01:00
Cameron Diver
65019bb5bc
Fix typo in image selection
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-06-11 13:10:44 +01:00
Cameron Diver
b1e4e8399b
Re-parallelise current image retrieval code
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-06-11 07:50:38 +01:00
Cameron Diver
9c486275c9
Revert "Add a controlling variable for mixpanel reporting"
This reverts commit 1a7ed0f95bb24474ef7326309696da091e68a3d9.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-06-10 20:53:33 +01:00
Cameron Diver
ea64f0b0b3 Upgrade typescript and fix warnings
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-06-10 15:35:43 +01:00
Cameron Diver
2276dd54e1 Standardise all supervisor output logging
Also use the supervisor's own container logging monitoring code when
running livepush on the supervisor container.

Change-type: minor
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-06-10 11:38:45 +01:00
Cameron Diver
1a7ed0f95b
Add a controlling variable for mixpanel reporting
Change-type: patch
Closes: #990
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-05-29 11:53:47 +01:00
Cameron Diver
8279678052
Add wrapper around container logs saving, to reduce db load
Changes are collected together and exist in memory, for querying and
saving. Once every 10 mins, every changed timestamp is flushed to the
database.

Change-type: patch
Closes: #987
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-05-28 15:56:11 +01:00
Cameron Diver
2a27b1d51f Move to resin-lint 3 and fix lint errors
This is a massive commit, but nothing related to runtime has actually
changed, only the lint errors have changed.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-05-26 18:54:09 +01:00
Cameron Diver
1155d757e4 Make delta application logs easier to parse for multicontainer
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-05-15 12:41:27 +01:00
Cameron Diver
4e783fcd86 Allow system messages to not be tracked
Also don't track update lock events

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-05-15 10:19:15 +01:00
Cameron Diver
50fdee4290 Format update lock logs with pretty-ms
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-05-15 10:19:14 +01:00
Cameron Diver
27b68b5845 Display update lock notification in dashboard logs
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-05-15 10:18:48 +01:00
Cameron Diver
5894a9baf3 Show better logs for UpdatesLockedError
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-05-15 10:18:47 +01:00
Cameron Diver
5ace459ad2 Clear backoff counter on new target state
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-05-15 10:18:47 +01:00
Cameron Diver
932a6f2997
Fix supervisor tests for node v10.12.0 and above
Changes in the node engine related to streams would cause the gzip
streams flush function to be called at the wrong times. The sinon fake
timers were also interacting with this.

We use setImmediate to call the flush function, and remove sinon timers
for the logging tests.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-05-10 12:36:46 +01:00
Cameron Diver
51703621a5
Cache delta auth token request results for 10 minutes
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-05-10 11:17:48 +01:00
Cameron Diver
76e0361b0e Fix dockerode type failures
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-05-03 11:31:13 +01:00
Cameron Diver
8424fb44f6 Perform case-insensitive checking when converting booleans from strings
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-05-01 12:33:20 +01:00
Cameron Diver
9fae4bab62 fix: Run db log cleanup on init - after migrations
Before this change the first time the cleanup code runs would be before
the migrations have had a chance to execute. This change makes it so
that the cleanup code always runs once the migrations have finished.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-04-25 16:56:29 +01:00
Cameron Diver
0fa47f635b
fix: Correctly handle multiple hosts ports pointing to a container port
When assigning multiple host ports to a single container port before
this change, the supervisor would incorrectly take only the first host
port into consideration. This change makes it so that every host port
per container port is considered.

Closes: #986
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-04-25 13:48:04 +01:00
Cameron Diver
9e3fae5852
compose: Remove unique expose entries after adding all entries
Prior to this change, we would `_.uniq` the expose value before adding
values from the port mappings. This could cause ports to get added
twice, which would cause the supervisor to think that there is a
configuration mismatch.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-04-11 10:20:20 +01:00
Cameron Diver
892cf1961e
Don't attempt to report any state during local mode
Even though this would never have attempted to report the state to the
api during local mode, it leaves behind artifacts which would cause the
state to be sometimes reported when exiting local mode. This would cause
the api to reject the update unecessarily.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-04-10 15:40:52 +01:00
Cameron Diver
80031b76e4
types: Upgrade dockerode types, and remove fixes which are superceded
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-04-08 10:46:28 +01:00
Cameron Diver
5943d3117c
Run database cleanup on startup in addition to once a day
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-04-05 10:06:29 +01:00
Cameron Diver
760b18dd2a
fix: Fix non-tty container message parsing
This had a bug where it was using the `in` operator on a list. It may
have worked for some cases, but would have failed for others.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-04-05 10:06:28 +01:00
Cameron Diver
e148ce0529
Report all logs from a container's runtime
We add a database table, which holds information about the last
timestamp of a log successfully reported to a backend (local or remote).
We then use this value to calculate from which point in time to start
reporting logs from the container. If this is the first time we've seen
a container, we get all logs, and for every log reported we save the
timestamp. If it is not the first time we've seen a container, we
request all logs since the last reported time, ensuring no interruption
of service.

Change-type: minor
Closes: #937
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-04-05 10:06:27 +01:00
Cameron Diver
25fd11bed3
Refactor container logging interface and rename logging-backends
Container logging is now handled by a class which attaches and emits
information from the container. We add these to the directory
logging-backends/, and rename it to logging/.

Change-type: minor
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-04-05 10:06:26 +01:00
Cameron Diver
196f173e13
ux: Show a supervisor starting log message in dashboard
Change-type: minor
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-04-05 10:06:25 +01:00
Cameron Diver
0504776169
ux: Remove service already running log message
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-04-05 10:06:24 +01:00
Pablo Carranza Velez
9961ebb41d In /v1/update, return 202 when we're not updating immediately
We also add a catch to any errors when getting configuration, and send 503 in this case, even if it's
unlikely.

Change-type: patch
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2019-04-03 09:15:50 -07:00
Pablo Carranza Velez
8f07bf62de Add a random jitter to target state polls, and a config var to ignore update notifications and not poll immediately after startup
This commit does two related things:

* We make the poll interval a random time between 0.5 and 1.5 times the configured interval.
* We introduce the BALENA_SUPERVISOR_INSTANT_UPDATE_TRIGGER configuration variable, that defaults to true. If this variable is set
to false, then calls to /v1/update are ignored, and on startup the supervisor waits for a poll interval before getting the target state.

This will help especially on cases where there's a large number of devices on a single network. By disabling instant updates and setting a large
poll interval, we can now achieve a sitation where not all devices apply an update at the same time, which can help avoid
overwhelming the network.

Change-type: minor
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2019-04-03 09:15:49 -07:00
Cameron Diver
9a343316b2
Fix service comparison when starting a stopped service
When comparing a stopped container after a start request, the container
ID will be present in the target state (where usually it is not). We
were already filtering this value out of the current state, but
neglected to do so for the target state. This change now ensures we
remove it from both alias lists if it exists.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-04-03 13:08:22 +01:00
Pablo Carranza Velez
c902706600 Fix migration of legacy apps when there's more than one app in the local DB
In an edge case observed in the field, a supervisor's database held two applications
because the device had been moved and the update lock was set in the old app. This causes
the updated supervisor to be unable to start, logging "No compatible releases found in API",
because it can't fetch the release for the app it was moved from.

This commit changes the migration code to iterate through all apps, and remove any for which
we can't get a release.

Change-type: patch
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2019-03-28 17:14:06 -07:00
Cameron Diver
175cbfee50
Fix typo in delta request error message
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-03-28 12:31:50 +00:00
Cameron Diver
c4b7fb481a
Remove log-timestamp due to having journald logs
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-03-28 10:27:22 +00:00
Cameron Diver
1570935b2c
misc: Fix lint errors
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-03-28 10:27:22 +00:00