Commit Graph

1356 Commits

Author SHA1 Message Date
Cameron Diver
76de276b92
Only consider certain array fields without order
Various fields returned from the docker daemon don't retain order (for
example the volumes field). We now only select certain array values to
compare taking order into account.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-08-19 14:55:06 +01:00
Cameron Diver
86e8a85640
Take order into account when comparing service config arrays
Change-type: patch
Closes: #1060
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-08-15 17:07:22 +01:00
Cameron Diver
43cbf7dbba
Add an endpoint and module for reading journald logs
Change-type: minor
Closes: #1003
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-08-09 11:08:29 +01:00
Cameron Diver
c62f4afe8d
api: Avoid cloneDeep when copying target state.
This can cause issues where the stack is exhausted
on large applications.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
Signed-off-by: Heds Simons <heds@balena.io>
2019-07-25 15:23:18 +01:00
Roman Mazur
4688524e45
Filter both null and omitted values
The docker daemon can sometimes return null for various fields,
or omit the field altogether. The best thing to do with using
the docker api is to always use == null to cover both instances.

Change-type: patch
Signed-off-by: Roman Mazur <roman@balena.io>
2019-07-25 13:03:04 +03:00
Roman Mazur
4b56022d93
Use typed error to report inspection problems
This will throw a typed error in case when supervisor cannot inspect
its own container.

Change-type: patch
Signed-off-by: Roman Mazur <roman@balena.io>
2019-07-25 13:03:03 +03:00
Roman Mazur
4846acb25e
Prevent supervisor from deleting itself
With recent changes to local mode behavior, supervisor removes all engine objects
that were not present before entering local mode.
With this change, supervisor also detects its own resources and does not delete them
if it was updated during local mode operations.

Change-type: patch
Signed-off-by: Roman Mazur <roman@balena.io>
2019-07-25 13:02:57 +03:00
Rich Bayliss
f9aa4681fa
api: Perform network checks if device is unmanaged
Without these checks the API GET /v1/device doesn't return a
value for `ip_address`.

Change-type: patch
Signed-off-by: Rich Bayliss <rich@balena.io>
2019-07-24 15:40:53 +01:00
Cameron Diver
c109a24874 Support network_modes of service:<servicename>
That way that this is performed is by first adding a depends_on entry
for the target service if it appears in a network mode. Then when we
generate the docker container for this service, we use the containerId
of the target container and replace the network_mode with
`container:<id>`.

When comparing state, we check that the containerId still points to the
contianerId of the target container, and in this way we ensure that
when a network mode target container changes, we change the dependent
container too.

Change-type: minor
Closes: #851
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-07-23 13:41:19 +01:00
Cameron Diver
e91fa6e7b3
fix: Always reboot when applying boot config
Change-type: patch
Closes: #1037
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-07-22 16:07:39 +01:00
Cameron Diver
cbb079e8dc
fix: Don't consider volume supervised labels when comparing
Change-type: patch
Closes: #1039
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-07-19 12:17:38 +01:00
Matthew McGinn
2cb6ecc150
device-api: fix up small typos
Change-type: patch
Signed-off-by: Matthew McGinn <matthew@balena.io>
2019-07-18 13:07:53 +01:00
Cameron Diver
a796777967 Don't attempt to cleanup any target state referenced volumes
The code before this change could potentially remove a volume which
should not be removed if a container was deleted before the call that
references said volume.

To avoid this, we additionally filter the list of volumes to cleanup by
any that are referenced in the target state. This means that cleanup
will never remove it, as long as it's still supposed to be there,
regardless of if a container references it or not.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-07-11 16:35:29 +01:00
Cameron Diver
e076622fe6 Don't remove volumes when switching into local mode
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-07-10 14:25:47 +01:00
Cameron Diver
3304825216 Add a supervisor endpoint to cleanup orphaned volumes
Change-type: minor
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-07-10 14:25:44 +01:00
Cameron Diver
5357d4729d Only remove volumes when we're moving between applications
Change-type: major
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-07-10 14:25:22 +01:00
Cameron Diver
bb549a445b Dont try to set the commit if it's undefined
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-07-10 14:25:21 +01:00
Cameron Diver
37945b4aa5 Don't attempt to setup a log stream to the cloud before provision
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-07-09 13:21:05 +01:00
Cameron Diver
20a83e8e0a Add more typescript conversions and export utilities from existing
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-07-09 13:21:05 +01:00
Michel Wohlert
39c9810bbc Merge remote-tracking branch 'balena/master' into instantly-apply-update-on-api-request 2019-07-09 13:10:15 +02:00
Michel Wohlert
7a42b6719a Apply targetState immediately after update request
Cancel delayed promise if exists and schedule a new one without delay, when /v1/update is called

Change-type: patch
2019-07-06 00:06:42 +02:00
Cameron Diver
89807c21fa
Fix network composition parsing, and correctly report compose config
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-07-04 11:47:39 +01:00
Cameron Diver
e5d7379b74
Keep the network and volume models consistent across usage
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-07-03 10:42:42 +01:00
Cameron Diver
eaff3a2ee5
Pass around instantiated Network objects when comparing state
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-07-03 10:36:38 +01:00
Cameron Diver
23e564389d
Seperate volume handling into manager and object classes
This change also makes sure that in the application-manager workflow we
pass around instances of the Volume class, rather than just the config.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-07-03 10:36:37 +01:00
Pablo Carranza Velez
743c8406c9 fix: When reporting initial config, avoid creating a "VPN enabled" unnecessary override
Since we were comparing the VPN's value before adding the explicit "true", there were cases
were the VPN is off, and therefore "value" didn't match the default, so the supervisor would
create a device specific SUPERVISOR_VPN_CONTROL = true, which is unnecessary and causes issues if
users don't expect this and move the device to an app that has VPN disabled. The correct behavior
is to compare "varValue" and only create a device config var if this value differs from the default.

(This was the behavior before the TS conversion in 01ed7bb103 )

Change-type: patch
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2019-07-01 12:28:46 -07:00
Roman Mazur
645bc6c185
Ensure we get input on parsing errors
We wrap JSON and date parsing code to ensure input data is logged in case of an error.

Change-type: minor
Signed-off-by: Roman Mazur <roman@balena.io>
2019-06-26 17:40:02 +03:00
Roman Mazur
7c4d8d7653
Ensure local mode switch runs before target state
This change makes DeviceState to wait until local mode switch is definitely
completed before actually applying the state, which avoids races in state cleanup.

Change-type: patch
Signed-off-by: Roman Mazur <roman@balena.io>
2019-06-26 17:40:02 +03:00
Roman Mazur
4974c9200c
Clean up local mode engine objects using snapshots
Snapshot is collected to compare with engine state when local mode is turned off.

Change-type: minor
Signed-off-by: Roman Mazur <roman@balena.io>
2019-06-26 17:40:01 +03:00
Roman Mazur
024b9c45f4
Report device state in local mode
In local mode, we now update device status on the backend,
but omit applications info in our updates.

Closes: #959
Change-type: minor
Signed-off-by: Roman Mazur <roman@balena.io>
2019-06-20 15:38:33 +03:00
Cameron Diver
8c69ae7645 Don't drop failed api patch data
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-06-20 12:56:10 +01:00
Cameron Diver
65019bb5bc
Fix typo in image selection
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-06-11 13:10:44 +01:00
Cameron Diver
b1e4e8399b
Re-parallelise current image retrieval code
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-06-11 07:50:38 +01:00
Cameron Diver
9c486275c9
Revert "Add a controlling variable for mixpanel reporting"
This reverts commit 1a7ed0f95b.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-06-10 20:53:33 +01:00
Cameron Diver
ea64f0b0b3 Upgrade typescript and fix warnings
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-06-10 15:35:43 +01:00
Cameron Diver
2276dd54e1 Standardise all supervisor output logging
Also use the supervisor's own container logging monitoring code when
running livepush on the supervisor container.

Change-type: minor
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-06-10 11:38:45 +01:00
Cameron Diver
1a7ed0f95b
Add a controlling variable for mixpanel reporting
Change-type: patch
Closes: #990
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-05-29 11:53:47 +01:00
Cameron Diver
8279678052
Add wrapper around container logs saving, to reduce db load
Changes are collected together and exist in memory, for querying and
saving. Once every 10 mins, every changed timestamp is flushed to the
database.

Change-type: patch
Closes: #987
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-05-28 15:56:11 +01:00
Cameron Diver
2a27b1d51f Move to resin-lint 3 and fix lint errors
This is a massive commit, but nothing related to runtime has actually
changed, only the lint errors have changed.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-05-26 18:54:09 +01:00
Cameron Diver
1155d757e4 Make delta application logs easier to parse for multicontainer
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-05-15 12:41:27 +01:00
Cameron Diver
4e783fcd86 Allow system messages to not be tracked
Also don't track update lock events

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-05-15 10:19:15 +01:00
Cameron Diver
50fdee4290 Format update lock logs with pretty-ms
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-05-15 10:19:14 +01:00
Cameron Diver
27b68b5845 Display update lock notification in dashboard logs
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-05-15 10:18:48 +01:00
Cameron Diver
5894a9baf3 Show better logs for UpdatesLockedError
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-05-15 10:18:47 +01:00
Cameron Diver
5ace459ad2 Clear backoff counter on new target state
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-05-15 10:18:47 +01:00
Cameron Diver
932a6f2997
Fix supervisor tests for node v10.12.0 and above
Changes in the node engine related to streams would cause the gzip
streams flush function to be called at the wrong times. The sinon fake
timers were also interacting with this.

We use setImmediate to call the flush function, and remove sinon timers
for the logging tests.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-05-10 12:36:46 +01:00
Cameron Diver
51703621a5
Cache delta auth token request results for 10 minutes
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-05-10 11:17:48 +01:00
Cameron Diver
76e0361b0e Fix dockerode type failures
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-05-03 11:31:13 +01:00
Cameron Diver
8424fb44f6 Perform case-insensitive checking when converting booleans from strings
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-05-01 12:33:20 +01:00
Cameron Diver
9fae4bab62 fix: Run db log cleanup on init - after migrations
Before this change the first time the cleanup code runs would be before
the migrations have had a chance to execute. This change makes it so
that the cleanup code always runs once the migrations have finished.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-04-25 16:56:29 +01:00
Cameron Diver
0fa47f635b
fix: Correctly handle multiple hosts ports pointing to a container port
When assigning multiple host ports to a single container port before
this change, the supervisor would incorrectly take only the first host
port into consideration. This change makes it so that every host port
per container port is considered.

Closes: #986
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-04-25 13:48:04 +01:00
Cameron Diver
9e3fae5852
compose: Remove unique expose entries after adding all entries
Prior to this change, we would `_.uniq` the expose value before adding
values from the port mappings. This could cause ports to get added
twice, which would cause the supervisor to think that there is a
configuration mismatch.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-04-11 10:20:20 +01:00
Cameron Diver
892cf1961e
Don't attempt to report any state during local mode
Even though this would never have attempted to report the state to the
api during local mode, it leaves behind artifacts which would cause the
state to be sometimes reported when exiting local mode. This would cause
the api to reject the update unecessarily.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-04-10 15:40:52 +01:00
Cameron Diver
80031b76e4
types: Upgrade dockerode types, and remove fixes which are superceded
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-04-08 10:46:28 +01:00
Cameron Diver
5943d3117c
Run database cleanup on startup in addition to once a day
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-04-05 10:06:29 +01:00
Cameron Diver
760b18dd2a
fix: Fix non-tty container message parsing
This had a bug where it was using the `in` operator on a list. It may
have worked for some cases, but would have failed for others.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-04-05 10:06:28 +01:00
Cameron Diver
e148ce0529
Report all logs from a container's runtime
We add a database table, which holds information about the last
timestamp of a log successfully reported to a backend (local or remote).
We then use this value to calculate from which point in time to start
reporting logs from the container. If this is the first time we've seen
a container, we get all logs, and for every log reported we save the
timestamp. If it is not the first time we've seen a container, we
request all logs since the last reported time, ensuring no interruption
of service.

Change-type: minor
Closes: #937
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-04-05 10:06:27 +01:00
Cameron Diver
25fd11bed3
Refactor container logging interface and rename logging-backends
Container logging is now handled by a class which attaches and emits
information from the container. We add these to the directory
logging-backends/, and rename it to logging/.

Change-type: minor
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-04-05 10:06:26 +01:00
Cameron Diver
196f173e13
ux: Show a supervisor starting log message in dashboard
Change-type: minor
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-04-05 10:06:25 +01:00
Cameron Diver
0504776169
ux: Remove service already running log message
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-04-05 10:06:24 +01:00
Pablo Carranza Velez
9961ebb41d In /v1/update, return 202 when we're not updating immediately
We also add a catch to any errors when getting configuration, and send 503 in this case, even if it's
unlikely.

Change-type: patch
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2019-04-03 09:15:50 -07:00
Pablo Carranza Velez
8f07bf62de Add a random jitter to target state polls, and a config var to ignore update notifications and not poll immediately after startup
This commit does two related things:

* We make the poll interval a random time between 0.5 and 1.5 times the configured interval.
* We introduce the BALENA_SUPERVISOR_INSTANT_UPDATE_TRIGGER configuration variable, that defaults to true. If this variable is set
to false, then calls to /v1/update are ignored, and on startup the supervisor waits for a poll interval before getting the target state.

This will help especially on cases where there's a large number of devices on a single network. By disabling instant updates and setting a large
poll interval, we can now achieve a sitation where not all devices apply an update at the same time, which can help avoid
overwhelming the network.

Change-type: minor
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2019-04-03 09:15:49 -07:00
Cameron Diver
9a343316b2
Fix service comparison when starting a stopped service
When comparing a stopped container after a start request, the container
ID will be present in the target state (where usually it is not). We
were already filtering this value out of the current state, but
neglected to do so for the target state. This change now ensures we
remove it from both alias lists if it exists.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-04-03 13:08:22 +01:00
Pablo Carranza Velez
c902706600 Fix migration of legacy apps when there's more than one app in the local DB
In an edge case observed in the field, a supervisor's database held two applications
because the device had been moved and the update lock was set in the old app. This causes
the updated supervisor to be unable to start, logging "No compatible releases found in API",
because it can't fetch the release for the app it was moved from.

This commit changes the migration code to iterate through all apps, and remove any for which
we can't get a release.

Change-type: patch
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2019-03-28 17:14:06 -07:00
Cameron Diver
175cbfee50
Fix typo in delta request error message
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-03-28 12:31:50 +00:00
Cameron Diver
c4b7fb481a
Remove log-timestamp due to having journald logs
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-03-28 10:27:22 +00:00
Cameron Diver
1570935b2c
misc: Fix lint errors
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-03-28 10:27:22 +00:00
Cameron Diver
b3192679c7
fix: Correctly compare and generate network membership aliases
Before this change, service name resolution would only occur in the
default network. This was because we were not explicitly adding aliases
of the service names to the aliases fields.

We also fix the comparison, which would do funny things based on
container IDs, which was correct but unnecessary.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-03-28 10:27:21 +00:00
CameronDiver
e956242d45
Merge pull request #949 from balena-io/use-default-device-config
When a device config variables requested value is not valid, fallback to the default
2019-03-28 09:43:30 +00:00
Cameron Diver
c7499a6b12
device-config: Show invalid values in dashboard logs
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-03-27 14:03:45 +00:00
Cameron Diver
83d53cfb56
events: Allow system messages to not be tracked
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-03-27 13:14:40 +00:00
Cameron Diver
c211efe399
device-config: Use default values for any invalid target values
If a value is requested which does not pass validation, we instead set
it to the default value, to ensure that the state engine continues to
work and move towards the target state.

Change-type: minor
Closes: #938
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-03-27 13:08:04 +00:00
Pablo Carranza Velez
dab5d7546c Ensure the supervisor0 network uses a subnet less likely to cause conflicts
We put the supervisor0 network in the 10.114.104.0/25 subnet to avoid issues when the device
is in a network using the 172.17.* network.

We also ensure we recreate this network if it was created in the incorrect subnet (i.e. if we're updating
from an old supervisor that didn't do this), for which we have to kill any containers using this network.

Closes #731

Change-type: patch
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2019-03-21 16:02:49 -07:00
Pablo Carranza Velez
22a5b33196 fix: When pinning a preloaded device, ensure the pinning is done when retrying after a failure
Without this patch, if for some reason device pinning fails (e.g. connectivity goes down) or anything
interrupts the initialization after provisioning completes but before pinning is completed, after a retry
the supervisor would just skip the pinning code, leaving the device unpinned. This patch ensures that the
pinning procedure is run even if the device was already provisioned (as long as the pinning flag has been set,
of course). This matches the behavior that the CoffeeScript code had from before the TypeScript conversion.

Change-type: patch
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2019-03-20 17:50:26 -07:00
Pablo Carranza Velez
6e3bedeb1d fix: Return a promise when retrying provisioning to avoid continuing after a failure
Otherwise we'll keep doing the rest of the APIBinder init steps, like reporting initial config,
potentially before completing the provisioning.

Change-type: patch
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2019-03-20 16:48:20 -07:00
Pablo Carranza Velez
b374bd81dd fix: Await reporting the initial config before continuing APIBinder initialization
This avoid a race condition, in which config.txt can be cleared if a target state is fetched before the
initial values have been created as config vars.

Change-type: patch
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2019-03-20 12:27:20 -07:00
Pablo Carranza Velez
f32de99aff Fix typo when getting device config default values
Change-type: patch
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2019-03-19 13:46:17 -07:00
Pablo Carranza Velez
d64dcb4b40 fix: Correct use of $expand to avoid an exception when updating from a legacy OS
The last update of pinejs-client to pinejs-client-request made the way we were
using $expand on the migration break. This switches to the correct way of doing it now.

Change-type: patch
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2019-03-18 09:49:28 -07:00
Cameron Diver
b922789dee
device-api: Add v2/device/tags api endpoint
This endpoint will fetch the device tags from the balena api

Change-type: minor
Closes: #890
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-03-13 14:21:23 +00:00
Cameron Diver
3f231e8ff3
device-api: Add v2/device/name endpoint
This endpoint returns the last known device name from the API. This
differs from the BALENA_DEVICE_NAME_AT_INIT env var because this will
not change throughout the runtime of the container.

Closes: #908
Change-type: minor
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-03-13 14:21:22 +00:00
Cameron Diver
84356b82b8
state-engine: Return a noop when waiting for a dependency
We run the risk of the state engine exiting early when a dependency is
not ready, especially in local mode. This changes forces a noop to be
returned when we are waiting on another service, which is the process
used elsewhere in the state engine.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-03-13 10:34:15 +00:00
Cameron Diver
8f2d6f4d7b
Skip dependency check on kill in local mode
This function would usually check that an image is present for a
dependency, but in local mode the images would have never been inserted
into the database.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-03-13 10:34:14 +00:00
Cameron Diver
1aa58fd7b9
state-engine: Add an exponential backoff for device-config noops
To avoid unnecesarilly using resources, we add an exponential backoff
when the noops explicitly come from the device-config module.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-03-07 18:40:09 +00:00
Cameron Diver
ea1b247d3f
fix: Fix connectivity active VPN check
During the conversion to typescript, the VPN active check was being
performed on the directory, and not the file that the VPN creates,
meaning it would always return true (as we explicitly create the
directory on startup if it does not exist).

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-03-07 18:39:32 +00:00
Cameron Diver
6f79702099
state-engine: Add rate limited steps to device-config
In the case of an airgapped supervisor, with a target state that
requests the vpn be enabled, the supervisor will constantly loop on
trying to set the vpn to on. Unfortunately the vpn requires an internet
connection to be configured, so it will never be turned on.

We add the concept of no-ops to the device-config state change steps,
and don't end the state engine transition while these are present
(similar to how image pulls are implemented).

Change-type: minor
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-03-07 18:39:30 +00:00
Rich Bayliss
aeb96aa807
feature: Add BALENA_API_URL environment variable when using the balena-api feature label
When using the label `io.balena.features.balena-api` the supervisor will inject 2 environment
variables into the container:
- BALENA_API_KEY
- BALENA_API_URL

This allows the container to access the currently associated API using the KEY.

Change-type: patch
Signed-off-by: Rich Bayliss <rich@balena.io>
Connects-to: #847
2019-02-28 11:41:28 +00:00
Cameron Diver
987de0e097 debug: Print more information about failing validations
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-02-25 13:22:09 +01:00
Cameron Diver
f9626a3ee4 device-config: Add migration for SUPERVISOR_DELTA_APPLY_TIMEOUT
The default value for the delta apply timeout was changed from `''` to
`'0'` (note strings as these are database values) - but if the value
existed in the database already, this would fail validation. We add a
migration which will look explcitily for the failing value and switch it
to the new default.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-02-25 13:22:02 +01:00
Cameron Diver
c9507e013c Increase max payload size in bodyparser to avoid PayloadTooLarge errors
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-02-25 13:18:55 +01:00
Cameron Diver
0e3f260978
Fix provisioning workflow when UUID already exists
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-02-22 10:53:24 +00:00
Cameron Diver
911ee7f009
Run iptables rules synchronous to avoid locking errors
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-02-19 17:43:04 +00:00
Cameron Diver
5f82f6fd3f
Apply iptables rules to ipv6
Change-type: patch
Closes: #867
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-02-13 20:57:05 +00:00
Cameron Diver
7bd7f7e025
Improve error messages, and add description to ImageAuth error
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-02-13 15:44:42 +00:00
Cameron Diver
81ec85c581
fix: Request image authentication token with explicitly as json
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-02-13 15:43:55 +00:00
Cameron Diver
d9177404b5
Always back off on image fetch failure
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-02-13 15:43:18 +00:00
Cameron Diver
06580bf437
Don't treat a non-200 status response on patch as report errors
Non-200 errors were causing the watchdog to restart the supervisor,
which in some cases could cause a restart loop. Instead we change the
code to only treat communication failures as an error, and report status
code failures directly.

Change-type: patch
Closes: #843
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-02-12 13:56:54 +00:00
Cameron Diver
49dbaaba12
Allow newlines to be part of environment variables
We were not allowing newlines previously by virtue of the regex not
allowing them. The docker daemon and supervisor handling code both
support them, so we allow them in the parsing code too.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-02-12 11:19:55 +00:00
Cameron Diver
6bf008cc85
Remove environment variable whitespace trimming
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-02-11 17:13:35 +00:00
Cameron Diver
3d6dc88eb0
Make sure to correctly convert config emit events after validation
We were validating the input configuration values by coercing them to
the correct type, and then using the initial value to be saved (which
currently is always converted to a string).

We now use the coerced value as the actual value we will store, and more
importantly emit. This means that the config.on('change' ...) calls will
always be properly typed, which before this change was not a guarantee
that we could make.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-02-11 11:22:08 +00:00
Cameron Diver
88f19b4147
Set default delta apply timeout of 0
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-02-11 10:21:04 +00:00
Cameron Diver
f3264862ca
fix: Normalize ports from compose file on instantiation
Adjacent ports are always grouped together by docker when reporting the
container state (from an inspect), so adjacent ports defined in the
compose file would not match as they would not have been normalized.

We make sure to always normalize the input port configuration, so that
it will match the docker output (if it should).

We also don't sort in the fromComposePorts function anymore as that is
handled by the normalize function.

Closes: #897
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-02-08 15:14:07 +00:00
Cameron Diver
e9b51bbcd7
fix: Rework delete-then-download handling in state engine
In the original implementation it was possible that the delete did not
wait for the kill step to be finished, so it would not be deleted.

We seperate this process into two steps, to allow for the container to
have stopped before proceeding.

Change-type: patch
Closes: #841
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-02-06 11:41:05 +00:00
Cameron Diver
828a0fc345
Make the failure backoff time the same as the appUpdatePollTime
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-02-05 18:40:18 +00:00
Cameron Diver
146267b402
Backoff on image download error
Change-type: patch
Closes: #873
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-02-05 18:40:18 +00:00
Cameron Diver
c0e68bb069
Add the release to the status endpoint
Change-type: minor
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-02-05 17:57:58 +00:00
Cameron Diver
769e2f3c51
Add application status endpoint
Change-type: minor
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-02-05 17:57:57 +00:00
Cameron Diver
d863326783
misc: Improve typings
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-02-05 17:57:57 +00:00
Cameron Diver
2e09ed071c
Ensure the balena-fin always has it's overlay assigned
Change-type: patch
Closes: #884
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-02-05 16:19:42 +00:00
Cameron Diver
d8085a6ef8
fix: Don't attempt a pull if the token requests fails
Change-type: patch
Closes: #879
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-02-04 17:50:26 +00:00
Cameron Diver
d75c58f09d
Move legacy renaming to formatting function
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-02-04 11:08:36 +00:00
Cameron Diver
f12f474516
Fix typo in OVERRIDE_LOCK
Closes: #885
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-02-04 11:08:26 +00:00
Cameron Diver
ef7d993db6
Wrap services.getAll in a bluebird promise
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-31 10:09:25 +00:00
Cameron Diver
6a9ca2a60d
fix: Properly bind context to healthchecks
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-29 14:19:45 +00:00
Cameron Diver
707ddb5f45
fix: Only apply supervisor api authentication after healthcheck entry
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-29 14:19:45 +00:00
Cameron Diver
8315413046
Add missing ServiceManager methods
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-29 11:47:41 +00:00
Cameron Diver
ba000a73fc
refactor: Convert ServiceManager to typescript
Change-type: minor
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-25 13:52:05 +00:00
Cameron Diver
e5893c0ee0
Improve error and logger typings
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-25 13:52:04 +00:00
Cameron Diver
ecffa12e41
misc: Lint fixes
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-25 13:52:04 +00:00
Cameron Diver
02736113a3
refactor: Convert docker-utils module to typescript
Change-type: patch
Closes: #868
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-25 13:52:04 +00:00
Cameron Diver
9a97451e7a
fix: Fix intial config reporting
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-23 17:51:52 +00:00
Cameron Diver
24ce72a2c3
Clear up images.ts module code for simiplicity
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-23 16:40:15 +00:00
Cameron Diver
6f9d9e5de6
Prefer optional type in ConfigChangeMap
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-23 16:31:33 +00:00
Cameron Diver
6a1e787eee
Use dictionary for db key
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-23 16:30:43 +00:00
Cameron Diver
32acde0a99
refactor: Convert volumes module to typescript
Change-type: minor
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-22 20:25:04 +00:00
Cameron Diver
91b553dd32
refactor: Convert compose/images module to typescript
Change-type: minor
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-22 20:20:05 +00:00
Cameron Diver
1af13d3373
typings: Improve database typings by allowing unknown dictionary keys
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-22 13:25:39 +00:00
Cameron Diver
fed01c355c
typings: Improve docker-utils typings
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-22 13:25:01 +00:00
Cameron Diver
d1e1297f6d
misc: Add Nullable<T> helper type
Change-type: minor
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-21 11:18:41 +00:00
Cameron Diver
0505c0f976
config: Properly type the change events from config module
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-21 11:13:14 +00:00
Cameron Diver
06cdaddd71
Send connection message when streaming local mode logs
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-16 13:14:49 +00:00
Cameron Diver
9a98fc4c84
refactor: Small code changes
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-11 10:38:05 +00:00
Cameron Diver
6a3148ff80
refactor: Remove unnecessary validations and casts
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-11 10:37:25 +00:00
Cameron Diver
db74e748a1
refactor: Fully type and validate config module set and get
We define the type for each config value, and validate the data when
retrieving and setting it.

Change-type: minor
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-11 10:37:25 +00:00
Cameron Diver
ac4866170e
refactor: Remove scaffolding for unused mutable config functions
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-11 10:37:24 +00:00
Cameron Diver
81b17faab1
fix: Use logind manager to request reboots and shutdowns
Change-type: patch
Closes: #861
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-09 11:23:43 +00:00
Cameron Diver
a970ec5377
Fix context bind errors by switching to async/await
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-08 16:22:47 +00:00
Cameron Diver
de21c2ab2b
Specify TODO in DeviceApplicationState comment
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-08 11:59:56 +00:00
Cameron Diver
01ed7bb103
refactor: Convert ApiBinder module to typescript
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-08 11:59:56 +00:00
Cameron Diver
d5f4ac690f
refactor: Only promisify read and write locks once
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-08 11:59:54 +00:00
Cameron Diver
9decea1d3b
refactor: Convert supervisor api module to typescript
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-08 11:59:53 +00:00
Cameron Diver
3a130f4f9c
refactor: Change config modules export type to new-style
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-02 18:12:22 +00:00
Cameron Diver
dc34025545
refactor: Generate and normalise PortMaps from compose ports in-class
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-02 12:19:30 +00:00
Cameron Diver
557c32b80e
compose: Normalise target ports for comparison with docker's output
Docker always returns ports in ascending order, so if they aren't
specified like that in the compose, a restart loop would occur. This
patch changes the port maps to be stored in ascending order, based on
an alphabetical sort of the internalStart port (not taking into account
the protocol). This is the same as how Docker returns them, so they will
match, regardless of input form.

Change-type: patch
Closes: #824
Signed-off-by: Cameron Diver <cameron@balena.io>
2019-01-02 11:24:56 +00:00
Cameron Diver
b32fba43e1
refactor: Convert DeviceConfig module to typescript
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-12-21 17:18:35 +00:00
Cameron Diver
7ba1ab981c
types: Make DeviceApplicationState reflect state endpoint
Also change the return format of ApplicationManager.getStatus(), which
does not conform to the above.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-12-21 17:18:33 +00:00
Cameron Diver
de5157c04c
refactor: Change export style of db to named
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-12-21 17:18:33 +00:00
Cameron Diver
64db38204f
refactor: Add default export to logger module
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-12-21 17:18:33 +00:00
Cameron Diver
f4f67a5afc
refactor: Convert host-config module to typescript
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-12-21 17:18:32 +00:00
Cameron Diver
9fab0fc5cc
fix: Correctly type top level network IPAM config
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-12-20 10:07:38 +00:00
Pagan Gazzard
019190646e Update pinejs-client to pinejs-client-request 5.x
Change-type: patch
2018-12-19 17:54:53 +00:00
Cameron Diver
45b322b7e0
device-api: Add morgan to log api requests
Change-type: minor
Closes: #834
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-12-19 14:30:37 +00:00
Cameron Diver
b977b30dfe
refactor: Convert update-lock module to typescript
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-12-19 14:07:47 +00:00
Cameron Diver
ec37db597d
refactor: Convert systemd module to typescript
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-12-19 14:07:45 +00:00
Cameron Diver
e00954babd
refactor: Convert request module to typescript
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-12-19 14:07:45 +00:00
Cameron Diver
2ea657c95d
refactor: Convert migration module to typescript
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-12-19 14:07:44 +00:00
Pablo Carranza Velez
42737cb9e9 Fix a race condition that could cause an unnecessary restart of a service immediately after download
Up to now, there was a slim but non-zero chance that an image would be downloaded between the call to `@getTarget` inside deviceState
(which gets the target state and creates Service objects using information from available images), and the call to
`@images.getAvailable` in ApplicationManager (which is used to determine whether we should keep waiting for a download or start the
service). If this race condition happened, then the ApplicationManager would infer that a service was ready to be started (because
the image appears as available), but would have incomplete information about the service because the image wasn't available when
the Service object was created. The result would be that the service would be started, and then immediately on the next applyTarget
the ApplicationManager would try to kill it and restart it to update it with the complete information from the image.

This patch changes this behavior by ensuring that all of the additional information about the current state, which includes available images,
is gathered *before* building the current and target states that we compare. This means that if the image is downloaded after the call to getAvailable, the Service might be constructed with all the information about the image, but it won't be started until the next pass, because ApplicationManager will treat it as still downloading.

Change-type: patch
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-12-17 15:41:12 -03:00
Cameron Diver
96b614ad6a
Replace offline -> unmanaged in logs and tests
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-12-17 10:53:24 +00:00
Cameron Diver
c533631f70
unmanaged: Default to local mode in target state when unmanaged
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-12-17 10:53:23 +00:00
Cameron Diver
151af309fb
config: Force config values to strings when storing in db
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-12-17 10:53:23 +00:00
Cameron Diver
3ca1d7c864
join: Disable local mode when joining a cloud
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-12-17 10:53:23 +00:00
Cameron Diver
82602abf8d
config: Replace supervisorOfflineMode and offlineMode with unmanaged
Change-type: major
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-12-14 15:01:41 +00:00
Cameron Diver
5bea0fdc9d
fix: Give unmanaged target states a source of 'local'
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-12-14 15:01:41 +00:00
Cameron Diver
178e80d92a
Require an apikey for supervisor api in production unmanaged
Change-type: minor
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-12-14 15:01:41 +00:00
Cameron Diver
5bb3820d6a
Switch to local mode in unmanaged mode
Change-type: major
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-12-14 15:01:41 +00:00
Cameron Diver
91a6340563
Move config.json flag back to the database
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-12-14 15:01:40 +00:00
Cameron Diver
19f8b75766
Revert "localMode: Remove local mode from device-config"
This reverts commit eda477ff86.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-12-14 11:05:57 +00:00
Pablo Carranza Velez
b94921263a Use rimraf package instead of handmade function
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-12-12 14:22:15 -03:00
Pablo Carranza Velez
af717a3761 Stricter validation for backup file contents
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-12-12 14:17:34 -03:00
Pablo Carranza Velez
501272266b Add the ability to restore volumes from a backup.tgz in the data partition
Change-type: minor
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-12-12 14:17:33 -03:00
Pablo Carranza Velez
d5b2fcd4dd Update knex to 0.15.2 and sqlite3 to 4.0.4
We also replace a createTableIfNotExists in the migrations with hasTable then createTable, to
avoid a warning message about it being not recommended.

Change-type: patch
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-12-12 13:42:29 -03:00
Pablo Carranza Velez
a52683ade0 Fix typo that prevented deleting old resin/ supervisor images
Change-type: patch
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-12-11 13:27:39 -03:00
Heds Simons
80203f29ad
api: Ensure Supervisor API returns IP addresses
The move from pure CoffeeScript to TypeScript has brought a
few changes to the way transpiling happens. Previously, through
serendipity, the way `startIPAddressUpdate` was called worked
because of the binding convention pre-transpiling.

However, with the move to TypeScript, this has altered and
the assumption that a lack of parentheses would call the
method before supplying a callback into the returned function
is incorrect. The method must be specifically called first.

Connects-to: #836
Change-type: patch
Signed-off-by: Heds Simons <heds@balena.io>
2018-12-07 10:37:00 +00:00
Cameron Diver
64a8c03eba
unmanged: Don't require a device name when setting a target state
Also set a default device name of 'local', to avoid an undefined value.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-11-29 11:35:08 +00:00
Cameron Diver
eda477ff86
localMode: Remove local mode from device-config
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-11-29 11:35:07 +00:00
Cameron Diver
5c04df2cb8
compose: Fix network option field names and update dockerode types
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-11-28 17:43:31 +00:00
Pablo Carranza Velez
502167e267
fix: When updating from a legacy supervisor, use updated resource ids and image URL from the API
When updating from old supervisors (<7.0.0), we've been so far using a fake id 1 for serviceId, imageId
and releaseId since these were not available in the old supervisor. This causes problems when the supervisor
tries to report these values to the API. Moreover, the app from the legacy supervisor has an image URL
that doesn't include the content hash - this causes the supervisor to believe the image is not really downloaded
and try to fetch it again.

To fix these issues, we add a request to the API when the supervisor starts up and detects that there's a legacy
app that needs to be normalised. We fetch the appropriate release, and use it to populate the resource ids
and the updated image URL.

This should avoid the unnecessary image download, and errors reporting target state after an update.

Change-type: patch
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-11-28 17:19:55 +00:00
Cameron Diver
311eaf0ac0
device-api: Add container id endpoint
Change-type: minor
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-11-28 16:44:19 +00:00
Cameron Diver
6317b16138
Convert network module to typescript
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-11-28 15:15:06 +00:00
Cameron Diver
b37ce5e3d9
config: Set default apiKey of empty string to avoid undefined keys
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-11-28 14:53:09 +00:00
Cameron Diver
142d2a7c6f
config: Move config.ts -> config/index.ts
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-11-28 14:53:08 +00:00
Cameron Diver
a2e2948a4e
Allow local mode to be controlled via config.json
Change-type: minor
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-11-28 14:53:08 +00:00
Cameron Diver
2e80b49da1
Don't start connectivity check when in offlineMode
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-11-28 14:53:07 +00:00
Cameron Diver
ce543d820f
Improve UX when apps.json is not present
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-11-28 14:53:06 +00:00
Cameron Diver
f56be737ee
config: Switch default device type to unknown
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-11-28 14:26:05 +00:00
Cameron Diver
dfa9034fe9
Handle empty apiEndpoint when detecting mixpanel host
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-11-28 14:26:02 +00:00
Cameron Diver
940d8c9862
fix: Set default apiEndpoint to empty
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-11-28 13:11:45 +00:00
Shaun Mulligan
aface89fd2 compose: Set default config for stopSignal to SIGTERM
Change-type: patch
Signed-off-by: Shaun Mulligan shaun@balena.io
2018-11-27 12:21:08 -08:00
Shaun Mulligan
646a510310 compose: Change default stop signal from '' to SIGTERM
Change-type: patch
Signed-off-by: Shaun Mulligan <shaun@balena.io>
2018-11-27 12:13:20 -08:00
Cameron Diver
fe44d496fb
compose: Change default grace period to 10s
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-11-22 12:18:18 +00:00
Cameron Diver
f48e85095b
events: Correctly proxy mixpanel events
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-11-21 13:09:06 +00:00
Cameron Diver
96ecef8052
Fix m00001 migration by using targetValues
Change-type: patch
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-11-13 22:14:06 +00:00
Pagan Gazzard
2a17184d32 Let typescript do the json resolution so we get proper types
Change-type: patch
2018-11-06 18:13:52 +00:00
Cameron Diver
00bd88c50d
fix: Rework code to avoid prettier errors
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-11-06 11:07:55 +00:00
Cameron Diver
dfdc371a9c
device-api: Also support serviceName in v2 service endpoints
Now you can either pass a serviceName or imageId to restart a specific
service, which is much easier from a device container.

Change-type: minor
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-11-05 11:44:29 +00:00
Cameron Diver
9963d24675
fix: Fix typo in image cleanup code
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-11-05 11:44:29 +00:00
Cameron Diver
2bdb34d78d
fix: Correctly type bluebird promises in application-manager typings
Signed-off-by: Cameron Diver <cameron@balena.io>
2018-11-05 11:44:26 +00:00
Pagan Gazzard
235f9f04af Remove some type casts
Change-type: patch
2018-11-02 20:45:13 +00:00
Pagan Gazzard
83927f36d8 Update to typescript 3
Change-type: patch
2018-11-02 15:41:14 +00:00
Pagan Gazzard
ab6401343b Run prettier on the typescript files 2018-11-02 14:50:13 +00:00
Pagan Gazzard
d6e9283a15 Fix coffee-script lint failures 2018-11-02 14:50:12 +00:00
Pablo Carranza Velez
4bf6f75a4f Remove unused codepaths that do deltas from resin/scratch
The supervisor has been doing regular pulls instead of deltas
from scratch for a while now. We remove remaining references to
resin/scratch, and add a handler in docker-utils to fall back
to a regular pull with a null deltaSource (which should never be
called anyways, but is left as a precaution).

Change-type: patch
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-11-01 17:00:13 -07:00
Pablo Carranza Velez
8298487a88 Rename most of the documentation and variable names from resin to balena
Change-type: minor
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-11-01 17:00:13 -07:00
Pablo Carranza Velez
007bd168bb When updating from resin-supervisor to balena-supervisor, try to delete old resin-supervisor images
Change-type: minor
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-11-01 14:59:41 -07:00
Akis Kesoglou
b71c59c002 api-binder: Use API v5
Change-type: major
2018-11-01 13:17:30 +02:00
Pablo Carranza Velez
989a5e2c6a fix: When calling the /v1/update endpoint, always trigger a target state apply
getAndSetTargetState in the APIBinder had a check for whether the target state has changed.
When triggering an update from the API, we want to *always* call triggerApplyTarget, especially
when the update is forced. Otherwise the API endpoint doesn't work for forced updates (and in general,
will rarely trigger an update unless a change in the API's target state has happened)

Change-type: patch
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-10-31 14:26:56 -07:00
Pablo Carranza Velez
d5094d8d3b fix: Properly check for the balena-api label to inject the API key env vars
Change-type: patch
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-10-31 13:13:09 -07:00
Pablo Carranza Velez
d6ee60100d fix: When setting target state, delete old apps from the same source
In commit 19cd310da3 this line was deleted,
probably to avoid deleting local mode apps when setting the API target and
viceversa but we need to delete old apps to avoid problems when moving
the device between apps.

We now filter by source to avoid the problem with local mode too.

Change-type: patch
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-10-31 11:20:42 -07:00
Pablo Carranza Velez
1c5891ec09 ApplicationManager: when comparing images to save metadata, take docker image ids into account
Otherwise we may skip saving a target image to the db when updating from legacy supervisors,
which in turn prevents from deleting the legacy image entry (with imageId = 1), leaving the
supervisor in a state where it can't report its current state to the API.

While we're at it, we also remove an unused variable in _getStatus.

Change-type: patch
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-10-23 15:43:59 -07:00
Cameron Diver
64de3d2f63
fix: Use fat arrow for cleanup method
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-10-23 19:13:34 +01:00
Cameron Diver
1879c76b00
fix: Run checkTruthy on config values before using them
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-10-23 18:32:38 +01:00
Pablo Carranza Velez
f5b005bcb2 fix: Properly get the config from target state before comparing during initial config reporting
Change-type: patch
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-10-22 13:18:53 -07:00
Pablo Carranza Velez
0eaae3d670 fix: When checking for legacy containers, properly get the labels from config
Change-type: patch
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-10-20 19:08:53 +02:00
Pablo Carranza Velez
922597e582 fix: Avoid trying to report duplicated values in initial config
We do this by formatting the keys from the target state before comparing them
with the ones from the current state (that are already formatted to strip the namespace
prefix).

Change-type: patch
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-10-20 18:58:13 +02:00
Pablo Carranza Velez
b3860b2b70 fix: Store and retrieve device config without namespaces
This avoids issues on provisioning where the current state
(esp. config.txt) that we want to save is retrieved without
a RESIN_ or BALENA_ prefix, causing those values to be lost.

Change-type: patch
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-10-20 04:40:55 +02:00
Pablo Carranza Velez
32d5e58ead Keep defaulting tty to true to avoid accidental breakage
Otherwise old releases (where applications expected tty to be true)
would break.

Change-type: patch
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-10-19 17:54:13 +02:00
Pablo Carranza Velez
6fb0147d3c Fix preloading in flasher images by reading apps.json if target hasn't been set
i.e. if we're not provisioned or if the target state is empty (of apps), then we
read apps.json to preload. We then mark that the target state has been set to avoid
trying to preload again if we ever get an empty target state from the API.

Change-type: patch
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-10-19 14:42:29 +02:00
Pablo Carranza Velez
583ce34c01 apiBinder: Keep prepending RESIN_ to reported config variables
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-10-18 17:20:53 +02:00
Pablo Carranza Velez
24cbfbb860 deviceConfig: allow BALENA_ config variables
They will take precedence over any existing RESIN_ variables. We strip both namespaces now
whenever we get the target values.

This also fixes preloading with a legacy config (the interface to get the config keys from
the legacy apps.json was broken).

Change-type: minor
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-10-18 17:20:53 +02:00
Cameron Diver
e04de0a170
fix: Add typescript cast to fix type error
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-10-18 15:14:12 +01:00
Cameron Diver
d939b2b9e6
fix: Remove debugging console logs
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-10-18 15:04:04 +01:00
Cameron Diver
6ad3a7981d
fix: Redact environment variable values in debug output
Chnage-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-10-18 15:04:04 +01:00
Cameron Diver
2a94a7b730
fix: Apply default config options when they're falsy
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-10-18 15:04:04 +01:00
Cameron Diver
5569e2bc84
fix: Always set the container name when generating creation options
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-10-18 15:04:04 +01:00
Pablo Carranza Velez
e17bb59844 config: Allow atomic config.json writes on balenaOS
Change-type: minor
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-10-18 15:01:49 +02:00
Pablo Carranza Velez
0a4eefa9c6 network: Ignore resin-redsocks and balena-redsocks when reporting IP addresses
Change-type: patch
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-10-18 15:01:49 +02:00
Pablo Carranza Velez
31e730600d Use the DOCKER_SOCKET env var for the user container's balena socket mount
Instead of hardcoding balena.sock we use this variable since the path changes
with the balena -> balenaEngine rename.

We keep also mounting into balena.sock for backwards compatibility (even though
most tools should transparently use the DOCKER_HOST env var).

Change-type: minor
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-10-18 15:01:49 +02:00
Pablo Carranza Velez
220f4b5810 service: Fix device name env vars by injecting and filtering in the interface with docker
Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-10-18 14:49:00 +02:00
Pablo Carranza Velez
88b3682d51 ServiceManager: add BALENA_DEVICE_NAME_AT_INIT to injected env vars
Change-type: minor
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-10-18 13:21:31 +02:00
Pablo Carranza Velez
ed3f5522ae Replace io.resin labels (and their env vars) with io.balena equivalents
But we keep backwards compatibility by normalizing existing io.resin labels
into io.balena ones, and adding both RESIN_ and BALENA_ env vars for these features.

Change-Type: minor
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-10-18 13:21:31 +02:00
Pablo Carranza Velez
83b4d5878e service-manager: Delete the correct handover complete file
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-10-17 17:38:49 +02:00
Pablo Carranza Velez
047ca2ae68 service: Fix typo and type errors
Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-10-17 11:34:13 +02:00
Pablo Carranza Velez
87a8d8f8f4 Env var changes in API docs, new private env vars, remove unused vars
We update the supervisor API docs to reflect the new `BALENA_` injected env vars.
We also add the new sensitive env vars to the list of vars not sent by the API.

And we remove the unused RESIN_DATA_PATH and RESIN_PROXYVISOR_HOOK_RECEIVER from the
vars the supervisor parses from its own environment.

Signed-off-by: Pablo Carranza Velez <pablo@balena.io>
2018-10-16 15:12:05 +02:00
Pablo Carranza Velez
802025c01f Make handover-complete an alternative to the resin-kill-me file
Also fixes the handover wait (since the service interface for it was missing).
We add some logging to this process too.

Change-type: minor
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
2018-10-16 15:12:04 +02:00
Pablo Carranza Velez
8003f25c3d Add /tmp/balena lock and handover paths and BALENA_ env vars
We change the lockfile to /tmp/balena/updates.lock, and the resin-kill-me file to /tmp/balena/handover-complete.
In the host, we change to use /tmp/balena-supervisor instead of /tmp/resin-supervisor.

We add BALENA_ env vars in addition to the RESIN_ env vars.

We keep backwards compatibility by using both paths for the lockfile and handover, and keeping the RESIN_ env vars.

Changelog-entry: Move the handover and lock files to /tmp/balena, rename them, and add BALENA_ env vars
Change-type: minor
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
2018-10-16 15:12:04 +02:00
Pagan Gazzard
660d9ff6f3 Disable express' x-powered-by header
Change-type: patch
Signed-off-by: Pagan Gazzard <page@resin.io>
2018-10-16 13:26:21 +02:00
Cameron Diver
b1434e44ae
fix: overlay compose fields on image for healthcheck
This commit changes a bug where the compose healthcheck would always
overwrite the healthcheck set by the image - even if no compose
healthcheck exists.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-10-16 10:34:37 +01:00
Cameron Diver
16e7522a1f
fix: Remove support for broken storageOpt option
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-10-16 08:22:19 +01:00
Zubair Lutfullah Kakakhel
73b9895bdf
backend: Remove a few blacklisted config.txt options
These options were discussed in an arch call and the conclusion was
that they don't need to be in the blacklist.

Change-type: patch
Changelog-entry: Remove a few blacklisted config.txt options
Signed-off-by: Zubair Lutfullah Kakakhel <zubair@resin.io>
2018-10-15 11:41:22 +01:00
Cameron Diver
a143a07aaa
fix: Return void from config setter
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-10-15 11:12:47 +01:00
Cameron Diver
d7edd3eb63
fix: Don't assume config is defined for legacy images in logger
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-10-15 10:53:11 +01:00
Cameron Diver
ed73946af5
device-api: Add supervisor version endpoint
Change-type: minor
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-10-13 20:24:25 +01:00
Cameron Diver
479e0a8bb8
state: Don't consider local mode when storing state
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-10-13 20:20:05 +01:00
Cameron Diver
5906e1427c
local mode: Add local mode manager module to handle cleanup
Change-type: minor
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-10-13 20:20:05 +01:00
Cameron Diver
3e665c0f4a
logger: Enforce timestamp value to avoid logging server 400 errors
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-10-13 20:20:04 +01:00
Cameron Diver
6156825293
Add logging endpoint to supervisor
Change-type: minor
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-10-13 20:19:38 +01:00
Cameron Diver
6e2801380b
Support local logging via standard logging interface
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-10-10 13:02:40 +01:00
Cameron Diver
19cd310da3
Support setting target state in local mode from supervisor API
Change-type: minor
Closes: #689
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-10-10 13:02:40 +01:00
Cameron Diver
e0237e826e
Add several local mode enabling endpoints
Change-type: minor
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-10-10 13:02:08 +01:00
Cameron Diver
524d185c4c
Correctly type service-manager logger and event tracker fields
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-10-10 13:02:08 +01:00
Cameron Diver
5c0c880a10
Don't set empty target state in local mode
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-10-10 13:02:06 +01:00
Cameron Diver
5537ae4e2e
fix: Correctly handle array based network definitions for service
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-10-08 11:29:32 +01:00
Cameron Diver
367dd876aa
fix: Only join the default network when creating the container
We were joining every network on container creation, which is currently
bugged in Docker. We were also joining networks afterwards, so the
non-default networks are joined post-creation, and only the networkMode
container is joined on creation.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-10-08 11:29:23 +01:00
Cameron Diver
06bbf9751a
compose: Support changing the tty option for compose services
This enables the switch to be added to the compose, and the handling of
docker messages has been changed to ensure that the multiplexed logs
which result are handled properly.

Change-type: minor
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-09-24 12:04:51 +01:00
Cameron Diver
e8a5edf774
types: Remove temporary Service type in favour of actual compose class
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-09-24 12:04:51 +01:00
Cameron Diver
892d227cc2
compose/service: Convert module to typescript and add network config
Stability improvements;
* Printing of unsupported compose fields
* Added a lot of tests
* All compose configuration has a default value, enabling better
comparison

Change-type: minor
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-09-21 10:30:27 +01:00
Cameron Diver
e0231f15e9
Compose: Support more network creation options
Change-type: minor
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-09-21 10:30:27 +01:00
Cameron Diver
a7551abe93
Refactor: Split Networks class to Network and NetworkManager
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-09-19 15:41:09 +01:00
Cameron Diver
c5540404af
logger: Fix reporting the configuration change variables
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-09-13 14:44:20 +01:00
Cameron Diver
d37eb8e8a9
lib/errors: Improve typings by extending Error class for predicates
When using the predicate functions in bluebird `.catch`es from
typescript, the compiler would complain that the predicates do not
accept a function which takes an error. Because these are specific
errors, I've extended the base `Error` class, and added the extra fields
we expect.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-09-13 12:01:10 +01:00
Cameron Diver
68a6b1aef1
fix: Use throttle not debounce when ensuring we don't spam mixpanel
Debounce will mean that in certain cases, the events will never be sent,
whereas with throttle we can be sure that it will be sent a minimum
amount per time slice.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-09-12 12:30:31 +01:00
Cameron Diver
8abbfe36e0
fix: Don't send internal state tracking information to the API
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-09-05 12:09:35 -07:00
Cameron Diver
08b1c90ea1
fix: Fix event-tracker being passed no options object
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-09-04 16:02:06 -07:00
Cameron Diver
c1748436d8
fix: Import logger properly and add startup test
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-09-04 15:53:49 -07:00
Cameron Diver
8ca42062a3
events: Report supervisor version with any mixpanel events
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-09-04 13:17:20 -07:00
Cameron Diver
607c722b0b
events: Add rate limiting based on event name when reporting to mixpanel
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-09-04 13:17:17 -07:00
Cameron Diver
8bd895d24f
refactor: Use Dictionary<T> type in ports module
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-09-03 09:39:07 -07:00
Cameron Diver
328b07a092
fix: Respect logging enabling on instantiation of logger
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-09-03 09:39:07 -07:00
Cameron Diver
d3a18da573
Refactor: Convert logging module to typescript
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-09-03 09:39:06 -07:00
Cameron Diver
99c807a48c
Refactor: Make exported log-types implement interface
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-09-03 09:36:21 -07:00
Cameron Diver
8ee26adbbe
Refactor: Convert event tracker module to typescript
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-09-03 09:36:21 -07:00
Cameron Diver
16e102b3ba
fix: Dont bind service class methods in declaration
This was causing a bug where the applications were cloned when
restarting all of them. These clones did not carry over the binds, so
when starting the containers back up, two different versions of `this`
were being used.

Change-type: patch
Closes: #736
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-08-29 15:32:03 -07:00
Cameron Diver
0a93b9c18a
feature(host_config): Support gpio field as an array
Change-type: minor
Closes: #734
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-08-29 08:58:05 -07:00
Cameron Diver
ae446c01b2
ux: Warn on invalid device name when trying to start a service
Device names with newlines cause reboot loops, due to newlines not being
supported by docker. This PR will warn when a device name contains a
newline.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-08-16 21:52:49 +01:00
Cameron Diver
f7240ccce1
fix: Fix importing of service type location
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-08-16 16:38:43 +01:00
Cameron Diver
3a4f1594e0
Refactor: Add docker-utils typings
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-08-16 15:35:31 +01:00
Cameron Diver
b504306f28
Refactor: Convert log-types module to typescript
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-08-16 15:35:30 +01:00
Cameron Diver
b7d991b98e
fix: Apply device name before generating container config
Currently the service has the device name applied after the docker
config is generated. This means that is has no effect until the next
restart.

This commit ensures that the device name is applied before the docker
config is generated, meaning that the env var gets applied to the
upcoming container.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-08-13 10:02:53 +01:00
Cameron Diver
430e45d6fb
Fix: Restart a service when it's memory limit changes
Change-type: patch
Closes: #720
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-08-07 13:15:37 +01:00
Pagan Gazzard
c6b77954a7 Add a backoff mechanism for state reporting errors
Change-type: patch
Signed-off-by: Pagan Gazzard <page@resin.io>
2018-08-01 11:56:55 -07:00
Cameron Diver
7ea48683be
Fix: Fix scoping issue with variables in service-manager
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-07-25 15:45:18 +01:00
Cameron Diver
f7958e6c6b
Dont restart service on device name change
Change-type: patch
Closes: #659
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-07-25 10:45:05 +01:00
Petros Angelatos
bba1da370b
logger: Remove pubnub leftovers
Change-type: minor
Signed-off-by: Petros Angelatos <petrosagg@gmail.com>
2018-07-18 12:31:10 -07:00
Petros Angelatos
6766c23bd9
logger: Only send logs produced after attaching
The previous approach had the bad side effect of resending tons of logs
in the case of a supervisor restart.

The approach can be improved by storing the last timestamp per container
and re-attaching at the correct point.

Change-type: minor
Signed-off-by: Petros Angelatos <petrosagg@gmail.com>
2018-07-18 12:31:06 -07:00
Petros Angelatos
0d812c272c
logger: Use the new logging backend
Change-type: minor
Signed-off-by: Petros Angelatos <petrosagg@gmail.com>
2018-07-18 12:30:59 -07:00
Cameron Diver
1604fdf276
Allow the enabling and disabling of persistent logging via env var
Closes: #698
Change-type: minor
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-07-17 19:43:54 +01:00
Cameron Diver
0e29e5f6f7
Refactor config code to be consistent in location
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-07-16 17:48:03 +01:00
Cameron Diver
c61b16655e
Remove resinApiEndpoint meta-endpoint and use config.json entry instead
The resinApiEndpoint config option existed for legacy reasons, where the
apiEndpoint was passed in via env vars, but this is no longer the case,
and the current supervisor wouldn't run on these older versions of
resinOS anymore anyway, so I've removed the references to this legacy
endpoint, as it made reasoning about offline mode weird.

Change-type: minor
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-07-12 13:33:57 +01:00
Cameron Diver
9259f231f8
Correctly apply current commit value to applications
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-07-12 12:42:22 +01:00
Cameron Diver
8abfd05718
Fix typo in config method call
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-07-09 11:09:42 +01:00
Cameron Diver
080fdd3774
Bind db handle to fn variable
During the conversion to typescript, the behaviour of the database
handle changed slightly, meaning storing a reference to the models
function also requires a bind to be applied too.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-06-28 14:25:06 +01:00
Cameron Diver
39d8ac0133
Change config function providers to be mutable
Also change logsChannelSecret value to be queried with the api backend,
so that logs are not shared between instances. This has been implemented
as the first config function provider with mutability.

Change-type: minor
Closes: #675
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-06-26 14:02:25 +01:00
Cameron Diver
380d40f9f8
Fix /v1/device endpoint returning null for commit after an update
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-06-25 17:32:48 +01:00
Cameron Diver
cf85ae524b
Add applications state v2 endpoint
Change-type: minor
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-06-25 17:32:48 +01:00
Cameron Diver
049398eec0
Move compose types to ./types and add partial definitions for compose modules
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-06-25 17:32:47 +01:00
Cameron Diver
6ca2701b85
Move v2 endpoints to separate module
Change-type: patch
Connects-to: #640
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-06-25 17:32:47 +01:00
Cameron Diver
981c7323ed
Refactor v1 api into seperate modules
Change-type: patch
Connects-to: #640
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-06-25 17:32:47 +01:00
Cameron Diver
bb8be44429
Reuse PortRange object in PortMap class, and other cleanup
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-06-25 09:46:07 +01:00
Cameron Diver
5b8068794e
Add fromDockerOpts and normalization to PortMap class, and use in fromContainer
This function takes the docker output representing ports, and generates
the port map values from them. This means that services can accurately
be compared and next steps can be inferred.

The normalization function ensures that regardless of source, PortMaps
that represent the same port setup will be represented correctly
internally.

Change-type: patch
Closes: #644
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-06-25 09:46:07 +01:00
Cameron Diver
7b77e45f69
Store port ranges as ranges, to reduce memory usage
Before this change, port ranges were iterated and stored as an object
per port mapping. Now the port ranges are stored as ranges until they
need to be converted to objects. The need to convert to objects still
exists as this is the format which the docker remote API expects, but
hopefully this should alleviate bugs like #644 by making the memory more
shorter-lived.

Also added more tests.

Change-type: patch
Closes: #644
Signed-of-by: Cameron Diver <cameron@resin.io>
2018-06-25 09:46:01 +01:00
Cameron Diver
809bc87b88
Convert config module to typescript
Extract config.json handling code out to separate modules too.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-06-18 13:41:18 +01:00
Cameron Diver
d656fbb1f5
Convert fs-utils module to typescript
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-06-18 13:39:08 +01:00
Cameron Diver
c9904d8b5d
Convert os-release module to typescript
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-06-18 13:39:08 +01:00
Cameron Diver
c5acb2f66d
Convert supervisor_version module to typescript, and add typings for json
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-06-18 13:39:08 +01:00
Cameron Diver
1b0fd82f51
Convert db module to typescript
Also had to change config module to bind `.this` value, due to
differences in setup.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-06-18 13:39:08 +01:00
Cameron Diver
3c9e3bba42
Show better UX when a delta download fails because the image is processing
Change-type: patch
Closes: #679
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-06-14 09:51:42 +01:00
Cameron Diver
31c6db2ccd
Upgrade TypedError and move docker-utils error to error module
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-06-13 16:55:34 +01:00
Cameron Diver
089f31cb5d
Pin a device to a commit when preload has a pinDevice field
Change-type: minor
Closes: #668
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-06-11 20:27:16 +01:00
Cameron Diver
0eb8b25b64
Fix typo in EEXIST error predicate
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-06-11 13:47:24 +01:00
Cameron Diver
5622ddce94
Convert iptables module to typescript
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-06-11 10:20:17 +01:00
Cameron Diver
352a5684b3
Convert errors module to typescript
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-06-06 21:24:08 +01:00
Cameron Diver
fa1d3da438
Use ES6 interpolation for string operations 2018-06-06 14:50:21 +01:00
Cameron Diver
51b5cff7b0
Rename ExtLinuxFile interface to match the codebase 2018-06-06 14:50:21 +01:00
Cameron Diver
1e48d02781
Move config backend code out to classes which implement a common base
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-06-06 14:50:21 +01:00
Cameron Diver
89627a3392
Add support for extlinux configuration files
Currently the only supported variable is RESIN_HOST_EXTLINUX_isolcpus

Change-type: minor
Closes: #671
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-06-05 17:00:45 +01:00
Cameron Diver
dc59c83409
Move boot config related code to config-utils module
This commit abstracts all of the boot config code out of the
device-config module, ready to extend with different config backends.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-06-05 17:00:45 +01:00
Cameron Diver
2b9d82e731
Add types for fs-utils module
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-06-05 14:27:27 +01:00
Akis Kesoglou
8479801674 Add support for Balena deltas
Resin’s delta server supports Balena deltas as version 3 deltas. This commit adds support for triggering delta generation for Balena deltas, and applying them locally to the device via a simple pull.

The delta version to use when updating has been abstracted away as an env var that is user-defined. The default value is still instructing use of rsync deltas (v2).

Change-Type: minor
2018-05-23 20:59:56 +03:00
Akis Kesoglou
56ecd845f7 Log delta debugging information
Change-Type: minor
2018-05-23 20:59:56 +03:00
Cameron Diver
bc37ee56e4
Check against application source for target applications
The supervisor will now check that a source of an application matches
the current source, and only start it if so.

Change-type: patch
Closes: #658
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-05-22 12:11:57 +01:00
Cameron Diver
343989f487
Add migration for source field to be added to app table
This field will represent the apiEndpoint that the application came
from, (or an empty string for local apps). This means that when
configuring an application to work on a different environment, as long
as the endpoint is different, the supervisor can know not to start the
old application.

Change-type: minor
Connects-to: #658
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-05-22 12:11:54 +01:00
Cameron Diver
bea0b00804
Ignore leading and trailing whitespace when parsing env vars
Change-type: patch
Closes: #644
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-05-22 11:32:31 +01:00
Cameron Diver
ea8e8d2f5f
Don't generate config fields in offline mode
Change-type: patch
Closes: #648
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-05-21 14:45:32 +01:00
Pablo Carranza Velez
3a815aa972 Add support for changing config.txt on balena fin devices
Change-Type: minor
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
2018-05-18 10:19:28 -07:00
Cameron Diver
73b19a320f
Change intialConfigReported value to be an api endpoint
This api endpoint is the endpoint which the intial config was reported.
Also changed the code to detect if the api endpoint has changed, and
therefore whether we need to re-report the initial config, to avoid
losing hardware specific information.

Change-type: minor
Closes: #649
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-05-16 14:18:03 +01:00
Cameron Diver
96d7bbefd0
Fix handling of systemd errors when polling for log to display
Change-type: patch
Closes: #610
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-05-08 15:12:13 +01:00
Cameron Diver
9150c3fdbc
Convert conversions module to typescript
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-05-02 09:36:21 +01:00
Cameron Diver
367e90d6e4
Move shared types to separate module
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-05-02 09:36:21 +01:00
Cameron Diver
273224547c
Convert blink module to typescript
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-05-02 09:36:21 +01:00
Cameron Diver
c1fbc2dc21
Convert lib/constants module to typescript
Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-05-02 09:36:21 +01:00
Cameron Diver
67b9ec6ca7
Type parameters for validation functions better
They can now be string | null | undefined.

Change-type: patch
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-05-02 09:36:21 +01:00
Pablo Carranza Velez
652b596c80 Add some more unit tests to the multicontainer supervisor
We add a bunch of additional unit tests, and also a coverage report using istanbul.

The tests are not meant to cover everything, but they're a first attempt at having *some* unit testing
on the supervisor. There's much to improve but hopefully it helps catch obvious errors.

Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
2018-04-30 19:39:01 -07:00
Cameron Diver
ddbf3418de
Remove trailing slashes from working directories of services
This is to combat when a working directory is in the compose file for a
service with a trailing slash. Docker will strip this slash and that
means service comparisons will fail going forward - even if they are the
same.

Change-type: patch
Closes: #635
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-04-27 18:03:36 +01:00
Cameron Diver
cfddbf65e4
Start initial typescript conversion, and add validation debugging
Add webpack config and dependencies to have typescript built, and also
convert src/lib/validation.coffee to typescript.

In this conversion I also added a lot of debugging which should help the
upcoming local mode development.

Change-type: minor
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-04-27 16:09:50 +01:00
Pablo Carranza Velez
6c4eaff02f On startup, only attach to logs for running containers, and remove any containers marked as dead
Closes #611
Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
2018-04-19 15:04:04 -07:00
Akis Kesoglou
1cde9dd16d Properly propagate delta failure
Change-Type: patch
Closes: #627
2018-04-18 18:34:42 +03:00
Pablo Carranza Velez
015d33f8f5 Allow services (and their images) to expose udp ports besides tcp
Closes #621
Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
2018-04-11 15:13:48 -07:00
Cameron Diver
1c27ebc354
Fix parsing of non-unit memory numbers and add tests
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-04-04 14:03:24 +01:00
Cameron Diver
7d233ea111
Fix regex for parsing memory numbers
It now allows a trailing `b`, as the docker-compose docs specify.

In addition the regex now specifies a case-insensitive flag, to catch
both upper and lower case memory numbers (the rest of the function
supported these already).

Change-type: patch
Closes: #603
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-04-04 11:25:37 +01:00
Cameron Diver
393671505c
Respond to reboot and shutdown endpoints with a success object
Change-type: patch
Closes: #607
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-03-29 13:11:02 +01:00
Pablo Carranza Velez
72a5f03b0c
Back off fetching the target state exponentially, for faster retries when there's no connectivity
Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
2018-03-29 10:20:52 +01:00
Cameron Diver
e458c4103c
Handle incorrectly parsed env vars from docker inspect
Change-type: patch
Closes: #604
Signed-off-by: Cameron Diver <cameron@resin.io>
2018-03-26 16:56:06 +01:00
Pablo Carranza Velez
348ff66cee
Replace the gosuper component with a node module that handles communication with systemd, and stop using an init system in the supervisor container
Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
2018-03-22 15:55:15 +00:00
Pablo Carranza Velez
c8d79c3b7d Remove any leftover knex migrations locks before running migrations
Closes #598
Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
2018-03-21 18:20:15 -03:00
Pablo Carranza Velez
d46d6513b5 Trim whitespace from the hostname file on the host before passing it as the hostname for containers with host network
Closes #594
Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
2018-03-19 17:19:59 -03:00
Pablo Carranza Velez
e1e33b376e Force reboots and shutdowns if lock override is enabled
Closes #440
Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
2018-03-19 16:30:59 -03:00
Pablo Carranza Velez
91ac11d0e8 In /v1/apps/:appId/stop, wait for the service to exit before responding
Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
2018-03-16 15:13:15 -03:00
Pablo Carranza Velez
5d7e8afcee Avoid trying to clean up the supervisor image if it has several tags (even though it would never succeed)
Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
2018-03-14 22:38:52 -03:00
Pablo Carranza Velez
75ce55e932 Fix typo passing apiKey to the resin API client when exchanging apikeys
Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
2018-03-14 21:50:31 -03:00
Pablo Carranza Velez
7e342e9d80 Fix the check for whether the device is provisioned by correctly getting registered_at and deviceId from config.json into the ApiBinder
Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
2018-03-14 21:50:31 -03:00
Pablo Carranza Velez
08a98f47e4 compose: Use the hostname on the host as default hostname for services with host network mode
Otherwise if the hostname on the supervisor container differs from the hostname on the host, the current and target
services will never match.

Change-Type: patch
Signed-off-by: Pablo Carranza Velez <pablo@resin.io>
2018-03-14 21:18:11 -03:00