v12.52.0

Merge pull request #2334 from balena-io/2005-os-esr-versions-hostapp
os versions, os download: Add support for balenaOS ESR versions
2025-06-24 18:45:07 +00:00 · 2021-11-20 03:19:00 +02:00 · 2021-11-20 01:17:17 +00:00 · 2021-11-20 00:43:15 +00:00 · 2021-11-20 00:43:15 +00:00 · 2021-11-16 19:10:36 +02:00
16 changed files with 459 additions and 168 deletions
--- a/.mocharc.js
+++ b/.mocharc.js
@ -3,5 +3,8 @@ module.exports = {
 	require: 'ts-node/register/transpile-only',
 	file: './tests/config-tests',
 	timeout: 12000,
+	// To test only, say, 'push.spec.ts', do it as follows so that
+	// requests are authenticated:
+	// spec: ['tests/auth/*.spec.ts', 'tests/**/deploy.spec.ts'],
 	spec: 'tests/**/*.spec.ts',
 };
--- a/.resinci.yml
+++ b/.resinci.yml
@ -5,13 +5,11 @@ npm:
      os: ubuntu
      architecture: x86_64
      node_versions:
-        - "10"
        - "12"
        - "14"
    - name: linux
      os: alpine
      architecture: x86_64
      node_versions:
-        - "10"
        - "12"
        - "14"
--- a/.versionbot/CHANGELOG.yml
+++ b/.versionbot/CHANGELOG.yml
@ -1,3 +1,186 @@
+- commits:
+    - subject: >-
+        os download: Display OS version actually downloaded (range or
+        'recommended')
+      hash: f46452f6de7bbfa00e073986d47a85608e5a9fea
+      body: ''
+      footer:
+        Change-type: patch
+        change-type: patch
+      author: Paulo Castro
+      nested: []
+    - subject: 'os versions, os download: Add support for balenaOS ESR versions'
+      hash: c166ec75979fcaa8bd123a1bf84f379279050fd4
+      body: ''
+      footer:
+        Change-type: minor
+        change-type: minor
+      author: Paulo Castro
+      nested: []
+  version: 12.52.0
+  date: 2021-11-20T00:51:09.369Z
+- commits:
+    - subject: 'deploy: Ensure the release fails if an image''s digest (hash) is missing'
+      hash: 23b07f8a41c82c0b23a38bac55cbd3874300dd58
+      body: ''
+      footer:
+        Change-type: patch
+        change-type: patch
+      author: Paulo Castro
+      nested: []
+  version: 12.51.3
+  date: 2021-11-16T11:58:31.895Z
+- commits:
+    - subject: Update balena CI configuration (remove Node v10 from npm pipeline list)
+      hash: ae5ea0f4e85008624ff46fe10998fadbd8a48b15
+      body: ''
+      footer:
+        Change-type: patch
+        change-type: patch
+      author: Paulo Castro
+      nested: []
+  version: 12.51.2
+  date: 2021-11-16T00:07:56.053Z
+- commits:
+    - subject: Fix forums support link in README.md
+      hash: 112a7b8194d098e9c66754eb256590e13f54fe29
+      body: ''
+      footer:
+        Change-type: patch
+        change-type: patch
+        Signed-off-by: Scott Lowe <scott@balena.io>
+        signed-off-by: Scott Lowe <scott@balena.io>
+      author: Scott Lowe
+      nested: []
+  version: 12.51.1
+  date: 2021-10-25T17:29:36.975Z
+- commits:
+    - subject: Add support for YAML anchors and aliases in 'docker-compose.yml'
+      hash: 8e712ac91055c4efde885854488000a27c6b483d
+      body: >
+        This allows project files to define services from generic fragments by
+        leveraging YAML's anchors and aliases. See here for an example:
+        https://github.com/compose-spec/compose-spec/blob/43f6537b2c8f01b6d3f0e184d13a0f3cb93d38d7/spec.md#fragments
+
+
+        Removing the FAILSAFE_SCHEMA flag is not expected to break existing
+        project files, since the default behaviour is more liberal, or cause
+        problems down the road given we perform validation immediately after.
+        Docs for the flag:
+        https://github.com/nodeca/js-yaml#load-string---options-
+      footer:
+        Change-type: minor
+        change-type: minor
+      author: dfunckt
+      nested: []
+  version: 12.51.0
+  date: 2021-10-22T16:50:50.004Z
+- commits:
+    - subject: 'preload: Avoid possible ValueError when parsing storage driver'
+      hash: 48053ecefc00e451f3ee5c9b82b2b398978ec229
+      body: |
+        Update balena-preload from 10.5.0 to 11.0.0
+      footer:
+        Change-type: patch
+        change-type: patch
+        Signed-off-by: Kyle Harding <kyle@balena.io>
+        signed-off-by: Kyle Harding <kyle@balena.io>
+      author: Kyle Harding
+      nested:
+        - commits:
+            - subject: Avoid creating multiple preload containers
+              hash: 6b5b6428833ce2cd5c53c2051d6f515f1b3e4c37
+              body: |
+                This was only caught when we started correctly naming
+                the preload container by switching from `Name` to `name` in
+                our createContainer options.
+
+                Previously we were creating two containers but they had unique
+                random names so we never saw the conflict.
+              footer:
+                Change-type: patch
+                change-type: patch
+                Signed-off-by: Kyle Harding <kyle@balena.io>
+                signed-off-by: Kyle Harding <kyle@balena.io>
+              author: Kyle Harding
+              nested: []
+            - subject: 'major: Remove balena-preload script in favor of use with CLI'
+              hash: 7de06434155913b199024683e62c85e94f1d1cb1
+              body: ''
+              footer:
+                Signed-off-by: >-
+                  Lorenzo Alberto Maria Ambrosi
+                  <lorenzothunder.ambrosi@gmail.com>
+                signed-off-by: >-
+                  Lorenzo Alberto Maria Ambrosi
+                  <lorenzothunder.ambrosi@gmail.com>
+              author: Lorenzo Alberto Maria Ambrosi
+              nested: []
+            - subject: Fix missing 'await' for getEdisonPartitions()
+              hash: 9a2ebfdb2a5375304ee5cf2ba6321b93f93886ed
+              body: ''
+              footer:
+                Change-type: patch
+                change-type: patch
+              author: Paulo Castro
+              nested: []
+            - subject: Add extra type information (refactor bind mount array)
+              hash: 5b8d21e68d9da902f7304a210cb1abe22420d1d0
+              body: ''
+              footer:
+                Change-type: patch
+                change-type: patch
+              author: Paulo Castro
+              nested: []
+            - subject: Run linter
+              hash: 456c727e6518317588b31e28ffb734377fb85e76
+              body: ''
+              footer:
+                Change-type: patch
+                change-type: patch
+              author: Paulo Castro
+              nested: []
+            - subject: 'major: Convert to typescript'
+              hash: ce241be0a780bdff87f4513e4d5a0ec63d72ac7e
+              body: ''
+              footer:
+                Signed-off-by: >-
+                  Lorenzo Alberto Maria Ambrosi
+                  <lorenzothunder.ambrosi@gmail.com>
+                signed-off-by: >-
+                  Lorenzo Alberto Maria Ambrosi
+                  <lorenzothunder.ambrosi@gmail.com>
+              author: Lorenzo Alberto Maria Ambrosi
+              nested: []
+            - subject: 'patch: Fix incorrect python List index check'
+              hash: 85d404000ac6e9b4ac83ce1feed33789deca182b
+              body: ''
+              footer:
+                Signed-off-by: >-
+                  Lorenzo Alberto Maria Ambrosi
+                  <lorenzothunder.ambrosi@gmail.com>
+                signed-off-by: >-
+                  Lorenzo Alberto Maria Ambrosi
+                  <lorenzothunder.ambrosi@gmail.com>
+              author: Lorenzo Alberto Maria Ambrosi
+              nested: []
+          version: balena-preload-11.0.0
+          date: 2021-10-13T18:20:43.867Z
+  version: 12.50.3
+  date: 2021-10-20T15:30:14.694Z
+- commits:
+    - subject: Error message when renaming a fleet now mentions the target name.
+      hash: c493c33e3896784ee60c9d4ac79721ca6b96a778
+      body: ''
+      footer:
+        Change-type: patch
+        change-type: patch
+        Signed-off-by: Carlo Miguel F. Cruz <carloc@balena.io>
+        signed-off-by: Carlo Miguel F. Cruz <carloc@balena.io>
+      author: Carlo Miguel F. Cruz
+      nested: []
+  version: 12.50.2
+  date: 2021-10-05T09:04:40.995Z
 - commits:
    - subject: Update dependencies (@sentry/node error reporting)
      hash: 08dfc945f3ed1c518a7d1830a5a37d72fd5739fd
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@ -4,6 +4,49 @@ All notable changes to this project will be documented in this file
 automatically by Versionist. DO NOT EDIT THIS FILE MANUALLY!
 This project adheres to [Semantic Versioning](http://semver.org/).

+## 12.52.0 - 2021-11-20
+
+* os download: Display OS version actually downloaded (range or 'recommended') [Paulo Castro]
+* os versions, os download: Add support for balenaOS ESR versions [Paulo Castro]
+
+## 12.51.3 - 2021-11-16
+
+* deploy: Ensure the release fails if an image's digest (hash) is missing [Paulo Castro]
+
+## 12.51.2 - 2021-11-16
+
+* Update balena CI configuration (remove Node v10 from npm pipeline list) [Paulo Castro]
+
+## 12.51.1 - 2021-10-25
+
+* Fix forums support link in README.md [Scott Lowe]
+
+## 12.51.0 - 2021-10-22
+
+* Add support for YAML anchors and aliases in 'docker-compose.yml' [dfunckt]
+
+## 12.50.3 - 2021-10-20
+
+
+<details>
+<summary> preload: Avoid possible ValueError when parsing storage driver [Kyle Harding] </summary>
+
+> ### balena-preload-11.0.0 - 2021-10-13
+> 
+> * Avoid creating multiple preload containers [Kyle Harding]
+> * major: Remove balena-preload script in favor of use with CLI [Lorenzo Alberto Maria Ambrosi]
+> * Fix missing 'await' for getEdisonPartitions() [Paulo Castro]
+> * Add extra type information (refactor bind mount array) [Paulo Castro]
+> * Run linter [Paulo Castro]
+> * major: Convert to typescript [Lorenzo Alberto Maria Ambrosi]
+> * patch: Fix incorrect python List index check [Lorenzo Alberto Maria Ambrosi]
+> 
+</details>
+
+## 12.50.2 - 2021-10-05
+
+* Error message when renaming a fleet now mentions the target name. [Carlo Miguel F. Cruz]
+
 ## 12.50.1 - 2021-09-30

 * Update dependencies (@sentry/node error reporting) [Paulo Castro]
--- a/README.md
+++ b/README.md
@ -144,7 +144,7 @@ To learn more, troubleshoot issues, or to contact us for support:

 * Check the [masterclass tutorials](https://www.balena.io/docs/learn/more/masterclasses/overview/)
 * Check our [FAQ / troubleshooting document](https://github.com/balena-io/balena-cli/blob/master/TROUBLESHOOTING.md)
-* Ask us a question through the [balenaCloud forum](https://forums.balena.io/c/balena-cloud)
+* Ask us a question in the [balena forums](https://forums.balena.io/c/product-support)

 For CLI bug reports or feature requests, check the
 [CLI GitHub issues](https://github.com/balena-io/balena-cli/issues/).
--- a/doc/cli.markdown
+++ b/doc/cli.markdown
@ -132,7 +132,7 @@ To learn more, troubleshoot issues, or to contact us for support:

 * Check the [masterclass tutorials](https://www.balena.io/docs/learn/more/masterclasses/overview/)
 * Check our [FAQ / troubleshooting document](https://github.com/balena-io/balena-cli/blob/master/TROUBLESHOOTING.md)
-* Ask us a question through the [balenaCloud forum](https://forums.balena.io/c/balena-cloud)
+* Ask us a question in the [balena forums](https://forums.balena.io/c/product-support)

 For CLI bug reports or feature requests, check the
 [CLI GitHub issues](https://github.com/balena-io/balena-cli/issues/).
@ -2240,6 +2240,9 @@ device UUID
 Show the available balenaOS versions for the given device type.
 Check available types with `balena devices supported`.

+balenaOS ESR versions can be listed with the '--esr' option. See also:
+https://www.balena.io/docs/reference/OS/extended-support-release/
+
 Examples:

 	$ balena os versions raspberrypi3
@ -2252,23 +2255,28 @@ device type

 ### Options

+#### --esr
+
+select balenaOS ESR versions
+
 ## os download &#60;type&#62;

-Download an unconfigured OS image for a certain device type.
-Check available types with `balena devices supported`
+Download an unconfigured OS image for the specified device type.
+Check available device types with 'balena devices supported'.

 Note: Currently this command only works with balenaCloud, not openBalena.
 If using openBalena, please download the OS from: https://www.balena.io/os/

-If version is not specified the newest stable (non-pre-release) version of OS
-is downloaded (if available), otherwise the newest version (if all existing
-versions for the given device type are pre-release).
+The '--version' option is used to select the balenaOS version. If omitted,
+the latest released version is downloaded (and if only pre-release versions
+exist, the latest pre-release version is downloaded).

-You can pass `--version menu` to pick the OS version from the interactive menu
-of all available versions.
+Use '--version menu' or '--version menu-esr' to interactively select the
+OS version. The latter lists ESR versions which are only available for
+download on Production and Enterprise plans. See also:
+https://www.balena.io/docs/reference/OS/extended-support-release/

-To download a development image append `.dev` to the version or select from
-the interactive menu.
+Development images can be selected by appending `.dev` to the version.

 Examples:

@ -2294,11 +2302,13 @@ output path

 #### --version VERSION

-exact version number, or a valid semver range,
+version number (ESR or non-ESR versions),
+or semver range (non-ESR versions only),
 or 'latest' (includes pre-releases),
-or 'default' (excludes pre-releases if at least one stable version is available),
+or 'default' (excludes pre-releases if at least one released version is available),
 or 'recommended' (excludes pre-releases, will fail if only pre-release versions are available),
-or 'menu' (will show the interactive menu)
+or 'menu' (interactive menu, non-ESR versions),
+or 'menu-esr' (interactive menu, ESR versions)

 ## os build-config &#60;image&#62; &#60;device-type&#62;

--- a/lib/commands/app/rename.ts
+++ b/lib/commands/app/rename.ts
@ -120,7 +120,7 @@ export class FleetRenameCmd extends Command {
 		} catch (e) {
 			// BalenaRequestError: Request error: "organization" and "app_name" must be unique.
 			if ((e.message || '').toLowerCase().includes('unique')) {
-				throw new ExpectedError(`Error: fleet ${params.fleet} already exists.`);
+				throw new ExpectedError(`Error: fleet ${newName} already exists.`);
 			}
 			throw e;
 		}
--- a/lib/commands/os/download.ts
+++ b/lib/commands/os/download.ts
@ -34,21 +34,22 @@ export default class OsDownloadCmd extends Command {
 	public static description = stripIndent`
 		Download an unconfigured OS image.

-		Download an unconfigured OS image for a certain device type.
-		Check available types with \`balena devices supported\`
+		Download an unconfigured OS image for the specified device type.
+		Check available device types with 'balena devices supported'.

 		Note: Currently this command only works with balenaCloud, not openBalena.
 		If using openBalena, please download the OS from: https://www.balena.io/os/

-		If version is not specified the newest stable (non-pre-release) version of OS
-		is downloaded (if available), otherwise the newest version (if all existing
-		versions for the given device type are pre-release).
+		The '--version' option is used to select the balenaOS version. If omitted,
+		the latest released version is downloaded (and if only pre-release versions
+		exist, the latest pre-release version is downloaded).

-		You can pass \`--version menu\` to pick the OS version from the interactive menu
-		of all available versions.
+		Use '--version menu' or '--version menu-esr' to interactively select the
+		OS version. The latter lists ESR versions which are only available for
+		download on Production and Enterprise plans. See also:
+		https://www.balena.io/docs/reference/OS/extended-support-release/

-		To download a development image append \`.dev\` to the version or select from
-		the interactive menu.
+		Development images can be selected by appending \`.dev\` to the version.
 `;
 	public static examples = [
 		'$ balena os download raspberrypi3 -o ../foo/bar/raspberry-pi.img',
@ -78,11 +79,13 @@ export default class OsDownloadCmd extends Command {
 		}),
 		version: flags.string({
 			description: stripIndent`
-				exact version number, or a valid semver range,
+				version number (ESR or non-ESR versions),
+				or semver range (non-ESR versions only),
 				or 'latest' (includes pre-releases),
-				or 'default' (excludes pre-releases if at least one stable version is available),
+				or 'default' (excludes pre-releases if at least one released version is available),
 				or 'recommended' (excludes pre-releases, will fail if only pre-release versions are available),
-				or 'menu' (will show the interactive menu)
+				or 'menu' (interactive menu, non-ESR versions),
+				or 'menu-esr' (interactive menu, ESR versions)
 				`,
 		}),
 		help: cf.help,
--- a/lib/commands/os/versions.ts
+++ b/lib/commands/os/versions.ts
@ -18,9 +18,10 @@
 import { flags } from '@oclif/command';
 import Command from '../../command';
 import * as cf from '../../utils/common-flags';
-import { getBalenaSdk, stripIndent } from '../../utils/lazy';
+import { stripIndent } from '../../utils/lazy';

 interface FlagsDef {
+	esr?: boolean;
 	help: void;
 }

@ -34,6 +35,9 @@ export default class OsVersionsCmd extends Command {

 		Show the available balenaOS versions for the given device type.
 		Check available types with \`balena devices supported\`.
+
+		balenaOS ESR versions can be listed with the '--esr' option. See also:
+		https://www.balena.io/docs/reference/OS/extended-support-release/
 	`;

 	public static examples = ['$ balena os versions raspberrypi3'];
@ -50,16 +54,20 @@ export default class OsVersionsCmd extends Command {

 	public static flags: flags.Input<FlagsDef> = {
 		help: cf.help,
+		esr: flags.boolean({
+			description: 'select balenaOS ESR versions',
+			default: false,
+		}),
 	};

 	public async run() {
-		const { args: params } = this.parse<FlagsDef, ArgsDef>(OsVersionsCmd);
+		const { args: params, flags: options } = this.parse<FlagsDef, ArgsDef>(
+			OsVersionsCmd,
+		);

-		const { versions: vs, recommended } =
-			await getBalenaSdk().models.os.getSupportedVersions(params.type);
+		const { getFormattedOsVersions } = await import('../../utils/cloud');
+		const vs = await getFormattedOsVersions(params.type, !!options.esr);

-		vs.forEach((v) => {
-			console.log(`v${v}` + (v === recommended ? ' (recommended)' : ''));
-		});
+		console.log(vs.map((v) => v.formattedVersion).join('\n'));
 	}
 }
--- a/lib/utils/cloud.ts
+++ b/lib/utils/cloud.ts
@ -130,15 +130,13 @@ export async function downloadOSImage(
 	console.info(`Getting device operating system for ${deviceType}`);

 	if (!OSVersion) {
-		console.warn('OS version not specified: using latest stable version');
+		console.warn('OS version not specified: using latest released version');
 	}

 	OSVersion = OSVersion
 		? await resolveOSVersion(deviceType, OSVersion)
 		: 'default';

-	const displayVersion = OSVersion === 'default' ? '' : ` ${OSVersion}`;
-
 	// Override the default zlib flush value as we've seen cases of
 	// incomplete files being identified as successful downloads when using Z_SYNC_FLUSH.
 	// Using Z_NO_FLUSH results in a Z_BUF_ERROR instead of a corrupt image file.
@ -150,10 +148,17 @@ export async function downloadOSImage(
 	const manager = await import('balena-image-manager');
 	const stream = await manager.get(deviceType, OSVersion);

+	const displayVersion = await new Promise((resolve, reject) => {
+		stream.on('error', reject);
+		stream.on('balena-image-manager:resolved-version', resolve);
+	});
+
 	const visuals = getVisuals();
-	const bar = new visuals.Progress(`Downloading Device OS${displayVersion}`);
+	const bar = new visuals.Progress(
+		`Downloading balenaOS version ${displayVersion}`,
+	);
 	const spinner = new visuals.Spinner(
-		`Downloading Device OS${displayVersion} (size unknown)`,
+		`Downloading balenaOS version ${displayVersion} (size unknown)`,
 	);

 	stream.on('progress', (state: any) => {
@ -183,28 +188,29 @@ export async function downloadOSImage(
 	const streamToPromise = await import('stream-to-promise');
 	await streamToPromise(stream.pipe(output));

-	console.info('The image was downloaded successfully');
+	console.info(
+		`balenaOS image version ${displayVersion} downloaded successfully`,
+	);

 	return outputPath;
 }

-async function resolveOSVersion(deviceType: string, version: string) {
-	if (version !== 'menu') {
+async function resolveOSVersion(
+	deviceType: string,
+	version: string,
+): Promise<string> {
+	if (!['menu', 'menu-esr'].includes(version)) {
 		if (version[0] === 'v') {
 			version = version.slice(1);
 		}
 		return version;
 	}

-	const vs = (
-		(await getBalenaSdk().models.hostapp.getAllOsVersions([deviceType]))[
-			deviceType
-		] ?? []
-	).filter((v) => v.osType === 'default');
+	const vs = await getFormattedOsVersions(deviceType, version === 'menu-esr');

 	const choices = vs.map((v) => ({
 		value: v.rawVersion,
-		name: `v${v.rawVersion}` + (v.isRecommended ? ' (recommended)' : ''),
+		name: v.formattedVersion,
 	}));

 	return getCliForm().ask({
@ -214,3 +220,35 @@ async function resolveOSVersion(deviceType: string, version: string) {
 		default: (vs.find((v) => v.isRecommended) ?? vs[0])?.rawVersion,
 	});
 }
+
+/**
+ * Return the output of sdk.models.hostapp.getAvailableOsVersions(), filtered
+ * regarding ESR or non-ESR versions, and having the `formattedVersion` field
+ * reformatted for compatibility with the pre-existing output format of the
+ * `os versions` and `os download` commands.
+ */
+export async function getFormattedOsVersions(
+	deviceType: string,
+	esr: boolean,
+): Promise<SDK.OsVersion[]> {
+	const versions: SDK.OsVersion[] = (
+		(await getBalenaSdk().models.hostapp.getAvailableOsVersions([deviceType]))[
+			deviceType
+		] ?? []
+	)
+		.filter((v: SDK.OsVersion) => v.osType === (esr ? 'esr' : 'default'))
+		.map((v: SDK.OsVersion) => {
+			const i = v.formattedVersion.indexOf(' ');
+			v.formattedVersion =
+				i < 0
+					? `v${v.rawVersion}`
+					: `v${v.rawVersion}${v.formattedVersion.substring(i)}`;
+			return v;
+		});
+	if (!versions.length) {
+		throw new ExpectedError(stripIndent`
+			Error: No balenaOS versions found for device type '${deviceType}'.
+			Double-check the device type slug with 'balena devices supported'.`);
+	}
+	return versions;
+}
--- a/lib/utils/compose.js
+++ b/lib/utils/compose.js
@ -63,9 +63,7 @@ export function createProject(
 	const compose = require('resin-compose-parse');

 	// both methods below may throw.
-	const rawComposition = yml.load(composeStr, {
-		schema: yml.FAILSAFE_SCHEMA,
-	});
+	const rawComposition = yml.load(composeStr);
 	const composition = compose.normalize(rawComposition);

 	projectName ||= path.basename(composePath);
@ -356,76 +354,6 @@ export const authorizePush = function (
 		.catch(() => '');
 };

-/**
- * @param {import('dockerode')} docker
- * @param {string} token
- * @param {Array<import('./compose-types').TaggedImage>} images
- * @param {(serviceImage: import('balena-release/build/models').ImageModel, props: object) => void} afterEach
- */
-export const pushAndUpdateServiceImages = function (
-	docker,
-	token,
-	images,
-	afterEach,
-) {
-	const { DockerProgress } = require('docker-progress');
-	const { retry } = require('./helpers');
-	const tty = require('./tty')(process.stdout);
-	const Bluebird = require('bluebird');
-
-	const opts = { authconfig: { registrytoken: token } };
-
-	const progress = new DockerProgress({ docker });
-	const renderer = pushProgressRenderer(
-		tty,
-		getChalk().blue('[Push]') + '    ',
-	);
-	const reporters = progress.aggregateProgress(images.length, renderer);
-
-	return Bluebird.using(tty.cursorHidden(), () =>
-		Promise.all(
-			images.map(({ serviceImage, localImage, props, logs }, index) =>
-				Promise.all([
-					localImage.inspect().then((img) => img.Size),
-					retry({
-						// @ts-ignore
-						func: () => progress.push(localImage.name, reporters[index], opts),
-						maxAttempts: 3, // try calling func 3 times (max)
-						// @ts-ignore
-						label: localImage.name, // label for retry log messages
-						initialDelayMs: 2000, // wait 2 seconds before the 1st retry
-						backoffScaler: 1.4, // wait multiplier for each retry
-					}).finally(renderer.end),
-				])
-					.then(
-						/** @type {([number, string]) => void} */
-						function ([size, digest]) {
-							serviceImage.image_size = size;
-							serviceImage.content_hash = digest;
-							serviceImage.build_log = logs;
-							serviceImage.dockerfile = props.dockerfile;
-							serviceImage.project_type = props.projectType;
-							if (props.startTime) {
-								serviceImage.start_timestamp = props.startTime;
-							}
-							if (props.endTime) {
-								serviceImage.end_timestamp = props.endTime;
-							}
-							serviceImage.push_timestamp = new Date();
-							serviceImage.status = 'success';
-						},
-					)
-					.catch(function (e) {
-						serviceImage.error_message = '' + e;
-						serviceImage.status = 'failed';
-						throw e;
-					})
-					.finally(() => afterEach?.(serviceImage, props)),
-			),
-		),
-	);
-};
-
 // utilities

 const renderProgressBar = function (percentage, stepCount) {
@ -437,7 +365,7 @@ const renderProgressBar = function (percentage, stepCount) {
 	return `${bar} ${_.padStart(percentage, 3)}%`;
 };

-var pushProgressRenderer = function (tty, prefix) {
+export const pushProgressRenderer = function (tty, prefix) {
 	const fn = function (e) {
 		const { error, percentage } = e;
 		if (error != null) {
--- a/lib/utils/compose_ts.ts
+++ b/lib/utils/compose_ts.ts
@ -1306,15 +1306,101 @@ async function getTokenForPreviousRepos(
 	return token;
 }

+async function pushAndUpdateServiceImages(
+	docker: Dockerode,
+	token: string,
+	images: TaggedImage[],
+	afterEach: (
+		serviceImage: import('balena-release/build/models').ImageModel,
+		props: object,
+	) => void,
+) {
+	const { DockerProgress } = await import('docker-progress');
+	const { retry } = await import('./helpers');
+	const { pushProgressRenderer } = await import('./compose');
+	const tty = (await import('./tty'))(process.stdout);
+	const opts = { authconfig: { registrytoken: token } };
+	const progress = new DockerProgress({ docker });
+	const renderer = pushProgressRenderer(
+		tty,
+		getChalk().blue('[Push]') + '    ',
+	);
+	const reporters = progress.aggregateProgress(images.length, renderer);
+
+	const pushImage = async (
+		localImage: Dockerode.Image,
+		index: number,
+	): Promise<string> => {
+		try {
+			// TODO 'localImage as any': find out exactly why tsc warns about
+			// 'name' that exists as a matter of fact, with a value similar to:
+			// "name": "registry2.balena-cloud.com/v2/aa27790dff571ec7d2b4fbcf3d4648d5:latest"
+			const imgName: string = (localImage as any).name || '';
+			const imageDigest: string = await retry({
+				func: () => progress.push(imgName, reporters[index], opts),
+				maxAttempts: 3, // try calling func 3 times (max)
+				label: imgName, // label for retry log messages
+				initialDelayMs: 2000, // wait 2 seconds before the 1st retry
+				backoffScaler: 1.4, // wait multiplier for each retry
+			});
+			if (!imageDigest) {
+				throw new ExpectedError(stripIndent`\
+					Unable to extract image digest (content hash) from image upload progress stream for image:
+					${imgName}`);
+			}
+			return imageDigest;
+		} finally {
+			renderer.end();
+		}
+	};
+
+	const inspectAndPushImage = async (
+		{ serviceImage, localImage, props, logs }: TaggedImage,
+		index: number,
+	) => {
+		try {
+			const [imgInfo, imgDigest] = await Promise.all([
+				localImage.inspect(),
+				pushImage(localImage, index),
+			]);
+			serviceImage.image_size = imgInfo.Size;
+			serviceImage.content_hash = imgDigest;
+			serviceImage.build_log = logs;
+			serviceImage.dockerfile = props.dockerfile;
+			serviceImage.project_type = props.projectType;
+			if (props.startTime) {
+				serviceImage.start_timestamp = props.startTime;
+			}
+			if (props.endTime) {
+				serviceImage.end_timestamp = props.endTime;
+			}
+			serviceImage.push_timestamp = new Date();
+			serviceImage.status = 'success';
+		} catch (error) {
+			serviceImage.error_message = '' + error;
+			serviceImage.status = 'failed';
+			throw error;
+		} finally {
+			await afterEach(serviceImage, props);
+		}
+	};
+
+	tty.hideCursor();
+	try {
+		await Promise.all(images.map(inspectAndPushImage));
+	} finally {
+		tty.showCursor();
+	}
+}
+
 async function pushServiceImages(
-	docker: import('dockerode'),
+	docker: Dockerode,
 	logger: Logger,
 	pineClient: ReturnType<typeof import('balena-release').createClient>,
 	taggedImages: TaggedImage[],
 	token: string,
 	skipLogUpload: boolean,
 ): Promise<void> {
-	const { pushAndUpdateServiceImages } = await import('./compose');
 	const releaseMod = await import('balena-release');
 	logger.logInfo('Pushing images to registry...');
 	await pushAndUpdateServiceImages(
@ -1337,7 +1423,7 @@ async function pushServiceImages(
 const PLAIN_SEMVER_REGEX = /^([0-9]+)\.([0-9]+)\.([0-9]+)$/;

 export async function deployProject(
-	docker: import('dockerode'),
+	docker: Dockerode,
 	logger: Logger,
 	composition: Composition,
 	images: BuiltImage[],
--- a/lib/utils/tty.ts
+++ b/lib/utils/tty.ts
@ -1,3 +1,20 @@
+/**
+ * @license
+ * Copyright 2018-2021 Balena Ltd.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
 const windowSize: { width?: number; height?: number } = {};

 const updateWindowSize = () => {
@ -29,13 +46,6 @@ export = (stream: NodeJS.WriteStream = process.stdout) => {

 	const cursorDown = (rows: number = 0) => stream.write(`\u001B[${rows}B`);

-	const cursorHidden = () => {
-		const Bluebird = require('bluebird') as typeof import('bluebird');
-		return Bluebird.try(hideCursor).disposer(() => {
-			showCursor();
-		});
-	};
-
 	const write = (str: string) => stream.write(str);

 	const writeLine = (str: string) => stream.write(`${str}\n`);
@ -54,7 +64,6 @@ export = (stream: NodeJS.WriteStream = process.stdout) => {
 		currentWindowSize,
 		hideCursor,
 		showCursor,
-		cursorHidden,
 		cursorUp,
 		cursorDown,
 		write,
--- a/npm-shrinkwrap.json
+++ b/npm-shrinkwrap.json
@ -1,6 +1,6 @@
 {
  "name": "balena-cli",
-  "version": "12.50.1",
+  "version": "12.52.0",
  "lockfileVersion": 1,
  "requires": true,
  "dependencies": {
@ -3785,9 +3785,9 @@
      }
    },
    "balena-image-manager": {
-      "version": "7.0.3",
-      "resolved": "https://registry.npmjs.org/balena-image-manager/-/balena-image-manager-7.0.3.tgz",
-      "integrity": "sha512-KLb/5JksYxCR7JClq+MAuCMYPNMAXB9MgqubUvzj4mI/Yec3XD4xZ2BMITwINrl4sMKui/G7t/R5tBNFZsFe9w==",
+      "version": "7.1.0",
+      "resolved": "https://registry.npmjs.org/balena-image-manager/-/balena-image-manager-7.1.0.tgz",
+      "integrity": "sha512-W8Etn0PBQJUlTJ1m1rCuQQ8TTLdkBoNP2SRHQqez2HANQp+T38wFjQXVx9PhQ6J7eqOYuW87SkJiKLpnmlz79Q==",
      "requires": {
        "balena-sdk": "^15.2.1",
        "mime": "^2.4.6",
@ -3814,9 +3814,9 @@
      }
    },
    "balena-preload": {
-      "version": "10.5.0",
-      "resolved": "https://registry.npmjs.org/balena-preload/-/balena-preload-10.5.0.tgz",
-      "integrity": "sha512-tgnTyOSOLB3HxIqlR1NFrTsy1eiiew5Vzmplb82/eZc/vJTrOqal2tFNn6aFay6UQ8+OASUJwANC99zBu1e8mQ==",
+      "version": "11.0.0",
+      "resolved": "https://registry.npmjs.org/balena-preload/-/balena-preload-11.0.0.tgz",
+      "integrity": "sha512-2LuPTw6LoVxuasGhn+cLyA5n7kjvwBtQmBsg08KNhcbEpWLkzrV5jhpNxlMPUuoC2xcMv5Rcg6FWbY87QV5zYQ==",
      "requires": {
        "archiver": "^3.1.1",
        "balena-sdk": "^15.44.0",
@ -3828,6 +3828,7 @@
        "get-port": "^3.2.0",
        "lodash": "^4.17.21",
        "node-cleanup": "^2.1.2",
+        "request": "^2.88.2",
        "request-promise": "^4.2.6",
        "resin-cli-visuals": "^1.8.0",
        "tar-fs": "^2.1.1",
@ -3885,11 +3886,6 @@
            "readable-stream": "^3.4.0"
          }
        },
-        "moment": {
-          "version": "2.29.1",
-          "resolved": "https://registry.npmjs.org/moment/-/moment-2.29.1.tgz",
-          "integrity": "sha512-kHmoybcPV8Sqy59DwNDY3Jefr64lK/by/da0ViFcuA4DH0vQg5Q6Ze5VimxkfQNSC+Mls/Kx53s7TjP1RhFEDQ=="
-        },
        "readable-stream": {
          "version": "3.6.0",
          "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-3.6.0.tgz",
@ -3900,14 +3896,6 @@
            "util-deprecate": "^1.0.1"
          }
        },
-        "semver": {
-          "version": "7.3.5",
-          "resolved": "https://registry.npmjs.org/semver/-/semver-7.3.5.tgz",
-          "integrity": "sha512-PoeGJYh8HK4BTO/a9Tf6ZG3veo/A7ZVsYrSA6J8ny9nb3B1VrpkuN+z9OE5wfE5p6H4LchYZsegiQgbJD94ZFQ==",
-          "requires": {
-            "lru-cache": "^6.0.0"
-          }
-        },
        "tar-fs": {
          "version": "2.1.1",
          "resolved": "https://registry.npmjs.org/tar-fs/-/tar-fs-2.1.1.tgz",
@ -3933,11 +3921,6 @@
            }
          }
        },
-        "tslib": {
-          "version": "2.3.1",
-          "resolved": "https://registry.npmjs.org/tslib/-/tslib-2.3.1.tgz",
-          "integrity": "sha512-77EbyPPpMz+FRFRuAFlWMtmgUWGe9UOG2Z25NqCwiIjRhOf5iKGuzSe5P2w1laq+FkRy4p+PCuVkJSGkzTEKVw=="
-        },
        "zip-stream": {
          "version": "2.1.3",
          "resolved": "https://registry.npmjs.org/zip-stream/-/zip-stream-2.1.3.tgz",
@ -4243,9 +4226,9 @@
      "dev": true
    },
    "big-integer": {
-      "version": "1.6.48",
-      "resolved": "https://registry.npmjs.org/big-integer/-/big-integer-1.6.48.tgz",
-      "integrity": "sha512-j51egjPa7/i+RdiRuJbPdJ2FIUYYPhvYLjzoYbcMMm62ooO6F94fETG4MTs46zPAF9Brs04OajboA/qTGuz78w=="
+      "version": "1.6.50",
+      "resolved": "https://registry.npmjs.org/big-integer/-/big-integer-1.6.50.tgz",
+      "integrity": "sha512-+O2uoQWFRo8ysZNo/rjtri2jIwjr3XfeAgRjAUADRqGG+ZITvyn8J1kvXLTaKVr3hhGXk+f23tKfdzmklVM9vQ=="
    },
    "big.js": {
      "version": "5.2.2",
--- a/package.json
+++ b/package.json
@ -1,6 +1,6 @@
 {
  "name": "balena-cli",
-  "version": "12.50.1",
+  "version": "12.52.0",
  "description": "The official balena Command Line Interface",
  "main": "./build/app.js",
  "homepage": "https://github.com/balena-io/balena-cli",
@ -205,8 +205,8 @@
    "balena-device-init": "^6.0.0",
    "balena-errors": "^4.7.1",
    "balena-image-fs": "^7.0.6",
-    "balena-image-manager": "^7.0.3",
-    "balena-preload": "^10.5.0",
+    "balena-image-manager": "^7.1.0",
+    "balena-preload": "^11.0.0",
    "balena-release": "^3.2.0",
    "balena-sdk": "^15.51.1",
    "balena-semver": "^2.3.0",
@ -289,6 +289,6 @@
    "windosu": "^0.3.0"
  },
  "versionist": {
-    "publishedAt": "2021-09-30T00:16:32.621Z"
+    "publishedAt": "2021-11-20T00:51:09.643Z"
  }
 }
--- a/tests/utils/eol-conversion.spec.ts
+++ b/tests/utils/eol-conversion.spec.ts
@ -74,7 +74,6 @@ describe('detectEncoding() function', function () {
 			'node_modules/.bin/gulp',
 			'node_modules/.bin/tsc',
 			'node_modules/.bin/balena-lint',
-			'node_modules/.bin/balena-preload',
 			'node_modules/.bin/catch-uncommitted',
 		];
Author	SHA1	Message	Date
Balena CI	55fc9b2ade	v12.52.0	2021-11-20 03:19:00 +02:00
bulldozer-balena[bot]	6c29d0ae27	Merge pull request #2334 from balena-io/2005-os-esr-versions-hostapp os versions, os download: Add support for balenaOS ESR versions	2021-11-20 01:17:17 +00:00
Paulo Castro	f46452f6de	os download: Display OS version actually downloaded (range or 'recommended') Change-type: patch	2021-11-20 00:43:15 +00:00
Paulo Castro	c166ec7597	os versions, os download: Add support for balenaOS ESR versions Change-type: minor	2021-11-20 00:43:15 +00:00
Balena CI	7325c79888	v12.51.3	2021-11-16 19:10:36 +02:00
bulldozer-balena[bot]	2a29b386eb	Merge pull request #2375 from balena-io/missing-digest deploy: Ensure the release fails if an image's digest (hash) is missing	2021-11-16 17:08:41 +00:00
Paulo Castro	23b07f8a41	deploy: Ensure the release fails if an image's digest (hash) is missing Change-type: patch	2021-11-16 11:55:07 +00:00
Balena CI	6d641b4841	v12.51.2	2021-11-16 13:50:52 +02:00
bulldozer-balena[bot]	7b498149b1	Merge pull request #2379 from balena-io/remove-node10-from-resinci.yml Update balena CI configuration (remove Node v10 from npm pipeline list)	2021-11-16 11:49:36 +00:00
Paulo Castro	ae5ea0f4e8	Update balena CI configuration (remove Node v10 from npm pipeline list) Change-type: patch	2021-11-15 23:51:15 +00:00
Balena CI	f635f648da	v12.51.1	2021-10-25 20:54:12 +03:00
bulldozer-balena[bot]	3d4e2cf823	Merge pull request #2256 from balena-io/forum-link Fix forums support link in README.md	2021-10-25 17:52:15 +00:00
Balena CI	ef3b630887	v12.51.0	2021-10-22 22:13:44 +03:00
bulldozer-balena[bot]	19040ccb6c	Merge pull request #2367 from balena-io/support-for-fragments Add support for YAML anchors and aliases in 'docker-compose.yml'	2021-10-22 19:11:47 +00:00
dfunckt	8e712ac910	Add support for YAML anchors and aliases in 'docker-compose.yml' This allows project files to define services from generic fragments by leveraging YAML's anchors and aliases. See here for an example: `43f6537b2c/spec.md (fragments)` Removing the FAILSAFE_SCHEMA flag is not expected to break existing project files, since the default behaviour is more liberal, or cause problems down the road given we perform validation immediately after. Docs for the flag: https://github.com/nodeca/js-yaml#load-string---options- Change-type: minor	2021-10-22 16:42:29 +03:00
Balena CI	c401ed35ac	v12.50.3	2021-10-20 19:33:37 +03:00
bulldozer-balena[bot]	94be97313b	Merge pull request #2359 from balena-io/klutchell/preload-11 preload: Avoid possible ValueError when parsing storage driver	2021-10-20 16:31:34 +00:00
Kyle Harding	48053ecefc	preload: Avoid possible ValueError when parsing storage driver Update balena-preload from 10.5.0 to 11.0.0 Change-type: patch Signed-off-by: Kyle Harding <kyle@balena.io>	2021-10-15 09:44:27 -04:00
Balena CI	cc60e86507	v12.50.2	2021-10-05 13:32:02 +03:00
bulldozer-balena[bot]	bd774e8553	Merge pull request #2357 from balena-io/fix-fleet-rename-error-message Error message when renaming a fleet now mentions the target name.	2021-10-05 10:27:51 +00:00
Carlo Miguel F. Cruz	c493c33e38	Error message when renaming a fleet now mentions the target name. Change-type: patch Signed-off-by: Carlo Miguel F. Cruz <carloc@balena.io>	2021-10-05 17:01:07 +08:00
Scott Lowe	112a7b8194	Fix forums support link in README.md Change-type: patch Signed-off-by: Scott Lowe <scott@balena.io>	2021-04-21 06:57:28 +00:00