v22.1.1

Deploy: Limit the submitted error_message of images that fail to build to 300K characters

.dockerignore

@@ -0,0 +1,37 @@
+# Reminders:
+# * Matching rules are different to `.gitignore`
+# * A pattern without '**' matches in the project's root directory only
+# * Leading and trailing '/' are discarded (it is not possible to
+#   distinguish between files and directories)
+# * More details: https://github.com/balena-io-modules/dockerignore
+
+# development and testing tools or IDEs
+**/*.log
+**/*.pid
+**/*.seed
+.idea
+.lock-wscript
+.nvmrc
+.nyc_output
+.vscode
+coverage
+lib-cov
+logs
+pids
+
+# OS cache files
+**/.DS_Store
+
+# balena CLI config and build files
+**/.balenaconf
+**/.fast-boot.json
+**/.resinconf
+balenarc.yml
+build
+build-bin
+dist
+node_modules
+oclif.manifest.json
+package-lock.json
+resinrc.yml
+tmp

.gitattributes

@@ -4,9 +4,13 @@
 *.* -eol
 
 *.sh text eol=lf
+.dockerignore eol=lf
+Dockerfile eol=lf
+Dockerfile.* eol=lf
+* text=auto eol=lf
 
 # lf for the docs as it's auto-generated and will otherwise trigger an uncommited error on windows
-doc/cli.markdown text eol=lf
+docs/balena-cli.md text eol=lf
 # crlf for the eol conversion test files
 tests/test-data/projects/docker-compose/basic/service2/file2-crlf.sh eol=crlf
 tests/test-data/projects/no-docker-compose/basic/src/windows-crlf.sh eol=crlf

.github/CODEOWNERS

@@ -1 +0,0 @@
-* @balena-io/balena-cli

.github/ISSUE_TEMPLATE.md

@@ -1,7 +1,7 @@
 
 # About this issue tracker
 
-*balenaCLI (Command Line Interface) is a tool used to interact with the balena platform.
+*The balena CLI (Command Line Interface) is a tool used to interact with the balena platform.
 This GitHub issue tracker is used for bug reports and feature requests regarding the CLI
 tool. General and troubleshooting questions (such as setting up your project to work with a
 balenalib base image) are encouraged to be posted to the [balena
@@ -12,7 +12,7 @@ community can both contribute and benefit from the answers.*
 the same problem or feature please add comments to the existing issue.*
 
 *Thank you for your time and effort creating the issue report, and helping us improve
-balenaCLI!*
+the balena CLI!*
 
 ---
 
@@ -32,11 +32,11 @@ Please describe what actually happened instead:
   Examples:
 
 ```
-balena push myApp
+balena push myFleet
 balena push 192.168.0.12
-balena deploy myApp
-balena deploy myApp --build
-balena build . -a myApp
+balena deploy myFleet
+balena deploy myFleet --build
+balena build . -f myFleet
 balena build . -A armv7hf -d raspberrypi3
 ```
 
@@ -48,7 +48,7 @@ additional information. The `--logs` option reveals additional information for t
 
 ```
 balena build . --logs
-balena deploy myApp --build --logs
+balena deploy myFleet --build --logs
 ```
 
 # Steps to Reproduce the Problem
@@ -63,11 +63,11 @@ fixed it.
 
 # Specifications
 
-- **balenaCLI version:** e.g. 1.2.3 (output of the `"balena version -a"` command)
+- **balena CLI version:** e.g. 1.2.3 (output of the `"balena version -a"` command)
 - **Cloud backend: openBalena or balenaCloud?** If unsure, it will be balenaCloud
 - **Operating system version:** e.g. Windows 10, Ubuntu 18.04, macOS 10.14.5
 - **32/64 bit OS and processor:** e.g. 32-bit Windows on 64-bit Intel processor
-- **Install method:** npm or zip package or executable installer
+- **Install method:** npm or standalone package or executable installer
 - **If npm install, Node.js and npm version:** e.g. Node v8.16.0 and npm v6.4.1
 
 # Additional References

.github/actions/publish/action.yml

@@ -0,0 +1,145 @@
+---
+name: package and draft GitHub release
+# https://github.com/product-os/flowzone/tree/master/.github/actions
+inputs:
+  json:
+    description: 'JSON stringified object containing all the inputs from the calling workflow'
+    required: true
+  secrets:
+    description: 'JSON stringified object containing all the secrets from the calling workflow'
+    required: true
+  variables:
+    description: 'JSON stringified object containing all the variables from the calling workflow'
+    required: true
+
+  # --- custom environment
+  XCODE_APP_LOADER_EMAIL:
+    type: string
+    default: 'accounts+apple@balena.io'
+  NODE_VERSION:
+    type: string
+    default: '22.x'
+  VERBOSE:
+    type: string
+    default: 'true'
+
+runs:
+  # https://docs.github.com/en/actions/creating-actions/creating-a-composite-action
+  using: 'composite'
+  steps:
+    - name: Download custom source artifact
+      uses: actions/download-artifact@cc203385981b70ca67e1cc392babf9cc229d5806 # v4.1.9
+      with:
+        name: custom-${{ github.event.pull_request.head.sha || github.event.head_commit.id }}-${{ runner.os }}-${{ runner.arch }}
+        path: ${{ runner.temp }}
+
+    - name: Extract custom source artifact
+      shell: pwsh
+      working-directory: .
+      run: tar -xf ${{ runner.temp }}/custom.tgz
+
+    - name: Setup Node.js
+      uses: actions/setup-node@cdca7365b2dadb8aad0a33bc7601856ffabcc48e # v4
+      with:
+        node-version: ${{ inputs.NODE_VERSION }}
+        cache: npm
+
+    - name: Set up Python 3.11
+      if: runner.os == 'macOS'
+      uses: actions/setup-python@65d7f2d534ac1bc67fcd62888c5f4f3d2cb2b236 # v4
+      with:
+        python-version: '3.11'
+
+    - name: Install additional tools
+      if: runner.os == 'Windows'
+      shell: bash
+      run: |
+        choco install yq
+
+    - name: Install additional tools
+      if: runner.os == 'macOS'
+      shell: bash
+      run: |
+        brew install coreutils
+
+    # https://www.electron.build/code-signing.html
+    # https://github.com/Apple-Actions/import-codesign-certs
+    - name: Import Apple code signing certificate
+      if: runner.os == 'macOS'
+      uses: apple-actions/import-codesign-certs@8f3fb608891dd2244cdab3d69cd68c0d37a7fe93 # v2
+      with:
+        p12-file-base64: ${{ fromJSON(inputs.secrets).APPLE_SIGNING }}
+        p12-password: ${{ fromJSON(inputs.secrets).APPLE_SIGNING_PASSWORD }}
+
+    - name: Import Windows code signing certificate
+      if: runner.os == 'Windows'
+      shell: powershell
+      run: |
+        Set-Content -Path ${{ runner.temp }}/certificate.base64 -Value $env:SM_CLIENT_CERT_FILE_B64
+        certutil -decode ${{ runner.temp }}/certificate.base64 ${{ runner.temp }}/Certificate_pkcs12.p12
+        Remove-Item -path ${{ runner.temp }} -include certificate.base64
+      env:
+        SM_CLIENT_CERT_FILE_B64: ${{ fromJSON(inputs.secrets).SM_CLIENT_CERT_FILE_B64 }}
+
+    # https://github.com/product-os/scripts/tree/master/shared
+    # https://github.com/product-os/balena-concourse/blob/master/pipelines/github-events/template.yml
+    - name: Package release
+      shell: bash
+      run: |
+        set -ea
+
+        [[ '${{ inputs.VERBOSE }}' =~ on|On|Yes|yes|true|True ]] && set -x
+
+        runner_os="$(echo "${RUNNER_OS}" | tr '[:upper:]' '[:lower:]')"
+        runner_arch="$(echo "${RUNNER_ARCH}" | tr '[:upper:]' '[:lower:]')"
+
+        if [[ $runner_os =~ darwin|macos|osx ]]; then
+            CSC_KEY_PASSWORD='${{ fromJSON(inputs.secrets).APPLE_SIGNING_PASSWORD }}'
+            CSC_KEYCHAIN=signing_temp
+            CSC_LINK=${{ fromJSON(inputs.secrets).APPLE_SIGNING }}
+
+        elif [[ $runner_os =~ windows|win ]]; then
+            SM_HOST=${{ fromJSON(inputs.secrets).SM_HOST }}
+            SM_API_KEY=${{ fromJSON(inputs.secrets).SM_API_KEY }}
+            SM_CLIENT_CERT_FILE='${{ runner.temp }}\Certificate_pkcs12.p12'
+            SM_CLIENT_CERT_PASSWORD=${{ fromJSON(inputs.secrets).SM_CLIENT_CERT_PASSWORD }}
+            SM_CODE_SIGNING_CERT_SHA1_HASH=${{ fromJSON(inputs.secrets).SM_CODE_SIGNING_CERT_SHA1_HASH }}
+
+            curl --silent --retry 3 --fail https://one.digicert.com/signingmanager/api-ui/v1/releases/smtools-windows-x64.msi/download \
+              -H "x-api-key:$SM_API_KEY" \
+              -o smtools-windows-x64.msi
+            msiexec -i smtools-windows-x64.msi -qn
+            PATH="/c/Program Files/DigiCert/DigiCert One Signing Manager Tools:${PATH}"
+            smksp_registrar.exe list
+            smctl.exe keypair ls
+            smctl.exe windows certsync
+            /c/Windows/System32/certutil.exe -csp "DigiCert Signing Manager KSP" -key -user
+
+            # (signtool.exe) https://github.com/actions/runner-images/blob/main/images/win/Windows2019-Readme.md#installed-windows-sdks
+            PATH="/c/Program Files (x86)/Windows Kits/10/bin/${runner_arch}:${PATH}"
+        fi
+
+        npm run package
+
+        find dist -type f -maxdepth 1
+
+      env:
+        # https://github.blog/2020-08-03-github-actions-improvements-for-fork-and-pull-request-workflows/#improvements-for-public-repository-forks
+        # https://docs.github.com/en/actions/managing-workflow-runs/approving-workflow-runs-from-public-forks#about-workflow-runs-from-public-forks
+        CSC_FOR_PULL_REQUEST: true
+        # https://docs.digicert.com/es/software-trust-manager/ci-cd-integrations/plugins/github-custom-action-for-keypair-signing.html
+        TIMESTAMP_SERVER: http://timestamp.digicert.com
+        # Apple notarization (automation/build-bin.ts)
+        XCODE_APP_LOADER_EMAIL: ${{ inputs.XCODE_APP_LOADER_EMAIL }}
+        XCODE_APP_LOADER_PASSWORD: ${{ fromJSON(inputs.secrets).XCODE_APP_LOADER_PASSWORD }}
+        XCODE_APP_LOADER_TEAM_ID: ${{ inputs.XCODE_APP_LOADER_TEAM_ID }}
+
+    - name: Upload artifacts
+      uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
+      with:
+        name: gh-release-${{ github.event.pull_request.head.sha || github.event.head_commit.id }}-${{ strategy.job-index }}
+        path: |
+          dist
+          !dist/balena
+        retention-days: 1
+        if-no-files-found: error

.github/actions/test/action.yml

@@ -0,0 +1,65 @@
+---
+name: test release
+# https://github.com/product-os/flowzone/tree/master/.github/actions
+inputs:
+  json:
+    description: "JSON stringified object containing all the inputs from the calling workflow"
+    required: true
+  secrets:
+    description: "JSON stringified object containing all the secrets from the calling workflow"
+    required: true
+  variables:
+    description: "JSON stringified object containing all the variables from the calling workflow"
+    required: true
+
+  # --- custom environment
+  NODE_VERSION:
+    type: string
+    default: '22.x'
+  VERBOSE:
+    type: string
+    default: "true"
+
+runs:
+  # https://docs.github.com/en/actions/creating-actions/creating-a-composite-action
+  using: "composite"
+  steps:
+    # https://github.com/actions/setup-node#caching-global-packages-data
+    - name: Setup Node.js
+      uses: actions/setup-node@cdca7365b2dadb8aad0a33bc7601856ffabcc48e # v4
+      with:
+        node-version: ${{ inputs.NODE_VERSION }}
+        cache: npm
+
+    - name: Set up Python 3.11
+      if: runner.os == 'macOS'
+      uses: actions/setup-python@65d7f2d534ac1bc67fcd62888c5f4f3d2cb2b236 # v4
+      with:
+        python-version: "3.11"
+
+    - name: Test release
+      shell: bash
+      run: |
+        set -ea
+
+        [[ '${{ inputs.VERBOSE }}' =~ on|On|Yes|yes|true|True ]] && set -x
+
+        if [[ -e package-lock.json ]] || [[ -e npm-shrinkwrap.json ]]; then
+            npm ci
+        else
+            npm i
+        fi
+
+        npm run build
+        npm run test:core
+
+    - name: Compress custom source
+      shell: pwsh
+      run: tar --exclude-vcs -acf ${{ runner.temp }}/custom.tgz .
+
+    - name: Upload custom artifact
+      uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
+      with:
+        name: custom-${{ github.event.pull_request.head.sha || github.event.head_commit.id }}-${{ runner.os }}-${{ runner.arch }}
+        path: ${{ runner.temp }}/custom.tgz
+        retention-days: 1

.github/renovate.json

@@ -0,0 +1,4 @@
+{
+	"extends": ["github>balena-io/renovate-config"],
+	"postUpdateOptions": ["npmDedupe"]
+}

.github/workflows/flowzone.yml

@@ -0,0 +1,45 @@
+name: Flowzone
+on:
+  pull_request:
+    types: [opened, synchronize, closed]
+    branches: [main, master]
+  pull_request_target:
+    types: [opened, synchronize, closed]
+    branches: [main, master]
+jobs:
+  flowzone:
+    name: Flowzone
+    uses: product-os/flowzone/.github/workflows/flowzone.yml@master
+    # prevent duplicate workflow executions for pull_request and pull_request_target
+    if: |
+      (
+        github.event.pull_request.head.repo.full_name == github.repository &&
+        github.event_name == 'pull_request'
+      ) || (
+        github.event.pull_request.head.repo.full_name != github.repository &&
+        github.event_name == 'pull_request_target'
+      )
+    secrets: inherit
+    with:
+      custom_test_matrix: >
+        {
+          "os": [
+            ["self-hosted", "X64"],
+            ["self-hosted", "ARM64"],
+            ["macos-13"],
+            ["windows-2019"],
+            ["macos-latest-xlarge"]
+          ]
+        }
+      custom_publish_matrix: >
+        {
+          "os": [
+            ["self-hosted", "X64"],
+            ["self-hosted", "ARM64"],
+            ["macos-13"],
+            ["windows-2019"],
+            ["macos-latest-xlarge"]
+          ]
+        }
+      github_prerelease: false
+      restrict_custom_actions: false

.gitignore

@@ -1,47 +1,36 @@
-# Logs
-logs
-*.log
+# Reminders:
+# * A pattern without '/' matches in subdirectories as well (files and directories)
+# * A leading '/' anchors matching to the directory where `.gitignore` is defined
+# * A trailing '/' makes the pattern match against directories only
+# More details: https://git-scm.com/docs/gitignore
 
-# Runtime data
-pids
+# development and testing tools or IDEs
+*.log
 *.pid
 *.seed
+/.idea/
+/.lock-wscript
+/.nvmrc
+/.nyc_output/
+/.vscode/
+/coverage/
+/lib-cov/
+/logs
+/pids
 
-# Directory for instrumented libs generated by jscoverage/JSCover
-lib-cov
-
-# Coverage directory used by tools like istanbul
-coverage
-.nyc_output
-
-# node-waf configuration
-.lock-wscript
-
-# Compiled binary addons (http://nodejs.org/api/addons.html)
-build/Release
-
-# Dependency directory
-# Commenting this out is preferred by some people, see
-# https://www.npmjs.org/doc/misc/npm-faq.html#should-i-check-my-node_modules-folder-into-git-
-node_modules
-
-package-lock.json
-.resinconf
-.balenaconf
-resinrc.yml
-balenarc.yml
-
+# OS cache files
 .DS_Store
-.idea
-.nvmrc
-.vscode
 
-/tmp
-build/
-build-bin/
-build-zip/
-dist/
-
-# Ignore fast-boot cache file
-**/.fast-boot.json
+# balena CLI config and build files
+.balenaconf
+.fast-boot.json
+.resinconf
+/balenarc.yml
+/build/
+/build-bin/
+/dist/
+/node_modules
 /oclif.manifest.json
+/package-lock.json
+/resinrc.yml
+/tmp/

.husky/pre-commit

@@ -0,0 +1 @@
+node automation/check-npm-version.js && ts-node automation/check-doc.ts

.mocharc-standalone.js

@@ -0,0 +1,6 @@
+const commonConfig = require('./.mocharc.js');
+
+module.exports = {
+	...commonConfig,
+	spec: ['tests/auth/*.spec.ts', 'tests/commands/**/*.spec.ts'],
+};

.mocharc.js

@@ -0,0 +1,10 @@
+module.exports = {
+	reporter: 'spec',
+	require: 'ts-node/register/transpile-only',
+	file: './tests/config-tests',
+	timeout: 48000,
+	// To test only, say, 'push.spec.ts', do it as follows so that
+	// requests are authenticated:
+	// spec: ['tests/auth/*.spec.ts', 'tests/**/deploy.spec.ts'],
+	spec: 'tests/**/*.spec.ts',
+};

.resinci.yml

@@ -1,20 +0,0 @@
----
-npm:
-  platforms:
-    - name: linux
-      os: ubuntu
-      architecture: x86_64
-      node_versions:
-        - "10"
-        - "12"
-        - "14"
-    - name: linux
-      os: alpine
-      architecture: x86_64
-      node_versions:
-        - "10"
-        - "12"
-        - "14"
-
-docker:
-  publish: false

.travis.yml

@@ -1,25 +0,0 @@
-language: node_js
-os:
-  - linux
-  - osx
-node_js:
-  - "10"
-matrix:
-  exclude:
-    node_js: "10"
-script:
-  - node --version
-  - npm --version
-  - npm run ci
-  # - npm run build:standalone
-  # - npm run build:installer
-notifications:
-  email: false
-deploy:
-  - provider: script
-    script: npm run release
-    skip_cleanup: true
-    on:
-      tags: true
-      condition: "$TRAVIS_TAG =~ ^v?[[:digit:]]+\\.[[:digit:]]+\\.[[:digit:]]+"
-      repo: balena-io/balena-cli

CONTRIBUTING.md

@@ -1,6 +1,6 @@
 # Contributing
 
-balenaCLI is an open source project and your contribution is welcome!
+The balena CLI is an open source project and your contribution is welcome!
 
 * Install the dependencies listed in the [NPM Installation
   section](./INSTALL-ADVANCED.md#npm-installation) section of the installation instructions. Check
@@ -14,7 +14,7 @@ balenaCLI is an open source project and your contribution is welcome!
 In order to ease development:
 
 * `npm run build:fast` skips some of the build steps for interactive testing, or
-* `npm run test:source` skips testing the standalone zip packages (which is rather slow)
+* `npm run test:source` skips testing the standalone packages (which is rather slow)
 * `./bin/balena-dev` uses `ts-node/register` to transpile on the fly.
 
 Before opening a PR, test your changes with `npm test`. Keep compatibility in mind, as the CLI is
@@ -23,25 +23,73 @@ this will only help if you add some test cases for your new code!
 
 ## Semantic versioning, commit messages and the ChangeLog
 
-The CLI version numbering adheres to [Semantic Versioning](http://semver.org/). The following
-header/row is required in the body of a commit message, and will cause the CI build to fail if absent:
+When a pull request is merged, Balena's versionbot / Continuous Integration system takes care of
+automatically creating a new CLI release on both the [npm
+registry](https://www.npmjs.com/package/balena-cli) and the GitHub [releases
+page](https://github.com/balena-io/balena-cli/releases). The release version numbering adheres to
+the [Semantic Versioning's](http://semver.org/) concept of patch, minor and major releases.
+Generally, bug fixes and documentation changes are classed as patch changes, while new features are
+classed as minor changes. If a change breaks backwards compatibility, it is a major change.
+
+A new version entry is also automatically added to the
+[CHANGELOG.md](https://github.com/balena-io/balena-cli/blob/master/CHANGELOG.md) file when a pull
+request is merged. Each pull request corresponds to a single version / release. Each commit in the
+pull request becomes a bullet point entry in the Changelog. The Changelog file should not be
+manually edited.
+
+To support this automation, a commit message should be structured as follows:
+
+```text
+The first line becomes a bullet point in the CHANGELOG file
+
+Optionally, a more detailed description in one or more paragraphs.
+The detailed description can be seen with `git log`, but it is not copied
+to the CHANGELOG file.
 
-```
 Change-type: patch|minor|major
 ```
 
-Version numbers and commit messages are automatically added to the `CHANGELOG.md` file by the CI
-build flow, after a pull request is merged. It should not be manually edited.
+Only the first line of the commit message is copied to the Changelog file. The `Change-type` footer
+must be preceded by a blank line, and indicates the commit's semver change type. When a PR consists
+of multiple commits, the commits may have different change type values. As a whole, the PR will
+produce a release of the "highest" change type. For example, two commits mixing patch and minor
+change types will produce a minor CLI release, while two commits mixing minor and major change
+types will produce a major CLI release.
+
+The commit message is parsed / checked by versionbot with the
+[resin-commit-lint](https://github.com/balena-io-modules/resin-commit-lint#resin-commit-lint)
+package.
+
+Because of the way that the Changelog file is automatically updated from commit messages, which
+become the source of "what's new" for CLI end users, we advocate "meaningful commits" and
+user-focused commit messages. A meaningful commit is one that, in isolation, introduces a fix or
+feature (or part of a fix or feature) that makes sense at the Changelog level, and which leaves the
+CLI in a non-broken state. Sometimes, in the course of preparing a single pull request, a developer
+creates several commits as a way of saving their "work in progress", which may even fail to build
+(e.g. `npm run build` fails), and which is then fixed or undone by further commits in the same PR.
+In this situation, the recommendation is to "squash" or "fixup" the work-in-progress commits into
+fewer, meaningful commits. Interactive rebase is a good tool to achieve this:
+[blog](https://thoughtbot.com/blog/git-interactive-rebase-squash-amend-rewriting-history),
+[docs](https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History).
+
+Mixing multiple distinct features or bug fixes in a single commit is discouraged, because the
+description will likely not fit in the single-line Changelog bullet point and also because it
+makes it harder to review the pull request (especially a large one) and harder to isolate and
+revert individual changes in case a bug is found later on. Create a separate commit for each
+feature / bug fix, or even separate pull requests.
+
+If you need to catch up with changes to the master branch while working on a pull request,
+use rebase instead of merge: [docs](https://git-scm.com/book/en/v2/Git-Branching-Rebasing).
 
 If `package.json` is updated for dependencies listed in the `repo.yml` file (like `balena-sdk`),
-the commit message body should include a line in the following format:
+the commit message body should also include a line in the following format:
 ```
 Update balena-sdk from 12.0.0 to 12.1.0
 ```
 
-This allows the CI to produce nested change logs (with expandable arrows), pulling in commit
-messages from the upstream repositories. The following npm script can be used to automatically
-produce a commit with a suitable commit message:
+This allows versionbot to produce nested Changelog entries (with expandable arrows), pulling in
+commit messages from the upstream repositories. The following npm script can be used to
+automatically produce a commit with a suitable commit message:
 ```
 npm run update balena-sdk ^12.1.0
 ```
@@ -49,7 +97,7 @@ npm run update balena-sdk ^12.1.0
 The script will create a new branch (only if `master` is currently checked out), run `npm update`
 with the given target version and commit the `package.json` and `npm-shrinkwrap.json` files. The
 script by default will set the `Change-type` to `patch` or `minor`, depending on the semver change
-of the updated dependency. For a `major` change type, it can specified as an extra argument:
+of the updated dependency. A `major` change type can specified as an extra argument:
 ```
 npm run update balena-sdk ^12.14.0 patch
 npm run update balena-sdk ^13.0.0 major
@@ -57,33 +105,83 @@ npm run update balena-sdk ^13.0.0 major
 
 ## Editing documentation files (README, INSTALL, Reference website...)
 
-The `doc/cli.markdown` file is automatically generated by running `npm run build:doc` (which also
+The `docs/balena-cli.md` file is automatically generated by running `npm run build:doc` (which also
 runs as part of `npm run build`). That file is then pulled by scripts in the
 [balena-io/docs](https://github.com/balena-io/docs/) GitHub repo for publishing at the [CLI
 Documentation page](https://www.balena.io/docs/reference/cli/).
 
-The content sources for the auto generation of `doc/cli.markdown` are:
+The content sources for the auto generation of `docs/balena-cli.md` are:
 
 * [Selected
   sections](https://github.com/balena-io/balena-cli/blob/v12.23.0/automation/capitanodoc/capitanodoc.ts#L199-L204)
   of the README file.
-* The CLI's command documentation in source code (`lib/commands/` folder), for example:
-  * `lib/commands/push.ts`
-  * `lib/commands/env/add.ts`
+* The CLI's command documentation in source code (`src/commands/` folder), for example:
+  * `src/commands/push.ts`
+  * `src/commands/env/add.ts`
 
 The README file is manually edited, but subsections are automatically extracted for inclusion in
-`doc/cli.markdown` by the `getCapitanoDoc()` function in
+`docs/balena-cli.md` by the `getCapitanoDoc()` function in
 [`automation/capitanodoc/capitanodoc.ts`](https://github.com/balena-io/balena-cli/blob/master/automation/capitanodoc/capitanodoc.ts).
 
+**IMPORTANT**
+
+The file [`capitanodoc.ts`](https://github.com/balena-io/balena-cli/blob/master/automation/capitanodoc/capitanodoc.ts) lists 
+commands to generate documentation from. At the moment, it's manually updated and maintained alphabetically. 
+
+To add a new command to be documented, 
+
+1. Find the resource which it is part of or create a new one. 
+2. List the location of the build file 
+3. Make sure to add your files in alphabetical order
+
+Once added, run the command `npm run build` to generate the documentation
+
 The `INSTALL*.md` and `TROUBLESHOOTING.md` files are also manually edited.
 
+## Patches folder
+
+The `patches` folder contains patch files created with the
+[patch-package](https://www.npmjs.com/package/patch-package) tool. Small code changes to
+third-party modules can be made by directly editing Javascript files under the `node_modules`
+folder and then running `patch-package` to create the patch files. The patch files are then
+applied immediately after `npm install`, through the `postinstall` script defined in
+`package.json`.
+
+The subfolders of the `patches` folder are documented in the
+[apply-patches.js](https://github.com/balena-io/balena-cli/blob/master/patches/apply-patches.js)
+script.
+
+To make changes to the patch files under the `patches` folder, **do not edit them directly,**
+not even for a "single character change" because the hash values in the patch files also need
+to be recomputed by `patch-packages`. Instead, edit the relevant files under `node_modules`
+directly, and then run `patch-packages` with the `--patch-dir` option to specify the subfolder
+where the patch should be saved. For example, edit `node_modules/exit-hook/index.js` and then
+run:
+
+```sh
+$ npx patch-package --patch-dir patches/all exit-hook
+```
+
+That said, these kinds of patches should be avoided in favour of creating pull requests
+upstream. Patch files create additional maintenance work over time as the patches need to be
+updated when the dependencies are updated, and they prevent the compounding community benefit
+that sharing fixes upstream have on open source projects like the balena CLI. The typical
+scenario where these patches are used is when the upstream maintainers are unresponsive or
+unwilling to merge the required fixes, the fixes are very small and specific to the balena CLI,
+and creating a fork of the upstream repo is likely to be more long-term effort than maintaining
+the patches.
+
 ## Windows
 
-The `npm run build:installer` script (which generates the `.exe` executable installer on Windows)
-specifically requires [MSYS2](https://www.msys2.org/) to be installed.  Other than that, the
-standard Command Prompt or PowerShell can be used (though MSYS2 is still handy, as it provides
-'git' and a number of common unix utilities). If changes are made to npm scripts in `package.json`,
-check that they also run on a standard Windows Command Prompt.
+Besides the regular npm installation dependencies, the `npm run build:installer` script
+that produces the `.exe` graphical installer on Windows also requires
+[NSIS](https://sourceforge.net/projects/nsis/) and [MSYS2](https://www.msys2.org/) to be
+installed. Be sure to add `C:\Program Files (x86)\NSIS` to the PATH, so that `makensis`
+is available. MSYS2 is recommended when developing the balena CLI on Windows.
+
+If changes are made to npm scripts in `package.json`, don't assume that a Unix shell like
+bash is available. For example, some Windows shells don't have the `cp` and `rm` commands,
+which is why you'll often find `ncp` and `rimraf` used in `package.json` scripts.
 
 ## Updating the 'npm-shrinkwrap.json' file
 
@@ -116,6 +214,24 @@ Optionally, these steps may be automated by installing the
 npx npm-merge-driver install -g
 ```
 
+## `fast-boot` and `npm link` - modifying the `node_modules` folder
+
+During development or debugging, it is sometimes useful to temporarily modify the `node_modules`
+folder (with or without making the respective changes to the `npm-shrinkwrap.json` file),
+replacing dependencies with different versions. This can be achieved with the `npm link`
+command, or by manually editing or copying files to the `node_modules` folder.
+
+Unexpected behavior may then be observed because of the CLI's use of the
+[fast-boot2](https://www.npmjs.com/package/fast-boot2) package that caches module resolution.
+`fast-boot2` is configured in `src/fast-boot.ts` to automatically invalidate the cache if
+changes are made to the `package.json` or `npm-shrinkwrap.json` files, but the cache won't
+be automatically invalidated if `npm link` is used or if manual modifications are made to the
+`node_modules` folder. In this situation:
+
+* Manually delete the module cache file (typically `~/.balena/cli-module-cache.json`), or
+* Use the `bin/balena-dev` entry point (instead of `bin/balena`) as it does not activate
+  `fast-boot2`.
+
 ## TypeScript and oclif
 
 The CLI currently contains a mix of plain JavaScript and
@@ -189,3 +305,11 @@ gotchas to bear in mind:
   `node_modules/balena-sdk/node_modules/balena-errors`  
   In the case of subclasses of `TypedError`, a string comparison may be used instead:  
   `error.name === 'BalenaApplicationNotFound'`
+
+## Further debugging notes
+
+* If you need to selectively run specific tests, `it.only` will not work in cases when authorization is required as part of the test cycle.  In order to target specific tests, control execution via `.mocharc.js` instead.  Here is an example of targeting the `deploy` tests.
+
+	replace: `spec: 'tests/**/*.spec.ts',`
+
+	with: `spec: ['tests/auth/*.spec.ts', 'tests/**/deploy.spec.ts'],`

INSTALL-ADVANCED.md

@@ -1,4 +1,4 @@
-# balenaCLI Advanced Installation Options
+# balena CLI Advanced Installation Options
 
 **These are alternative, advanced installation options. Most users would prefer the [recommended,
 streamlined installation
@@ -8,11 +8,11 @@ There are 3 options to choose from to install balena's CLI:
 
 * [Executable Installer](#executable-installer): the easiest method on Windows and macOS, using the
   traditional graphical desktop application installers.
-* [Standalone Zip Package](#standalone-zip-package): these are plain zip files with the balenaCLI
-  executable in them: extract and run. Available for all platforms: Linux, Windows, macOS.
+* [Standalone tar.gz Package](#standalone-targz-package): these are plain tar.gz files with the balena CLI
+  bundled within. Available for all platforms: Linux, Windows, macOS.
   Recommended also for scripted installation in CI (continuous integration) environments.
 * [NPM Installation](#npm-installation): recommended for Node.js developers who may be interested
-  in integrating balenaCLI in their existing projects or workflow.
+  in integrating the balena CLI in their existing projects or workflow.
 
 Some specific CLI commands have a few extra installation steps: see section [Additional
 Dependencies](#additional-dependencies).
@@ -30,9 +30,9 @@ instructions:
 > If you would like to use WSL, follow the [installations instructions for
 > Linux](./INSTALL-LINUX.md) rather than Windows, as WSL consists of a Linux environment.
 
-If you had previously installed the CLI using a standalone zip package, it may be a good idea to
+If you had previously installed the CLI using a standalone tar package, it may be a good idea to
 check your system's `PATH` environment variable for duplicate entries, as the terminal will use the
-entry that comes first. Check the [Standalone Zip Package](#standalone-zip-package) instructions
+entry that comes first. Check the [Standalone tar.gz Package](#standalone-targz-package) instructions
 for how to modify the PATH variable.
 
 By default, the CLI is installed to the following folders:
@@ -40,94 +40,111 @@ By default, the CLI is installed to the following folders:
 OS  | Folders
 --- | ---
 Windows: | `C:\Program Files\balena-cli\`
-macOS:   | `/usr/local/lib/balena-cli/` <br> `/usr/local/bin/balena`
+macOS:   | `/usr/local/src/balena-cli/` <br> `/usr/local/bin/balena`
 
-## Standalone Zip Package
+## Standalone tar.gz Package
 
-1. Download the latest zip file from the [releases page](https://github.com/balena-io/balena-cli/releases).
+1. Download the latest tar.gz file from the [releases page](https://github.com/balena-io/balena-cli/releases).
    Look for a file name that ends with the word "standalone", for example:  
-   `balena-cli-vX.Y.Z-linux-x64-standalone.zip`  ← _also for the Windows Subsystem for Linux_  
-   `balena-cli-vX.Y.Z-macOS-x64-standalone.zip`  
-   `balena-cli-vX.Y.Z-windows-x64-standalone.zip`
+   `balena-cli-vX.Y.Z-linux-x64-standalone.tar.gz`  ← _also for the Windows Subsystem for Linux_  
+   `balena-cli-vX.Y.Z-macOS-x64-standalone.tar.gz`
+   `balena-cli-vX.Y.Z-windows-x64-standalone.tar.gz`
 
-2. Extract the zip file contents to any folder you choose. The extracted contents will include a
-   `balena-cli` folder.
+2. Extract the tar.gz file contents to any folder you choose. The extracted contents will be a `balena` folder containing a `bin` subdirectory.
 
-3. Add the `balena-cli` folder to the system's `PATH` environment variable.  
+3. Add the `balena/bin` folder to the system's `PATH` environment variable.
    See instructions for:
    [Linux](https://stackoverflow.com/questions/14637979/how-to-permanently-set-path-on-linux-unix) |
    [macOS](https://www.architectryan.com/2012/10/02/add-to-the-path-on-mac-os-x-mountain-lion/#.Uydjga1dXDg) |
    [Windows](https://www.computerhope.com/issues/ch000549.htm)
 
-> * If you are using macOS Catalina (10.15), [check this known issue and
->   workaround](https://github.com/balena-io/balena-cli/issues/1479).
-> * **Linux Alpine** and **Busybox:** the standalone zip package is not currently compatible with
+> * If you are using macOS 10.15 or later (Catalina, Big Sur), [check this known issue and
+>   workaround](https://github.com/balena-io/balena-cli/issues/2244).
+> * **Linux Alpine** and **Busybox:** the standalone tar.gz package is not currently compatible with
 >   these "compact" Linux distributions, because of the alternative C libraries they ship with.
 >   For these, consider the [NPM Installation](#npm-installation) option.
-> * Note that moving the `balena` executable out of the extracted `balena-cli` folder on its own
+> * Note that moving the `balena/bin/balena` executable out of the extracted `balena` folder on its own
 >   (e.g. moving it to `/usr/local/bin/balena`) will **not** work, as it depends on the other
->   folders and files also present in the `balena-cli` folder.
+>   folders and files also present in the `balena` folder.
 
-To update the CLI to a new version, download a new release zip file and replace the previous
+To update the CLI to a new version, download a new release tar.gz file and replace the previous
 installation folder. To uninstall, simply delete the folder and edit the PATH environment variable
 as described above.
 
 ## NPM Installation
 
-If you are a Node.js developer, you may wish to install balenaCLI via [npm](https://www.npmjs.com).
+If you are a Node.js developer, you may wish to install the balena CLI via [npm](https://www.npmjs.com).
 The npm installation involves building native (platform-specific) binary modules, which require
-some additional development tools to be installed first:
+some development tools to be installed first, as follows.
 
-* [Node.js](https://nodejs.org/) version 10 (min **10.20.0**) or 12 (version 14 is not yet fully supported)
-  * **Linux, macOS** and **Windows Subsystem for Linux (WSL):**  
-    Installing Node via [nvm](https://github.com/nvm-sh/nvm/blob/master/README.md) is recommended.
-    When the "system" or "default" Node.js and npm packages are installed with "apt-get" in Linux
-    distributions like Ubuntu, users often report permission or compilation errors when running
-    "npm install". This [sample
-    Dockerfile](https://gist.github.com/pdcastro/5d4d96652181e7da685a32caf629dd44) shows the CLI
-    installation steps on an Ubuntu 18.04 base image.
-* [Python 2.7](https://www.python.org/), [git](https://git-scm.com/), [make](https://www.gnu.org/software/make/), [g++](https://gcc.gnu.org/)
-  * **Linux** and **Windows Subsystem for Linux (WSL):**  
-    `sudo apt-get install -y python git make g++`
-  * **macOS:** install Apple's Command Line Tools by running on a Terminal window:  
-    `xcode-select --install`
+> **The balena CLI currently requires Node.js version >=20.6.0**
+> **Versions 23 and later are not yet fully supported.**
 
-On **Windows (not WSL),** the dependencies above and additional ones can be met by installing:
+### Install development tools
 
-* Node.js from the [Nodejs.org download page](https://nodejs.org/en/download/).
-* The [MSYS2 shell](https://www.msys2.org/), which provides `git`, `make`, `g++`, `ssh`, `rsync`
-  and more:
-  * `pacman -S git openssh rsync gcc make`
+#### **Linux or WSL** (Windows Subsystem for Linux)
+
+```sh
+$ sudo apt-get update && sudo apt-get -y install curl python3 git make g++
+$ curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.38.0/install.sh | bash
+$ . ~/.bashrc
+$ nvm install 22
+```
+
+The `curl` command line above uses
+[nvm](https://github.com/nvm-sh/nvm/blob/master/README.md#install--update-script) to install
+Node.js, instead of using `apt-get`. Installing Node.js through `apt-get` is a common source of
+problems from permission errors to conflict with other system packages, and therefore not
+recommended.
+
+#### **macOS**
+
+* Download and install Apple's Command Line Tools from https://developer.apple.com/downloads/
+* Install Node.js through [nvm](https://github.com/nvm-sh/nvm/blob/master/README.md#install--update-script):
+
+```sh
+$ curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.38.0/install.sh | bash
+$ . ~/.bashrc
+$ nvm install 22
+```
+
+#### **Windows** (not WSL)
+
+Install:
+
+  * If you'd like the ability to switch between Node.js versions, install
+- Node.js v22 from the [Nodejs.org releases page](https://nodejs.org/en/download/releases/).
+    [nvm-windows](https://github.com/coreybutler/nvm-windows#node-version-manager-nvm-for-windows)
+    instead.
+* The [MSYS2 shell](https://www.msys2.org/), which provides `git`, `make`, `g++` and more:
+  * `pacman -S git gcc make openssh p7zip`
   * [Set a Windows environment variable](https://www.onmsft.com/how-to/how-to-set-an-environment-variable-in-windows-10): `MSYS2_PATH_TYPE=inherit`
   * Note that a bug in the MSYS2 launch script (`msys2_shell.cmd`) makes text-based
     interactive CLI menus to misbehave. [Check this Github issue for a
     workaround](https://github.com/msys2/MINGW-packages/issues/1633#issuecomment-240583890).
 * The Windows Driver Kit (WDK), which is needed to compile some native Node modules. It is **not**
   necessary to install Visual Studio, only the WDK, which is "step 2" in the following guides:
-  * [WDK for Windows 10](https://docs.microsoft.com/en-us/windows-hardware/drivers/download-the-wdk#download-icon-step-2-install-wdk-for-windows-10-version-1903)
+  * [WDK for Windows 10](https://docs.microsoft.com/en-us/windows-hardware/drivers/download-the-wdk#download-icon-step-2-install-refreshed-wdk-for-windows-10-version-2004)
   * [WDK for earlier versions of Windows](https://docs.microsoft.com/en-us/windows-hardware/drivers/other-wdk-downloads#step-2-install-the-wdk)
-* The [windows-build-tools](https://www.npmjs.com/package/windows-build-tools) npm package (which
-  provides Python 2.7 and more), by running the following command on an [administrator
-  console](https://www.howtogeek.com/194041/how-to-open-the-command-prompt-as-administrator-in-windows-8.1/):
-  
-  `npm install -g --production windows-build-tools`
+* The [windows-build-tools](https://www.npmjs.com/package/windows-build-tools) npm package,
+  by running the following command on an [administrator
+  console](https://www.howtogeek.com/194041/how-to-open-the-command-prompt-as-administrator-in-windows-8.1/):  
+  `npm install --global --production windows-build-tools`
 
-With these dependencies in place, the balenaCLI installation command is:
+### Install the balena CLI
+
+After installing the development tools, install the balena CLI with:
 
 ```sh
-$ npm install balena-cli -g --production --unsafe-perm
+$ npm install balena-cli --global --production --unsafe-perm
 ```
 
-`--unsafe-perm` is required when `npm install` is executed as the root user, or on systems where
-the global install directory is not user-writable. It allows npm install steps to download and save
-prebuilt native binaries, and also allows the execution of npm scripts like `postinstall` that are
-used to patch dependencies. It is usually possible to omit `--unsafe-perm` if installing under a
-regular (non-root) user account, especially if using a user-managed node installation such as
-[nvm](https://github.com/creationix/nvm).
+`--unsafe-perm` is needed when `npm install` is executed as the `root` user (e.g. in a Docker
+container) in order to allow npm scripts like `postinstall` to be executed.
 
 ## Additional Dependencies
 
-The `balena ssh`, `scan`, `build`, `deploy`, `preload` and `os configure` commands may require
+The `balena device ssh`, `device detect`, `build`, `deploy` and `preload` commands may require
 additional software to be installed. Check the Additional Dependencies sections for each operating
 system:
 
@@ -135,9 +152,9 @@ system:
 * [macOS](./INSTALL-MAC.md#additional-dependencies)
 * [Linux](./INSTALL-LINUX.md#additional-dependencies)
 
-The `build` and `deploy` commands are also capable of using Docker or balenaEngine on a remote
-server, or on a balenaOS device running a [balenaOS development
-image](https://www.balena.io/docs/reference/OS/overview/2.x/#dev-vs-prod-images)). Reasons why this
+Where Docker or balenaEngine are required, they may be installed on the local machine (where the
+balena CLI is executed), on a remote server, or on a balenaOS device running a [balenaOS development
+image](https://www.balena.io/docs/reference/OS/overview/2.x/#dev-vs-prod-images). Reasons why this
 may be desirable include:
 
 * To avoid having to install Docker on the development machine / laptop.
@@ -145,6 +162,7 @@ may be desirable include:
 * To build or run images "natively" on an ARM device, avoiding the need for QEMU emulation.
 
 To use a remote Docker Engine (daemon) or balenaEngine, specify the remote machine's IP address and
-port number with the `--dockerHost` and `--dockerPort` command-line options. For more details,
-check `balena help build` or the [online
+port number with the `--dockerHost` and `--dockerPort` command-line options. The `preload` command
+has additional requirements because the bind mount feature is used. For more details, see
+`balena help` for each command or the [online
 reference](https://www.balena.io/docs/reference/cli/#cli-command-reference).

INSTALL-LINUX.md

@@ -1,60 +1,79 @@
-# balenaCLI Installation Instructions for Linux
+# balena CLI Installation Instructions for Linux
 
-These instructions are for the recommended installation option. They are suitable for most Linux
-distributions, except notably for **Linux Alpine** or **Busybox**. For these distros, see [advanced
-installation options](./INSTALL-ADVANCED.md).
+These instructions are suitable for most Linux distributions on Intel x86, such as
+Ubuntu, Debian, Fedora, Arch Linux and other glibc-based distributions.
+For the ARM architecture and for Linux distributions not based on glibc, such as
+Alpine Linux, follow the [NPM Installation](./INSTALL-ADVANCED.md#npm-installation)
+method.
 
 Selected operating system: **Linux**
 
-1. Download the latest zip file from the [latest release
+1. Download the latest tar.gz file from the [latest release
    page](https://github.com/balena-io/balena-cli/releases/latest). Look for a file name that ends
-   with "-standalone.zip", for example:  
-   `balena-cli-vX.Y.Z-linux-x64-standalone.zip`
+   with "-standalone.tar.gz", for example:  
+   `balena-cli-vX.Y.Z-linux-x64-standalone.tar.gz`
 
-2. Extract the zip file contents to any folder you choose. The extracted contents will include a
-   `balena-cli` folder.
+2. Extract the tar.gz file contents to any folder you choose, for example `/home/james`.
+   The extracted contents will include a `balena/bin` folder.
 
-3. Add the `balena-cli` folder to the system's `PATH` environment variable. There are several
-   ways of achieving this on Linux: See this [StackOverflow post](https://stackoverflow.com/questions/14637979/how-to-permanently-set-path-on-linux-unix). Close and reopen the terminal window
-   so that the changes to PATH can take effect.
+3. Add that folder (e.g. `/home/james/balena/bin`) to the `PATH` environment variable.
+   Check this [StackOverflow
+   post](https://stackoverflow.com/questions/14637979/how-to-permanently-set-path-on-linux-unix)
+   for instructions. Close and reopen the terminal window so that the changes to `PATH`
+   can take effect.
 
 4. Check that the installation was successful by running the following commands on a
-   command terminal:  
-   * `balena version` - should print balenaCLI's version
+   terminal window:  
+   * `balena version` - should print the CLI's version
    * `balena help` - should print a list of available commands
 
-No further steps are required to run most balenaCLI commands. The `balena ssh`, `scan`, `build`,
-`deploy` and `preload` commands may require additional software to be installed, as described
-below.
-
-To update balenaCLI to a new version, download a new release zip file and replace the previous
+To update the balena CLI to a new version, download a new release tar.gz file and replace the previous
 installation folder. To uninstall, simply delete the folder and edit the PATH environment variable
 as described above.
 
+## sudo configuration
+
+A few CLI commands require execution through sudo, e.g. `sudo balena device detect`.  
+If your Linux distribution has an `/etc/sudoers` file that defines a `secure_path`
+setting, run `sudo visudo` to edit it and add the balena CLI's installation folder to
+the ***pre-existing*** `secure_path` setting, for example:
+
+```text
+Defaults   secure_path="/home/james/balena-cli:<pre-existing entries go here>"
+```
+
+If an `/etc/sudoers` file does not exist, or if it does not contain a pre-existing
+`secure_path` setting, do not change it.
+
+If you also have Docker installed, ensure that it can be executed ***without*** `sudo`, so that
+CLI commands like `balena build` and `balena preload` can also be executed without `sudo`.
+Check Docker's [post-installation
+steps](https://docs.docker.com/engine/install/linux-postinstall/) on how to achieve this.
+
 ## Additional Dependencies
 
 ### build, deploy
 
 These commands require [Docker](https://docs.docker.com/install/overview/) or
-[balenaEngine](https://www.balena.io/engine/) to be available (on a local or remote machine). Most
-users will simply follow [Docker's installation
-instructions](https://docs.docker.com/install/overview/) to install Docker on the same laptop (dev
-machine) where balenaCLI is installed. The [advanced installation
-options](./INSTALL-ADVANCED.md) document describes other possibilities.
+[balenaEngine](https://www.balena.io/engine/) to be available on a local or remote
+machine. Most users will follow [Docker's installation
+instructions](https://docs.docker.com/install/overview/) to install Docker on the same
+workstation as the balena CLI. The [advanced installation
+options](./INSTALL-ADVANCED.md#additional-dependencies) document describes other possibilities.
 
-### balena ssh
+### balena device ssh
 
-The `balena ssh` command requires the `ssh` command-line tool to be available. Most Linux
+The `balena device ssh` command requires the `ssh` command-line tool to be available. Most Linux
 distributions will already have it installed. Otherwise, `sudo apt-get install openssh-client`
 should do the trick on Debian or Ubuntu.
 
-The `balena ssh` command also requires an SSH key to be added to your balena account: see [SSH
+The `balena device ssh` command also requires an SSH key to be added to your balena account: see [SSH
 Access documentation](https://www.balena.io/docs/learn/manage/ssh-access/). The `balena key*`
 command set can also be used to list and manage SSH keys: see `balena help -v`.
 
-### balena scan
+### balena device detect
 
-The `balena scan` command requires a multicast DNS (mDNS) service like
+The `balena device detect` command requires a multicast DNS (mDNS) service like
 [Avahi](https://en.wikipedia.org/wiki/Avahi_(software)), which is installed by default on most
 desktop Linux distributions. Otherwise, on Debian or Ubuntu, the installation command would be
 `sudo apt-get install avahi-daemon`.

INSTALL-MAC.md

@@ -1,4 +1,4 @@
-# balenaCLI Installation Instructions for macOS
+# balena CLI Installation Instructions for macOS
 
 These instructions are for the recommended installation option. Advanced users may also be
 interested in [advanced installation options](./INSTALL-ADVANCED.md).
@@ -7,37 +7,43 @@ Selected operating system: **macOS**
 
 1. Download the installer from the [latest release
    page](https://github.com/balena-io/balena-cli/releases/latest).
-   Look for a file name that ends with "-installer.pkg":  
-   `balena-cli-vX.Y.Z-macOS-x64-installer.pkg`  
+   Look for a file name that ends with "-installer.pkg":
+   `balena-cli-vX.Y.Z-macOS-x64-installer.pkg`
 
-2. Double click the downloaded file to run the installer. After the installation completes,
-   close and re-open any open [command
-   terminal](https://www.balena.io/docs/reference/cli/#choosing-a-shell-command-promptterminal)
-   windows (so that the changes made by the installer to the PATH environment variable can take
-   effect).
+2. Double click on the downloaded file to run the installer and follow the installer's
+   instructions.
 
-3. Check that the installation was successful by running the following commands on a
-   command terminal:  
-   * `balena version` - should print balenaCLI's version
-   * `balena help` - should print a list of available commands
+3. Check that the installation was successful:
+   - [Open the Terminal
+   app](https://support.apple.com/en-gb/guide/terminal/apd5265185d-f365-44cb-8b09-71a064a42125/mac).
+   - On the terminal prompt, type `balena version` and hit Enter. It should display
+     the version of the balena CLI that you have installed.
 
-No further steps are required to run most balenaCLI commands. The `balena ssh`, `build`, `deploy`
-and `preload` commands may require additional software to be installed, as described below.
+No further steps are required to run most CLI commands. The `balena device ssh`, `build`, `deploy`
+and `preload` commands may require additional software to be installed, as described
+in the next section.
+
+To update the balena CLI, repeat the steps above for the new version.  
+To uninstall it, run the following command on a terminal prompt:
+
+```text
+sudo /usr/local/src/balena-cli/bin/uninstall
+```
 
 ## Additional Dependencies
 
 ### build and deploy
 
 These commands require [Docker](https://docs.docker.com/install/overview/) or
-[balenaEngine](https://www.balena.io/engine/) to be available (on a local or remote machine). Most
-users will simply follow [Docker's installation
-instructions](https://docs.docker.com/install/overview/) to install Docker on the same laptop (dev
-machine) where balenaCLI is installed. The [advanced installation
-options](./INSTALL-ADVANCED.md) document describes other possibilities.
+[balenaEngine](https://www.balena.io/engine/) to be available on a local or remote
+machine. Most users will follow [Docker's installation
+instructions](https://docs.docker.com/install/overview/) to install Docker on the same
+workstation as the balena CLI. The [advanced installation
+options](./INSTALL-ADVANCED.md#additional-dependencies) document describes other possibilities.
 
-### balena ssh
+### balena device ssh
 
-The `balena ssh` command requires the `ssh` command-line tool to be available. To check whether
+The `balena device ssh` command requires the `ssh` command-line tool to be available. To check whether
 it is already installed, run `ssh` on a Terminal window. If it is not yet installed, the options
 include:
 
@@ -46,23 +52,23 @@ include:
   Components → Command Line Tools → Install.
 * Or, install [Homebrew](https://brew.sh/), then `brew install openssh`
 
-The `balena ssh` command also requires an SSH key to be added to your balena account: see [SSH
+The `balena device ssh` command also requires an SSH key to be added to your balena account: see [SSH
 Access documentation](https://www.balena.io/docs/learn/manage/ssh-access/). The `balena key*`
 command set can also be used to list and manage SSH keys: see `balena help -v`.
 
 ### balena preload
 
-Like the `build` and `deploy` commands, the `preload` command requires Docker, with the additional
-restriction that Docker must be installed on the local machine (because Docker's bind mounting
-feature is used). Also, for some device types (such as the Raspberry Pi), the `preload` command
-requires Docker to support the [AUFS storage
+Like the `build` and `deploy` commands, the `preload` command requires Docker.
+Preloading balenaOS images for some older device types (like the Raspberry
+Pi 3, but not the Raspberry 4) requires Docker to support the [AUFS storage
 driver](https://docs.docker.com/storage/storagedriver/aufs-driver/). Unfortunately, Docker Desktop
-for Windows dropped support for the AUFS filesystem in Docker CE versions greater than 18.06.1. The
-present workaround is to either:
+for Windows and macOS dropped support for the AUFS filesystem in Docker CE versions greater than
+18.06.1. The present workarounds are to either:
 
+* Install the balena CLI on Linux (e.g. Ubuntu) with a virtual machine like VirtualBox.
+  This works because Docker for Linux still supports AUFS. Hint: if using a virtual machine,
+  copy the image file over, rather than accessing it through "file sharing", to avoid errors.
 * Downgrade Docker Desktop to version 18.06.1. Link: [Docker CE for
   Mac](https://docs.docker.com/docker-for-mac/release-notes/#docker-community-edition-18061-ce-mac73-2018-08-29)
-* Install balenaCLI on a Linux machine (as Docker for Linux still supports AUFS). A Linux Virtual
-  Machine also works, but a Docker container is _not_ recommended.
 
-Long term, we are working on replacing AUFS with overlay2 for the affected device types.
+We are working on replacing AUFS with overlay2 in balenaOS images of the affected device types.

INSTALL-WINDOWS.md

@@ -1,4 +1,4 @@
-# balenaCLI Installation Instructions for Windows
+# balena CLI Installation Instructions for Windows
 
 These instructions are for the recommended installation option. Advanced users may also be
 interested in [advanced installation options](./INSTALL-ADVANCED.md).
@@ -8,21 +8,19 @@ Selected operating system: **Windows**
 1. Download the installer from the [latest release
    page](https://github.com/balena-io/balena-cli/releases/latest).
    Look for a file name that ends with "-installer.exe":  
-   `balena-cli-vX.Y.Z-windows-x64-installer.exe`  
+   `balena-cli-vX.Y.Z-windows-x64-installer.exe`
 
-2. Double click the downloaded file to run the installer. After the installation completes,
-   close and re-open any open [command
-   terminal](https://www.balena.io/docs/reference/cli/#choosing-a-shell-command-promptterminal)
-   windows (so that the changes made by the installer to the PATH environment variable can take
-   effect).
+2. Double click on the downloaded file to run the installer and follow the installer's
+   instructions.
 
-3. Check that the installation was successful by running the following commands on a
-   command terminal:  
-   * `balena version` - should print balenaCLI's version
-   * `balena help` - should print a list of available commands
+3. Check that the installation was successful:
+   - Click on the Windows Start Menu, type PowerShell, and then click
+     on Windows PowerShell.
+   - On the command prompt, type `balena version` and hit Enter. It should display
+     the version of the balena CLI that you have installed.
 
-No further steps are required to run most balenaCLI commands. The `balena ssh`, `scan`, `build`,
-`deploy`, `preload` and `os configure` commands may require additional software to be installed, as
+No further steps are required to run most CLI commands. The `balena device ssh`, `device detect`, `build`,
+`deploy` and `preload` commands may require additional software to be installed, as
 described below.
 
 ## Additional Dependencies
@@ -30,15 +28,15 @@ described below.
 ### build and deploy
 
 These commands require [Docker](https://docs.docker.com/install/overview/) or
-[balenaEngine](https://www.balena.io/engine/) to be available (on a local or remote machine). Most
-users will simply follow [Docker's installation
-instructions](https://docs.docker.com/install/overview/) to install Docker on the same laptop (dev
-machine) where balenaCLI is installed. The [advanced installation
-options](./INSTALL-ADVANCED.md) document describes other possibilities.
+[balenaEngine](https://www.balena.io/engine/) to be available on a local or remote
+machine. Most users will follow [Docker's installation
+instructions](https://docs.docker.com/install/overview/) to install Docker on the same
+workstation as the balena CLI. The [advanced installation
+options](./INSTALL-ADVANCED.md#additional-dependencies) document describes other possibilities.
 
-### balena ssh
+### balena device ssh
 
-The `balena ssh` command requires the `ssh` command-line tool to be available. Microsoft started
+The `balena device ssh` command requires the `ssh` command-line tool to be available. Microsoft started
 distributing an SSH client with Windows 10, which is automatically installed through Windows
 Update. To check whether it is installed, run `ssh` on a Windows Command Prompt or PowerShell. It
 can also be [manually
@@ -46,37 +44,30 @@ installed](https://docs.microsoft.com/en-us/windows-server/administration/openss
 if needed. For older versions of Windows, there are several ssh/OpenSSH clients provided by 3rd
 parties.
 
-The `balena ssh` command also requires an SSH key to be added to your balena account: see [SSH
+The `balena device ssh` command also requires an SSH key to be added to your balena account: see [SSH
 Access documentation](https://www.balena.io/docs/learn/manage/ssh-access/). The `balena key*`
 command set can also be used to list and manage SSH keys: see `balena help -v`.
 
-### balena scan
+### balena device detect
 
-The `balena scan` command requires a multicast DNS (mDNS) service like Apple's Bonjour.
+The `balena device detect` command requires a multicast DNS (mDNS) service like Apple's Bonjour.
 Many Windows machines will already have this service installed, as it is bundled in popular
 applications such as Skype (Wikipedia lists [several others](https://en.wikipedia.org/wiki/Bonjour_(software))).
 Otherwise, Bonjour for Windows can be downloaded and installed from: https://support.apple.com/kb/DL999
 
 ### balena preload
 
-Like the `build` and `deploy` commands, the `preload` command requires Docker, with the additional
-restriction that Docker must be installed on the local machine (because Docker's bind mounting
-feature is used). Also, for some device types (such as the Raspberry Pi), the `preload` command
-requires Docker to support the [AUFS storage
+Like the `build` and `deploy` commands, the `preload` command requires Docker.
+Preloading balenaOS images for some older device types (like the Raspberry
+Pi 3, but not the Raspberry 4) requires Docker to support the [AUFS storage
 driver](https://docs.docker.com/storage/storagedriver/aufs-driver/). Unfortunately, Docker Desktop
-for Windows dropped support for the AUFS filesystem in Docker CE versions greater than 18.06.1. The
-present workaround is to either:
+for Windows and macOS dropped support for the AUFS filesystem in Docker CE versions greater than
+18.06.1. The present workarounds are to either:
 
+* Install the balena CLI on Linux (e.g. Ubuntu) with a virtual machine like VirtualBox.
+  This works because Docker for Linux still supports AUFS. Hint: if using a virtual machine,
+  copy the image file over, rather than accessing it through "file sharing", to avoid errors.
 * Downgrade Docker Desktop to version 18.06.1. Link: [Docker CE for
   Windows](https://docs.docker.com/docker-for-windows/release-notes/#docker-community-edition-18061-ce-win73-2018-08-29)
-* Install balenaCLI on a Linux machine (as Docker for Linux still supports AUFS). A Linux Virtual
-  Machine also works, but a Docker container is _not_ recommended.
 
-Long term, we are working on replacing AUFS with overlay2 for the affected device types.
-
-### balena os configure
-
-* The `balena os configure` command is currently not supported on Windows natively, but works with
-  the [Windows Subsystem for Linux](https://docs.microsoft.com/en-us/windows/wsl/about) (WSL). When
-  using WSL, [install balenaCLI for
-  Linux](https://github.com/balena-io/balena-cli/blob/master/INSTALL-LINUX.md).
+We are working on replacing AUFS with overlay2 in balenaOS images of the affected device types.

INSTALL.md

@@ -1,4 +1,4 @@
-# balenaCLI Installation Instructions
+# balena CLI Installation Instructions
 
 Please select your operating system:
 

MIGRATIONS.md

@@ -0,0 +1,45 @@
+## Migrating to balena CLI v22
+
+This guide outlines the changes introduced in balena CLI v22 and provides instructions for when and how to migrate.
+
+---
+
+### For Installer Users (Windows .exe, macOS .pkg)
+
+If you are using the Windows executable (.exe) or macOS package (.pkg) installers, **no changes** are required for this update. You can continue to use the installers as before.
+
+---
+
+### For npm Installation Users
+
+If you installed balena CLI via npm, **no changes** are required for this update. Your existing installation and update process remains the same.
+
+---
+
+### For Standalone Installation Users
+
+Users of the standalone balena CLI will need to make the following adjustments:
+
+1.  **Archive Format Change**: The distribution archive format has changed from `.zip` to `.tar.gz`. You will need to use the `tar` command instead of `unzip` to extract the CLI.
+
+    * **Previous command (v21.x.x and older):**
+        ```bash
+        unzip balena-cli-v21.1.12-linux-x64-standalone.zip
+        ```
+    * **New command (v22.0.0 and newer):**
+        ```bash
+        tar -xzf balena-cli-v22.0.0-linux-x64-standalone.tar.gz
+        ```
+
+2.  **Executable Path Change**: The path to the balena CLI executable within the extracted folder has been updated.
+
+    * **Previous path (v21.x.x and older):**
+        ```
+        balena-cli/balena
+        ```
+    * **New path (v22.0.0 and newer):**
+        ```
+        balena/bin/balena
+        ```
+
+Please update your scripts and any aliases to reflect these changes if you are using the standalone version.

README.md

@@ -1,4 +1,4 @@
-# balenaCLI
+# balena CLI
 
 The official balena Command Line Interface.
 
@@ -7,19 +7,21 @@ The official balena Command Line Interface.
 
 ## About
 
-balenaCLI is a Command Line Interface for [balenaCloud](https://www.balena.io/cloud/) or
+The balena CLI is a Command Line Interface for [balenaCloud](https://www.balena.io/cloud/) or
 [openBalena](https://www.balena.io/open/). It is a software tool available for Windows, macOS and
 Linux, used through a command prompt / terminal window. It can be used interactively or invoked in
-scripts. balenaCLI builds on the [balena API](https://www.balena.io/docs/reference/api/overview/)
+scripts. The balena CLI builds on the [balena API](https://www.balena.io/docs/reference/api/overview/)
 and the [balena SDK](https://www.balena.io/docs/reference/sdk/node-sdk/), and can also be directly
-imported in Node.js applications. balenaCLI is an [open-source project on
+imported in Node.js applications. The balena CLI is an [open-source project on
 GitHub](https://github.com/balena-io/balena-cli/), and your contribution is also welcome!
 
 ## Installation
 
-Check the [balenaCLI installation instructions on
+Check the [balena CLI installation instructions on
 GitHub](https://github.com/balena-io/balena-cli/blob/master/INSTALL.md).
 
+Note for v22 Migration: If you are upgrading to balena CLI v22 from a previous standalone installation, please [see the v22 Migration Guide](https://github.com/balena-io/balena-cli/blob/master/MIGRATIONS.md) for installation changes.
+
 ## Choosing a shell (command prompt/terminal)
 
 On **Windows,** the standard Command Prompt (`cmd.exe`) and
@@ -28,34 +30,36 @@ are supported. Alternative shells include:
 
 * [MSYS2](https://www.msys2.org/):
   * Install additional packages with the command:  
-    `pacman -S git openssh rsync`
+    `pacman -S git gcc make openssh p7zip`
   * [Set a Windows environment variable](https://www.onmsft.com/how-to/how-to-set-an-environment-variable-in-windows-10): `MSYS2_PATH_TYPE=inherit`
   * Note that a bug in the MSYS2 launch script (`msys2_shell.cmd`) makes text-based interactive CLI
     menus to break. [Check this Github issue for a
     workaround](https://github.com/msys2/MINGW-packages/issues/1633#issuecomment-240583890).
-* [MSYS](http://www.mingw.org/wiki/MSYS): select the `msys-rsync` and `msys-openssh` packages too
+
+* [MSYS](http://www.mingw.org/wiki/MSYS)
 * [Git for Windows](https://git-for-windows.github.io/)
   * During the installation, you will be prompted to choose between _"Use MinTTY"_ and _"Use
     Windows' default console window"._ Choose the latter, because of the same [MSYS2
     bug](https://github.com/msys2/MINGW-packages/issues/1633) mentioned above (Git for Windows
     actually uses MSYS2). For a screenshot, check this
     [comment](https://github.com/balena-io/balena-cli/issues/598#issuecomment-556513098).
+
 * Microsoft's [Windows Subsystem for Linux](https://docs.microsoft.com/en-us/windows/wsl/about)
   (WSL). In this case, a Linux distribution like Ubuntu is installed via the Microsoft Store, and a
-  balenaCLI release **for Linux** should be selected. See
-  [FAQ](https://github.com/balena-io/balena-cli/blob/master/TROUBLESHOOTING.md) for using balenaCLI
-  with WSL and Docker Desktop for Windows.
+  balena CLI release **for Linux** should be selected. See
+  [FAQ](https://github.com/balena-io/balena-cli/blob/master/TROUBLESHOOTING.md) for using the
+  balena CLI with WSL and Docker Desktop for Windows.
 
 On **macOS** and **Linux,** the standard terminal window is supported. Optionally, `bash` command
 auto completion may be enabled by copying the
-[balena-completion.bash](https://github.com/balena-io/balena-cli/blob/master/balena-completion.bash)
+[balena_comp](https://github.com/balena-io/balena-cli/blob/master/completion/balena-completion.bash)
 file to your system's `bash_completion` directory: check [Docker's command completion
 guide](https://docs.docker.com/compose/completion/) for system setup instructions.
 
 ## Logging in
 
 Several CLI commands require access to your balenaCloud account, for example in order to push a
-new release to your application. Those commands require creating a CLI login session by running:
+new release to your fleet. Those commands require creating a CLI login session by running:
 
 ```sh
 $ balena login
@@ -75,7 +79,6 @@ HTTP(S) proxies can be configured through any of the following methods, in prece
   file](https://www.npmjs.com/package/balena-settings-client#documentation). It may be:
   * A string in URL format, e.g. `proxy: 'http://localhost:8000'`
   * An object in the format:
-
     ```yaml
     proxy:
         protocol: 'http'
@@ -87,9 +90,9 @@ HTTP(S) proxies can be configured through any of the following methods, in prece
 * The `HTTPS_PROXY` and/or `HTTP_PROXY` environment variables, in the same URL format as
   `BALENARC_PROXY`.
 
-### Proxy setup for balena ssh
+### Proxy setup for balena device ssh
 
-In order to work behind a proxy server, the `balena ssh` command requires the
+In order to work behind a proxy server, the `balena device ssh` command requires the
 [`proxytunnel`](http://proxytunnel.sourceforge.net/) package (command-line tool) to be installed.
 `proxytunnel` is available for Linux distributions like Ubuntu/Debian (`apt install proxytunnel`),
 and for macOS through [Homebrew](https://brew.sh/). Windows support is limited to the [Windows
@@ -106,10 +109,10 @@ server, it should be configured with the following rules in the `squid.conf` fil
 
 The `BALENARC_NO_PROXY` variable may be used to exclude specified destinations from proxying.
 
-> * This feature requires balenaCLI version 11.30.8 or later. In the case of the npm [installation
+> * This feature requires CLI version 11.30.8 or later. In the case of the npm [installation
 >   option](https://github.com/balena-io/balena-cli/blob/master/INSTALL.md), it also requires
 >   Node.js version 10.16.0 or later.
-> * To exclude a `balena ssh` target from proxying (IP address or `.local` hostname), the
+> * To exclude a `balena device ssh` target from proxying (IP address or `.local` hostname), the
 >   `--noproxy` option should be specified in addition to the `BALENARC_NO_PROXY` variable.
 
 By default (if `BALENARC_NO_PROXY` is not defined), all [private IPv4
@@ -139,28 +142,34 @@ The full CLI command reference is available [on the web](https://www.balena.io/d
 
 ## Support, FAQ and troubleshooting
 
-If you come across any problems or would like to get in touch:
+To learn more, troubleshoot issues, or to contact us for support:
 
-* Check our [FAQ / troubleshooting document](https://github.com/balena-io/balena-cli/blob/master/TROUBLESHOOTING.md).
-* Ask us a question through the [balenaCloud forum](https://forums.balena.io/c/balena-cloud).
-* For bug reports or feature requests,
-  [have a look at the GitHub issues or create a new one](https://github.com/balena-io/balena-cli/issues/).
+* Check the [masterclass tutorials](https://www.balena.io/docs/learn/more/masterclasses/overview/)
+* Check our [FAQ / troubleshooting document](https://github.com/balena-io/balena-cli/blob/master/TROUBLESHOOTING.md)
+* Ask us a question in the [balena forums](https://forums.balena.io/c/product-support)
+
+For CLI bug reports or feature requests, check the
+[CLI GitHub issues](https://github.com/balena-io/balena-cli/issues/).
 
 ## Deprecation policy
 
-balenaCLI uses [semver versioning](https://semver.org/), with the concepts
+The balena CLI uses [semver versioning](https://semver.org/), with the concepts
 of major, minor and patch version releases.
 
-The latest release of a major version of balenaCLI will remain compatible with
+The latest release of a major version of the balena CLI will remain compatible with
 the balenaCloud backend services for at least one year from the date when the
-following major version is released. For example, balenaCLI v10.17.5, as the
-latest v10 release, would remain compatible with the balenaCloud backend for one
-year from the date when v11.0.0 is released.
+following major version is released. For example, balena CLI v11.36.0, as the
+latest v11 release, would remain compatible with the balenaCloud backend for one
+year from the date when v12.0.0 was released.
 
-At the end of this period, the older major version is considered deprecated and
-some of the functionality that depends on balenaCloud services may stop working
-at any time.
-Users are encouraged to regularly update balenaCLI to the latest version.
+Half way through to that period (6 months after the release of the next major
+version), older major versions of the balena CLI will start printing a deprecation
+warning message when it is used interactively (when `stderr` is attached to a TTY
+device file). At the end of that period, older major versions will exit with an
+error message unless the `--unsupported` flag is used.  This behavior was
+introduced in CLI version 12.47.0 and is also documented by `balena help`.
+To take advantage of the latest backend features and ensure compatibility, users
+are encouraged to regularly update the balena CLI to the latest version.
 
 ## Contributing (including editing documentation files)
 

TROUBLESHOOTING.md

@@ -1,23 +1,23 @@
-# balenaCLI FAQ & Troubleshooting
+# balena CLI FAQ & Troubleshooting
 
-## Where is balenaCLI's configuration file located?
+## Where is the balena CLI's configuration file located?
 
 The per-user configuration file lives in `$HOME/.balenarc.yml` or `%UserProfile%\_balenarc.yml`, in
 Unix based operating systems and Windows respectively.
 
-balenaCLI also attempts to read a `balenarc.yml` file in the current directory, which takes
+The balena CLI also attempts to read a `balenarc.yml` file in the current directory, which takes
 precedence over the per-user configuration file.
 
-## How do I point balenaCLI to the staging environment?
+## How do I point the balena CLI to the staging environment?
 
 Set the `BALENARC_BALENA_URL=balena-staging.com` environment variable, or add
-`balenaUrl: balena-staging.com` to balenaCLI's configuration file.
+`balenaUrl: balena-staging.com` to the balena CLI's configuration file.
 
-## How do I make balenaCLI persist data in another directory?
+## How do I make the balena CLI persist data in another directory?
 
-balenaCLI persists the session token, as well as cached assets, to `$HOME/.balena` or
+The balena CLI persists the session token, as well as cached assets, to `$HOME/.balena` or
 `%UserProfile%\_balena`. This directory can be changed by setting an environment variable,
-`BALENARC_DATA_DIRECTORY=/opt/balena`, or by adding `dataDirectory: /opt/balena` to balenaCLI's
+`BALENARC_DATA_DIRECTORY=/opt/balena`, or by adding `dataDirectory: /opt/balena` to the CLI's
 configuration file, replacing `/opt/balena` with the desired directory.
 
 ## After burning to an SD card, my device doesn't boot
@@ -31,6 +31,11 @@ command again.
 
 Check whether the SD card is locked (a physical switch on the side of the card).
 
+## I get `connect ETIMEDOUT` with `balena device tunnel`
+
+Please update the CLI to the latest version. This issue was fixed in v12.38.5.
+For more details, see: https://github.com/balena-io/balena-cli/issues/2172
+
 ## I get EINVAL errors on Cygwin
 
 The errors may look something like this:
@@ -64,9 +69,9 @@ Or in Windows:
 
 ## I get `EACCES: permission denied` when logging in
 
-balenaCLI stores the session token in `$HOME/.balena` or `C:\Users\<user>\_balena` in UNIX based
+The balena CLI stores the session token in `$HOME/.balena` or `C:\Users\<user>\_balena` in UNIX based
 operating systems and Windows respectively. This error usually indicates that the user doesn't have
-permissions over that directory, which can happen if balenaCLI was executed as the `root` user.
+permissions over that directory, which can happen if the CLI was executed as the `root` user.
 
 Try resetting the ownership by running:
 
@@ -74,9 +79,17 @@ Try resetting the ownership by running:
 $ sudo chown -R <user> $HOME/.balena
 ```
 
-## Broken line wrapping / cursor behavior with `balena ssh`
+## Broken line wrapping / cursor behavior with `balena device ssh`
 
-Users sometimes come across broken line wrapping or cursor behavior in text terminals, for example when long command lines are typed in a `balena ssh` session, or when using text editors like `vim` or `nano`. This is not something specific to balenaCLI, being also a commonly reported issue with standard remote terminal tools like `ssh` or `telnet`. It is often a remote shell configuration issue (files like `/etc/profile`, `~/.bash_profile`, `~/.bash_login`, `~/.profile` and the like), including UTF-8 misconfiguration, the use of unsupported ASCII control characters in shell prompt formatting (e.g. the `$PS1` env var) or the output of tools or log files that use colored text. The issue can sometimes be fixed by resizing the client terminal window, or by running one or more of the following commands on the shell:
+Users sometimes come across broken line wrapping or cursor behavior in text terminals, for example
+when long command lines are typed in a `balena device ssh` session, or when using text editors like `vim`
+or `nano`. This is not something specific to the balena CLI, being also a commonly reported issue
+with standard remote terminal tools like `ssh` or `telnet`.  It is often a remote shell
+configuration issue (files like `/etc/profile`, `~/.bash_profile`, `~/.bash_login`, `~/.profile`
+and the like on the remote machine), including UTF-8 misconfiguration, the use of unsupported ASCII
+control characters in shell prompt formatting (e.g. the `$PS1` env var) or the output of tools or
+log files that use colored text. The issue can sometimes be fixed by simply resizing the client
+terminal window, or by running one or more of the following commands on the shell:
 
 ```sh
 export TERMINAL=linux
@@ -102,7 +115,7 @@ If nothing seems to help, consider also using a different client-side terminal a
 ## "Docker seems to be unavailable" error when using Windows Subsystem for Linux (WSL)
 
 When running on WSL, the recommendation is to install a CLI release for Linux, like the standalone
-zip package for Linux. However, commands like "balena build" will, by default, attempt to reach the
+tar.gz package for Linux. However, commands like "balena build" will, by default, attempt to reach the
 Docker daemon at the Unix socket path `/var/run/docker.sock`, while Docker Desktop for Windows uses
 a Windows named pipe at `//./pipe/docker_engine` (which the Linux CLI on WSL cannot use). A
 solution is:

appveyor.yml

@@ -1,43 +0,0 @@
-# appveyor file
-# http://www.appveyor.com/docs/appveyor-yml
-
-image: Visual Studio 2017
-
-init:
-  - git config --global core.autocrlf input
-
-cache:
-  - C:\Users\appveyor\.node-gyp
-  - '%AppData%\npm-cache'
-
-matrix:
-  fast_finish: true
-
-# what combinations to test
-environment:
-  matrix:
-    - nodejs_version: 10
-
-install:
-  - ps: Install-Product node $env:nodejs_version x64
-  - set PATH=%APPDATA%\npm;%PATH%
-  - npm config set python 'C:\Python27\python.exe'
-  - npm --version
-  # - npm install
-
-build: off
-test: off
-deploy: off
-
-test_script:
-  - node --version
-  - npm --version
-  # - npm test
-
-deploy_script:
-  - node --version
-  - npm --version
-  # - npm run build:standalone
-  # - npm run build:installer
-  # - IF "%APPVEYOR_REPO_TAG%" == "true" (npm run release)
-  # - IF NOT "%APPVEYOR_REPO_TAG%" == "true" (echo 'Not tagged, skipping deploy')

automation/build-bin.ts

@@ -15,30 +15,21 @@
  * limitations under the License.
  */
 
-import type { JsonVersions } from '../lib/commands/version';
-
-import { run as oclifRun } from '@oclif/dev-cli';
-import * as archiver from 'archiver';
-import * as Bluebird from 'bluebird';
-import { execFile } from 'child_process';
-import * as filehound from 'filehound';
+import { run as oclifRun } from '@oclif/core';
+import { exec, execFile } from 'child_process';
+import type { Stats } from 'fs';
 import * as fs from 'fs-extra';
-import * as _ from 'lodash';
+import * as klaw from 'klaw';
 import * as path from 'path';
 import * as rimraf from 'rimraf';
-import * as semver from 'semver';
-import * as util from 'util';
+import { promisify } from 'util';
+import { notarize } from '@electron/notarize';
 
-import { stripIndent } from '../lib/utils/lazy';
-import {
-	diffLines,
-	getSubprocessStdout,
-	loadPackageJson,
-	MSYS2_BASH,
-	ROOT,
-	StdOutTap,
-	whichSpawn,
-} from './utils';
+import { loadPackageJson, ROOT, whichSpawn } from './utils';
+
+const execFileAsync = promisify(execFile);
+const execAsync = promisify(exec);
+const rimrafAsync = promisify(rimraf);
 
 export const packageJSON = loadPackageJson();
 export const version = 'v' + packageJSON.version;
@@ -52,15 +43,13 @@ interface PathByPlatform {
 	[platform: string]: string;
 }
 
-const standaloneZips: PathByPlatform = {
-	linux: dPath(`balena-cli-${version}-linux-${arch}-standalone.zip`),
-	darwin: dPath(`balena-cli-${version}-macOS-${arch}-standalone.zip`),
-	win32: dPath(`balena-cli-${version}-windows-${arch}-standalone.zip`),
-};
-
-const oclifInstallers: PathByPlatform = {
-	darwin: dPath('macos', `balena-${version}.pkg`),
-	win32: dPath('win', `balena-${version}-${arch}.exe`),
+const getOclifInstallersOriginalNames = async (): Promise<PathByPlatform> => {
+	const { stdout } = await execAsync('git rev-parse --short HEAD');
+	const sha = stdout.trim();
+	return {
+		darwin: dPath('macos', `balena-${version}-${sha}-${arch}.pkg`),
+		win32: dPath('win32', `balena-${version}-${sha}-${arch}.exe`),
+	};
 };
 
 const renamedOclifInstallers: PathByPlatform = {
@@ -68,248 +57,149 @@ const renamedOclifInstallers: PathByPlatform = {
 	win32: dPath(`balena-cli-${version}-windows-${arch}-installer.exe`),
 };
 
-export const finalReleaseAssets: { [platform: string]: string[] } = {
-	win32: [standaloneZips['win32'], renamedOclifInstallers['win32']],
-	darwin: [standaloneZips['darwin'], renamedOclifInstallers['darwin']],
-	linux: [standaloneZips['linux']],
+const getOclifStandaloneOriginalNames = async (): Promise<PathByPlatform> => {
+	const { stdout } = await execAsync('git rev-parse --short HEAD');
+	const sha = stdout.trim();
+	return {
+		linux: dPath(`balena-${version}-${sha}-linux-${arch}.tar.gz`),
+		darwin: dPath(`balena-${version}-${sha}-darwin-${arch}.tar.gz`),
+		win32: dPath(`balena-${version}-${sha}-win32-${arch}.tar.gz`),
+	};
 };
 
-/**
- * Given the output of `pkg` as a string (containing warning messages),
- * diff it against previously saved output of known "safe" warnings.
- * Throw an error if the diff is not empty.
- */
-async function diffPkgOutput(pkgOut: string) {
-	const { monochrome } = await import('../tests/helpers');
-	const relSavedPath = path.join(
-		'tests',
-		'test-data',
-		'pkg',
-		`expected-warnings-${process.platform}.txt`,
-	);
-	const absSavedPath = path.join(ROOT, relSavedPath);
-	const ignoreStartsWith = [
-		'> pkg@',
-		'> Fetching base Node.js binaries',
-		'  fetched-',
-	];
-	const modulesRE =
-		process.platform === 'win32'
-			? /(?<=[ '])([A-Z]:)?\\.+?\\node_modules(?=\\)/
-			: /(?<=[ '])\/.+?\/node_modules(?=\/)/;
-	const buildRE =
-		process.platform === 'win32'
-			? /(?<=[ '])([A-Z]:)?\\.+\\build(?=\\)/
-			: /(?<=[ '])\/.+\/build(?=\/)/;
+const renamedOclifStandalone: PathByPlatform = {
+	linux: dPath(`balena-cli-${version}-linux-${arch}-standalone.tar.gz`),
+	darwin: dPath(`balena-cli-${version}-macOS-${arch}-standalone.tar.gz`),
+	win32: dPath(`balena-cli-${version}-windows-${arch}-standalone.tar.gz`),
+};
 
-	const cleanLines = (chunks: string | string[]) => {
-		const lines = typeof chunks === 'string' ? chunks.split('\n') : chunks;
-		return lines
-			.map((line: string) => monochrome(line)) // remove ASCII colors
-			.filter((line: string) => !/^\s*$/.test(line)) // blank lines
-			.filter((line: string) =>
-				ignoreStartsWith.every((i) => !line.startsWith(i)),
-			)
-			.map((line: string) => {
-				// replace absolute paths with relative paths
-				let replaced = line.replace(modulesRE, 'node_modules');
-				if (replaced === line) {
-					replaced = line.replace(buildRE, 'build');
-				}
-				return replaced;
-			});
-	};
-
-	pkgOut = cleanLines(pkgOut).join('\n');
-	const { readFile } = (await import('fs')).promises;
-	const expectedOut = cleanLines(await readFile(absSavedPath, 'utf8')).join(
-		'\n',
-	);
-	if (expectedOut !== pkgOut) {
-		const sep =
-			'================================================================================';
-		const diff = diffLines(expectedOut, pkgOut);
-		const msg = `pkg output does not match expected output from "${relSavedPath}"
-Diff:
-${sep}
-${diff}
-${sep}
-Check whether the new or changed pkg warnings are safe to ignore, then update
-"${relSavedPath}"
-and share the result of your investigation as comments on the pull request.
-Hint: the fix is often a matter of updating the 'pkg.scripts' or 'pkg.assets'
-sections in the CLI's 'package.json' file, or a matter of updating the
-'buildPkg' function in 'automation/build-bin.ts'.  Sometimes it requires
-patching dependencies: See for example 'patches/all/open+7.0.2.patch'.
-${sep}
-`;
-		throw new Error(msg);
+export async function signFilesForNotarization() {
+	console.log('Signing files for notarization');
+	// If signFilesForNotarization is called on the test CI environment (which will not set CSC_LINK)
+	// then we skip the signing process.
+	if (process.platform !== 'darwin' || !process.env.CSC_LINK) {
+		console.log('Skipping signing for notarization');
+		return;
 	}
-}
-
-/**
- * Call `pkg.exec` to generate the standalone zip file, capturing its warning
- * messages (stdout and stderr) in order to call diffPkgOutput().
- */
-async function execPkg(...args: any[]) {
-	const { exec: pkgExec } = await import('pkg');
-	const outTap = new StdOutTap(true);
-	try {
-		outTap.tap();
-		await (pkgExec as any)(...args);
-	} catch (err) {
-		outTap.untap();
-		console.log(outTap.stdoutBuf.join(''));
-		console.error(outTap.stderrBuf.join(''));
-		throw err;
-	}
-	outTap.untap();
-	await diffPkgOutput(outTap.allBuf.join(''));
-}
-
-/**
- * Use the 'pkg' module to create a single large executable file with
- * the contents of 'node_modules' and the CLI's javascript code.
- * Also copy a number of native modules (binary '.node' files) that are
- * compiled during 'npm install' to the 'build-bin' folder, alongside
- * the single large executable file created by pkg. (This is necessary
- * because of a pkg limitation that does not allow binary executables
- * to be directly executed from inside another binary executable.)
- */
-async function buildPkg() {
-	const args = [
-		'--target',
-		'host',
-		'--output',
-		'build-bin/balena',
-		'package.json',
-	];
-	console.log('=======================================================');
-	console.log(`execPkg ${args.join(' ')}`);
-	console.log(`cwd="${process.cwd()}" ROOT="${ROOT}"`);
-	console.log('=======================================================');
-
-	await execPkg(args);
-
-	const paths: Array<[string, string[], string[]]> = [
-		// [platform, [source path], [destination path]]
-		['*', ['open', 'xdg-open'], ['xdg-open']],
-		['*', ['opn', 'xdg-open'], ['xdg-open-402']],
-		['darwin', ['denymount', 'bin', 'denymount'], ['denymount']],
-	];
-	await Promise.all(
-		paths.map(([platform, source, dest]) => {
-			if (platform === '*' || platform === process.platform) {
-				// eg copy from node_modules/open/xdg-open to build-bin/xdg-open
-				return fs.copy(
-					path.join(ROOT, 'node_modules', ...source),
-					path.join(ROOT, 'build-bin', ...dest),
-				);
-			}
-		}),
-	);
-	const nativeExtensionPaths: string[] = await filehound
-		.create()
-		.paths(path.join(ROOT, 'node_modules'))
-		.ext(['node', 'dll'])
-		.find();
-
-	console.log(`\nCopying to build-bin:\n${nativeExtensionPaths.join('\n')}`);
-
-	await Promise.all(
-		nativeExtensionPaths.map((extPath) =>
-			fs.copy(
-				extPath,
-				extPath.replace(
-					path.join(ROOT, 'node_modules'),
-					path.join(ROOT, 'build-bin'),
-				),
-			),
-		),
-	);
-}
-
-/**
- * Run some basic tests on the built pkg executable.
- * TODO: test more than just `balena version -j`; integrate with the
- * existing mocha/chai CLI command testing.
- */
-async function testPkg() {
-	const pkgBalenaPath = path.join(
-		ROOT,
-		'build-bin',
-		process.platform === 'win32' ? 'balena.exe' : 'balena',
-	);
-	console.log(`Testing standalone package "${pkgBalenaPath}"...`);
-	// Run `balena version -j`, parse its stdout as JSON, and check that the
-	// reported Node.js major version matches semver.major(process.version)
-	const stdout = await getSubprocessStdout(pkgBalenaPath, ['version', '-j']);
-	let pkgNodeVersion = '';
-	let pkgNodeMajorVersion = 0;
-	try {
-		const balenaVersions: JsonVersions = JSON.parse(stdout);
-		pkgNodeVersion = balenaVersions['Node.js'];
-		pkgNodeMajorVersion = semver.major(pkgNodeVersion);
-	} catch (err) {
-		throw new Error(stripIndent`
-			Error parsing JSON output of "balena version -j": ${err}
-			Original output: "${stdout}"`);
-	}
-	if (semver.major(process.version) !== pkgNodeMajorVersion) {
-		throw new Error(
-			`Mismatched major version: built-in pkg Node version="${pkgNodeVersion}" vs process.version="${process.version}"`,
-		);
-	}
-	console.log('Success! (standalone package test successful)');
-}
-
-/**
- * Create the zip file for the standalone 'pkg' bundle previously created
- * by the buildPkg() function in 'build-bin.ts'.
- */
-async function zipPkg() {
-	const outputFile = standaloneZips[process.platform];
-	if (!outputFile) {
-		throw new Error(
-			`Standalone installer unavailable for platform "${process.platform}"`,
-		);
-	}
-	await fs.mkdirp(path.dirname(outputFile));
+	console.log('Deleting unneeded zip files...');
 	await new Promise((resolve, reject) => {
-		console.log(`Zipping standalone package to "${outputFile}"...`);
-
-		const archive = archiver('zip', {
-			zlib: { level: 7 },
-		});
-		archive.directory(path.join(ROOT, 'build-bin'), 'balena-cli');
-
-		const outputStream = fs.createWriteStream(outputFile);
-
-		outputStream.on('close', resolve);
-		outputStream.on('error', reject);
-
-		archive.on('error', reject);
-		archive.on('warning', console.warn);
-
-		archive.pipe(outputStream);
-		archive.finalize();
+		klaw('node_modules/')
+			.on('data', (item: { path: string; stats: Stats }) => {
+				if (!item.stats.isFile()) {
+					return;
+				}
+				if (path.basename(item.path).endsWith('.node.bak')) {
+					console.log('Removing pkg .node.bak file', item.path);
+					fs.unlinkSync(item.path);
+				}
+				if (
+					path.basename(item.path).endsWith('.zip') &&
+					path.dirname(item.path).includes('test')
+				) {
+					console.log('Removing zip', item.path);
+					fs.unlinkSync(item.path);
+				}
+			})
+			.on('end', resolve)
+			.on('error', reject);
 	});
+	// Sign all .node files first
+	console.log('Signing .node files...');
+	await new Promise((resolve, reject) => {
+		klaw('node_modules/')
+			.on('data', async (item: { path: string; stats: Stats }) => {
+				if (!item.stats.isFile()) {
+					return;
+				}
+				if (path.basename(item.path).endsWith('.node')) {
+					console.log('running command:', 'codesign', [
+						'-d',
+						'-f',
+						'-s',
+						'Developer ID Application: Balena Ltd (66H43P8FRG)',
+						item.path,
+					]);
+					await whichSpawn('codesign', [
+						'-d',
+						'-f',
+						'-s',
+						'Developer ID Application: Balena Ltd (66H43P8FRG)',
+						item.path,
+					]);
+				}
+			})
+			.on('end', resolve)
+			.on('error', reject);
+	});
+	console.log('Signing other binaries...');
+	console.log('running command:', 'codesign', [
+		'-d',
+		'-f',
+		'--options=runtime',
+		'-s',
+		'Developer ID Application: Balena Ltd (66H43P8FRG)',
+		'node_modules/denymount/bin/denymount',
+	]);
+	await whichSpawn('codesign', [
+		'-d',
+		'-f',
+		'--options=runtime',
+		'-s',
+		'Developer ID Application: Balena Ltd (66H43P8FRG)',
+		'node_modules/denymount/bin/denymount',
+	]);
+	console.log('running command:', 'codesign', [
+		'-d',
+		'-f',
+		'--options=runtime',
+		'-s',
+		'Developer ID Application: Balena Ltd (66H43P8FRG)',
+		'node_modules/macmount/bin/macmount',
+	]);
+	await whichSpawn('codesign', [
+		'-d',
+		'-f',
+		'--options=runtime',
+		'-s',
+		'Developer ID Application: Balena Ltd (66H43P8FRG)',
+		'node_modules/macmount/bin/macmount',
+	]);
 }
 
-export async function buildStandaloneZip() {
-	console.log(`Building standalone zip package for CLI ${version}`);
+export async function buildStandalone() {
+	console.log(`Building standalone tarball for CLI ${version}`);
+	fs.rmSync('./tmp', { recursive: true, force: true });
+	fs.rmSync('./dist', { recursive: true, force: true });
+	fs.mkdirSync('./dist');
 	try {
-		await buildPkg();
-		await testPkg();
-		await zipPkg();
-		console.log(`Standalone zip package build completed`);
+		let packOpts = ['-r', ROOT, '--no-xz'];
+		if (process.platform === 'darwin') {
+			packOpts = packOpts.concat('--targets', `darwin-${arch}`);
+		} else if (process.platform === 'win32') {
+			packOpts = packOpts.concat('--targets', 'win32-x64');
+		} else if (process.platform === 'linux') {
+			packOpts = packOpts.concat('--targets', `linux-${arch}`);
+		}
+
+		console.log(`Building oclif installer for CLI ${version}`);
+		const packCmd = `pack:tarballs`;
+		console.log('=======================================================');
+		console.log(`oclif ${packCmd} ${packOpts.join(' ')}`);
+		console.log(`cwd="${process.cwd()}" ROOT="${ROOT}"`);
+		console.log('=======================================================');
+		const oclifPath = path.join(ROOT, 'node_modules', 'oclif');
+		await oclifRun([packCmd].concat(...packOpts), oclifPath);
+		await renameStandalone();
+
+		console.log(`Standalone tarball package build completed`);
 	} catch (error) {
-		console.error(`Error creating or testing standalone zip package`);
+		console.error(`Error creating or testing standalone tarball package`);
 		throw error;
 	}
 }
 
-async function renameInstallerFiles() {
+async function renameInstallers() {
+	const oclifInstallers = await getOclifInstallersOriginalNames();
 	if (await fs.pathExists(oclifInstallers[process.platform])) {
 		await fs.rename(
 			oclifInstallers[process.platform],
@@ -318,24 +208,42 @@ async function renameInstallerFiles() {
 	}
 }
 
+async function renameStandalone() {
+	const oclifStandalone = await getOclifStandaloneOriginalNames();
+	if (await fs.pathExists(oclifStandalone[process.platform])) {
+		await fs.rename(
+			oclifStandalone[process.platform],
+			renamedOclifStandalone[process.platform],
+		);
+	}
+}
+
 /**
  * If the CSC_LINK and CSC_KEY_PASSWORD env vars are set, digitally sign the
- * executable installer by running the balena-io/scripts/shared/sign-exe.sh
- * script (which must be in the PATH) using a MSYS2 bash shell.
+ * executable installer using Microsoft SignTool.exe (Sign Tool)
+ * https://learn.microsoft.com/en-us/dotnet/framework/tools/signtool-exe
  */
 async function signWindowsInstaller() {
-	if (process.env.CSC_LINK && process.env.CSC_KEY_PASSWORD) {
-		const exeName = renamedOclifInstallers[process.platform];
-		const execFileAsync = util.promisify<string, string[], void>(execFile);
-
+	if (process.env.SM_CODE_SIGNING_CERT_SHA1_HASH) {
+		const exeName = (await getOclifInstallersOriginalNames())[process.platform];
 		console.log(`Signing installer "${exeName}"`);
-		await execFileAsync(MSYS2_BASH, [
-			'sign-exe.sh',
-			'-f',
-			exeName,
+		// trust ...
+		await execFileAsync('signtool.exe', [
+			'sign',
+			'-sha1',
+			process.env.SM_CODE_SIGNING_CERT_SHA1_HASH,
+			'-tr',
+			process.env.TIMESTAMP_SERVER || 'http://timestamp.comodoca.com',
+			'-td',
+			'SHA256',
+			'-fd',
+			'SHA256',
 			'-d',
 			`balena-cli ${version}`,
+			exeName,
 		]);
+		// ... but verify
+		await execFileAsync('signtool.exe', ['verify', '-pa', '-v', exeName]);
 	} else {
 		console.log(
 			'Skipping installer signing step because CSC_* env vars are not set',
@@ -344,7 +252,29 @@ async function signWindowsInstaller() {
 }
 
 /**
- * Run the `oclif-dev pack:win` or `pack:macos` command (depending on the value
+ * Wait for Apple Installer Notarization to continue
+ */
+async function notarizeMacInstaller(): Promise<void> {
+	const teamId = process.env.XCODE_APP_LOADER_TEAM_ID || '66H43P8FRG';
+	const appleId =
+		process.env.XCODE_APP_LOADER_EMAIL || 'accounts+apple@balena.io';
+	const appleIdPassword = process.env.XCODE_APP_LOADER_PASSWORD;
+	const appPath = (await getOclifInstallersOriginalNames())[process.platform];
+	console.log(`Notarizing file "${appPath}"`);
+
+	if (appleIdPassword && teamId) {
+		await notarize({
+			tool: 'notarytool',
+			teamId,
+			appPath,
+			appleId,
+			appleIdPassword,
+		});
+	}
+}
+
+/**
+ * Run the `oclif pack:win` or `pack:macos` command (depending on the value
  * of process.platform) to generate the native installers (which end up under
  * the 'dist' folder). There are some harcoded options such as selecting only
  * 64-bit binaries under Windows.
@@ -354,9 +284,10 @@ export async function buildOclifInstaller() {
 	let packOpts = ['-r', ROOT];
 	if (process.platform === 'darwin') {
 		packOS = 'macos';
+		packOpts = packOpts.concat('--targets', `darwin-${arch}`);
 	} else if (process.platform === 'win32') {
 		packOS = 'win';
-		packOpts = packOpts.concat('-t', 'win32-x64');
+		packOpts = packOpts.concat('--targets', 'win32-x64');
 	}
 	if (packOS) {
 		console.log(`Building oclif installer for CLI ${version}`);
@@ -367,21 +298,26 @@ export async function buildOclifInstaller() {
 		}
 		for (const dir of dirs) {
 			console.log(`rimraf(${dir})`);
-			await Bluebird.fromCallback((cb) => rimraf(dir, cb));
+			await rimrafAsync(dir);
 		}
 		console.log('=======================================================');
-		console.log(`oclif-dev "${packCmd}" "${packOpts.join('" "')}"`);
+		console.log(`oclif ${packCmd} ${packOpts.join(' ')}`);
 		console.log(`cwd="${process.cwd()}" ROOT="${ROOT}"`);
 		console.log('=======================================================');
-		await oclifRun([packCmd].concat(...packOpts));
-		await renameInstallerFiles();
+		const oclifPath = path.join(ROOT, 'node_modules', 'oclif');
+		await oclifRun([packCmd].concat(...packOpts), oclifPath);
 		// The Windows installer is explicitly signed here (oclif doesn't do it).
 		// The macOS installer is automatically signed by oclif (which runs the
 		// `pkgbuild` tool), using the certificate name given in package.json
 		// (`oclif.macos.sign` section).
 		if (process.platform === 'win32') {
 			await signWindowsInstaller();
+		} else if (process.platform === 'darwin') {
+			console.log('Notarizing package...');
+			await notarizeMacInstaller(); // Notarize
+			console.log('Package notarized.');
 		}
+		await renameInstallers();
 		console.log(`oclif installer build completed`);
 	}
 }
@@ -417,4 +353,5 @@ export async function testShrinkwrap(): Promise<void> {
 	if (process.platform !== 'win32') {
 		await whichSpawn(path.resolve(__dirname, 'test-lock-deduplicated.sh'));
 	}
+	await Promise.resolve();
 }

automation/capitanodoc/capitanodoc.ts

@@ -17,6 +17,7 @@
 
 import * as path from 'path';
 import { MarkdownFileParser } from './utils';
+import { GlobSync } from 'glob';
 
 /**
  * This is the skeleton of CLI documentation/reference web page at:
@@ -24,161 +25,111 @@ import { MarkdownFileParser } from './utils';
  *
  * The `getCapitanoDoc` function in this module parses README.md and adds
  * some content to this object.
+ *
+ * IMPORTANT
+ *
+ * All commands need to be stored under a folder in src/commands to maintain uniformity
+ * Generating docs will error out if directive not followed
+ * To add a custom heading for command docs, add the heading next to the folder name
+ * in the `commandHeadings` dictionary.
+ *
+ * This dictionary is the source of truth that creates the docs config which is used
+ * to generate the CLI documentation. By default, the folder name will be used.
+ *
  */
-const capitanoDoc = {
-	title: 'balenaCLI Documentation',
-	introduction: '',
-	categories: [
-		{
-			title: 'API keys',
-			files: ['build/commands/api-key/generate.js'],
-		},
-		{
-			title: 'Application',
-			files: [
-				'build/commands/apps.js',
-				'build/commands/app/index.js',
-				'build/commands/app/create.js',
-				'build/commands/app/rm.js',
-				'build/commands/app/rename.js',
-				'build/commands/app/restart.js',
-			],
-		},
-		{
-			title: 'Authentication',
-			files: [
-				'build/commands/login.js',
-				'build/commands/logout.js',
-				'build/commands/whoami.js',
-			],
-		},
-		{
-			title: 'Device',
-			files: [
-				'build/commands/device/identify.js',
-				'build/commands/device/init.js',
-				'build/commands/device/index.js',
-				'build/commands/device/move.js',
-				'build/commands/device/reboot.js',
-				'build/commands/device/register.js',
-				'build/commands/device/rename.js',
-				'build/commands/device/rm.js',
-				'build/commands/device/restart.js',
-				'build/commands/device/shutdown.js',
-				'build/commands/devices/index.js',
-				'build/commands/devices/supported.js',
-				'build/commands/device/os-update.js',
-				'build/commands/device/public-url.js',
-			],
-		},
-		{
-			title: 'Environment Variables',
-			files: [
-				'build/commands/envs.js',
-				'build/commands/env/add.js',
-				'build/commands/env/rename.js',
-				'build/commands/env/rm.js',
-			],
-		},
-		{
-			title: 'Tags',
-			files: [
-				'build/commands/tags.js',
-				'build/commands/tag/rm.js',
-				'build/commands/tag/set.js',
-			],
-		},
-		{
-			title: 'Help and Version',
-			files: ['help', 'build/commands/version.js'],
-		},
-		{
-			title: 'Keys',
-			files: [
-				'build/commands/keys.js',
-				'build/commands/key/index.js',
-				'build/commands/key/add.js',
-				'build/commands/key/rm.js',
-			],
-		},
-		{
-			title: 'Logs',
-			files: ['build/commands/logs.js'],
-		},
-		{
-			title: 'Network',
-			files: [
-				'build/commands/scan.js',
-				'build/commands/ssh.js',
-				'build/commands/tunnel.js',
-			],
-		},
-		{
-			title: 'Notes',
-			files: ['build/commands/note.js'],
-		},
-		{
-			title: 'OS',
-			files: [
-				'build/commands/os/build-config.js',
-				'build/commands/os/configure.js',
-				'build/commands/os/versions.js',
-				'build/commands/os/download.js',
-				'build/commands/os/initialize.js',
-			],
-		},
-		{
-			title: 'Config',
-			files: [
-				'build/commands/config/generate.js',
-				'build/commands/config/inject.js',
-				'build/commands/config/read.js',
-				'build/commands/config/reconfigure.js',
-				'build/commands/config/write.js',
-			],
-		},
-		{
-			title: 'Preload',
-			files: ['build/commands/preload.js'],
-		},
-		{
-			title: 'Push',
-			files: ['build/commands/push.js'],
-		},
-		{
-			title: 'Settings',
-			files: ['build/commands/settings.js'],
-		},
-		{
-			title: 'Local',
-			files: [
-				'build/commands/local/configure.js',
-				'build/commands/local/flash.js',
-			],
-		},
-		{
-			title: 'Deploy',
-			files: ['build/commands/build.js', 'build/commands/deploy.js'],
-		},
-		{
-			title: 'Platform',
-			files: ['build/commands/join.js', 'build/commands/leave.js'],
-		},
-		{
-			title: 'Utilities',
-			files: ['build/commands/util/available-drives.js'],
-		},
-		{
-			title: 'Support',
-			files: ['build/commands/support.js'],
-		},
-	],
+
+interface Category {
+	title: string;
+	files: string[];
+}
+
+interface Documentation {
+	title: string;
+	introduction: string;
+	categories: Category[];
+}
+
+// Mapping folders names to custom headings in the docs
+const commandHeadings: { [key: string]: string } = {
+	'api-key': 'API Keys',
+	login: 'Authentication',
+	whoami: 'Authentication',
+	logout: 'Authentication',
+	env: 'Environment Variables',
+	help: 'Help and Version',
+	'ssh-key': 'SSH Keys',
+	organization: 'Organizations',
+	os: 'OS',
+	util: 'Utilities',
+	build: 'Deploy',
+	join: 'Platform',
+	leave: 'Platform',
+	app: 'Apps',
+	block: 'Blocks',
+	device: 'Devices',
+	fleet: 'Fleets',
+	release: 'Releases',
+	tag: 'Tags',
 };
 
+// Fetch all available commands
+const allCommandsPaths = new GlobSync('build/commands/**/*.js', {
+	ignore: 'build/commands/internal/**',
+}).found;
+
+// Throw error if any commands found outside of command directories
+const illegalCommandPaths = allCommandsPaths.filter((commandPath: string) =>
+	/^build\/commands\/[^/]+\.js$/.test(commandPath),
+);
+
+if (illegalCommandPaths.length !== 0) {
+	throw new Error(
+		`Found the following commands without a command directory: ${illegalCommandPaths}\n
+		To resolve this error, move the respective commands to their resource directories or create new ones.\n
+		Refer to the automation/capitanodoc/capitanodoc.ts file for more information.`,
+	);
+}
+
+// Docs config template
+const capitanoDoc: Documentation = {
+	title: 'balena CLI Documentation',
+	introduction: '',
+	categories: [],
+};
+
+// Helper function to capitalize each word of directory name
+function formatTitle(dir: string): string {
+	return dir.replace(/(^\w|\s\w)/g, (word) => word.toUpperCase());
+}
+
+// Create a map to track the categories for faster lookup
+const categoriesMap: { [key: string]: Category } = {};
+
+for (const commandPath of allCommandsPaths) {
+	const commandDir = path.basename(path.dirname(commandPath));
+	const heading = commandHeadings[commandDir] || formatTitle(commandDir);
+
+	if (!categoriesMap[heading]) {
+		categoriesMap[heading] = { title: heading, files: [] };
+		capitanoDoc.categories.push(categoriesMap[heading]);
+	}
+
+	categoriesMap[heading].files.push(commandPath);
+}
+
+// Sort Category titles alphabetically
+capitanoDoc.categories = capitanoDoc.categories.sort((a, b) =>
+	a.title.localeCompare(b.title),
+);
+
+// Sort Category file paths alphabetically
+capitanoDoc.categories.forEach((category) => {
+	category.files.sort((a, b) => a.localeCompare(b));
+});
+
 /**
- * Modify and return the `capitanoDoc` object above in order to render the
- * CLI documentation/reference web page at:
- * https://www.balena.io/docs/reference/cli/
+ * Modify and return the `capitanoDoc` object above in order to generate the
+ * CLI documentation at docs/balena-cli.md
  *
  * This function parses the README.md file to extract relevant sections
  * for the documentation web page.
@@ -194,7 +145,7 @@ export async function getCapitanoDoc(): Promise<typeof capitanoDoc> {
 				throw new Error(`Error parsing section title`);
 			}
 			// match[1] has the title, match[2] has the rest
-			return match && match[2];
+			return match?.[2];
 		}),
 		mdParser.getSectionOfTitle('Installation'),
 		mdParser.getSectionOfTitle('Choosing a shell (command prompt/terminal)'),

automation/capitanodoc/doc-types.d.ts

@@ -14,7 +14,7 @@
  * See the License for the specific language governing permissions and
  * limitations under the License.
  */
-import { Command as OclifCommandClass } from '@oclif/command';
+import type { Command as OclifCommandClass } from '@oclif/core';
 
 type OclifCommand = typeof OclifCommandClass;
 
@@ -26,7 +26,7 @@ export interface Document {
 
 export interface Category {
 	title: string;
-	commands: OclifCommand[];
+	commands: Array<OclifCommand & { name: string }>;
 }
 
 export { OclifCommand };

automation/capitanodoc/index.ts

@@ -16,9 +16,8 @@
  */
 import * as path from 'path';
 import { getCapitanoDoc } from './capitanodoc';
-import { Category, Document, OclifCommand } from './doc-types';
+import type { Category, Document, OclifCommand } from './doc-types';
 import * as markdown from './markdown';
-import { stripIndent } from '../../lib/utils/lazy';
 
 /**
  * Generates the markdown document (as a string) for the CLI documentation
@@ -39,7 +38,7 @@ export async function renderMarkdown(): Promise<string> {
 		};
 
 		for (const jsFilename of commandCategory.files) {
-			category.commands.push(...importOclifCommands(jsFilename));
+			category.commands.push(await importOclifCommands(jsFilename));
 		}
 		result.categories.push(category);
 	}
@@ -47,49 +46,23 @@ export async function renderMarkdown(): Promise<string> {
 	return markdown.render(result);
 }
 
-// Help is now managed via a plugin
-// This fake command allows capitanodoc to include help in docs
-class FakeHelpCommand {
-	description = stripIndent`
-		List balena commands, or get detailed help for a specific command.
+async function importOclifCommands(jsFilename: string) {
+	const command = (await import(path.join(process.cwd(), jsFilename)))
+		.default as OclifCommand;
 
-		List balena commands, or get detailed help for a specific command.
-	`;
-
-	examples = [
-		'$ balena help',
-		'$ balena help apps',
-		'$ balena help os download',
-	];
-
-	args = [
-		{
-			name: 'command',
-			description: 'command to show help for',
-		},
-	];
-
-	usage = 'help [command]';
-
-	flags = {
-		verbose: {
-			description: 'show additional commands',
-			char: '-v',
-		},
-	};
-}
-
-function importOclifCommands(jsFilename: string): OclifCommand[] {
-	// TODO: Currently oclif commands with no `usage` overridden will cause
-	//  an error when parsed.  This should be improved so that `usage` does not have
-	//  to be overridden if not necessary.
-
-	const command: OclifCommand =
-		jsFilename === 'help'
-			? ((new FakeHelpCommand() as unknown) as OclifCommand)
-			: (require(path.join(process.cwd(), jsFilename)).default as OclifCommand);
-
-	return [command];
+	return {
+		...command,
+		// build/commands/device/index.js -> device
+		// build/commands/device/list.js -> device list
+		name: jsFilename
+			.split('/')
+			.slice(2)
+			.join(' ')
+			.split('.')
+			.slice(0, 1)
+			.join(' ')
+			.split(' index')[0],
+	} as Category['commands'][0];
 }
 
 /**
@@ -101,8 +74,9 @@ async function printMarkdown() {
 		console.log(await renderMarkdown());
 	} catch (error) {
 		console.error(error);
-		process.exit(1);
+		process.exitCode = 1;
 	}
 }
 
+// eslint-disable-next-line @typescript-eslint/no-floating-promises
 printMarkdown();

automation/capitanodoc/markdown.ts

@@ -14,16 +14,31 @@
  * See the License for the specific language governing permissions and
  * limitations under the License.
  */
-import { flagUsages } from '@oclif/parser';
+import { Parser } from '@oclif/core';
 import * as ent from 'ent';
 import * as _ from 'lodash';
 
-import { getManualSortCompareFunction } from '../../lib/utils/helpers';
-import { capitanoizeOclifUsage } from '../../lib/utils/oclif-utils';
-import { Category, Document, OclifCommand } from './doc-types';
+import { capitanoizeOclifUsage } from '../../src/utils/oclif-utils';
+import type { Category, Document } from './doc-types';
 
-function renderOclifCommand(command: OclifCommand): string[] {
-	const result = [`## ${ent.encode(command.usage || '')}`];
+function renderOclifCommand(command: Category['commands'][0]): string[] {
+	const result = [`## ${ent.encode(command.name || '')}`];
+	if (command.aliases?.length) {
+		result.push('### Aliases');
+		result.push(
+			command.aliases
+				.map(
+					(alias) =>
+						`- \`${alias}\`${command.deprecateAliases ? ' *(deprecated)*' : ''}`,
+				)
+				.join('\n'),
+		);
+		result.push(
+			`\nTo use one of the aliases, replace \`${command.name}\` with the alias.`,
+		);
+	}
+
+	result.push('### Description');
 	const description = (command.description || '')
 		.split('\n')
 		.slice(1) // remove the first line, which oclif uses as help header
@@ -37,8 +52,8 @@ function renderOclifCommand(command: OclifCommand): string[] {
 
 	if (!_.isEmpty(command.args)) {
 		result.push('### Arguments');
-		for (const arg of command.args!) {
-			result.push(`#### ${arg.name.toUpperCase()}`, arg.description || '');
+		for (const [name, arg] of Object.entries(command.args!)) {
+			result.push(`#### ${name.toUpperCase()}`, arg.description || '');
 		}
 	}
 
@@ -49,7 +64,7 @@ function renderOclifCommand(command: OclifCommand): string[] {
 				continue;
 			}
 			flag.name = name;
-			const flagUsage = flagUsages([flag])
+			const flagUsage = Parser.flagUsages([flag])
 				.map(([usage, _description]) => usage)
 				.join()
 				.trim();
@@ -80,7 +95,7 @@ function renderToc(categories: Category[]): string[] {
 		result.push(
 			category.commands
 				.map((command) => {
-					const signature = capitanoizeOclifUsage(command.usage);
+					const signature = capitanoizeOclifUsage(command.name);
 					return `\t- [${ent.encode(signature)}](${getAnchor(signature)})`;
 				})
 				.join('\n'),
@@ -89,33 +104,7 @@ function renderToc(categories: Category[]): string[] {
 	return result;
 }
 
-const manualCategorySorting: { [category: string]: string[] } = {
-	'Environment Variables': ['envs', 'env rm', 'env add', 'env rename'],
-	OS: [
-		'os versions',
-		'os download',
-		'os build config',
-		'os configure',
-		'os initialize',
-	],
-};
-
-function sortCommands(doc: Document): void {
-	for (const category of doc.categories) {
-		if (category.title in manualCategorySorting) {
-			category.commands = category.commands.sort(
-				getManualSortCompareFunction<OclifCommand, string>(
-					manualCategorySorting[category.title],
-					(cmd: OclifCommand, x: string) =>
-						(cmd.usage || '').toString().replace(/\W+/g, ' ').includes(x),
-				),
-			);
-		}
-	}
-}
-
 export function render(doc: Document) {
-	sortCommands(doc);
 	const result = [
 		`# ${doc.title}`,
 		doc.introduction,

automation/capitanodoc/utils.ts

@@ -15,41 +15,9 @@
  * limitations under the License.
  */
 
-import type { OptionDefinition } from 'capitano';
-import * as ent from 'ent';
 import * as fs from 'fs';
 import * as readline from 'readline';
 
-export function getOptionPrefix(signature: string) {
-	if (signature.length > 1) {
-		return '--';
-	} else {
-		return '-';
-	}
-}
-
-export function getOptionSignature(signature: string) {
-	return `${getOptionPrefix(signature)}${signature}`;
-}
-
-export function parseCapitanoOption(option: OptionDefinition): string {
-	let result = getOptionSignature(option.signature);
-
-	if (Array.isArray(option.alias)) {
-		for (const alias of option.alias) {
-			result += `, ${getOptionSignature(alias)}`;
-		}
-	} else if (typeof option.alias === 'string') {
-		result += `, ${getOptionSignature(option.alias)}`;
-	}
-
-	if (option.parameter) {
-		result += ` <${option.parameter}>`;
-	}
-
-	return ent.encode(result);
-}
-
 export class MarkdownFileParser {
 	constructor(public mdFilePath: string) {}
 
@@ -57,7 +25,7 @@ export class MarkdownFileParser {
 	 * Extract the lines of a markdown document section with the given title.
 	 * For example, consider this sample markdown document:
 	 * ```
-	 *     # balenaCLI
+	 *     # balena CLI
 	 *
 	 *     ## Introduction
 	 *     Lorem ipsum dolor sit amet, consectetur adipiscing elit,

automation/check-doc.ts

@@ -15,24 +15,25 @@
  * limitations under the License.
  */
 
-const stripIndent = require('common-tags/lib/stripIndent');
-const _ = require('lodash');
-const { promises: fs } = require('fs');
-const path = require('path');
-const simplegit = require('simple-git/promise');
+// eslint-disable-next-line no-restricted-imports
+import { stripIndent } from 'common-tags';
+import * as _ from 'lodash';
+import { promises as fs } from 'fs';
+import * as path from 'path';
+import { simpleGit } from 'simple-git';
 
 const ROOT = path.normalize(path.join(__dirname, '..'));
 
 /**
- * Compare the timestamp of cli.markdown with the timestamp of staged files,
- * issuing an error if cli.markdown is older.
- * If cli.markdown does not require updating and the developer cannot run
+ * Compare the timestamp of balena-cli.md with the timestamp of staged files,
+ * issuing an error if balena-cli.md is older.
+ * If balena-cli.md does not require updating and the developer cannot run
  * `npm run build` on their laptop, the error message suggests a workaround
  * using `touch`.
  */
 async function checkBuildTimestamps() {
-	const git = simplegit(ROOT);
-	const docFile = path.join(ROOT, 'doc', 'cli.markdown');
+	const git = simpleGit(ROOT);
+	const docFile = path.join(ROOT, 'docs', 'balena-cli.md');
 	const [docStat, gitStatus] = await Promise.all([
 		fs.stat(docFile),
 		git.status(),
@@ -42,8 +43,8 @@ async function checkBuildTimestamps() {
 		...gitStatus.staged,
 		...gitStatus.renamed.map((o) => o.to),
 	])
-		// select only staged files that start with lib/ or typings/
-		.filter((f) => f.match(/^(lib|typings)[/\\]/))
+		// select only staged files that start with src/ or typings/
+		.filter((f) => f.match(/^(src|typings)[/\\]/))
 		.map((f) => path.join(ROOT, f));
 
 	const fStats = await Promise.all(stagedFiles.map((f) => fs.stat(f)));
@@ -81,4 +82,5 @@ async function run() {
 	}
 }
 
+// eslint-disable-next-line @typescript-eslint/no-floating-promises
 run();

automation/check-npm-version.js

@@ -6,6 +6,8 @@
  *
  * We don't `require('semver')` to allow this script to be run as a npm
  * 'preinstall' hook, at which point no dependencies have been installed.
+ *
+ * @param {string} version
  */
 function parseSemver(version) {
 	const match = /v?(\d+)\.(\d+).(\d+)/.exec(version);
@@ -16,9 +18,13 @@ function parseSemver(version) {
 	return [parseInt(major, 10), parseInt(minor, 10), parseInt(patch, 10)];
 }
 
+/**
+ * @param {string} v1
+ * @param {string} v2
+ */
 function semverGte(v1, v2) {
-	let v1Array = parseSemver(v1);
-	let v2Array = parseSemver(v2);
+	const v1Array = parseSemver(v1);
+	const v2Array = parseSemver(v2);
 	for (let i = 0; i < 3; i++) {
 		if (v1Array[i] < v2Array[i]) {
 			return false;
@@ -41,17 +47,25 @@ function checkNpmVersion() {
 		// the reason is that it would unnecessarily prevent end users from
 		// using npm v6.4.1 that ships with Node 8.  (It is OK for the
 		// shrinkwrap file to get damaged if it is not going to be reused.)
-		console.error(`\
--------------------------------------------------------------------------------
+		throw new Error(`\
+-----------------------------------------------------------------------------
 Error: npm version '${npmVersion}' detected. Please upgrade to npm v${requiredVersion} or later
 because of a bug that causes the 'npm-shrinkwrap.json' file to be damaged.
 At this point, however, your 'npm-shrinkwrap.json' file has already been
 damaged. Please revert it to the master branch state with a command such as:
 "git checkout master -- npm-shrinkwrap.json"
 Then re-run "npm install" using npm version ${requiredVersion} or later.
--------------------------------------------------------------------------------`);
-		process.exit(1);
+-----------------------------------------------------------------------------`);
 	}
 }
 
-checkNpmVersion();
+function main() {
+	try {
+		checkNpmVersion();
+	} catch (e) {
+		console.error(e.message || e);
+		process.exitCode = 1;
+	}
+}
+
+main();

automation/deploy-bin.ts

@@ -1,252 +0,0 @@
-/**
- * @license
- * Copyright 2019 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-import * as Bluebird from 'bluebird';
-import * as _ from 'lodash';
-import * as semver from 'semver';
-
-import { finalReleaseAssets, version } from './build-bin';
-
-const { GITHUB_TOKEN } = process.env;
-
-/**
- * Create or update a release in GitHub's releases page, uploading the
- * installer files (standalone zip + native oclif installers).
- */
-export async function createGitHubRelease() {
-	console.log(`Publishing release ${version} to GitHub`);
-	const publishRelease = await import('publish-release');
-	const ghRelease = await Bluebird.fromCallback(
-		publishRelease.bind(null, {
-			token: GITHUB_TOKEN || '',
-			owner: 'balena-io',
-			repo: 'balena-cli',
-			tag: version,
-			name: `balena-CLI ${version}`,
-			reuseRelease: true,
-			assets: finalReleaseAssets[process.platform],
-		}),
-	);
-	console.log(`Release ${version} successful: ${ghRelease.html_url}`);
-}
-
-/**
- * Top-level function to create a CLI release in GitHub's releases page:
- * call zipStandaloneInstaller(), rename the files as we'd like them to
- * display on the releases page, and call createGitHubRelease() to upload
- * the files.
- */
-export async function release() {
-	try {
-		await createGitHubRelease();
-	} catch (err) {
-		console.error('Release failed');
-		console.error(err);
-		process.exit(1);
-	}
-}
-
-/** Return a cached Octokit instance, creating a new one as needed. */
-const getOctokit = _.once(function () {
-	const Octokit = (require('@octokit/rest') as typeof import('@octokit/rest')).Octokit.plugin(
-		(require('@octokit/plugin-throttling') as typeof import('@octokit/plugin-throttling'))
-			.throttling,
-	);
-	return new Octokit({
-		auth: GITHUB_TOKEN,
-		throttle: {
-			onRateLimit: (retryAfter: number, options: any) => {
-				console.warn(
-					`Request quota exhausted for request ${options.method} ${options.url}`,
-				);
-				// retries 3 times
-				if (options.request.retryCount < 3) {
-					console.log(`Retrying after ${retryAfter} seconds!`);
-					return true;
-				}
-			},
-			onAbuseLimit: (_retryAfter: number, options: any) => {
-				// does not retry, only logs a warning
-				console.warn(
-					`Abuse detected for request ${options.method} ${options.url}`,
-				);
-			},
-		},
-	});
-});
-
-/**
- * Extract pagination information (current page, total pages, ordinal number)
- * from the 'link' response header (example below), using the parse-link-header
- * npm package:
- * "link": "<https://api.github.com/repositories/187370853/releases?per_page=2&page=2>; rel=\"next\",
- *          <https://api.github.com/repositories/187370853/releases?per_page=2&page=3>; rel=\"last\""
- *
- * @param response Octokit response object (including response.headers.link)
- * @param perPageDefault Default per_page pagination value if missing in URL
- * @return Object where 'page' is the current page number (1-based),
- * 'pages' is the total number of pages, and 'ordinal' is the ordinal number
- * (3rd, 4th, 5th...) of the first item in the current page.
- */
-function getPageNumbers(
-	response: any,
-	perPageDefault: number,
-): { page: number; pages: number; ordinal: number } {
-	const res = { page: 1, pages: 1, ordinal: 1 };
-	if (!response.headers.link) {
-		return res;
-	}
-	const parse = require('parse-link-header') as typeof import('parse-link-header');
-	const parsed = parse(response.headers.link);
-	if (parsed == null) {
-		throw new Error(`Failed to parse link header: '${response.headers.link}'`);
-	}
-	let perPage = perPageDefault;
-	if (parsed.next) {
-		if (parsed.next.per_page) {
-			perPage = parseInt(parsed.next.per_page, 10);
-		}
-		res.page = parseInt(parsed.next.page, 10) - 1;
-		res.pages = parseInt(parsed.last.page, 10);
-	} else {
-		if (parsed.prev.per_page) {
-			perPage = parseInt(parsed.prev.per_page, 10);
-		}
-		res.page = res.pages = parseInt(parsed.prev.page, 10) + 1;
-	}
-	res.ordinal = (res.page - 1) * perPage + 1;
-	return res;
-}
-
-/**
- * Iterate over every GitHub release in the given owner/repo, check whether
- * its tag_name matches against the affectedVersions semver spec, and if so
- * replace its release description (body) with the given newDescription value.
- * @param owner GitHub repo owner, e.g. 'balena-io' or 'pdcastro'
- * @param repo GitHub repo, e.g. 'balena-cli'
- * @param affectedVersions Semver spec, e.g. '2.6.1 - 7.10.9 || 8.0.0'
- * @param newDescription New release description (body)
- * @param editID Short string present in newDescription, e.g. '[AA101]', that
- * can be searched to determine whether that release has already been updated.
- */
-async function updateGitHubReleaseDescriptions(
-	owner: string,
-	repo: string,
-	affectedVersions: string,
-	newDescription: string,
-	editID: string,
-) {
-	const perPage = 30;
-	const octokit = getOctokit();
-	const options = await octokit.repos.listReleases.endpoint.merge({
-		owner,
-		repo,
-		per_page: perPage,
-	});
-	let errCount = 0;
-	for await (const response of octokit.paginate.iterator(options)) {
-		const { page: thisPage, pages: totalPages, ordinal } = getPageNumbers(
-			response,
-			perPage,
-		);
-		let i = 0;
-		for (const cliRelease of response.data) {
-			const prefix = `[#${ordinal + i++} pg ${thisPage}/${totalPages}]`;
-			if (!cliRelease.id) {
-				console.error(
-					`${prefix} Error: missing release ID (errCount=${++errCount})`,
-				);
-				continue;
-			}
-			const skipMsg = `${prefix} skipping release "${cliRelease.tag_name}" (${cliRelease.id})`;
-			if (cliRelease.draft === true) {
-				console.info(`${skipMsg}: draft release`);
-				continue;
-			} else if (cliRelease.body && cliRelease.body.includes(editID)) {
-				console.info(`${skipMsg}: already updated`);
-				continue;
-			} else if (!semver.satisfies(cliRelease.tag_name, affectedVersions)) {
-				console.info(`${skipMsg}: outside version range`);
-				continue;
-			} else {
-				const updatedRelease = {
-					owner,
-					repo,
-					release_id: cliRelease.id,
-					body: newDescription,
-				};
-				let oldBodyPreview = cliRelease.body;
-				if (oldBodyPreview) {
-					oldBodyPreview = oldBodyPreview.replace(/\s+/g, ' ').trim();
-					if (oldBodyPreview.length > 12) {
-						oldBodyPreview = oldBodyPreview.substring(0, 9) + '...';
-					}
-				}
-				console.info(
-					`${prefix} updating release "${cliRelease.tag_name}" (${cliRelease.id}) old body="${oldBodyPreview}"`,
-				);
-				try {
-					await octokit.repos.updateRelease(updatedRelease);
-				} catch (err) {
-					console.error(
-						`${skipMsg}: Error: ${err.message} (count=${++errCount})`,
-					);
-					continue;
-				}
-			}
-		}
-	}
-}
-
-/**
- * Add a warning description to CLI releases affected by a mixpanel tracking
- * security issue (#1359). This function can be executed "manually" with the
- * following command line:
- *
- * npx ts-node --type-check -P automation/tsconfig.json automation/run.ts fix1359
- */
-export async function updateDescriptionOfReleasesAffectedByIssue1359() {
-	// Run only on Linux/Node10, instead of all platform/Node combinations.
-	// (It could have been any other platform, as long as it only runs once.)
-	if (process.platform !== 'linux' || semver.major(process.version) !== 10) {
-		return;
-	}
-	const owner = 'balena-io';
-	const repo = 'balena-cli';
-	const affectedVersions =
-		'2.6.1 - 7.10.9 || 8.0.0 - 8.1.0 || 9.0.0 - 9.15.6 || 10.0.0 - 10.17.5 || 11.0.0 - 11.7.2';
-	const editID = '[AA100]';
-	let newDescription = `
-		Please note: the "login" command in this release is affected by a
-		security issue fixed in versions
-		[7.10.10](https://github.com/balena-io/balena-cli/releases/tag/v7.10.10),
-		[8.1.1](https://github.com/balena-io/balena-cli/releases/tag/v8.1.1),
-		[9.15.7](https://github.com/balena-io/balena-cli/releases/tag/v9.15.7),
-		[10.17.6](https://github.com/balena-io/balena-cli/releases/tag/v10.17.6),
-		[11.7.3](https://github.com/balena-io/balena-cli/releases/tag/v11.7.3)
-		and later. If you need to use this version, avoid passing your password,
-		keys or tokens as command-line arguments. ${editID}`;
-	// remove line breaks and collapse white space
-	newDescription = newDescription.replace(/\s+/g, ' ').trim();
-	await updateGitHubReleaseDescriptions(
-		owner,
-		repo,
-		affectedVersions,
-		newDescription,
-		editID,
-	);
-}

automation/run.ts

@@ -19,15 +19,11 @@ import * as _ from 'lodash';
 
 import {
 	buildOclifInstaller,
-	buildStandaloneZip,
+	buildStandalone,
 	catchUncommitted,
+	signFilesForNotarization,
 	testShrinkwrap,
 } from './build-bin';
-import {
-	release,
-	updateDescriptionOfReleasesAffectedByIssue1359,
-} from './deploy-bin';
-import { fixPathForMsys, ROOT, runUnderMsys } from './utils';
 
 // DEBUG set to falsy for negative values else is truthy
 process.env.DEBUG = ['0', 'no', 'false', '', undefined].includes(
@@ -36,88 +32,56 @@ process.env.DEBUG = ['0', 'no', 'false', '', undefined].includes(
 	? ''
 	: '1';
 
-function exitWithError(error: Error | string): never {
-	console.error(`Error: ${error}`);
-	process.exit(1);
-}
-
 /**
  * Trivial command-line parser. Check whether the command-line argument is one
  * of the following strings, then call the appropriate functions:
  *     'build:installer'   (to build a native oclif installer)
- * 	   'build:standalone'  (to build a standalone pkg package)
- *     'release'           (to create/update a GitHub release)
- *
- * In the case of 'build:installer', also call runUnderMsys() to switch the
- * shell from cmd.exe to MSYS2 bash.exe.
+ * 	   'build:standalone'  (to build a standalone package)
  *
  * @param args Arguments to parse (default is process.argv.slice(2))
  */
-export async function run(args?: string[]) {
+async function parse(args?: string[]) {
 	args = args || process.argv.slice(2);
-	console.log(`automation/run.ts process.argv=[${process.argv}]\n`);
-	console.log(`automation/run.ts args=[${args}]`);
+	console.error(`[debug] automation/run.ts process.argv=[${process.argv}]`);
+	console.error(`[debug] automation/run.ts args=[${args}]`);
 	if (_.isEmpty(args)) {
-		return exitWithError('missing command-line arguments');
+		throw new Error('missing command-line arguments');
 	}
 	const commands: { [cmd: string]: () => void | Promise<void> } = {
 		'build:installer': buildOclifInstaller,
-		'build:standalone': buildStandaloneZip,
+		'build:standalone': buildStandalone,
+		'sign:binaries': signFilesForNotarization,
 		'catch-uncommitted': catchUncommitted,
 		'test-shrinkwrap': testShrinkwrap,
-		fix1359: updateDescriptionOfReleasesAffectedByIssue1359,
-		release,
 	};
 	for (const arg of args) {
-		if (!commands.hasOwnProperty(arg)) {
-			return exitWithError(`command unknown: ${arg}`);
+		if (!Object.hasOwn(commands, arg)) {
+			throw new Error(`command unknown: ${arg}`);
 		}
 	}
 
-	// If runUnderMsys() is called to re-execute this script under MSYS2,
-	// the current working dir becomes the MSYS2 homedir, so we change back.
-	process.chdir(ROOT);
-
-	// The BUILD_TMP env var is used as an alternative location for oclif
-	// (patched) to copy/extract the CLI files, run npm install and then
-	// create the NSIS executable installer for Windows. This was necessary
-	// to avoid issues with a 260-char limit on Windows paths (possibly a
-	// limitation of some library used by NSIS), as the "current working dir"
-	// provided by balena CI is a rather long path to start with.
-	if (process.platform === 'win32' && !process.env.BUILD_TMP) {
-		const randID = (await import('crypto'))
-			.randomBytes(6)
-			.toString('base64')
-			.replace(/\+/g, '-')
-			.replace(/\//g, '_'); // base64url (RFC 4648)
-		process.env.BUILD_TMP = `C:\\tmp\\${randID}`;
-	}
-
 	for (const arg of args) {
 		try {
-			if (arg === 'build:installer' && process.platform === 'win32') {
-				// ensure running under MSYS2
-				if (!process.env.MSYSTEM) {
-					process.env.MSYS2_PATH_TYPE = 'inherit';
-					await runUnderMsys([
-						fixPathForMsys(process.argv[0]),
-						fixPathForMsys(process.argv[1]),
-						arg,
-					]);
-					continue;
-				}
-				if (process.env.MSYS2_PATH_TYPE !== 'inherit') {
-					throw new Error(
-						'the MSYS2_PATH_TYPE env var must be set to "inherit"',
-					);
-				}
-			}
 			const cmdFunc = commands[arg];
 			await cmdFunc();
 		} catch (err) {
-			return exitWithError(`"${arg}": ${err}`);
+			if (typeof err === 'object') {
+				err.message = `"${arg}": ${err.message}`;
+			}
+			throw err;
 		}
 	}
 }
 
+/** See jsdoc for parse() function above */
+export async function run(args?: string[]) {
+	try {
+		await parse(args);
+	} catch (e) {
+		console.error(e.message ? `Error: ${e.message}` : e);
+		process.exitCode = 1;
+	}
+}
+
+// eslint-disable-next-line @typescript-eslint/no-floating-promises
 run();

automation/test-lock-deduplicated.sh

@@ -10,7 +10,10 @@ npm i
 if ! diff -q npm-shrinkwrap.json npm-shrinkwrap.json.old > /dev/null; then
   rm npm-shrinkwrap.json.old
   echo "** npm-shrinkwrap.json was not deduplicated or not fully committed - FAIL **";
-  echo "** Please run 'npm ci', followed by 'npm dedupe' **";
+  echo "** This can usually be fixed with: **";
+  echo "** git checkout master -- npm-shrinkwrap.json **";
+  echo "** rm -rf node_modules **";
+  echo "** npm install && npm dedupe && npm install **";
   exit 1;
 fi
 

automation/update-module.ts

@@ -3,7 +3,7 @@ import * as semver from 'semver';
 
 const changeTypes = ['major', 'minor', 'patch'] as const;
 
-const validateChangeType = (maybeChangeType: string = 'minor') => {
+const validateChangeType = (maybeChangeType = 'minor') => {
 	maybeChangeType = maybeChangeType.toLowerCase();
 	switch (maybeChangeType) {
 		case 'patch':
@@ -11,8 +11,7 @@ const validateChangeType = (maybeChangeType: string = 'minor') => {
 		case 'major':
 			return maybeChangeType;
 		default:
-			console.error(`Invalid change type: '${maybeChangeType}'`);
-			return process.exit(1);
+			throw new Error(`Invalid change type: '${maybeChangeType}'`);
 	}
 };
 
@@ -37,8 +36,8 @@ const run = async (cmd: string) => {
 			}
 			resolve({ stdout, stderr });
 		});
-		p.stdout.pipe(process.stdout);
-		p.stderr.pipe(process.stderr);
+		p.stdout?.pipe(process.stdout);
+		p.stderr?.pipe(process.stderr);
 	});
 };
 
@@ -58,31 +57,24 @@ const getUpstreams = async () => {
 	const repoYaml = fs.readFileSync(__dirname + '/../repo.yml', 'utf8');
 
 	const yaml = await import('js-yaml');
-	const { upstream } = yaml.safeLoad(repoYaml) as {
+	const { upstream } = yaml.load(repoYaml) as {
 		upstream: Upstream[];
 	};
 
 	return upstream;
 };
 
-const printUsage = (upstreams: Upstream[], upstreamName: string) => {
-	console.error(
-		`
+const getUsage = (upstreams: Upstream[], upstreamName: string) => `
 Usage: npm run update ${upstreamName} $version [$changeType=minor]
 
 Upstream names: ${upstreams.map(({ repo }) => repo).join(', ')}
-`,
-	);
-	return process.exit(1);
-};
+`;
 
-// TODO: Drop the wrapper function once we move to TS 3.8,
-// which will support top level await.
-async function main() {
+async function $main() {
 	const upstreams = await getUpstreams();
 
 	if (process.argv.length < 3) {
-		return printUsage(upstreams, '$upstreamName');
+		throw new Error(getUsage(upstreams, '$upstreamName'));
 	}
 
 	const upstreamName = process.argv[2];
@@ -90,16 +82,15 @@ async function main() {
 	const upstream = upstreams.find((v) => v.repo === upstreamName);
 
 	if (!upstream) {
-		console.error(
+		throw new Error(
 			`Invalid upstream name '${upstreamName}', valid options: ${upstreams
 				.map(({ repo }) => repo)
 				.join(', ')}`,
 		);
-		return process.exit(1);
 	}
 
 	if (process.argv.length < 4) {
-		printUsage(upstreams, upstreamName);
+		throw new Error(getUsage(upstreams, upstreamName));
 	}
 
 	const packageName = upstream.module || upstream.repo;
@@ -108,8 +99,7 @@ async function main() {
 	await run(`npm install ${packageName}@${process.argv[3]}`);
 	const newVersion = await getVersion(packageName);
 	if (newVersion === oldVersion) {
-		console.error(`Already on version '${newVersion}'`);
-		return process.exit(1);
+		throw new Error(`Already on version '${newVersion}'`);
 	}
 
 	console.log(`Updated ${upstreamName} from ${oldVersion} to ${newVersion}`);
@@ -117,11 +107,11 @@ async function main() {
 
 	const changeType = process.argv[4]
 		? // if the caller specified a change type, use that one
-		  validateChangeType(process.argv[4])
+			validateChangeType(process.argv[4])
 		: // use the same change type as in the dependency, but avoid major bumps
-		semverChangeType && semverChangeType !== 'major'
-		? semverChangeType
-		: 'minor';
+			semverChangeType && semverChangeType !== 'major'
+			? semverChangeType
+			: 'minor';
 	console.log(`Using Change-type: ${changeType}`);
 
 	let { stdout: currentBranch } = await run('git rev-parse --abbrev-ref HEAD');
@@ -137,4 +127,13 @@ async function main() {
 	);
 }
 
-main();
+async function main() {
+	try {
+		await $main();
+	} catch (e) {
+		console.error(e);
+		process.exitCode = 1;
+	}
+}
+
+void main();

automation/utils.ts

@@ -16,165 +16,17 @@
  */
 
 import { spawn } from 'child_process';
-import * as _ from 'lodash';
 import * as path from 'path';
-import * as shellEscape from 'shell-escape';
+import * as fs from 'fs';
+import * as whichMod from 'which';
 
-export const MSYS2_BASH = 'C:\\msys64\\usr\\bin\\bash.exe';
 export const ROOT = path.join(__dirname, '..');
 
-/** Tap and buffer this process' stdout and stderr */
-export class StdOutTap {
-	public stdoutBuf: string[] = [];
-	public stderrBuf: string[] = [];
-	public allBuf: string[] = []; // both stdout and stderr
-
-	protected origStdoutWrite: typeof process.stdout.write;
-	protected origStderrWrite: typeof process.stdout.write;
-
-	constructor(protected printDots = false) {}
-
-	tap() {
-		this.origStdoutWrite = process.stdout.write;
-		this.origStderrWrite = process.stderr.write;
-
-		process.stdout.write = (chunk: string, ...args: any[]): boolean => {
-			this.stdoutBuf.push(chunk);
-			this.allBuf.push(chunk);
-			const str = this.printDots ? '.' : chunk;
-			return this.origStdoutWrite.call(process.stdout, str, ...args);
-		};
-
-		process.stderr.write = (chunk: string, ...args: any[]): boolean => {
-			this.stderrBuf.push(chunk);
-			this.allBuf.push(chunk);
-			const str = this.printDots ? '.' : chunk;
-			return this.origStderrWrite.call(process.stderr, str, ...args);
-		};
-	}
-
-	untap() {
-		process.stdout.write = this.origStdoutWrite;
-		process.stderr.write = this.origStderrWrite;
-		if (this.printDots) {
-			console.error('');
-		}
-	}
-}
-
-/**
- * Diff strings by line, using the 'diff' npm package:
- * https://www.npmjs.com/package/diff
- */
-export function diffLines(str1: string, str2: string): string {
-	const { diffTrimmedLines } = require('diff');
-	const diffObjs = diffTrimmedLines(str1, str2);
-	const prefix = (chunk: string, char: string) =>
-		chunk
-			.split('\n')
-			.map((line: string) => `${char} ${line}`)
-			.join('\n');
-	const diffStr = diffObjs
-		.map((part: any) => {
-			return part.added
-				? prefix(part.value, '+')
-				: part.removed
-				? prefix(part.value, '-')
-				: prefix(part.value, ' ');
-		})
-		.join('\n');
-	return diffStr;
-}
-
 export function loadPackageJson() {
-	return require(path.join(ROOT, 'package.json'));
-}
+	const packageJsonPath = path.join(ROOT, 'package.json');
 
-/**
- * Convert e.g. 'C:\myfolder' -> '/C/myfolder' so that the path can be given
- * as argument to "unix tools" like 'tar' under MSYS or MSYS2 on Windows.
- */
-export function fixPathForMsys(p: string): string {
-	return p.replace(/\\/g, '/').replace(/^([a-zA-Z]):/, '/$1');
-}
-
-/**
- * Run the MSYS2 bash.exe shell in a child process (child_process.spawn()).
- * The given argv arguments are escaped using the 'shell-escape' package,
- * so that backslashes in Windows paths, and other bash-special characters,
- * are preserved. If argv is not provided, defaults to process.argv, to the
- * effect that this current (parent) process is re-executed under MSYS2 bash.
- * This is useful to change the default shell from cmd.exe to MSYS2 bash on
- * Windows.
- * @param argv Arguments to be shell-escaped and given to MSYS2 bash.exe.
- */
-export async function runUnderMsys(argv?: string[]) {
-	const newArgv = argv || process.argv;
-	await new Promise((resolve, reject) => {
-		const args = ['-lc', shellEscape(newArgv)];
-		const child = spawn(MSYS2_BASH, args, { stdio: 'inherit' });
-		child.on('close', (code) => {
-			if (code) {
-				console.log(`runUnderMsys: child process exited with code ${code}`);
-				reject(code);
-			} else {
-				resolve();
-			}
-		});
-	});
-}
-
-/**
- * Run the executable at execPath as a child process, and resolve a promise
- * to the executable's stdout output as a string. Reject the promise if
- * anything is printed to stderr, or if the child process exits with a
- * non-zero exit code.
- * @param execPath Executable path
- * @param args Command-line argument for the executable
- */
-export async function getSubprocessStdout(
-	execPath: string,
-	args: string[],
-): Promise<string> {
-	const child = spawn(execPath, args);
-	return new Promise((resolve, reject) => {
-		let stdout = '';
-		child.stdout.on('error', reject);
-		child.stderr.on('error', reject);
-		child.stdout.on('data', (data: Buffer) => {
-			try {
-				stdout = data.toString();
-			} catch (err) {
-				reject(err);
-			}
-		});
-		child.stderr.on('data', (data: Buffer) => {
-			try {
-				const stderr = data.toString();
-
-				// ignore any debug lines, but ensure that we parse
-				// every line provided to the stderr stream
-				const lines = _.filter(
-					stderr.trim().split(/\r?\n/),
-					(line) => !line.startsWith('[debug]'),
-				);
-				if (lines.length > 0) {
-					reject(
-						new Error(`"${execPath}": non-empty stderr "${lines.join('\n')}"`),
-					);
-				}
-			} catch (err) {
-				reject(err);
-			}
-		});
-		child.on('exit', (code: number) => {
-			if (code) {
-				reject(new Error(`"${execPath}": non-zero exit code "${code}"`));
-			} else {
-				resolve(stdout);
-			}
-		});
-	});
+	const packageJson = fs.readFileSync(packageJsonPath, 'utf8');
+	return JSON.parse(packageJson);
 }
 
 /**
@@ -187,7 +39,6 @@ export async function getSubprocessStdout(
  * @returns The program's full path, e.g. 'C:\WINDOWS\System32\OpenSSH\ssh.EXE'
  */
 export async function which(program: string): Promise<string> {
-	const whichMod = await import('which');
 	let programPath: string;
 	try {
 		programPath = await whichMod(program);
@@ -206,7 +57,7 @@ export async function which(program: string): Promise<string> {
  */
 export async function whichSpawn(
 	programName: string,
-	args?: string[],
+	args: string[] = [],
 ): Promise<void> {
 	const program = await which(programName);
 	let error: Error | undefined;
@@ -218,7 +69,7 @@ export async function whichSpawn(
 					.on('error', reject)
 					.on('close', resolve);
 			} catch (err) {
-				reject(err);
+				reject(err as Error);
 			}
 		});
 	} catch (err) {

balena-completion.bash

@@ -1,73 +0,0 @@
-#!/bin/bash
-
-_balena_complete()
-{
-  local cur prev
-
-  # Valid top-level completions
-  commands="app apps build config deploy device devices env envs help key \
-            keys local login logout logs note os preload quickstart settings \
-            scan ssh util version whoami"
-  # Sub-completions
-  app_cmds="create restart rm"
-  config_cmds="generate inject read reconfigure write"
-  device_cmds="identify init move public-url reboot register rename rm \
-               shutdown"
-  device_public_url_cmds="disable enable status"
-  env_cmds="add rename rm"
-  key_cmds="add rm"
-  local_cmds="configure flash"
-  os_cmds="build-config configure download initialize versions"
-  util_cmds="available-drives"
-
-
-  COMPREPLY=()
-  cur=${COMP_WORDS[COMP_CWORD]}
-  prev=${COMP_WORDS[COMP_CWORD-1]}
-
-  if [ $COMP_CWORD -eq 1 ]
-  then
-    COMPREPLY=( $(compgen -W "${commands}" -- $cur) )
-  elif [ $COMP_CWORD -eq 2 ]
-  then
-    case "$prev" in
-      "app")
-        COMPREPLY=( $(compgen -W "$app_cmds" -- $cur) )
-        ;;
-      "config")
-        COMPREPLY=( $(compgen -W "$config_cmds" -- $cur) )
-        ;;
-      "device")
-        COMPREPLY=( $(compgen -W "$device_cmds" -- $cur) )
-        ;;
-      "env")
-        COMPREPLY=( $(compgen -W "$env_cmds" -- $cur) )
-        ;;
-      "key")
-        COMPREPLY=( $(compgen -W "$key_cmds" -- $cur) )
-        ;;
-      "local")
-        COMPREPLY=( $(compgen -W "$local_cmds" -- $cur) )
-        ;;
-      "os")
-        COMPREPLY=( $(compgen -W "$os_cmds" -- $cur) )
-        ;;
-      "util")
-        COMPREPLY=( $(compgen -W "$util_cmds" -- $cur) )
-        ;;
-      "*")
-        ;;
-    esac
-  elif [ $COMP_CWORD -eq 3 ]
-  then
-    case "$prev" in
-      "public-url")
-        COMPREPLY=( $(compgen -W "$device_public_url_cmds" -- $cur) )
-        ;;
-      "*")
-        ;;
-    esac
-  fi
-
-}
-complete -F _balena_complete balena

bin/balena

@@ -1,22 +0,0 @@
-#!/usr/bin/env node
-
-// tslint:disable:no-var-requires
-
-// We boost the threadpool size as ext2fs can deadlock with some
-// operations otherwise, if the pool runs out.
-process.env.UV_THREADPOOL_SIZE = '64';
-
-// Disable oclif registering ts-node
-process.env.OCLIF_TS_NODE = 0;
-
-// Use fast-boot to cache require lookups, speeding up startup
-require('fast-boot2').start({
-	cacheScope: __dirname + '/..',
-	cacheFile: __dirname + '/.fast-boot.json',
-});
-
-// Set the desired es version for downstream modules that support it
-require('@balena/es-version').set('es2018');
-
-// Run the CLI
-require('../build/app').run();

bin/balena

@@ -0,0 +1 @@
+run.js

bin/balena-dev

@@ -1,74 +0,0 @@
-#!/usr/bin/env node
-
-// ****************************************************************************
-// THIS IS FOR DEV PURPOSES ONLY AND WILL NOT BE PART OF THE PUBLISHED PACKAGE
-// Before opening a PR you should build and test your changes using bin/balena
-// ****************************************************************************
-
-// tslint:disable:no-var-requires
-
-// We boost the threadpool size as ext2fs can deadlock with some
-// operations otherwise, if the pool runs out.
-process.env.UV_THREADPOOL_SIZE = '64';
-
-const path = require('path');
-const rootDir = path.join(__dirname, '..');
-
-// Allow balena-dev to work with oclif by temporarily
-// pointing oclif config options to lib/ instead of build/
-modifyOclifPaths();
-// Undo changes on exit
-process.on('exit', function () {
-	modifyOclifPaths(true);
-});
-// Undo changes in case of ctrl-v
-process.on('SIGINT', function () {
-	modifyOclifPaths(true);
-});
-
-// Use fast-boot to cache require lookups, speeding up startup
-require('fast-boot2').start({
-	cacheScope: __dirname + '/..',
-	cacheFile: '.fast-boot.json',
-});
-
-// Set the desired es version for downstream modules that support it
-require('@balena/es-version').set('es2018');
-
-// Note: before ts-node v6.0.0, 'transpile-only' (no type checking) was the
-// default option. We upgraded ts-node and found that adding 'transpile-only'
-// was necessary to avoid a mysterious 'null' error message. On the plus side,
-// it is supposed to run faster. We still benefit from type checking when
-// running 'npm run build'.
-require('ts-node').register({
-	project: path.join(rootDir, 'tsconfig.json'),
-	transpileOnly: true,
-});
-require('../lib/app').run();
-
-// Modify package.json oclif paths from build/ -> lib/, or vice versa
-function modifyOclifPaths(revert) {
-	const fs = require('fs');
-	const packageJsonPath = path.join(rootDir, 'package.json');
-
-	const packageJson = fs.readFileSync(packageJsonPath, 'utf8');
-	const packageObj = JSON.parse(packageJson);
-
-	if (!packageObj.oclif) {
-		return;
-	}
-
-	let oclifSectionText = JSON.stringify(packageObj.oclif);
-	if (!revert) {
-		oclifSectionText = oclifSectionText.replace(/\/build\//g, '/lib/');
-	} else {
-		oclifSectionText = oclifSectionText.replace(/\/lib\//g, '/build/');
-	}
-
-	packageObj.oclif = JSON.parse(oclifSectionText);
-	fs.writeFileSync(
-		packageJsonPath,
-		`${JSON.stringify(packageObj, null, 2)}\n`,
-		'utf8',
-	);
-}

bin/balena-dev

@@ -0,0 +1 @@
+dev.js

bin/dev.cmd

@@ -0,0 +1,3 @@
+@echo off
+
+node "%~dp0\run" %*

bin/dev.js

@@ -0,0 +1,90 @@
+#!/usr/bin/env node
+
+// ****************************************************************************
+// THIS IS FOR DEV PURPOSES ONLY AND WILL NOT BE PART OF THE PUBLISHED PACKAGE
+// Before opening a PR you should build and test your changes using bin/balena
+// ****************************************************************************
+
+// We boost the threadpool size as ext2fs can deadlock with some
+// operations otherwise, if the pool runs out.
+process.env.UV_THREADPOOL_SIZE = '64';
+
+// Note on `fast-boot2`: We do not use `fast-boot2` with `balena-dev` because:
+// * fast-boot2's cacheKiller option is configured to include the timestamps of
+//   the package.json and npm-shrinkwrap.json files, to avoid unexpected CLI
+//   behavior when changes are made to dependencies during development. This is
+//   generally a good thing, however, `balena-dev` (a few lines below) edits
+//   `package.json` to modify oclif paths, and this results in cache
+//   invalidation and a performance hit rather than speedup.
+// * Even if the timestamps are removed from cacheKiller, so that there is no
+//   cache invalidation, fast-boot's speedup is barely noticeable when ts-node
+//   is used, e.g. 1.43s vs 1.4s when running `balena version`.
+// * `fast-boot` causes unexpected behavior when used with `npm link` or
+//   when the `node_modules` folder is manually modified (affecting transitive
+//   dependencies) during development (e.g. bug investigations). A workaround
+//   is to use `balena-dev` without `fast-boot`. See also notes in
+//   `CONTRIBUTING.md`.
+
+const path = require('path');
+const rootDir = path.join(__dirname, '..');
+
+// Allow balena-dev to work with oclif by temporarily
+// pointing oclif config options to src/ instead of build/
+modifyOclifPaths();
+// Undo changes on exit
+process.on('exit', function () {
+	modifyOclifPaths(true);
+});
+// Undo changes in case of ctrl-c
+process.on('SIGINT', function () {
+	modifyOclifPaths(true);
+	// Note process exit here will interfere with commands that do their own SIGINT handling,
+	// but without it commands can not be exited.
+	// So currently using balena-dev does not guarantee proper exit behaviour when using ctrl-c.
+	// Ideally a better solution is needed.
+	process.exit();
+});
+
+// Set the desired es version for downstream modules that support it
+require('@balena/es-version').set('es2018');
+
+// Note: before ts-node v6.0.0, 'transpile-only' (no type checking) was the
+// default option. We upgraded ts-node and found that adding 'transpile-only'
+// was necessary to avoid a mysterious 'null' error message. On the plus side,
+// it is supposed to run faster. We still benefit from type checking when
+// running 'npm run build'.
+require('ts-node').register({
+	project: path.join(rootDir, 'tsconfig.json'),
+	transpileOnly: true,
+});
+void require('../src/app').run(undefined, {
+	dir: __dirname,
+	development: true,
+});
+
+// Modify package.json oclif paths from build/ -> src/, or vice versa
+function modifyOclifPaths(revert) {
+	const fs = require('fs');
+	const packageJsonPath = path.join(rootDir, 'package.json');
+
+	const packageJson = fs.readFileSync(packageJsonPath, 'utf8');
+	const packageObj = JSON.parse(packageJson);
+
+	if (!packageObj.oclif) {
+		return;
+	}
+
+	let oclifSectionText = JSON.stringify(packageObj.oclif);
+	if (!revert) {
+		oclifSectionText = oclifSectionText.replace(/\/build\//g, '/src/');
+	} else {
+		oclifSectionText = oclifSectionText.replace(/\/src\//g, '/build/');
+	}
+
+	packageObj.oclif = JSON.parse(oclifSectionText);
+	fs.writeFileSync(
+		packageJsonPath,
+		`${JSON.stringify(packageObj, null, 2)}\n`,
+		'utf8',
+	);
+}

bin/run.cmd

@@ -0,0 +1,3 @@
+@echo off
+
+node "%~dp0\run" %*

bin/run.js

@@ -0,0 +1,21 @@
+#!/usr/bin/env node
+
+// We boost the threadpool size as ext2fs can deadlock with some
+// operations otherwise, if the pool runs out.
+process.env.UV_THREADPOOL_SIZE = '64';
+
+// Disable oclif registering ts-node
+process.env.OCLIF_TS_NODE = '0';
+
+async function run() {
+	// Use fast-boot to cache require lookups, speeding up startup
+	await require('../build/fast-boot').start();
+
+	// Set the desired es version for downstream modules that support it
+	require('@balena/es-version').set('es2018');
+
+	// Run the CLI
+	await require('../build/app').run(undefined, { dir: __dirname });
+}
+
+void run();

completion/_balena

@@ -0,0 +1,95 @@
+#compdef balena
+#autoload
+
+#GENERATED FILE DON'T MODIFY#
+
+_balena() {
+  typeset -A opt_args
+  local context state line curcontext="$curcontext"
+
+  # Valid top-level completions
+  main_commands=( api-key app block build config deploy device device-type env fleet internal join leave local login logout organization os preload push release settings ssh-key support tag util version whoami )
+  # Sub-completions
+  api_key_cmds=( generate list revoke )
+  app_cmds=( create )
+  block_cmds=( create )
+  config_cmds=( generate inject read reconfigure write )
+  device_type_cmds=( list )
+  device_cmds=( deactivate detect identify init list local-mode logs move note os-update pin public-url purge reboot register rename restart rm shutdown ssh start-service stop-service track-fleet tunnel )
+  env_cmds=( list rename rm set )
+  fleet_cmds=( create list pin purge rename restart rm track-latest )
+  internal_cmds=( osinit )
+  local_cmds=( configure flash )
+  organization_cmds=( list )
+  os_cmds=( build-config configure download initialize versions )
+  release_cmds=( finalize invalidate list validate )
+  ssh_key_cmds=( add list rm )
+  tag_cmds=( list rm set )
+
+
+  _arguments -C \
+    '(- 1 *)--version[show version and exit]' \
+    '(- 1 *)--help[show help options and exit]' \
+    '1:first command:_balena_main_cmds' \
+    '2:second command:_balena_sec_cmds' \
+  && ret=0
+}
+
+(( $+functions[_balena_main_cmds] )) ||
+_balena_main_cmds() {
+  _describe -t main_commands 'command' main_commands "$@" && ret=0
+}
+
+(( $+functions[_balena_sec_cmds] )) ||
+_balena_sec_cmds() {
+  case $line[1] in
+      "api-key")
+        _describe -t api_key_cmds 'api-key_cmd' api_key_cmds "$@" && ret=0
+      ;;
+      "app")
+        _describe -t app_cmds 'app_cmd' app_cmds "$@" && ret=0
+      ;;
+      "block")
+        _describe -t block_cmds 'block_cmd' block_cmds "$@" && ret=0
+      ;;
+      "config")
+        _describe -t config_cmds 'config_cmd' config_cmds "$@" && ret=0
+      ;;
+      "device-type")
+        _describe -t device_type_cmds 'device-type_cmd' device_type_cmds "$@" && ret=0
+      ;;
+      "device")
+        _describe -t device_cmds 'device_cmd' device_cmds "$@" && ret=0
+      ;;
+      "env")
+        _describe -t env_cmds 'env_cmd' env_cmds "$@" && ret=0
+      ;;
+      "fleet")
+        _describe -t fleet_cmds 'fleet_cmd' fleet_cmds "$@" && ret=0
+      ;;
+      "internal")
+        _describe -t internal_cmds 'internal_cmd' internal_cmds "$@" && ret=0
+      ;;
+      "local")
+        _describe -t local_cmds 'local_cmd' local_cmds "$@" && ret=0
+      ;;
+      "organization")
+        _describe -t organization_cmds 'organization_cmd' organization_cmds "$@" && ret=0
+      ;;
+      "os")
+        _describe -t os_cmds 'os_cmd' os_cmds "$@" && ret=0
+      ;;
+      "release")
+        _describe -t release_cmds 'release_cmd' release_cmds "$@" && ret=0
+      ;;
+      "ssh-key")
+        _describe -t ssh_key_cmds 'ssh-key_cmd' ssh_key_cmds "$@" && ret=0
+      ;;
+      "tag")
+        _describe -t tag_cmds 'tag_cmd' tag_cmds "$@" && ret=0
+      ;;
+
+  esac
+}
+
+_balena "$@"

completion/balena-completion.bash

@@ -0,0 +1,92 @@
+#!/bin/bash
+
+#GENERATED FILE DON'T MODIFY#
+
+_balena_complete()
+{
+  local cur prev
+
+  # Valid top-level completions
+  main_commands="api-key app block build config deploy device device-type env fleet internal join leave local login logout organization os preload push release settings ssh-key support tag util version whoami"
+  # Sub-completions
+  api_key_cmds="generate list revoke"
+  app_cmds="create"
+  block_cmds="create"
+  config_cmds="generate inject read reconfigure write"
+  device_type_cmds="list"
+  device_cmds="deactivate detect identify init list local-mode logs move note os-update pin public-url purge reboot register rename restart rm shutdown ssh start-service stop-service track-fleet tunnel"
+  env_cmds="list rename rm set"
+  fleet_cmds="create list pin purge rename restart rm track-latest"
+  internal_cmds="osinit"
+  local_cmds="configure flash"
+  organization_cmds="list"
+  os_cmds="build-config configure download initialize versions"
+  release_cmds="finalize invalidate list validate"
+  ssh_key_cmds="add list rm"
+  tag_cmds="list rm set"
+
+
+
+  COMPREPLY=()
+  cur=${COMP_WORDS[COMP_CWORD]}
+  prev=${COMP_WORDS[COMP_CWORD-1]}
+
+  if [ $COMP_CWORD -eq 1 ]
+  then
+    COMPREPLY=( $(compgen -W "${main_commands}" -- $cur) )
+  elif [ $COMP_CWORD -eq 2 ]
+  then
+    case "$prev" in
+      api-key)
+        COMPREPLY=( $(compgen -W "$api_key_cmds" -- $cur) )
+        ;;
+      app)
+        COMPREPLY=( $(compgen -W "$app_cmds" -- $cur) )
+        ;;
+      block)
+        COMPREPLY=( $(compgen -W "$block_cmds" -- $cur) )
+        ;;
+      config)
+        COMPREPLY=( $(compgen -W "$config_cmds" -- $cur) )
+        ;;
+      device-type)
+        COMPREPLY=( $(compgen -W "$device_type_cmds" -- $cur) )
+        ;;
+      device)
+        COMPREPLY=( $(compgen -W "$device_cmds" -- $cur) )
+        ;;
+      env)
+        COMPREPLY=( $(compgen -W "$env_cmds" -- $cur) )
+        ;;
+      fleet)
+        COMPREPLY=( $(compgen -W "$fleet_cmds" -- $cur) )
+        ;;
+      internal)
+        COMPREPLY=( $(compgen -W "$internal_cmds" -- $cur) )
+        ;;
+      local)
+        COMPREPLY=( $(compgen -W "$local_cmds" -- $cur) )
+        ;;
+      organization)
+        COMPREPLY=( $(compgen -W "$organization_cmds" -- $cur) )
+        ;;
+      os)
+        COMPREPLY=( $(compgen -W "$os_cmds" -- $cur) )
+        ;;
+      release)
+        COMPREPLY=( $(compgen -W "$release_cmds" -- $cur) )
+        ;;
+      ssh-key)
+        COMPREPLY=( $(compgen -W "$ssh_key_cmds" -- $cur) )
+        ;;
+      tag)
+        COMPREPLY=( $(compgen -W "$tag_cmds" -- $cur) )
+        ;;
+
+      "*")
+        ;;
+    esac
+  fi
+
+}
+complete -F _balena_complete balena

completion/generate-completion.js

@@ -0,0 +1,175 @@
+/**
+ * @license
+ * Copyright 2021 Balena Ltd.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+const path = require('path');
+const rootDir = path.join(__dirname, '..');
+const fs = require('fs');
+const manifestFile = 'oclif.manifest.json';
+
+commandsFilePath = path.join(rootDir, manifestFile);
+if (fs.existsSync(commandsFilePath)) {
+	console.log('Generating shell auto completion files...');
+} else {
+	console.error(`generate-completion.js: Could not find "${manifestFile}"`);
+	process.exitCode = 1;
+	return;
+}
+
+const commandsJson = JSON.parse(fs.readFileSync(commandsFilePath, 'utf8'));
+
+const mainCommands = [];
+const additionalCommands = [];
+for (const key of Object.keys(commandsJson.commands).sort()) {
+	const cmd = key.split(':');
+	if (cmd.length > 1) {
+		additionalCommands.push(cmd);
+		if (!mainCommands.includes(cmd[0])) {
+			mainCommands.push(cmd[0]);
+		}
+	} else {
+		mainCommands.push(cmd[0]);
+	}
+}
+const mainCommandsStr = mainCommands.join(' ');
+
+// GENERATE BASH COMPLETION FILE
+bashFilePathIn = path.join(__dirname, '/templates/bash.template');
+bashFilePathOut = path.join(__dirname, 'balena-completion.bash');
+
+try {
+	fs.unlinkSync(bashFilePathOut);
+} catch (error) {
+	process.exitCode = 1;
+	return console.error(error);
+}
+
+fs.readFile(bashFilePathIn, 'utf8', function (err, data) {
+	if (err) {
+		process.exitCode = 1;
+		return console.error(err);
+	}
+
+	data = data.replace(
+		'#TEMPLATE FILE FOR BASH COMPLETION#',
+		"#GENERATED FILE DON'T MODIFY#",
+	);
+
+	data = data.replace(
+		/\$main_commands\$/g,
+		'main_commands="' + mainCommandsStr + '"',
+	);
+	let subCommands = [];
+	let prevElement = additionalCommands[0][0];
+	additionalCommands.forEach(function (element) {
+		if (element[0] === prevElement) {
+			subCommands.push(element[1]);
+		} else {
+			const prevElement2 = prevElement.replace(/-/g, '_') + '_cmds';
+			data = data.replace(
+				/\$sub_cmds\$/g,
+				'  ' + prevElement2 + '="' + subCommands.join(' ') + '"\n$sub_cmds$',
+			);
+			data = data.replace(
+				/\$sub_cmds_prev\$/g,
+				'      ' +
+					prevElement +
+					')\n        COMPREPLY=( $(compgen -W "$' +
+					prevElement2 +
+					'" -- $cur) )\n        ;;\n$sub_cmds_prev$',
+			);
+			prevElement = element[0];
+			subCommands = [];
+			subCommands.push(element[1]);
+		}
+	});
+	// cleanup placeholders
+	data = data.replace(/\$sub_cmds\$/g, '');
+	data = data.replace(/\$sub_cmds_prev\$/g, '');
+
+	fs.writeFile(bashFilePathOut, data, 'utf8', function (error) {
+		if (error) {
+			process.exitCode = 1;
+			return console.error(error);
+		}
+	});
+});
+
+// GENERATE ZSH COMPLETION FILE
+zshFilePathIn = path.join(__dirname, '/templates/zsh.template');
+zshFilePathOut = path.join(__dirname, '_balena');
+
+try {
+	fs.unlinkSync(zshFilePathOut);
+} catch (error) {
+	process.exitCode = 1;
+	return console.error(error);
+}
+
+fs.readFile(zshFilePathIn, 'utf8', function (err, data) {
+	if (err) {
+		process.exitCode = 1;
+		return console.error(err);
+	}
+
+	data = data.replace(
+		'#TEMPLATE FILE FOR ZSH COMPLETION#',
+		"#GENERATED FILE DON'T MODIFY#",
+	);
+
+	data = data.replace(
+		/\$main_commands\$/g,
+		'main_commands=( ' + mainCommandsStr + ' )',
+	);
+	let subCommands = [];
+	let prevElement = additionalCommands[0][0];
+	additionalCommands.forEach(function (element) {
+		if (element[0] === prevElement) {
+			subCommands.push(element[1]);
+		} else {
+			const prevElement2 = prevElement.replace(/-/g, '_') + '_cmds';
+			data = data.replace(
+				/\$sub_cmds\$/g,
+				'  ' + prevElement2 + '=( ' + subCommands.join(' ') + ' )\n$sub_cmds$',
+			);
+			data = data.replace(
+				/\$sub_cmds_prev\$/g,
+				'      "' +
+					prevElement +
+					'")\n        _describe -t ' +
+					prevElement2 +
+					" '" +
+					prevElement +
+					"_cmd' " +
+					prevElement2 +
+					' "$@" && ret=0\n      ;;\n$sub_cmds_prev$',
+			);
+			prevElement = element[0];
+			subCommands = [];
+			subCommands.push(element[1]);
+		}
+	});
+	// cleanup placeholders
+	data = data.replace(/\$sub_cmds\$/g, '');
+	data = data.replace(/\$sub_cmds_prev\$/g, '');
+
+	fs.writeFile(zshFilePathOut, data, 'utf8', function (error) {
+		if (error) {
+			process.exitCode = 1;
+			return console.error(error);
+		}
+	});
+});

completion/templates/bash.template

@@ -0,0 +1,32 @@
+#!/bin/bash
+
+#TEMPLATE FILE FOR BASH COMPLETION#
+
+_balena_complete()
+{
+  local cur prev
+
+  # Valid top-level completions
+  $main_commands$
+  # Sub-completions
+$sub_cmds$
+
+
+  COMPREPLY=()
+  cur=${COMP_WORDS[COMP_CWORD]}
+  prev=${COMP_WORDS[COMP_CWORD-1]}
+
+  if [ $COMP_CWORD -eq 1 ]
+  then
+    COMPREPLY=( $(compgen -W "${main_commands}" -- $cur) )
+  elif [ $COMP_CWORD -eq 2 ]
+  then
+    case "$prev" in
+$sub_cmds_prev$
+      "*")
+        ;;
+    esac
+  fi
+
+}
+complete -F _balena_complete balena

completion/templates/zsh.template

@@ -0,0 +1,35 @@
+#compdef balena
+#autoload
+
+#TEMPLATE FILE FOR ZSH COMPLETION#
+
+_balena() {
+  typeset -A opt_args
+  local context state line curcontext="$curcontext"
+
+  # Valid top-level completions
+  $main_commands$
+  # Sub-completions
+$sub_cmds$
+
+  _arguments -C \
+    '(- 1 *)--version[show version and exit]' \
+    '(- 1 *)--help[show help options and exit]' \
+    '1:first command:_balena_main_cmds' \
+    '2:second command:_balena_sec_cmds' \
+  && ret=0
+}
+
+(( $+functions[_balena_main_cmds] )) ||
+_balena_main_cmds() {
+  _describe -t main_commands 'command' main_commands "$@" && ret=0
+}
+
+(( $+functions[_balena_sec_cmds] )) ||
+_balena_sec_cmds() {
+  case $line[1] in
+$sub_cmds_prev$
+  esac
+}
+
+_balena "$@"

doc/automated-init.md

@@ -1,112 +0,0 @@
-# Provisioning balena devices in automated (non-interactive) mode
-
-This document describes how to run the `device init` command in non-interactive mode.
-
-It requires collecting some preliminary information _once_.
-
-The final command to provision the device looks like this:
-
-```bash
-balena device init --app APP_ID --os-version OS_VERSION --drive DRIVE --config CONFIG_FILE --yes
-
-```
-
-You can run this command as many times as you need, putting the new medium (SD card / USB stick) each time.
-
-But before you can run it you need to collect the parameters and build the configuration file. Keep reading to figure out how to do it.
-
-
-## Collect all the required parameters.
-
-1. `DEVICE_TYPE`. Run
-	```bash
-	balena devices supported
-	```
-	and find the _slug_ for your target device type, like _raspberrypi3_.
-
-1. `APP_ID`. Create an application (`balena app create APP_NAME --type DEVICE_TYPE`) or find an existing one (`balena apps`) and notice its ID.
-
-1. `OS_VERSION`. Run
-	```bash
-	balena os versions DEVICE_TYPE
-	```
-	and pick the version that you need, like _v2.0.6+rev1.prod_.
-	_Note_ that even though we support _semver ranges_ it's recommended to use the exact version when doing the automated provisioning as it
-	guarantees full compatibility between the steps.
-
-1. `DRIVE`. Plug in your target medium (SD card or the USB stick, depending on your device type) and run
-	```bash
-	balena util available-drives
-	```
-	and get the drive name, like _/dev/sdb_ or _/dev/mmcblk0_.
-	The balena CLI will not display the system drives to protect you,
-	but still please check very carefully that you've picked the correct drive as it will be erased during the provisioning process.
-
-Now we have all the parameters -- time to build the config file.
-
-## Build the config file
-
-Interactive device provisioning process often includes collecting some extra device configuration, like the networking mode and wifi credentials.
-
-To skip this interactive step we need to buid this configuration once and save it to the JSON file for later reuse.
-
-Let's say we will place it into the `CONFIG_FILE` path, like _./balena-os/raspberrypi3-config.json_.
-
-We also need to put the OS image somewhere, let's call this path `OS_IMAGE_PATH`, it can be something like _./balena-os/raspberrypi3-v2.0.6+rev1.prod.img_.
-
-1. First we need to download the OS image once. That's needed for building the config, and will speedup the subsequent operations as the downloaded OS image is placed into the local cache.
-
-	Run:
-	```bash
-	balena os download DEVICE_TYPE --output OS_IMAGE_PATH --version OS_VERSION
-	```
-
-1. Now we're ready to build the config:
-
-	```bash
-	balena os build-config OS_IMAGE_PATH DEVICE_TYPE --output CONFIG_FILE
-	```
-
-	This will run you through the interactive configuration wizard and in the end save the generated config as `CONFIG_FILE`. You can then verify it's not empty:
-
-	```bash
-	cat CONFIG_FILE
-	```
-
-## Done
-
-Now you're ready to run the command in the beginning of this guide.
-
-Please note again that all of these steps only need to be done once (unless you need to change something), and once all the parameters are collected the main init command can be run unchanged.
-
-But there are still some nuances to cover, please read below. 
-
-## Nuances
-
-### `sudo` password on *nix systems
-
-In order to write the image to the raw device we need the root permissions, this is unavoidable.
-
-To improve the security we only run the minimal subcommand with `sudo`.
-
-This means that with the default setup you're interrupted closer to the end of the device init process to enter your sudo password for this subcommand to work.
-
-There are several ways to eliminate it and make the process fully non-interactive.
-
-#### Option 1: make passwordless sudo.
-
-Obviously you shouldn't do that if the machine you're working on has access to any sensitive resources or information.
-
-But if you're using a machine dedicated to balena provisioning this can be fine, and also the simplest thing to do.
-
-#### Option 2: `NOPASSWD` directive
-
-You can configure the `balena` CLI command to be sudo-runnable without the password. Check [this post](https://askubuntu.com/questions/159007/how-do-i-run-specific-sudo-commands-without-a-password) for an example.
-
-### Extra initialization config
-
-As of June 2017 all the supported devices should not require any other interactive configuration.
-
-But by the design of our system it is _possible_ (though it doesn't look very likely it's going to happen any time soon) that some extra initialization options may be requested for the specific device types.
-
-If that is the case please raise the issue in the balena CLI repository and the maintainers will add the necessary options to build the similar JSON config for this step.

eslint.config.js

@@ -0,0 +1,32 @@
+const { FlatCompat } = require('@eslint/eslintrc');
+
+const compat = new FlatCompat({
+	baseDirectory: __dirname,
+});
+module.exports = [
+	...require('@balena/lint/config/eslint.config'),
+	...compat.config({
+		parserOptions: {
+			project: 'tsconfig.dev.json',
+		},
+		ignorePatterns: ['**/generate-completion.js', '**/bin/**/*'],
+		rules: {
+			ignoreDefinitionFiles: 0,
+			'@typescript-eslint/no-non-null-assertion': 'off',
+			'@typescript-eslint/no-shadow': 'off',
+			'@typescript-eslint/no-var-requires': 'off',
+			'@typescript-eslint/no-require-imports': 'off',
+			'@typescript-eslint/no-unnecessary-type-assertion': 'off',
+			'@typescript-eslint/prefer-nullish-coalescing': 'warn',
+
+			'no-restricted-imports': ['error', {
+				paths: ['resin-cli-visuals', 'chalk', 'common-tags', 'resin-cli-form'],
+			}],
+
+			'@typescript-eslint/no-unused-vars': ['error', {
+				argsIgnorePattern: '^_',
+				caughtErrorsIgnorePattern: '^_',
+			}],
+		},
+	}),
+];

gulpfile.js

@@ -1,15 +0,0 @@
-const gulp = require('gulp');
-const inlinesource = require('gulp-inline-source');
-
-const OPTIONS = {
-	files: {
-		pages: 'lib/auth/pages/*.ejs',
-	},
-};
-
-gulp.task('pages', () =>
-	gulp
-		.src(OPTIONS.files.pages)
-		.pipe(inlinesource())
-		.pipe(gulp.dest('build/auth/pages')),
-);

lib/app.ts

@@ -1,152 +0,0 @@
-/**
- * @license
- * Copyright 2019-2020 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-import * as packageJSON from '../package.json';
-import { CliSettings } from './utils/bootstrap';
-import { onceAsync, stripIndent } from './utils/lazy';
-
-/**
- * Sentry.io setup
- * @see https://docs.sentry.io/error-reporting/quickstart/?platform=node
- */
-export const setupSentry = onceAsync(async () => {
-	const config = await import('./config');
-	const Sentry = await import('@sentry/node');
-	Sentry.init({
-		dsn: config.sentryDsn,
-		release: packageJSON.version,
-	});
-	Sentry.configureScope((scope) => {
-		scope.setExtras({
-			is_pkg: !!(process as any).pkg,
-			node_version: process.version,
-			platform: process.platform,
-		});
-	});
-	return Sentry.getCurrentHub();
-});
-
-async function checkNodeVersion() {
-	const validNodeVersions = packageJSON.engines.node;
-	if (!(await import('semver')).satisfies(process.version, validNodeVersions)) {
-		console.warn(stripIndent`
-			------------------------------------------------------------------------------
-			Warning: Node version "${process.version}" does not match required versions "${validNodeVersions}".
-			This may cause unexpected behavior. To upgrade Node, visit:
-			https://nodejs.org/en/download/
-			------------------------------------------------------------------------------
-			`);
-	}
-}
-
-/** Setup balena-sdk options that are shared with imported packages */
-function setupBalenaSdkSharedOptions(settings: CliSettings) {
-	const BalenaSdk = require('balena-sdk') as typeof import('balena-sdk');
-	BalenaSdk.setSharedOptions({
-		apiUrl: settings.get<string>('apiUrl'),
-		dataDirectory: settings.get<string>('dataDirectory'),
-	});
-}
-
-/**
- * Addresses the console warning:
- * (node:49500) MaxListenersExceededWarning: Possible EventEmitter memory
- * leak detected. 11 error listeners added. Use emitter.setMaxListeners() to
- * increase limit
- */
-export function setMaxListeners(maxListeners: number) {
-	require('events').EventEmitter.defaultMaxListeners = maxListeners;
-}
-
-/** Selected CLI initialization steps */
-async function init() {
-	if (process.env.BALENARC_NO_SENTRY) {
-		console.error(`WARN: disabling Sentry.io error reporting`);
-	} else {
-		await setupSentry();
-	}
-	checkNodeVersion();
-
-	const settings = new CliSettings();
-
-	// Proxy setup should be done early on, before loading balena-sdk
-	await (await import('./utils/proxy')).setupGlobalHttpProxy(settings);
-
-	setupBalenaSdkSharedOptions(settings);
-
-	// check for CLI updates once a day
-	(await import('./utils/update')).notify();
-}
-
-/** Execute the oclif parser and the CLI command. */
-async function oclifRun(
-	command: string[],
-	options: import('./preparser').AppOptions,
-) {
-	const { CustomMain } = await import('./utils/oclif-utils');
-	const runPromise = CustomMain.run(command).then(
-		() => {
-			if (!options.noFlush) {
-				return require('@oclif/command/flush');
-			}
-		},
-		(error) => {
-			// oclif sometimes exits with ExitError code 0 (not an error)
-			// (Avoid `error instanceof ExitError` here for the reasons explained
-			// in the CONTRIBUTING.md file regarding the `instanceof` operator.)
-			if (error.oclif?.exit === 0) {
-				return;
-			} else {
-				throw error;
-			}
-		},
-	);
-	const { trackPromise } = await import('./hooks/prerun/track');
-	await Promise.all([trackPromise, runPromise]);
-}
-
-/** CLI entrypoint. Called by the `bin/balena` and `bin/balena-dev` scripts. */
-export async function run(
-	cliArgs = process.argv,
-	options: import('./preparser').AppOptions = {},
-) {
-	try {
-		const { normalizeEnvVars, pkgExec } = await import('./utils/bootstrap');
-		normalizeEnvVars();
-
-		// The 'pkgExec' special/internal command provides a Node.js interpreter
-		// for use of the standalone zip package. See pkgExec function.
-		if (cliArgs.length > 3 && cliArgs[2] === 'pkgExec') {
-			return pkgExec(cliArgs[3], cliArgs.slice(4));
-		}
-
-		await init();
-
-		const { preparseArgs, checkDeletedCommand } = await import('./preparser');
-
-		// Look for commands that have been removed and if so, exit with a notice
-		checkDeletedCommand(cliArgs.slice(2));
-
-		const args = await preparseArgs(cliArgs);
-		await oclifRun(args, options);
-	} catch (err) {
-		await (await import('./errors')).handleError(err);
-	} finally {
-		// Windows fix: reading from stdin prevents the process from exiting
-		process.stdin.pause();
-	}
-}

lib/command.ts

@@ -1,132 +0,0 @@
-/**
- * @license
- * Copyright 2020 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-import Command from '@oclif/command';
-import { InsufficientPrivilegesError } from './errors';
-
-export default abstract class BalenaCommand extends Command {
-	/**
-	 * When set to true, command will be listed in `help`,
-	 * otherwise listed in `help --verbose` with secondary commands.
-	 */
-	public static primary = false;
-
-	/**
-	 * Require elevated privileges to run.
-	 * When set to true, command will exit with an error
-	 * if executed without root on Mac/Linux
-	 * or if executed by non-Administrator on Windows.
-	 */
-	public static root = false;
-
-	/**
-	 * Require authentication to run.
-	 * When set to true, command will exit with an error
-	 * if user is not already logged in.
-	 */
-	public static authenticated = false;
-
-	/**
-	 * Accept piped input.
-	 * When set to true, command will read from stdin during init
-	 * and make contents available on member `stdin`.
-	 */
-	public static readStdin = false;
-
-	public stdin: string;
-
-	/**
-	 * Throw InsufficientPrivilegesError if not root on Mac/Linux
-	 * or non-Administrator on Windows.
-	 *
-	 * Called automatically if `root=true`.
-	 * Can be called explicitly by command implementation, if e.g.:
-	 *  - check should only be done conditionally
-	 *  - other code needs to execute before check
-	 */
-	protected static async checkElevatedPrivileges() {
-		const isElevated = await (await import('is-elevated'))();
-		if (!isElevated) {
-			throw new InsufficientPrivilegesError(
-				'You need root/admin privileges to run this command',
-			);
-		}
-	}
-
-	/**
-	 * Throw NotLoggedInError if not logged in.
-	 *
-	 * Called automatically if `authenticated=true`.
-	 * Can be called explicitly by command implementation, if e.g.:
-	 *  - check should only be done conditionally
-	 *  - other code needs to execute before check
-	 *
-	 *  Note, currently public to allow use outside of derived commands
-	 *  (as some command implementations require this. Can be made protected
-	 *  if this changes).
-	 *
-	 * @throws {NotLoggedInError}
-	 */
-	public static async checkLoggedIn() {
-		await (await import('./utils/patterns')).checkLoggedIn();
-	}
-
-	/**
-	 * Throw NotLoggedInError if not logged in when condition true.
-	 *
-	 * @param {boolean} doCheck - will check if true.
-	 * @throws {NotLoggedInError}
-	 */
-	public static async checkLoggedInIf(doCheck: boolean) {
-		if (doCheck) {
-			await this.checkLoggedIn();
-		}
-	}
-
-	/**
-	 * Read stdin contents and make available to command.
-	 *
-	 * This approach could be improved in the future to automatically set argument
-	 * values from stdin based in configuration, minimising command implementation.
-	 */
-	protected async getStdin() {
-		this.stdin = await (await import('get-stdin'))();
-	}
-
-	/**
-	 * Get a logger instance.
-	 */
-	protected static async getLogger() {
-		return (await import('./utils/logger')).getLogger();
-	}
-
-	protected async init() {
-		const ctr = this.constructor as typeof BalenaCommand;
-
-		if (ctr.root) {
-			await BalenaCommand.checkElevatedPrivileges();
-		}
-
-		if (ctr.authenticated) {
-			await BalenaCommand.checkLoggedIn();
-		}
-
-		if (ctr.readStdin) {
-			await this.getStdin();
-		}
-	}
-}

lib/commands/api-key/generate.ts

@@ -1,85 +0,0 @@
-/**
- * @license
- * Copyright 2016-2020 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-import { flags } from '@oclif/command';
-import Command from '../../command';
-import { ExpectedError } from '../../errors';
-import * as cf from '../../utils/common-flags';
-import { getBalenaSdk, stripIndent } from '../../utils/lazy';
-
-interface FlagsDef {
-	help: void;
-}
-
-interface ArgsDef {
-	name: string;
-}
-
-export default class GenerateCmd extends Command {
-	public static description = stripIndent`
-		Generate a new balenaCloud API key.
-
-		Generate a new balenaCloud API key for the current user, with the given
-		name. The key will be logged to the console.
-
-		This key can be used to log into the CLI using 'balena login --token <key>',
-		or to authenticate requests to the API with an 'Authorization: Bearer <key>' header.
-`;
-	public static examples = ['$ balena api-key generate "Jenkins Key"'];
-
-	public static args = [
-		{
-			name: 'name',
-			description: 'the API key name',
-			required: true,
-		},
-	];
-
-	public static usage = 'api-key generate <name>';
-
-	public static flags: flags.Input<FlagsDef> = {
-		help: cf.help,
-	};
-
-	public static authenticated = true;
-
-	public async run() {
-		const { args: params } = this.parse<FlagsDef, ArgsDef>(GenerateCmd);
-
-		let key;
-		try {
-			key = await getBalenaSdk().models.apiKey.create(params.name);
-		} catch (e) {
-			if (e.name === 'BalenaNotLoggedIn') {
-				throw new ExpectedError(stripIndent`
-					This command cannot be run when logged in with an API key.
-					Please login again with 'balena login' and select an alternative method.
-				`);
-			} else {
-				throw e;
-			}
-		}
-
-		console.log(stripIndent`
-			Registered api key '${params.name}':
-
-			${key}
-
-			This key will not be shown again, so please save it now.
-		`);
-	}
-}

lib/commands/app/create.ts

@@ -1,104 +0,0 @@
-/**
- * @license
- * Copyright 2016-2020 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-import { flags } from '@oclif/command';
-import Command from '../../command';
-import { ExpectedError } from '../../errors';
-import * as cf from '../../utils/common-flags';
-import { getBalenaSdk, stripIndent } from '../../utils/lazy';
-import type * as BalenaSDK from 'balena-sdk';
-
-interface FlagsDef {
-	type?: string; // application device type
-	help: void;
-}
-
-interface ArgsDef {
-	name: string;
-}
-
-export default class AppCreateCmd extends Command {
-	public static description = stripIndent`
-		Create an application.
-
-		Create a new balena application.
-
-		You can specify the application device type with the \`--type\` option.
-		Otherwise, an interactive dropdown will be shown for you to select from.
-
-		You can see a list of supported device types with:
-
-		$ balena devices supported
-`;
-	public static examples = [
-		'$ balena app create MyApp',
-		'$ balena app create MyApp --type raspberry-pi',
-	];
-
-	public static args = [
-		{
-			name: 'name',
-			description: 'application name',
-			required: true,
-		},
-	];
-
-	public static usage = 'app create <name>';
-
-	public static flags: flags.Input<FlagsDef> = {
-		type: flags.string({
-			char: 't',
-			description:
-				'application device type (Check available types with `balena devices supported`)',
-		}),
-		help: cf.help,
-	};
-
-	public static authenticated = true;
-
-	public async run() {
-		const { args: params, flags: options } = this.parse<FlagsDef, ArgsDef>(
-			AppCreateCmd,
-		);
-
-		const balena = getBalenaSdk();
-
-		// Create application
-		const deviceType =
-			options.type ||
-			(await (await import('../../utils/patterns')).selectDeviceType());
-		let application: BalenaSDK.Application;
-		try {
-			application = await balena.models.application.create({
-				name: params.name,
-				deviceType,
-				organization: (await balena.auth.whoami())!,
-			});
-		} catch (err) {
-			// BalenaRequestError: Request error: Unique key constraint violated
-			if ((err.message || '').toLowerCase().includes('unique')) {
-				throw new ExpectedError(
-					`Error: application "${params.name}" already exists`,
-				);
-			}
-			throw err;
-		}
-		console.info(
-			`Application created: ${application.slug} (${deviceType}, id ${application.id})`,
-		);
-	}
-}

lib/commands/app/index.ts

@@ -1,87 +0,0 @@
-/**
- * @license
- * Copyright 2016-2020 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-import { flags } from '@oclif/command';
-import Command from '../../command';
-import * as cf from '../../utils/common-flags';
-import { getBalenaSdk, getVisuals, stripIndent } from '../../utils/lazy';
-import { tryAsInteger } from '../../utils/validation';
-import { Release } from 'balena-sdk';
-
-interface FlagsDef {
-	help: void;
-}
-
-interface ArgsDef {
-	name: string;
-}
-
-export default class AppCmd extends Command {
-	public static description = stripIndent`
-		Display information about a single application.
-
-		Display detailed information about a single balena application.
-`;
-	public static examples = ['$ balena app MyApp'];
-
-	public static args = [
-		{
-			name: 'name',
-			description: 'application name or numeric ID',
-			required: true,
-		},
-	];
-
-	public static usage = 'app <name>';
-
-	public static flags: flags.Input<FlagsDef> = {
-		help: cf.help,
-	};
-
-	public static authenticated = true;
-	public static primary = true;
-
-	public async run() {
-		const { args: params } = this.parse<FlagsDef, ArgsDef>(AppCmd);
-
-		const application = (await getBalenaSdk().models.application.get(
-			tryAsInteger(params.name),
-			{
-				$expand: {
-					is_for__device_type: { $select: 'slug' },
-					should_be_running__release: { $select: 'commit' },
-				},
-			},
-		)) as ApplicationWithDeviceType & {
-			should_be_running__release: [Release?];
-		};
-
-		// @ts-expect-error
-		application.device_type = application.is_for__device_type[0].slug;
-		// @ts-expect-error
-		application.commit = application.should_be_running__release[0]?.commit;
-		console.log(
-			getVisuals().table.vertical(application, [
-				`$${application.app_name}$`,
-				'id',
-				'device_type',
-				'slug',
-				'commit',
-			]),
-		);
-	}
-}

lib/commands/app/rename.ts

@@ -1,135 +0,0 @@
-/**
- * @license
- * Copyright 2016-2020 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-import { flags } from '@oclif/command';
-import type { IArg } from '@oclif/parser/lib/args';
-import Command from '../../command';
-import * as cf from '../../utils/common-flags';
-import { getBalenaSdk, stripIndent, getCliForm } from '../../utils/lazy';
-import type { Application, ApplicationType, BalenaSDK } from 'balena-sdk';
-
-interface FlagsDef {
-	help: void;
-}
-
-interface ArgsDef {
-	name: string;
-	newName?: string;
-}
-
-export default class AppRenameCmd extends Command {
-	public static description = stripIndent`
-		Rename an application.
-
-		Rename an application.
-
-		Note, if the \`newName\` parameter is omitted, it will be
-		prompted for interactively.
-		`;
-	public static examples = [
-		'$ balena app rename OldName',
-		'$ balena app rename OldName NewName',
-	];
-
-	public static args: Array<IArg<any>> = [
-		{
-			name: 'name',
-			description: 'application name or numeric ID',
-			required: true,
-		},
-		{
-			name: 'newName',
-			description: 'the new name for the application',
-		},
-	];
-
-	public static usage = 'app rename <name> [newName]';
-
-	public static flags: flags.Input<FlagsDef> = {
-		help: cf.help,
-	};
-
-	public static authenticated = true;
-
-	public async run() {
-		const { args: params } = this.parse<FlagsDef, ArgsDef>(AppRenameCmd);
-
-		const { ExpectedError, instanceOf } = await import('../../errors');
-		const balena = getBalenaSdk();
-
-		// Get app
-		let app;
-		try {
-			app = await balena.models.application.get(params.name, {
-				$expand: {
-					application_type: {
-						$select: ['is_legacy'],
-					},
-				},
-			});
-		} catch (e) {
-			const { BalenaApplicationNotFound } = await import('balena-errors');
-			if (instanceOf(e, BalenaApplicationNotFound)) {
-				throw new ExpectedError(`Application ${params.name} not found.`);
-			} else {
-				throw e;
-			}
-		}
-
-		// Check app supports renaming
-		const appType = (app.application_type as ApplicationType[])?.[0];
-		if (appType.is_legacy) {
-			throw new ExpectedError(
-				`Application ${params.name} is of 'legacy' type, and cannot be renamed.`,
-			);
-		}
-
-		const { validateApplicationName } = await import('../../utils/validation');
-		const newName =
-			params.newName ||
-			(await getCliForm().ask({
-				message: 'Please enter the new name for this application:',
-				type: 'input',
-				validate: validateApplicationName,
-			})) ||
-			'';
-
-		try {
-			await this.renameApplication(balena, app.id, newName);
-		} catch (e) {
-			// BalenaRequestError: Request error: Unique key constraint violated
-			if ((e.message || '').toLowerCase().includes('unique')) {
-				throw new ExpectedError(
-					`Error: application ${params.name} already exists.`,
-				);
-			}
-			throw e;
-		}
-
-		console.log(`Application ${params.name} renamed to ${newName}`);
-	}
-
-	async renameApplication(balena: BalenaSDK, id: number, newName: string) {
-		return balena.pine.patch<Application>({
-			resource: 'application',
-			id,
-			body: {
-				app_name: newName,
-			},
-		});
-	}
-}

lib/commands/app/restart.ts

@@ -1,61 +0,0 @@
-/**
- * @license
- * Copyright 2016-2020 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-import { flags } from '@oclif/command';
-import Command from '../../command';
-import * as cf from '../../utils/common-flags';
-import { getBalenaSdk, stripIndent } from '../../utils/lazy';
-import { tryAsInteger } from '../../utils/validation';
-
-interface FlagsDef {
-	help: void;
-}
-
-interface ArgsDef {
-	name: string;
-}
-
-export default class AppRestartCmd extends Command {
-	public static description = stripIndent`
-		Restart an application.
-
-		Restart all devices that belongs to a certain application.
-`;
-	public static examples = ['$ balena app restart MyApp'];
-
-	public static args = [
-		{
-			name: 'name',
-			description: 'application name or numeric ID',
-			required: true,
-		},
-	];
-
-	public static usage = 'app restart <name>';
-
-	public static flags: flags.Input<FlagsDef> = {
-		help: cf.help,
-	};
-
-	public static authenticated = true;
-
-	public async run() {
-		const { args: params } = this.parse<FlagsDef, ArgsDef>(AppRestartCmd);
-
-		await getBalenaSdk().models.application.restart(tryAsInteger(params.name));
-	}
-}

lib/commands/app/rm.ts

@@ -1,79 +0,0 @@
-/**
- * @license
- * Copyright 2016-2020 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-import { flags } from '@oclif/command';
-import Command from '../../command';
-import * as cf from '../../utils/common-flags';
-import { getBalenaSdk, stripIndent } from '../../utils/lazy';
-import { tryAsInteger } from '../../utils/validation';
-
-interface FlagsDef {
-	yes: boolean;
-	help: void;
-}
-
-interface ArgsDef {
-	name: string;
-}
-
-export default class AppRmCmd extends Command {
-	public static description = stripIndent`
-		Remove an application.
-
-		Permanently remove a balena application.
-
-		The --yes option may be used to avoid interactive confirmation.
-`;
-	public static examples = [
-		'$ balena app rm MyApp',
-		'$ balena app rm MyApp --yes',
-	];
-
-	public static args = [
-		{
-			name: 'name',
-			description: 'application name or numeric ID',
-			required: true,
-		},
-	];
-
-	public static usage = 'app rm <name>';
-
-	public static flags: flags.Input<FlagsDef> = {
-		yes: cf.yes,
-		help: cf.help,
-	};
-
-	public static authenticated = true;
-
-	public async run() {
-		const { args: params, flags: options } = this.parse<FlagsDef, ArgsDef>(
-			AppRmCmd,
-		);
-
-		const patterns = await import('../../utils/patterns');
-
-		// Confirm
-		await patterns.confirm(
-			options.yes ?? false,
-			`Are you sure you want to delete application ${params.name}?`,
-		);
-
-		// Remove
-		await getBalenaSdk().models.application.remove(tryAsInteger(params.name));
-	}
-}

lib/commands/apps.ts

@@ -1,97 +0,0 @@
-/**
- * @license
- * Copyright 2016-2020 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-import { flags } from '@oclif/command';
-import Command from '../command';
-import * as cf from '../utils/common-flags';
-import { getBalenaSdk, getVisuals, stripIndent } from '../utils/lazy';
-import { isV12 } from '../utils/version';
-
-interface ExtendedApplication extends ApplicationWithDeviceType {
-	device_count?: number;
-	online_devices?: number;
-}
-
-interface FlagsDef {
-	help: void;
-	verbose?: boolean;
-}
-
-export default class AppsCmd extends Command {
-	public static description = stripIndent`
-		List all applications.
-
-		list all your balena applications.
-
-		For detailed information on a particular application,
-		use \`balena app <name> instead\`.
-`;
-	public static examples = ['$ balena apps'];
-
-	public static usage = 'apps';
-
-	public static flags: flags.Input<FlagsDef> = {
-		help: cf.help,
-		verbose: flags.boolean({
-			char: 'v',
-			description: isV12()
-				? 'No-op since release v12.0.0'
-				: 'add extra columns in the tabular output (SLUG)',
-		}),
-	};
-
-	public static authenticated = true;
-	public static primary = true;
-
-	public async run() {
-		const { flags: options } = this.parse<FlagsDef, {}>(AppsCmd);
-
-		const balena = getBalenaSdk();
-
-		// Get applications
-		const applications = (await balena.models.application.getAll({
-			$select: ['id', 'app_name', 'slug'],
-			$expand: {
-				is_for__device_type: { $select: 'slug' },
-				owns__device: { $select: 'is_online' },
-			},
-		})) as ExtendedApplication[];
-
-		const _ = await import('lodash');
-		// Add extended properties
-		applications.forEach((application) => {
-			application.device_count = application.owns__device?.length ?? 0;
-			application.online_devices = _.sumBy(application.owns__device, (d) =>
-				d.is_online === true ? 1 : 0,
-			);
-			// @ts-expect-error
-			application.device_type = application.is_for__device_type[0].slug;
-		});
-
-		// Display
-		console.log(
-			getVisuals().table.horizontal(applications, [
-				'id',
-				'app_name',
-				options.verbose || isV12() ? 'slug' : '',
-				'device_type',
-				'online_devices',
-				'device_count',
-			]),
-		);
-	}
-}

lib/commands/build.ts

@@ -1,258 +0,0 @@
-/**
- * @license
- * Copyright 2016-2020 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-import { flags } from '@oclif/command';
-import Command from '../command';
-import { getBalenaSdk } from '../utils/lazy';
-import * as compose from '../utils/compose';
-import type { Application, ApplicationType, BalenaSDK } from 'balena-sdk';
-import { dockerignoreHelp, registrySecretsHelp } from '../utils/messages';
-import type { ComposeCliFlags, ComposeOpts } from '../utils/compose-types';
-import { composeCliFlags } from '../utils/compose_ts';
-import type { DockerCliFlags } from '../utils/docker';
-import { dockerCliFlags } from '../utils/docker';
-
-interface FlagsDef extends ComposeCliFlags, DockerCliFlags {
-	arch?: string;
-	deviceType?: string;
-	application?: string;
-	source?: string; // Not part of command profile - source param copied here.
-	help: void;
-}
-
-interface ArgsDef {
-	source?: string;
-}
-
-export default class BuildCmd extends Command {
-	public static description = `\
-Build a project locally.
-
-Use this command to build an image or a complete multicontainer project with
-the provided docker daemon in your development machine or balena device.
-(See also the \`balena push\` command for the option of building images in the
-balenaCloud build servers.)
-
-You must provide either an application or a device-type/architecture pair.
-
-This command will look into the given source directory (or the current working
-directory if one isn't specified) for a docker-compose.yml file, and if found,
-each service defined in the compose file will be built. If a compose file isn't
-found, it will look for a Dockerfile[.template] file (or alternative Dockerfile
-specified with the \`--dockerfile\` option), and if no dockerfile is found, it
-will try to generate one.
-
-${registrySecretsHelp}
-
-${dockerignoreHelp}
-`;
-	public static examples = [
-		'$ balena build --application myApp',
-		'$ balena build ./source/ --application myApp',
-		'$ balena build --deviceType raspberrypi3 --arch armv7hf --emulated',
-		'$ balena build --docker /var/run/docker.sock --application myApp   # Linux, Mac',
-		'$ balena build --docker //./pipe/docker_engine --application myApp # Windows',
-		'$ balena build --dockerHost my.docker.host --dockerPort 2376 --ca ca.pem --key key.pem --cert cert.pem -a myApp',
-	];
-
-	public static args = [
-		{
-			name: 'source',
-			description: 'path of project source directory',
-		},
-	];
-
-	public static usage = 'build [source]';
-
-	public static flags: flags.Input<FlagsDef> = {
-		arch: flags.string({
-			description: 'the architecture to build for',
-			char: 'A',
-		}),
-		deviceType: flags.string({
-			description: 'the type of device this build is for',
-			char: 'd',
-		}),
-		application: flags.string({
-			description: 'name of the target balena application this build is for',
-			char: 'a',
-		}),
-		...composeCliFlags,
-		...dockerCliFlags,
-		// NOTE: Not supporting -h for help, because of clash with -h in DockerCliFlags
-		// Revisit this in future release.
-		help: flags.help({}),
-	};
-
-	public static primary = true;
-
-	public async run() {
-		const { args: params, flags: options } = this.parse<FlagsDef, ArgsDef>(
-			BuildCmd,
-		);
-
-		await Command.checkLoggedInIf(!!options.application);
-
-		// compositions with many services trigger misleading warnings
-		// @ts-ignore editing property that isn't typed but does exist
-		(await import('events')).defaultMaxListeners = 1000;
-
-		const sdk = getBalenaSdk();
-
-		const logger = await Command.getLogger();
-		logger.logDebug('Parsing input...');
-
-		// `build` accepts `source` as a parameter, but compose expects it as an option
-		options.source = params.source;
-		delete params.source;
-
-		await this.validateOptions(options, sdk);
-
-		const app = await this.getAppAndResolveArch(options);
-
-		const { docker, buildOpts, composeOpts } = await this.prepareBuild(options);
-
-		try {
-			await this.buildProject(docker, logger, composeOpts, {
-				app,
-				arch: options.arch!,
-				deviceType: options.deviceType!,
-				buildEmulated: options.emulated,
-				buildOpts,
-			});
-		} catch (err) {
-			logger.logError('Build failed.');
-			throw err;
-		}
-
-		logger.outputDeferredMessages();
-		logger.logSuccess('Build succeeded!');
-	}
-
-	protected async validateOptions(opts: FlagsDef, sdk: BalenaSDK) {
-		// Validate option combinations
-		if (
-			(opts.application == null &&
-				(opts.arch == null || opts.deviceType == null)) ||
-			(opts.application != null &&
-				(opts.arch != null || opts.deviceType != null))
-		) {
-			const { ExpectedError } = await import('../errors');
-			throw new ExpectedError(
-				'You must specify either an application or an arch/deviceType pair to build for',
-			);
-		}
-
-		// Validate project directory
-		const { validateProjectDirectory } = await import('../utils/compose_ts');
-		const { dockerfilePath, registrySecrets } = await validateProjectDirectory(
-			sdk,
-			{
-				dockerfilePath: opts.dockerfile,
-				noParentCheck: opts['noparent-check'] || false,
-				projectPath: opts.source || '.',
-				registrySecretsPath: opts['registry-secrets'],
-			},
-		);
-
-		opts.dockerfile = dockerfilePath;
-		opts['registry-secrets'] = registrySecrets;
-	}
-
-	protected async getAppAndResolveArch(opts: FlagsDef) {
-		if (opts.application) {
-			const { getAppWithArch } = await import('../utils/helpers');
-			const app = await getAppWithArch(opts.application);
-			opts.arch = app.arch;
-			opts.deviceType = app.is_for__device_type[0].slug;
-			return app;
-		}
-	}
-
-	protected async prepareBuild(options: FlagsDef) {
-		const { getDocker, generateBuildOpts } = await import('../utils/docker');
-		const [docker, buildOpts, composeOpts] = await Promise.all([
-			getDocker(options),
-			generateBuildOpts(options),
-			compose.generateOpts(options),
-		]);
-		return {
-			docker,
-			buildOpts,
-			composeOpts,
-		};
-	}
-
-	/**
-	 * Opts must be an object with the following keys:
-	 *   app: the app this build is for (optional)
-	 *   arch: the architecture to build for
-	 *   deviceType: the device type to build for
-	 *   buildEmulated
-	 *   buildOpts: arguments to forward to docker build command
-	 *
-	 * @param {DockerToolbelt} docker
-	 * @param {Logger} logger
-	 * @param {ComposeOpts} composeOpts
-	 * @param opts
-	 */
-	protected async buildProject(
-		docker: import('docker-toolbelt'),
-		logger: import('../utils/logger'),
-		composeOpts: ComposeOpts,
-		opts: {
-			app?: Application;
-			arch: string;
-			deviceType: string;
-			buildEmulated: boolean;
-			buildOpts: any;
-		},
-	) {
-		const { loadProject } = await import('../utils/compose_ts');
-
-		const project = await loadProject(logger, composeOpts);
-
-		const appType = (opts.app?.application_type as ApplicationType[])?.[0];
-		if (
-			appType != null &&
-			project.descriptors.length > 1 &&
-			!appType.supports_multicontainer
-		) {
-			logger.logWarn(
-				'Target application does not support multiple containers.\n' +
-					'Continuing with build, but you will not be able to deploy.',
-			);
-		}
-
-		await compose.buildProject(
-			docker,
-			logger,
-			project.path,
-			project.name,
-			project.composition,
-			opts.arch,
-			opts.deviceType,
-			opts.buildEmulated,
-			opts.buildOpts,
-			composeOpts.inlineLogs,
-			composeOpts.convertEol,
-			composeOpts.dockerfilePath,
-			composeOpts.nogitignore,
-			composeOpts.multiDockerignore,
-		);
-	}
-}

lib/commands/config/generate.ts

@@ -1,246 +0,0 @@
-/**
- * @license
- * Copyright 2016-2020 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-import { flags } from '@oclif/command';
-import Command from '../../command';
-import * as cf from '../../utils/common-flags';
-import { getBalenaSdk, getCliForm, stripIndent } from '../../utils/lazy';
-import type { Application, PineDeferred } from 'balena-sdk';
-
-interface FlagsDef {
-	version: string; // OS version
-	application?: string;
-	app?: string; // application alias
-	device?: string;
-	deviceApiKey?: string;
-	deviceType?: string;
-	'generate-device-api-key': boolean;
-	output?: string;
-	// Options for non-interactive configuration
-	network?: string;
-	wifiSsid?: string;
-	wifiKey?: string;
-	appUpdatePollInterval?: string;
-	help: void;
-}
-
-export default class ConfigGenerateCmd extends Command {
-	public static description = stripIndent`
-		Generate a config.json file.
-
-		Generate a config.json file for a device or application.
-
-		Calling this command with the exact version number of the targeted image is required.
-
-		This command is interactive by default, but you can do this automatically without interactivity
-		by specifying an option for each question on the command line, if you know the questions
-		that will be asked for the relevant device type.
-
-		In case that you want to configure an image for an application with mixed device types,
-		you can pass the --device-type argument along with --app to specify the target device type.
-	`;
-
-	public static examples = [
-		'$ balena config generate --device 7cf02a6 --version 2.12.7',
-		'$ balena config generate --device 7cf02a6 --version 2.12.7 --generate-device-api-key',
-		'$ balena config generate --device 7cf02a6 --version 2.12.7 --device-api-key <existingDeviceKey>',
-		'$ balena config generate --device 7cf02a6 --version 2.12.7 --output config.json',
-		'$ balena config generate --app MyApp --version 2.12.7',
-		'$ balena config generate --app MyApp --version 2.12.7 --device-type fincm3',
-		'$ balena config generate --app MyApp --version 2.12.7 --output config.json',
-		'$ balena config generate --app MyApp --version 2.12.7 --network wifi --wifiSsid mySsid --wifiKey abcdefgh --appUpdatePollInterval 1',
-	];
-
-	public static usage = 'config generate';
-
-	public static flags: flags.Input<FlagsDef> = {
-		version: flags.string({
-			description: 'a balenaOS version',
-			required: true,
-		}),
-		application: flags.string({
-			description: 'application name',
-			char: 'a',
-			exclusive: ['app', 'device'],
-		}),
-		app: flags.string({
-			description: "same as '--application'",
-			exclusive: ['application', 'device'],
-		}),
-		device: flags.string({
-			description: 'device uuid',
-			char: 'd',
-			exclusive: ['application', 'app'],
-		}),
-		deviceApiKey: flags.string({
-			description:
-				'custom device key - note that this is only supported on balenaOS 2.0.3+',
-			char: 'k',
-		}),
-		deviceType: flags.string({
-			description: 'device type slug',
-		}),
-		'generate-device-api-key': flags.boolean({
-			description: 'generate a fresh device key for the device',
-		}),
-		output: flags.string({
-			description: 'path of output file',
-			char: 'o',
-		}),
-		// Options for non-interactive configuration
-		network: flags.string({
-			description: 'the network type to use: ethernet or wifi',
-			options: ['ethernet', 'wifi'],
-		}),
-		wifiSsid: flags.string({
-			description:
-				'the wifi ssid to use (used only if --network is set to wifi)',
-		}),
-		wifiKey: flags.string({
-			description:
-				'the wifi key to use (used only if --network is set to wifi)',
-		}),
-		appUpdatePollInterval: flags.string({
-			description:
-				'how frequently (in minutes) to poll for application updates',
-		}),
-		help: cf.help,
-	};
-
-	public static authenticated = true;
-
-	public async run() {
-		const { flags: options } = this.parse<FlagsDef, {}>(ConfigGenerateCmd);
-
-		const balena = getBalenaSdk();
-
-		await this.validateOptions(options);
-
-		let deviceType = options.deviceType;
-		// Get device | application
-		let resource;
-		if (options.device != null) {
-			const { tryAsInteger } = await import('../../utils/validation');
-			resource = (await balena.models.device.get(tryAsInteger(options.device), {
-				$expand: {
-					is_of__device_type: { $select: 'slug' },
-				},
-			})) as DeviceWithDeviceType & { belongs_to__application: PineDeferred };
-			deviceType = deviceType || resource.is_of__device_type[0].slug;
-		} else {
-			resource = (await balena.models.application.get(options.application!, {
-				$expand: {
-					is_for__device_type: { $select: 'slug' },
-				},
-			})) as ApplicationWithDeviceType;
-			deviceType = deviceType || resource.is_for__device_type[0].slug;
-		}
-
-		const deviceManifest = await balena.models.device.getManifestBySlug(
-			deviceType!,
-		);
-
-		// Check compatibility if application and deviceType provided
-		if (options.application && options.deviceType) {
-			const appDeviceManifest = await balena.models.device.getManifestBySlug(
-				deviceType!,
-			);
-
-			const helpers = await import('../../utils/helpers');
-			if (
-				!helpers.areDeviceTypesCompatible(appDeviceManifest, deviceManifest)
-			) {
-				throw new balena.errors.BalenaInvalidDeviceType(
-					`Device type ${options.deviceType} is incompatible with application ${options.application}`,
-				);
-			}
-		}
-
-		// Prompt for values
-		// Pass params as an override: if there is any param with exactly the same name as a
-		// required option, that value is used (and the corresponding question is not asked)
-		const answers = await getCliForm().run(deviceManifest.options, {
-			override: options,
-		});
-		answers.version = options.version;
-
-		// Generate config
-		const { generateDeviceConfig, generateApplicationConfig } = await import(
-			'../../utils/config'
-		);
-
-		let config;
-		if ('uuid' in resource && resource.uuid != null) {
-			config = await generateDeviceConfig(
-				resource,
-				options.deviceApiKey || options['generate-device-api-key'] || undefined,
-				answers,
-			);
-		} else {
-			answers.deviceType = deviceType;
-			config = await generateApplicationConfig(
-				resource as Application,
-				answers,
-			);
-		}
-
-		// Output
-		if (options.output != null) {
-			const fs = await import('fs');
-			await fs.promises.writeFile(options.output, JSON.stringify(config));
-		}
-
-		const prettyjson = await import('prettyjson');
-		console.log(prettyjson.render(config));
-	}
-
-	protected readonly missingDeviceOrAppMessage = stripIndent`
-		Either a device or an application must be specified.
-
-		See the help page for examples:
-
-		  $ balena help config generate
-  	`;
-
-	protected readonly deviceTypeNotAllowedMessage = stripIndent`
-		Specifying a different device type is only supported when
-		generating a config for an application:
-
-		* An application, with --app <appname>
-		* A specific device type, with --device-type <deviceTypeSlug>
-
-		See the help page for examples:
-
-		  $ balena help config generate
-	`;
-
-	protected async validateOptions(options: FlagsDef) {
-		const { ExpectedError } = await import('../../errors');
-
-		// Prefer options.application over options.app
-		options.application = options.application || options.app;
-		delete options.app;
-
-		if (options.device == null && options.application == null) {
-			throw new ExpectedError(this.missingDeviceOrAppMessage);
-		}
-
-		if (!options.application && options.deviceType) {
-			throw new ExpectedError(this.deviceTypeNotAllowedMessage);
-		}
-	}
-}

lib/commands/config/inject.ts

@@ -1,96 +0,0 @@
-/**
- * @license
- * Copyright 2016-2020 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-import { flags } from '@oclif/command';
-import Command from '../../command';
-import * as cf from '../../utils/common-flags';
-import { getVisuals, stripIndent } from '../../utils/lazy';
-
-interface FlagsDef {
-	type: string;
-	drive?: string;
-	help: void;
-}
-
-interface ArgsDef {
-	file: string;
-}
-
-export default class ConfigInjectCmd extends Command {
-	public static description = stripIndent`
-		Inject a configuration file into a device or OS image.
-
-		Inject a config.json file to the mounted filesystem,
-		e.g. the SD card of a provisioned device or balenaOS image.
-	`;
-
-	public static examples = [
-		'$ balena config inject my/config.json --type raspberrypi3',
-		'$ balena config inject my/config.json --type raspberrypi3 --drive /dev/disk2',
-	];
-
-	public static args = [
-		{
-			name: 'file',
-			description: 'the path to the config.json file to inject',
-			required: true,
-		},
-	];
-
-	public static usage = 'config inject <file>';
-
-	public static flags: flags.Input<FlagsDef> = {
-		type: flags.string({
-			description:
-				'device type (Check available types with `balena devices supported`)',
-			char: 't',
-			required: true,
-		}),
-		drive: flags.string({
-			description: 'device filesystem or OS image location',
-			char: 'd',
-		}),
-		help: cf.help,
-	};
-
-	public static authenticated = true;
-
-	public static root = true;
-
-	public async run() {
-		const { args: params, flags: options } = this.parse<FlagsDef, ArgsDef>(
-			ConfigInjectCmd,
-		);
-
-		const { promisify } = await import('util');
-		const umountAsync = promisify((await import('umount')).umount);
-
-		const drive =
-			options.drive || (await getVisuals().drive('Select the device/OS drive'));
-		await umountAsync(drive);
-
-		const fs = await import('fs');
-		const configJSON = JSON.parse(
-			await fs.promises.readFile(params.file, 'utf8'),
-		);
-
-		const config = await import('balena-config-json');
-		await config.write(drive, options.type, configJSON);
-
-		console.info('Done');
-	}
-}

lib/commands/config/read.ts

@@ -1,78 +0,0 @@
-/**
- * @license
- * Copyright 2016-2020 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-import { flags } from '@oclif/command';
-import Command from '../../command';
-import * as cf from '../../utils/common-flags';
-import { getVisuals, stripIndent } from '../../utils/lazy';
-
-interface FlagsDef {
-	type: string;
-	drive?: string;
-	help: void;
-}
-
-export default class ConfigReadCmd extends Command {
-	public static description = stripIndent`
-		Read the configuration of a device or OS image.
-
-		Read the config.json file from the mounted filesystem,
-		e.g. the SD card of a provisioned device or balenaOS image.
-	`;
-
-	public static examples = [
-		'$ balena config read --type raspberrypi3',
-		'$ balena config read --type raspberrypi3 --drive /dev/disk2',
-	];
-
-	public static usage = 'config read';
-
-	public static flags: flags.Input<FlagsDef> = {
-		type: flags.string({
-			description:
-				'device type (Check available types with `balena devices supported`)',
-			char: 't',
-			required: true,
-		}),
-		drive: flags.string({
-			description: 'device filesystem or OS image location',
-			char: 'd',
-		}),
-		help: cf.help,
-	};
-
-	public static authenticated = true;
-
-	public static root = true;
-
-	public async run() {
-		const { flags: options } = this.parse<FlagsDef, {}>(ConfigReadCmd);
-
-		const { promisify } = await import('util');
-		const umountAsync = promisify((await import('umount')).umount);
-
-		const drive =
-			options.drive || (await getVisuals().drive('Select the device drive'));
-		await umountAsync(drive);
-
-		const config = await import('balena-config-json');
-		const configJSON = await config.read(drive, options.type);
-
-		const prettyjson = await import('prettyjson');
-		console.info(prettyjson.render(configJSON));
-	}
-}

lib/commands/config/write.ts

@@ -1,106 +0,0 @@
-/**
- * @license
- * Copyright 2016-2020 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-import { flags } from '@oclif/command';
-import Command from '../../command';
-import * as cf from '../../utils/common-flags';
-import { getVisuals, stripIndent } from '../../utils/lazy';
-
-interface FlagsDef {
-	type: string;
-	drive?: string;
-	help: void;
-}
-
-interface ArgsDef {
-	key: string;
-	value: string;
-}
-
-export default class ConfigWriteCmd extends Command {
-	public static description = stripIndent`
-		Write a key-value pair to configuration of a device or OS image.
-
-		Write a key-value pair to the config.json file on the mounted filesystem,
-		e.g. the SD card of a provisioned device or balenaOS image.
-	`;
-
-	public static examples = [
-		'$ balena config write --type raspberrypi3 username johndoe',
-		'$ balena config write --type raspberrypi3 --drive /dev/disk2 username johndoe',
-		'$ balena config write --type raspberrypi3 files.network/settings "..."',
-	];
-
-	public static args = [
-		{
-			name: 'key',
-			description: 'the key of the config parameter to write',
-			required: true,
-		},
-		{
-			name: 'value',
-			description: 'the value of the config parameter to write',
-			required: true,
-		},
-	];
-
-	public static usage = 'config write <key> <value>';
-
-	public static flags: flags.Input<FlagsDef> = {
-		type: flags.string({
-			description:
-				'device type (Check available types with `balena devices supported`)',
-			char: 't',
-			required: true,
-		}),
-		drive: flags.string({
-			description: 'device filesystem or OS image location',
-			char: 'd',
-		}),
-		help: cf.help,
-	};
-
-	public static authenticated = true;
-
-	public static root = true;
-
-	public async run() {
-		const { args: params, flags: options } = this.parse<FlagsDef, ArgsDef>(
-			ConfigWriteCmd,
-		);
-
-		const { promisify } = await import('util');
-		const umountAsync = promisify((await import('umount')).umount);
-
-		const drive =
-			options.drive || (await getVisuals().drive('Select the device drive'));
-		await umountAsync(drive);
-
-		const config = await import('balena-config-json');
-		const configJSON = await config.read(drive, options.type);
-
-		console.info(`Setting ${params.key} to ${params.value}`);
-		const _ = await import('lodash');
-		_.set(configJSON, params.key, params.value);
-
-		await umountAsync(drive);
-
-		await config.write(drive, options.type, configJSON);
-
-		console.info('Done');
-	}
-}

lib/commands/deploy.ts

@@ -1,343 +0,0 @@
-/**
- * @license
- * Copyright 2016-2020 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-import { flags } from '@oclif/command';
-import Command from '../command';
-import { ExpectedError } from '../errors';
-import { getBalenaSdk, getChalk } from '../utils/lazy';
-import { dockerignoreHelp, registrySecretsHelp } from '../utils/messages';
-import * as compose from '../utils/compose';
-import type { ComposeCliFlags, ComposeOpts } from '../utils/compose-types';
-import type { DockerCliFlags } from '../utils/docker';
-import { composeCliFlags } from '../utils/compose_ts';
-import { dockerCliFlags } from '../utils/docker';
-import type { Application, ApplicationType, DeviceType } from 'balena-sdk';
-
-interface ApplicationWithArch extends Application {
-	arch: string;
-}
-
-interface FlagsDef extends ComposeCliFlags, DockerCliFlags {
-	source?: string;
-	build: boolean;
-	nologupload: boolean;
-	help: void;
-}
-
-interface ArgsDef {
-	appName: string;
-	image?: string;
-}
-
-export default class DeployCmd extends Command {
-	public static description = `\
-Deploy a single image or a multicontainer project to a balena application.
-
-Usage: \`deploy <appName> ([image] | --build [--source build-dir])\`
-
-Use this command to deploy an image or a complete multicontainer project to an
-application, optionally building it first. The source images are searched for
-(and optionally built) using the docker daemon in your development machine or
-balena device. (See also the \`balena push\` command for the option of building
-the image in the balenaCloud build servers.)
-
-Unless an image is specified, this command will look into the current directory
-(or the one specified by --source) for a docker-compose.yml file.  If one is
-found, this command will deploy each service defined in the compose file,
-building it first if an image for it doesn't exist. If a compose file isn't
-found, the command will look for a Dockerfile[.template] file (or alternative
-Dockerfile specified with the \`-f\` option), and if yet that isn't found, it
-will try to generate one.
-
-To deploy to an app on which you're a collaborator, use
-\`balena deploy <appOwnerUsername>/<appName>\`.
-
-${registrySecretsHelp}
-
-${dockerignoreHelp}
-`;
-
-	public static examples = [
-		'$ balena deploy myApp',
-		'$ balena deploy myApp --build --source myBuildDir/',
-		'$ balena deploy myApp myApp/myImage',
-	];
-
-	public static args = [
-		{
-			name: 'appName',
-			description: 'the name of the application to deploy to',
-			required: true,
-		},
-		{
-			name: 'image',
-			description: 'the image to deploy',
-		},
-	];
-
-	public static usage = 'deploy <appName> [image]';
-
-	public static flags: flags.Input<FlagsDef> = {
-		source: flags.string({
-			description:
-				'specify an alternate source directory; default is the working directory',
-			char: 's',
-		}),
-		build: flags.boolean({
-			description: 'force a rebuild before deploy',
-			char: 'b',
-		}),
-		nologupload: flags.boolean({
-			description:
-				"don't upload build logs to the dashboard with image (if building)",
-		}),
-		...composeCliFlags,
-		...dockerCliFlags,
-		// NOTE: Not supporting -h for help, because of clash with -h in DockerCliFlags
-		// Revisit this in future release.
-		help: flags.help({}),
-	};
-
-	public static authenticated = true;
-
-	public static primary = true;
-
-	public async run() {
-		const { args: params, flags: options } = this.parse<FlagsDef, ArgsDef>(
-			DeployCmd,
-		);
-
-		// compositions with many services trigger misleading warnings
-		// @ts-ignore editing property that isn't typed but does exist
-		(await import('events')).defaultMaxListeners = 1000;
-
-		const logger = await Command.getLogger();
-		logger.logDebug('Parsing input...');
-
-		const { appName, image } = params;
-
-		if (image != null && options.build) {
-			throw new ExpectedError(
-				'Build option is not applicable when specifying an image',
-			);
-		}
-
-		const sdk = getBalenaSdk();
-		const { getRegistrySecrets, validateProjectDirectory } = await import(
-			'../utils/compose_ts'
-		);
-
-		if (image) {
-			options['registry-secrets'] = await getRegistrySecrets(
-				sdk,
-				options['registry-secrets'],
-			);
-		} else {
-			const {
-				dockerfilePath,
-				registrySecrets,
-			} = await validateProjectDirectory(sdk, {
-				dockerfilePath: options.dockerfile,
-				noParentCheck: options['noparent-check'] || false,
-				projectPath: options.source || '.',
-				registrySecretsPath: options['registry-secrets'],
-			});
-			options.dockerfile = dockerfilePath;
-			options['registry-secrets'] = registrySecrets;
-		}
-
-		const helpers = await import('../utils/helpers');
-		const app = await helpers.getAppWithArch(appName);
-
-		const dockerUtils = await import('../utils/docker');
-		const [docker, buildOpts, composeOpts] = await Promise.all([
-			dockerUtils.getDocker(options),
-			dockerUtils.generateBuildOpts(options),
-			compose.generateOpts(options),
-		]);
-
-		await this.deployProject(docker, logger, composeOpts, {
-			app,
-			appName, // may be prefixed by 'owner/', unlike app.app_name
-			image,
-			shouldPerformBuild: !!options.build,
-			shouldUploadLogs: !options.nologupload,
-			buildEmulated: !!options.emulated,
-			buildOpts,
-		});
-	}
-
-	async deployProject(
-		docker: import('docker-toolbelt'),
-		logger: import('../utils/logger'),
-		composeOpts: ComposeOpts,
-		opts: {
-			app: ApplicationWithArch; // the application instance to deploy to
-			appName: string;
-			image?: string;
-			dockerfilePath?: string; // alternative Dockerfile
-			shouldPerformBuild: boolean;
-			shouldUploadLogs: boolean;
-			buildEmulated: boolean;
-			buildOpts: any; // arguments to forward to docker build command
-		},
-	) {
-		const _ = await import('lodash');
-		const doodles = await import('resin-doodles');
-		const sdk = getBalenaSdk();
-		const { deployProject: $deployProject, loadProject } = await import(
-			'../utils/compose_ts'
-		);
-
-		const appType = (opts.app?.application_type as ApplicationType[])?.[0];
-
-		try {
-			const project = await loadProject(logger, composeOpts, opts.image);
-			if (project.descriptors.length > 1 && !appType?.supports_multicontainer) {
-				throw new ExpectedError(
-					'Target application does not support multiple containers. Aborting!',
-				);
-			}
-
-			// find which services use images that already exist locally
-			let servicesToSkip = await Promise.all(
-				project.descriptors.map(async function (d: any) {
-					// unconditionally build (or pull) if explicitly requested
-					if (opts.shouldPerformBuild) {
-						return d;
-					}
-					try {
-						await docker
-							.getImage(
-								(typeof d.image === 'string' ? d.image : d.image.tag) || '',
-							)
-							.inspect();
-
-						return d.serviceName;
-					} catch {
-						// Ignore
-					}
-				}),
-			);
-			servicesToSkip = servicesToSkip.filter((d) => !!d);
-
-			// multibuild takes in a composition and always attempts to
-			// build or pull all services. we workaround that here by
-			// passing a modified composition.
-			const compositionToBuild = _.cloneDeep(project.composition);
-			compositionToBuild.services = _.omit(
-				compositionToBuild.services,
-				servicesToSkip,
-			);
-			let builtImagesByService: Dictionary<any> = {};
-			if (_.size(compositionToBuild.services) === 0) {
-				logger.logInfo(
-					'Everything is up to date (use --build to force a rebuild)',
-				);
-			} else {
-				const builtImages = await compose.buildProject(
-					docker,
-					logger,
-					project.path,
-					project.name,
-					compositionToBuild,
-					opts.app.arch,
-					(opts.app?.is_for__device_type as DeviceType[])?.[0].slug,
-					opts.buildEmulated,
-					opts.buildOpts,
-					composeOpts.inlineLogs,
-					composeOpts.convertEol,
-					composeOpts.dockerfilePath,
-					composeOpts.nogitignore,
-					composeOpts.multiDockerignore,
-				);
-				builtImagesByService = _.keyBy(builtImages, 'serviceName');
-			}
-			const images = project.descriptors.map(
-				(d) =>
-					builtImagesByService[d.serviceName] ?? {
-						serviceName: d.serviceName,
-						name: typeof d.image === 'string' ? d.image : d.image.tag,
-						logs: 'Build skipped; image for service already exists.',
-						props: {},
-					},
-			);
-
-			let release;
-			if (appType?.is_legacy) {
-				const { deployLegacy } = require('../utils/deploy-legacy');
-
-				const msg = getChalk().yellow(
-					'Target application requires legacy deploy method.',
-				);
-				logger.logWarn(msg);
-
-				const [token, username, url, options] = await Promise.all([
-					sdk.auth.getToken(),
-					sdk.auth.whoami(),
-					sdk.settings.get('balenaUrl'),
-					{
-						// opts.appName may be prefixed by 'owner/', unlike opts.app.app_name
-						appName: opts.appName,
-						imageName: images[0].name,
-						buildLogs: images[0].logs,
-						shouldUploadLogs: opts.shouldUploadLogs,
-					},
-				]);
-				const releaseId = await deployLegacy(
-					docker,
-					logger,
-					token,
-					username,
-					url,
-					options,
-				);
-
-				release = await sdk.models.release.get(releaseId, {
-					$select: ['commit'],
-				});
-			} else {
-				const [userId, auth, apiEndpoint] = await Promise.all([
-					sdk.auth.getUserId(),
-					sdk.auth.getToken(),
-					sdk.settings.get('apiUrl'),
-				]);
-				release = await $deployProject(
-					docker,
-					logger,
-					project.composition,
-					images,
-					opts.app.id,
-					userId,
-					`Bearer ${auth}`,
-					apiEndpoint,
-					!opts.shouldUploadLogs,
-				);
-			}
-
-			logger.outputDeferredMessages();
-			logger.logSuccess('Deploy succeeded!');
-			logger.logSuccess(`Release: ${release.commit}`);
-			console.log();
-			console.log(doodles.getDoodle()); // Show charlie
-			console.log();
-		} catch (err) {
-			logger.logError('Deploy failed');
-			throw err;
-		}
-	}
-}

lib/commands/device/move.ts

@@ -1,174 +0,0 @@
-/**
- * @license
- * Copyright 2016-2020 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-import { flags } from '@oclif/command';
-import type { IArg } from '@oclif/parser/lib/args';
-import type { Application, BalenaSDK } from 'balena-sdk';
-import Command from '../../command';
-import * as cf from '../../utils/common-flags';
-import { expandForAppName } from '../../utils/helpers';
-import { getBalenaSdk, stripIndent } from '../../utils/lazy';
-import { tryAsInteger } from '../../utils/validation';
-import { ExpectedError } from '../../errors';
-
-interface ExtendedDevice extends DeviceWithDeviceType {
-	application_name?: string;
-}
-
-interface FlagsDef {
-	application?: string;
-	app?: string;
-	help: void;
-}
-
-interface ArgsDef {
-	uuid: string;
-}
-
-export default class DeviceMoveCmd extends Command {
-	public static description = stripIndent`
-		Move one or more devices to another application.
-
-		Move one or more devices to another application.
-
-		Note, if the application option is omitted it will be prompted
-		for interactively.
-		`;
-	public static examples = [
-		'$ balena device move 7cf02a6',
-		'$ balena device move 7cf02a6,dc39e52',
-		'$ balena device move 7cf02a6 --application MyNewApp',
-	];
-
-	public static args: Array<IArg<any>> = [
-		{
-			name: 'uuid',
-			description:
-				'comma-separated list (no blank spaces) of device UUIDs to be moved',
-			required: true,
-		},
-	];
-
-	public static usage = 'device move <uuid(s)>';
-
-	public static flags: flags.Input<FlagsDef> = {
-		application: cf.application,
-		app: cf.app,
-		help: cf.help,
-	};
-
-	public static authenticated = true;
-
-	public async run() {
-		const { args: params, flags: options } = this.parse<FlagsDef, ArgsDef>(
-			DeviceMoveCmd,
-		);
-
-		const balena = getBalenaSdk();
-
-		options.application = options.application || options.app;
-		delete options.app;
-
-		// Parse ids string into array of correct types
-		const deviceIds: Array<string | number> = params.uuid
-			.split(',')
-			.map((id) => tryAsInteger(id));
-
-		// Get devices
-		const devices = await Promise.all(
-			deviceIds.map(
-				(uuid) =>
-					balena.models.device.get(uuid, expandForAppName) as Promise<
-						ExtendedDevice
-					>,
-			),
-		);
-
-		// Map application name for each device
-		for (const device of devices) {
-			const belongsToApplication = device.belongs_to__application as Application[];
-			device.application_name = belongsToApplication?.[0]
-				? belongsToApplication[0].app_name
-				: 'N/a';
-		}
-
-		// Get destination application
-		const application =
-			options.application ||
-			(await this.interactivelySelectApplication(balena, devices));
-
-		// Move each device
-		for (const uuid of deviceIds) {
-			try {
-				await balena.models.device.move(uuid, tryAsInteger(application));
-				console.info(`${uuid} was moved to ${application}`);
-			} catch (err) {
-				console.info(`${err.message}, uuid: ${uuid}`);
-				process.exitCode = 1;
-			}
-		}
-	}
-
-	async interactivelySelectApplication(
-		balena: BalenaSDK,
-		devices: ExtendedDevice[],
-	) {
-		const [deviceDeviceTypes, deviceTypes] = await Promise.all([
-			Promise.all(
-				devices.map((device) =>
-					balena.models.device.getManifestBySlug(
-						device.is_of__device_type[0].slug,
-					),
-				),
-			),
-			balena.models.config.getDeviceTypes(),
-		]);
-
-		const compatibleDeviceTypes = deviceTypes.filter((dt) =>
-			deviceDeviceTypes.every(
-				(deviceDeviceType) =>
-					balena.models.os.isArchitectureCompatibleWith(
-						deviceDeviceType.arch,
-						dt.arch,
-					) &&
-					!!dt.isDependent === !!deviceDeviceType.isDependent &&
-					dt.state !== 'DISCONTINUED',
-			),
-		);
-
-		const patterns = await import('../../utils/patterns');
-		try {
-			const application = await patterns.selectApplication(
-				(app) =>
-					compatibleDeviceTypes.some(
-						(dt) => dt.slug === app.is_for__device_type[0].slug,
-					) &&
-					// @ts-ignore using the extended device object prop
-					devices.some((device) => device.application_name !== app.app_name),
-				true,
-			);
-			return application;
-		} catch (err) {
-			if (deviceDeviceTypes.length) {
-				throw new ExpectedError(
-					`${err.message}\nDo all devices have a compatible architecture?`,
-				);
-			}
-			throw err;
-		}
-	}
-}

lib/commands/device/os-update.ts

@@ -1,150 +0,0 @@
-/**
- * @license
- * Copyright 2016-2020 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-import { flags } from '@oclif/command';
-import type { IArg } from '@oclif/parser/lib/args';
-import Command from '../../command';
-import * as cf from '../../utils/common-flags';
-import { getBalenaSdk, stripIndent, getCliForm } from '../../utils/lazy';
-import { tryAsInteger } from '../../utils/validation';
-import type { Device } from 'balena-sdk';
-import { ExpectedError } from '../../errors';
-
-interface FlagsDef {
-	version?: string;
-	yes: boolean;
-	help: void;
-}
-
-interface ArgsDef {
-	uuid: string;
-}
-
-export default class DeviceOsUpdateCmd extends Command {
-	public static description = stripIndent`
-		Start a Host OS update for a device.
-
-		Start a Host OS update for a device.
-
-		Note this command will ask for confirmation interactively.
-		This can be avoided by passing the \`--yes\` option.
-
-		Requires balenaCloud; will not work with openBalena or standalone balenaOS.
-		`;
-	public static examples = [
-		'$ balena device os-update 23c73a1',
-		'$ balena device os-update 23c73a1 --version 2.31.0+rev1.prod',
-	];
-
-	public static args: Array<IArg<any>> = [
-		{
-			name: 'uuid',
-			description: 'the uuid of the device to update',
-			parse: (dev) => tryAsInteger(dev),
-			required: true,
-		},
-	];
-
-	public static usage = 'device os-update <uuid>';
-
-	public static flags: flags.Input<FlagsDef> = {
-		version: flags.string({
-			description: 'a balenaOS version',
-		}),
-		yes: cf.yes,
-		help: cf.help,
-	};
-
-	public static authenticated = true;
-
-	public async run() {
-		const { args: params, flags: options } = this.parse<FlagsDef, ArgsDef>(
-			DeviceOsUpdateCmd,
-		);
-
-		const sdk = getBalenaSdk();
-
-		// Get device info
-		const {
-			uuid,
-			is_of__device_type,
-			os_version,
-			os_variant,
-		} = (await sdk.models.device.get(params.uuid, {
-			$select: ['uuid', 'os_version', 'os_variant'],
-			$expand: {
-				is_of__device_type: {
-					$select: 'slug',
-				},
-			},
-		})) as DeviceWithDeviceType;
-
-		// Get current device OS version
-		const currentOsVersion = sdk.models.device.getOsVersion({
-			os_version,
-			os_variant,
-		} as Device);
-		if (!currentOsVersion) {
-			throw new ExpectedError(
-				'The current os version of the device is not available',
-			);
-		}
-
-		// Get supported OS update versions
-		const hupVersionInfo = await sdk.models.os.getSupportedOsUpdateVersions(
-			is_of__device_type[0].slug,
-			currentOsVersion,
-		);
-		if (hupVersionInfo.versions.length === 0) {
-			throw new ExpectedError(
-				'There are no available Host OS update targets for this device',
-			);
-		}
-
-		// Get target OS version
-		let targetOsVersion = options.version;
-		if (targetOsVersion != null) {
-			if (!hupVersionInfo.versions.includes(targetOsVersion)) {
-				throw new ExpectedError(
-					`The provided version ${targetOsVersion} is not in the Host OS update targets for this device`,
-				);
-			}
-		} else {
-			targetOsVersion = await getCliForm().ask({
-				message: 'Target OS version',
-				type: 'list',
-				choices: hupVersionInfo.versions.map((version) => ({
-					name:
-						hupVersionInfo.recommended === version
-							? `${version} (recommended)`
-							: version,
-					value: version,
-				})),
-			});
-		}
-
-		const patterns = await import('../../utils/patterns');
-		// Confirm and start update
-		await patterns.confirm(
-			options.yes || false,
-			'Host OS updates require a device restart when they complete. Are you sure you want to proceed?',
-		);
-
-		await sdk.models.device.startOsUpdate(uuid, targetOsVersion);
-		await patterns.awaitDeviceOsUpdate(uuid, targetOsVersion);
-	}
-}

lib/commands/device/register.ts

@@ -1,82 +0,0 @@
-/**
- * @license
- * Copyright 2016-2020 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-import { flags } from '@oclif/command';
-import type { IArg } from '@oclif/parser/lib/args';
-import Command from '../../command';
-import * as cf from '../../utils/common-flags';
-import { getBalenaSdk, stripIndent } from '../../utils/lazy';
-import { tryAsInteger } from '../../utils/validation';
-
-interface FlagsDef {
-	uuid?: string;
-	help: void;
-}
-
-interface ArgsDef {
-	application: string;
-}
-
-export default class DeviceRegisterCmd extends Command {
-	public static description = stripIndent`
-		Register a device.
-
-		Register a device to an application.
-		`;
-	public static examples = [
-		'$ balena device register MyApp',
-		'$ balena device register MyApp --uuid <uuid>',
-	];
-
-	public static args: Array<IArg<any>> = [
-		{
-			name: 'application',
-			description: 'the name or id of application to register device with',
-			parse: (app) => tryAsInteger(app),
-			required: true,
-		},
-	];
-
-	public static usage = 'device register <application>';
-
-	public static flags: flags.Input<FlagsDef> = {
-		uuid: flags.string({
-			description: 'custom uuid',
-			char: 'u',
-		}),
-		help: cf.help,
-	};
-
-	public static authenticated = true;
-
-	public async run() {
-		const { args: params, flags: options } = this.parse<FlagsDef, ArgsDef>(
-			DeviceRegisterCmd,
-		);
-
-		const balena = getBalenaSdk();
-
-		const application = await balena.models.application.get(params.application);
-		const uuid = options.uuid ?? balena.models.device.generateUniqueKey();
-
-		console.info(`Registering to ${application.app_name}: ${uuid}`);
-
-		const result = await balena.models.device.register(application.id, uuid);
-
-		return result && result.uuid;
-	}
-}

lib/commands/devices/index.ts

@@ -1,136 +0,0 @@
-/**
- * @license
- * Copyright 2016-2020 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-import { flags } from '@oclif/command';
-import Command from '../../command';
-import * as cf from '../../utils/common-flags';
-import { expandForAppName } from '../../utils/helpers';
-import { getBalenaSdk, getVisuals, stripIndent } from '../../utils/lazy';
-import { tryAsInteger } from '../../utils/validation';
-import type { Application } from 'balena-sdk';
-
-interface ExtendedDevice extends DeviceWithDeviceType {
-	dashboard_url?: string;
-	application_name?: string;
-}
-
-interface FlagsDef {
-	application?: string;
-	app?: string;
-	help: void;
-	json: boolean;
-}
-
-export default class DevicesCmd extends Command {
-	public static description = stripIndent`
-		List all devices.
-
-		list all devices that belong to you.
-
-		You can filter the devices by application by using the \`--application\` option.
-
-		The --json option is recommended when scripting the output of this command,
-		because field names are less likely to change in JSON format and because it
-		better represents data types like arrays and empty strings. The 'jq' utility
-		may also be helpful in shell scripts (https://stedolan.github.io/jq/manual/).
-	`;
-	public static examples = [
-		'$ balena devices',
-		'$ balena devices --application MyApp',
-		'$ balena devices --app MyApp',
-		'$ balena devices -a MyApp',
-	];
-
-	public static usage = 'devices';
-
-	public static flags: flags.Input<FlagsDef> = {
-		application: cf.application,
-		app: cf.app,
-		help: cf.help,
-		json: flags.boolean({
-			char: 'j',
-			description: 'produce JSON output instead of tabular output',
-		}),
-	};
-
-	public static primary = true;
-
-	public static authenticated = true;
-
-	public async run() {
-		const { flags: options } = this.parse<FlagsDef, {}>(DevicesCmd);
-
-		const balena = getBalenaSdk();
-
-		// Consolidate application options
-		options.application = options.application || options.app;
-		delete options.app;
-
-		let devices;
-
-		if (options.application != null) {
-			devices = (await balena.models.device.getAllByApplication(
-				tryAsInteger(options.application),
-				expandForAppName,
-			)) as ExtendedDevice[];
-		} else {
-			devices = (await balena.models.device.getAll(
-				expandForAppName,
-			)) as ExtendedDevice[];
-		}
-
-		devices = devices.map(function (device) {
-			device.dashboard_url = balena.models.device.getDashboardUrl(device.uuid);
-
-			const belongsToApplication = device.belongs_to__application as Application[];
-			device.application_name = belongsToApplication?.[0]
-				? belongsToApplication[0].app_name
-				: 'N/a';
-
-			device.uuid = device.uuid.slice(0, 7);
-
-			// @ts-ignore
-			device.device_type = device.is_of__device_type[0].slug;
-			return device;
-		});
-
-		const fields = [
-			'id',
-			'uuid',
-			'device_name',
-			'device_type',
-			'application_name',
-			'status',
-			'is_online',
-			'supervisor_version',
-			'os_version',
-			'dashboard_url',
-		];
-		if (options.json) {
-			const _ = await import('lodash');
-			console.log(
-				JSON.stringify(
-					devices.map((device) => _.pick(device, fields)),
-					null,
-					4,
-				),
-			);
-		} else {
-			console.log(getVisuals().table.horizontal(devices, fields));
-		}
-	}
-}

lib/commands/devices/supported.ts

@@ -1,119 +0,0 @@
-/**
- * @license
- * Copyright 2016-2019 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-import { flags } from '@oclif/command';
-import type * as SDK from 'balena-sdk';
-import * as _ from 'lodash';
-import Command from '../../command';
-
-import * as cf from '../../utils/common-flags';
-import { getBalenaSdk, getVisuals, stripIndent } from '../../utils/lazy';
-import { CommandHelp } from '../../utils/oclif-utils';
-
-interface FlagsDef {
-	discontinued: boolean;
-	help: void;
-	json?: boolean;
-	verbose?: boolean;
-}
-
-export default class DevicesSupportedCmd extends Command {
-	public static description = stripIndent`
-		List the supported device types (like 'raspberrypi3' or 'intel-nuc').
-
-		List the supported device types (like 'raspberrypi3' or 'intel-nuc').
-
-		The --verbose option adds extra columns/fields to the output, including the
-		"STATE" column whose values are one of 'new', 'released' or 'discontinued'.
-		However, 'discontinued' device types are only listed if the '--discontinued'
-		option is used.
-
-		The --json option is recommended when scripting the output of this command,
-		because the JSON format is less likely to change and it better represents data
-		types like lists and empty strings (for example, the ALIASES column contains a
-		list of zero or more values). The 'jq' utility may be helpful in shell scripts
-		(https://stedolan.github.io/jq/manual/).
-`;
-	public static examples = [
-		'$ balena devices supported',
-		'$ balena devices supported --verbose',
-		'$ balena devices supported -vj',
-	];
-
-	public static usage = (
-		'devices supported ' +
-		new CommandHelp({ args: DevicesSupportedCmd.args }).defaultUsage()
-	).trim();
-
-	public static flags: flags.Input<FlagsDef> = {
-		discontinued: flags.boolean({
-			description: 'include "discontinued" device types',
-		}),
-		help: cf.help,
-		json: flags.boolean({
-			char: 'j',
-			description: 'produce JSON output instead of tabular output',
-		}),
-		verbose: flags.boolean({
-			char: 'v',
-			description:
-				'add extra columns in the tabular output (ALIASES, ARCH, STATE)',
-		}),
-	};
-
-	public async run() {
-		const { flags: options } = this.parse<FlagsDef, {}>(DevicesSupportedCmd);
-		const dts = await getBalenaSdk().models.config.getDeviceTypes();
-		let deviceTypes: Array<Partial<SDK.DeviceTypeJson.DeviceType>> = dts.map(
-			(d) => {
-				if (d.aliases && d.aliases.length) {
-					// remove aliases that are equal to the slug
-					d.aliases = d.aliases.filter((alias: string) => alias !== d.slug);
-					if (!options.json) {
-						// stringify the aliases array with commas and spaces
-						d.aliases = [d.aliases.join(', ')];
-					}
-				} else {
-					// ensure it is always an array (for the benefit of JSON output)
-					d.aliases = [];
-				}
-				return d;
-			},
-		);
-		if (!options.discontinued) {
-			deviceTypes = deviceTypes.filter((dt) => dt.state !== 'DISCONTINUED');
-		}
-		const fields = options.verbose
-			? ['slug', 'aliases', 'arch', 'state', 'name']
-			: ['slug', 'aliases', 'arch', 'name'];
-		deviceTypes = _.sortBy(
-			deviceTypes.map((d) => {
-				const picked = _.pick(d, fields);
-				// 'BETA' renamed to 'NEW'
-				picked.state = picked.state === 'BETA' ? 'NEW' : picked.state;
-				return picked;
-			}),
-			fields,
-		);
-		if (options.json) {
-			console.log(JSON.stringify(deviceTypes, null, 4));
-		} else {
-			const visuals = getVisuals();
-			const output = await visuals.table.horizontal(deviceTypes, fields);
-			console.log(output);
-		}
-	}
-}

lib/commands/envs.ts

@@ -1,442 +0,0 @@
-/**
- * @license
- * Copyright 2016-2019 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-import { flags } from '@oclif/command';
-import type * as SDK from 'balena-sdk';
-import * as _ from 'lodash';
-import Command from '../command';
-
-import { ExpectedError } from '../errors';
-import * as cf from '../utils/common-flags';
-import { getBalenaSdk, getVisuals, stripIndent } from '../utils/lazy';
-import { CommandHelp } from '../utils/oclif-utils';
-import { isV12 } from '../utils/version';
-
-interface FlagsDef {
-	all?: boolean; // whether to include application-wide, device-wide variables //TODO: REMOVE
-	application?: string; // application name
-	config: boolean;
-	device?: string; // device UUID
-	json: boolean;
-	help: void;
-	service?: string; // service name
-	verbose: boolean;
-}
-
-interface EnvironmentVariableInfo extends SDK.EnvironmentVariableBase {
-	appName?: string | null; // application name
-	deviceUUID?: string; // device UUID
-	serviceName?: string; // service name
-}
-
-interface DeviceServiceEnvironmentVariableInfo
-	extends SDK.DeviceServiceEnvironmentVariable {
-	appName?: string; // application name
-	deviceUUID?: string; // device UUID
-	serviceName?: string; // service name
-}
-
-interface ServiceEnvironmentVariableInfo
-	extends SDK.ServiceEnvironmentVariable {
-	appName?: string; // application name
-	deviceUUID?: string; // device UUID
-	serviceName?: string; // service name
-}
-
-export default class EnvsCmd extends Command {
-	public static description = isV12()
-		? stripIndent`
-		List the environment or config variables of an application, device or service.
-
-		List the environment or configuration variables of an application, device or
-		service, as selected by the respective command-line options. (A service is
-		an application container in a "microservices" application.)
-
-		The results include application-wide (fleet), device-wide (multiple services on
-		a device) and service-specific variables that apply to the selected application,
-		device or service. It can be thought of as including "inherited" variables;
-		for example, a service inherits device-wide variables, and a device inherits
-		application-wide variables.
-
-		The printed output may include DEVICE and/or SERVICE columns to distinguish
-		between application-wide, device-specific and service-specific variables.
-		An asterisk in these columns indicates that the variable applies to
-		"all devices" or "all services".
-
-		The --config option is used to list "configuration variables" that control
-		balena platform features, as opposed to custom environment variables defined
-		by the user. The --config and the --service options are mutually exclusive
-		because configuration variables cannot be set for specific services.
-
-		The --json option is recommended when scripting the output of this command,
-		because the JSON format is less likely to change and it better represents data
-		types like lists and empty strings. The 'jq' utility may be helpful in shell
-		scripts (https://stedolan.github.io/jq/manual/). When --json is used, an empty
-		JSON array ([]) is printed instead of an error message when no variables exist
-		for the given query. When querying variables for a device, note that the
-		application name may be null in JSON output (or 'N/A' in tabular output) if the
-		application linked to the device is no longer accessible by the current user
-		(for example, in case the current user has been removed from the application
-		by its owner).
-`
-		: stripIndent`
-		List the environment or config variables of an application, device or service.
-
-		List the environment or configuration variables of an application, device or
-		service, as selected by the respective command-line options. (A service is
-		an application container in a "microservices" application.)
-
-		The --config option is used to list "configuration variables" that control
-		balena platform features, as opposed to custom environment variables defined
-		by the user. The --config and the --service options are mutually exclusive
-		because configuration variables cannot be set for specific services.
-
-		The --all option is used to include application-wide (fleet), device-wide
-		(multiple services on a device) and service-specific variables that apply to
-		the selected application, device or service. It can be thought of as including
-		"inherited" variables: for example, a service inherits device-wide variables,
-		and a device inherits application-wide variables. Variables are still filtered
-		out by type with the --config option, such that configuration and non-
-		configuration variables are never listed together.
-
-		When the --all option is used, the printed output may include DEVICE and/or
-		SERVICE columns to distinguish between application-wide, device-specific and
-		service-specific variables. An asterisk in these columns indicates that the
-		variable applies to "all devices" or "all services".
-
-		The --json option is recommended when scripting the output of this command,
-		because the JSON format is less likely to change and it better represents data
-		types like lists and empty strings. The 'jq' utility may be helpful in shell
-		scripts (https://stedolan.github.io/jq/manual/). When --json is used, an empty
-		JSON array ([]) is printed instead of an error message when no variables exist
-		for the given query. When querying variables for a device, note that the
-		application name may be null in JSON output (or 'N/A' in tabular output) if the
-		application linked to the device is no longer accessible by the current user
-		(for example, in case the current user has been removed from the application
-		by its owner).
-`;
-	public static examples = isV12()
-		? [
-				'$ balena envs --application MyApp',
-				'$ balena envs --application MyApp --json',
-				'$ balena envs --application MyApp --service MyService',
-				'$ balena envs --application MyApp --service MyService',
-				'$ balena envs --application MyApp --config',
-				'$ balena envs --device 7cf02a6',
-				'$ balena envs --device 7cf02a6 --json',
-				'$ balena envs --device 7cf02a6 --config --json',
-				'$ balena envs --device 7cf02a6 --service MyService',
-		  ]
-		: [
-				'$ balena envs --application MyApp',
-				'$ balena envs --application MyApp --all --json',
-				'$ balena envs --application MyApp --service MyService',
-				'$ balena envs --application MyApp --all --service MyService',
-				'$ balena envs --application MyApp --config',
-				'$ balena envs --device 7cf02a6',
-				'$ balena envs --device 7cf02a6 --all --json',
-				'$ balena envs --device 7cf02a6 --config --all --json',
-				'$ balena envs --device 7cf02a6 --all --service MyService',
-		  ];
-
-	public static usage = (
-		'envs ' + new CommandHelp({ args: EnvsCmd.args }).defaultUsage()
-	).trim();
-
-	public static flags: flags.Input<FlagsDef> = {
-		...(isV12()
-			? {
-					all: flags.boolean({
-						description: stripIndent`
-				No-op since balenaCLI v12.0.0.`,
-						hidden: true,
-					}),
-			  }
-			: {
-					all: flags.boolean({
-						description: stripIndent`
-				include app-wide, device-wide variables that apply to the selected device or service.
-				Variables are still filtered out by type with the --config option.`,
-					}),
-			  }),
-		application: { exclusive: ['device'], ...cf.application },
-		config: flags.boolean({
-			char: 'c',
-			description: 'show configuration variables only',
-			exclusive: ['service'],
-		}),
-		device: { exclusive: ['application'], ...cf.device },
-		help: cf.help,
-		json: flags.boolean({
-			char: 'j',
-			description: 'produce JSON output instead of tabular output',
-		}),
-		verbose: cf.verbose,
-		service: { exclusive: ['config'], ...cf.service },
-	};
-
-	public async run() {
-		const { flags: options } = this.parse<FlagsDef, {}>(EnvsCmd);
-		const variables: EnvironmentVariableInfo[] = [];
-
-		options.all = options.all || isV12();
-
-		await Command.checkLoggedIn();
-
-		if (!options.application && !options.device) {
-			throw new ExpectedError('You must specify an application or device');
-		}
-
-		const balena = getBalenaSdk();
-
-		let appName = options.application;
-		let fullUUID: string | undefined; // as oppposed to the short, 7-char UUID
-
-		if (options.device) {
-			const { getDeviceAndMaybeAppFromUUID } = await import('../utils/cloud');
-			const [device, app] = await getDeviceAndMaybeAppFromUUID(
-				balena,
-				options.device,
-				['uuid'],
-				['app_name'],
-			);
-			fullUUID = device.uuid;
-			if (app) {
-				appName = app.app_name;
-			}
-		}
-		if (appName && options.service) {
-			await validateServiceName(balena, options.service, appName);
-		}
-		if (options.application || options.all) {
-			variables.push(...(await getAppVars(balena, appName, options)));
-		}
-		if (fullUUID) {
-			variables.push(
-				...(await getDeviceVars(balena, fullUUID, appName, options)),
-			);
-		}
-		if (!options.json && variables.length === 0) {
-			const target =
-				(options.service ? `service "${options.service}" of ` : '') +
-				(options.application
-					? `application "${options.application}"`
-					: `device "${options.device}"`);
-			throw new ExpectedError(`No environment variables found for ${target}`);
-		}
-
-		await this.printVariables(variables, options);
-	}
-
-	protected async printVariables(
-		varArray: EnvironmentVariableInfo[],
-		options: FlagsDef,
-	) {
-		const fields = ['id', 'name', 'value'];
-
-		if (options.all) {
-			// Replace undefined app names with 'N/A' or null
-			varArray = varArray.map((i: EnvironmentVariableInfo) => {
-				i.appName = i.appName || (options.json ? null : 'N/A');
-				return i;
-			});
-
-			fields.push(options.json ? 'appName' : 'appName => APPLICATION');
-			if (options.device) {
-				fields.push(options.json ? 'deviceUUID' : 'deviceUUID => DEVICE');
-			}
-			if (!options.config) {
-				fields.push(options.json ? 'serviceName' : 'serviceName => SERVICE');
-			}
-		}
-
-		if (options.json) {
-			this.log(
-				stringifyVarArray<SDK.EnvironmentVariableBase>(varArray, fields),
-			);
-		} else {
-			this.log(
-				getVisuals().table.horizontal(
-					_.sortBy(varArray, (v: SDK.EnvironmentVariableBase) => v.name),
-					fields,
-				),
-			);
-		}
-	}
-}
-
-async function validateServiceName(
-	sdk: SDK.BalenaSDK,
-	serviceName: string,
-	appName: string,
-) {
-	const services = await sdk.models.service.getAllByApplication(appName, {
-		$filter: { service_name: serviceName },
-	});
-	if (services.length === 0) {
-		throw new ExpectedError(
-			`Service "${serviceName}" not found for application "${appName}"`,
-		);
-	}
-}
-
-/**
- * Fetch application-wide config / env / service vars.
- * If options.application is undefined, an attempt is made to obtain the
- * application name from the device UUID (options.device). If this attempt
- * fails because the device does not belong to any application, an emtpy
- * array is returned.
- */
-async function getAppVars(
-	sdk: SDK.BalenaSDK,
-	appName: string | undefined,
-	options: FlagsDef,
-): Promise<EnvironmentVariableInfo[]> {
-	const appVars: EnvironmentVariableInfo[] = [];
-	if (!appName) {
-		return appVars;
-	}
-	if (options.config || options.all || !options.service) {
-		const vars = await sdk.models.application[
-			options.config ? 'configVar' : 'envVar'
-		].getAllByApplication(appName);
-		fillInInfoFields(vars, appName);
-		appVars.push(...vars);
-	}
-	if (!options.config && (options.service || options.all)) {
-		const pineOpts: SDK.PineOptions<SDK.ServiceEnvironmentVariable> = {
-			$expand: {
-				service: {},
-			},
-		};
-		if (options.service) {
-			pineOpts.$filter = {
-				service: {
-					service_name: options.service,
-				},
-			};
-		}
-		const serviceVars = await sdk.models.service.var.getAllByApplication(
-			appName,
-			pineOpts,
-		);
-		fillInInfoFields(serviceVars, appName);
-		appVars.push(...serviceVars);
-	}
-	return appVars;
-}
-
-/**
- * Fetch config / env / service vars when the '--device' option is provided.
- * Precondition: options.device must be defined.
- */
-async function getDeviceVars(
-	sdk: SDK.BalenaSDK,
-	fullUUID: string,
-	appName: string | undefined,
-	options: FlagsDef,
-): Promise<EnvironmentVariableInfo[]> {
-	const printedUUID = options.json ? fullUUID : options.device!;
-	const deviceVars: EnvironmentVariableInfo[] = [];
-	if (options.config) {
-		const deviceConfigVars = await sdk.models.device.configVar.getAllByDevice(
-			fullUUID,
-		);
-		fillInInfoFields(deviceConfigVars, appName, printedUUID);
-		deviceVars.push(...deviceConfigVars);
-	} else {
-		if (options.service || options.all) {
-			const pineOpts: SDK.PineOptions<SDK.DeviceServiceEnvironmentVariable> = {
-				$expand: {
-					service_install: {
-						$expand: 'installs__service',
-					},
-				},
-			};
-			if (options.service) {
-				pineOpts.$filter = {
-					service_install: {
-						installs__service: { service_name: options.service },
-					},
-				};
-			}
-			const deviceServiceVars = await sdk.models.device.serviceVar.getAllByDevice(
-				fullUUID,
-				pineOpts,
-			);
-			fillInInfoFields(deviceServiceVars, appName, printedUUID);
-			deviceVars.push(...deviceServiceVars);
-		}
-		if (!options.service || options.all) {
-			const deviceEnvVars = await sdk.models.device.envVar.getAllByDevice(
-				fullUUID,
-			);
-			fillInInfoFields(deviceEnvVars, appName, printedUUID);
-			deviceVars.push(...deviceEnvVars);
-		}
-	}
-	return deviceVars;
-}
-
-/**
- * For each env var object in varArray, fill in its top-level serviceName
- * and deviceUUID fields. An asterisk is used to indicate that the variable
- * applies to "all services" or "all devices".
- */
-function fillInInfoFields(
-	varArray:
-		| EnvironmentVariableInfo[]
-		| DeviceServiceEnvironmentVariableInfo[]
-		| ServiceEnvironmentVariableInfo[],
-	appName?: string,
-	deviceUUID?: string,
-) {
-	for (const envVar of varArray) {
-		if ('service' in envVar) {
-			// envVar is of type ServiceEnvironmentVariableInfo
-			envVar.serviceName = (envVar.service as SDK.Service[])[0]?.service_name;
-		} else if ('service_install' in envVar) {
-			// envVar is of type DeviceServiceEnvironmentVariableInfo
-			envVar.serviceName = ((envVar.service_install as SDK.ServiceInstall[])[0]
-				?.installs__service as SDK.Service[])[0]?.service_name;
-		}
-		envVar.appName = appName;
-		envVar.serviceName = envVar.serviceName || '*';
-		envVar.deviceUUID = deviceUUID || '*';
-	}
-}
-
-/**
- * Transform each object (item) of varArray to preserve only the
- * fields (keys) listed in the fields argument.
- */
-function stringifyVarArray<T = Dictionary<any>>(
-	varArray: T[],
-	fields: string[],
-): string {
-	const transformed = varArray.map((o: Dictionary<any>) =>
-		_.transform(
-			o,
-			(result, value, key) => {
-				if (fields.includes(key)) {
-					result[key] = value;
-				}
-			},
-			{} as Dictionary<any>,
-		),
-	);
-	return JSON.stringify(transformed, null, 4);
-}

lib/commands/internal/scandevices.ts

@@ -1,55 +0,0 @@
-/**
- * @license
- * Copyright 2016-2020 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-import Command from '../../command';
-import { stripIndent } from '../../utils/lazy';
-
-// 'Internal' commands are called during the execution of other commands.
-// `scandevices` is called during by `join`,`leave'.
-// TODO: These should be refactored to modules/functions, and removed
-// 	See previous `internal sudo` refactor:
-//    - https://github.com/balena-io/balena-cli/pull/1455/files
-//    - https://github.com/balena-io/balena-cli/pull/1455#discussion_r334308357
-//    - https://github.com/balena-io/balena-cli/pull/1455#discussion_r334308526
-
-export default class ScandevicesCmd extends Command {
-	public static description = stripIndent`
-		Scan for local balena-enabled devices and show a picker to choose one.
-
-		Don't use this command directly!
-	`;
-
-	public static usage = 'internal scandevices';
-
-	public static root = true;
-	public static hidden = true;
-
-	public async run() {
-		const { forms } = await import('balena-sync');
-		try {
-			const hostnameOrIp = await forms.selectLocalBalenaOsDevice();
-			return console.error(`==> Selected device: ${hostnameOrIp}`);
-		} catch (e) {
-			if (e.message.toLowerCase().includes('could not find any')) {
-				const { ExpectedError } = await import('../../errors');
-				throw new ExpectedError(e);
-			} else {
-				throw e;
-			}
-		}
-	}
-}

lib/commands/os/download.ts

@@ -1,96 +0,0 @@
-/**
- * @license
- * Copyright 2016-2020 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-import { flags } from '@oclif/command';
-import Command from '../../command';
-import * as cf from '../../utils/common-flags';
-import { stripIndent } from '../../utils/lazy';
-
-interface FlagsDef {
-	output: string;
-	version?: string;
-	help: void;
-}
-
-interface ArgsDef {
-	type: string;
-}
-
-export default class OsDownloadCmd extends Command {
-	public static description = stripIndent`
-		Download an unconfigured OS image.
-
-		Download an unconfigured OS image for a certain device type.
-		Check available types with \`balena devices supported\`
-
-		Note: Currently this command only works with balenaCloud, not openBalena.
-		If using openBalena, please download the OS from: https://www.balena.io/os/
-
-		If version is not specified the newest stable (non-pre-release) version of OS
-		is downloaded (if available), otherwise the newest version (if all existing
-		versions for the given device type are pre-release).
-
-		You can pass \`--version menu\` to pick the OS version from the interactive menu
-		of all available versions.
-`;
-	public static examples = [
-		'$ balena os download raspberrypi3 -o ../foo/bar/raspberry-pi.img',
-		'$ balena os download raspberrypi3 -o ../foo/bar/raspberry-pi.img --version 1.24.1',
-		'$ balena os download raspberrypi3 -o ../foo/bar/raspberry-pi.img --version ^1.20.0',
-		'$ balena os download raspberrypi3 -o ../foo/bar/raspberry-pi.img --version latest',
-		'$ balena os download raspberrypi3 -o ../foo/bar/raspberry-pi.img --version default',
-		'$ balena os download raspberrypi3 -o ../foo/bar/raspberry-pi.img --version menu',
-	];
-
-	public static args = [
-		{
-			name: 'type',
-			description: 'the device type',
-			required: true,
-		},
-	];
-
-	public static usage = 'os download <type>';
-
-	public static flags: flags.Input<FlagsDef> = {
-		output: flags.string({
-			description: 'output path',
-			char: 'o',
-			required: true,
-		}),
-		version: flags.string({
-			description: stripIndent`
-				exact version number, or a valid semver range,
-				or 'latest' (includes pre-releases),
-				or 'default' (excludes pre-releases if at least one stable version is available),
-				or 'recommended' (excludes pre-releases, will fail if only pre-release versions are available),
-				or 'menu' (will show the interactive menu)
-				`,
-		}),
-		help: cf.help,
-	};
-
-	public async run() {
-		const { args: params, flags: options } = this.parse<FlagsDef, ArgsDef>(
-			OsDownloadCmd,
-		);
-
-		const { downloadOSImage } = await import('../../utils/cloud');
-
-		await downloadOSImage(params.type, options.output, options.version);
-	}
-}

lib/commands/os/versions.ts

@@ -1,67 +0,0 @@
-/**
- * @license
- * Copyright 2016-2020 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-import { flags } from '@oclif/command';
-import Command from '../../command';
-import * as cf from '../../utils/common-flags';
-import { getBalenaSdk, stripIndent } from '../../utils/lazy';
-
-interface FlagsDef {
-	help: void;
-}
-
-interface ArgsDef {
-	type: string;
-}
-
-export default class OsVersionsCmd extends Command {
-	public static description = stripIndent`
-		Show available balenaOS versions for the given device type.
-
-		Show the available balenaOS versions for the given device type.
-		Check available types with \`balena devices supported\`.
-	`;
-
-	public static examples = ['$ balena os versions raspberrypi3'];
-
-	public static args = [
-		{
-			name: 'type',
-			description: 'device type',
-			required: true,
-		},
-	];
-
-	public static usage = 'os versions <type>';
-
-	public static flags: flags.Input<FlagsDef> = {
-		help: cf.help,
-	};
-
-	public async run() {
-		const { args: params } = this.parse<FlagsDef, ArgsDef>(OsVersionsCmd);
-
-		const {
-			versions: vs,
-			recommended,
-		} = await getBalenaSdk().models.os.getSupportedVersions(params.type);
-
-		vs.forEach((v) => {
-			console.log(`v${v}` + (v === recommended ? ' (recommended)' : ''));
-		});
-	}
-}

lib/commands/push.ts

@@ -1,419 +0,0 @@
-/**
- * @license
- * Copyright 2016-2020 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-import { flags } from '@oclif/command';
-import Command from '../command';
-import * as cf from '../utils/common-flags';
-import { getBalenaSdk, stripIndent } from '../utils/lazy';
-import { dockerignoreHelp, registrySecretsHelp } from '../utils/messages';
-import type { BalenaSDK, Application, Organization } from 'balena-sdk';
-import { ExpectedError, instanceOf } from '../errors';
-
-enum BuildTarget {
-	Cloud,
-	Device,
-}
-
-interface FlagsDef {
-	source?: string;
-	emulated: boolean;
-	dockerfile?: string; // DeviceDeployOptions.dockerfilePath (alternative Dockerfile)
-	nocache?: boolean;
-	pull?: boolean;
-	'noparent-check'?: boolean;
-	'registry-secrets'?: string;
-	gitignore?: boolean;
-	nogitignore?: boolean;
-	nolive?: boolean;
-	detached?: boolean;
-	service?: string[];
-	system?: boolean;
-	env?: string[];
-	'convert-eol'?: boolean;
-	'noconvert-eol'?: boolean;
-	'multi-dockerignore'?: boolean;
-	help: void;
-}
-
-interface ArgsDef {
-	applicationOrDevice: string;
-}
-
-export default class PushCmd extends Command {
-	public static description = stripIndent`
-		Start a remote build on the balenaCloud build servers or a local mode device.
-
-		Start a build on the remote balenaCloud builders, or a local mode balena device.
-
-		When building on the balenaCloud servers, the given source directory will be
-		sent to the remote server. This can be used as a drop-in replacement for the
-		"git push" deployment method.
-
-		When building on a local mode device, the given source directory will be
-		built on the device, and the resulting containers will be run on the device.
-		Logs will be streamed back from the device as part of the same invocation.
-		The web dashboard can be used to switch a device to local mode:
-		https://www.balena.io/docs/learn/develop/local-mode/
-		Note that local mode requires a supervisor version of at least v7.21.0.
-		The logs from only a single service can be shown with the --service flag, and
-		showing only the system logs can be achieved with --system. Note that these
-		flags can be used together.
-
-		When pushing to a local device a live session will be started.
-		The project source folder is watched for filesystem events, and changes
-		to files and folders are automatically synchronized to the running
-		containers. The synchronization is only in one direction, from this machine to
-		the device, and changes made on the device itself may be overwritten.
-		This feature requires a device running supervisor version v9.7.0 or greater.
-
-		${registrySecretsHelp.split('\n').join('\n\t\t')}
-
-		${dockerignoreHelp.split('\n').join('\n\t\t')}
-
-		Note: the --service and --env flags must come after the applicationOrDevice
-		parameter, as per examples.
-	`;
-
-	public static examples = [
-		'$ balena push myApp',
-		'$ balena push myApp --source <source directory>',
-		'$ balena push myApp -s <source directory>',
-		'',
-		'$ balena push 10.0.0.1',
-		'$ balena push 10.0.0.1 --source <source directory>',
-		'$ balena push 10.0.0.1 --service my-service',
-		'$ balena push 10.0.0.1 --env MY_ENV_VAR=value --env my-service:SERVICE_VAR=value',
-		'$ balena push 10.0.0.1 --nolive',
-		'',
-		'$ balena push 23c73a1.local --system',
-		'$ balena push 23c73a1.local --system --service my-service',
-	];
-
-	public static args = [
-		{
-			name: 'applicationOrDevice',
-			description: 'application name, or device address (for local pushes)',
-			required: true,
-		},
-	];
-
-	public static usage = 'push <applicationOrDevice>';
-
-	public static flags: flags.Input<FlagsDef> = {
-		source: flags.string({
-			description: stripIndent`
-				Source directory to be sent to balenaCloud or balenaOS device
-				(default: current working dir)`,
-			char: 's',
-		}),
-		emulated: flags.boolean({
-			description: stripIndent`
-				Don't use native ARM servers; force QEMU ARM emulation on Intel x86-64
-				servers during the image build (balenaCloud).`,
-			char: 'e',
-		}),
-		dockerfile: flags.string({
-			description:
-				'Alternative Dockerfile name/path, relative to the source folder',
-		}),
-		nocache: flags.boolean({
-			description: stripIndent`
-				Don't use cached layers of previously built images for this project. This
-				ensures that the latest base image and packages are pulled. Note that build
-				logs may still display the message _"Pulling previous images for caching
-				purposes" (as the cloud builder needs previous images to compute delta
-				updates), but the logs will not display the "Using cache" lines for each
-				build step of a Dockerfile.`,
-			char: 'c',
-		}),
-		pull: flags.boolean({
-			description: stripIndent`
-				When pushing to a local device, force the base images to be pulled again.
-				Currently this option is ignored when pushing to the balenaCloud builders.`,
-		}),
-		'noparent-check': flags.boolean({
-			description: stripIndent`
-				Disable project validation check of 'docker-compose.yml' file in parent folder`,
-		}),
-		'registry-secrets': flags.string({
-			description: stripIndent`
-				Path to a local YAML or JSON file containing Docker registry passwords used
-				to pull base images. Note that if registry-secrets are not provided on the
-				command line, a secrets configuration file from the balena directory will be
-				used (usually $HOME/.balena/secrets.yml|.json)`,
-			char: 'R',
-		}),
-		nolive: flags.boolean({
-			description: stripIndent`
-				Don't run a live session on this push. The filesystem will not be monitored,
-				and changes will not be synchronized to any running containers. Note that both
-				this flag and --detached and required to cause the process to end once the
-				initial build has completed.`,
-		}),
-		detached: flags.boolean({
-			description: stripIndent`
-				When pushing to the cloud, this option will cause the build to start, then
-				return execution back to the shell, with the status and release ID (if
-				applicable).  When pushing to a local mode device, this option will cause
-				the command to not tail application logs when the build has completed.`,
-			char: 'd',
-		}),
-		service: flags.string({
-			description: stripIndent`
-				Reject logs not originating from this service.
-				This can be used in combination with --system and other --service flags.
-				Only valid when pushing to a local mode device.`,
-			multiple: true,
-		}),
-		system: flags.boolean({
-			description: stripIndent`
-				Only show system logs. This can be used in combination with --service.
-				Only valid when pushing to a local mode device.`,
-		}),
-		env: flags.string({
-			description: stripIndent`
-				When performing a push to device, run the built containers with environment
-				variables provided with this argument. Environment variables can be applied
-				to individual services by adding their service name before the argument,
-				separated by a colon, e.g:
-					--env main:MY_ENV=value
-				Note that if the service name cannot be found in the composition, the entire
-				left hand side of the = character will be treated as the variable name.
-			`,
-			multiple: true,
-		}),
-		'convert-eol': flags.boolean({
-			description: 'No-op and deprecated since balenaCLI v12.0.0',
-			char: 'l',
-			hidden: true,
-		}),
-		'noconvert-eol': flags.boolean({
-			description: `Don't convert line endings from CRLF (Windows format) to LF (Unix format).`,
-		}),
-		'multi-dockerignore': flags.boolean({
-			description:
-				'Have each service use its own .dockerignore file. See "balena help push".',
-			char: 'm',
-			exclusive: ['gitignore'],
-		}),
-		nogitignore: flags.boolean({
-			description:
-				'No-op (default behavior) since balenaCLI v12.0.0. See "balena help push".',
-			char: 'G',
-			hidden: true,
-		}),
-		gitignore: flags.boolean({
-			description: stripIndent`
-				Consider .gitignore files in addition to the .dockerignore file. This reverts
-				to the CLI v11 behavior/implementation (deprecated) if compatibility is
-				required until your project can be adapted.`,
-			char: 'g',
-			exclusive: ['multi-dockerignore'],
-		}),
-		help: cf.help,
-	};
-
-	public static primary = true;
-
-	public async run() {
-		const { args: params, flags: options } = this.parse<FlagsDef, ArgsDef>(
-			PushCmd,
-		);
-
-		const sdk = getBalenaSdk();
-		const { validateProjectDirectory } = await import('../utils/compose_ts');
-
-		const source = options.source || '.';
-		if (process.env.DEBUG) {
-			console.error(`[debug] Using ${source} as build source`);
-		}
-
-		const { dockerfilePath, registrySecrets } = await validateProjectDirectory(
-			sdk,
-			{
-				dockerfilePath: options.dockerfile,
-				noParentCheck: options['noparent-check'] || false,
-				projectPath: source,
-				registrySecretsPath: options['registry-secrets'],
-			},
-		);
-
-		const nogitignore = !options.gitignore;
-		const convertEol = !options['noconvert-eol'];
-
-		const appOrDevice = params.applicationOrDevice;
-		const buildTarget = await this.getBuildTarget(appOrDevice);
-		switch (buildTarget) {
-			case BuildTarget.Cloud:
-				const remote = await import('../utils/remote-build');
-
-				// Check for invalid options
-				const localOnlyOptions = ['nolive', 'service', 'system', 'env'];
-
-				localOnlyOptions.forEach((opt) => {
-					// @ts-ignore : Not sure why typescript wont let me do this?
-					if (options[opt]) {
-						throw new ExpectedError(
-							`The --${opt} flag is only valid when pushing to a local mode device`,
-						);
-					}
-				});
-
-				const app = appOrDevice;
-				await Command.checkLoggedIn();
-				const [token, baseUrl, owner] = await Promise.all([
-					sdk.auth.getToken(),
-					sdk.settings.get('balenaUrl'),
-					this.getAppOwner(sdk, app),
-				]);
-
-				const opts = {
-					dockerfilePath,
-					emulated: options.emulated || false,
-					multiDockerignore: options['multi-dockerignore'] || false,
-					nocache: options.nocache || false,
-					registrySecrets,
-					headless: options.detached || false,
-					convertEol,
-				};
-				const args = {
-					app,
-					owner,
-					source,
-					auth: token,
-					baseUrl,
-					nogitignore,
-					sdk,
-					opts,
-				};
-				await remote.startRemoteBuild(args);
-				break;
-
-			case BuildTarget.Device:
-				const deviceDeploy = await import('../utils/device/deploy');
-				const device = appOrDevice;
-				const servicesToDisplay = options.service;
-
-				// TODO: Support passing a different port
-				try {
-					await deviceDeploy.deployToDevice({
-						source,
-						deviceHost: device,
-						dockerfilePath,
-						registrySecrets,
-						multiDockerignore: options['multi-dockerignore'] || false,
-						nocache: options.nocache || false,
-						pull: options.pull || false,
-						nogitignore,
-						noParentCheck: options['noparent-check'] || false,
-						nolive: options.nolive || false,
-						detached: options.detached || false,
-						services: servicesToDisplay,
-						system: options.system || false,
-						env: options.env || [],
-						convertEol,
-					});
-				} catch (e) {
-					const { BuildError } = await import('../utils/device/errors');
-					if (instanceOf(e, BuildError)) {
-						throw new ExpectedError(e.toString());
-					} else {
-						throw e;
-					}
-				}
-				break;
-
-			default:
-				throw new ExpectedError(stripIndent`
-					Build target not recognized. Please provide either an application name or
-					device IP address.`);
-		}
-	}
-
-	async getBuildTarget(appOrDevice: string): Promise<BuildTarget | null> {
-		const {
-			validateApplicationName,
-			validateDotLocalUrl,
-			validateIPAddress,
-		} = await import('../utils/validation');
-
-		// First try the application regex from the api
-		if (validateApplicationName(appOrDevice)) {
-			return BuildTarget.Cloud;
-		}
-
-		if (validateIPAddress(appOrDevice) || validateDotLocalUrl(appOrDevice)) {
-			return BuildTarget.Device;
-		}
-
-		return null;
-	}
-
-	async getAppOwner(sdk: BalenaSDK, appName: string) {
-		const _ = await import('lodash');
-
-		const applications = (await sdk.models.application.getAll({
-			$expand: {
-				organization: {
-					$select: ['handle'],
-				},
-			},
-			$filter: {
-				$eq: [{ $tolower: { $: 'app_name' } }, appName.toLowerCase()],
-			},
-			$select: ['id'],
-		})) as Array<
-			Application & {
-				organization: [Organization];
-			}
-		>;
-
-		if (applications == null || applications.length === 0) {
-			throw new ExpectedError(
-				stripIndent`
-			No applications found with name: ${appName}.
-
-			This could mean that the application does not exist, or you do
-			not have the permissions required to access it.`,
-			);
-		}
-
-		if (applications.length === 1) {
-			return applications[0].organization[0].handle;
-		}
-
-		// If we got more than one application with the same name it means that the
-		// user has access to a collab app with the same name as a personal app. We
-		// present a list to the user which shows the fully qualified application
-		// name (user/appname) and allows them to select
-		const entries = _.map(applications, (app) => {
-			const username = app.organization[0].handle;
-			return {
-				name: `${username}/${appName}`,
-				extra: username,
-			};
-		});
-
-		const { selectFromList } = await import('../utils/patterns');
-		const selected = await selectFromList(
-			`${entries.length} applications found with that name, please select the application you would like to push to`,
-			entries,
-		);
-
-		return selected.extra;
-	}
-}

lib/commands/scan.ts

@@ -1,180 +0,0 @@
-/**
- * @license
- * Copyright 2020 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-import { flags } from '@oclif/command';
-import type { LocalBalenaOsDevice } from 'balena-sync';
-import Command from '../command';
-import * as cf from '../utils/common-flags';
-import { getVisuals, stripIndent } from '../utils/lazy';
-
-interface FlagsDef {
-	json?: boolean;
-	verbose: boolean;
-	timeout?: number;
-	help: void;
-}
-
-export default class ScanCmd extends Command {
-	public static description = stripIndent`
-		Scan for balenaOS devices on your local network.
-
-		Scan for balenaOS devices on your local network.
-`;
-
-	public static examples = [
-		'$ balena scan',
-		'$ balena scan --timeout 120',
-		'$ balena scan --verbose',
-	];
-
-	public static usage = 'scan';
-
-	public static flags: flags.Input<FlagsDef> = {
-		verbose: flags.boolean({
-			char: 'v',
-			default: false,
-			description: 'display full info',
-		}),
-		timeout: flags.integer({
-			char: 't',
-			description: 'scan timeout in seconds',
-		}),
-		help: cf.help,
-		json: flags.boolean({
-			char: 'j',
-			description: 'produce JSON output instead of tabular output',
-		}),
-	};
-
-	public static primary = true;
-	public static root = true;
-
-	public async run() {
-		const _ = await import('lodash');
-		const { SpinnerPromise } = getVisuals();
-		const { discover } = await import('balena-sync');
-		const prettyjson = await import('prettyjson');
-		const { ExpectedError } = await import('../errors');
-		const dockerUtils = await import('../utils/docker');
-
-		const dockerPort = 2375;
-		const dockerTimeout = 2000;
-
-		const { flags: options } = this.parse<FlagsDef, {}>(ScanCmd);
-
-		const discoverTimeout =
-			options.timeout != null ? options.timeout * 1000 : undefined;
-
-		// Find active local devices
-		const activeLocalDevices: LocalBalenaOsDevice[] = await new SpinnerPromise({
-			promise: discover.discoverLocalBalenaOsDevices(discoverTimeout),
-			startMessage: 'Scanning for local balenaOS devices..',
-			stopMessage: options.json ? '' : 'Reporting scan results',
-		}).filter(async ({ address }: { address: string }) => {
-			const docker = dockerUtils.createClient({
-				host: address,
-				port: dockerPort,
-				timeout: dockerTimeout,
-			}) as any;
-			try {
-				await docker.pingAsync();
-				return true;
-			} catch (err) {
-				return false;
-			}
-		});
-
-		// Exit with message if no devices found
-		if (_.isEmpty(activeLocalDevices)) {
-			// TODO: Consider whether this should really be an error
-			throw new ExpectedError(
-				process.platform === 'win32'
-					? ScanCmd.noDevicesFoundMessage + ScanCmd.windowsTipMessage
-					: ScanCmd.noDevicesFoundMessage,
-			);
-		}
-
-		// Query devices for info
-		const devicesInfo = await Promise.all(
-			activeLocalDevices.map(async ({ host, address }) => {
-				const docker = dockerUtils.createClient({
-					host: address,
-					port: dockerPort,
-					timeout: dockerTimeout,
-				}) as any;
-				const [dockerInfo, dockerVersion] = await Promise.all([
-					docker.infoAsync().catchReturn('Could not get Docker info'),
-					docker.versionAsync().catchReturn('Could not get Docker version'),
-				]);
-				return {
-					host,
-					address,
-					dockerInfo,
-					dockerVersion,
-				};
-			}),
-		);
-
-		// Reduce properties if not --verbose
-		if (!options.verbose) {
-			devicesInfo.forEach((d: any) => {
-				d.dockerInfo = _.isObject(d.dockerInfo)
-					? _.pick(d.dockerInfo, ScanCmd.dockerInfoProperties)
-					: d.dockerInfo;
-				d.dockerVersion = _.isObject(d.dockerVersion)
-					? _.pick(d.dockerVersion, ScanCmd.dockerVersionProperties)
-					: d.dockerVersion;
-			});
-		}
-
-		// Output results
-		console.log(
-			options.json
-				? JSON.stringify(devicesInfo, null, 4)
-				: prettyjson.render(devicesInfo, { noColor: true }),
-		);
-	}
-
-	protected static dockerInfoProperties = [
-		'Containers',
-		'ContainersRunning',
-		'ContainersPaused',
-		'ContainersStopped',
-		'Images',
-		'Driver',
-		'SystemTime',
-		'KernelVersion',
-		'OperatingSystem',
-		'Architecture',
-	];
-
-	protected static dockerVersionProperties = ['Version', 'ApiVersion'];
-
-	protected static noDevicesFoundMessage =
-		'Could not find any balenaOS devices on the local network.';
-
-	protected static windowsTipMessage = `
-
-Note for Windows users:
-  The 'scan' command relies on the Bonjour service. Check whether Bonjour is
-  installed (Control Panel > Programs and Features). If not, you can download
-  Bonjour for Windows (included with Bonjour Print Services) from here:
-  https://support.apple.com/kb/DL999
-
-  After installing Bonjour, restart your PC and run the 'balena scan' command
-  again.`;
-}

lib/commands/ssh.ts

@@ -1,393 +0,0 @@
-/**
- * @license
- * Copyright 2016-2020 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-import { flags } from '@oclif/command';
-import Command from '../command';
-import * as cf from '../utils/common-flags';
-import { getBalenaSdk, stripIndent } from '../utils/lazy';
-import {
-	parseAsInteger,
-	validateDotLocalUrl,
-	validateIPAddress,
-} from '../utils/validation';
-import * as BalenaSdk from 'balena-sdk';
-
-interface FlagsDef {
-	port?: number;
-	tty: boolean;
-	verbose: boolean;
-	noproxy: boolean;
-	help: void;
-}
-
-interface ArgsDef {
-	applicationOrDevice: string;
-	service?: string;
-}
-
-export default class NoteCmd extends Command {
-	public static description = stripIndent`
-		SSH into the host or application container of a device.
-
-		Start a shell on a local or remote device. If a service name is not provided,
-		a shell will be opened on the host OS.
-
-		If an application name is provided, an interactive menu will be presented
-		for the selection of an online device. A shell will then be opened for the
-		host OS or service container of the chosen device.
-
-		For local devices, the IP address and .local domain name are supported.
-		If the device is referenced by IP or \`.local\` address, the connection
-		is initiated directly to balenaOS on port \`22222\` via an
-		openssh-compatible client. Otherwise, any connection initiated remotely
-		traverses the balenaCloud VPN.
-
-		Commands may be piped to the standard input for remote execution (see examples).
-		Note however that remote command execution on service containers (as opposed to
-		the host OS) is not currently possible when a device UUID is used (instead of
-		an IP address) because of a balenaCloud backend limitation.
-
-		Note: \`balena ssh\` requires an openssh-compatible client to be correctly
-		installed in your shell environment. For more information (including Windows
-		support) please check:
-			https://github.com/balena-io/balena-cli/blob/master/INSTALL.md#additional-dependencies,
-	`;
-
-	public static examples = [
-		'$ balena ssh MyApp',
-		'$ balena ssh f49cefd',
-		'$ balena ssh f49cefd my-service',
-		'$ balena ssh f49cefd --port <port>',
-		'$ balena ssh 192.168.0.1 --verbose',
-		'$ balena ssh f49cefd.local my-service',
-		'$ echo "uptime; exit;" | balena ssh f49cefd',
-		'$ echo "uptime; exit;" | balena ssh 192.168.0.1 myService',
-	];
-
-	public static args = [
-		{
-			name: 'applicationOrDevice',
-			description: 'application name, device uuid, or address of local device',
-			required: true,
-		},
-		{
-			name: 'service',
-			description: 'service name, if connecting to a container',
-			required: false,
-		},
-	];
-
-	public static usage = 'ssh <applicationOrDevice> [service]';
-
-	public static flags: flags.Input<FlagsDef> = {
-		port: flags.integer({
-			description: stripIndent`
-				SSH server port number (default 22222) if the target is an IP address or .local
-				hostname. Otherwise, port number for the balenaCloud gateway (default 22).`,
-			char: 'p',
-			parse: (p) => parseAsInteger(p, 'port'),
-		}),
-		tty: flags.boolean({
-			description:
-				'Force pseudo-terminal allocation (bypass TTY autodetection for stdin)',
-			char: 't',
-		}),
-		verbose: flags.boolean({
-			description: 'Increase verbosity',
-			char: 'v',
-		}),
-		noproxy: flags.boolean({
-			description: 'Bypass global proxy configuration for the ssh connection',
-		}),
-		help: cf.help,
-	};
-
-	public static primary = true;
-
-	public async run() {
-		const { args: params, flags: options } = this.parse<FlagsDef, ArgsDef>(
-			NoteCmd,
-		);
-
-		// if we're doing a direct SSH connection locally...
-		if (
-			validateDotLocalUrl(params.applicationOrDevice) ||
-			validateIPAddress(params.applicationOrDevice)
-		) {
-			const { performLocalDeviceSSH } = await import('../utils/device/ssh');
-			return await performLocalDeviceSSH({
-				address: params.applicationOrDevice,
-				port: options.port,
-				forceTTY: options.tty,
-				verbose: options.verbose,
-				service: params.service,
-			});
-		}
-
-		const { getProxyConfig, which } = await import('../utils/helpers');
-		const { checkLoggedIn, getOnlineTargetUuid } = await import(
-			'../utils/patterns'
-		);
-		const sdk = getBalenaSdk();
-
-		const proxyConfig = getProxyConfig();
-		const useProxy = !!proxyConfig && !options.noproxy;
-
-		// this will be a tunnelled SSH connection...
-		await checkLoggedIn();
-		const uuid = await getOnlineTargetUuid(sdk, params.applicationOrDevice);
-		let version: string | undefined;
-		let id: number | undefined;
-
-		const device = await sdk.models.device.get(uuid, {
-			$select: ['id', 'supervisor_version', 'is_online'],
-		});
-		id = device.id;
-		version = device.supervisor_version;
-
-		const [whichProxytunnel, username, proxyUrl] = await Promise.all([
-			useProxy ? which('proxytunnel', false) : undefined,
-			sdk.auth.whoami(),
-			// note that `proxyUrl` refers to the balenaCloud "resin-proxy"
-			// service, currently "balena-devices.com", rather than some
-			// local proxy server URL
-			sdk.settings.get('proxyUrl'),
-		]);
-
-		const getSshProxyCommand = () => {
-			if (!proxyConfig) {
-				return;
-			}
-			if (!whichProxytunnel) {
-				console.warn(stripIndent`
-					Proxy is enabled but the \`proxytunnel\` binary cannot be found.
-					Please install it if you want to route the \`balena ssh\` requests through the proxy.
-					Alternatively you can pass \`--noproxy\` param to the \`balena ssh\` command to ignore the proxy config
-					for the \`ssh\` requests.
-
-					Attempting the unproxied request for now.`);
-				return;
-			}
-
-			const p = proxyConfig;
-			if (p.username && p.password) {
-				// proxytunnel understands these variables for proxy authentication.
-				// Setting the variables instead of command-line options avoids the
-				// need for shell-specific escaping of special characters like '$'.
-				process.env.PROXYUSER = p.username;
-				process.env.PROXYPASS = p.password;
-			}
-
-			return [
-				'proxytunnel',
-				`--proxy=${p.host}:${p.port}`,
-				// ssh replaces these %h:%p variables in the ProxyCommand option
-				// https://linux.die.net/man/5/ssh_config
-				'--dest=%h:%p',
-				...(options.verbose ? ['--verbose'] : []),
-			];
-		};
-
-		const proxyCommand = useProxy ? getSshProxyCommand() : undefined;
-
-		if (username == null) {
-			const { ExpectedError } = await import('../errors');
-			throw new ExpectedError(
-				`Opening an SSH connection to a remote device requires you to be logged in.`,
-			);
-		}
-
-		// At this point, we have a long uuid with a device
-		// that we know exists and is accessible
-		let containerId: string | undefined;
-		if (params.service != null) {
-			containerId = await this.getContainerId(
-				sdk,
-				uuid,
-				params.service,
-				{
-					port: options.port,
-					proxyCommand,
-					proxyUrl: proxyUrl || '',
-					username: username!,
-				},
-				version,
-				id,
-			);
-		}
-
-		let accessCommand: string;
-		if (containerId != null) {
-			accessCommand = `enter ${uuid} ${containerId}`;
-		} else {
-			accessCommand = `host ${uuid}`;
-		}
-
-		const command = this.generateVpnSshCommand({
-			uuid,
-			command: accessCommand,
-			verbose: options.verbose,
-			port: options.port,
-			proxyCommand,
-			proxyUrl: proxyUrl || '',
-			username: username!,
-		});
-
-		const { spawnSshAndThrowOnError } = await import('../utils/ssh');
-		return spawnSshAndThrowOnError(command);
-	}
-
-	async getContainerId(
-		sdk: BalenaSdk.BalenaSDK,
-		uuid: string,
-		serviceName: string,
-		sshOpts: {
-			port?: number;
-			proxyCommand?: string[];
-			proxyUrl: string;
-			username: string;
-		},
-		version?: string,
-		id?: number,
-	): Promise<string> {
-		const semver = await import('balena-semver');
-
-		if (version == null || id == null) {
-			const device = await sdk.models.device.get(uuid, {
-				$select: ['id', 'supervisor_version'],
-			});
-			version = device.supervisor_version;
-			id = device.id;
-		}
-
-		let containerId: string | undefined;
-		if (semver.gte(version, '8.6.0')) {
-			const apiUrl = await sdk.settings.get('apiUrl');
-			// TODO: Move this into the SDKs device model
-			const request = await sdk.request.send({
-				method: 'POST',
-				url: '/supervisor/v2/containerId',
-				baseUrl: apiUrl,
-				body: {
-					method: 'GET',
-					deviceId: id,
-				},
-			});
-			if (request.status !== 200) {
-				throw new Error(
-					`There was an error connecting to device ${uuid}, HTTP response code: ${request.status}.`,
-				);
-			}
-			const body = request.body;
-			if (body.status !== 'success') {
-				throw new Error(
-					`There was an error communicating with device ${uuid}.\n\tError: ${body.message}`,
-				);
-			}
-			containerId = body.services[serviceName];
-		} else {
-			console.error(stripIndent`
-			Using legacy method to detect container ID. This will be slow.
-			To speed up this process, please update your device to an OS
-			which has a supervisor version of at least v8.6.0.
-		`);
-			// We need to execute a balena ps command on the device,
-			// and parse the output, looking for a specific
-			// container
-			const childProcess = await import('child_process');
-			const { escapeRegExp } = await import('lodash');
-			const { which } = await import('../utils/helpers');
-			const { deviceContainerEngineBinary } = await import(
-				'../utils/device/ssh'
-			);
-
-			const sshBinary = await which('ssh');
-			const sshArgs = this.generateVpnSshCommand({
-				uuid,
-				verbose: false,
-				port: sshOpts.port,
-				command: `host ${uuid} "${deviceContainerEngineBinary}" ps --format "{{.ID}} {{.Names}}"`,
-				proxyCommand: sshOpts.proxyCommand,
-				proxyUrl: sshOpts.proxyUrl,
-				username: sshOpts.username,
-			});
-
-			if (process.env.DEBUG) {
-				console.error(`[debug] [${sshBinary}, ${sshArgs.join(', ')}]`);
-			}
-			const subProcess = childProcess.spawn(sshBinary, sshArgs, {
-				stdio: [null, 'pipe', null],
-			});
-			const containers = await new Promise<string>((resolve, reject) => {
-				const output: string[] = [];
-				subProcess.stdout.on('data', (chunk) => output.push(chunk.toString()));
-				subProcess.on('close', (code: number) => {
-					if (code !== 0) {
-						reject(
-							new Error(
-								`Non-zero error code when looking for service container: ${code}`,
-							),
-						);
-					} else {
-						resolve(output.join(''));
-					}
-				});
-			});
-
-			const lines = containers.split('\n');
-			const regex = new RegExp(`\\/?${escapeRegExp(serviceName)}_\\d+_\\d+`);
-			for (const container of lines) {
-				const [cId, name] = container.split(' ');
-				if (regex.test(name)) {
-					containerId = cId;
-					break;
-				}
-			}
-		}
-
-		if (containerId == null) {
-			throw new Error(
-				`Could not find a service ${serviceName} on device ${uuid}.`,
-			);
-		}
-		return containerId;
-	}
-
-	generateVpnSshCommand(opts: {
-		uuid: string;
-		command: string;
-		verbose: boolean;
-		port?: number;
-		username: string;
-		proxyUrl: string;
-		proxyCommand?: string[];
-	}) {
-		return [
-			...(opts.verbose ? ['-vvv'] : []),
-			'-t',
-			...['-o', 'LogLevel=ERROR'],
-			...['-o', 'StrictHostKeyChecking=no'],
-			...['-o', 'UserKnownHostsFile=/dev/null'],
-			...(opts.proxyCommand && opts.proxyCommand.length
-				? ['-o', `ProxyCommand=${opts.proxyCommand.join(' ')}`]
-				: []),
-			...(opts.port ? ['-p', opts.port.toString()] : []),
-			`${opts.username}@ssh.${opts.proxyUrl}`,
-			opts.command,
-		];
-	}
-}

lib/commands/tags.ts

@@ -1,131 +0,0 @@
-/**
- * @license
- * Copyright 2016-2020 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-import { flags } from '@oclif/command';
-import Command from '../command';
-import { ExpectedError } from '../errors';
-import * as cf from '../utils/common-flags';
-import { getBalenaSdk, getVisuals, stripIndent } from '../utils/lazy';
-import { disambiguateReleaseParam } from '../utils/normalization';
-import { tryAsInteger } from '../utils/validation';
-import { isV12 } from '../utils/version';
-
-interface FlagsDef {
-	application?: string;
-	device?: string;
-	release?: string;
-	help: void;
-	app?: string;
-}
-
-export default class TagsCmd extends Command {
-	public static description = stripIndent`
-		List all tags for an application, device or release.
-
-		List all tags and their values for a particular application,
-		device or release.
-	`;
-
-	public static examples = [
-		'$ balena tags --application MyApp',
-		'$ balena tags --device 7cf02a6',
-		'$ balena tags --release 1234',
-		'$ balena tags --release b376b0e544e9429483b656490e5b9443b4349bd6',
-	];
-
-	public static usage = 'tags';
-
-	public static flags: flags.Input<FlagsDef> = {
-		application: {
-			...cf.application,
-			exclusive: ['app', 'device', 'release'],
-		},
-		device: {
-			...cf.device,
-			exclusive: ['app', 'application', 'release'],
-		},
-		release: {
-			...cf.release,
-			exclusive: ['app', 'application', 'device'],
-		},
-		help: cf.help,
-		app: flags.string({
-			description: "same as '--application'",
-			exclusive: ['application', 'device', 'release'],
-		}),
-	};
-
-	public static authenticated = true;
-
-	public async run() {
-		const { flags: options } = this.parse<FlagsDef, {}>(TagsCmd);
-
-		// Prefer options.application over options.app
-		options.application = options.application || options.app;
-		delete options.app;
-
-		const balena = getBalenaSdk();
-
-		// Check user has specified one of application/device/release
-		if (!options.application && !options.device && !options.release) {
-			throw new ExpectedError(this.missingResourceMessage);
-		}
-
-		let tags;
-
-		if (options.application) {
-			tags = await balena.models.application.tags.getAllByApplication(
-				tryAsInteger(options.application),
-			);
-		}
-		if (options.device) {
-			tags = await balena.models.device.tags.getAllByDevice(
-				tryAsInteger(options.device),
-			);
-		}
-		if (options.release) {
-			const releaseParam = await disambiguateReleaseParam(
-				balena,
-				options.release,
-			);
-
-			tags = await balena.models.release.tags.getAllByRelease(releaseParam);
-		}
-
-		if (!tags || tags.length === 0) {
-			throw new ExpectedError('No tags found');
-		}
-
-		console.log(
-			isV12()
-				? getVisuals().table.horizontal(tags, ['tag_key', 'value'])
-				: getVisuals().table.horizontal(tags, ['id', 'tag_key', 'value']),
-		);
-	}
-
-	protected missingResourceMessage = stripIndent`
-					To list tags for a resource, you must provide exactly one of:
-
-					  * An application, with --application <appname>
-					  * A device, with --device <uuid>
-					  * A release, with --release <id or commit>
-
-					See the help page for examples:
-
-					  $ balena help tags
-	`;
-}

lib/events.ts

@@ -1,118 +0,0 @@
-/**
- * @license
- * Copyright 2019-2020 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-import * as _ from 'lodash';
-import * as Mixpanel from 'mixpanel';
-
-import * as packageJSON from '../package.json';
-import { getBalenaSdk } from './utils/lazy';
-
-const getMixpanel = _.once((balenaUrl: string) => {
-	return Mixpanel.init('balena-main', {
-		host: `api.${balenaUrl}`,
-		path: '/mixpanel',
-		protocol: 'https',
-	});
-});
-
-interface CachedUsername {
-	token: string;
-	username: string;
-}
-
-/**
- * Mixpanel.com analytics tracking (information on balenaCLI usage).
- *
- * @param commandSignature A string like, for example:
- *      "push <applicationOrDevice>"
- * That's literally so: "applicationOrDevice" is NOT replaced with the actual
- * application ID or device ID. The purpose is to find out the most / least
- * used command verbs, so we can focus our development effort where it is most
- * beneficial to end users.
- *
- * The username and command signature are also added as extra context
- * information in Sentry.io error reporting, for CLI debugging purposes
- * (mainly unexpected/unhandled exceptions -- see also `lib/errors.ts`).
- */
-export async function trackCommand(commandSignature: string) {
-	try {
-		let Sentry: typeof import('@sentry/node');
-		if (!process.env.BALENARC_NO_SENTRY) {
-			Sentry = await import('@sentry/node');
-			Sentry.configureScope((scope) => {
-				scope.setExtra('command', commandSignature);
-			});
-		}
-		const settings = await import('balena-settings-client');
-		const balenaUrl = settings.get('balenaUrl') as string;
-
-		const username = await (async () => {
-			const getStorage = await import('balena-settings-storage');
-			const dataDirectory = settings.get('dataDirectory') as string;
-			const storage = getStorage({ dataDirectory });
-			let token;
-			try {
-				token = await storage.get('token');
-			} catch {
-				// If we can't get a token then we can't get a username
-				return;
-			}
-			try {
-				const result = (await storage.get('cachedUsername')) as CachedUsername;
-				if (result.token === token) {
-					return result.username;
-				}
-			} catch {
-				// ignore
-			}
-			try {
-				const balena = getBalenaSdk();
-				const $username = await balena.auth.whoami();
-				storage.set('cachedUsername', {
-					token,
-					username: $username,
-				} as CachedUsername);
-				return $username;
-			} catch {
-				return;
-			}
-		})();
-
-		const mixpanel = getMixpanel(balenaUrl);
-
-		if (!process.env.BALENARC_NO_SENTRY) {
-			Sentry!.configureScope((scope) => {
-				scope.setUser({
-					id: username,
-					username,
-				});
-			});
-		}
-		// Don't actually call mixpanel.track() while running test cases
-		if (!process.env.BALENA_CLI_TEST_TYPE) {
-			await mixpanel.track(`[CLI] ${commandSignature}`, {
-				distinct_id: username,
-				version: packageJSON.version,
-				node: process.version,
-				arch: process.arch,
-				balenaUrl, // e.g. 'balena-cloud.com' or 'balena-staging.com'
-				platform: process.platform,
-			});
-		}
-	} catch {
-		// ignore
-	}
-}

lib/help.ts

@@ -1,189 +0,0 @@
-import Help from '@oclif/plugin-help';
-import * as indent from 'indent-string';
-import { getChalk } from './utils/lazy';
-import { renderList } from '@oclif/plugin-help/lib/list';
-import { ExpectedError } from './errors';
-
-// Partially overrides standard implementation of help plugin
-// https://github.com/oclif/plugin-help/blob/master/src/index.ts
-
-function getHelpSubject(args: string[]): string | undefined {
-	for (const arg of args) {
-		if (arg === '--') {
-			return;
-		}
-		if (arg === 'help' || arg === '--help' || arg === '-h') {
-			continue;
-		}
-		if (arg.startsWith('-')) {
-			return;
-		}
-		return arg;
-	}
-}
-
-export default class BalenaHelp extends Help {
-	public static usage: 'help [command]';
-
-	public showHelp(argv: string[]) {
-		const chalk = getChalk();
-		const subject = getHelpSubject(argv);
-		if (!subject) {
-			const verbose = argv.includes('-v') || argv.includes('--verbose');
-			this.showCustomRootHelp(verbose);
-			return;
-		}
-
-		const command = this.config.findCommand(subject);
-		if (command) {
-			this.showCommandHelp(command);
-			return;
-		}
-
-		// If they've typed a topic (e.g. `balena os`) that isn't also a command (e.g. `balena device`)
-		// then list the associated commands.
-		const topicCommands = this.config.commands.filter((c) => {
-			return c.id.startsWith(`${subject}:`);
-		});
-		if (topicCommands.length > 0) {
-			console.log(`${chalk.yellow(subject)} commands include:`);
-			console.log(this.formatCommands(topicCommands));
-			console.log(
-				`\nRun ${chalk.cyan.bold(
-					'balena help -v',
-				)} for a list of all available commands,`,
-			);
-			console.log(
-				` or ${chalk.cyan.bold(
-					'balena help <command>',
-				)} for detailed help on a specific command.`,
-			);
-			return;
-		}
-
-		throw new ExpectedError(`command ${chalk.cyan.bold(subject)} not found`);
-	}
-
-	showCustomRootHelp(showAllCommands: boolean): void {
-		const chalk = getChalk();
-		const bold = chalk.bold;
-		const cmd = chalk.cyan.bold;
-
-		let commands = this.config.commands;
-		commands = commands.filter((c) => this.opts.all || !c.hidden);
-
-		// Get Primary Commands, sorted as in manual list
-		const primaryCommands = this.manuallySortedPrimaryCommands.map((pc) => {
-			return commands.find((c) => c.id === pc.replace(' ', ':'));
-		});
-
-		// Get the rest as Additional Commands
-		const additionalCommands = commands.filter(
-			(c) =>
-				!this.manuallySortedPrimaryCommands.includes(c.id.replace(':', ' ')),
-		);
-
-		// Find longest usage, and pad usage of first command in each category
-		// This is to ensure that both categories align visually
-		const usageLength = commands
-			.map((c) => c.usage?.length || 0)
-			.reduce((longest, l) => {
-				return l > longest ? l : longest;
-			});
-
-		if (
-			typeof primaryCommands[0]?.usage === 'string' &&
-			typeof additionalCommands[0]?.usage === 'string'
-		) {
-			primaryCommands[0].usage = primaryCommands[0].usage.padEnd(usageLength);
-			additionalCommands[0].usage = additionalCommands[0].usage.padEnd(
-				usageLength,
-			);
-		}
-
-		// Output help
-		console.log(bold('USAGE'));
-		console.log('$ balena [COMMAND] [OPTIONS]');
-
-		console.log(bold('\nPRIMARY COMMANDS'));
-		console.log(this.formatCommands(primaryCommands));
-
-		if (showAllCommands) {
-			console.log(bold('\nADDITIONAL COMMANDS'));
-			console.log(this.formatCommands(additionalCommands));
-		} else {
-			console.log(
-				`\n${bold('...MORE')} run ${cmd(
-					'balena help --verbose',
-				)} to list additional commands.`,
-			);
-		}
-
-		console.log(bold('\nGLOBAL OPTIONS'));
-		console.log('  --help, -h');
-		console.log('  --debug\n');
-
-		console.log(
-			`For help, visit our support forums: ${chalk.grey(
-				'https://forums.balena.io',
-			)}`,
-		);
-		console.log(
-			`For bug reports or feature requests, see: ${chalk.grey(
-				'https://github.com/balena-io/balena-cli/issues/',
-			)}\n`,
-		);
-	}
-
-	protected formatCommands(commands: any[]): string {
-		if (commands.length === 0) {
-			return '';
-		}
-
-		const body = renderList(
-			commands
-				.filter((c) => c.usage != null && c.usage !== '')
-				.map((c) => [c.usage, this.formatDescription(c.description)]),
-			{
-				spacer: '\n',
-				stripAnsi: this.opts.stripAnsi,
-				maxWidth: this.opts.maxWidth - 2,
-			},
-		);
-
-		return indent(body, 2);
-	}
-
-	protected formatDescription(desc: string = '') {
-		const chalk = getChalk();
-
-		desc = desc.split('\n')[0];
-		// Remove any ending .
-		if (desc[desc.length - 1] === '.') {
-			desc = desc.substring(0, desc.length - 1);
-		}
-		// Lowercase first letter if second char is lowercase, to preserve e.g. 'SSH ...')
-		if (desc[1] === desc[1]?.toLowerCase()) {
-			desc = `${desc[0].toLowerCase()}${desc.substring(1)}`;
-		}
-		return chalk.grey(desc);
-	}
-
-	readonly manuallySortedPrimaryCommands = [
-		'login',
-		'push',
-		'logs',
-		'ssh',
-		'apps',
-		'app',
-		'devices',
-		'device',
-		'tunnel',
-		'preload',
-		'build',
-		'deploy',
-		'join',
-		'leave',
-		'scan',
-	];
-}

lib/hooks/prerun/track.ts

@@ -1,46 +0,0 @@
-/**
- * @license
- * Copyright 2019-2020 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-import type { Hook } from '@oclif/config';
-
-let trackResolve: (result: Promise<any>) => void;
-
-// note: trackPromise is subject to a Bluebird.timeout, defined in events.ts
-export const trackPromise = new Promise((resolve) => {
-	trackResolve = resolve;
-});
-
-/**
- * This is an oclif 'prerun' hook. This hook runs after the command line is
- * parsed by oclif, but before the command's run() function is called.
- * See: https://oclif.io/docs/hooks
- *
- * This hook is used to track CLI command signatures with mixpanel.
- * A command signature is something like "env add NAME [VALUE]". That's
- * literally so: 'NAME' and 'VALUE' are NOT replaced with actual values.
- */
-const hook: Hook<'prerun'> = async function (options) {
-	const events = await import('../../events');
-	const usage: string | string[] | undefined = options.Command.usage;
-	const cmdSignature =
-		usage == null ? '*' : typeof usage === 'string' ? usage : usage.join(' ');
-
-	// Intentionally do not await for the track promise here, in order to
-	// run the command tracking and the command itself in parallel.
-	trackResolve(events.trackCommand(cmdSignature));
-};
-
-export default hook;

lib/utils/bootstrap.ts

@@ -1,103 +0,0 @@
-/**
- * @license
- * Copyright 2019-2020 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-/*
- * THIS MODULE SHOULD NOT IMPORT / REQUIRE ANYTHING AT THE GLOBAL LEVEL.
- * It is meant to contain elementary helper functions or classes that
- * can be used very early on during CLI startup, before anything else
- * like Sentry error reporting, preparser, oclif parser and the like.
- */
-
-export class CliSettings {
-	public readonly settings: any;
-	constructor() {
-		this.settings = require('balena-settings-client') as typeof import('balena-settings-client');
-	}
-
-	public get<T>(name: string): T {
-		return this.settings.get(name);
-	}
-
-	/**
-	 * Like settings.get(), but return `undefined` instead of throwing an
-	 * error if the setting is not found / not defined.
-	 */
-	public getCatch<T>(name: string): T | undefined {
-		try {
-			return this.settings.get(name);
-		} catch (err) {
-			if (!/Setting not found/i.test(err.message)) {
-				throw err;
-			}
-		}
-	}
-}
-
-export function parseBoolEnvVar(varName: string): boolean {
-	return !['0', 'no', 'false', '', undefined].includes(
-		process.env[varName]?.toLowerCase(),
-	);
-}
-
-export function normalizeEnvVar(varName: string) {
-	process.env[varName] = parseBoolEnvVar(varName) ? '1' : '';
-}
-
-const bootstrapVars = ['DEBUG', 'BALENARC_NO_SENTRY'];
-
-export function normalizeEnvVars(varNames: string[] = bootstrapVars) {
-	for (const varName of varNames) {
-		normalizeEnvVar(varName);
-	}
-}
-
-/**
- * Implements the 'pkgExec' command, used as a way to provide a Node.js
- * interpreter for child_process.spawn()-like operations when the CLI is
- * executing as a standalone zip package (built-in Node interpreter) and
- * the system may not have a separate Node.js installation. A present use
- * case is a patched version of the 'windosu' package that requires a
- * Node.js interpreter to spawn a privileged child process.
- *
- * @param modFunc Path to a JS module that will be executed via require().
- * The modFunc argument may optionally contain a function name separated
- * by '::', for example '::main' in:
- * 'C:\\snapshot\\balena-cli\\node_modules\\windosu\\lib\\pipe.js::main'
- * in which case that function is executed in the require'd module.
- * @param args Optional arguments to passed through process.argv and as
- * arguments to the function specified via modFunc.
- */
-export async function pkgExec(modFunc: string, args: string[]) {
-	const [modPath, funcName] = modFunc.split('::');
-	let replacedModPath = modPath;
-	const match = modPath
-		.replace(/\\/g, '/')
-		.match(/\/snapshot\/balena-cli\/(.+)/);
-	if (match) {
-		replacedModPath = `../${match[1]}`;
-	}
-	process.argv = [process.argv[0], process.argv[1], ...args];
-	try {
-		const mod: any = await import(replacedModPath);
-		if (funcName) {
-			await mod[funcName](...args);
-		}
-	} catch (err) {
-		console.error(`Error executing pkgExec "${modFunc}" [${args.join()}]`);
-		console.error(err);
-	}
-}

lib/utils/common-flags.ts

@@ -1,77 +0,0 @@
-/**
- * @license
- * Copyright 2019 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-import { flags } from '@oclif/command';
-
-import type { IBooleanFlag } from '@oclif/parser/lib/flags';
-import { stripIndent } from './lazy';
-
-export const application = flags.string({
-	char: 'a',
-	description: 'application name',
-});
-// TODO: Consider remove second alias 'app' when we can, to simplify.
-export const app = flags.string({
-	description: "same as '--application'",
-});
-
-export const device = flags.string({
-	char: 'd',
-	description: 'device UUID',
-});
-
-export const help: IBooleanFlag<void> = flags.help({ char: 'h' });
-
-export const quiet: IBooleanFlag<boolean> = flags.boolean({
-	char: 'q',
-	description: 'suppress warning messages',
-	default: false,
-});
-
-export const release = flags.string({
-	char: 'r',
-	description: 'release id',
-});
-
-export const service = flags.string({
-	char: 's',
-	description: 'service name',
-});
-
-export const verbose: IBooleanFlag<boolean> = flags.boolean({
-	char: 'v',
-	description: 'produce verbose output',
-});
-
-export const yes: IBooleanFlag<boolean> = flags.boolean({
-	char: 'y',
-	description: 'answer "yes" to all questions (non interactive use)',
-});
-
-export const force: IBooleanFlag<boolean> = flags.boolean({
-	char: 'f',
-	description: 'force action if the update lock is set',
-});
-
-export const drive = flags.string({
-	char: 'd',
-	description: stripIndent`
-		the drive to write the image to, eg. \`/dev/sdb\` or \`/dev/mmcblk0\`.
-		Careful with this as you can erase your hard drive.
-		Check \`balena util available-drives\` for available options.
-	`,
-});

lib/utils/compose_ts.ts

@@ -1,966 +0,0 @@
-/**
- * @license
- * Copyright 2018-2020 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-import { BalenaSDK } from 'balena-sdk';
-import type * as Dockerode from 'dockerode';
-import * as _ from 'lodash';
-import { promises as fs } from 'fs';
-import * as path from 'path';
-import type { Composition } from 'resin-compose-parse';
-import type * as MultiBuild from 'resin-multibuild';
-import type { Readable } from 'stream';
-import type { Pack } from 'tar-stream';
-import { ExpectedError } from '../errors';
-import { getBalenaSdk, getChalk, stripIndent } from './lazy';
-import {
-	BuiltImage,
-	ComposeCliFlags,
-	ComposeOpts,
-	ComposeProject,
-	Release,
-	TaggedImage,
-	TarDirectoryOptions,
-} from './compose-types';
-import { DeviceInfo } from './device/api';
-import Logger = require('./logger');
-import { flags } from '@oclif/command';
-
-export interface RegistrySecrets {
-	[registryAddress: string]: {
-		username: string;
-		password: string;
-	};
-}
-
-const exists = async (filename: string) => {
-	try {
-		await fs.access(filename);
-		return true;
-	} catch {
-		return false;
-	}
-};
-
-const compositionFileNames = ['docker-compose.yml', 'docker-compose.yaml'];
-
-const hr =
-	'----------------------------------------------------------------------';
-
-/**
- * high-level function resolving a project and creating a composition out
- * of it in one go. if image is given, it'll create a default project for
- * that without looking for a project. falls back to creating a default
- * project if none is found at the given projectPath.
- */
-export async function loadProject(
-	logger: Logger,
-	opts: ComposeOpts,
-	image?: string,
-): Promise<ComposeProject> {
-	const compose = await import('resin-compose-parse');
-	const { createProject } = await import('./compose');
-	let composeName: string;
-	let composeStr: string;
-
-	logger.logDebug('Loading project...');
-
-	if (image) {
-		logger.logInfo(`Creating default composition with image: "${image}"`);
-		composeStr = compose.defaultComposition(image);
-	} else {
-		logger.logDebug('Resolving project...');
-		[composeName, composeStr] = await resolveProject(logger, opts.projectPath);
-		if (composeName) {
-			if (opts.dockerfilePath) {
-				logger.logWarn(
-					`Ignoring alternative dockerfile "${opts.dockerfilePath}" because composition file "${composeName}" exists`,
-				);
-			}
-		} else {
-			logger.logInfo(
-				`Creating default composition with source: "${opts.projectPath}"`,
-			);
-			composeStr = compose.defaultComposition(undefined, opts.dockerfilePath);
-		}
-	}
-	logger.logDebug('Creating project...');
-	return createProject(opts.projectPath, composeStr, opts.projectName);
-}
-
-/**
- * Look into the given directory for valid compose files and return
- * the contents of the first one found.
- */
-async function resolveProject(
-	logger: Logger,
-	projectRoot: string,
-	quiet = false,
-): Promise<[string, string]> {
-	let composeFileName = '';
-	let composeFileContents = '';
-	for (const fname of compositionFileNames) {
-		const fpath = path.join(projectRoot, fname);
-		if (await exists(fpath)) {
-			logger.logDebug(`${fname} file found at "${projectRoot}"`);
-			composeFileName = fname;
-			try {
-				composeFileContents = await fs.readFile(fpath, 'utf8');
-			} catch (err) {
-				logger.logError(`Error reading composition file "${fpath}":\n${err}`);
-				throw err;
-			}
-			break;
-		}
-	}
-	if (!quiet && !composeFileName) {
-		logger.logInfo(`No "docker-compose.yml" file found at "${projectRoot}"`);
-	}
-	return [composeFileName, composeFileContents];
-}
-
-/**
- * Load the ".balena/balena.yml" file (or resin.yml, or yaml or json),
- * which contains "build metadata" for features like "build secrets" and
- * "build variables".
- * @returns Pair of metadata object and metadata file path
- */
-async function loadBuildMetatada(
-	sourceDir: string,
-): Promise<[MultiBuild.ParsedBalenaYml, string]> {
-	let metadataPath = '';
-	let rawString = '';
-
-	outer: for (const fName of ['balena', 'resin']) {
-		for (const fExt of ['yml', 'yaml', 'json']) {
-			metadataPath = path.join(sourceDir, `.${fName}`, `${fName}.${fExt}`);
-			try {
-				rawString = await fs.readFile(metadataPath, 'utf8');
-				break outer;
-			} catch (err) {
-				if (err.code === 'ENOENT') {
-					// file not found, try the next name.extension combination
-					continue;
-				} else {
-					throw err;
-				}
-			}
-		}
-	}
-	if (!rawString) {
-		return [{}, ''];
-	}
-	let buildMetadata: MultiBuild.ParsedBalenaYml;
-	try {
-		if (metadataPath.endsWith('json')) {
-			buildMetadata = JSON.parse(rawString);
-		} else {
-			buildMetadata = require('js-yaml').safeLoad(rawString);
-		}
-	} catch (err) {
-		throw new ExpectedError(
-			`Error parsing file "${metadataPath}":\n ${err.message}`,
-		);
-	}
-	return [buildMetadata, metadataPath];
-}
-
-/**
- * Return a map of service name to service subdirectory, obtained from the given
- * composition object. If a composition object is not provided, an attempt will
- * be made to parse a 'docker-compose.yml' file at the given sourceDir.
- * Entries will be NOT be returned for subdirectories equal to '.' (e.g. the
- * 'main' "service" of a single-container application).
- *
- * @param sourceDir Project source directory (project root)
- * @param composition Optional previously parsed composition object
- */
-async function getServiceDirsFromComposition(
-	sourceDir: string,
-	composition?: Composition,
-): Promise<Dictionary<string>> {
-	const { createProject } = await import('./compose');
-	const serviceDirs: Dictionary<string> = {};
-	if (!composition) {
-		const [, composeStr] = await resolveProject(
-			Logger.getLogger(),
-			sourceDir,
-			true,
-		);
-		if (composeStr) {
-			composition = createProject(sourceDir, composeStr).composition;
-		}
-	}
-	if (composition?.services) {
-		const relPrefix = '.' + path.sep;
-		for (const [serviceName, service] of Object.entries(composition.services)) {
-			let dir =
-				typeof service.build === 'string'
-					? service.build
-					: service.build?.context || '.';
-			// Convert forward slashes to backslashes on Windows
-			dir = path.normalize(dir);
-			// Make sure the path is relative to the project directory
-			if (path.isAbsolute(dir)) {
-				dir = path.relative(sourceDir, dir);
-			}
-			// remove a trailing '/' (or backslash on Windows)
-			dir = dir.endsWith(path.sep) ? dir.slice(0, -1) : dir;
-			// remove './' prefix (or '.\\' on Windows)
-			dir = dir.startsWith(relPrefix) ? dir.slice(2) : dir;
-			// filter out a '.' service directory (e.g. for the 'main' service
-			// of a single-container application)
-			if (dir && dir !== '.') {
-				serviceDirs[serviceName] = dir;
-			}
-		}
-	}
-	return serviceDirs;
-}
-
-/**
- * Create a tar stream out of the local filesystem at the given directory,
- * while optionally applying file filters such as '.dockerignore' and
- * optionally converting text file line endings (CRLF to LF).
- * @param dir Source directory
- * @param param Options
- * @returns {Promise<import('stream').Readable>}
- */
-export async function tarDirectory(
-	dir: string,
-	{
-		composition,
-		convertEol = false,
-		multiDockerignore = false,
-		nogitignore = false,
-		preFinalizeCallback,
-	}: TarDirectoryOptions,
-): Promise<import('stream').Readable> {
-	(await import('assert')).strict.equal(nogitignore, true);
-	const { filterFilesWithDockerignore } = await import('./ignore');
-	const { toPosixPath } = (await import('resin-multibuild')).PathUtils;
-
-	const serviceDirs = multiDockerignore
-		? await getServiceDirsFromComposition(dir, composition)
-		: {};
-
-	let readFile: (file: string) => Promise<Buffer>;
-	if (process.platform === 'win32') {
-		const { readFileWithEolConversion } = require('./eol-conversion');
-		readFile = (file) => readFileWithEolConversion(file, convertEol);
-	} else {
-		readFile = fs.readFile;
-	}
-	const tar = await import('tar-stream');
-	const pack = tar.pack();
-	const {
-		filteredFileList,
-		dockerignoreFiles,
-	} = await filterFilesWithDockerignore(dir, serviceDirs);
-	printDockerignoreWarn(dockerignoreFiles, serviceDirs, multiDockerignore);
-	for (const fileStats of filteredFileList) {
-		pack.entry(
-			{
-				name: toPosixPath(fileStats.relPath),
-				mtime: fileStats.stats.mtime,
-				mode: fileStats.stats.mode,
-				size: fileStats.stats.size,
-			},
-			await readFile(fileStats.filePath),
-		);
-	}
-	if (preFinalizeCallback) {
-		await preFinalizeCallback(pack);
-	}
-	pack.finalize();
-	return pack;
-}
-
-/**
- * Print warning messages for unused .dockerignore files, and info messages if
- * the --multi-dockerignore (-m) option is used in certain circumstances.
- * @param dockerignoreFiles All .dockerignore files found in the project
- * @param serviceDirsByService Map of service names to service subdirectories
- * @param multiDockerignore Whether --multi-dockerignore (-m) was provided
- */
-export function printDockerignoreWarn(
-	dockerignoreFiles: Array<import('./ignore').FileStats>,
-	serviceDirsByService: Dictionary<string>,
-	multiDockerignore: boolean,
-) {
-	let rootDockerignore: import('./ignore').FileStats | undefined;
-	const logger = Logger.getLogger();
-	const relPrefix = '.' + path.sep;
-	const serviceDirs = Object.values(serviceDirsByService || {});
-	// compute a list of unused .dockerignore files
-	const unusedFiles = dockerignoreFiles.filter(
-		(dockerignoreStats: import('./ignore').FileStats) => {
-			let dirname = path.dirname(dockerignoreStats.relPath);
-			dirname = dirname.startsWith(relPrefix) ? dirname.slice(2) : dirname;
-			const isProjectRootDir = !dirname || dirname === '.';
-			if (isProjectRootDir) {
-				rootDockerignore = dockerignoreStats;
-				return false; // a root .dockerignore file is always used
-			}
-			if (multiDockerignore) {
-				for (const serviceDir of serviceDirs) {
-					if (serviceDir === dirname) {
-						return false;
-					}
-				}
-			}
-			return true;
-		},
-	);
-	const msg: string[] = [];
-	let logFunc = logger.logWarn;
-	// Warn about unused .dockerignore files
-	if (unusedFiles.length) {
-		msg.push(
-			'The following .dockerignore file(s) will not be used:',
-			...unusedFiles.map((fileStats) => `* ${fileStats.filePath}`),
-		);
-		if (multiDockerignore) {
-			msg.push(stripIndent`
-				When --multi-dockerignore (-m) is used, only .dockerignore files at the root of
-				each service's build context (in a microservices/multicontainer application),
-				plus a .dockerignore file at the overall project root, are used.
-				See "balena help ${Logger.command}" for more details.`);
-		} else {
-			msg.push(stripIndent`
-				By default, only one .dockerignore file at the source folder (project root)
-				is used. Microservices (multicontainer) applications may use a separate
-				.dockerignore file for each service with the --multi-dockerignore (-m) option.
-				See "balena help ${Logger.command}" for more details.`);
-		}
-	}
-	// No unused .dockerignore files. Print info-level advice in some cases.
-	else if (multiDockerignore) {
-		logFunc = logger.logInfo;
-		// multi-container app with a root .dockerignore file
-		if (serviceDirs.length && rootDockerignore) {
-			msg.push(
-				stripIndent`
-				The --multi-dockerignore option is being used, and a .dockerignore file was
-				found at the project source (root) directory. Note that this file will not
-				be used to filter service subdirectories. See "balena help ${Logger.command}".`,
-			);
-		}
-		// single-container app
-		else if (serviceDirs.length === 0) {
-			msg.push(
-				stripIndent`
-				The --multi-dockerignore (-m) option was specified, but it has no effect for
-				single-container (non-microservices) apps. Only one .dockerignore file at the
-				project source (root) directory, if any, is used. See "balena help ${Logger.command}".`,
-			);
-		}
-	}
-	if (msg.length) {
-		logFunc.call(logger, [' ', hr, ...msg, hr].join('\n'));
-	}
-}
-
-/**
- * Print a deprecation warning if any '.gitignore' or '.dockerignore' file is
- * found and the --gitignore (-g) option has been provided (v11 compatibility).
- * @param dockerignoreFile Absolute path to a .dockerignore file
- * @param gitignoreFiles Array of absolute paths to .gitginore files
- */
-export function printGitignoreWarn(
-	dockerignoreFile: string,
-	gitignoreFiles: string[],
-) {
-	const ignoreFiles = [dockerignoreFile, ...gitignoreFiles].filter((e) => e);
-	if (ignoreFiles.length === 0) {
-		return;
-	}
-	const msg = [' ', hr, 'Using file ignore patterns from:'];
-	msg.push(...ignoreFiles.map((e) => `* ${e}`));
-	if (gitignoreFiles.length) {
-		msg.push(stripIndent`
-			.gitignore files are being considered because the --gitignore option was used.
-			This option is deprecated and will be removed in the next major version release.
-			For more information, see 'balena help ${Logger.command}'.
-		`);
-		msg.push(hr);
-		Logger.getLogger().logWarn(msg.join('\n'));
-	} else if (dockerignoreFile && process.platform === 'win32') {
-		msg.push(stripIndent`
-			The --gitignore option was used, but no .gitignore files were found.
-			The --gitignore option is deprecated and will be removed in the next major
-			version release. It prevents the use of a better dockerignore parser and
-			filter library that fixes several issues on Windows and improves compatibility
-			with 'docker build'. For more information, see 'balena help ${Logger.command}'.
-		`);
-		msg.push(hr);
-		Logger.getLogger().logWarn(msg.join('\n'));
-	}
-}
-
-/**
- * Check whether the "build secrets" feature is being used and, if so,
- * verify that the target docker daemon is balenaEngine. If the
- * requirement is not satisfied, reject with an ExpectedError.
- * @param docker Dockerode instance
- * @param sourceDir Project directory where to find .balena/balena.yml
- */
-export async function checkBuildSecretsRequirements(
-	docker: Dockerode,
-	sourceDir: string,
-) {
-	const [metaObj, metaFilename] = await loadBuildMetatada(sourceDir);
-	if (metaObj && !_.isEmpty(metaObj['build-secrets'])) {
-		const dockerUtils = await import('./docker');
-		const isBalenaEngine = await dockerUtils.isBalenaEngine(docker);
-		if (!isBalenaEngine) {
-			throw new ExpectedError(stripIndent`
-				The "build secrets" feature currently requires balenaEngine, but a standard Docker
-				daemon was detected. Please use command-line options to specify the hostname and
-				port number (or socket path) of a balenaEngine daemon, running on a balena device
-				or a virtual machine with balenaOS. If the build secrets feature is not required,
-				comment out or delete the 'build-secrets' entry in the file:
-				"${metaFilename}"
-				`);
-		}
-	}
-}
-
-export async function getRegistrySecrets(
-	sdk: BalenaSDK,
-	inputFilename?: string,
-): Promise<RegistrySecrets> {
-	if (inputFilename != null) {
-		return await parseRegistrySecrets(inputFilename);
-	}
-
-	const directory = await sdk.settings.get('dataDirectory');
-	const potentialPaths = [
-		path.join(directory, 'secrets.yml'),
-		path.join(directory, 'secrets.yaml'),
-		path.join(directory, 'secrets.json'),
-	];
-
-	for (const potentialPath of potentialPaths) {
-		if (await exists(potentialPath)) {
-			return await parseRegistrySecrets(potentialPath);
-		}
-	}
-
-	return {};
-}
-
-async function parseRegistrySecrets(
-	secretsFilename: string,
-): Promise<RegistrySecrets> {
-	try {
-		let isYaml = false;
-		if (/.+\.ya?ml$/i.test(secretsFilename)) {
-			isYaml = true;
-		} else if (!/.+\.json$/i.test(secretsFilename)) {
-			throw new ExpectedError('Filename must end with .json, .yml or .yaml');
-		}
-		const raw = (await fs.readFile(secretsFilename)).toString();
-		const multiBuild = await import('resin-multibuild');
-		const registrySecrets = new multiBuild.RegistrySecretValidator().validateRegistrySecrets(
-			isYaml ? require('js-yaml').safeLoad(raw) : JSON.parse(raw),
-		);
-		multiBuild.addCanonicalDockerHubEntry(registrySecrets);
-		return registrySecrets;
-	} catch (error) {
-		throw new ExpectedError(
-			`Error validating registry secrets file "${secretsFilename}":\n${error.message}`,
-		);
-	}
-}
-
-/**
- * Create a BuildTask array of "resolved build tasks" by calling multibuild
- * .splitBuildStream() and performResolution(), and add build stream error
- * handlers and debug logging.
- * Both `balena build` and `balena deploy` call this function.
- */
-export async function makeBuildTasks(
-	composition: Composition,
-	tarStream: Readable,
-	deviceInfo: DeviceInfo,
-	logger: Logger,
-	projectName: string,
-	releaseHash: string = 'unavailable',
-	preprocessHook?: (dockerfile: string) => string,
-): Promise<MultiBuild.BuildTask[]> {
-	const multiBuild = await import('resin-multibuild');
-	const buildTasks = await multiBuild.splitBuildStream(composition, tarStream);
-
-	logger.logDebug('Found build tasks:');
-	_.each(buildTasks, (task) => {
-		let infoStr: string;
-		if (task.external) {
-			infoStr = `image pull [${task.imageName}]`;
-		} else {
-			infoStr = `build [${task.context}]`;
-		}
-		logger.logDebug(`    ${task.serviceName}: ${infoStr}`);
-	});
-
-	logger.logDebug(
-		`Resolving services with [${deviceInfo.deviceType}|${deviceInfo.arch}]`,
-	);
-
-	await performResolution(
-		buildTasks,
-		deviceInfo,
-		projectName,
-		releaseHash,
-		preprocessHook,
-	);
-
-	logger.logDebug('Found project types:');
-	_.each(buildTasks, (task) => {
-		if (task.external) {
-			logger.logDebug(`    ${task.serviceName}: External image`);
-		} else {
-			logger.logDebug(`    ${task.serviceName}: ${task.projectType}`);
-		}
-	});
-
-	return buildTasks;
-}
-
-async function performResolution(
-	tasks: MultiBuild.BuildTask[],
-	deviceInfo: DeviceInfo,
-	appName: string,
-	releaseHash: string,
-	preprocessHook?: (dockerfile: string) => string,
-): Promise<MultiBuild.BuildTask[]> {
-	const { cloneTarStream } = await import('tar-utils');
-	const multiBuild = await import('resin-multibuild');
-
-	return await new Promise<MultiBuild.BuildTask[]>((resolve, reject) => {
-		const buildTasks = multiBuild.performResolution(
-			tasks,
-			deviceInfo.arch,
-			deviceInfo.deviceType,
-			{ error: [reject] },
-			{
-				BALENA_RELEASE_HASH: releaseHash,
-				BALENA_APP_NAME: appName,
-			},
-			preprocessHook,
-		);
-		(async () => {
-			try {
-				// Do one task at a time in order to reduce peak memory usage. Resolves to buildTasks.
-				for (const buildTask of buildTasks) {
-					// buildStream is falsy for "external" tasks (image pull)
-					if (!buildTask.buildStream) {
-						continue;
-					}
-					// Consume each task.buildStream in order to trigger the
-					// resolution events that define fields like:
-					//     task.dockerfile, task.dockerfilePath,
-					//     task.projectType, task.resolved
-					// This mimics what is currently done in `resin-builder`.
-					const clonedStream: Pack = await cloneTarStream(
-						buildTask.buildStream,
-					);
-					buildTask.buildStream = clonedStream;
-					if (!buildTask.external && !buildTask.resolved) {
-						throw new ExpectedError(
-							`Project type for service "${buildTask.serviceName}" could not be determined. Missing a Dockerfile?`,
-						);
-					}
-				}
-				resolve(buildTasks);
-			} catch (e) {
-				reject(e);
-			}
-		})();
-	});
-}
-
-/**
- * Enforce that, for example, if 'myProject/MyDockerfile.template' is specified
- * as an alternativate Dockerfile name, then 'myProject/MyDockerfile' must not
- * exist.
- * Return the tar stream path (Posix, normalized) for the given dockerfilePath.
- * For example, on Windows, given a dockerfilePath of 'foo\..\bar\Dockerfile',
- * return 'bar/Dockerfile'. On Linux, given './bar/Dockerfile', return 'bar/Dockerfile'.
- *
- * @param projectPath The project source folder (-s command-line option)
- * @param dockerfilePath The alternative Dockerfile specified by the user
- * @return A normalized posix representation of dockerfilePath
- */
-async function validateSpecifiedDockerfile(
-	projectPath: string,
-	dockerfilePath: string,
-): Promise<string> {
-	const { contains, toNativePath, toPosixPath } = (
-		await import('resin-multibuild')
-	).PathUtils;
-
-	const nativeProjectPath = path.normalize(projectPath);
-	const nativeDockerfilePath = path.normalize(toNativePath(dockerfilePath));
-
-	// reminder: native windows paths may start with a drive specificaton,
-	// e.g. 'C:\absolute' or 'C:relative'.
-	if (path.isAbsolute(nativeDockerfilePath)) {
-		throw new ExpectedError(stripIndent`
-			Error: the specified Dockerfile cannot be an absolute path. The path must be
-			relative to, and not a parent folder of, the project's source folder.
-			Specified dockerfile: "${nativeDockerfilePath}"
-			Project's source folder: "${nativeProjectPath}"
-		`);
-	}
-
-	// note that path.normalize('a/../../b') results in '../b'
-	if (nativeDockerfilePath.startsWith('..')) {
-		throw new ExpectedError(stripIndent`
-			Error: the specified Dockerfile cannot be in a parent folder of the project's
-			source folder. Note that the path should be relative to the project's source
-			folder, not the current folder.
-			Specified dockerfile: "${nativeDockerfilePath}"
-			Project's source folder: "${nativeProjectPath}"
-		`);
-	}
-
-	const fullDockerfilePath = path.join(nativeProjectPath, nativeDockerfilePath);
-
-	if (!(await exists(fullDockerfilePath))) {
-		throw new ExpectedError(stripIndent`
-			Error: specified Dockerfile not found:
-			Specified dockerfile: "${fullDockerfilePath}"
-			Project's source folder: "${nativeProjectPath}"
-			Note that the specified Dockerfile path should be relative to the source folder.
-		`);
-	}
-
-	if (!contains(nativeProjectPath, fullDockerfilePath)) {
-		throw new ExpectedError(stripIndent`
-			Error: the specified Dockerfile must be in a subfolder of the source folder:
-			Specified dockerfile: "${fullDockerfilePath}"
-			Project's source folder: "${nativeProjectPath}"
-		`);
-	}
-
-	return toPosixPath(nativeDockerfilePath);
-}
-
-export interface ProjectValidationResult {
-	dockerfilePath: string;
-	registrySecrets: RegistrySecrets;
-}
-
-/**
- * Perform "sanity checks" on the project directory, e.g. for the existence
- * of a 'Dockerfile[.*]' or 'docker-compose.yml' file or 'package.json' file.
- * Also validate registry secrets if any, and perform checks around an
- * alternative specified dockerfile (--dockerfile) if any.
- *
- * Return the parsed registry secrets if any, and the "tar stream path" for
- * an alternative specified Dockerfile if any (see validateSpecifiedDockerfile()).
- */
-export async function validateProjectDirectory(
-	sdk: BalenaSDK,
-	opts: {
-		dockerfilePath?: string;
-		noParentCheck: boolean;
-		projectPath: string;
-		registrySecretsPath?: string;
-	},
-): Promise<ProjectValidationResult> {
-	if (
-		!(await exists(opts.projectPath)) ||
-		!(await fs.stat(opts.projectPath)).isDirectory()
-	) {
-		throw new ExpectedError(
-			`Could not access source folder: "${opts.projectPath}"`,
-		);
-	}
-
-	const result: ProjectValidationResult = {
-		dockerfilePath: opts.dockerfilePath || '',
-		registrySecrets: {},
-	};
-
-	if (opts.dockerfilePath) {
-		result.dockerfilePath = await validateSpecifiedDockerfile(
-			opts.projectPath,
-			opts.dockerfilePath,
-		);
-	} else {
-		const files = await fs.readdir(opts.projectPath);
-		const projectMatch = (file: string) =>
-			/^(Dockerfile|Dockerfile\.\S+|docker-compose.ya?ml|package.json)$/.test(
-				file,
-			);
-		if (!_.some(files, projectMatch)) {
-			throw new ExpectedError(stripIndent`
-				Error: no "Dockerfile[.*]", "docker-compose.yml" or "package.json" file
-				found in source folder "${opts.projectPath}"
-			`);
-		}
-		if (!opts.noParentCheck) {
-			const checkCompose = async (folder: string) => {
-				return _.some(
-					await Promise.all(
-						compositionFileNames.map((filename) =>
-							exists(path.join(folder, filename)),
-						),
-					),
-				);
-			};
-			const [hasCompose, hasParentCompose] = await Promise.all([
-				checkCompose(opts.projectPath),
-				checkCompose(path.join(opts.projectPath, '..')),
-			]);
-			if (!hasCompose && hasParentCompose) {
-				const { isV12 } = await import('./version');
-				const msg = stripIndent`
-					"docker-compose.y[a]ml" file found in parent directory: please check that
-					the correct source folder was specified. (Suppress with '--noparent-check'.)`;
-				if (isV12()) {
-					throw new ExpectedError(`Error: ${msg}`);
-				} else {
-					Logger.getLogger().logWarn(msg);
-				}
-			}
-		}
-	}
-	result.registrySecrets = await getRegistrySecrets(
-		sdk,
-		opts.registrySecretsPath,
-	);
-
-	return result;
-}
-
-async function getTokenForPreviousRepos(
-	docker: import('docker-toolbelt'),
-	logger: Logger,
-	appId: number,
-	apiEndpoint: string,
-	taggedImages: TaggedImage[],
-): Promise<string> {
-	logger.logDebug('Authorizing push...');
-	const { authorizePush, getPreviousRepos } = await import('./compose');
-	const sdk = getBalenaSdk();
-	const previousRepos = await getPreviousRepos(sdk, docker, logger, appId);
-
-	const token = await authorizePush(
-		sdk,
-		apiEndpoint,
-		taggedImages[0].registry,
-		_.map(taggedImages, 'repo'),
-		previousRepos,
-	);
-	return token;
-}
-
-async function pushServiceImages(
-	docker: import('docker-toolbelt'),
-	logger: Logger,
-	pineClient: ReturnType<typeof import('balena-release').createClient>,
-	taggedImages: TaggedImage[],
-	token: string,
-	skipLogUpload: boolean,
-): Promise<void> {
-	const { pushAndUpdateServiceImages } = await import('./compose');
-	const releaseMod = await import('balena-release');
-	logger.logInfo('Pushing images to registry...');
-	await pushAndUpdateServiceImages(docker, token, taggedImages, async function (
-		serviceImage,
-	) {
-		logger.logDebug(
-			`Saving image ${serviceImage.is_stored_at__image_location}`,
-		);
-		if (skipLogUpload) {
-			delete serviceImage.build_log;
-		}
-		await releaseMod.updateImage(pineClient, serviceImage.id, serviceImage);
-	});
-}
-
-export async function deployProject(
-	docker: import('docker-toolbelt'),
-	logger: Logger,
-	composition: import('resin-compose-parse').Composition,
-	images: BuiltImage[],
-	appId: number,
-	userId: number,
-	auth: string,
-	apiEndpoint: string,
-	skipLogUpload: boolean,
-): Promise<Partial<import('balena-release/build/models').ReleaseModel>> {
-	const releaseMod = await import('balena-release');
-	const { createRelease, tagServiceImages } = await import('./compose');
-	const tty = (await import('./tty'))(process.stdout);
-
-	const prefix = getChalk().cyan('[Info]') + '    ';
-	const spinner = createSpinner();
-	let runloop = runSpinner(tty, spinner, `${prefix}Creating release...`);
-
-	let $release: Release;
-	try {
-		$release = await createRelease(
-			apiEndpoint,
-			auth,
-			userId,
-			appId,
-			composition,
-		);
-	} finally {
-		runloop.end();
-	}
-	const { client: pineClient, release, serviceImages } = $release;
-
-	try {
-		logger.logDebug('Tagging images...');
-		const taggedImages = await tagServiceImages(docker, images, serviceImages);
-		try {
-			const token = await getTokenForPreviousRepos(
-				docker,
-				logger,
-				appId,
-				apiEndpoint,
-				taggedImages,
-			);
-			await pushServiceImages(
-				docker,
-				logger,
-				pineClient,
-				taggedImages,
-				token,
-				skipLogUpload,
-			);
-			release.status = 'success';
-		} catch (err) {
-			release.status = 'failed';
-			throw err;
-		} finally {
-			logger.logDebug('Untagging images...');
-			await Promise.all(
-				taggedImages.map(({ localImage }) => localImage.remove()),
-			);
-		}
-	} finally {
-		runloop = runSpinner(tty, spinner, `${prefix}Saving release...`);
-		release.end_timestamp = new Date();
-		if (release.id != null) {
-			try {
-				await releaseMod.updateRelease(pineClient, release.id, release);
-			} finally {
-				runloop.end();
-			}
-		}
-	}
-	return release;
-}
-
-export function createSpinner() {
-	const chars = '|/-\\';
-	let index = 0;
-	return () => chars[index++ % chars.length];
-}
-
-function runSpinner(
-	tty: ReturnType<typeof import('./tty')>,
-	spinner: () => string,
-	msg: string,
-) {
-	const runloop = createRunLoop(function () {
-		tty.clearLine();
-		tty.writeLine(`${msg} ${spinner()}`);
-		return tty.cursorUp();
-	});
-	runloop.onEnd = function () {
-		tty.clearLine();
-		return tty.writeLine(msg);
-	};
-	return runloop;
-}
-
-export function createRunLoop(tick: (...args: any[]) => void) {
-	const timerId = setInterval(tick, 1000 / 10);
-	const runloop = {
-		onEnd() {
-			// noop
-		},
-		end() {
-			clearInterval(timerId);
-			return runloop.onEnd();
-		},
-	};
-	return runloop;
-}
-
-export const composeCliFlags: flags.Input<ComposeCliFlags> = {
-	emulated: flags.boolean({
-		description:
-			'Use QEMU for ARM architecture emulation during the image build',
-		char: 'e',
-	}),
-	dockerfile: flags.string({
-		description:
-			'Alternative Dockerfile name/path, relative to the source folder',
-	}),
-	logs: flags.boolean({
-		description:
-			'No-op and deprecated since balenaCLI v12.0.0. Build logs are now shown by default.',
-	}),
-	nologs: flags.boolean({
-		description:
-			'Hide the image build log output (produce less verbose output)',
-	}),
-	gitignore: flags.boolean({
-		description: stripIndent`
-				Consider .gitignore files in addition to the .dockerignore file. This reverts
-				to the CLI v11 behavior/implementation (deprecated) if compatibility is required
-				until your project can be adapted.`,
-		char: 'g',
-	}),
-	'multi-dockerignore': flags.boolean({
-		description:
-			'Have each service use its own .dockerignore file. See "balena help build".',
-		char: 'm',
-	}),
-	nogitignore: flags.boolean({
-		description: `No-op (default behavior) since balenaCLI v12.0.0. See "balena help build".`,
-		char: 'G',
-	}),
-	'noparent-check': flags.boolean({
-		description:
-			"Disable project validation check of 'docker-compose.yml' file in parent folder",
-	}),
-	'registry-secrets': flags.string({
-		description:
-			'Path to a YAML or JSON file with passwords for a private Docker registry',
-		char: 'R',
-	}),
-	'convert-eol': flags.boolean({
-		description: 'No-op and deprecated since balenaCLI v12.0.0',
-		char: 'l',
-	}),
-	'noconvert-eol': flags.boolean({
-		description:
-			"Don't convert line endings from CRLF (Windows format) to LF (Unix format).",
-	}),
-	projectName: flags.string({
-		description:
-			'Specify an alternate project name; default is the directory name',
-		char: 'n',
-	}),
-};

lib/utils/config.ts

@@ -1,173 +0,0 @@
-/*
-Copyright 2016-2019 Balena
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-   http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-import type * as BalenaSdk from 'balena-sdk';
-import * as semver from 'balena-semver';
-import { getBalenaSdk } from './lazy';
-
-export interface ImgConfig {
-	applicationName: string;
-	applicationId: number;
-	deviceType: string;
-	userId: number;
-	username: string;
-	appUpdatePollInterval: number;
-	listenPort: number;
-	vpnPort: number;
-	apiEndpoint: string;
-	vpnEndpoint: string;
-	registryEndpoint: string;
-	deltaEndpoint: string;
-	mixpanelToken: string;
-	wifiSsid?: string;
-	wifiKey?: string;
-	initialDeviceName?: string;
-
-	// props for older OS versions
-	connectivity?: string;
-	files?: {
-		[filepath: string]: string;
-	};
-
-	// device specific config props
-	deviceId?: number;
-	uuid?: string;
-	registered_at?: number;
-
-	os?: {
-		sshKeys?: string[];
-	};
-}
-
-export async function generateBaseConfig(
-	application: BalenaSdk.Application,
-	options: {
-		version: string;
-		appUpdatePollInterval?: number;
-		deviceType?: string;
-		os?: {
-			sshKeys?: string[];
-		};
-	},
-): Promise<ImgConfig> {
-	options = {
-		...options,
-		appUpdatePollInterval: options.appUpdatePollInterval || 10,
-	};
-
-	const config = (await getBalenaSdk().models.os.getConfig(
-		application.app_name,
-		options,
-	)) as ImgConfig & { apiKey?: string };
-	// os.getConfig always returns a config for an app
-	delete config.apiKey;
-
-	// merge sshKeys to config, when they have been specified
-	if (options.os && options.os.sshKeys) {
-		// Create config.os object if it does not exist
-		config.os = config.os ? config.os : {};
-		config.os.sshKeys = config.os.sshKeys
-			? [...config.os.sshKeys, ...options.os.sshKeys]
-			: options.os.sshKeys;
-	}
-
-	return config;
-}
-
-export async function generateApplicationConfig(
-	application: BalenaSdk.Application,
-	options: {
-		version: string;
-		deviceType?: string;
-		appUpdatePollInterval?: number;
-	},
-) {
-	const config = await generateBaseConfig(application, options);
-
-	if (semver.satisfies(options.version, '<2.7.8')) {
-		await addApplicationKey(config, application.id);
-	} else {
-		await addProvisioningKey(config, application.id);
-	}
-
-	return config;
-}
-
-export function generateDeviceConfig(
-	device: DeviceWithDeviceType & {
-		belongs_to__application: BalenaSdk.PineDeferred;
-	},
-	deviceApiKey: string | true | undefined,
-	options: { version: string },
-) {
-	return getBalenaSdk()
-		.models.application.get(device.belongs_to__application.__id)
-		.then(async (application) => {
-			const baseConfigOpts = {
-				...options,
-				deviceType: device.is_of__device_type[0].slug,
-			};
-			const config = await generateBaseConfig(application, baseConfigOpts);
-
-			if (deviceApiKey == null && semver.satisfies(options.version, '<2.0.3')) {
-				await addApplicationKey(config, application.id);
-			} else {
-				await addDeviceKey(config, device.uuid, deviceApiKey || true);
-			}
-
-			return config;
-		})
-		.then((config) => {
-			// Associate a device, to prevent the supervisor
-			// from creating another one on its own.
-			config.registered_at = Math.floor(Date.now() / 1000);
-			config.deviceId = device.id;
-			config.uuid = device.uuid;
-
-			return config;
-		});
-}
-
-function addApplicationKey(config: any, applicationNameOrId: string | number) {
-	return getBalenaSdk()
-		.models.application.generateApiKey(applicationNameOrId)
-		.then((apiKey) => {
-			config.apiKey = apiKey;
-			return apiKey;
-		});
-}
-
-function addProvisioningKey(config: any, applicationNameOrId: string | number) {
-	return getBalenaSdk()
-		.models.application.generateProvisioningKey(applicationNameOrId)
-		.then((apiKey) => {
-			config.apiKey = apiKey;
-			return apiKey;
-		});
-}
-
-async function addDeviceKey(
-	config: any,
-	uuid: string,
-	customDeviceApiKey: string | true,
-) {
-	if (customDeviceApiKey === true) {
-		config.deviceApiKey = await getBalenaSdk().models.device.generateDeviceKey(
-			uuid,
-		);
-	} else {
-		config.deviceApiKey = customDeviceApiKey;
-	}
-}

lib/utils/device/logs.ts

@@ -1,119 +0,0 @@
-import ColorHash = require('color-hash');
-import * as _ from 'lodash';
-import type { Readable } from 'stream';
-
-import { getChalk } from '../lazy';
-import Logger = require('../logger');
-
-interface Log {
-	message: string;
-	timestamp?: number;
-	serviceName?: string;
-
-	// There's also a serviceId and imageId, but they're
-	// meaningless in local mode
-}
-
-interface BuildLog {
-	serviceName: string;
-	message: string;
-}
-
-/**
- * Display logs from a device logging stream. This function will return
- * when the log stream ends.
- *
- * @param logs A stream which produces newline seperated log
- * 	objects
- * @param logger A Logger instance which the logs will be
- * 	displayed through
- * @param system Only show system (and potentially the
- * 	filterService) logs
- * @param filterService Filter the logs so that only logs
- * 	from a single service will be displayed
- */
-export function displayDeviceLogs(
-	logs: Readable,
-	logger: Logger,
-	system: boolean,
-	filterServices?: string[],
-): Promise<void> {
-	return new Promise((resolve, reject) => {
-		logs.on('data', (log) => {
-			displayLogLine(log, logger, system, filterServices);
-		});
-
-		logs.on('error', reject);
-		logs.on('end', () => {
-			logger.logError('Connection to device lost');
-			resolve();
-		});
-	});
-}
-
-export function displayBuildLog(log: BuildLog, logger: Logger): void {
-	const toPrint = `${getServiceColourFn(log.serviceName)(
-		`[${log.serviceName}]`,
-	)} ${log.message}`;
-	logger.logBuild(toPrint);
-}
-
-// mutates serviceColours
-function displayLogLine(
-	log: string | Buffer,
-	logger: Logger,
-	system: boolean,
-	filterServices?: string[],
-): void {
-	try {
-		const obj: Log = JSON.parse(log.toString());
-		displayLogObject(obj, logger, system, filterServices);
-	} catch (e) {
-		logger.logDebug(`Dropping device log due to failed parsing: ${e}`);
-	}
-}
-
-export function displayLogObject<T extends Log>(
-	obj: T,
-	logger: Logger,
-	system: boolean,
-	filterServices?: string[],
-): void {
-	let toPrint: string;
-	if (obj.timestamp != null) {
-		toPrint = `[${new Date(obj.timestamp).toLocaleString()}]`;
-	} else {
-		toPrint = `[${new Date().toLocaleString()}]`;
-	}
-
-	if (obj.serviceName != null) {
-		if (filterServices) {
-			if (!_.includes(filterServices, obj.serviceName)) {
-				return;
-			}
-		} else if (system) {
-			return;
-		}
-
-		const colourFn = getServiceColourFn(obj.serviceName);
-
-		toPrint += ` ${colourFn(`[${obj.serviceName}]`)}`;
-	} else if (filterServices != null && !system) {
-		// We have a system log here but we are filtering based
-		// on a service, so drop this too
-		return;
-	}
-
-	toPrint += ` ${obj.message}`;
-
-	logger.logLogs(toPrint);
-}
-
-export const getServiceColourFn = _.memoize(_getServiceColourFn);
-
-const colorHash = new ColorHash();
-function _getServiceColourFn(serviceName: string): (msg: string) => string {
-	const [r, g, b] = colorHash.rgb(serviceName);
-
-	return getChalk().rgb(r, g, b);
-}

lib/utils/device/ssh.ts

@@ -1,120 +0,0 @@
-/*
-Copyright 2016-2020 Balena
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-   http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-import type { ContainerInfo } from 'dockerode';
-import { stripIndent } from '../lazy';
-
-export interface DeviceSSHOpts {
-	address: string;
-	port?: number;
-	forceTTY?: boolean;
-	verbose: boolean;
-	service?: string;
-}
-
-export const deviceContainerEngineBinary = `$(if [ -f /usr/bin/balena ]; then echo "balena"; else echo "docker"; fi)`;
-
-export async function performLocalDeviceSSH(
-	opts: DeviceSSHOpts,
-): Promise<void> {
-	const { escapeRegExp, reduce } = await import('lodash');
-	const { spawnSshAndThrowOnError } = await import('../ssh');
-	const { ExpectedError } = await import('../../errors');
-
-	let command = '';
-
-	if (opts.service != null) {
-		// Get the containers which are on-device. Currently we
-		// are single application, which means we can assume any
-		// container which fulfills the form of
-		// $serviceName_$appId_$releaseId is what we want. Once
-		// we have multi-app, we should show a dialog which
-		// allows the user to choose the correct container
-
-		const Docker = await import('dockerode');
-		const docker = new Docker({
-			host: opts.address,
-			port: 2375,
-		});
-
-		const regex = new RegExp(`\\/?${escapeRegExp(opts.service)}_\\d+_\\d+`);
-		const nameRegex = /\/?([a-zA-Z0-9_]+)_\d+_\d+/;
-		let allContainers: ContainerInfo[];
-		try {
-			allContainers = await docker.listContainers();
-		} catch (_e) {
-			throw new ExpectedError(stripIndent`
-				Could not access docker daemon on device ${opts.address}.
-				Please ensure the device is in local mode.`);
-		}
-
-		const serviceNames: string[] = [];
-		const containers = allContainers
-			.map((container) => {
-				for (const name of container.Names) {
-					if (regex.test(name)) {
-						return { id: container.Id, name };
-					}
-					const match = name.match(nameRegex);
-					if (match) {
-						serviceNames.push(match[1]);
-					}
-				}
-				return;
-			})
-			.filter((c) => c != null);
-
-		if (containers.length === 0) {
-			throw new ExpectedError(
-				`Could not find a service on device with name ${opts.service}. ${
-					serviceNames.length > 0
-						? `Available services:\n${reduce(
-								serviceNames,
-								(str, name) => `${str}\t${name}\n`,
-								'',
-						  )}`
-						: ''
-				}`,
-			);
-		}
-		if (containers.length > 1) {
-			throw new ExpectedError(stripIndent`
-				Found more than one container with a service name ${opts.service}.
-				This state is not supported, please contact support.
-			`);
-		}
-
-		const containerId = containers[0]!.id;
-		const shellCmd = `/bin/sh -c "if [ -e /bin/bash ]; then exec /bin/bash; else exec /bin/sh; fi"`;
-		// stdin (fd=0) is not a tty when data is piped in, for example
-		// echo 'ls -la; exit;' | balena ssh 192.168.0.20 service1
-		// See https://www.balena.io/blog/balena-monthly-roundup-january-2020/#charliestipsntricks
-		//     https://assets.balena.io/newsletter/2020-01/pipe.png
-		const isTTY = !!opts.forceTTY || (await import('tty')).isatty(0);
-		const ttyFlag = isTTY ? '-t' : '';
-		command = `${deviceContainerEngineBinary} exec -i ${ttyFlag} ${containerId} ${shellCmd}`;
-	}
-
-	return spawnSshAndThrowOnError([
-		...(opts.verbose ? ['-vvv'] : []),
-		'-t',
-		...['-p', opts.port ? opts.port.toString() : '22222'],
-		...['-o', 'LogLevel=ERROR'],
-		...['-o', 'StrictHostKeyChecking=no'],
-		...['-o', 'UserKnownHostsFile=/dev/null'],
-		`root@${opts.address}`,
-		...(command ? [command] : []),
-	]);
-}

lib/utils/docker-js.js

@@ -1,216 +0,0 @@
-/**
- * @license
- * Copyright 2017-2020 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-// Functions to help actions which rely on using docker
-
-import * as _ from 'lodash';
-import { ExpectedError } from '../errors';
-
-const generateConnectOpts = async function (opts) {
-	const { promises: fs } = await import('fs');
-
-	const connectOpts = {};
-
-	// Start with docker-modem defaults which take several env vars into account,
-	// including DOCKER_HOST, DOCKER_TLS_VERIFY, DOCKER_CERT_PATH, SSH_AUTH_SOCK
-	// https://github.com/apocas/docker-modem/blob/v2.1.3/lib/modem.js#L15-L65
-	const Modem = await import('docker-modem');
-	const defaultOpts = new Modem();
-	const optsOfInterest = [
-		'ca',
-		'cert',
-		'key',
-		'host',
-		'port',
-		'socketPath',
-		'protocol',
-		'username',
-		'sshAuthAgent',
-		'timeout',
-	];
-	for (const opt of optsOfInterest) {
-		connectOpts[opt] = defaultOpts[opt];
-	}
-
-	// Now override the default options with any explicit command line options
-	if (opts.docker != null && opts.dockerHost == null) {
-		// good, local docker socket
-		connectOpts.socketPath = opts.docker;
-		delete connectOpts.host;
-		delete connectOpts.port;
-	} else if (opts.dockerHost != null && opts.docker == null) {
-		// Good a host is provided, and local socket isn't
-		connectOpts.host = opts.dockerHost;
-		connectOpts.port = opts.dockerPort || 2376;
-		delete connectOpts.socketPath;
-	} else if (opts.docker != null && opts.dockerHost != null) {
-		// Both provided, no obvious way to continue
-		throw new ExpectedError(
-			"Both a local docker socket and docker host have been provided. Don't know how to continue.",
-		);
-	}
-
-	// Now need to check if the user wants to connect over TLS
-	// to the host
-
-	// If any are set...
-	if (opts.ca != null || opts.cert != null || opts.key != null) {
-		// but not all
-		if (!(opts.ca != null && opts.cert != null && opts.key != null)) {
-			throw new ExpectedError(
-				'You must provide a CA, certificate and key in order to use TLS',
-			);
-		}
-
-		const [ca, cert, key] = await Promise.all([
-			fs.readFile(opts.ca, 'utf-8'),
-			fs.readFile(opts.cert, 'utf-8'),
-			fs.readFile(opts.key, 'utf-8'),
-		]);
-		return _.merge(connectOpts, {
-			ca,
-			cert,
-			key,
-		});
-	}
-
-	return connectOpts;
-};
-
-const parseBuildArgs = function (args) {
-	if (!Array.isArray(args)) {
-		args = [args];
-	}
-	const buildArgs = {};
-	args.forEach(function (arg) {
-		// note: [^] matches any character, including line breaks
-		const pair = /^([^\s]+?)=([^]*)$/.exec(arg);
-		if (pair != null) {
-			buildArgs[pair[1]] = pair[2] ?? '';
-		} else {
-			throw new ExpectedError(`Could not parse build argument: '${arg}'`);
-		}
-	});
-	return buildArgs;
-};
-
-export function generateBuildOpts(options) {
-	const opts = {};
-	if (options.tag != null) {
-		opts.t = options.tag;
-	}
-	if (options.nocache != null) {
-		opts.nocache = true;
-	}
-	if (options['cache-from']?.trim()) {
-		opts.cachefrom = options['cache-from'].split(',').filter((i) => !!i.trim());
-	}
-	if (options.pull != null) {
-		opts.pull = true;
-	}
-	if (options.squash != null) {
-		opts.squash = true;
-	}
-	if (options.buildArg != null) {
-		opts.buildargs = parseBuildArgs(options.buildArg);
-	}
-	if (!_.isEmpty(options['registry-secrets'])) {
-		opts.registryconfig = options['registry-secrets'];
-	}
-	return opts;
-}
-/**
- * @param {{
- * 	ca?: string; // path to ca (Certificate Authority) file (TLS)
- * 	cert?: string; // path to cert (Certificate) file (TLS)
- * 	key?: string; // path to key file (TLS)
- * 	docker?: string; // dockerode DockerOptions.socketPath
- * 	dockerHost?: string; // dockerode DockerOptions.host
- * 	dockerPort?: number; // dockerode DockerOptions.port
- * 	host?: string;
- * 	port?: number;
- * 	timeout?: number;
- * }} options
- * @returns {Promise<import('docker-toolbelt')>}
- */
-export async function getDocker(options) {
-	const connectOpts = await generateConnectOpts(options);
-	const client = createClient(connectOpts);
-	await ensureDockerSeemsAccessible(client);
-	return client;
-}
-
-const getDockerToolbelt = _.once(function () {
-	const Docker = require('docker-toolbelt');
-	const Bluebird = require('bluebird');
-	Bluebird.promisifyAll(Docker.prototype, {
-		filter(name) {
-			return name === 'run';
-		},
-		multiArgs: true,
-	});
-	Bluebird.promisifyAll(Docker.prototype);
-	// @ts-ignore `getImage()` should have a param but this whole thing is a hack that should be removed
-	Bluebird.promisifyAll(new Docker({}).getImage().constructor.prototype);
-	// @ts-ignore `getContainer()` should have a param but this whole thing is a hack that should be removed
-	Bluebird.promisifyAll(new Docker({}).getContainer().constructor.prototype);
-	return Docker;
-});
-
-// docker-toolbelt v3 is not backwards compatible as it removes all *Async
-// methods that are in wide use in the CLI. The workaround for now is to
-// manually promisify the client and replace all `new Docker()` calls with
-// this shared function that returns a promisified client.
-//
-//    **New code must not use the *Async methods.**
-//
-/**
- * @param {{
- * 	host: string;
- * 	port: number;
- * 	timeout?: number;
- * 	socketPath?: string
- * }} opts
- * @returns {import('docker-toolbelt')}
- */
-export const createClient = function (opts) {
-	const Docker = getDockerToolbelt();
-	const docker = new Docker(opts);
-	const { modem } = docker;
-	// Workaround for a docker-modem 2.0.x bug where it sets a default
-	// socketPath on Windows even if the input options specify a host/port.
-	if (modem.socketPath && modem.host) {
-		if (opts.socketPath) {
-			modem.host = undefined;
-			modem.port = undefined;
-		} else if (opts.host) {
-			modem.socketPath = undefined;
-		}
-	}
-	return docker;
-};
-
-var ensureDockerSeemsAccessible = function (docker) {
-	const { exitWithExpectedError } = require('../errors');
-	return docker
-		.ping()
-		.catch((e) =>
-			exitWithExpectedError(
-				`Docker seems to be unavailable. Is it installed and running?\n${e}`,
-			),
-		);
-};

lib/utils/docker.ts

@@ -1,109 +0,0 @@
-/**
- * @license
- * Copyright 2018-2019 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-import type * as dockerode from 'dockerode';
-import { flags } from '@oclif/command';
-import { parseAsInteger } from './validation';
-
-export * from './docker-js';
-
-interface BalenaEngineVersion extends dockerode.DockerVersion {
-	Engine?: string;
-}
-
-export interface DockerConnectionCliFlags {
-	docker?: string;
-	dockerHost?: string;
-	dockerPort?: number;
-	ca?: string;
-	cert?: string;
-	key?: string;
-}
-
-export interface DockerCliFlags extends DockerConnectionCliFlags {
-	tag?: string;
-	buildArg?: string[];
-	'cache-from'?: string;
-	nocache: boolean;
-	pull?: boolean;
-	squash: boolean;
-}
-
-export const dockerConnectionCliFlags: flags.Input<DockerConnectionCliFlags> = {
-	docker: flags.string({
-		description: 'Path to a local docker socket (e.g. /var/run/docker.sock)',
-		char: 'P',
-	}),
-	dockerHost: flags.string({
-		description:
-			'Docker daemon hostname or IP address (dev machine or balena device) ',
-		char: 'h',
-	}),
-	dockerPort: flags.integer({
-		description:
-			'Docker daemon TCP port number (hint: 2375 for balena devices)',
-		char: 'p',
-		parse: (p) => parseAsInteger(p, 'dockerPort'),
-	}),
-	ca: flags.string({
-		description: 'Docker host TLS certificate authority file',
-	}),
-	cert: flags.string({
-		description: 'Docker host TLS certificate file',
-	}),
-	key: flags.string({
-		description: 'Docker host TLS key file',
-	}),
-};
-
-export const dockerCliFlags: flags.Input<DockerCliFlags> = {
-	tag: flags.string({
-		description: 'The alias to the generated image',
-		char: 't',
-	}),
-	buildArg: flags.string({
-		description:
-			'Set a build-time variable (eg. "-B \'ARG=value\'"). Can be specified multiple times.',
-		char: 'B',
-		multiple: true,
-	}),
-	'cache-from': flags.string({
-		description: `\
-Comma-separated list (no spaces) of image names for build cache resolution. \
-Implements the same feature as the "docker build --cache-from" option.`,
-	}),
-	nocache: flags.boolean({
-		description: "Don't use docker layer caching when building",
-	}),
-	pull: flags.boolean({
-		description: 'Pull the base images again even if they exist locally',
-	}),
-	squash: flags.boolean({
-		description: 'Squash newly built layers into a single new layer',
-	}),
-	...dockerConnectionCliFlags,
-};
-
-export async function isBalenaEngine(docker: dockerode): Promise<boolean> {
-	// dockerVersion.Engine should equal 'balena-engine' for the current/latest
-	// version of balenaEngine, but it was at one point (mis)spelt 'balaena':
-	// https://github.com/balena-os/balena-engine/pull/32/files
-	const dockerVersion = (await docker.version()) as BalenaEngineVersion;
-	return !!(
-		dockerVersion.Engine && dockerVersion.Engine.match(/balena|balaena/)
-	);
-}

lib/utils/ignore.ts

@@ -1,331 +0,0 @@
-/**
- * @license
- * Copyright 2019 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-import * as _ from 'lodash';
-import { promises as fs, Stats } from 'fs';
-import * as path from 'path';
-import * as MultiBuild from 'resin-multibuild';
-
-import dockerIgnore = require('@zeit/dockerignore');
-import ignore from 'ignore';
-
-import { ExpectedError } from '../errors';
-
-const { toPosixPath } = MultiBuild.PathUtils;
-
-export enum IgnoreFileType {
-	DockerIgnore,
-	GitIgnore,
-}
-
-interface IgnoreEntry {
-	pattern: string;
-	// The relative file path from the base path of the build context
-	filePath: string;
-}
-
-/**
- * This class is used by the CLI v10 / v11 "original" tarDirectory function
- * in `compose.js`. It is still around for the benefit of the `--gitignore`
- * option, but is expected to be deleted in CLI v13.
- */
-export class FileIgnorer {
-	private dockerIgnoreEntries: IgnoreEntry[];
-	private gitIgnoreEntries: IgnoreEntry[];
-
-	private static ignoreFiles: Array<{
-		pattern: string;
-		type: IgnoreFileType;
-		allowSubdirs: boolean;
-	}> = [
-		{
-			pattern: '.gitignore',
-			type: IgnoreFileType.GitIgnore,
-			allowSubdirs: true,
-		},
-		{
-			pattern: '.dockerignore',
-			type: IgnoreFileType.DockerIgnore,
-			allowSubdirs: false,
-		},
-	];
-
-	public constructor(public basePath: string) {
-		this.dockerIgnoreEntries = [];
-		this.gitIgnoreEntries = [];
-	}
-	/**
-	 * @param  {string} relativePath
-	 * 	The relative pathname from the build context, for example a root level .gitignore should be
-	 * 		./.gitignore
-	 * @returns IgnoreFileType
-	 * 	The type of ignore file, or null
-	 */
-	public getIgnoreFileType(relativePath: string): IgnoreFileType | null {
-		for (const { pattern, type, allowSubdirs } of FileIgnorer.ignoreFiles) {
-			if (
-				path.basename(relativePath) === pattern &&
-				(allowSubdirs || path.dirname(relativePath) === '.')
-			) {
-				return type;
-			}
-		}
-
-		return null;
-	}
-	/**
-	 * @param  {string} fullPath
-	 * 	The full path on disk of the ignore file
-	 * @param  {IgnoreFileType} type
-	 * @returns Promise
-	 */
-	public async addIgnoreFile(
-		fullPath: string,
-		type: IgnoreFileType,
-	): Promise<void> {
-		const contents = await fs.readFile(fullPath, 'utf8');
-
-		contents.split('\n').forEach((line) => {
-			// ignore empty lines and comments
-			if (/\s*#/.test(line) || _.isEmpty(line)) {
-				return;
-			}
-
-			this.addEntry(line, fullPath, type);
-		});
-
-		return;
-	}
-
-	// Pass this function as a predicate to a filter function, and it will filter
-	// any ignored files
-	public filter = (filename: string): boolean => {
-		const relFile = path.relative(this.basePath, filename);
-
-		// Don't ignore any metadata files
-		// The regex below matches `.balena/qemu` and `myservice/.balena/qemu`
-		// but not `some.dir.for.balena/qemu`.
-		if (/(^|\/)\.(balena|resin)\//.test(toPosixPath(relFile))) {
-			return true;
-		}
-
-		// Don't ignore Dockerfile (with or without extension) or docker-compose.yml
-		if (
-			/^Dockerfile$|^Dockerfile\.\S+/.test(path.basename(relFile)) ||
-			path.basename(relFile) === 'docker-compose.yml'
-		) {
-			return true;
-		}
-
-		const dockerIgnoreHandle = dockerIgnore();
-		const gitIgnoreHandle = ignore();
-
-		interface IgnoreHandle {
-			add: (pattern: string) => void;
-			ignores: (file: string) => boolean;
-		}
-
-		const ignoreTypes: Array<{
-			handle: IgnoreHandle;
-			entries: IgnoreEntry[];
-		}> = [
-			{ handle: dockerIgnoreHandle, entries: this.dockerIgnoreEntries },
-			{ handle: gitIgnoreHandle, entries: this.gitIgnoreEntries },
-		];
-
-		_.each(ignoreTypes, ({ handle, entries }) => {
-			_.each(entries, ({ pattern, filePath }) => {
-				if (FileIgnorer.contains(path.posix.dirname(filePath), filename)) {
-					handle.add(pattern);
-				}
-			});
-		});
-
-		return !_.some(ignoreTypes, ({ handle }) => handle.ignores(relFile));
-	}; // tslint:disable-line:semicolon
-
-	private addEntry(
-		pattern: string,
-		filePath: string,
-		type: IgnoreFileType,
-	): void {
-		const entry: IgnoreEntry = { pattern, filePath };
-		switch (type) {
-			case IgnoreFileType.DockerIgnore:
-				this.dockerIgnoreEntries.push(entry);
-				break;
-			case IgnoreFileType.GitIgnore:
-				this.gitIgnoreEntries.push(entry);
-				break;
-		}
-	}
-
-	/**
-	 * Given two paths, check whether the first contains the second
-	 * @param path1 The potentially containing path
-	 * @param path2 The potentially contained path
-	 * @return A boolean indicating whether `path1` contains `path2`
-	 */
-	private static contains(path1: string, path2: string): boolean {
-		// First normalise the input, to remove any path weirdness
-		path1 = path.posix.normalize(path1);
-		path2 = path.posix.normalize(path2);
-
-		// Now test if the start of the relative path contains ../ ,
-		// which would tell us that path1 is not part of path2
-		return !/^\.\.\//.test(path.posix.relative(path1, path2));
-	}
-}
-
-export interface FileStats {
-	filePath: string;
-	relPath: string;
-	stats: Stats;
-}
-
-/**
- * Create a list of files (FileStats[]) for the filesystem subtree rooted at
- * projectDir, listing each file with both a full path and a relative path,
- * but excluding entries for directories themselves.
- * @param projectDir Source directory (root of subtree to be listed)
- * @param dir Used for recursive calls only (omit on first function call)
- */
-async function listFiles(
-	projectDir: string,
-	dir: string = projectDir,
-): Promise<FileStats[]> {
-	const files: FileStats[] = [];
-	const dirEntries = await fs.readdir(dir);
-	await Promise.all(
-		dirEntries.map(async (entry) => {
-			const filePath = path.join(dir, entry);
-			const stats = await fs.stat(filePath);
-			if (stats.isDirectory()) {
-				files.push(...(await listFiles(projectDir, filePath)));
-			} else if (stats.isFile()) {
-				files.push({
-					filePath,
-					relPath: path.relative(projectDir, filePath),
-					stats,
-				});
-			}
-		}),
-	);
-	return files;
-}
-
-/**
- * Return the contents of a .dockerignore file at projectDir, as a string.
- * Return an empty string if a .dockerignore file does not exist.
- * @param projectDir Source directory
- * @returns Contents of the .dockerignore file, as a UTF-8 string
- */
-async function readDockerIgnoreFile(projectDir: string): Promise<string> {
-	const dockerIgnorePath = path.join(projectDir, '.dockerignore');
-	let dockerIgnoreStr = '';
-	try {
-		dockerIgnoreStr = await fs.readFile(dockerIgnorePath, 'utf8');
-	} catch (err) {
-		if (err.code !== 'ENOENT') {
-			throw new ExpectedError(
-				`Error reading file "${dockerIgnorePath}": ${err.message}`,
-			);
-		}
-	}
-	return dockerIgnoreStr;
-}
-
-/**
- * Create an instance of '@balena/dockerignore', initialized with the contents
- * of a .dockerignore file (if any) found at the given directory argument, plus
- * a set of default/hardcoded patterns.
- * @param directory Directory where to look for a .dockerignore file
- */
-async function getDockerIgnoreInstance(
-	directory: string,
-): Promise<import('@balena/dockerignore').Ignore> {
-	const dockerIgnoreStr = await readDockerIgnoreFile(directory);
-	const $dockerIgnore = (await import('@balena/dockerignore')).default;
-	const ig = $dockerIgnore({ ignorecase: false });
-
-	ig.add(['**/.git']);
-	if (dockerIgnoreStr) {
-		ig.add(dockerIgnoreStr);
-	}
-	ig.add([
-		'!**/.balena',
-		'!**/.resin',
-		'!**/Dockerfile',
-		'!**/Dockerfile.*',
-		'!**/docker-compose.yml',
-	]);
-	return ig;
-}
-
-export interface ServiceDirs {
-	[service: string]: string;
-}
-
-/**
- * Create a list of files (FileStats[]) for the filesystem subtree rooted at
- * projectDir, filtered against the applicable .dockerignore files, including
- * a few default/hardcoded dockerignore patterns.
- * @param projectDir Source directory to
- * @param serviceDirsByService Map of service names to their subdirectories.
- * The service directory names/paths must be relative to the project root dir
- * and be "normalized" (path.normalize()) before the call to this function:
- * they should use backslashes on Windows, not contain '.' or '..' segments and
- * not contain multiple consecutive path separators like '//'. Also, relative
- * paths must not start with './' (e.g. 'a/b' instead of './a/b').
- */
-export async function filterFilesWithDockerignore(
-	projectDir: string,
-	serviceDirsByService?: ServiceDirs,
-): Promise<{ filteredFileList: FileStats[]; dockerignoreFiles: FileStats[] }> {
-	// path.resolve() also converts forward slashes to backslashes on Windows
-	projectDir = path.resolve(projectDir);
-	// ignoreByDir stores an instance of the dockerignore filter for each service dir
-	const ignoreByDir: {
-		[serviceDir: string]: import('@balena/dockerignore').Ignore;
-	} = {
-		'.': await getDockerIgnoreInstance(projectDir),
-	};
-	const serviceDirs: string[] = Object.values(serviceDirsByService || {})
-		// filter out the project source/root dir
-		.filter((dir) => dir && dir !== '.')
-		// add a trailing '/' (or '\' on Windows) to the path
-		.map((dir) => (dir.endsWith(path.sep) ? dir : dir + path.sep));
-
-	for (const serviceDir of serviceDirs) {
-		ignoreByDir[serviceDir] = await getDockerIgnoreInstance(
-			path.join(projectDir, serviceDir),
-		);
-	}
-	const files = await listFiles(projectDir);
-	const dockerignoreFiles: FileStats[] = [];
-	const filteredFileList = files.filter((file: FileStats) => {
-		if (path.basename(file.relPath) === '.dockerignore') {
-			dockerignoreFiles.push(file);
-		}
-		for (const dir of serviceDirs) {
-			if (file.relPath.startsWith(dir)) {
-				return !ignoreByDir[dir].ignores(file.relPath.substring(dir.length));
-			}
-		}
-		return !ignoreByDir['.'].ignores(file.relPath);
-	});
-	return { filteredFileList, dockerignoreFiles };
-}

lib/utils/messages.ts

@@ -1,121 +0,0 @@
-/**
- * @license
- * Copyright 2017-2020 Balena Ltd.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-const DEBUG_MODE = !!process.env.DEBUG;
-
-export const reachingOut = `\
-If you need help, or just want to say hi, don't hesitate in reaching out
-through our discussion and support forums at https://forums.balena.io
-
-For bug reports or feature requests, have a look at the GitHub issues or
-create a new one at: https://github.com/balena-io/balena-cli/issues/\
-`;
-
-const debugHint = `\
-Additional information may be available with the \`--debug\` flag.
-`;
-
-export const help = `\
-For help, visit our support forums: https://forums.balena.io
-For bug reports or feature requests, see: https://github.com/balena-io/balena-cli/issues/
-`;
-
-export const getHelp = (DEBUG_MODE ? '' : debugHint) + help;
-
-export const balenaAsciiArt = `\
- _            _
-| |__   __ _ | |  ____  _ __    __ _
-| '_ \\ / _\` || | / __ \\| '_ \\  / _\` |
-| |_) | (_) || ||  ___/| | | || (_) |
-|_.__/ \\__,_||_| \\____/|_| |_| \\__,_|
-`;
-
-export const registrySecretsHelp = `\
-REGISTRY SECRETS  
-The --registry-secrets option specifies a JSON or YAML file containing private
-Docker registry usernames and passwords to be used when pulling base images.
-Sample registry-secrets YAML file:
-\`\`\`
-	'my-registry-server.com:25000':
-		username: ann
-		password: hunter2
-	'':  # Use the empty string to refer to the Docker Hub
-		username: mike
-		password: cze14
-	'eu.gcr.io':  # Google Container Registry
-		username: '_json_key'
-		password: '{escaped contents of the GCR keyfile.json file}'
-\`\`\`
-For a sample project using registry secrets with the Google Container Registry,
-check: https://github.com/balena-io-examples/sample-gcr-registry-secrets
-
-If the --registry-secrets option is not specified, and a secrets.yml or
-secrets.json file exists in the balena directory (usually $HOME/.balena),
-this file will be used instead.`;
-
-export const dockerignoreHelp = `\
-DOCKERIGNORE AND GITIGNORE FILES  
-By default, balenaCLI will use a single ".dockerignore" file (if any) at
-the project root (--source directory) in order to decide which source files to
-exclude from the "build context" (tar stream) sent to balenaCloud, Docker
-daemon or balenaEngine. In a microservices (multicontainer) application, the
-source directory is the directory that contains the "docker-compose.yml" file.
-
-The --multi-dockerignore (-m) option may be used with microservices
-(multicontainer) applications that define a docker-compose.yml file. When
-this option is used, each service subdirectory (defined by the \`build\` or
-\`build.context\` service properties in the docker-compose.yml file) is
-filtered separately according to a .dockerignore file defined in the service
-subdirectory. If no .dockerignore file exists in a service subdirectory, then
-only the default .dockerignore patterns (see below) apply for that service
-subdirectory.
-
-When the --multi-dockerignore (-m) option is used, the .dockerignore file (if
-any) defined at the overall project root will be used to filter files and
-subdirectories other than service subdirectories. It will not have any effect
-on service subdirectories, whether or not a service subdirectory defines its
-own .dockerignore file. Multiple .dockerignore files are not merged or added
-together, and cannot override or extend other files. This behavior maximises
-compatibility with the standard docker-compose tool, while still allowing a
-root .dockerignore file (at the overall project root) to filter files and
-folders that are outside service subdirectories.
-
-balenaCLI releases older than v12.0.0 also took .gitignore files into account.
-This behavior is deprecated, but may still be enabled with the --gitignore (-g)
-option if compatibility is required. This option is mutually exclusive with
---multi-dockerignore (-m) and will be removed in the CLI's next major version
-release (v13).
-
-Default .dockerignore patterns  
-When --gitignore (-g) is NOT used (i.e. when not in v11 compatibility mode), a
-few default/hardcoded dockerignore patterns are "merged" (in memory) with the
-patterns found in the applicable .dockerignore files, in the following order:
-\`\`\`
-    **/.git
-    < user's patterns from the applicable '.dockerignore' file, if any >
-    !**/.balena
-    !**/.resin
-    !**/Dockerfile
-    !**/Dockerfile.*
-    !**/docker-compose.yml
-\`\`\`
-These patterns always apply, whether or not .dockerignore files exist in the
-project. If necessary, the effect of the \`**/.git\` pattern may be modified by
-adding counter patterns to the applicable .dockerignore file(s), for example
-\`!mysubmodule/.git\`. For documentation on pattern format, see:
-- https://docs.docker.com/engine/reference/builder/#dockerignore-file
-- https://www.npmjs.com/package/@balena/dockerignore`;