diff --git a/doc/cli.markdown b/doc/cli.markdown index c9169a54..b104a53a 100644 --- a/doc/cli.markdown +++ b/doc/cli.markdown @@ -1943,11 +1943,11 @@ containers. The synchronization is only in one direction, from this machine to the device, and changes made on the device itself may be overwritten. This feature requires a device running supervisor version v9.7.0 or greater. -REGISTRY SECRETS +REGISTRY SECRETS The --registry-secrets option specifies a JSON or YAML file containing private Docker registry usernames and passwords to be used when pulling base images. Sample registry-secrets YAML file: - +``` 'my-registry-server.com:25000': username: ann password: hunter2 @@ -1957,7 +1957,7 @@ Sample registry-secrets YAML file: 'eu.gcr.io': # Google Container Registry username: '_json_key' password: '{escaped contents of the GCR keyfile.json file}' - +``` For a sample project using registry secrets with the Google Container Registry, check: https://github.com/balena-io-playground/sample-gcr-registry-secrets @@ -1965,37 +1965,54 @@ If the --registry-secrets option is not specified, and a secrets.yml or secrets.json file exists in the balena directory (usually $HOME/.balena), this file will be used instead. -DOCKERIGNORE AND GITIGNORE FILES -The balena CLI will use a '.dockerignore' file (if any) at the source directory -in order to decide which source files to exclude from the "build context" sent -to balenaCloud, Docker or balenaEngine. In a microservices / multicontainer -application, the source directory is usually where the 'docker-compose.yml' -file is located, and therefore the '.dockerignore' file should be located -alongside the 'docker-compose.yml' file. Matching patterns may be prefixed with -the service's directory name (relative to the source directory) in order to -apply to that service only (e.g. 'service1/node_modules'). +DOCKERIGNORE AND GITIGNORE FILES +By default, the balena CLI will use a single ".dockerignore" file (if any) at +the project root (--source directory) in order to decide which source files to +exclude from the "build context" (tar stream) sent to balenaCloud, Docker daemon +or balenaEngine. In a microservices (multicontainer) application, the source +directory is the directory that contains the "docker-compose.yml" file. -Previous balena CLI releases (before v12.0.0) also took '.gitignore' files -into account. This behavior is deprecated, but may still be enabled with the ---gitignore (-g) option if compatibility is required. This option will be -removed in the CLI's next major version release (v13). +The --multi-dockerignore (-m) option may be used with microservices (multicontainer) +applications that define a docker-compose.yml file. When this option is used, +each service subdirectory (defined by the `build` or `build.context` service +properties in the docker-compose.yml file) is filtered separately according to +a .dockerignore file defined in the service subdirectory. If no .dockerignore +file exists in a service subdirectory, then only the default .dockerignore +patterns (see below) apply for that service subdirectory. -When --gitignore (-g) is NOT provided (i.e. when not in v11 compatibility mode), -a few "hardcoded" dockerignore patterns are also used and "merged" (in memory) -with the patterns found in the '.dockerignore' file (if any), in the following -order: +When the --multi-dockerignore (-m) option is used, the .dockerignore file (if +any) defined at the overall project root will be used to filter files and +subdirectories other than service subdirectories. It will not have any effect +on service subdirectories, whether or not a service subdirectory defines its +own .dockerignore file. Multiple .dockerignore files are not merged or added +together, and cannot override or extend other files. This behavior maximises +compatibility with the standard docker-compose tool, while still allowing a +root .dockerignore file (at the overall project root) to filter files and +folders that are outside service subdirectories. +Balena CLI releases older than v12.0.0 also took .gitignore files into account. +This behavior is deprecated, but may still be enabled with the --gitignore (-g) +option if compatibility is required. This option is mutually exclusive with +--multi-dockerignore (-m) and will be removed in the CLI's next major version +release (v13). + +Default .dockerignore patterns +When --gitignore (-g) is NOT used (i.e. when not in v11 compatibility mode), a +few default/hardcoded dockerignore patterns are "merged" (in memory) with the +patterns found in the applicable .dockerignore files, in the following order: +``` **/.git - < user's patterns from the '.dockerignore' file, if any > + < user's patterns from the applicable '.dockerignore' file, if any > !**/.balena !**/.resin !**/Dockerfile !**/Dockerfile.* !**/docker-compose.yml - -If necessary, the effect of the '**/.git' pattern may be modified by adding -"counter patterns" to the '.dockerignore' file, for example '!service1/.git'. -For documentation on pattern format, see: +``` +These patterns always apply, whether or not .dockerignore files exist in the +project. If necessary, the effect of the `**/.git` pattern may be modified by +adding counter patterns to the applicable .dockerignore file(s), for example +`!mysubmodule/.git`. For documentation on pattern format, see: - https://docs.docker.com/engine/reference/builder/#dockerignore-file - https://www.npmjs.com/package/@balena/dockerignore @@ -2085,6 +2102,10 @@ No-op and deprecated since balena CLI v12.0.0 Don't convert line endings from CRLF (Windows format) to LF (Unix format). +#### --multi-dockerignore, -m + +Have each service use its own .dockerignore file. See "balena help push". + #### --nogitignore, -G No-op (default behavior) since balena CLI v12.0.0. See "balena help push". @@ -2157,11 +2178,11 @@ found, it will look for a Dockerfile[.template] file (or alternative Dockerfile specified with the `--dockerfile` option), and if no dockerfile is found, it will try to generate one. -REGISTRY SECRETS +REGISTRY SECRETS The --registry-secrets option specifies a JSON or YAML file containing private Docker registry usernames and passwords to be used when pulling base images. Sample registry-secrets YAML file: - +``` 'my-registry-server.com:25000': username: ann password: hunter2 @@ -2171,7 +2192,7 @@ Sample registry-secrets YAML file: 'eu.gcr.io': # Google Container Registry username: '_json_key' password: '{escaped contents of the GCR keyfile.json file}' - +``` For a sample project using registry secrets with the Google Container Registry, check: https://github.com/balena-io-playground/sample-gcr-registry-secrets @@ -2179,37 +2200,54 @@ If the --registry-secrets option is not specified, and a secrets.yml or secrets.json file exists in the balena directory (usually $HOME/.balena), this file will be used instead. -DOCKERIGNORE AND GITIGNORE FILES -The balena CLI will use a '.dockerignore' file (if any) at the source directory -in order to decide which source files to exclude from the "build context" sent -to balenaCloud, Docker or balenaEngine. In a microservices / multicontainer -application, the source directory is usually where the 'docker-compose.yml' -file is located, and therefore the '.dockerignore' file should be located -alongside the 'docker-compose.yml' file. Matching patterns may be prefixed with -the service's directory name (relative to the source directory) in order to -apply to that service only (e.g. 'service1/node_modules'). +DOCKERIGNORE AND GITIGNORE FILES +By default, the balena CLI will use a single ".dockerignore" file (if any) at +the project root (--source directory) in order to decide which source files to +exclude from the "build context" (tar stream) sent to balenaCloud, Docker daemon +or balenaEngine. In a microservices (multicontainer) application, the source +directory is the directory that contains the "docker-compose.yml" file. -Previous balena CLI releases (before v12.0.0) also took '.gitignore' files -into account. This behavior is deprecated, but may still be enabled with the ---gitignore (-g) option if compatibility is required. This option will be -removed in the CLI's next major version release (v13). +The --multi-dockerignore (-m) option may be used with microservices (multicontainer) +applications that define a docker-compose.yml file. When this option is used, +each service subdirectory (defined by the `build` or `build.context` service +properties in the docker-compose.yml file) is filtered separately according to +a .dockerignore file defined in the service subdirectory. If no .dockerignore +file exists in a service subdirectory, then only the default .dockerignore +patterns (see below) apply for that service subdirectory. -When --gitignore (-g) is NOT provided (i.e. when not in v11 compatibility mode), -a few "hardcoded" dockerignore patterns are also used and "merged" (in memory) -with the patterns found in the '.dockerignore' file (if any), in the following -order: +When the --multi-dockerignore (-m) option is used, the .dockerignore file (if +any) defined at the overall project root will be used to filter files and +subdirectories other than service subdirectories. It will not have any effect +on service subdirectories, whether or not a service subdirectory defines its +own .dockerignore file. Multiple .dockerignore files are not merged or added +together, and cannot override or extend other files. This behavior maximises +compatibility with the standard docker-compose tool, while still allowing a +root .dockerignore file (at the overall project root) to filter files and +folders that are outside service subdirectories. +Balena CLI releases older than v12.0.0 also took .gitignore files into account. +This behavior is deprecated, but may still be enabled with the --gitignore (-g) +option if compatibility is required. This option is mutually exclusive with +--multi-dockerignore (-m) and will be removed in the CLI's next major version +release (v13). + +Default .dockerignore patterns +When --gitignore (-g) is NOT used (i.e. when not in v11 compatibility mode), a +few default/hardcoded dockerignore patterns are "merged" (in memory) with the +patterns found in the applicable .dockerignore files, in the following order: +``` **/.git - < user's patterns from the '.dockerignore' file, if any > + < user's patterns from the applicable '.dockerignore' file, if any > !**/.balena !**/.resin !**/Dockerfile !**/Dockerfile.* !**/docker-compose.yml - -If necessary, the effect of the '**/.git' pattern may be modified by adding -"counter patterns" to the '.dockerignore' file, for example '!service1/.git'. -For documentation on pattern format, see: +``` +These patterns always apply, whether or not .dockerignore files exist in the +project. If necessary, the effect of the `**/.git` pattern may be modified by +adding counter patterns to the applicable .dockerignore file(s), for example +`!mysubmodule/.git`. For documentation on pattern format, see: - https://docs.docker.com/engine/reference/builder/#dockerignore-file - https://www.npmjs.com/package/@balena/dockerignore @@ -2263,6 +2301,10 @@ Consider .gitignore files in addition to the .dockerignore file. This reverts to the CLI v11 behavior/implementation (deprecated) if compatibility is required until your project can be adapted. +#### --multi-dockerignore, -m + +Have each service use its own .dockerignore file. See "balena help build". + #### --nogitignore, -G No-op (default behavior) since balena CLI v12.0.0. See "balena help build". @@ -2351,11 +2393,11 @@ To deploy to an app on which you're a collaborator, use When --build is used, all options supported by `balena build` are also supported by this command. -REGISTRY SECRETS +REGISTRY SECRETS The --registry-secrets option specifies a JSON or YAML file containing private Docker registry usernames and passwords to be used when pulling base images. Sample registry-secrets YAML file: - +``` 'my-registry-server.com:25000': username: ann password: hunter2 @@ -2365,7 +2407,7 @@ Sample registry-secrets YAML file: 'eu.gcr.io': # Google Container Registry username: '_json_key' password: '{escaped contents of the GCR keyfile.json file}' - +``` For a sample project using registry secrets with the Google Container Registry, check: https://github.com/balena-io-playground/sample-gcr-registry-secrets @@ -2373,37 +2415,54 @@ If the --registry-secrets option is not specified, and a secrets.yml or secrets.json file exists in the balena directory (usually $HOME/.balena), this file will be used instead. -DOCKERIGNORE AND GITIGNORE FILES -The balena CLI will use a '.dockerignore' file (if any) at the source directory -in order to decide which source files to exclude from the "build context" sent -to balenaCloud, Docker or balenaEngine. In a microservices / multicontainer -application, the source directory is usually where the 'docker-compose.yml' -file is located, and therefore the '.dockerignore' file should be located -alongside the 'docker-compose.yml' file. Matching patterns may be prefixed with -the service's directory name (relative to the source directory) in order to -apply to that service only (e.g. 'service1/node_modules'). +DOCKERIGNORE AND GITIGNORE FILES +By default, the balena CLI will use a single ".dockerignore" file (if any) at +the project root (--source directory) in order to decide which source files to +exclude from the "build context" (tar stream) sent to balenaCloud, Docker daemon +or balenaEngine. In a microservices (multicontainer) application, the source +directory is the directory that contains the "docker-compose.yml" file. -Previous balena CLI releases (before v12.0.0) also took '.gitignore' files -into account. This behavior is deprecated, but may still be enabled with the ---gitignore (-g) option if compatibility is required. This option will be -removed in the CLI's next major version release (v13). +The --multi-dockerignore (-m) option may be used with microservices (multicontainer) +applications that define a docker-compose.yml file. When this option is used, +each service subdirectory (defined by the `build` or `build.context` service +properties in the docker-compose.yml file) is filtered separately according to +a .dockerignore file defined in the service subdirectory. If no .dockerignore +file exists in a service subdirectory, then only the default .dockerignore +patterns (see below) apply for that service subdirectory. -When --gitignore (-g) is NOT provided (i.e. when not in v11 compatibility mode), -a few "hardcoded" dockerignore patterns are also used and "merged" (in memory) -with the patterns found in the '.dockerignore' file (if any), in the following -order: +When the --multi-dockerignore (-m) option is used, the .dockerignore file (if +any) defined at the overall project root will be used to filter files and +subdirectories other than service subdirectories. It will not have any effect +on service subdirectories, whether or not a service subdirectory defines its +own .dockerignore file. Multiple .dockerignore files are not merged or added +together, and cannot override or extend other files. This behavior maximises +compatibility with the standard docker-compose tool, while still allowing a +root .dockerignore file (at the overall project root) to filter files and +folders that are outside service subdirectories. +Balena CLI releases older than v12.0.0 also took .gitignore files into account. +This behavior is deprecated, but may still be enabled with the --gitignore (-g) +option if compatibility is required. This option is mutually exclusive with +--multi-dockerignore (-m) and will be removed in the CLI's next major version +release (v13). + +Default .dockerignore patterns +When --gitignore (-g) is NOT used (i.e. when not in v11 compatibility mode), a +few default/hardcoded dockerignore patterns are "merged" (in memory) with the +patterns found in the applicable .dockerignore files, in the following order: +``` **/.git - < user's patterns from the '.dockerignore' file, if any > + < user's patterns from the applicable '.dockerignore' file, if any > !**/.balena !**/.resin !**/Dockerfile !**/Dockerfile.* !**/docker-compose.yml - -If necessary, the effect of the '**/.git' pattern may be modified by adding -"counter patterns" to the '.dockerignore' file, for example '!service1/.git'. -For documentation on pattern format, see: +``` +These patterns always apply, whether or not .dockerignore files exist in the +project. If necessary, the effect of the `**/.git` pattern may be modified by +adding counter patterns to the applicable .dockerignore file(s), for example +`!mysubmodule/.git`. For documentation on pattern format, see: - https://docs.docker.com/engine/reference/builder/#dockerignore-file - https://www.npmjs.com/package/@balena/dockerignore @@ -2453,6 +2512,10 @@ Consider .gitignore files in addition to the .dockerignore file. This reverts to the CLI v11 behavior/implementation (deprecated) if compatibility is required until your project can be adapted. +#### --multi-dockerignore, -m + +Have each service use its own .dockerignore file. See "balena help build". + #### --nogitignore, -G No-op (default behavior) since balena CLI v12.0.0. See "balena help build". diff --git a/lib/actions/build.js b/lib/actions/build.js index f8b38519..4fa8adea 100644 --- a/lib/actions/build.js +++ b/lib/actions/build.js @@ -24,15 +24,19 @@ import * as compose from '../utils/compose'; import { dockerignoreHelp, registrySecretsHelp } from '../utils/messages'; import { getBalenaSdk } from '../utils/lazy'; -/* -Opts must be an object with the following keys: - - app: the app this build is for (optional) - arch: the architecture to build for - deviceType: the device type to build for - buildEmulated - buildOpts: arguments to forward to docker build command -*/ +/** + * Opts must be an object with the following keys: + * app: the app this build is for (optional) + * arch: the architecture to build for + * deviceType: the device type to build for + * buildEmulated + * buildOpts: arguments to forward to docker build command + * + * @param {import('docker-toolbelt')} docker + * @param {import('../utils/logger')} logger + * @param {import('../utils/compose-types').ComposeOpts} composeOpts + * @param {any} opts + */ const buildProject = function (docker, logger, composeOpts, opts) { const { loadProject } = require('../utils/compose_ts'); return Bluebird.resolve(loadProject(logger, composeOpts)) @@ -63,6 +67,7 @@ const buildProject = function (docker, logger, composeOpts, opts) { composeOpts.convertEol, composeOpts.dockerfilePath, composeOpts.nogitignore, + composeOpts.multiDockerignore, ); }) .then(function () { diff --git a/lib/actions/deploy.js b/lib/actions/deploy.js index a4c5a8c3..e0af7a48 100644 --- a/lib/actions/deploy.js +++ b/lib/actions/deploy.js @@ -25,17 +25,21 @@ import { dockerignoreHelp, registrySecretsHelp } from '../utils/messages'; import { ExpectedError } from '../errors'; import { getBalenaSdk, getChalk } from '../utils/lazy'; -/* -Opts must be an object with the following keys: - - app: the application instance to deploy to - image: the image to deploy; optional - dockerfilePath: name of an alternative Dockerfile; optional - shouldPerformBuild - shouldUploadLogs - buildEmulated - buildOpts: arguments to forward to docker build command -*/ +/** + * Opts must be an object with the following keys: + * app: the application instance to deploy to + * image: the image to deploy; optional + * dockerfilePath: name of an alternative Dockerfile; optional + * shouldPerformBuild + * shouldUploadLogs + * buildEmulated + * buildOpts: arguments to forward to docker build command + * + * @param {any} docker + * @param {import('../utils/logger')} logger + * @param {import('../utils/compose-types').ComposeOpts} composeOpts + * @param {any} opts + */ const deployProject = function (docker, logger, composeOpts, opts) { const _ = require('lodash'); const doodles = require('resin-doodles'); @@ -100,6 +104,7 @@ const deployProject = function (docker, logger, composeOpts, opts) { composeOpts.convertEol, composeOpts.dockerfilePath, composeOpts.nogitignore, + composeOpts.multiDockerignore, ) .then((builtImages) => _.keyBy(builtImages, 'serviceName')); }) diff --git a/lib/actions/push.ts b/lib/actions/push.ts index 31b2b351..fc300627 100644 --- a/lib/actions/push.ts +++ b/lib/actions/push.ts @@ -118,6 +118,7 @@ export const push: CommandDefinition< env?: string | string[]; 'convert-eol'?: boolean; 'noconvert-eol'?: boolean; + 'multi-dockerignore'?: boolean; } > = { signature: 'push ', @@ -276,6 +277,13 @@ export const push: CommandDefinition< }, ] : []), + { + signature: 'multi-dockerignore', + alias: 'm', + description: + 'Have each service use its own .dockerignore file. See "balena help push".', + boolean: true, + }, { signature: 'nogitignore', alias: 'G', @@ -307,6 +315,11 @@ export const push: CommandDefinition< if (appOrDevice == null) { throw new ExpectedError('You must specify an application or a device'); } + if (options.gitignore && options['multi-dockerignore']) { + throw new ExpectedError( + 'The --gitignore and --multi-dockerignore options cannot be used together', + ); + } const source = options.source || '.'; if (process.env.DEBUG) { @@ -363,6 +376,7 @@ export const push: CommandDefinition< const opts = { dockerfilePath, emulated: options.emulated || false, + multiDockerignore: options['multi-dockerignore'] || false, nocache: options.nocache || false, registrySecrets, headless: options.detached || false, @@ -397,6 +411,7 @@ export const push: CommandDefinition< deviceHost: device, dockerfilePath, registrySecrets, + multiDockerignore: options['multi-dockerignore'] || false, nocache: options.nocache || false, nogitignore, noParentCheck: options['noparent-check'] || false, diff --git a/lib/utils/compose-types.d.ts b/lib/utils/compose-types.d.ts index c8a13fae..1ae4c62a 100644 --- a/lib/utils/compose-types.d.ts +++ b/lib/utils/compose-types.d.ts @@ -48,6 +48,8 @@ export interface ComposeOpts { convertEol: boolean; dockerfilePath?: string; inlineLogs?: boolean; + multiDockerignore: boolean; + nogitignore: boolean; noParentCheck: boolean; projectName: string; projectPath: string; @@ -67,7 +69,9 @@ export interface Release { } interface TarDirectoryOptions { + composition?: Composition; convertEol?: boolean; - preFinalizeCallback?: (pack: Pack) => void | Promise; + multiDockerignore?: boolean; nogitignore: boolean; + preFinalizeCallback?: (pack: Pack) => void | Promise; } diff --git a/lib/utils/compose.js b/lib/utils/compose.js index d3968608..689d43ee 100644 --- a/lib/utils/compose.js +++ b/lib/utils/compose.js @@ -18,6 +18,7 @@ import * as Bluebird from 'bluebird'; import * as path from 'path'; +import { ExpectedError } from '../errors'; import { getChalk, stripIndent } from './lazy'; export const appendProjectOptions = (opts) => @@ -72,6 +73,13 @@ export function appendOptions(opts) { until your project can be adapted.`, boolean: true, }, + { + signature: 'multi-dockerignore', + alias: 'm', + description: + 'Have each service use its own .dockerignore file. See "balena help build".', + boolean: true, + }, { signature: 'nogitignore', description: `No-op (default behavior) since balena CLI v12.0.0. See "balena help build".`, @@ -120,12 +128,19 @@ Source files are not modified.`, export function generateOpts(options) { const { promises: fs } = require('fs'); const { isV12 } = require('./version'); + + if (options.gitignore && options['multi-dockerignore']) { + throw new ExpectedError( + 'The --gitignore and --multi-dockerignore options cannot be used together', + ); + } return fs.realpath(options.source || '.').then((projectPath) => ({ projectName: options.projectName, projectPath, inlineLogs: !options.nologs && (!!options.logs || isV12()), convertEol: isV12() ? !options['noconvert-eol'] : !!options['convert-eol'], dockerfilePath: options.dockerfile, + multiDockerignore: !!options['multi-dockerignore'], nogitignore: !options.gitignore, noParentCheck: options['noparent-check'], })); @@ -309,6 +324,7 @@ export function buildProject( convertEol, dockerfilePath, nogitignore, + multiDockerignore, ) { const _ = require('lodash'); const humanize = require('humanize'); @@ -362,7 +378,12 @@ export function buildProject( .then(( needsQemu, // Tar up the directory, ready for the build stream ) => - tarDirectory(projectPath, { convertEol, nogitignore }) + tarDirectory(projectPath, { + composition, + convertEol, + multiDockerignore, + nogitignore, + }) .then((tarStream) => makeBuildTasks( composition, diff --git a/lib/utils/compose_ts.ts b/lib/utils/compose_ts.ts index bc6d612b..cd8a6548 100644 --- a/lib/utils/compose_ts.ts +++ b/lib/utils/compose_ts.ts @@ -57,6 +57,9 @@ const exists = async (filename: string) => { const compositionFileNames = ['docker-compose.yml', 'docker-compose.yaml']; +const hr = + '----------------------------------------------------------------------'; + /** * high-level function resolving a project and creating a composition out * of it in one go. if image is given, it'll create a default project for @@ -105,6 +108,7 @@ export async function loadProject( async function resolveProject( logger: Logger, projectRoot: string, + quiet = false, ): Promise<[string, string]> { let composeFileName = ''; let composeFileContents = ''; @@ -122,7 +126,7 @@ async function resolveProject( break; } } - if (!composeFileName) { + if (!quiet && !composeFileName) { logger.logInfo(`No "docker-compose.yml" file found at "${projectRoot}"`); } return [composeFileName, composeFileContents]; @@ -174,6 +178,59 @@ async function loadBuildMetatada( return [buildMetadata, metadataPath]; } +/** + * Return a map of service name to service subdirectory, obtained from the given + * composition object. If a composition object is not provided, an attempt will + * be made to parse a 'docker-compose.yml' file at the given sourceDir. + * Entries will be NOT be returned for subdirectories equal to '.' (e.g. the + * 'main' "service" of a single-container application). + * + * @param sourceDir Project source directory (project root) + * @param composition Optional previously parsed composition object + */ +async function getServiceDirsFromComposition( + sourceDir: string, + composition?: Composition, +): Promise> { + const { createProject } = await import('./compose'); + const serviceDirs: Dictionary = {}; + if (!composition) { + const [, composeStr] = await resolveProject( + Logger.getLogger(), + sourceDir, + true, + ); + if (composeStr) { + composition = createProject(sourceDir, composeStr).composition; + } + } + if (composition?.services) { + const relPrefix = '.' + path.sep; + for (const [serviceName, service] of Object.entries(composition.services)) { + let dir = + typeof service.build === 'string' + ? service.build + : service.build?.context || '.'; + // Convert forward slashes to backslashes on Windows + dir = path.normalize(dir); + // Make sure the path is relative to the project directory + if (path.isAbsolute(dir)) { + dir = path.relative(sourceDir, dir); + } + // remove a trailing '/' (or backslash on Windows) + dir = dir.endsWith(path.sep) ? dir.slice(0, -1) : dir; + // remove './' prefix (or '.\\' on Windows) + dir = dir.startsWith(relPrefix) ? dir.slice(2) : dir; + // filter out a '.' service directory (e.g. for the 'main' service + // of a single-container application) + if (dir && dir !== '.') { + serviceDirs[serviceName] = dir; + } + } + } + return serviceDirs; +} + /** * Create a tar stream out of the local filesystem at the given directory, * while optionally applying file filters such as '.dockerignore' and @@ -185,15 +242,21 @@ async function loadBuildMetatada( export async function tarDirectory( dir: string, { - preFinalizeCallback, + composition, convertEol = false, + multiDockerignore = false, nogitignore = false, + preFinalizeCallback, }: TarDirectoryOptions, ): Promise { (await import('assert')).strict.strictEqual(nogitignore, true); const { filterFilesWithDockerignore } = await import('./ignore'); const { toPosixPath } = (await import('resin-multibuild')).PathUtils; + const serviceDirs = multiDockerignore + ? await getServiceDirsFromComposition(dir, composition) + : {}; + let readFile: (file: string) => Promise; if (process.platform === 'win32') { const { readFileWithEolConversion } = require('./eol-conversion'); @@ -205,8 +268,8 @@ export async function tarDirectory( const { filteredFileList, dockerignoreFiles, - } = await filterFilesWithDockerignore(dir); - printDockerignoreWarn(dockerignoreFiles); + } = await filterFilesWithDockerignore(dir, serviceDirs); + printDockerignoreWarn(dockerignoreFiles, serviceDirs, multiDockerignore); for (const fileStats of filteredFileList) { pack.entry( { @@ -225,35 +288,89 @@ export async function tarDirectory( return pack; } +/** + * Print warning messages for unused .dockerignore files, and info messages if + * the --multi-dockerignore (-m) option is used in certain circumstances. + * @param dockerignoreFiles All .dockerignore files found in the project + * @param serviceDirsByService Map of service names to service subdirectories + * @param multiDockerignore Whether --multi-dockerignore (-m) was provided + */ export function printDockerignoreWarn( dockerignoreFiles: Array, + serviceDirsByService: Dictionary, + multiDockerignore: boolean, ) { - const nonRootFiles = dockerignoreFiles.filter( - (fileStats: import('./ignore').FileStats) => { - const dirname = path.dirname(fileStats.relPath); - return !!dirname && dirname !== '.'; + let rootDockerignore: import('./ignore').FileStats | undefined; + const logger = Logger.getLogger(); + const relPrefix = '.' + path.sep; + const serviceDirs = Object.values(serviceDirsByService || {}); + // compute a list of unused .dockerignore files + const unusedFiles = dockerignoreFiles.filter( + (dockerignoreStats: import('./ignore').FileStats) => { + let dirname = path.dirname(dockerignoreStats.relPath); + dirname = dirname.startsWith(relPrefix) ? dirname.slice(2) : dirname; + const isProjectRootDir = !dirname || dirname === '.'; + if (isProjectRootDir) { + rootDockerignore = dockerignoreStats; + return false; // a root .dockerignore file is always used + } + if (multiDockerignore) { + for (const serviceDir of serviceDirs) { + if (serviceDir === dirname) { + return false; + } + } + } + return true; }, ); - if (nonRootFiles.length === 0) { - return; + const msg: string[] = []; + let logFunc = logger.logWarn; + // Warn about unused .dockerignore files + if (unusedFiles.length) { + msg.push( + 'The following .dockerignore file(s) will not be used:', + ...unusedFiles.map((fileStats) => `* ${fileStats.filePath}`), + ); + if (multiDockerignore) { + msg.push(stripIndent` + When --multi-dockerignore (-m) is used, only .dockerignore files at the root of + each service's build context (in a microservices/multicontainer application), + plus a .dockerignore file at the overall project root, are used. + See "balena help ${Logger.command}" for more details.`); + } else { + msg.push(stripIndent` + By default, only one .dockerignore file at the source folder (project root) + is used. Microservices (multicontainer) applications may use a separate + .dockerignore file for each service with the --multi-dockerignore (-m) option. + See "balena help ${Logger.command}" for more details.`); + } + } + // No unused .dockerignore files. Print info-level advice in some cases. + else if (multiDockerignore) { + logFunc = logger.logInfo; + // multi-container app with a root .dockerignore file + if (serviceDirs.length && rootDockerignore) { + msg.push( + stripIndent` + The --multi-dockerignore option is being used, and a .dockerignore file was + found at the project source (root) directory. Note that this file will not + be used to filter service subdirectories. See "balena help ${Logger.command}".`, + ); + } + // single-container app + else if (serviceDirs.length === 0) { + msg.push( + stripIndent` + The --multi-dockerignore (-m) option was specified, but it has no effect for + single-container (non-microservices) apps. Only one .dockerignore file at the + project source (root) directory, if any, is used. See "balena help ${Logger.command}".`, + ); + } + } + if (msg.length) { + logFunc.call(logger, [' ', hr, ...msg, hr].join('\n')); } - const hr = - '-------------------------------------------------------------------------------'; - const msg = [ - ' ', - hr, - 'The following .dockerignore file(s) will not be used:', - ]; - msg.push(...nonRootFiles.map((fileStats) => `* ${fileStats.filePath}`)); - msg.push(stripIndent` - Only one .dockerignore file at the source folder (project root) is used. - Additional .dockerignore files are disregarded. Microservices (multicontainer) - apps should place the .dockerignore file alongside the docker-compose.yml file. - See issue: https://github.com/balena-io/balena-cli/issues/1870 - See also CLI v12 release notes: https://git.io/Jf7hz - `); - msg.push(hr); - Logger.getLogger().logWarn(msg.join('\n')); } /** @@ -270,8 +387,6 @@ export function printGitignoreWarn( if (ignoreFiles.length === 0) { return; } - const hr = - '-------------------------------------------------------------------------------'; const msg = [' ', hr, 'Using file ignore patterns from:']; msg.push(...ignoreFiles.map((e) => `* ${e}`)); if (gitignoreFiles.length) { diff --git a/lib/utils/device/deploy.ts b/lib/utils/device/deploy.ts index 8193d981..b883af5a 100644 --- a/lib/utils/device/deploy.ts +++ b/lib/utils/device/deploy.ts @@ -54,6 +54,7 @@ export interface DeviceDeployOptions { devicePort?: number; dockerfilePath?: string; registrySecrets: RegistrySecrets; + multiDockerignore: boolean; nocache: boolean; nogitignore: boolean; noParentCheck: boolean; @@ -180,6 +181,8 @@ export async function deployToDevice(opts: DeviceDeployOptions): Promise { const project = await loadProject(globalLogger, { convertEol: opts.convertEol, dockerfilePath: opts.dockerfilePath, + multiDockerignore: opts.multiDockerignore, + nogitignore: opts.nogitignore, noParentCheck: opts.noParentCheck, projectName: 'local', projectPath: opts.source, @@ -194,7 +197,9 @@ export async function deployToDevice(opts: DeviceDeployOptions): Promise { await checkBuildSecretsRequirements(docker, opts.source); globalLogger.logDebug('Tarring all non-ignored files...'); const tarStream = await tarDirectory(opts.source, { + composition: project.composition, convertEol: opts.convertEol, + multiDockerignore: opts.multiDockerignore, nogitignore: opts.nogitignore, }); @@ -407,7 +412,9 @@ export async function rebuildSingleTask( }; const tarStream = await tarDirectory(source, { + composition, convertEol: opts.convertEol, + multiDockerignore: opts.multiDockerignore, nogitignore: opts.nogitignore, }); diff --git a/lib/utils/ignore.ts b/lib/utils/ignore.ts index 4903c8cb..8a1c5883 100644 --- a/lib/utils/ignore.ts +++ b/lib/utils/ignore.ts @@ -249,17 +249,15 @@ async function readDockerIgnoreFile(projectDir: string): Promise { } /** - * Create a list of files (FileStats[]) for the filesystem subtree rooted at - * projectDir, filtered against a .dockerignore file (if any) also at projectDir, - * plus a few hardcoded dockerignore patterns. - * @param projectDir Source directory to + * Create an instance of '@balena/dockerignore', initialized with the contents + * of a .dockerignore file (if any) found at the given directory argument, plus + * a set of default/hardcoded patterns. + * @param directory Directory where to look for a .dockerignore file */ -export async function filterFilesWithDockerignore( - projectDir: string, -): Promise<{ filteredFileList: FileStats[]; dockerignoreFiles: FileStats[] }> { - // path.resolve() also converts forward slashes to backslashes on Windows - projectDir = path.resolve(projectDir); - const dockerIgnoreStr = await readDockerIgnoreFile(projectDir); +async function getDockerIgnoreInstance( + directory: string, +): Promise { + const dockerIgnoreStr = await readDockerIgnoreFile(directory); const $dockerIgnore = (await import('@balena/dockerignore')).default; const ig = $dockerIgnore({ ignorecase: false }); @@ -274,14 +272,60 @@ export async function filterFilesWithDockerignore( '!**/Dockerfile.*', '!**/docker-compose.yml', ]); + return ig; +} +export interface ServiceDirs { + [service: string]: string; +} + +/** + * Create a list of files (FileStats[]) for the filesystem subtree rooted at + * projectDir, filtered against the applicable .dockerignore files, including + * a few default/hardcoded dockerignore patterns. + * @param projectDir Source directory to + * @param serviceDirsByService Map of service names to their subdirectories. + * The service directory names/paths must be relative to the project root dir + * and be "normalized" (path.normalize()) before the call to this function: + * they should use backslashes on Windows, not contain '.' or '..' segments and + * not contain multiple consecutive path separators like '//'. Also, relative + * paths must not start with './' (e.g. 'a/b' instead of './a/b'). + */ +export async function filterFilesWithDockerignore( + projectDir: string, + serviceDirsByService?: ServiceDirs, +): Promise<{ filteredFileList: FileStats[]; dockerignoreFiles: FileStats[] }> { + // path.resolve() also converts forward slashes to backslashes on Windows + projectDir = path.resolve(projectDir); + // ignoreByDir stores an instance of the dockerignore filter for each service dir + const ignoreByDir: { + [serviceDir: string]: import('@balena/dockerignore').Ignore; + } = { + '.': await getDockerIgnoreInstance(projectDir), + }; + const serviceDirs: string[] = Object.values(serviceDirsByService || {}) + // filter out the project source/root dir + .filter((dir) => dir && dir !== '.') + // add a trailing '/' (or '\' on Windows) to the path + .map((dir) => (dir.endsWith(path.sep) ? dir : dir + path.sep)); + + for (const serviceDir of serviceDirs) { + ignoreByDir[serviceDir] = await getDockerIgnoreInstance( + path.join(projectDir, serviceDir), + ); + } const files = await listFiles(projectDir); const dockerignoreFiles: FileStats[] = []; const filteredFileList = files.filter((file: FileStats) => { if (path.basename(file.relPath) === '.dockerignore') { dockerignoreFiles.push(file); } - return !ig.ignores(file.relPath); + for (const dir of serviceDirs) { + if (file.relPath.startsWith(dir)) { + return !ignoreByDir[dir].ignores(file.relPath.substring(dir.length)); + } + } + return !ignoreByDir['.'].ignores(file.relPath); }); return { filteredFileList, dockerignoreFiles }; } diff --git a/lib/utils/messages.ts b/lib/utils/messages.ts index a198a2ec..49cbb501 100644 --- a/lib/utils/messages.ts +++ b/lib/utils/messages.ts @@ -45,11 +45,11 @@ export const balenaAsciiArt = `\ `; export const registrySecretsHelp = `\ -REGISTRY SECRETS +REGISTRY SECRETS The --registry-secrets option specifies a JSON or YAML file containing private Docker registry usernames and passwords to be used when pulling base images. Sample registry-secrets YAML file: - +\`\`\` 'my-registry-server.com:25000': username: ann password: hunter2 @@ -59,7 +59,7 @@ Sample registry-secrets YAML file: 'eu.gcr.io': # Google Container Registry username: '_json_key' password: '{escaped contents of the GCR keyfile.json file}' - +\`\`\` For a sample project using registry secrets with the Google Container Registry, check: https://github.com/balena-io-playground/sample-gcr-registry-secrets @@ -68,36 +68,53 @@ secrets.json file exists in the balena directory (usually $HOME/.balena), this file will be used instead.`; export const dockerignoreHelp = `\ -DOCKERIGNORE AND GITIGNORE FILES -The balena CLI will use a '.dockerignore' file (if any) at the source directory -in order to decide which source files to exclude from the "build context" sent -to balenaCloud, Docker or balenaEngine. In a microservices / multicontainer -application, the source directory is usually where the 'docker-compose.yml' -file is located, and therefore the '.dockerignore' file should be located -alongside the 'docker-compose.yml' file. Matching patterns may be prefixed with -the service's directory name (relative to the source directory) in order to -apply to that service only (e.g. 'service1/node_modules'). +DOCKERIGNORE AND GITIGNORE FILES +By default, the balena CLI will use a single ".dockerignore" file (if any) at +the project root (--source directory) in order to decide which source files to +exclude from the "build context" (tar stream) sent to balenaCloud, Docker daemon +or balenaEngine. In a microservices (multicontainer) application, the source +directory is the directory that contains the "docker-compose.yml" file. -Previous balena CLI releases (before v12.0.0) also took '.gitignore' files -into account. This behavior is deprecated, but may still be enabled with the ---gitignore (-g) option if compatibility is required. This option will be -removed in the CLI's next major version release (v13). +The --multi-dockerignore (-m) option may be used with microservices (multicontainer) +applications that define a docker-compose.yml file. When this option is used, +each service subdirectory (defined by the \`build\` or \`build.context\` service +properties in the docker-compose.yml file) is filtered separately according to +a .dockerignore file defined in the service subdirectory. If no .dockerignore +file exists in a service subdirectory, then only the default .dockerignore +patterns (see below) apply for that service subdirectory. -When --gitignore (-g) is NOT provided (i.e. when not in v11 compatibility mode), -a few "hardcoded" dockerignore patterns are also used and "merged" (in memory) -with the patterns found in the '.dockerignore' file (if any), in the following -order: +When the --multi-dockerignore (-m) option is used, the .dockerignore file (if +any) defined at the overall project root will be used to filter files and +subdirectories other than service subdirectories. It will not have any effect +on service subdirectories, whether or not a service subdirectory defines its +own .dockerignore file. Multiple .dockerignore files are not merged or added +together, and cannot override or extend other files. This behavior maximises +compatibility with the standard docker-compose tool, while still allowing a +root .dockerignore file (at the overall project root) to filter files and +folders that are outside service subdirectories. +Balena CLI releases older than v12.0.0 also took .gitignore files into account. +This behavior is deprecated, but may still be enabled with the --gitignore (-g) +option if compatibility is required. This option is mutually exclusive with +--multi-dockerignore (-m) and will be removed in the CLI's next major version +release (v13). + +Default .dockerignore patterns +When --gitignore (-g) is NOT used (i.e. when not in v11 compatibility mode), a +few default/hardcoded dockerignore patterns are "merged" (in memory) with the +patterns found in the applicable .dockerignore files, in the following order: +\`\`\` **/.git - < user's patterns from the '.dockerignore' file, if any > + < user's patterns from the applicable '.dockerignore' file, if any > !**/.balena !**/.resin !**/Dockerfile !**/Dockerfile.* !**/docker-compose.yml - -If necessary, the effect of the '**/.git' pattern may be modified by adding -"counter patterns" to the '.dockerignore' file, for example '!service1/.git'. -For documentation on pattern format, see: +\`\`\` +These patterns always apply, whether or not .dockerignore files exist in the +project. If necessary, the effect of the \`**/.git\` pattern may be modified by +adding counter patterns to the applicable .dockerignore file(s), for example +\`!mysubmodule/.git\`. For documentation on pattern format, see: - https://docs.docker.com/engine/reference/builder/#dockerignore-file - https://www.npmjs.com/package/@balena/dockerignore`; diff --git a/lib/utils/remote-build.ts b/lib/utils/remote-build.ts index 50f892f5..c4a55b54 100644 --- a/lib/utils/remote-build.ts +++ b/lib/utils/remote-build.ts @@ -43,6 +43,7 @@ export interface BuildOpts { registrySecrets: RegistrySecrets; headless: boolean; convertEol: boolean; + multiDockerignore: boolean; } export interface RemoteBuild { @@ -306,6 +307,7 @@ async function getTarStream(build: RemoteBuild): Promise { return await tarDirectory(path.resolve(build.source), { preFinalizeCallback: preFinalizeCb, convertEol: build.opts.convertEol, + multiDockerignore: build.opts.multiDockerignore, nogitignore: build.nogitignore, }); } finally { diff --git a/tests/commands/build.spec.ts b/tests/commands/build.spec.ts index 8f9c55d6..66b672de 100644 --- a/tests/commands/build.spec.ts +++ b/tests/commands/build.spec.ts @@ -61,6 +61,9 @@ const commonComposeQueryParams = [ ['labels', ''], ]; +const hr = + '----------------------------------------------------------------------'; + // "itSS" means "it() Skip Standalone" const itSS = process.env.BALENA_CLI_TEST_TYPE === 'standalone' ? it.skip : it; @@ -172,15 +175,14 @@ describe('balena build', function () { '[Info] Building for rpi/raspberry-pi', '[Info] Emulation is enabled', ...[ - '[Warn] -------------------------------------------------------------------------------', + `[Warn] ${hr}`, '[Warn] The following .dockerignore file(s) will not be used:', `[Warn] * ${path.join(projectPath, 'src', '.dockerignore')}`, - '[Warn] Only one .dockerignore file at the source folder (project root) is used.', - '[Warn] Additional .dockerignore files are disregarded. Microservices (multicontainer)', - '[Warn] apps should place the .dockerignore file alongside the docker-compose.yml file.', - '[Warn] See issue: https://github.com/balena-io/balena-cli/issues/1870', - '[Warn] See also CLI v12 release notes: https://git.io/Jf7hz', - '[Warn] -------------------------------------------------------------------------------', + '[Warn] By default, only one .dockerignore file at the source folder (project root)', + '[Warn] is used. Microservices (multicontainer) applications may use a separate', + '[Warn] .dockerignore file for each service with the --multi-dockerignore (-m) option.', + '[Warn] See "balena help build" for more details.', + `[Warn] ${hr}`, ], '[Build] main Step 1/4 : FROM busybox', '[Success] Build succeeded!', @@ -230,7 +232,7 @@ describe('balena build', function () { } }); - it('should create the expected tar stream (single container, --[no]convert-eol)', async () => { + it('should create the expected tar stream (single container, --[no]convert-eol, --multi-dockerignore)', async () => { const projectPath = path.join(projectsPath, 'no-docker-compose', 'basic'); const expectedFiles: ExpectedTarStreamFiles = { 'src/.dockerignore': { fileSize: 16, type: 'file' }, @@ -252,15 +254,14 @@ describe('balena build', function () { `[Info] No "docker-compose.yml" file found at "${projectPath}"`, `[Info] Creating default composition with source: "${projectPath}"`, ...[ - '[Warn] -------------------------------------------------------------------------------', + `[Warn] ${hr}`, '[Warn] The following .dockerignore file(s) will not be used:', `[Warn] * ${path.join(projectPath, 'src', '.dockerignore')}`, - '[Warn] Only one .dockerignore file at the source folder (project root) is used.', - '[Warn] Additional .dockerignore files are disregarded. Microservices (multicontainer)', - '[Warn] apps should place the .dockerignore file alongside the docker-compose.yml file.', - '[Warn] See issue: https://github.com/balena-io/balena-cli/issues/1870', - '[Warn] See also CLI v12 release notes: https://git.io/Jf7hz', - '[Warn] -------------------------------------------------------------------------------', + '[Warn] When --multi-dockerignore (-m) is used, only .dockerignore files at the root of', + "[Warn] each service's build context (in a microservices/multicontainer application),", + '[Warn] plus a .dockerignore file at the overall project root, are used.', + '[Warn] See "balena help build" for more details.', + `[Warn] ${hr}`, ], '[Build] main Step 1/4 : FROM busybox', ]; @@ -273,7 +274,7 @@ describe('balena build', function () { } docker.expectGetInfo({}); await testDockerBuildStream({ - commandLine: `build ${projectPath} --deviceType nuc --arch amd64 --noconvert-eol`, + commandLine: `build ${projectPath} --deviceType nuc --arch amd64 --noconvert-eol -m`, dockerMock: docker, expectedFilesByService: { main: expectedFiles }, expectedQueryParamsByService: { main: commonQueryParams }, @@ -304,7 +305,93 @@ describe('balena build', function () { 'file1.sh': { fileSize: 12, type: 'file' }, }, service2: { - '.dockerignore': { fileSize: 14, type: 'file' }, + '.dockerignore': { fileSize: 12, type: 'file' }, + 'Dockerfile-alt': { fileSize: 40, type: 'file' }, + 'file2-crlf.sh': { + fileSize: isWindows ? 12 : 14, + testStream: isWindows ? expectStreamNoCRLF : undefined, + type: 'file', + }, + 'src/file1.sh': { fileSize: 12, type: 'file' }, + }, + }; + const responseFilename = 'build-POST.json'; + const responseBody = await fs.readFile( + path.join(dockerResponsePath, responseFilename), + 'utf8', + ); + const expectedQueryParamsByService = { + service1: [ + ['t', '${tag}'], + [ + 'buildargs', + '{"MY_VAR_1":"This is a variable","MY_VAR_2":"Also a variable","SERVICE1_VAR":"This is a service specific variable"}', + ], + ['labels', ''], + ], + service2: [...commonComposeQueryParams, ['dockerfile', 'Dockerfile-alt']], + }; + const expectedResponseLines: string[] = [ + ...commonResponseLines[responseFilename], + ...[ + '[Build] service1 Step 1/4 : FROM busybox', + '[Build] service2 Step 1/4 : FROM busybox', + ], + ...[ + `[Warn] ${hr}`, + '[Warn] The following .dockerignore file(s) will not be used:', + `[Warn] * ${path.join(projectPath, 'service2', '.dockerignore')}`, + '[Warn] By default, only one .dockerignore file at the source folder (project root)', + '[Warn] is used. Microservices (multicontainer) applications may use a separate', + '[Warn] .dockerignore file for each service with the --multi-dockerignore (-m) option.', + '[Warn] See "balena help build" for more details.', + `[Warn] ${hr}`, + ], + ]; + if (isWindows) { + expectedResponseLines.push( + `[Info] Converting line endings CRLF -> LF for file: ${path.join( + projectPath, + 'service2', + 'file2-crlf.sh', + )}`, + ); + } + docker.expectGetInfo({}); + await testDockerBuildStream({ + commandLine: `build ${projectPath} --deviceType nuc --arch amd64 --convert-eol -G`, + dockerMock: docker, + expectedFilesByService, + expectedQueryParamsByService, + expectedResponseLines, + projectPath, + responseBody, + responseCode: 200, + services: ['service1', 'service2'], + }); + }); + + it('should create the expected tar stream (docker-compose, --multi-dockerignore)', async () => { + const projectPath = path.join(projectsPath, 'docker-compose', 'basic'); + const service1Dockerfile = ( + await fs.readFile( + path.join(projectPath, 'service1', 'Dockerfile.template'), + 'utf8', + ) + ).replace('%%BALENA_MACHINE_NAME%%', 'nuc'); + const expectedFilesByService: ExpectedTarStreamFilesByService = { + service1: { + Dockerfile: { + contents: service1Dockerfile, + fileSize: service1Dockerfile.length, + type: 'file', + }, + 'Dockerfile.template': { fileSize: 144, type: 'file' }, + 'file1.sh': { fileSize: 12, type: 'file' }, + 'test-ignore.txt': { fileSize: 12, type: 'file' }, + }, + service2: { + '.dockerignore': { fileSize: 12, type: 'file' }, 'Dockerfile-alt': { fileSize: 40, type: 'file' }, 'file2-crlf.sh': { fileSize: isWindows ? 12 : 14, @@ -336,14 +423,11 @@ describe('balena build', function () { '[Build] service2 Step 1/4 : FROM busybox', ], ...[ - '[Warn] The following .dockerignore file(s) will not be used:', - `[Warn] * ${path.join(projectPath, 'service2', '.dockerignore')}`, - '[Warn] Only one .dockerignore file at the source folder (project root) is used.', - '[Warn] Additional .dockerignore files are disregarded. Microservices (multicontainer)', - '[Warn] apps should place the .dockerignore file alongside the docker-compose.yml file.', - '[Warn] See issue: https://github.com/balena-io/balena-cli/issues/1870', - '[Warn] See also CLI v12 release notes: https://git.io/Jf7hz', - '[Warn] -------------------------------------------------------------------------------', + `[Info] ${hr}`, + '[Info] The --multi-dockerignore option is being used, and a .dockerignore file was', + '[Info] found at the project source (root) directory. Note that this file will not', + '[Info] be used to filter service subdirectories. See "balena help build".', + `[Info] ${hr}`, ], ]; if (isWindows) { @@ -357,7 +441,7 @@ describe('balena build', function () { } docker.expectGetInfo({}); await testDockerBuildStream({ - commandLine: `build ${projectPath} --deviceType nuc --arch amd64 --convert-eol -G`, + commandLine: `build ${projectPath} --deviceType nuc --arch amd64 --convert-eol -m`, dockerMock: docker, expectedFilesByService, expectedQueryParamsByService, diff --git a/tests/commands/deploy.spec.ts b/tests/commands/deploy.spec.ts index 28ea3274..15773eb4 100644 --- a/tests/commands/deploy.spec.ts +++ b/tests/commands/deploy.spec.ts @@ -27,7 +27,10 @@ import { BalenaAPIMock } from '../balena-api-mock'; import { expectStreamNoCRLF, testDockerBuildStream } from '../docker-build'; import { DockerMock, dockerResponsePath } from '../docker-mock'; import { cleanOutput, runCommand, switchSentry } from '../helpers'; -import { ExpectedTarStreamFiles } from '../projects'; +import { + ExpectedTarStreamFiles, + ExpectedTarStreamFilesByService, +} from '../projects'; const repoPath = path.normalize(path.join(__dirname, '..', '..')); const projectsPath = path.join(repoPath, 'tests', 'test-data', 'projects'); @@ -38,7 +41,7 @@ const commonResponseLines = { '[Info] Docker Desktop detected (daemon architecture: "x86_64")', '[Info] Docker itself will determine and enable architecture emulation if required,', '[Info] without balena-cli intervention and regardless of the --emulated option.', - '[Build] main Step 1/4 : FROM busybox', + // '[Build] main Step 1/4 : FROM busybox', '[Info] Creating release...', '[Info] Pushing images to registry...', '[Info] Saving release...', @@ -53,6 +56,18 @@ const commonQueryParams = [ ['labels', ''], ]; +const commonComposeQueryParams = [ + ['t', '${tag}'], + [ + 'buildargs', + '{"MY_VAR_1":"This is a variable","MY_VAR_2":"Also a variable"}', + ], + ['labels', ''], +]; + +const hr = + '----------------------------------------------------------------------'; + describe('balena deploy', function () { let api: BalenaAPIMock; let docker: DockerMock; @@ -73,8 +88,8 @@ describe('balena deploy', function () { api.expectGetAuth(); api.expectPostImage(); api.expectPostImageIsPartOfRelease(); - api.expectPostImageLabel(); + docker.expectGetImages(); docker.expectGetPing(); docker.expectGetInfo({}); docker.expectGetVersion({ persist: true }); @@ -112,15 +127,14 @@ describe('balena deploy', function () { `[Info] No "docker-compose.yml" file found at "${projectPath}"`, `[Info] Creating default composition with source: "${projectPath}"`, ...[ - '[Warn] -------------------------------------------------------------------------------', + `[Warn] ${hr}`, '[Warn] The following .dockerignore file(s) will not be used:', `[Warn] * ${path.join(projectPath, 'src', '.dockerignore')}`, - '[Warn] Only one .dockerignore file at the source folder (project root) is used.', - '[Warn] Additional .dockerignore files are disregarded. Microservices (multicontainer)', - '[Warn] apps should place the .dockerignore file alongside the docker-compose.yml file.', - '[Warn] See issue: https://github.com/balena-io/balena-cli/issues/1870', - '[Warn] See also CLI v12 release notes: https://git.io/Jf7hz', - '[Warn] -------------------------------------------------------------------------------', + '[Warn] By default, only one .dockerignore file at the source folder (project root)', + '[Warn] is used. Microservices (multicontainer) applications may use a separate', + '[Warn] .dockerignore file for each service with the --multi-dockerignore (-m) option.', + '[Warn] See "balena help deploy" for more details.', + `[Warn] ${hr}`, ], ]; if (isWindows) { @@ -132,6 +146,7 @@ describe('balena deploy', function () { api.expectPatchImage({}); api.expectPatchRelease({}); + api.expectPostImageLabel(); await testDockerBuildStream({ commandLine: `deploy testApp --build --source ${projectPath} -G`, @@ -189,6 +204,7 @@ describe('balena deploy', function () { expect(releaseBody.status).to.equal('failed'); }, }); + api.expectPostImageLabel(); try { sentryStatus = await switchSentry(false); @@ -213,6 +229,98 @@ describe('balena deploy', function () { process.exit.restore(); } }); + + it('should create the expected tar stream (docker-compose, --multi-dockerignore)', async () => { + const projectPath = path.join(projectsPath, 'docker-compose', 'basic'); + const service1Dockerfile = ( + await fs.readFile( + path.join(projectPath, 'service1', 'Dockerfile.template'), + 'utf8', + ) + ).replace('%%BALENA_MACHINE_NAME%%', 'raspberrypi3'); + + console.error( + `Dockerfile.template (replaced) length=${service1Dockerfile.length}`, + ); + console.error(service1Dockerfile); + + const expectedFilesByService: ExpectedTarStreamFilesByService = { + service1: { + Dockerfile: { + contents: service1Dockerfile, + fileSize: service1Dockerfile.length, + type: 'file', + }, + 'Dockerfile.template': { fileSize: 144, type: 'file' }, + 'file1.sh': { fileSize: 12, type: 'file' }, + 'test-ignore.txt': { fileSize: 12, type: 'file' }, + }, + service2: { + '.dockerignore': { fileSize: 12, type: 'file' }, + 'Dockerfile-alt': { fileSize: 40, type: 'file' }, + 'file2-crlf.sh': { + fileSize: isWindows ? 12 : 14, + testStream: isWindows ? expectStreamNoCRLF : undefined, + type: 'file', + }, + }, + }; + const responseFilename = 'build-POST.json'; + const responseBody = await fs.readFile( + path.join(dockerResponsePath, responseFilename), + 'utf8', + ); + const expectedQueryParamsByService = { + service1: [ + ['t', '${tag}'], + [ + 'buildargs', + '{"MY_VAR_1":"This is a variable","MY_VAR_2":"Also a variable","SERVICE1_VAR":"This is a service specific variable"}', + ], + ['labels', ''], + ], + service2: [...commonComposeQueryParams, ['dockerfile', 'Dockerfile-alt']], + }; + const expectedResponseLines: string[] = [ + ...commonResponseLines[responseFilename], + ...[ + '[Build] service1 Step 1/4 : FROM busybox', + '[Build] service2 Step 1/4 : FROM busybox', + ], + ...[ + `[Info] ${hr}`, + '[Info] The --multi-dockerignore option is being used, and a .dockerignore file was', + '[Info] found at the project source (root) directory. Note that this file will not', + '[Info] be used to filter service subdirectories. See "balena help deploy".', + `[Info] ${hr}`, + ], + ]; + if (isWindows) { + expectedResponseLines.push( + `[Info] Converting line endings CRLF -> LF for file: ${path.join( + projectPath, + 'service2', + 'file2-crlf.sh', + )}`, + ); + } + + // docker.expectGetImages(); + api.expectPatchImage({}); + api.expectPatchRelease({}); + + await testDockerBuildStream({ + commandLine: `deploy testApp --build --source ${projectPath} --multi-dockerignore`, + dockerMock: docker, + expectedFilesByService, + expectedQueryParamsByService, + expectedResponseLines, + projectPath, + responseBody, + responseCode: 200, + services: ['service1', 'service2'], + }); + }); }); describe('balena deploy: project validation', function () { diff --git a/tests/commands/push.spec.ts b/tests/commands/push.spec.ts index 04611097..a730806d 100644 --- a/tests/commands/push.spec.ts +++ b/tests/commands/push.spec.ts @@ -77,6 +77,9 @@ const commonQueryParams = [ ['headless', 'false'], ]; +const hr = + '----------------------------------------------------------------------'; + describe('balena push', function () { let api: BalenaAPIMock; let builder: BuilderMock; @@ -126,14 +129,14 @@ describe('balena push', function () { const expectedResponseLines = [ ...commonResponseLines[responseFilename], ...[ + `[Warn] ${hr}`, '[Warn] The following .dockerignore file(s) will not be used:', `[Warn] * ${path.join(projectPath, 'src', '.dockerignore')}`, - '[Warn] Only one .dockerignore file at the source folder (project root) is used.', - '[Warn] Additional .dockerignore files are disregarded. Microservices (multicontainer)', - '[Warn] apps should place the .dockerignore file alongside the docker-compose.yml file.', - '[Warn] See issue: https://github.com/balena-io/balena-cli/issues/1870', - '[Warn] See also CLI v12 release notes: https://git.io/Jf7hz', - '[Warn] -------------------------------------------------------------------------------', + '[Warn] By default, only one .dockerignore file at the source folder (project root)', + '[Warn] is used. Microservices (multicontainer) applications may use a separate', + '[Warn] .dockerignore file for each service with the --multi-dockerignore (-m) option.', + '[Warn] See "balena help push" for more details.', + `[Warn] ${hr}`, ], ]; if (isWindows) { @@ -173,14 +176,14 @@ describe('balena push', function () { const expectedResponseLines = [ ...commonResponseLines[responseFilename], ...[ + `[Warn] ${hr}`, '[Warn] The following .dockerignore file(s) will not be used:', `[Warn] * ${path.join(projectPath, 'src', '.dockerignore')}`, - '[Warn] Only one .dockerignore file at the source folder (project root) is used.', - '[Warn] Additional .dockerignore files are disregarded. Microservices (multicontainer)', - '[Warn] apps should place the .dockerignore file alongside the docker-compose.yml file.', - '[Warn] See issue: https://github.com/balena-io/balena-cli/issues/1870', - '[Warn] See also CLI v12 release notes: https://git.io/Jf7hz', - '[Warn] -------------------------------------------------------------------------------', + '[Warn] By default, only one .dockerignore file at the source folder (project root)', + '[Warn] is used. Microservices (multicontainer) applications may use a separate', + '[Warn] .dockerignore file for each service with the --multi-dockerignore (-m) option.', + '[Warn] See "balena help push" for more details.', + `[Warn] ${hr}`, ], ]; const expectedQueryParams = commonQueryParams.map((i) => @@ -220,14 +223,14 @@ describe('balena push', function () { const expectedResponseLines = [ ...commonResponseLines[responseFilename], ...[ + `[Warn] ${hr}`, '[Warn] The following .dockerignore file(s) will not be used:', `[Warn] * ${path.join(projectPath, 'src', '.dockerignore')}`, - '[Warn] Only one .dockerignore file at the source folder (project root) is used.', - '[Warn] Additional .dockerignore files are disregarded. Microservices (multicontainer)', - '[Warn] apps should place the .dockerignore file alongside the docker-compose.yml file.', - '[Warn] See issue: https://github.com/balena-io/balena-cli/issues/1870', - '[Warn] See also CLI v12 release notes: https://git.io/Jf7hz', - '[Warn] -------------------------------------------------------------------------------', + '[Warn] By default, only one .dockerignore file at the source folder (project root)', + '[Warn] is used. Microservices (multicontainer) applications may use a separate', + '[Warn] .dockerignore file for each service with the --multi-dockerignore (-m) option.', + '[Warn] See "balena help push" for more details.', + `[Warn] ${hr}`, ], ]; if (isWindows) { @@ -291,6 +294,7 @@ describe('balena push', function () { ); const expectedResponseLines = [ ...[ + `[Warn] ${hr}`, '[Warn] Using file ignore patterns from:', `[Warn] * ${path.join(projectPath, '.dockerignore')}`, `[Warn] * ${path.join(projectPath, '.gitignore')}`, @@ -298,6 +302,7 @@ describe('balena push', function () { '[Warn] .gitignore files are being considered because the --gitignore option was used.', '[Warn] This option is deprecated and will be removed in the next major version release.', "[Warn] For more information, see 'balena help push'.", + `[Warn] ${hr}`, ], ...commonResponseLines[responseFilename], ]; @@ -364,45 +369,19 @@ describe('balena push', function () { 'dockerignore2', ); const expectedFiles: ExpectedTarStreamFiles = { - '.dockerignore': { fileSize: 34, type: 'file' }, + '.dockerignore': { fileSize: 33, type: 'file' }, 'b.txt': { fileSize: 1, type: 'file' }, Dockerfile: { fileSize: 13, type: 'file' }, + 'lib/.dockerignore': { fileSize: 10, type: 'file' }, + 'lib/src-b.txt': { fileSize: 5, type: 'file' }, 'src/src-b.txt': { fileSize: 5, type: 'file' }, 'symlink-a.txt': { fileSize: 5, type: 'file' }, - ...(isWindows ? { 'src/src-a.txt': { fileSize: 5, type: 'file' } } : {}), - }; - const regSecretsPath = await addRegSecretsEntries(expectedFiles); - const responseFilename = 'build-POST-v3.json'; - const responseBody = await fs.readFile( - path.join(builderResponsePath, responseFilename), - 'utf8', - ); - - await testPushBuildStream({ - builderMock: builder, - commandLine: `push testApp -s ${projectPath} -R ${regSecretsPath} -l --gitignore`, - expectedFiles, - expectedQueryParams: commonQueryParams, - expectedResponseLines: commonResponseLines[responseFilename], - projectPath, - responseBody, - responseCode: 200, - }); - }); - - it('should create the expected tar stream (single container, dockerignore warn)', async () => { - const projectPath = path.join( - projectsPath, - 'no-docker-compose', - 'dockerignore2', - ); - const expectedFiles: ExpectedTarStreamFiles = { - '.dockerignore': { fileSize: 34, type: 'file' }, - 'b.txt': { fileSize: 1, type: 'file' }, - Dockerfile: { fileSize: 13, type: 'file' }, - 'src/src-b.txt': { fileSize: 5, type: 'file' }, - 'symlink-a.txt': { fileSize: 5, type: 'file' }, - ...(isWindows ? { 'src/src-a.txt': { fileSize: 5, type: 'file' } } : {}), + ...(isWindows + ? { + 'lib/src-a.txt': { fileSize: 5, type: 'file' }, + 'src/src-a.txt': { fileSize: 5, type: 'file' }, + } + : {}), }; const regSecretsPath = await addRegSecretsEntries(expectedFiles); const responseFilename = 'build-POST-v3.json'; @@ -412,6 +391,7 @@ describe('balena push', function () { ); const expectedResponseLines = isWindows ? [ + `[Warn] ${hr}`, '[Warn] Using file ignore patterns from:', `[Warn] * ${path.join(projectPath, '.dockerignore')}`, '[Warn] The --gitignore option was used, but no .gitignore files were found.', @@ -419,13 +399,61 @@ describe('balena push', function () { '[Warn] version release. It prevents the use of a better dockerignore parser and', '[Warn] filter library that fixes several issues on Windows and improves compatibility', "[Warn] with 'docker build'. For more information, see 'balena help push'.", + `[Warn] ${hr}`, ...commonResponseLines[responseFilename], ] : commonResponseLines[responseFilename]; await testPushBuildStream({ builderMock: builder, - commandLine: `push testApp -s ${projectPath} -R ${regSecretsPath} -l -g`, + commandLine: `push testApp -s ${projectPath} -R ${regSecretsPath} -l --gitignore`, + expectedFiles, + expectedQueryParams: commonQueryParams, + expectedResponseLines, + projectPath, + responseBody, + responseCode: 200, + }); + }); + + it('should create the expected tar stream (single container, --multi-dockerignore)', async () => { + const projectPath = path.join( + projectsPath, + 'no-docker-compose', + 'dockerignore2', + ); + const expectedFiles: ExpectedTarStreamFiles = { + '.dockerignore': { fileSize: 33, type: 'file' }, + 'b.txt': { fileSize: 1, type: 'file' }, + Dockerfile: { fileSize: 13, type: 'file' }, + 'lib/.dockerignore': { fileSize: 10, type: 'file' }, + 'lib/src-b.txt': { fileSize: 5, type: 'file' }, + 'src/src-b.txt': { fileSize: 5, type: 'file' }, + 'symlink-a.txt': { fileSize: 5, type: 'file' }, + }; + const regSecretsPath = await addRegSecretsEntries(expectedFiles); + const responseFilename = 'build-POST-v3.json'; + const responseBody = await fs.readFile( + path.join(builderResponsePath, responseFilename), + 'utf8', + ); + const expectedResponseLines: string[] = [ + ...[ + `[Warn] ${hr}`, + '[Warn] The following .dockerignore file(s) will not be used:', + `[Warn] * ${path.join(projectPath, 'lib', '.dockerignore')}`, + '[Warn] When --multi-dockerignore (-m) is used, only .dockerignore files at the root of', + "[Warn] each service's build context (in a microservices/multicontainer application),", + '[Warn] plus a .dockerignore file at the overall project root, are used.', + '[Warn] See "balena help push" for more details.', + `[Warn] ${hr}`, + ], + ...commonResponseLines[responseFilename], + ]; + + await testPushBuildStream({ + builderMock: builder, + commandLine: `push testApp -s ${projectPath} -R ${regSecretsPath} -l -m`, expectedFiles, expectedQueryParams: commonQueryParams, expectedResponseLines, @@ -444,12 +472,13 @@ describe('balena push', function () { 'service1/Dockerfile.template': { fileSize: 144, type: 'file' }, 'service1/file1.sh': { fileSize: 12, type: 'file' }, 'service2/Dockerfile-alt': { fileSize: 40, type: 'file' }, - 'service2/.dockerignore': { fileSize: 14, type: 'file' }, + 'service2/.dockerignore': { fileSize: 12, type: 'file' }, 'service2/file2-crlf.sh': { fileSize: isWindows ? 12 : 14, testStream: isWindows ? expectStreamNoCRLF : undefined, type: 'file', }, + 'service2/src/file1.sh': { fileSize: 12, type: 'file' }, }; const regSecretsPath = await addRegSecretsEntries(expectedFiles); const responseFilename = 'build-POST-v3.json'; @@ -460,14 +489,14 @@ describe('balena push', function () { const expectedResponseLines: string[] = [ ...commonResponseLines[responseFilename], ...[ + `[Warn] ${hr}`, '[Warn] The following .dockerignore file(s) will not be used:', `[Warn] * ${path.join(projectPath, 'service2', '.dockerignore')}`, - '[Warn] Only one .dockerignore file at the source folder (project root) is used.', - '[Warn] Additional .dockerignore files are disregarded. Microservices (multicontainer)', - '[Warn] apps should place the .dockerignore file alongside the docker-compose.yml file.', - '[Warn] See issue: https://github.com/balena-io/balena-cli/issues/1870', - '[Warn] See also CLI v12 release notes: https://git.io/Jf7hz', - '[Warn] -------------------------------------------------------------------------------', + '[Warn] By default, only one .dockerignore file at the source folder (project root)', + '[Warn] is used. Microservices (multicontainer) applications may use a separate', + '[Warn] .dockerignore file for each service with the --multi-dockerignore (-m) option.', + '[Warn] See "balena help push" for more details.', + `[Warn] ${hr}`, ], ]; if (isWindows) { @@ -491,6 +520,61 @@ describe('balena push', function () { responseCode: 200, }); }); + + it('should create the expected tar stream (docker-compose, --multi-dockerignore)', async () => { + const projectPath = path.join(projectsPath, 'docker-compose', 'basic'); + const expectedFiles: ExpectedTarStreamFiles = { + '.balena/balena.yml': { fileSize: 197, type: 'file' }, + '.dockerignore': { fileSize: 22, type: 'file' }, + 'docker-compose.yml': { fileSize: 245, type: 'file' }, + 'service1/Dockerfile.template': { fileSize: 144, type: 'file' }, + 'service1/file1.sh': { fileSize: 12, type: 'file' }, + 'service1/test-ignore.txt': { fileSize: 12, type: 'file' }, + 'service2/Dockerfile-alt': { fileSize: 40, type: 'file' }, + 'service2/.dockerignore': { fileSize: 12, type: 'file' }, + 'service2/file2-crlf.sh': { + fileSize: isWindows ? 12 : 14, + testStream: isWindows ? expectStreamNoCRLF : undefined, + type: 'file', + }, + }; + const regSecretsPath = await addRegSecretsEntries(expectedFiles); + const responseFilename = 'build-POST-v3.json'; + const responseBody = await fs.readFile( + path.join(builderResponsePath, responseFilename), + 'utf8', + ); + const expectedResponseLines: string[] = [ + ...commonResponseLines[responseFilename], + ...[ + `[Info] ${hr}`, + '[Info] The --multi-dockerignore option is being used, and a .dockerignore file was', + '[Info] found at the project source (root) directory. Note that this file will not', + '[Info] be used to filter service subdirectories. See "balena help push".', + `[Info] ${hr}`, + ], + ]; + if (isWindows) { + expectedResponseLines.push( + `[Info] Converting line endings CRLF -> LF for file: ${path.join( + projectPath, + 'service2', + 'file2-crlf.sh', + )}`, + ); + } + + await testPushBuildStream({ + builderMock: builder, + commandLine: `push testApp -s ${projectPath} -R ${regSecretsPath} -l -m`, + expectedFiles, + expectedQueryParams: commonQueryParams, + expectedResponseLines, + projectPath, + responseBody, + responseCode: 200, + }); + }); }); describe('balena push: project validation', function () { diff --git a/tests/docker-build.ts b/tests/docker-build.ts index 5ca534ca..99fda0e1 100644 --- a/tests/docker-build.ts +++ b/tests/docker-build.ts @@ -95,7 +95,11 @@ export async function inspectTarStream( expect($expected).to.deep.equal(found); } catch (e) { const { diff } = require('deep-object-diff'); - const diffStr = JSON.stringify(diff($expected, found), null, 4); + const diffStr = JSON.stringify( + diff($expected, found), + (_k, v) => (v === undefined ? 'undefined' : v), + 4, + ); console.error(`\nexpected vs. found diff:\n${diffStr}\n`); throw e; } @@ -181,7 +185,9 @@ export async function testDockerBuildStream(o: { inspectTarStream(buildRequestBody, expectedFiles, projectPath), tag, }); - o.dockerMock.expectGetImages(); + if (o.commandLine.startsWith('build')) { + o.dockerMock.expectGetImages(); + } } const { exitCode, out, err } = await runCommand(o.commandLine); diff --git a/tests/test-data/projects/docker-compose/basic/service2/.dockerignore b/tests/test-data/projects/docker-compose/basic/service2/.dockerignore index fe46fc1e..08012d02 100644 --- a/tests/test-data/projects/docker-compose/basic/service2/.dockerignore +++ b/tests/test-data/projects/docker-compose/basic/service2/.dockerignore @@ -1 +1 @@ -file2-crlf.sh +**/file1.sh diff --git a/tests/test-data/projects/docker-compose/basic/service2/src/file1.sh b/tests/test-data/projects/docker-compose/basic/service2/src/file1.sh new file mode 100644 index 00000000..c0d0fb45 --- /dev/null +++ b/tests/test-data/projects/docker-compose/basic/service2/src/file1.sh @@ -0,0 +1,2 @@ +line1 +line2 diff --git a/tests/test-data/projects/no-docker-compose/dockerignore2/.dockerignore b/tests/test-data/projects/no-docker-compose/dockerignore2/.dockerignore index dc701109..9b948498 100644 --- a/tests/test-data/projects/no-docker-compose/dockerignore2/.dockerignore +++ b/tests/test-data/projects/no-docker-compose/dockerignore2/.dockerignore @@ -1,3 +1,3 @@ a.txt -src/src-a.txt +**/src-a.txt symlink-b.txt diff --git a/tests/test-data/projects/no-docker-compose/dockerignore2/lib/.dockerignore b/tests/test-data/projects/no-docker-compose/dockerignore2/lib/.dockerignore new file mode 100644 index 00000000..7cb33e74 --- /dev/null +++ b/tests/test-data/projects/no-docker-compose/dockerignore2/lib/.dockerignore @@ -0,0 +1 @@ +src-b.txt diff --git a/tests/test-data/projects/no-docker-compose/dockerignore2/lib/src-a.txt b/tests/test-data/projects/no-docker-compose/dockerignore2/lib/src-a.txt new file mode 100644 index 00000000..a0286011 --- /dev/null +++ b/tests/test-data/projects/no-docker-compose/dockerignore2/lib/src-a.txt @@ -0,0 +1 @@ +lib-a \ No newline at end of file diff --git a/tests/test-data/projects/no-docker-compose/dockerignore2/lib/src-b.txt b/tests/test-data/projects/no-docker-compose/dockerignore2/lib/src-b.txt new file mode 100644 index 00000000..84dfcf75 --- /dev/null +++ b/tests/test-data/projects/no-docker-compose/dockerignore2/lib/src-b.txt @@ -0,0 +1 @@ +lib-b \ No newline at end of file diff --git a/tests/utils/tarDirectory.spec.ts b/tests/utils/tarDirectory.spec.ts index d40ac8c8..2775cf69 100644 --- a/tests/utils/tarDirectory.spec.ts +++ b/tests/utils/tarDirectory.spec.ts @@ -99,9 +99,11 @@ describe('compare new and old tarDirectory implementations', function () { 'dockerignore2', ); const expectedFiles = { - '.dockerignore': { fileSize: 34, type: 'file' }, + '.dockerignore': { fileSize: 33, type: 'file' }, 'b.txt': { fileSize: 1, type: 'file' }, Dockerfile: { fileSize: 13, type: 'file' }, + 'lib/.dockerignore': { fileSize: 10, type: 'file' }, + 'lib/src-b.txt': { fileSize: 5, type: 'file' }, 'src/src-b.txt': { fileSize: 5, type: 'file' }, 'symlink-a.txt': { fileSize: 5, type: 'file' }, };