ExternalVendorCode/balena-cli
1
0
Fork 0
mirror of https://github.com/balena-io/balena-cli.git synced 2025-01-18 18:56:25 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge pull request #1869 from balena-io/gitignore-option

Browse Source 
Add --gitignore (-g) option to push/build/deploy commands for v11 compatibility
This commit is contained in:
bulldozer-balena[bot] 2020-06-17 14:57:57 +00:00 committed by GitHub
commit 599b897721
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
12 changed files with 216 additions and 173 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

78
doc/cli.markdown
View File

@ -1867,12 +1867,20 @@ this file will be used instead.
DOCKERIGNORE AND GITIGNORE FILES DOCKERIGNORE AND GITIGNORE FILES
The balena CLI will use a '.dockerignore' file (if any) at the source directory The balena CLI will use a '.dockerignore' file (if any) at the source directory
in order to decide which source files to exclude from the "build context" sent in order to decide which source files to exclude from the "build context" sent
to balenaCloud, Docker or balenaEngine. Previous balena CLI releases (before to balenaCloud, Docker or balenaEngine. In a microservices / multicontainer
v12.0.0) also took '.gitignore' files into account, but this is no longer the application, the source directory is usually where the 'docker-compose.yml'
case. This allows files to be used for an image build even if they are listed file is located, and therefore the '.dockerignore' file should be located
in '.gitignore'. alongside the 'docker-compose.yml' file. Matching patterns may be prefixed with
the service's directory name (relative to the source directory) in order to
apply to that service only (e.g. 'service1/node_modules').
A few "hardcoded" dockerignore patterns are also used and "merged" (in memory) Previous balena CLI releases (before v12.0.0) also took '.gitignore' files
into account. This behavior is deprecated, but may still be enabled with the
--gitignore (-g) option if compatibility is required. This option will be
removed in the CLI's next major version release (v13).
When --gitignore (-g) is NOT provided (i.e. when not in v11 compatibility mode),
a few "hardcoded" dockerignore patterns are also used and "merged" (in memory)
with the patterns found in the '.dockerignore' file (if any), in the following with the patterns found in the '.dockerignore' file (if any), in the following
order: order:
@ -1978,7 +1986,13 @@ Don't convert line endings from CRLF (Windows format) to LF (Unix format).
#### --nogitignore, -G #### --nogitignore, -G
No-op and deprecated since balena CLI v12.0.0. See "balena help push". No-op (default behavior) since balena CLI v12.0.0. See "balena help push".
#### --gitignore, -g
Consider .gitignore files in addition to the .dockerignore file. This reverts
to the CLI v11 behavior/implementation (deprecated) if compatibility is required
until your project can be adapted.
# Settings # Settings
@ -2067,12 +2081,20 @@ this file will be used instead.
DOCKERIGNORE AND GITIGNORE FILES DOCKERIGNORE AND GITIGNORE FILES
The balena CLI will use a '.dockerignore' file (if any) at the source directory The balena CLI will use a '.dockerignore' file (if any) at the source directory
in order to decide which source files to exclude from the "build context" sent in order to decide which source files to exclude from the "build context" sent
to balenaCloud, Docker or balenaEngine. Previous balena CLI releases (before to balenaCloud, Docker or balenaEngine. In a microservices / multicontainer
v12.0.0) also took '.gitignore' files into account, but this is no longer the application, the source directory is usually where the 'docker-compose.yml'
case. This allows files to be used for an image build even if they are listed file is located, and therefore the '.dockerignore' file should be located
in '.gitignore'. alongside the 'docker-compose.yml' file. Matching patterns may be prefixed with
the service's directory name (relative to the source directory) in order to
apply to that service only (e.g. 'service1/node_modules').
A few "hardcoded" dockerignore patterns are also used and "merged" (in memory) Previous balena CLI releases (before v12.0.0) also took '.gitignore' files
into account. This behavior is deprecated, but may still be enabled with the
--gitignore (-g) option if compatibility is required. This option will be
removed in the CLI's next major version release (v13).
When --gitignore (-g) is NOT provided (i.e. when not in v11 compatibility mode),
a few "hardcoded" dockerignore patterns are also used and "merged" (in memory)
with the patterns found in the '.dockerignore' file (if any), in the following with the patterns found in the '.dockerignore' file (if any), in the following
order: order:
@ -2134,9 +2156,15 @@ No-op and deprecated since balena CLI v12.0.0. Build logs are now shown by defau
Hide the image build log output (produce less verbose output) Hide the image build log output (produce less verbose output)
#### --gitignore, -g
Consider .gitignore files in addition to the .dockerignore file. This reverts
to the CLI v11 behavior/implementation (deprecated) if compatibility is required
until your project can be adapted.
#### --nogitignore, -G #### --nogitignore, -G
No-op and deprecated since balena CLI v12.0.0. See "balena help undefined". No-op (default behavior) since balena CLI v12.0.0. See "balena help build".
#### --noparent-check #### --noparent-check
@ -2247,12 +2275,20 @@ this file will be used instead.
DOCKERIGNORE AND GITIGNORE FILES DOCKERIGNORE AND GITIGNORE FILES
The balena CLI will use a '.dockerignore' file (if any) at the source directory The balena CLI will use a '.dockerignore' file (if any) at the source directory
in order to decide which source files to exclude from the "build context" sent in order to decide which source files to exclude from the "build context" sent
to balenaCloud, Docker or balenaEngine. Previous balena CLI releases (before to balenaCloud, Docker or balenaEngine. In a microservices / multicontainer
v12.0.0) also took '.gitignore' files into account, but this is no longer the application, the source directory is usually where the 'docker-compose.yml'
case. This allows files to be used for an image build even if they are listed file is located, and therefore the '.dockerignore' file should be located
in '.gitignore'. alongside the 'docker-compose.yml' file. Matching patterns may be prefixed with
the service's directory name (relative to the source directory) in order to
apply to that service only (e.g. 'service1/node_modules').
A few "hardcoded" dockerignore patterns are also used and "merged" (in memory) Previous balena CLI releases (before v12.0.0) also took '.gitignore' files
into account. This behavior is deprecated, but may still be enabled with the
--gitignore (-g) option if compatibility is required. This option will be
removed in the CLI's next major version release (v13).
When --gitignore (-g) is NOT provided (i.e. when not in v11 compatibility mode),
a few "hardcoded" dockerignore patterns are also used and "merged" (in memory)
with the patterns found in the '.dockerignore' file (if any), in the following with the patterns found in the '.dockerignore' file (if any), in the following
order: order:
@ -2310,9 +2346,15 @@ No-op and deprecated since balena CLI v12.0.0. Build logs are now shown by defau
Hide the image build log output (produce less verbose output) Hide the image build log output (produce less verbose output)
#### --gitignore, -g
Consider .gitignore files in addition to the .dockerignore file. This reverts
to the CLI v11 behavior/implementation (deprecated) if compatibility is required
until your project can be adapted.
#### --nogitignore, -G #### --nogitignore, -G
No-op and deprecated since balena CLI v12.0.0. See "balena help undefined". No-op (default behavior) since balena CLI v12.0.0. See "balena help build".
#### --noparent-check #### --noparent-check

21
lib/actions/push.ts
View File

@ -110,6 +110,7 @@ export const push: CommandDefinition<
nocache?: boolean; nocache?: boolean;
'noparent-check'?: boolean; 'noparent-check'?: boolean;
'registry-secrets'?: string; 'registry-secrets'?: string;
gitignore?: boolean;
nogitignore?: boolean; nogitignore?: boolean;
nolive?: boolean; nolive?: boolean;
detached?: boolean; detached?: boolean;
@ -279,13 +280,17 @@ export const push: CommandDefinition<
{ {
signature: 'nogitignore', signature: 'nogitignore',
alias: 'G', alias: 'G',
description: isV12() description:
? 'No-op and deprecated since balena CLI v12.0.0. See "balena help push".' 'No-op (default behavior) since balena CLI v12.0.0. See "balena help push".',
: stripIndent` boolean: true,
Disregard all .gitignore files, and consider only the .dockerignore file (if any) },
at the source directory. This will be the default behavior in an upcoming major {
version release. For more information, see 'balena help push'. signature: 'gitignore',
`, alias: 'g',
description: stripIndent`
Consider .gitignore files in addition to the .dockerignore file. This reverts
to the CLI v11 behavior/implementation (deprecated) if compatibility is required
until your project can be adapted.`,
boolean: true, boolean: true,
}, },
], ],
@ -319,7 +324,7 @@ export const push: CommandDefinition<
}, },
); );
const nogitignore = !!options.nogitignore || isV12(); const nogitignore = !options.gitignore;
const convertEol = isV12() const convertEol = isV12()
? !options['noconvert-eol'] ? !options['noconvert-eol']
: !!options['convert-eol']; : !!options['convert-eol'];

23
lib/utils/compose.js
View File

@ -33,7 +33,6 @@ export const appendProjectOptions = (opts) =>
]); ]);
export function appendOptions(opts) { export function appendOptions(opts) {
const Logger = require('./logger');
const { isV12 } = require('./version'); const { isV12 } = require('./version');
return appendProjectOptions(opts).concat([ return appendProjectOptions(opts).concat([
{ {
@ -65,15 +64,18 @@ export function appendOptions(opts) {
}, },
] ]
: []), : []),
{
signature: 'gitignore',
alias: 'g',
description: stripIndent`
Consider .gitignore files in addition to the .dockerignore file. This reverts
to the CLI v11 behavior/implementation (deprecated) if compatibility is required
until your project can be adapted.`,
boolean: true,
},
{ {
signature: 'nogitignore', signature: 'nogitignore',
description: isV12() description: `No-op (default behavior) since balena CLI v12.0.0. See "balena help build".`,
? `No-op and deprecated since balena CLI v12.0.0. See "balena help ${Logger.command}".`
: stripIndent`
Disregard all .gitignore files, and consider only the .dockerignore file (if any)
at the source directory. This will be the default behavior in an upcoming major
version release. For more information, see 'balena help ${Logger.command}'.
`,
boolean: true, boolean: true,
alias: 'G', alias: 'G',
}, },
@ -125,7 +127,7 @@ export function generateOpts(options) {
inlineLogs: !options.nologs && (!!options.logs || isV12()), inlineLogs: !options.nologs && (!!options.logs || isV12()),
convertEol: isV12() ? !options['noconvert-eol'] : !!options['convert-eol'], convertEol: isV12() ? !options['noconvert-eol'] : !!options['convert-eol'],
dockerfilePath: options.dockerfile, dockerfilePath: options.dockerfile,
nogitignore: !!options.nogitignore || isV12(), nogitignore: !options.gitignore,
noParentCheck: options['noparent-check'], noParentCheck: options['noparent-check'],
})); }));
} }
@ -192,6 +194,9 @@ export function tarDirectory(dir, param) {
} }
/** /**
* This is the CLI v10 / v11 "original" tarDirectory function. It is still
* around for the benefit of the `--gitignore` option, but is expected to be
* deleted in CLI v13.
* @param {string} dir Source directory * @param {string} dir Source directory
* @param {import('./compose-types').TarDirectoryOptions} param * @param {import('./compose-types').TarDirectoryOptions} param
* @returns {Promise<import('stream').Readable>} * @returns {Promise<import('stream').Readable>}

20
lib/utils/compose_ts.ts
View File

@ -214,7 +214,7 @@ export async function tarDirectory(
/** /**
* Print a deprecation warning if any '.gitignore' or '.dockerignore' file is * Print a deprecation warning if any '.gitignore' or '.dockerignore' file is
* found and the --nogitignore (-G) option has not been provided. * found and the --gitignore (-g) option has been provided.
* @param dockerignoreFile Absolute path to a .dockerignore file * @param dockerignoreFile Absolute path to a .dockerignore file
* @param gitignoreFiles Array of absolute paths to .gitginore files * @param gitignoreFiles Array of absolute paths to .gitginore files
*/ */
@ -232,21 +232,19 @@ export function printGitignoreWarn(
msg.push(...ignoreFiles); msg.push(...ignoreFiles);
if (gitignoreFiles.length) { if (gitignoreFiles.length) {
msg.push(stripIndent` msg.push(stripIndent`
balena CLI currently uses gitgnore and dockerignore files, but an upcoming major Note: .gitignore files are being considered because the --gitignore option was
version release will disregard gitignore files and use a dockerignore file only. used. This option is deprecated and will be removed in the next major version
Use the --nogitignore (-G) option to enable the new behavior already now and release. For more information, see 'balena help ${Logger.command}'.
suppress this warning. For more information, see 'balena help ${Logger.command}'.
`); `);
msg.push(hr); msg.push(hr);
Logger.getLogger().logWarn(msg.join('\n')); Logger.getLogger().logWarn(msg.join('\n'));
} else if (dockerignoreFile && process.platform === 'win32') { } else if (dockerignoreFile && process.platform === 'win32') {
msg.push(stripIndent` msg.push(stripIndent`
Use the --nogitignore (-G) option to suppress this warning and enable the use The --gitignore option was used, but not .gitignore files were found.
of a better dockerignore parser and filter library that fixes several issues The --gitignore option is deprecated and will be removed in the next major
on Windows and improves compatibility with "docker build", but which may also version release. It prevents the use of a better dockerignore parser and
cause a different set of files to be filtered out (because of the bug fixes). filter library that fixes several issues on Windows and improves compatibility
The --nogitignore option will be the default behavior in an upcoming balena CLI with "docker build". For more information, see 'balena help ${Logger.command}'.
major version release. For more information, see 'balena help ${Logger.command}'.
`); `);
msg.push(hr); msg.push(hr);
Logger.getLogger().logWarn(msg.join('\n')); Logger.getLogger().logWarn(msg.join('\n'));

5
lib/utils/ignore.ts
View File

@ -37,6 +37,11 @@ interface IgnoreEntry {
filePath: string; filePath: string;
} }
/**
* This class is used by the CLI v10 / v11 "original" tarDirectory function
* in `compose.js`. It is still around for the benefit of the `--gitignore`
* option, but is expected to be deleted in CLI v13.
*/
export class FileIgnorer { export class FileIgnorer {
private dockerIgnoreEntries: IgnoreEntry[]; private dockerIgnoreEntries: IgnoreEntry[];
private gitIgnoreEntries: IgnoreEntry[]; private gitIgnoreEntries: IgnoreEntry[];

55
lib/utils/messages.ts
View File

@ -15,8 +15,6 @@
* limitations under the License. * limitations under the License.
*/ */
import { isV12 } from './version';
const DEBUG_MODE = !!process.env.DEBUG; const DEBUG_MODE = !!process.env.DEBUG;
export const reachingOut = `\ export const reachingOut = `\
@ -69,27 +67,26 @@ If the --registry-secrets option is not specified, and a secrets.yml or
secrets.json file exists in the balena directory (usually $HOME/.balena), secrets.json file exists in the balena directory (usually $HOME/.balena),
this file will be used instead.`; this file will be used instead.`;
const dockerignoreHelpV11 = `\ export const dockerignoreHelp = `\
DOCKERIGNORE AND GITIGNORE FILES DOCKERIGNORE AND GITIGNORE FILES
By default, both '.dockerignore' and '.gitignore' files are taken into account The balena CLI will use a '.dockerignore' file (if any) at the source directory
in order to prevent files from being sent to the balenaCloud builder or Docker in order to decide which source files to exclude from the "build context" sent
or balenaEngine (balenaOS device). to balenaCloud, Docker or balenaEngine. In a microservices / multicontainer
application, the source directory is usually where the 'docker-compose.yml'
file is located, and therefore the '.dockerignore' file should be located
alongside the 'docker-compose.yml' file. Matching patterns may be prefixed with
the service's directory name (relative to the source directory) in order to
apply to that service only (e.g. 'service1/node_modules').
However, this behavior has been DEPRECATED and will change in an upcoming major Previous balena CLI releases (before v12.0.0) also took '.gitignore' files
version release. The --nogitignore (-G) option should be used to enable the new into account. This behavior is deprecated, but may still be enabled with the
behavior already now. This option will cause the CLI to: --gitignore (-g) option if compatibility is required. This option will be
removed in the CLI's next major version release (v13).
* Disregard all '.gitignore' files at the source directory and subdirectories, When --gitignore (-g) is NOT provided (i.e. when not in v11 compatibility mode),
and consider only the '.dockerignore' file (if any) at the source directory. a few "hardcoded" dockerignore patterns are also used and "merged" (in memory)
* Consequently, allow files to be sent to balenaCloud / Docker / balenaEngine with the patterns found in the '.dockerignore' file (if any), in the following
even if they are listed in '.gitignore' files (a longstanding feature request). order:
* Use a new '.dockerignore' parser and filter library that improves compatibility
with "docker build" and fixes several issues (mainly on Windows).
* Prevent a warning message from being printed.
When --nogitignore (-G) is provided, a few "hardcoded" dockerignore patterns are
also used and "merged" (in memory) with the patterns found in the '.dockerignore'
file (if any), in the following order:
**/.git **/.git
< user's patterns from the '.dockerignore' file, if any > < user's patterns from the '.dockerignore' file, if any >
@ -104,21 +101,3 @@ If necessary, the effect of the '**/.git' pattern may be modified by adding
For documentation on pattern format, see: For documentation on pattern format, see:
- https://docs.docker.com/engine/reference/builder/#dockerignore-file - https://docs.docker.com/engine/reference/builder/#dockerignore-file
- https://www.npmjs.com/package/@balena/dockerignore`; - https://www.npmjs.com/package/@balena/dockerignore`;
const dockerignoreHelpV12 =
`DOCKERIGNORE AND GITIGNORE FILES
The balena CLI will use a '.dockerignore' file (if any) at the source directory
in order to decide which source files to exclude from the "build context" sent
to balenaCloud, Docker or balenaEngine. Previous balena CLI releases (before
v12.0.0) also took '.gitignore' files into account, but this is no longer the
case. This allows files to be used for an image build even if they are listed
in '.gitignore'.
A few "hardcoded" dockerignore patterns are also used and "merged" (in memory)
with the patterns found in the '.dockerignore' file (if any), in the following
order:
` + dockerignoreHelpV11.substring(dockerignoreHelpV11.indexOf('\n **/.git'));
export const dockerignoreHelp = isV12()
? dockerignoreHelpV12
: dockerignoreHelpV11;

6
npm-shrinkwrap.json generated
View File

@ -4448,6 +4448,12 @@
"integrity": "sha1-s2nW+128E+7PUk+RsHD+7cNXzzQ=", "integrity": "sha1-s2nW+128E+7PUk+RsHD+7cNXzzQ=",
"dev": true "dev": true
}, },
"deep-object-diff": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/deep-object-diff/-/deep-object-diff-1.1.0.tgz",
"integrity": "sha512-b+QLs5vHgS+IoSNcUE4n9HP2NwcHj7aqnJWsjPtuG75Rh5TOaGt0OjAYInh77d5T16V5cRDC+Pw/6ZZZiETBGw==",
"dev": true
},
"default-compare": { "default-compare": {
"version": "1.0.0", "version": "1.0.0",
"resolved": "https://registry.npmjs.org/default-compare/-/default-compare-1.0.0.tgz", "resolved": "https://registry.npmjs.org/default-compare/-/default-compare-1.0.0.tgz",

1
package.json
View File

@ -153,6 +153,7 @@
"chai": "^4.2.0", "chai": "^4.2.0",
"chai-as-promised": "^7.1.1", "chai-as-promised": "^7.1.1",
"cross-env": "^7.0.2", "cross-env": "^7.0.2",
"deep-object-diff": "^1.1.0",
"ent": "^2.2.0", "ent": "^2.2.0",
"filehound": "^1.17.4", "filehound": "^1.17.4",
"fs-extra": "^8.0.1", "fs-extra": "^8.0.1",

4
tests/commands/build.spec.ts
View File

@ -126,7 +126,7 @@ describe('balena build', function () {
} }
docker.expectGetInfo({}); docker.expectGetInfo({});
await testDockerBuildStream({ await testDockerBuildStream({
commandLine: `build ${projectPath} --deviceType nuc --arch amd64 -G`, commandLine: `build ${projectPath} --deviceType nuc --arch amd64 -g`,
dockerMock: docker, dockerMock: docker,
expectedFilesByService: { main: expectedFiles }, expectedFilesByService: { main: expectedFiles },
expectedQueryParamsByService: { main: commonQueryParams }, expectedQueryParamsByService: { main: commonQueryParams },
@ -138,6 +138,8 @@ describe('balena build', function () {
}); });
}); });
// Skip Standalone because we patch the source code with `mock-require` to avoid
// downloading and installing QEMU
itSS('should create the expected tar stream (--emulated)', async () => { itSS('should create the expected tar stream (--emulated)', async () => {
const projectPath = path.join(projectsPath, 'no-docker-compose', 'basic'); const projectPath = path.join(projectsPath, 'no-docker-compose', 'basic');
const isV12W = isWindows && isV12(); const isV12W = isWindows && isV12();

162
tests/commands/push.spec.ts
View File

@ -78,8 +78,6 @@ const commonQueryParams = [
['headless', 'false'], ['headless', 'false'],
]; ];
const itSkipWindows = process.platform === 'win32' ? it.skip : it;
describe('balena push', function () { describe('balena push', function () {
let api: BalenaAPIMock; let api: BalenaAPIMock;
let builder: BuilderMock; let builder: BuilderMock;
@ -173,7 +171,7 @@ describe('balena push', function () {
await testPushBuildStream({ await testPushBuildStream({
builderMock: builder, builderMock: builder,
commandLine: `push testApp -s ${projectPath} -R ${regSecretsPath} --dockerfile Dockerfile-alt --noconvert-eol --nogitignore`, commandLine: `push testApp -s ${projectPath} -R ${regSecretsPath} --dockerfile Dockerfile-alt --noconvert-eol`,
expectedFiles, expectedFiles,
expectedQueryParams, expectedQueryParams,
expectedResponseLines: commonResponseLines[responseFilename], expectedResponseLines: commonResponseLines[responseFilename],
@ -231,72 +229,71 @@ describe('balena push', function () {
}); });
}); });
// Skip Windows because the old tarDirectory() implementation (still used when it('should create the expected tar stream (single container, --gitignore)', async () => {
// '--nogitignore' is not provided) uses the old `zeit/dockerignore` npm package const projectPath = path.join(
// that is broken on Windows (reason why we created `@balena/dockerignore`). projectsPath,
itSkipWindows( 'no-docker-compose',
'should create the expected tar stream (single container, with gitignore)', 'dockerignore1',
async () => { );
const projectPath = path.join( const expectedFiles: ExpectedTarStreamFiles = {
projectsPath, '.balena/balena.yml': { fileSize: 12, type: 'file' },
'no-docker-compose', '.dockerignore': { fileSize: 438, type: 'file' },
'dockerignore1', '.gitignore': { fileSize: 20, type: 'file' },
); '.git/bar.txt': { fileSize: 4, type: 'file' },
const expectedFiles: ExpectedTarStreamFiles = { '.git/foo.txt': { fileSize: 4, type: 'file' },
'.balena/balena.yml': { fileSize: 12, type: 'file' }, 'c.txt': { fileSize: 1, type: 'file' },
'.dockerignore': { fileSize: 438, type: 'file' }, Dockerfile: { fileSize: 13, type: 'file' },
'.gitignore': { fileSize: 20, type: 'file' }, 'src/.balena/balena.yml': { fileSize: 16, type: 'file' },
'.git/foo.txt': { fileSize: 4, type: 'file' }, 'src/.gitignore': { fileSize: 10, type: 'file' },
'c.txt': { fileSize: 1, type: 'file' }, 'vendor/.git/vendor-git-contents': { fileSize: 20, type: 'file' },
Dockerfile: { fileSize: 13, type: 'file' }, // When --gitignore (-g) is provided for v11 compatibility, the old
'src/.balena/balena.yml': { fileSize: 16, type: 'file' }, // `zeit/dockerignore` npm package is still used but it is broken on
'src/.gitignore': { fileSize: 10, type: 'file' }, // Windows (reason why we created `@balena/dockerignore`).
'vendor/.git/vendor-git-contents': { fileSize: 20, type: 'file' }, ...(isWindows
...(isV12() ? {
? { 'src/src-b.txt': { fileSize: 5, type: 'file' },
'a.txt': { fileSize: 1, type: 'file' }, 'dot.git/bar.txt': { fileSize: 4, type: 'file' },
'src/src-a.txt': { fileSize: 5, type: 'file' }, 'dot.git/foo.txt': { fileSize: 4, type: 'file' },
'src/src-c.txt': { fileSize: 5, type: 'file' }, 'vendor/dot.git/vendor-git-contents': {
} fileSize: 20,
: { type: 'file',
'.git/bar.txt': { fileSize: 4, type: 'file' }, },
}), }
}; : {}),
};
const regSecretsPath = await addRegSecretsEntries(expectedFiles); const regSecretsPath = await addRegSecretsEntries(expectedFiles);
const responseFilename = 'build-POST-v3.json'; const responseFilename = 'build-POST-v3.json';
const responseBody = await fs.readFile( const responseBody = await fs.readFile(
path.join(builderResponsePath, responseFilename), path.join(builderResponsePath, responseFilename),
'utf8', 'utf8',
); );
const expectedResponseLines = [ const expectedResponseLines = [
...(!isV12() ...(isV12()
? [ ? [
'[Warn] Using file ignore patterns from:', '[Warn] Using file ignore patterns from:',
`[Warn] ${path.join(projectPath, '.dockerignore')}`, `[Warn] ${path.join(projectPath, '.dockerignore')}`,
`[Warn] ${path.join(projectPath, '.gitignore')}`, `[Warn] ${path.join(projectPath, '.gitignore')}`,
`[Warn] ${path.join(projectPath, 'src', '.gitignore')}`, `[Warn] ${path.join(projectPath, 'src', '.gitignore')}`,
'[Warn] balena CLI currently uses gitgnore and dockerignore files, but an upcoming major', '[Warn] Note: .gitignore files are being considered because the --gitignore option was',
'[Warn] version release will disregard gitignore files and use a dockerignore file only.', '[Warn] used. This option is deprecated and will be removed in the next major version',
'[Warn] Use the --nogitignore (-G) option to enable the new behavior already now and', "[Warn] release. For more information, see 'balena help push'.",
"[Warn] suppress this warning. For more information, see 'balena help push'.", ]
] : []),
: []), ...commonResponseLines[responseFilename],
...commonResponseLines[responseFilename], ];
];
await testPushBuildStream({ await testPushBuildStream({
builderMock: builder, builderMock: builder,
commandLine: `push testApp -s ${projectPath} -R ${regSecretsPath} -l`, commandLine: `push testApp -s ${projectPath} -R ${regSecretsPath} -l -g`,
expectedFiles, expectedFiles,
expectedQueryParams: commonQueryParams, expectedQueryParams: commonQueryParams,
expectedResponseLines, expectedResponseLines,
projectPath, projectPath,
responseBody, responseBody,
responseCode: 200, responseCode: 200,
}); });
}, });
);
it('should create the expected tar stream (single container, --nogitignore)', async () => { it('should create the expected tar stream (single container, --nogitignore)', async () => {
const projectPath = path.join( const projectPath = path.join(
@ -341,7 +338,7 @@ describe('balena push', function () {
// (with a mismatched fileSize 13 vs 5 for 'symlink-a.txt'), ensure that the // (with a mismatched fileSize 13 vs 5 for 'symlink-a.txt'), ensure that the
// `core.symlinks` property is set to `true` in the `.git/config` file. Ref: // `core.symlinks` property is set to `true` in the `.git/config` file. Ref:
// https://git-scm.com/docs/git-config#Documentation/git-config.txt-coresymlinks // https://git-scm.com/docs/git-config#Documentation/git-config.txt-coresymlinks
it('should create the expected tar stream (single container, symbolic links, --nogitignore)', async () => { it('should create the expected tar stream (single container, symbolic links, --gitignore)', async () => {
const projectPath = path.join( const projectPath = path.join(
projectsPath, projectsPath,
'no-docker-compose', 'no-docker-compose',
@ -353,6 +350,7 @@ describe('balena push', function () {
Dockerfile: { fileSize: 13, type: 'file' }, Dockerfile: { fileSize: 13, type: 'file' },
'src/src-b.txt': { fileSize: 5, type: 'file' }, 'src/src-b.txt': { fileSize: 5, type: 'file' },
'symlink-a.txt': { fileSize: 5, type: 'file' }, 'symlink-a.txt': { fileSize: 5, type: 'file' },
...(isWindows ? { 'src/src-a.txt': { fileSize: 5, type: 'file' } } : {}),
}; };
const regSecretsPath = await addRegSecretsEntries(expectedFiles); const regSecretsPath = await addRegSecretsEntries(expectedFiles);
const responseFilename = 'build-POST-v3.json'; const responseFilename = 'build-POST-v3.json';
@ -363,7 +361,7 @@ describe('balena push', function () {
await testPushBuildStream({ await testPushBuildStream({
builderMock: builder, builderMock: builder,
commandLine: `push testApp -s ${projectPath} -R ${regSecretsPath} -l -G`, commandLine: `push testApp -s ${projectPath} -R ${regSecretsPath} -l --gitignore`,
expectedFiles, expectedFiles,
expectedQueryParams: commonQueryParams, expectedQueryParams: commonQueryParams,
expectedResponseLines: commonResponseLines[responseFilename], expectedResponseLines: commonResponseLines[responseFilename],
@ -385,36 +383,30 @@ describe('balena push', function () {
Dockerfile: { fileSize: 13, type: 'file' }, Dockerfile: { fileSize: 13, type: 'file' },
'src/src-b.txt': { fileSize: 5, type: 'file' }, 'src/src-b.txt': { fileSize: 5, type: 'file' },
'symlink-a.txt': { fileSize: 5, type: 'file' }, 'symlink-a.txt': { fileSize: 5, type: 'file' },
...(isWindows ? { 'src/src-a.txt': { fileSize: 5, type: 'file' } } : {}),
}; };
const noV12W = isWindows && !isV12();
if (noV12W) {
// this test uses the old tarDirectory implementation, which uses
// the zeit/dockerignore library that has bugs on Windows
expectedFiles['src/src-a.txt'] = { fileSize: 5, type: 'file' };
}
const regSecretsPath = await addRegSecretsEntries(expectedFiles); const regSecretsPath = await addRegSecretsEntries(expectedFiles);
const responseFilename = 'build-POST-v3.json'; const responseFilename = 'build-POST-v3.json';
const responseBody = await fs.readFile( const responseBody = await fs.readFile(
path.join(builderResponsePath, responseFilename), path.join(builderResponsePath, responseFilename),
'utf8', 'utf8',
); );
const expectedResponseLines = noV12W const expectedResponseLines = isWindows
? [ ? [
'[Warn] Using file ignore patterns from:', '[Warn] Using file ignore patterns from:',
`[Warn] ${path.join(projectPath, '.dockerignore')}`, `[Warn] ${path.join(projectPath, '.dockerignore')}`,
'[Warn] Use the --nogitignore (-G) option to suppress this warning and enable the use', '[Warn] The --gitignore option was used, but not .gitignore files were found.',
'[Warn] of a better dockerignore parser and filter library that fixes several issues', '[Warn] The --gitignore option is deprecated and will be removed in the next major',
'[Warn] on Windows and improves compatibility with "docker build", but which may also', '[Warn] version release. It prevents the use of a better dockerignore parser and',
'[Warn] cause a different set of files to be filtered out (because of the bug fixes).', '[Warn] filter library that fixes several issues on Windows and improves compatibility',
'[Warn] The --nogitignore option will be the default behavior in an upcoming balena CLI', '[Warn] with "docker build". For more information, see \'balena help push\'.',
"[Warn] major version release. For more information, see 'balena help push'.",
...commonResponseLines[responseFilename], ...commonResponseLines[responseFilename],
] ]
: commonResponseLines[responseFilename]; : commonResponseLines[responseFilename];
await testPushBuildStream({ await testPushBuildStream({
builderMock: builder, builderMock: builder,
commandLine: `push testApp -s ${projectPath} -R ${regSecretsPath} -l`, commandLine: `push testApp -s ${projectPath} -R ${regSecretsPath} -l -g`,
expectedFiles, expectedFiles,
expectedQueryParams: commonQueryParams, expectedQueryParams: commonQueryParams,
expectedResponseLines, expectedResponseLines,
@ -460,7 +452,7 @@ describe('balena push', function () {
await testPushBuildStream({ await testPushBuildStream({
builderMock: builder, builderMock: builder,
commandLine: `push testApp -s ${projectPath} -R ${regSecretsPath} -l -G`, commandLine: `push testApp -s ${projectPath} -R ${regSecretsPath} -l`,
expectedFiles, expectedFiles,
expectedQueryParams: commonQueryParams, expectedQueryParams: commonQueryParams,
expectedResponseLines, expectedResponseLines,
@ -484,7 +476,7 @@ describe('balena push: project validation', function () {
]; ];
const { out, err } = await runCommand( const { out, err } = await runCommand(
`push testApp --source ${projectPath} --nogitignore`, `push testApp --source ${projectPath} --gitignore`,
); );
expect(cleanOutput(err, true)).to.include.members(expectedErrorLines); expect(cleanOutput(err, true)).to.include.members(expectedErrorLines);
expect(out).to.be.empty; expect(out).to.be.empty;

12
tests/docker-build.ts
View File

@ -88,9 +88,17 @@ export async function inspectTarStream(
sourceTarStream.pipe(extract); sourceTarStream.pipe(extract);
}); });
expect(found).to.deep.equal( const $expected = _.mapValues(expectedFiles, (v) =>
_.mapValues(expectedFiles, (v) => _.omit(v, 'testStream', 'contents')), _.omit(v, 'testStream', 'contents'),
); );
try {
expect($expected).to.deep.equal(found);
} catch (e) {
const { diff } = require('deep-object-diff');
const diffStr = JSON.stringify(diff($expected, found), null, 4);
console.error(`\nexpected vs. found diff:\n${diffStr}\n`);
throw e;
}
} }
/** Check that a tar stream entry matches the project contents in the filesystem */ /** Check that a tar stream entry matches the project contents in the filesystem */

2
tests/utils/tarDirectory.spec.ts
View File

@ -113,7 +113,7 @@ describe('compare new and old tarDirectory implementations', function () {
}); });
// Skip Windows because the old tarDirectory() implementation (still used when // Skip Windows because the old tarDirectory() implementation (still used when
// '--nogitignore' is not provided) uses the old `zeit/dockerignore` npm package // '--gitignore' is provided) uses the old `zeit/dockerignore` npm package
// that is broken on Windows (reason why we created `@balena/dockerignore`). // that is broken on Windows (reason why we created `@balena/dockerignore`).
itSkipWindows('should produce a compatible tar stream', async function () { itSkipWindows('should produce a compatible tar stream', async function () {
const dockerignoreProjDir = path.join( const dockerignoreProjDir = path.join(