If runTests() terminated with a FATAL condition, the cause was not
always apparent, because the error message usually got overwritten by
the test progress output on the user's terminal. To fix this, the main
loop's standard error is now collected in a temporary file which is sent
to standard error just before exit.
The test framework was not always handling its internal FATAL error code
(255) correctly, so this has been fixed.
One cause of a FATAL termination was if a test's unique temporary
directory, which was based on its Process ID, already existed. Anything
that left a temporary directory behind increased the likelihood of a
FATAL in a subsequent test run. (For example, one keyring test case was
not killing its Serval DNA daemon, and the daemon was re-creating its
instance directory before eventually terminating itself. This test case
has been fixed.)
The test framework now allocates its temporary directory by re-trying
different random numbers until mkdir(1) succeeds, and only fataling
after 20 failures.
Use the conventional autoconf @srcdir@ substitution instead of requiring
the SOURCE_PREFIX variable to be passed to the Makefile.
Also support running test scripts in an out-of-tree build directory.
The execute() and executeOk() functions now set $TFWEXECUTED instead
of $executed.
The execute() and executeOk() functions now make a hard link from the
--stdout-file=PATH and --stderr-file=PATH files to the default files
of 'stdout' and 'stderr' so that post-fork handling can still cat their
contents into the log file.
The fork() function now sets the variables $TFWFORKSTDOUT_<label>
$TFWFORKSTDERR_<label> $TFWFORKPID_<label>.
Use GNU grep, sed and awk instead of BSD variants. Developers will have
to install these using a package manager like homebrew. Updated the
INSTALL.md and doc/Development.md tech docs with instructions.
So that if execute() is used within a forked process and if that process
is terminated prematurely, the log will show what the execute'd command
was doing
Always pass exactly one argument to sleep(1) since the BSD sleep (used
on OSX) does not sum its arguments like GNU sleep(1)
Perform more stringent checking on various timeout parameters to ensure
they are well-formed decimal floats
Remove the subshell parentheses around the command invoked using fork(),
so that the forked command inherits the file descriptors. This allows
it to be used like "fork some command < input".
When in a the finalise part of a test (possibly as a result of a
failure or error), make sure that shell backtraces do not list the
test case functions that failed, since they have already been
reported in a backtrace at the point of failure
Replace forkKillAll with fork_terminate_all
Replace forkWaitAll with fork_wait_all
Add fork_terminate and fork_wait functions
Introduce fork labels '%<alphanumeric>' as arguments to all fork
functions
Treat a forked process exit status 143 (killed with SIGTERM) as a
failure or error -- it happens when a forked process is terminated
using fork_terminate or fork_terminate_all
Add -h|--hexdump option to tfw_cat() and fix faulty logic in tfw_cat()
-v option wof files not ending in \n (newline)
Add TFWSTDOUT and TFWSTDERR env vars set by execute() with absolute
path names of files containing stdout and stderr respectively (as an
alternative to using replayStdout() and replayStderr()).
Default timeout is 60 seconds, can be overridden by --timeout=N on command-line
or by TFW_EXECUTE_TIMEOUT var or by --timeout option to execute() primitive
Cull timeout sleep processes created by wait_until() primitive
The 'includeTests' function failed if /bin/bash was not available or not
the same as the Bash used to run the top test script. This failed on
Solaris, where /usr/bin/bash is too old, and a privately-built Bash must
be used to run the test scripts.
New tfw_quieten() is similar to tfw_nolog(), but only silences the chatty log
output for asserts and other actions that succeed. Failures are still logged.
New tfw_multicolumn() encapsulates an awk(1) script to format a list into
columns that fit into $COLUMNS width.
Improve exit status handling for test ERROR and FAIL conditions.
The non-deterministic bug of "job %N terminated without result" is now fixed.
The cause was that Bash job numbers are allocated as max(current job numbers) +
1, so if the most recent job exited was not harvested before the next job was
started, then the job number would get re-used, deleting the un-harvested job's
result file.
SIGINT (ctrl-C interrupt) handling has been improved for the -j (parallel) and
non-parallel cases.
Test case execution in the non-parallel case is faster, by fixing a bug that
waited for a one-second timeout before starting each test job.
Now that tests can be included from other scripts, there is a possibility of
test name collision, so the main loop logic now uses the test sequence number
instead of the name internally to identify the test.
