* The log history of a test is stored as a whole for the lifetime of the test.
* Matching of the log history against log patterns is done correctly now
(previously, a pattern like "AAB" on an input like "AAAB" wouldn't have
triggered).
* Use memcmp, memcpy, memmove instead of the former character-wise operations.
* Sanitizing of log input and log patterns now works more generic through the
new Filter class for all replacements/removals.
* Sanitizing is done as soon as a string is available and remains for the
lifetime of the test.
* Sanitizing doesn't interfer with the matching algorithm.
* Decomposing into small clearly named functions.
Ref #3555
There are still nightly tests like test-tcp_bulk_lxip on sel4 x86_64 qemu
that manage to hit the test timeout of the run script although the test was
successful. So, raise the extra time added by the run script to 30 seconds.
Ref #3411
Some platforms (sel4 imx6/imx7) cannot manage to execute all tests in a single
boot. Thus, we re-boot them periodically after a given maximum number of tests
to avoid that arbitrary tests always fail due to the long uptime and not due to
the tests themselves.
If the maximum number of tests is set to 0, no limit is applied.
Fixes#3411
It might happen, in CPU intensive tests (like TCP bulk lxip), or when
printing debugging output after a failed test (as done currently on
staging), that the run script on the host gives up and reboots the
platform too early. Thus, we raise the buffer time. A reboot should
be necessary only in rare cases anyway.
Fixes#3387
The fs_tool component performs file operations according to its
configuration. This initial version implements only the operation
<remove-file> as needed for Sculpt CE.
Issue #3222
Issue #3193
The minimal-footprint Ada runtime for implementing library-like
functionality in SPARK is now called "spark" runtime.
The full Ada runtime for entire components written in Ada and using the
libc as glue to the underlying system will move to the world repository
as "ada" runtime.
Issue #3144
The 'run_genode_until' procedure only called 'run_power_on' to reset
the target machine. That works will with the softreset module, which
is used by all x86-based test system but falls short regarding ARM
boards. The way those boards are connected requires turning the power
off and on for a complete cycle.
In less interactive mode, the run script doesn't give up on missing test
archives but instead removes the corresponding tests and marks them "missing".
This mode avoids total failure of a platform in automated test infrastructures
when only a few archives are missing.
Fixes#3120
After a certain number of tests, presumably some resource in core is exceeded
and loading the successive test fails. This quickfix looks out for the
characteristic Core error and then reboots to avoid that all successive tests
are marked as failed.
Adds an config attribute to the Depot Autopilot component:
:<config repeat>:
Can be one of
"false" - process the given test list only once,
"until_forever" - endlessly repeat processing the given test list,
"until_failed" - repeat processing the given test list until it fails.
Adds an environment variable to the Depot Autopilot Run script:
:TEST_REPEAT:
Same as the <config repeat> attribute of the Depot Autopilot.
This is useful when having to debug very sporadic errors during one test
or a series of tests.
This ensures that the depot_autopilot.run script, when exiting, always prints
a result overview of the so far available test results, except the Depot
Autopilot component has managed to print the result overview up to this point.
Unequal numbers of double quotes let the XML parser of init fail,
therefore replace all double quotes when transfering previous
results after a reboot.
Ref #3027
The number of tests to run is the number of test package-archives minus the
the those that are skipped for the given platform. The number is printed
directly after checking if the given platform is supported by the run script.
It helps the surrounding test infrastructure to ensure that, for instance, a
result graph always reflects the same total number of tests, even though there
is a sporadic problem with booting the platform.
The test for python requires x86 to be built. Therefore, there is no test
binary available when trying to execute that test on ARM with depot_autopilot.
Print a line like "succeeded: 35 failed: 11 skipped: 2" below the list of test
results. Adds further attributes to <previous-results> to communicate also the
previous statistics.
