Commit Graph

94 Commits

Author SHA1 Message Date
Martin Stein
fe5ada85cb depot_autopilot.run: max nr of tests per boot
Some platforms (sel4 imx6/imx7) cannot manage to execute all tests in a single
boot. Thus, we re-boot them periodically after a given maximum number of tests
to avoid that arbitrary tests always fail due to the long uptime and not due to
the tests themselves.

If the maximum number of tests is set to 0, no limit is applied.

Fixes #3411
2019-06-13 12:22:49 +02:00
Christian Helmuth
1b059649ff depot_autopilot: skip test-spark_exception on ARM
Issue #3395
2019-06-13 12:22:49 +02:00
Martin Stein
a09150e6b4 depot_autopilot.run: relax reboot timeout
It might happen, in CPU intensive tests (like TCP bulk lxip), or when
printing debugging output after a failed test (as done currently on
staging), that the run script on the host gives up and reboots the
platform too early. Thus, we raise the buffer time. A reboot should
be necessary only in rare cases anyway.

Fixes #3387
2019-05-29 10:20:52 +02:00
Martin Stein
da3c52668d depot_autopilot.run: fix insufficient Qemu RAM
Fixes an error output in init at the beginning of the test and insufficient
transfer of RAM to the runtime of RAM-intensive test packages.

Ref #3387
2019-05-29 10:20:52 +02:00
Emery Hemingway
5efa6d5273 Stack smashing test
A test to check if -fstack-protector can be enabled and is effective.

Ref #3066
2019-04-01 19:33:46 +02:00
Norman Feske
cac3f3b24e gems: initial version of fs_tool
The fs_tool component performs file operations according to its
configuration. This initial version implements only the operation
<remove-file> as needed for Sculpt CE.

Issue #3222
Issue #3193
2019-03-18 15:56:58 +01:00
Norman Feske
c25fbc010d depot_query: support for querying index files
Issue #3172
2019-02-28 11:34:06 +01:00
Alexander Senier
5ccae43552 Rename Ada runtime to SPARK runtime
The minimal-footprint Ada runtime for implementing library-like
functionality in SPARK is now called "spark" runtime.

The full Ada runtime for entire components written in Ada and using the
libc as glue to the underlying system will move to the world repository
as "ada" runtime.

Issue #3144
2019-02-19 11:08:17 +01:00
Josef Söntgen
1c0541e7cb run: add power cycle procedure
The 'run_genode_until' procedure only called 'run_power_on' to reset
the target machine. That works will with the softreset module, which
is used by all x86-based test system but falls short regarding ARM
boards. The way those boards are connected requires turning the power
off and on for a complete cycle.
2019-02-12 10:33:15 +01:00
Emery Hemingway
f529871162 Add [depot_user] to run tool builtins
Now that the depot_user procedure has found use in the depot_autopilot
it has become a convention.

Fix #3127
2019-01-30 13:49:55 +01:00
Martin Stein
ffd4e231d7 run/depot_autopilot: less interactive mode
In less interactive mode, the run script doesn't give up on missing test
archives but instead removes the corresponding tests and marks them "missing".
This mode avoids total failure of a platform in automated test infrastructures
when only a few archives are missing.

Fixes #3120
2019-01-30 13:35:29 +01:00
Martin Stein
e0af049124 depot_autopilot: fix regression with nr. of tests
The output of the number of tests to run was removed accidentally with the
commit "depot_autopilot: provide repeat mode".
2019-01-30 13:35:28 +01:00
Christian Prochaska
12c10dbcd1 libc: 'connect()' test
Fixes #3114
2019-01-30 13:35:28 +01:00
Martin Stein
3f60dcfae8 sel4 run/depot_autopilot: handle resource leak
After a certain number of tests, presumably some resource in core is exceeded
and loading the successive test fails. This quickfix looks out for the
characteristic Core error and then reboots to avoid that all successive tests
are marked as failed.
2019-01-14 12:34:41 +01:00
Norman Feske
81fb10daaa Consistently name block components
This patch replaces abbreviations like "blk", "cli", and "srv" by their
full forms "block", "client", and "server".

Fixes #1258
2019-01-14 12:34:39 +01:00
Alexander Senier
14cd115c82 Support GNAT.IO in Ada programs 2019-01-14 12:21:10 +01:00
Martin Stein
f0842a27c5 depot_autopilot: provide repeat mode
Adds an config attribute to the Depot Autopilot component:

:<config repeat>:

  Can be one of

    "false"         - process the given test list only once,
    "until_forever" - endlessly repeat processing the given test list,
    "until_failed"  - repeat processing the given test list until it fails.

Adds an environment variable to the Depot Autopilot Run script:

:TEST_REPEAT:

  Same as the <config repeat> attribute of the Depot Autopilot.

This is useful when having to debug very sporadic errors during one test
or a series of tests.
2019-01-07 12:33:56 +01:00
Stefan Kalkowski
c20c643b66 depot_autopilot: skip test-libc on rpi and sel4
Fix #3086
2019-01-07 12:33:55 +01:00
Christian Prochaska
9c8e76b190 base: 'sanitizer' test
Issue #3072
2019-01-07 12:33:54 +01:00
Martin Stein
f213a07c83 run/depot_autopilot: always log available results
This ensures that the depot_autopilot.run script, when exiting, always prints
a result overview of the so far available test results, except the Depot
Autopilot component has managed to print the result overview up to this point.
2019-01-07 12:33:50 +01:00
Stefan Kalkowski
59a8856773 depot_autopilot: sanitize XML results
Unequal numbers of double quotes let the XML parser of init fail,
therefore replace all double quotes when transfering previous
results after a reboot.

Ref #3027
2019-01-07 12:30:40 +01:00
Martin Stein
19e928271b run/depot_autopilot: reboot on some kernel faults 2019-01-07 12:30:37 +01:00
Martin Stein
54d1a676b7 run/depot_autopilot: reboot on log EOF 2019-01-07 12:25:46 +01:00
Martin Stein
f3a520e75d run/depot_autopilot: do not run on foc+panda 2019-01-07 12:25:46 +01:00
Martin Stein
abe80a4bfe run/depot_autopilot: raise initial timeout to 40s
At least sel4 on qemu/x86_32 on autopilot needs more time.
2019-01-07 12:25:45 +01:00
Martin Stein
6a8a5d2167 run/depot_autopilot: print nr of tests to run
The number of tests to run is the number of test package-archives minus the
the those that are skipped for the given platform. The number is printed
directly after checking if the given platform is supported by the run script.
It helps the surrounding test infrastructure to ensure that, for instance, a
result graph always reflects the same total number of tests, even though there
is a sporadic problem with booting the platform.
2019-01-07 12:25:45 +01:00
Martin Stein
da3e5fd3d6 run/depot_autopilot: reset qemu args correctly 2019-01-07 12:25:45 +01:00
Martin Stein
8db02b0a39 run/depot_autopilot: env-variables user-interface 2019-01-07 12:25:44 +01:00
Stefan Kalkowski
76966ed61a depot_autopilot: add route to IO_MEM for timer
* some timer drivers like epit for i.MX* need I/O memory access

Ref #3027
2019-01-07 12:25:44 +01:00
Stefan Kalkowski
6fb9c802d3 depot: enable package building for armv6
* Allow depot_autopilot to be run on top of arm_v6 too (Ref #3027)
2019-01-07 12:25:43 +01:00
Stefan Kalkowski
216dc49741 depot_autopilot: skip test-libc with low memory
* Skip test-libc on top of foc_pbxa9 and hw_imx53_tz as both platforms
  have to few memory in their configuration
2019-01-07 12:25:43 +01:00
Martin Stein
271fd0ba06 run/depot_autopilot: show depot size 2019-01-07 12:25:43 +01:00
Martin Stein
7f959a06f6 depot_autopilot: tune for multi-platform support
Issue #3027
2019-01-07 12:25:43 +01:00
Emery Hemingway
f2df40f58b Add simple Solo5 tests to depot autopilot
Add the following Solo5 tests to Autopilot: hello, fpu, globals, quiet,
blk. The remaining tests require a Rtc service or IP routing.

Ref #3027
2019-01-07 12:25:43 +01:00
Stefan Kalkowski
9857a0c956 depot_autopilot: disable test-python for ARM
The test for python requires x86 to be built. Therefore, there is no test
binary available when trying to execute that test on ARM with depot_autopilot.
2018-11-29 11:54:31 +01:00
Stefan Kalkowski
d7fa4cfb8b hw: enable eager FPU context switch for ARM
* Add an ieee754 FPU test
* Remove simple fpu test

Fix #2822
2018-11-29 11:54:31 +01:00
Martin Stein
89020bc3c0 depot_autopilot: move RTC test to autopilot.list 2018-11-29 11:54:30 +01:00
Emery Hemingway
a20d37c50a Add [depot_user] hook to depot_autopilot 2018-11-29 11:54:29 +01:00
Christian Prochaska
fe322b8e82 test-xml_generator: enable code coverage analysis
Issue #3048
2018-11-29 11:46:01 +01:00
Christian Prochaska
5639f31295 depot_autopilot.run: integrate gcov tool
Issue #3048
2018-11-29 11:46:01 +01:00
Martin Stein
789d908cee depot_autopilot: show result statistic
Print a line like "succeeded: 35 failed: 11 skipped: 2" below the list of test
results. Adds further attributes to <previous-results> to communicate also the
previous statistics.
2018-11-27 11:36:36 +01:00
Martin Stein
c65a13b3a4 run/depot_autopilot: power off before reboot 2018-11-27 11:36:36 +01:00
Martin Stein
eab7f54139 depot_autopilot: skip libc_getenv on foc x86
When doing the libc_getenv test on autopilot+foc+x86 and one of the
subsequent tests crashes the system so it gets rebooted by the run
script, the system doesn't come up again. It gets stuck after core
initialization.

Issue #3027
2018-11-27 11:36:35 +01:00
Martin Stein
2d041ac0fc depot_autopilot: evaluate sets of test packages 2018-11-16 15:07:53 +01:00