Commit Graph

580 Commits

Author SHA1 Message Date
Roman Iten
f65a7650c5 depot: allow to inject version command
Issue #2991
2018-09-24 11:18:23 +02:00
Alexander Boettcher
9eabe316bf depot: support for gpg command override
Fixes #2981
2018-09-21 15:49:52 +02:00
Emery Hemingway
b51ee34b11 Depot: support for GPG command line override
Allow the gpg utility to be overridden on the command line. For
example, './tool/depot/publish GPG="ssh 10.0.1.3 gpg" ...' would invoke
a remote gpg over SSH.

Fix #2981
2018-09-17 14:11:37 +02:00
Christian Helmuth
a6d7d5da07 tool/port: limit destructive git operations to port
Issue #2959
2018-09-05 11:04:24 +02:00
Emery Hemingway
a6acdac1cf Stop port recipe Git fallthrough with CONTRIB_DIR/.git
Ref #2959
2018-09-05 11:04:24 +02:00
Emery Hemingway
1370fa7631 Remove Noux networking support
Noux networking is not well tested and not in practical use. It also
uses a number of deprecated libraries.

Ref #2535
Fix #2955
2018-09-05 11:04:21 +02:00
Christian Prochaska
ecc4ac795a tool_chain: fix build problems on Ubuntu 18.04
Fixes #2939
2018-08-30 09:24:46 +02:00
Christian Prochaska
9d712b2ce9 vfs: libc_vfs_fs_chained.run script
Fixes #2944
2018-08-28 17:10:58 +02:00
Christian Helmuth
240e70d989 run: configurable AMT power-on timeout
The default timeout is 5 seconds and can be changed like follows.

  RUN_OPT += --power-on-amt-timeout 11
2018-08-28 17:10:54 +02:00
Norman Feske
c270e4fb30 base: exception safety during registry iteration
This patch fixes a problem in the non-const 'for_each' method of the
'Registry' data structure. If an exception was thrown from within the
functor of the 'for_each' operation, the not yet processed items of the
registry were dropped from the registry, which is not expected.
2018-08-28 17:10:54 +02:00
Norman Feske
b4f596b197 Tool for querying information from file system
Issue #2936
2018-08-28 17:10:53 +02:00
Martin Stein
d2a36c5958 base: define generic config XSD types globally
The new base/xsd/config.inc defines generic XSD types such as 'Boolean' or
'Session_label'. It can be included in config XSD files by using:

! <xs:include schemaLocation="file://${GENODE_CONFIG_INC}"/>

The string ${GENODE_CONFIG_INC} is replaced by the run tool with the
above mentionened file path.

Issue #2897
2018-08-28 16:48:47 +02:00
Alexander Boettcher
a149131dc2 run: add microcode chain bootloader to nova
Fixes #2762
2018-08-28 16:48:43 +02:00
Johannes Kliemann
3b3e155c79 dde_zircon: add pc-ps2 driver
Fixes #2926
2018-08-28 16:45:26 +02:00
Emery Hemingway
c75d9e2ee0 Rename network scenarios to make use of lxip explicit 2018-08-28 16:45:23 +02:00
Emery Hemingway
d9a4773194 LwIP VFS plugin
This patch reintroduces the LwIP stack to libc as a VFS plugin
implementing the socket_fs interface. Rather than use LwIP's socket
emulation layer this plugin interfaces directly to LwIP raw API and is
single threaded.

The internal TCP parameters of the stack are untuned.

Fix #2050
Fix #2335
2018-08-02 14:36:48 +02:00
Sebastian Sumpf
b6bc44fd10 run: add support to preserve 'genode' directory
Add '--preserve-genode-dir' to RUN_OPT in order to preserve the 'genode'
directory in '<build_dir>/var/run/<run-script>'
2018-08-02 14:36:46 +02:00
Emery Hemingway
59ac5b10c7 Plugin for importing VFS content
This new vfs_import plugin allows a VFS instance to be populated during
construction using a sub-VFS configured in an '<import>' configuration
node. This allows the ram_fs File_system server to be replaced by the
VFS server by reimplementing the ram_fs 'content' feature.  At the
moment the copying of symlinks is not enabled, and the resources
obtained by the import file-system may not be freed after the import is
finished.

Fix #2906
2018-08-02 14:36:42 +02:00
Emery Hemingway
f3abee631a File_system packet processing test
Test of File_system I/O scheduling. It appears that a client not
submitting I/O packets slower than the VFS server can process will
starve other clients.

Ref #2900
2018-08-02 14:36:36 +02:00
Pirmin Duss
6bb5b7190c depot: verify that runtime file is valid
issue #2894
2018-08-02 14:36:34 +02:00
Christian Prochaska
1d3ec6f0ae Test successive thread creation and destruction
Fixes #2887
2018-07-03 09:39:34 +02:00
Christian Prochaska
54a7301225 bender: update to fix foc boot issues
Fixes #2881
2018-07-03 09:39:31 +02:00
Sebastian Sumpf
10485b0d39 run: rename 'check_installed' to 'installed_command'
Also remove 'requires_installation_of', while also checking sbin
directories in 'have_installed'. The run scripts have been adjusted
accordingly.

Fixes #2853
2018-07-03 09:39:31 +02:00
Roman Iten
7e126f7fb6 run: use sgdisk instead of parted in image/uefi
Issue #2862
2018-06-29 10:44:54 +02:00
Roman Iten
4067cc7962 run: use sgdisk instead of parted in image/disk
Issue #2862
2018-06-29 10:44:54 +02:00
Martin Stein
14357b0b44 nic_router_flood.run: test DoS against router
Currently has three clients that continuously create new UDP/TCP/ICMP
connections through the NIC router with NAT to the outer world and they
get never closed.  A fourth client does normal ping through the same
domain to the outer world that must succeed even after the RAM quota of
the other session at the router is exhausted. The test is restricted to
Qemu to not being at risk to flood real networks.

Issue #2857
2018-06-29 10:40:13 +02:00
Norman Feske
7088e4faaa os: add init_loop test
This test monitors the RAM quota of a dynamic init and a server hosted
within the dynamic init in the presence of a repeatedly created and
destructed client.
2018-06-12 12:11:46 +02:00
Christian Helmuth
7d4174bceb run: disable heuristics for apple gpt in parted
Newer revisions of parted require special privileges due to use of the
dmidecode tool, which logs permission errors to standard error. In these
cases parted still succeeds with its operation, so just ignore the
jabbering and prevent expect to exit.
2018-06-12 12:11:45 +02:00
Alexander Boettcher
193e14a489 grub2: avoid switching modes
to reduce flickering during boot

Issue #2778
2018-06-12 12:11:44 +02:00
Alexander Boettcher
70028f8445 nova: export kernel messages via memory
Fixes #2207
2018-06-12 12:11:37 +02:00
Christian Helmuth
a7dadd9dae depot: ignore some files on pkg generation
Now, the typical gitignore patterns

  *~ *.rej *.orig *.swp

are not added to the pkg.
2018-05-31 17:07:21 +02:00
Christian Helmuth
5d5756d0cb Sort autopilot test list 2018-05-31 14:02:20 +02:00
Alexander Boettcher
4979221152 tool: support to use graphical grub2 2018-05-31 13:51:23 +02:00
Alexander Boettcher
30d004ab64 base: extend rm_nested test 2018-05-31 12:28:16 +02:00
Martin Stein
6a384df412 autopilot.list: add nic_router_uplinks
Issue #2815
2018-05-30 13:36:35 +02:00
Norman Feske
b1b70e6c35 tool/link.h: remove outdated comment 2018-05-30 13:36:29 +02:00
Norman Feske
096c1e1f7f run/image/disk: omit * from GENODE partition label
This way, the boot partition is not considered as default partition by
Sculpt unless the user explicitly marks it as such.

Issue #2778
2018-05-30 13:36:22 +02:00
Christian Helmuth
66d44289e1 tool/ports: streamline hash tool usage 2018-05-30 13:36:22 +02:00
Emery Hemingway
82897d672e Remove SHA1 verification for port downloads
Revert support for both SHA1 and SHA256 verification and support SHA256
only.

Ref #2767
2018-05-30 13:36:22 +02:00
Alexander Boettcher
b1389416d2 run: created hybrid MBR for image/disk
Issue #2778
2018-05-30 13:36:21 +02:00
Roman Iten
874815ebf6 run: improve error handling regarding depots
Fixes #2821
2018-05-30 13:36:21 +02:00
Alexander Boettcher
501d82b99f platform_drv: skip assign device if iommu missing
Avoid red messages in the log on machines with no IOMMU, which recurring
confuse people.

Issue #2801
2018-05-30 13:36:18 +02:00
Roman Iten
5d23ef9447 run: make run script fail on unmet dependencies
Fixes #2820
2018-05-30 13:36:15 +02:00
Christian Helmuth
0f7de9268d check_port_source: download first kilobytes of file
Unfortunately, some sites do not support the HTTP/1.1 range header and
just serve the whole file, which takes it's time and may result in a
timeout.

Fixes #2819
2018-05-30 13:36:14 +02:00
Alexander Boettcher
bbfa65a88a tool: remove rump tools
image/disk was the last user of the tools.

Issue #2778
2018-05-30 13:36:14 +02:00
Norman Feske
48bc69b14b run/image/disk: consider ext2 overhead
The original value of 256K does not suffice for images of the size of
sculpt.
2018-05-30 13:36:10 +02:00
Alexander Boettcher
de9d1afa99 tool/run: move binaries to boot directory
Issue #2778
2018-05-30 13:36:08 +02:00
Alexander Boettcher
25f2c44874 run: use GPT for image/disk BIOS legacy
Issue #2778

The "create_grub2" script functionality is moved to the g2fg contrib sources
and also the GRUB2 head image (tool/grub2-head.img). The head image is now
partitioned as GPT and bootable in BIOS legacy mode.
2018-05-30 13:36:08 +02:00
Christian Prochaska
6080f4fb56 Fix 'vbox5_genode_usb_hid.run' / add to autopilot
Fixes #2802
2018-05-30 13:36:08 +02:00
Emery Hemingway
51722c1fe6 Update Nim to 0.18.0
Fix #2804
2018-05-30 12:26:20 +02:00
Christian Prochaska
f347cb90f1 depot: qt5 recipes
Fixes #2792
2018-05-30 12:26:19 +02:00
Roman Iten
058b92ca99 depot: tool for publishing current versions
In order to simplify the automation of publishing archives, this tool
publishes the current version as specified in the recipes.

Fixes #2799
2018-05-03 16:11:53 +02:00
Martin Stein
09ad962418 nic_bridge.run: don't execute on hardware
The nic_bridge test is not suitable to be executed on hardware so it should,
in case, print an appropriate message and return without doing anything else.

Issue #2788
2018-05-03 15:31:57 +02:00
Johannes Schlatow
48aa50b97c test/trace_logger: fully automate test
Issue #2735

Co-authored-by: Martin Stein <martin.stein@genode-labs.com>
2018-05-03 15:31:55 +02:00
Christian Helmuth
a496360c5b tool/ports: utility for checking port hashes
Our nightly did not detect the current stale hashes because only updated
(in the sense of a changed hash) ports are prepared. Others were left
untouched besides the check_port_source step. Now, check_hash also
checks for missing hash file updates.
2018-05-03 15:31:53 +02:00
Emery Hemingway
80b3994500 prepare_port: prefer SHA256 file verification
SHA1 is susceptible to collision attacks and is generally deprecated.
Source code archives are particularly vulnerable because the hash digest
can be tweaked by hiding by arbitrary data in code comments and files
not processed during build.

With this in mind the 'prepare_port' tool now attempts to verify digests
as SHA256 with a fallback to SHA1. When CHECK_HASH=no is set the tool
will refuse to verify digests as SHA1. The use of SHA1 for creating
unique port versions is retained because the hashes are produced locally
from inputs stored in a git history.

Issue #2767
2018-05-03 15:31:19 +02:00
Christian Helmuth
169d5bc376 run: non-fatal exit on unsupported base build config
The kernel-specific boot dir scripts exit with 0 and log "Test
requires..." to inform the logger frontend of the base-archive
unavailability.
2018-04-19 12:39:20 +02:00
Norman Feske
cb78516bf1 os: add tests to autopilot
Since the autopilot has become able to use the depot, futher tests can
be enabled by default.
2018-04-19 12:38:53 +02:00
Josef Söntgen
04516a0d39 nvme_drv: add driver for NVMe storage devices
This driver component provides support for using consumer NVMe storage
devices, i.e. it omits name space managment and will always use the
first name space, on Genode. For now it defaults to a reasonable low
configuration:

  -    1 I/O queue (completion/submission tuple)
  -  128 entries in the I/O queue
  - 4096 as the only I/O transaction memory page size

Fixes #2747.
2018-04-19 12:38:22 +02:00
Christian Helmuth
49f64a0cac run: adapt qemu RAM size for okl4/x86
Our static okl4 system-integration configuration specifies a fixed RAM
range from 32 to 800 MiB, which must be satisfied by Qemu or strange
errors happen due to accesses beyond RAM. (Fixes current issues with
nic_router.run.)
2018-04-10 11:20:46 +02:00
Martin Stein
72ddbf84ee autopilot.list: add ping_nic_router test
Fixes #2732
2018-04-10 11:20:44 +02:00
Martin Stein
29685f48ef autopilot.list: add ping test
Issue #2732
2018-04-10 11:20:44 +02:00
Emery Hemingway
1ca2b6fab3 run: fix depot tool regex to support '.' versions
Fix #2720
2018-04-10 11:11:50 +02:00
Alexander Boettcher
bc2a998261 demo: test demo scenario bootup by autopilot 2018-03-29 14:59:06 +02:00
Reto Buerki
47724c68c2 platform_drv/x86: Switch to ECAM/MMCONF
Switch port I/O based PCI config space access to memory-mapped IO.  The
base address of the PCI configuration space is acquired by mapping the
ACPI ROM and reading the first <bdf> node. An exception is thrown if the
first <bdf> node is not for PCI domain zero or if multiple <bdf> nodes
exist. This is to reduce complexity and also because multiple PCI
domains are rare.

The PCI configuration space is accessed via I/O mem dataspace which is
created in the platform_drv root and then passed on to the PCI session,
device components and finally to the actual PCI config access instances.

The memory access code is implemented in a way to make it work with Muen
subject monitor (SM) device emulation and also general x86 targets. On
Muen, the simplified device emulation code (which works also for Linux)
always returns 0xffff in EAX to indicate a non-existing device.
Therefore, EAX is enforced in the assembly templates.

Fixes #2547
2018-03-29 14:59:04 +02:00
Emery Hemingway
c3b483d12a Prepate_port: create directories when downloading bare files
Ref #2718
2018-03-27 13:44:27 +02:00
Norman Feske
d59d07b5e3 os: util/utf8.h for UTF-8 string handling
This patch adds a simple UTF-8 decoder at 'os/include/util/utf8.h'
along with a test at 'os/run/utf8.run'.

Fixes #2717, related to issue #2716
2018-03-27 13:44:27 +02:00
Norman Feske
eaa412022f tool/run: improve depot/create suggestion
When a pkg is missing, the user should create <arch>/pkg instead
of only the missing pkg. This way, all depending binaries are created in
one step. Otherwise the missing binaries are detected at the next time
the run script is executed. This patch relieves the user from iterating
manually.
2018-03-27 13:43:09 +02:00
Christian Helmuth
2eb13c9047 depot: execute content rules sequentially 2018-03-08 12:05:09 +01:00
Stefan Kalkowski
bad002acb1 hw: link core with debug symbols optionally
This commit changes the semantic of run_boot_dir to link a core
binary with debug symbols only when the designated library is in place.

Follow-up fix refering to issues #2339 and #2700
2018-03-08 12:05:06 +01:00
Adrian-Ken Rueegsegger
9cb71d0260 run: Move shared procedure to load/pxe.inc
The install_pxe_bootloader_to_run_dir procedure is required by the tftp
as well as the ipxe load script. Move it to a separate file which is
include by both.
2018-03-08 12:05:04 +01:00
Christian Helmuth
ad56976131 tool/publish: fix help for PUBLIC_DIR 2018-02-28 11:05:03 +01:00
Adrian-Ken Rueegsegger
9e4ff8c196 Update Muen port
- Use latest Muen version
- Sync VirtualBox Muen subject state
- Drop unneccessary subject IP patch
- Adapt Muen RUN_OPTs
- Update documentation

Note: the GPL 2017 toolchain is now required and as the debug output
      format has changed the mulog-subject.py script must be updated on
      autopilot instances.
2018-02-28 11:04:57 +01:00
Norman Feske
7670f048c1 tool/depot: publish missing archives only
The timestamp-based rules of make do not work well for the publish tool
because depot archives are often re-created with the same content but a
different modification time, in particular when creating archives via
'FORCE'. This way, those archives are re-published every time, which
becomes a time-consuming operation since all archives must be signed.

This patch filters the targets based on the content that is already
present in the public/ location. All existing archives are skipped.
2018-02-14 20:41:04 +01:00
Martin Stein
f0cfaaa147 create_uboot: better documentation
Issue #2654
2018-02-09 13:34:19 +01:00
Norman Feske
6d6aff0093 run: no duplicates in create_tar_from_depot
This patch ensures that depot snapshots created by the run tool
won't contain duplicated content, which would result cyclic hard
links.
2018-02-09 13:34:14 +01:00
Norman Feske
dcce0a1032 depot: force SHA256 digest for signing archives 2018-02-09 13:34:14 +01:00
Martin Stein
47b11bf9b7 autopilot.list: add netperf_*_router tests
Fix #2624
2018-02-09 13:26:03 +01:00
Norman Feske
95785a9c46 Tool for checking consistency between lib and ABI
Fixes #2639
2018-01-17 12:14:42 +01:00
Josef Söntgen
67869a697b run: add deport-dir argument
By specifying '--depot-dir' in the RUN_OPTS it is possible to override
the default depot ($GENODE_DIR/deport) location with the given path.
2018-01-17 12:14:40 +01:00
Josef Söntgen
114af00913 depot: make DEPOT_DIR overridable 2018-01-17 12:14:40 +01:00
Christian Helmuth
f44edd407b run: fix import_from_depot for variable args
Apply the approach to join the argument list 'args' to pass a single
list argument to _collect_from_depot. Actually, this story teaches me to
shy away from {*} because of its special semantics, which are

  {*} makes each item in a list an individual argument of the current
  command (https://wiki.tcl.tk/17158)

This is a direct follow up to "run: let import_from_depot accept list
variables", which broke calling import_from_depot with a number of
individual arguments.

Issue #2619
2017-12-22 09:31:36 +01:00
Martin Stein
75df14f190 run: use XSD files from depots too
Fix #2619
2017-12-21 15:01:55 +01:00
Martin Stein
f01bdb9949 run: let import_from_depot accept list variables
When doing something like this in a run script ...

lappend arg X
lappend arg Y
lappend arg Z
import_from_depot arg

... the internals of import_from_depot do not treat $arg as list but as one
string and interprets it as malformed depot path. We can handle this by
applying {*} to $arg inside the procedure.

Issue #2619
2017-12-21 15:01:54 +01:00
Christian Prochaska
bd36933636 abi_symbols: annotate the size for 'R'-type symbols
Fixes #2622
2017-12-21 15:01:51 +01:00
Norman Feske
2f7217b29f depot: add REBUILD argument to create, build tools
Fixes #2617
2017-12-21 15:01:50 +01:00
Alexander Boettcher
892f51ab1c os: add app to transform kernel/core output to LOG
Issue #2207
2017-12-21 15:01:48 +01:00
Alexander Boettcher
30fd10933a okl4: support run script name containing "core"
Issue #2207
2017-12-21 15:01:46 +01:00
Norman Feske
17bf2564fa depot: improved args check in dependencies tool
This patch lets the dependencies tool give better feedback to the user
whenever an argument lacks the needed version information.
2017-12-21 15:01:45 +01:00
Norman Feske
a52541de18 depot: move versions to subdirectory
This patch changes the depot layout such that each archive is
represented as a directory that contains the versions of the archive as
subdirectories.

Issue #2610
2017-12-21 15:01:45 +01:00
Norman Feske
2d041f0e9c depot: switch from .tgz to .tar.xz
Issue #2610
2017-12-21 15:01:44 +01:00
Martin Stein
fb2398dbf2 run: target-specific config schemata
The run tool now by default checks configurations with target-specific
XML schemata. Each component may define a config schema file in its
target.mk via the CONFIG_XSD variable. When the run tool has checked an
configuration of an init instance, it additionally goes through the
start nodes of the config. For each start node it checks whether there
is an XSD file that matches. If so, the run tool also checks the config
of the start node (if existant). This is done recursively. I.e., also
the child configs of a sub-init of a sub-init of the top-level init
receive a config check.

Issue #2600
2017-12-21 15:01:36 +01:00
Christian Helmuth
18fcbbe1a1 depot: support path names including spaces
Related to #2582
2017-11-30 11:23:09 +01:00
Alexander Boettcher
858f5732ba hw: add mbi2 framebuffer support
Issue #2555
2017-11-30 11:23:09 +01:00
Alexander Boettcher
e1ac124a4d hw: evaluate also ACPI RSDP v1 with MBI2
Issue #2526
2017-11-30 11:23:09 +01:00
Alexander Boettcher
2b6ae514b5 sel4: add mbi2 framebuffer support
Issue #2555
2017-11-30 11:23:09 +01:00
Emery Hemingway
caba2d3021 Enforce use of a custom Nim compiler
Use a custom Nim compiler at '/usr/local/genode-nim' that can be built
with a makefile at 'tool/tool_chain_nim'.

Fix #2545
2017-11-30 11:23:06 +01:00
Alexander Boettcher
9646c17f2e bender: fix free memory calculation for Multiboot2
Fixes #2487
2017-11-30 11:23:04 +01:00
Martin Stein
054b8de67b tool/run/grub2: user-friendly missing message
Ref #2526
2017-11-30 11:23:03 +01:00
Martin Stein
f6dd1f9b9c tool/depot: user-friendly error on missing port
In general, make the message conform to the according message of the
common build system. Adds instruction how to fix the missing/outdated
port.

Fixes #2573
2017-11-30 11:23:03 +01:00