The former encoding was UTF-8, which works quite well if LC_CTYPE is
ensured to be an UTF-8 codeset (e.g., en_US.UTF-8 or C.UTF-8 . But, if
LC_CTYPE is set to C or latin1 for example, the Tcl regex library enters
an infinite loop because of unexpected characters used as markers
n the strings (e.g., SECTION SIGN U+00A7).
Therefore, the extract tool was converted to latin1 with the following
commands and now works for LC_CTYPE C and UTF-8 codesets.
iconv -f utf-8 -t latin1 tool/dts/extract > /tmp/e
cp /tmp/e tool/dts/extract
The NIC router used to ignore the IPv4 header fields "More fragments" and
"Fragment offset" completely. Therefore higher-level protocols of fragmented
IPv4 were interpreted wrong because each fragment was considered a self-
standing packet, expecting, for instance UDP/TCP headers somewhere inside of
the UDP/TCP data field. Normally, such packets were dropped as soon as the
UDP/TCP checksum check failed because of the misinterpretation. However,
it was also possible for fragmented IPv4 to pass the router although normally
only partially.
IPv4 fragmentation support in the router would introduce some potential
security risks and is presumably not an easy endeavor. So, for now, we settled
on not supporting IPv4 fragmentation. With this commit, the router simply drops
all fragmented IPv4. This is reflected to the log for each fragment as "drop
packet (fragmented IPv4 not supported)" when 'verbose_packet_drop="yes"' is
configured.
The new test 'run/nic_router_ipv4_fragm' is an automated test for this
behavior. The test is added to the autopilot list.
Ref #4236
This patch lifts the limitation for 32-bit ARM platforms and makes the
parameter --load-fastboot-device optional. If only one device is
present, it can be omitted.
Fixes#4232
This patch replaces the enumeration of boards in the drivers_nic_pkg and
drivers_interactive_pkg functions by inferring the package names from
the [board] name. This eases the addition of driver packages in external
repositories.
Fixes#4221
The depot tool 'build_bin_archive' used to ask only whether CCACHE is empty or
not and, if it wasn't empty, the tool added 'CCACHE := yes' to the build.conf
of the temporary build dir. This caused the sub-build processes to use ccache
even when the top-level CCACHE variable was set to a bogus value.
Also adds documentation of the CCACHE variable to the help message of the
'build' and 'build_bin_archive' tool.
Ref #4212
This patch adjusts the 'extract_initcall_order' utility to the changed
initcall symbol syntax introduced in Linux by the following commit:
https://github.com/torvalds/linux/commit/a8cccdd9
It uses an additional sed step to strip away the kmod prefix,
__COUNTER__, and __LINE__ information so that the symbol names match
those of earlier kernel versions.
Issue #4188
This patch enables the tool/dde_linux/create_dummies tool to operate on
a out-of-tree build directory. The Linux source tree is determined by
examining the build directory's Makefile.
Issue #4188
The new tool `extract_initcall_order` generates a header file out of a
Linux kernel build, which provides a sorted array with the initcall function
names of the kernel build. The order states the order in which the initcall
have to be called. It gets extrated out of the Linux kernel System.map.
Fixes#4172
The new tool `create_dummies` easily lists missing references,
while porting subsystems of the Linux kernel to Genode, e.g., device drivers.
Moreover it can automatically generate dummy implementations for functions
and global variables.
Fixes#4156
We used to export 'LC_COLLATE=C' for the environment of Genode tools. This was
meant to ensure that sorting is always done C-style and not dependent on the
users locale settings. This is required, for instance, to ensure that the same
archive always yields the same hash.
However, 'export LC_COLLATE=C' is not sufficient. It can be outruled by an
'LC_ALL' setting in the users environment. The manual of 'sort' recommends to
set 'LC_ALL=C' locally if you want reliable results and this is what this
commit does. Furthermore it removes the former 'export LC_COLLATE=C'
directives.
Note that I couldn't find a way to set 'LC_ALL' local to 'exec ... sort' in
TCL. This is why I set it global instead using TCLs 'env' array.
Note that the Make directive '$(sort ...)' and the TCL directive 'lsort',
unlike the Shell directive 'sort', are not affected by the users locale
settings.
Fixes#4144
Download gmp, mpc and mpfr with the download script provided by the
gcc source tree and let the gcc build system handle the build of these
libraries with the correct compile options. This fixes build issues on
armhf Linux and removes the need to maintain mpc and mpfr ports in
the Genode tree.
Issue #4094
The 'gprbuild' version which is provided by Ubuntu 18.04 has a bug
which prevents it from building 'ali2dep' with gcc/gnat 10. With
this commit, a newer 'gprbuild' version is built from source.
Issue #4094
This patch adds support for running Genode/Linux on the AARCH64
architecture.
- The kernel-agnostic startup code (crt0) had to be extended to
capture the initial stack pointer, which the Linux kernel uses
to pass the process environment. This is in line with the
existing startup code for x86_32 and x86_64.
- The link order of the host libraries linked to lx_hybrid
programs had to be adjusted such that libgcc appears at last
because the other libraries depend on symbols provided by
libgcc.
- When using AARCH64 Linux as host, one can execute run scripts
via 'make run/<script> KERNEL=linux BOARD=linux' now.
Issue #4136
This new runscript demonstrates how a file can be overwritten by vfs
import with the flag overwrite set to false. This is due to a bug
in lx_fs that is not propagating an ernno EEXIST syscall error.
Issue genodelabs#4104
Until now, Genode's Linux system call bindings were based on original
Unix system calls that were later superseded by more flexibile variants.
E.g., 'openat' is a modern version of 'open'. Even though Linux upholds
the compatiblity with the original versions for existing architectures
like x86, the legacy syscalls are absent for the recently added AARCH64
architecture. A good overview of the system calls accross the prominent
architectures can be found at
https://chromium.googlesource.com/chromiumos/docs/+/master/constants/syscalls.md
This patch updates Genode's syscall bindings to avoid legacy versions,
thereby easing the support for AARCH64. The used "modern" versions
were introduced back in Linux version 2 days. So we are fine to rely
on them.
The patch slightly changes the signature for lx_stat because this system
call is merely used to check for the existance of a file and its size.
The new name 'lx_stat_size' draws a closer connection to its use case.
That said, the stat syscall has not been updated to the modern statx
since statx is still a fairly recent addition.
Issue #4136
I created a test program which stresses the interplay between libc,
pthreads and vfs_pipe and may detect regressions.
The program starts a thread that spawns a bunch of workers, sends and
receives data to them through a pipe. When all data of a worker is
collected, the worker is finished (join()) and a new worker is started.
Both the number of bytes sent to the worker and the size of its
answer are random.
Issue #3967
- build debug version of core and bootstrap found under
_var/run/<runscript>.core_ (.bootstrap). This is a regression when
switching from object to archive library linkage of core.
fixes#4027
Instruct bender to enable Intel HWP on supported CPUs unconditionally
to gain more insight by using the feature most prominently in Sculpt.
Fixes#4040.
With this patch, the build directory's CCACHE configuration is applied
to the building of depot archives when using the --depot-auto-update
feature.
Note that this is effective only when resetting the hash files before
building so that each depot-auto-update results in the same-named
versions. Otherwise, the include paths (that include the version name as
part of the path) make cache hits impossible.
Issue #4004
This commit complements issue #4004 with support for the depot/create
tool. If CCACHE=1 is specified at the command line, the build directory
for building a binary archive is configured with 'CCACHE := yes'.
Do not link base and core libraries into on large relocatable .o file,
which is linked later to core - causing long link times. Create an
independent library archive out of the base and core libraries that can
be linked faster.
issue #4027
- remove Spike/BBL support in favour of Qemu (>=4.2.1)
- add 'riscv_qemu' board, remove 'spike' board'
- update to privileged ISA v1.10 (from v1.9.1)
- use direct system calls for privileged core threads (they call into
the kernel and don't use mode changing system calls, i.e. 'ecall',
semantics)
- use 'OpenSBI' semtantics for SBI calls (to machine mode) instead of
BBL
issue #4012
This patch simplifies the use of ccache with the build system. Up until
now, each developer had to set up the ccache hooks manually, adjust the
PATH variable, and customize the etc/tools.conf in each build directory.
With the patch, ccache can be enabled by un-commenting a single line in
the etc/build.conf file.
Fixes#4004
This patch moves the bootstrap-link-address information from the
tool/run/boot_dir/hw file to board-specific property files that can be
accessed by using the board as key. This eliminates the need to
customize boot_dir/hw when hosting board-support in an external
repository.
Fixes#3998
This patch allows for supplementing the existing run-tool plugins with
additional plugins hosted inside a repository. E.g., it enables the run
tool to find world/tool/run/boot_dir/hw automatically.
Fixes#3993
* Remove SPEC declarations from mk/spec
* Remove all board-specific REQUIRE declaratiions left
* Replace [have_spec <board>] run-script declarations with have_board where necessary
* Remove addition of BOARD variable to SPECS in toplevel Makefile
* Move board-specific directories in base-hw out of specs
The new helper function returns a boolean value analoque to 'have_spec' if the
BOARD variable corresponds to the given value. It shall replace [have_spec <board>]
declarations in run-scripts.
Ref #3971
* Introduce 'use_arp' configuration flag at the NIC router in order to disable
requesting IP-to-MAC address resolutions via ARP for domains.
* Add automated run/nic_router_disable_arp test to the autopilot that tests
the effect of the new flag
* Document the new flag and the new test in the NIC router README
Fixes#3935
* The NIC router now considers, memorizes, and, if configured, reports
multiple DHCP option 6 entries from DHCP replies that it received as DHCP
client
* A DHCP server at the NIC router can now be configured statically with
multiple DNS server addresses to propagate
* The 'dns_server_from' attribute of the DHCP server of the NIC router now
supports the forwarding of multiple DNS server addresses
* The automated run/nic_router_dhcp test tests all the above mentioned new
functionality and reconfiguring it at runtime. The test was added to the
autopilot.
* All run scripts were adapted to fit the new NIC router configuration
interface
Fixes#3952
The CBE repository contained a lot of Genode-specific code despite the fact
that the CBE core logic is not bound to Genode in any way. Therefore the
Genode-specific CBE code is moved to the 'gems' repository to form part of
Genode mainline. The remaining CBE code becomes a port in Genode instead of
being invoked as sub-repository.
The commit combines the following work steps:
* add all files removed from CBE repository
* add CBE port files
* make all CBE libs and targets build again
* make all CBE run scripts succeed again
* make all CBE recipes build again
* make CBE autopilot succeed again
* let CBE autopilot use 'libsparcrypto' contrib dir and Genode build dir
instead of '.ci' dir in CBE contrib dir (remove '.ci' dir from CBE repo)
* let CBE autopilot always check for all ports
* make CBE autopilot directly executable
* fix license headers in all Genode CBE files
* remove unused VFS replay component
* remove unused CBE test
* remove unused external crypto
* remove unused files in run dir
* remove unused external trust anchor
* add cbe_tester test to autopilot list
* get rid of directories 'include/cbe_*' and 'include/utils'
Fixes#3937
The driver is faily simple and does not support fancy features like
TCP checksum offloading or vlan filtering, but it is fully capable of
running every Genode network based scenario I've tried. Its currently
known to work on virt_qemu arm platforms and x86_64.
Fix#3825
Right now the same code dealing with nic setup on qemu is duplicated
in many different run scripts. It makes it unnecesarily complex to
change the existing config or add support for new nic types. Lets move
all this common code to qemu.inc.
Ref #3825
It seems Qemu 5.0.0 no longer supports gic_version parameter.
Apparently this only worked due to some compat feature that
was dropped in Qemu 5.0.0. The replacement which works on both
old and new Qemu versions is gic-version.
Fixes#3823
This commit applies the transition from the "Input" session to the "Event"
session to the event-filtering mechansim. The functionality of the
input_filter is now provided by the event_filter. The event filter
requests only one "Event" session as destination for the filter result,
which is usually routed to the nitpicker GUI server. It provides an
"Event" service to which any number of event sources can connect.
The configuration of the filter chain remains almost the same. Only the
declaration of the <input> nodes is no longer needed. Instead, the
configuration must specify <policy> nodes, which define the mapping of
"Event" clients (event sources) to the inputs used in the filter chain.
The patch adjusts all uses of the nitpicker GUI server accordingly such
that the event filter reports events to nitpicker's event service
instead of having nitpicker request an "Input" session. This dissolves
the dependency of nitpicker from input drivers.
Issue #3827
* Differentiate in between different architectures with assembler routines
for correct measures
* Automate first step measuring of 10G bogomips across different hardware
Fix#3785
base-linux uses seccomp to reduce the available system calls
to the minimum set needed to run base-linux. There are still
some syscalls that allow accessing global state which should
be further reduced.
The combination of seccomp and socket descriptor caps should
provide an intermediate level of security for base-linux
thereby enabling base-linux as a migration path from using
the Linux kernel to the use of microkernel-based Genode.
Fixes#3581
This patch adds support for booting base-hw kernel on qemu-arm virt
machines. The arm_virt machine has 2GB of RAM, 2 Cortex A15 cores and
uses GICv2 interrupt controller. The arm_64_virt machine also has 2GB of
RAM, but has 4 Cortex A53 cores and uses GICv3. Both machines use PSCI
to boot additional CPU cores.
Fixes#3673
This patch fixes a regression with run/log_core introduced by
2064ffd64b nova: support multidimensional affinity space
The run tool uses a log message 'run_boot_string' to detect successful
boot as well as to detect unexpected reboots. This message should never
be logged twice and, thus, should not be part of the core_log. The patch
mentioned above moved the former run_boot_string after the core_log
initialization.
This patch replaces the former Noux instance by a sub init that hosts
the combination of VFS + fs_rom + bash. If bash exits, the whole sub
init exits, which implicitly tears down the terminal session used by the
sub init. The latter is expected by the test sequence.
Issue #3696
From the man page of expect:
> Both expect and interact will detect when the current process exits
> and implicitly do a close. But if you kill the process by, say, "exec
> kill $pid", you will need to explicitly call close.
Fixes#3569
The check was accidentally in
66d44289e1 tool/ports: streamline hash tool usage
because $(call VAR, ...) just expands to nothing without errors if VAR
is undefined.
Introduce 'XZ_THREADS' to override the default number of threads used
for compression, which is still set to '1'. As using multiple threads
will increase the amount of memory needed during the compression and
potentially influences the size of the created archive, setting the
variable limits the impact.
Fixes#3431.
In case RUN_OPT_AUTOPILOT is changed on consecutive calls of "autopilot
--keep", the changes are now effectively propagated to the run tool.
I also documented the RUN_OPT_AUTOPILOT environment variable.
Replace the notion of board-specific platforms by clear definition
architecture and board.
Also added new command line switch for "targets"
-t <target> test target as triple of architecture-board-kernel,
e.g., arm_v7a-pbxa9-hw
Related to #3316Fixes#3417
This patch handles a few corner cases that appear in the headers of
Genode 19.05, in particular the new client-side block API in
'block_session/connection.h'.
* The extract tool determines and reports all missing ports at once.
* The extract tool automatically prepares all missing ports if PREPARE_PORTS=1.
* The missing_ports tool prints a list of missing ports for given archives.
Fixes#3353
Components like kernel, core, and bootstrap that are built for a
specific board need to reside inside the same architectural dependent
build directory. For instance there are sel4, foc, and hw kernel builds
for imx6q_sabrelite and imx7d_sabre, which have to reside inside the same
arm_v7 build directory.
This commit names those components explicitely, and adapts the run-tool to it.
Fix#3316
nullglob is needed to get an empty shell glob in case no matching binary
archive (path) exists. Otherwise, the original glob string including the
* is returned and used in the for loop.
This patch handles version collisions of binary archives. If a binary
archive for an (automatically) updated src-archive version already
exists in the depot, the extract tool removes the binary archive because
its existing content may stem from another src content (a version from a
different topic branch). This ensures that the new version is always
built, not skipped, when 'UPDATE_VERSIONS=1 REBUILD=' is specified.
Fixes#3267
Until now, Genode referenced a fork of the outdated elfloader-tool
to enable bootstrapping of sel4 on ARM platforms. Because the
elfloader is inherently dependent on the used platforms newer
ARM versions supported by the kernel could not be loaded by the
outdated elfloader. This commit uses a fresh fork of the nowadays
used sel4_tools repository.
Ref #3251
In cases where the signing failed (maybe the passphrase for the key was
not available at this time), there remained an empty .sig file, which is
newer than the to-be-signed file and, therefore, prevents subsequent
signing processes.
The input for the pkg index is located at gems/run/sculpt/index.
The sculpt.run script uses this input for generating the depot index
file at depot/<user>/index/<version>.
The tool/depot/publish tool support arguments of the form
<user>/index/<version> where <version> corresponds to the Sculpt
version.
Issue #3172
The 'run_genode_until' procedure only called 'run_power_on' to reset
the target machine. That works will with the softreset module, which
is used by all x86-based test system but falls short regarding ARM
boards. The way those boards are connected requires turning the power
off and on for a complete cycle.
