When doing something like this in a run script ...
lappend arg X
lappend arg Y
lappend arg Z
import_from_depot arg
... the internals of import_from_depot do not treat $arg as list but as one
string and interprets it as malformed depot path. We can handle this by
applying {*} to $arg inside the procedure.
Issue #2619
This patch changes the depot layout such that each archive is
represented as a directory that contains the versions of the archive as
subdirectories.
Issue #2610
The run tool now by default checks configurations with target-specific
XML schemata. Each component may define a config schema file in its
target.mk via the CONFIG_XSD variable. When the run tool has checked an
configuration of an init instance, it additionally goes through the
start nodes of the config. For each start node it checks whether there
is an XSD file that matches. If so, the run tool also checks the config
of the start node (if existant). This is done recursively. I.e., also
the child configs of a sub-init of a sub-init of the top-level init
receive a config check.
Issue #2600
In general, make the message conform to the according message of the
common build system. Adds instruction how to fix the missing/outdated
port.
Fixes#2573
If there is no debug/core-<kernel>.o library available, e.g., when
running a scenario in a fresh build directory where core is provided
via a depot package, the build_core_image step of the run tool was
canceled in the past, when trying to access the non-existing library.
This commit changes the semantic of build_core_image to link a core
binary with debug symbols only when the designated library is in place.
This fixup is related to issue #2339
* introduce new syscall (core-only) to create privileged threads
* take the privilege level of the thread into account
when doing a context switch
* map kernel segment as accessable for privileged code only
Ref #2091
* introduces central memory map for core/kernel
* on 32-bit platforms the kernel/core starts at 0x80000000
* on 64-bit platforms the kernel/core starts at 0xffffffc000000000
* mark kernel/core mappings as global ones (tagged TLB)
* move the exception vector to begin of core's binary,
thereby bootstrap knows from where to map it appropriately
* do not map boot modules into core anymore
* constrain core's virtual heap memory area
* differentiate in between user's and core's main thread's UTCB,
which now resides inside the kernel segment
Ref #2091
In the image.elf file, the very last boot module is followed
by arbitrary other core-local data. Since those boot modules are
exported as page-granular dataspace to the outside of core via core's
ROM service, we need to ensure that the last page is padded with zeros.
When building Genode on a Linux system running in a Xen Dom0, the 'xen'
run target can run a Genode scenario in a Xen DomU.
Usage: in build/x86_*/etc/build.conf, define:
RUN_OPT = --include boot_dir/$(KERNEL) --include image/iso --include power_on/xen --include log/xen --include power_off/xen
The Xen DomU runs in HVM mode and loads Genode from an ISO image. Serial
log output is printed to the console and graphical output is shown in an
SDL window.
The Xen DomU ist managed using the 'xl' command line tool and it is
possible to add configuration options in the 'xen_args' variable in a run
script. Common options are:
- disabling the graphical output:
append xen_args { sdl="0" }
- configuring a network device:
append xen_args { vif=\["model=e1000,mac=02:00:00:00:01:01,bridge=xenbr0"\] }
- configuring USB input devices:
append xen_args { usbdevice=\["mouse","keyboard"\] }
Note: the 'xl' tool requires super-user permissions and interactive
password input can be troublesome in combination with 'expect' and is not
practical for automatic tests. For this reason, the current implementation
assumes that no password input is needed when running 'sudo xl', which can
be achieved by creating a file '/etc/sudoers.d/xl' with the content
'user ALL=(root) NOPASSWD: /usr/sbin/xl'
(where 'user' is the Linux user name).
Fixes#2504
This patch enables one to supply a list of arguments to
'install_config'. The arguments appear concatenated in the resulting
config file. This is useful for embedding dynamically generated content
into an otherwise static configuration. For example, in the following
config, the 'pkg' attribute of the '<query>' config node is defined by
the return value of the 'query_pkg' function:
install_config {
<config>
...
<start name="depot_query">
...
<config>
<query pkg="genodelabs/pkg/} [query_pkg] {"/>
</config>
</start>
</config>}
- Update FatFS port from 0.07e to 0.13
- Multi-device support
- Basic test at run/fatfs
- Adaption of existing components
Note, ffat is now consistently renamed to fatfs.
Ref #2410
relocation is required, to avoid overlapping phdr in the elf binaries, but
sel4 will fail to boot if bender relocates the images at the end of physical
memory. Limit the physical relocation address to a fixed 256M value, so that
sel4 will work.
Issue #2451
The new version of the test exercises the combination of fs_report with
ram_fs and fs_rom as a more flexible alternative to report_rom.
It covers two corner cases that remained unaddressed by fs_rom and
ram_fs so far: First, the late installation of a ROM-update signal
handler at fs_rom right before the content of the file is modified.
Second, the case where the requested file is not present on the file
system at the creation time of the ROM session. Here, the ram_fs missed
to inform listeners for the compound directory about the later created
file.
The former use of 'arm_v7' as architecture was too unspecific. Depot
binary archives for Cortex-A8/A9 platforms should be generated for
the 'arm_v7a' architecture to define the required compile flags
(i.e., -march).
When building with multiple jobs, the 'Makefile' rule may be executed
before any of its sibling rules, which implicitly create the target
directory. In this case, the attempt to create the symlink for
'Makefile' fails and the 'build_bin_archive' aborts. Analogously to the
sibling rules, this fix creates the target directory as a side effect of
the 'Makefile' rule.
This patch makes the build step of the create tool conditional. If
merely creating api, src, pkg, or raw archives, the invocation of the
'build' tool can be skipped. Otherwise, the heap message for the 'build'
tool is displayed (because it is called w/o any arguments), which is
misleading to the user.
The depot tool did not consistently distinct libraries from regular
targets when handling binary archives. The binary archive of a regular
target is located at <user>/<bin>/<arch>/<name> whereas a library is
located at <user>/<bin>/<arch>/<api>/<name>.
Fixes#2438
This patch replaces the toy downloader (that merely copied files
locally, for testing) with the mechanics needed to download files from a
www server.
It also changes the use of GPG to use detached signatures.
Issue #2339
We do not have debug symbols in the bin/ binaries anymore. Thus, use the
debug/ binaries instead. We now have kernel specific binary names for
ld.lib.so and some others. Adapt, to this fact as well. For doing so
without unnecessary output, provide a new parameter "silent" at the
"kernel_specific_binary" procedure.
Ref #2398
Removes the following Fiasco.OC specific features:
* GDB extensions for Fiasco.OC
* i.MX53 support for Fiasco.OC
* Kernel debugger terminal driver
* Obsolete interface Native_pd
* Obsolete function of interface Native_cpu
Previously, the debugging version of the core image
(<build_dir>/var/run/<run_name>.core) was created by the run tool from
the common core object file
(<build_dir>/var/run/<run_name>/core-<kernel>.o). This object file,
however, was already stripped by the build system and thus didn't
contain debug info anymore. The build system also creates an unstripped
version in <build_dir>/debug/core-<kernel>.o, so we now use this one
for <build_dir>/var/run/<run_name>.core.
Ref #2254
This way, files copied from the depot are incorporated in addition to
the files explicitly specified as boot modules.
The patch also adds an automatic check for the validity of the XML
syntax of boot modules ending with '.config'.
Issue #2339
Run scripts can use the new 'import_from_depot' function to incorporate
archive content from the depot into a scenario. The function must be
called after the 'create_boot_directory' function and takes any number
of pkg, src, or raw archives as arguments. An archive is specified as
depot-relative path of the form <user>/<type>/name. Run scripts may
call 'import_from_depot' repeatedly.
An argument can refer to a specific version of an archive or just the
version-less archive name. In the latter case, the current version (as
defined by a corresponding archive recipe in the source tree) is used.
If a 'src' archive is specified, the run tool integrates the content of
the corrsponding binary archive into the scenario. The binary archives
are selected according the spec values as defined for the build directory.
As of now, only x86_32 and x86_64 are supported by the 'depot_spec'
function.
Issue #2339
The <build-dir>/bin/ directory used to contain symbolic links to the
unstripped build results. However, since the upcoming depot tool
extracts the content of binary archives from bin/, the resulting
archives would contain overly large unstripped binaries, which is
undesired. On the other hand, always stripping the build results is not
a good option either because we rely of symbol information during
debugging.
This patch changes the installation of build results such that a new
'debug/' directory is populated besides the existing 'bin/' directory.
The debug directory contains symbolic links to the unstripped build
results whereas the bin directory contains stripped binaries that are
palatable for packaging (depot tool) and for assembling boot images (run
tool).
By installing the core object to bin/, we follow the same convention as
for regular binaries. This, in turn, enables us to ship core in a
regular binary archive. The patch also adjusts the run tool to pick up
the core object from bin/ for the final linking stage.
'unzip' can uncompress and load raw multiboot images and ELF files.
Usage together with the PXE bootloader ease life running the muen hypervisor
together with a Genode/x86_64 VM, where the raw boot image is typically quite
large.
Ref #2358
The information log file contains a list of all log files that are
expected to exist after the autopilot has finished its execution.
By using the '-i' option the default filename of 'autopilot.log'
can be overriden.
Debian has dropped mkisofs long time ago, hence hardcoding it prevents
Genode from being built on Debian easily. This patch makes the tool
configurable (defaulting to mkisofs). Building on Debian can then be
done by something like:
ISOTOOL=genisoimage make run/foobar
Fixes#2234Fixes#2235
An inline attribute mismatch in the gcc-4.9.2 source causes its
compilation to fail on modern gcc versions (verified with gcc 6.3.1,
but probably since gcc 5)
The patch is courtesy of the gcc-patches mailing list:
https://gcc.gnu.org/ml/gcc-patches/2015-08/msg00375.htmlFixes#2341
This enables to use append_if like the append command in the following
situation.
proc optional_binary {} {...}
append_if $optional config {
<start name="} [optional_binary] {">
...
}
We call curl a second time if the first check fails. This gives download
sites time to reconsider their response and helps, for example, to check
the qemu-usb port.
The input_filter is the successor of the input_merger. In addition to
merging input streams, the component applies several forms of input
transformations such as the application of keyboard layouts.
Issue #2264
Put the initialization of the cpu cores, setup of page-tables, enabling of
MMU and caches into a separate component that is only used to bootstrap
the kernel resp. core.
Ref #2092
Instead of implicitly asking for core's link address when linking
core within the run tool, deliver it explicitly to the build_core
routine. Thereby we gain the freedom to use the build_core tool for
different targets like core, and bootstrap.
Ref #2092
This test reproduces an issue of the VFS block file system when the
underlying block device has a coarser granularity than the block
requests issued by the VFS client. I.e., if the underlying block device
has a block size of 4K, writing a sequence of (non-4K-aligned) 512 blocks
that crosss a 4K boundary corrupts the data on the block device.
Issue #2262
Parse ``<env key="..." value=".."/>`` nodes from the config ROM and
populate a list at the 'genode_envp' and 'environ' symbols.
Test script at run/libc_getenv.
Fix#2236
In combination with run/image/uboot, run/boot_dir/foc expected a file link it
created itself to be a directory by trying to create another file link inside
it.
Ref #1987
This commit mostly removes the globally visible NR_OF_CPUS define
from the global makefile specifiers defined in the base-hw repository.
Whereever necessary it adds platform specific makefiles to the base
repository when they were missing.
Ref #2190
This patch make the ABI mechanism available to shared libraries other
than Genode's dynamic linker. It thereby allows us to introduce
intermediate ABIs at the granularity of shared libraries. This is useful
for slow-moving ABIs such as the libc's interface but it will also
become handy for the package management.
To implement the feature, the build system had to be streamlined a bit.
In particular, archive dependencies and shared-lib dependencies are now
handled separately, and the global list of 'SHARED_LIBS' is no more.
Now, the variable with the same name holds the per-target list of shared
libraries used by the target.
This commit adds support for testing multiple kernels within the same
build directory. In addition to the existing -p arguments, the new
version expects one or more -k arguments that denote the kernels to be
used for executing the specified run scripts.
Consequently, the autopilot executes the 3-dimensional matrix of
platforms x kernels x run scripts, e.g.,
autopilot --force -p x86_32 -k nova -k okl4 -k sel4 -k linux -r log
Issue #2190
This patch makes the benefit of the recently introduced unified Genode
ABI available to developers by enabling the use of multiple kernels from
within a single build directory. The create_builddir tool has gained a
new set of kernel-agnostic platform arguments such as x86_32, or panda.
Most build targets within directories are in principle compatible with
all kernels that support the selected hardware platform. To execute a
scenario via the run tool, one has to select the kernel to use by
setting the 'KERNEL' argument in the build configuration
(etc/build.conf). Alternatively, the 'KERNEL' can be specified as
command-line argument of the Genode build system, e.g.:
make run/log KERNEL=nova
This allows us to easily switch from one kernel to another without
rebuilding any Genode component except for the very few kernel-specific
ones.
The new version of the 'create_builddir' tool is still compatible with
the old version. The old kernel-specific build directories can still be
created. However, those variants will eventually be removed.
Note that the commit removes the 'ports-foc' repository from the
generated 'build.conf' files. As this is only meaningful for 'foc',
I did not want to include it in the list of regular repositories (as
visible in a 'x86_32' build directory). Hence, the repository must
now be manually added in order to use L4Linux.
Issue #2190
This patch removes possible ambiguities with respect to the naming of
kernel-dependent binaries and libraries. It also removes the use of
kernel-specific global side effects from the build system. The reach of
kernel-specific peculiarities has thereby become limited to the actual
users of the respective 'syscall-<kernel>' libraries.
Kernel-specific build artifacts are no longer generated at magic places
within the build directory (like okl4's includes, or the L4 build
directories of L4/Fiasco and Fiasco.OC, or the build directories of
various kernels). Instead, such artifacts have been largely moved to the
libcache. E.g., the former '<build-dir>/l4/' build directory for the L4
build system resides at '<build-dir>/var/libcache/syscall-foc/build/'.
This way, the location is unique to the kernel. Note that various tools
are still generated somewhat arbitrarily under '<build-dir>/tool/' as
there is no proper formalism for building host tools yet.
As the result of this work, it has become possible to use a joint Genode
build directory that is usable with all kernels of a given hardware
platform. E.g., on x86_32, one can now seamlessly switch between linux,
nova, sel4, okl4, fiasco, foc, and pistachio without rebuilding any
components except for core, the kernel, the dynamic linker, and the timer
driver. At the current stage, such a build directory must still be
created manually. A change of the 'create_builddir' tool will follow to
make this feature easily available.
This patch also simplifies various 'run/boot_dir' plugins by removing
the option for an externally hosted kernel. This option remained unused
for many years now.
Issue #2190
This patch decouples the kernel-specific implementation of the dynamic
linker from its kernel-agnostic binary interface. The name of the
kernel-specific dynamic linker binary now corresponds to the kernel,
e.g., 'ld-linux.lib.so' or 'ld-nova.lib.so'. Applications are no longer
linked directly against a concrete instance of the dynamic linker but
against a shallow stub called 'ld.lib.so'. This stub contains nothing
but the symbols provided by the dynamic linker. It thereby represents
the Genode ABI.
At system-integration time, the kernel-specific run/boot_dir back ends
integrate the matching the kernel-specific variant of the dynamic linker
as 'ld.lib.so' into the boot image.
The ABI symbol file for the dynamic linker is located at
'base/lib/symbols/ld'. It contains the joint ABI of all supported
architectures. The new utility 'tool/abi_symbols' eases the creation of
such an ABI symbol file for a given shared library. Its result should be
manually inspected and edited as needed.
The patch removes the 'syscall' library from 'base_libs.mk' to avoid
polluting the kernel-agnostic ABI with kernel-specific interfaces.
Issue #2190
Issue #2195
Prepared for internal test machine, which has 8 logical cores.
6 Win7 64bit VMs are started, each having 2 vCPUs, using the same image and
different overlays. Changes to the VM are written to the overlays of ram_fs
and dropped after the test.
lCPU 0 : Genode base system and drivers
lCPU 1-2: VM1 2 vCPUs
lCPU 2-3: VM2 "
lCPU 3-4: VM3 "
lCPU 4-5: VM4 "
lCPU 5-6: VM5 "
lCPU 6-7: VM6 "
Fixes#2143
Instead of solving the problem to deliver ROM modules to core while booting
differently for the several kernels (multi-boot, elfweaver, core re-linking),
this commit unifies the approaches. It always builds core as a library, and
after all binaries are built from a run-script, the run-tool will link an
ELF image out of the core-library and all boot modules. Thereby, core can
access its ROM modules directly.
This approach now works for all kernels except Linux.
With this solution, there is no [build_dir]/bin/core binary available anymore.
For debugging purposes you will find a core binary without boot modules, but
with debug symbols under [run_dir].core.
Fix#2095
base generic code:
* Remove unused verbosity code from mmio framework
* Remove escape sequence end heuristic from LOG
* replace Core_console with Core_log (no format specifiers)
* move test/printf to test/log
* remove `printf()` tests from the log test
* check for exact match of the log test output
base-fiasco:
* remove unused Fiasco::print_l4_threadid function
base-nova:
* remove unused hexdump utility from core
base-hw:
* remove unused Kernel::Thread::_print_* debug utilities
* always print resource summary of core during startup
* remove Kernel::Ipc_node::pd_label (not used anymore)
base*:
* Turn `printf`,`PWRN`, etc. calls into their log equivalents
Ref #1987Fix#2119
* Remove 'test' routine from kernel/core
* Move 'cpu_scheduler' and 'double_list' test to user-land
* Remove 'hw_info' target at all (can be recycled in a topic branch)
When run with the '--autopilot' run option, the 'usb_hid.run' script tests
the input events generated by a 'Pro Micro' microcontroller board. Setup
instructions for the Pro Micro can be found in the run script.
Fixes#2087
A run script to demonstrate the combination of the cpu_sampler, fs_log,
ram_fs and noux components for saving the sampled data in files and
accessing these in a Noux environment. When the script is run by the
autopilot, it will check if one of the generated sample data files
contains the correct instruction pointer. When the script is run
interactively, the generated files can be accessed from a Noux Bash shell.
Fixes#2078
The '--autopilot' option makes it possible for a run script to detect if
it is being executed by the autopilot by
if { [get_cmd_switch --autopilot] } { ... }
In that case, the run script could do an automated test, whereas the
test could be interactive when run without this option.
Fixes#2076
The new parameter specifies the additional timeout duration in seconds
which is incurred by AMT log processing, e.g. time spent waiting for the
system to boot.
Use a seperate handle at each session.
Use SEEK_TAIL to append messages to files.
Increase packet buffer.
Refactor to component framework.
Fixes#1777
Issue #2060
This patch revives our ds_ownership test from 2012, which just revealed
a regression in core where the dataspace-free operation of the RAM
service would unconditionally destroy dataspace objects from foreign
sessions. The patch fixes the bug and adds an updated version of the
test to the autopilot.
Fixes#2065
The main feature for this version upgrade is the use of the instruction
emulator (IEM) to speed up execution and less often the slow recompiler.
issue #2059
- disable iommu
- increase root_cnode further for native boot
- support vesa driver on native hardware
- don't mask edge triggered ioapic irqs
- increase various allocators to get noux_tool_chain_* booting natively
Issue #2044
- adjust syscall bindings to support -fPIC
- read serial i/o ports from BIOS data area
- use autoconf.h provided by sel4
-- to avoid ambiguity between sel4 kernel and user libraries
-- remove manual set defines
- remove debug messages
- increase user virtual area to 3GB
Issue #1720
Issue #2044
When running the same kernel in a VM as on the host system and the
kernel boot message from the VM appears on the log output, the run tool
assumes that the host machine has rebooted unexpectedly. With this
commit, an unexpected reboot is assumed only if the kernel boot message
appears at the beginning of a line. On base-hw, we enforce a line feed
at the beginning of the boot message as the SPIKE emulator log starts
with the first message of the kernel lacking a line feed.
Fixes#2041
This patch equips the build system with the feature of building an
individual library with its dependencies by specifying the library
as 'LIB' argument. E.g., 'make LIB=libc' builds the libc.
This is a generalisation approach of the hw_zynq target. As the boards
typically use UART1 instead of UART0 (used by qemu), we have to
distinguish between those. Moreover, in general hw_zynq does not imply
zynq_qemu anymore, so that the support of particular boards can be
placed in third-party or community repositories (e.g. Genode world).
Fixes#1926
If ports need preparation in order to execute a run script, use the
knowledge about the actual genode directory to suggest a command to the
user that can be directly copied to the command line.
Previously, ports that were needed for a scenario and that were not
prepared or outdated, triggered one assertion each during the second
build stage. The commit slots a mechanism in ahead that gathers all
these ports during the first build stage and reports them in form of a
list before the second build stage is entered. This list can be used
directly as argument for tool/ports/prepare_port to prepare respectively
update the ports. If, however, this mechanism is not available, for
example because a target is build without the first build stage, the old
assertion still prevents the target from running into troubles with a
missing port.
Fixes#1872
To raise readability when preparing multiple ports in parallel we prefix
also the git clone output with the port name dark-yellow-coloured. To
achieve this we sed the git output. In sed \x1b[ resolves to an escape
sequence and \033[, that we use normally, doesn't. The echo command, at
the other hand, resolves both to an escape sequence. Thus we use the
sed-compatible version in general. This commit inhibits the progress
output of git clone as it can't be redirected to sed.
Ref #1872
The tool/prepare_port tool is now able to handle a list of ports that
shall be prepared. Additionally, one may state the number of ports that
shall be prepared in parallel at a max by using the -j parameter. If -j
is not set by the user, the tool acts as with -j1. The previous
implementation of the tool that prepares only a single port was moved to
tool/ports/mk/prepare_single_port.mk and acts as back end to the new
prepare_port tool. The interface of the new prepare_port tool is
backwards compatible. When called for one port only, the behavior is the
same as when calling tool/ports/mk/prepare_single_port.mk directly.
Removes "usage" rule from prepare_single_port.mk. Removes shebang line
from prepare_single_port.mk.
Ref #1872
Enable the ACPI functionality in the platform_drv on hw_x86_64_muen and
provide a simple generated XML report as ROM session in order to make
the PCI configuration space available.
This is a requirement to implement support for MSI on hw_x86_64_muen.
Enable automatic build of Muen system image by adding the image/muen run
script to RUN_OPT. Also specify --image-muen parameters with explanatory
comments.
Note: The ISO image is not built by default anymore.
The script takes the following RUN_OPT parameters:
--image-muen-external-build Muen system is built automatically or externally
--image-muen-system Muen system policy
--image-muen-components Muen system components
--image-muen-hardware Muen hardware platform
--image-muen-gnat-path Path to GNAT toolchain
--image-muen-spark-path Path to SPARK toolchain
This enables installation of the bootloader image without wiping the
partition table which is needed at least for the tz_vmm tutorial with
hw_usb_armory.
Ref #1497
