This patch make the ABI mechanism available to shared libraries other
than Genode's dynamic linker. It thereby allows us to introduce
intermediate ABIs at the granularity of shared libraries. This is useful
for slow-moving ABIs such as the libc's interface but it will also
become handy for the package management.
To implement the feature, the build system had to be streamlined a bit.
In particular, archive dependencies and shared-lib dependencies are now
handled separately, and the global list of 'SHARED_LIBS' is no more.
Now, the variable with the same name holds the per-target list of shared
libraries used by the target.
This patch makes the benefit of the recently introduced unified Genode
ABI available to developers by enabling the use of multiple kernels from
within a single build directory. The create_builddir tool has gained a
new set of kernel-agnostic platform arguments such as x86_32, or panda.
Most build targets within directories are in principle compatible with
all kernels that support the selected hardware platform. To execute a
scenario via the run tool, one has to select the kernel to use by
setting the 'KERNEL' argument in the build configuration
(etc/build.conf). Alternatively, the 'KERNEL' can be specified as
command-line argument of the Genode build system, e.g.:
make run/log KERNEL=nova
This allows us to easily switch from one kernel to another without
rebuilding any Genode component except for the very few kernel-specific
ones.
The new version of the 'create_builddir' tool is still compatible with
the old version. The old kernel-specific build directories can still be
created. However, those variants will eventually be removed.
Note that the commit removes the 'ports-foc' repository from the
generated 'build.conf' files. As this is only meaningful for 'foc',
I did not want to include it in the list of regular repositories (as
visible in a 'x86_32' build directory). Hence, the repository must
now be manually added in order to use L4Linux.
Issue #2190
This patch removes possible ambiguities with respect to the naming of
kernel-dependent binaries and libraries. It also removes the use of
kernel-specific global side effects from the build system. The reach of
kernel-specific peculiarities has thereby become limited to the actual
users of the respective 'syscall-<kernel>' libraries.
Kernel-specific build artifacts are no longer generated at magic places
within the build directory (like okl4's includes, or the L4 build
directories of L4/Fiasco and Fiasco.OC, or the build directories of
various kernels). Instead, such artifacts have been largely moved to the
libcache. E.g., the former '<build-dir>/l4/' build directory for the L4
build system resides at '<build-dir>/var/libcache/syscall-foc/build/'.
This way, the location is unique to the kernel. Note that various tools
are still generated somewhat arbitrarily under '<build-dir>/tool/' as
there is no proper formalism for building host tools yet.
As the result of this work, it has become possible to use a joint Genode
build directory that is usable with all kernels of a given hardware
platform. E.g., on x86_32, one can now seamlessly switch between linux,
nova, sel4, okl4, fiasco, foc, and pistachio without rebuilding any
components except for core, the kernel, the dynamic linker, and the timer
driver. At the current stage, such a build directory must still be
created manually. A change of the 'create_builddir' tool will follow to
make this feature easily available.
This patch also simplifies various 'run/boot_dir' plugins by removing
the option for an externally hosted kernel. This option remained unused
for many years now.
Issue #2190
This patch decouples the kernel-specific implementation of the dynamic
linker from its kernel-agnostic binary interface. The name of the
kernel-specific dynamic linker binary now corresponds to the kernel,
e.g., 'ld-linux.lib.so' or 'ld-nova.lib.so'. Applications are no longer
linked directly against a concrete instance of the dynamic linker but
against a shallow stub called 'ld.lib.so'. This stub contains nothing
but the symbols provided by the dynamic linker. It thereby represents
the Genode ABI.
At system-integration time, the kernel-specific run/boot_dir back ends
integrate the matching the kernel-specific variant of the dynamic linker
as 'ld.lib.so' into the boot image.
The ABI symbol file for the dynamic linker is located at
'base/lib/symbols/ld'. It contains the joint ABI of all supported
architectures. The new utility 'tool/abi_symbols' eases the creation of
such an ABI symbol file for a given shared library. Its result should be
manually inspected and edited as needed.
The patch removes the 'syscall' library from 'base_libs.mk' to avoid
polluting the kernel-agnostic ABI with kernel-specific interfaces.
Issue #2190
Issue #2195
Replace 'dump()' debug utilities within Allocator_avl with Output::print
equivalents, and use the new Avl_tree::for_each utility to simplify
the implementation.
Ref #2159
Instead of solving the problem to deliver ROM modules to core while booting
differently for the several kernels (multi-boot, elfweaver, core re-linking),
this commit unifies the approaches. It always builds core as a library, and
after all binaries are built from a run-script, the run-tool will link an
ELF image out of the core-library and all boot modules. Thereby, core can
access its ROM modules directly.
This approach now works for all kernels except Linux.
With this solution, there is no [build_dir]/bin/core binary available anymore.
For debugging purposes you will find a core binary without boot modules, but
with debug symbols under [run_dir].core.
Fix#2095
base generic code:
* Remove unused verbosity code from mmio framework
* Remove escape sequence end heuristic from LOG
* replace Core_console with Core_log (no format specifiers)
* move test/printf to test/log
* remove `printf()` tests from the log test
* check for exact match of the log test output
base-fiasco:
* remove unused Fiasco::print_l4_threadid function
base-nova:
* remove unused hexdump utility from core
base-hw:
* remove unused Kernel::Thread::_print_* debug utilities
* always print resource summary of core during startup
* remove Kernel::Ipc_node::pd_label (not used anymore)
base*:
* Turn `printf`,`PWRN`, etc. calls into their log equivalents
Ref #1987Fix#2119
Besides adapting the components to the use of base/log.h, the patch
cleans up a few base headers, i.e., it removes unused includes from
root/component.h, specifically base/heap.h and
ram_session/ram_session.h. Hence, components that relied on the implicit
inclusion of those headers have to manually include those headers now.
While adjusting the log messages, I repeatedly stumbled over the problem
that printing char * arguments is ambiguous. It is unclear whether to
print the argument as pointer or null-terminated string. To overcome
this problem, the patch introduces a new type 'Cstring' that allows the
caller to express that the argument should be handled as null-terminated
string. As a nice side effect, with this type in place, the optional len
argument of the 'String' class could be removed. Instead of supplying a
pair of (char const *, size_t), the constructor accepts a 'Cstring'.
This, in turn, clears the way let the 'String' constructor use the new
output mechanism to assemble a string from multiple arguments (and
thereby getting rid of snprintf within Genode in the near future).
To enforce the explicit resolution of the char * ambiguity, the 'char *'
overload of the 'print' function is marked as deleted.
Issue #1987
Required to boot on hardware. The fragmentation is such unfortunate with the
old allocator that alloc() will throw an exception during very early core boot
phase.
Issue #2044
- disable iommu
- increase root_cnode further for native boot
- support vesa driver on native hardware
- don't mask edge triggered ioapic irqs
- increase various allocators to get noux_tool_chain_* booting natively
Issue #2044
That seems nowadays the right place in order to tell the caller that the
thread couldn't be completely constructed. The return value false of
bind_thread causes in Cpu_thread_component the throwing of
Thread_creation_failed.
thread.run now passes
pthread.run now passes
Issue #2044
Additionally, exclude boot-module from page_table_registry. Core does not use
the boot-modules inside core.
Adding it otherwise to the registry will use up the meta data allocator of
the page_table_registry in core in scenarios with lot of boot modules,
e.g. noux and friends.
Issue #2044
- adjust syscall bindings to support -fPIC
- read serial i/o ports from BIOS data area
- use autoconf.h provided by sel4
-- to avoid ambiguity between sel4 kernel and user libraries
-- remove manual set defines
- remove debug messages
- increase user virtual area to 3GB
Issue #1720
Issue #2044
This patch establishes the sole use of generic headers across all
kernels. The common 'native_capability.h' is based on the version of
base-sel4. All traditional L4 kernels and Linux use the same
implementation of the capability-lifetime management. On base-hw, NOVA,
Fiasco.OC, and seL4, custom implementations (based on their original
mechanisms) are used, with the potential to unify them further in the
future.
This change achieves binary compatibility of dynamically linked programs
across all kernels.
Furthermore, the patch introduces a Native_capability::print method,
which allows the easy output of the kernel-specific capability
representation using the base/log.h API.
Issue #1993
This patch alleviates the need for a Native_capability::Dst at the API
level. The former use case of this type as argument to
Deprecated_env::reinit uses the opaque Native_capability::Raw type
instead. The 'Raw' type contains the portion of the capability that is
transferred as-is when delegating the capability (i.e., when installing
the parent capability into a new component, or when installing a new
parent capability into a new forked Noux process). This information can
be retrieved via the new Native_capability::raw method.
Furthermore, this patch moves the functions for retriving the parent
capability to base/internal/parent_cap.h, which is meant to be
implemented in platform-specific ways. It replaces the former set of
startup/internal/_main_parent_cap.h headers.
Issue #1993
This patch introduces the Genode::raw function that prints output
directly via a low-level kernel mechanism, if available.
On base-linux, it replaces the former 'raw_write_str' function.
On base-hw, it replaces the former kernel/log.h interface.
Fixes#2012
This is a follow-up commit to "slab: throw exception if initial
allocation fails". It is needed because on seL4, the PD session quota is
used as backing store for the meta data of the PD's virtual memory. For
this reason, PD sessions on seL4 require more session quota compared to
the other kernels. Apparently, the default quota is too scarce to
complete the PD-session construction. By shrinking the number of
supported page-table entries, we reduce the size of the PD session
component so that the creation succeeds.
Issue #1983
This patch moves the thread operations from the 'Cpu_session'
to the 'Cpu_thread' interface.
A noteworthy semantic change is the meaning of the former
'exception_handler' function, which used to define both, the default
exception handler or a thread-specific signal handler. Now, the
'Cpu_session::exception_sigh' function defines the CPU-session-wide
default handler whereas the 'Cpu_thread::exception_sigh' function
defines the thread-specific one.
To retain the ability to create 'Child' objects without invoking a
capability, the child's initial thread must be created outside the
'Child::Process'. It is now represented by the 'Child::Initial_thread',
which is passed as argument to the 'Child' constructor.
Fixes#1939
This patch cleans up the thread API and comes with the following
noteworthy changes:
- Introduced Cpu_session::Weight type that replaces a formerly used
plain integer value to prevent the accidental mix-up of
arguments.
- The enum definition of Cpu_session::DEFAULT_WEIGHT moved to
Cpu_session::Weight::DEFAULT_WEIGHT
- New Thread constructor that takes a 'Env &' as first argument.
The original constructors are now marked as deprecated. For the
common use case where the default 'Weight' and 'Affinity' are
used, a shortcut is provided. In the long term, those two
constructors should be the only ones to remain.
- The former 'Thread<>' class template has been renamed to
'Thread_deprecated'.
- The former 'Thread_base' class is now called 'Thread'.
- The new 'name()' accessor returns the thread's name as 'Name'
object as centrally defined via 'Cpu_session::Name'. It is meant to
replace the old-fashioned 'name' method that takes a buffer and size
as arguments.
- Adaptation of the thread test to the new API
Issue #1954
This patch moves the base library from src/base to src/lib/base,
flattens the library-internal directory structure, and moves the common
parts of the library-description files to base/lib/mk/base.inc and
base/lib/mk/base-common.inc.
Furthermore, the patch fixes a few cosmetic issues (whitespace and
comments only) that I encountered while browsing the result.
Fixes#1952
This patch makes the former 'Process' class private to the 'Child'
class and changes the constructor of the 'Child' in a way that
principally enables the implementation of single-threaded runtime
environments that virtualize the CPU, PD, and RAM services. The
new interfaces has become free from side effects. I.e., instead
of implicitly using Genode::env()->rm_session(), it takes the reference
to the local region map as argument. Also, the handling of the dynamic
linker via global variables is gone. Now, the linker binary must be
provided as constructor argument.
Fixes#1949
This patch replaces the former 'Pd_session::bind_thread' function by a
PD-capability argument of the 'Cpu_session::create_thread' function, and
removes the ancient thread-start protocol via 'Rm_session::add_client' and
'Cpu_session::set_pager'. Threads are now bound to PDs at their creation
time and implicitly paged according to the address space of the PD.
Note the API change:
This patch changes the signature of the 'Child' and 'Process' constructors.
There is a new 'address_space' argument, which represents the region map
representing the child's address space. It is supplied separately to the
PD session capability (which principally can be invoked to obtain the
PD's address space) to allow the population of the address space
without relying on an 'Pd_session::address_space' RPC call.
Furthermore, a new (optional) env_pd argument allows the explicit
overriding of the PD capability handed out to the child as part of its
environment. It can be used to intercept the interaction of the child
with its PD session at core. This is used by Noux.
Issue #1938
This patch integrates three region maps into each PD session to
reduce the session overhead and to simplify the PD creation procedure.
Please refer to the issue cited below for an elaborative discussion.
Note the API change:
With this patch, the semantics of core's RM service have changed. Now,
the service is merely a tool for creating and destroying managed
dataspaces, which are rarely needed. Regular components no longer need a
RM session. For this reason, the corresponding argument for the
'Process' and 'Child' constructors has been removed.
The former interface of the 'Rm_session' is not named 'Region_map'. As a
minor refinement, the 'Fault_type' enum values are now part of the
'Region_map::State' struct.
Issue #1938
The return code of assign_parent remained unused. So this patch
removes it.
The bind_thread function fails only due to platform-specific limitations
such as the exhaustion of ID name spaces, which cannot be sensibly
handled by the PD-session client. If occurred, such conditions used to
be reflected by integer return codes that were used for diagnostic
messages only. The patch removes the return codes and leaves the
diagnostic output to core.
Fixes#1842
Besides unifying the Msgbuf_base classes across all platforms, this
patch merges the Ipc_marshaller functionality into Msgbuf_base, which
leads to several further simplifications. For example, this patch
eventually moves the Native_connection_state and removes all state
from the former Ipc_server to the actual server loop, which not only
makes the flow of control and information much more obvious, but is
also more flexible. I.e., on NOVA, we don't even have the notion of
reply-and-wait. Now, we are no longer forced to pretend otherwise.
Issue #1832
This patch unifies the CPU session interface across all platforms. The
former differences are moved to respective "native-CPU" interfaces.
NOVA is not covered by the patch and still relies on a custom version of
the core-internal 'cpu_session_component.h'. However, this will soon be
removed once the ongoing rework of pause/single-step on NOVA is
completed.
Fixes#1922
This patch removes the dynamically growing slab allocator from the
page-table registry. This has two benefits. First, we alleviate the
corner cases where the slab allocator needed to extend its backing store
while establishing a core-local memory mapping, thereby triggering a
nested core-local mapping. Without this corner case, no reentrant lock
is needed any longer. Second, it removes the dependency from the overly
large old API of the slab allocator. So we can tighten the slab
interface.
This commit introduces the new `Component` interface in the form of the
headers base/component.h and base/entrypoint.h. The os/server.h API
has become merely a compatibilty wrapper and will eventually be removed.
The same holds true for os/signal_rpc_dispatcher.h. The mechanism has
moved to base/signal.h and is now called 'Signal_handler'.
Since the patch shuffles headers around, please do a 'make clean' in the
build directory.
Issue #1832
This commit replaces the stateful 'Ipc_client' type with the plain
function 'ipc_call' that takes all the needed state as arguments.
The stateful 'Ipc_server' class is retained but it moved from the public
API to the internal ipc_server.h header. The kernel-specific
implementations were cleaned up and simplified. E.g., the 'wait'
function does no longer exist. The badge and exception code are no
longer carried in the message buffers but are handled in kernel-specific
ways.
Issue #610
Issue #1832
This patch moves details about the stack allocation and organization
the base-internal headers. Thereby, I replaced the notion of "thread
contexts" by "stacks" as this term is much more intuitive. The fact that
we place thread-specific information at the bottom of the stack is not
worth introducing new terminology.
Issue #1832
On seL4 and L4/Fiasco, we employ a simple yielding spinlock as lock
implementation. Consequently these base platforms used to have a
simplified header. However, since the regular cancelable_lock has all
the member variables needed to implement a spinlock, we can simply use
the generic header on those two platforms too, just leaving some other
parts of the generic header unused. So at API level, the difference is
not visible.
Issue #1832
By moving the stub implementation to rm_session_client.cc, we can use
the generic base/include/rm_session/client.h for base-linux and
base-nova and merely use platform-specific implementations.
Issue #1832
This patch establishes a common organization of header files
internal to the base framework. The internal headers are located at
'<repository>/src/include/base/internal/'. This structure has been
choosen to make the nature of those headers immediately clear when
included:
#include <base/internal/lock_helper.h>
Issue #1832
This patch integrates the functionality of the former CAP session into
the PD session and unifies the approch of supplementing the generic PD
session with kernel-specific functionality. The latter is achieved by
the new 'Native_pd' interface. The kernel-specific interface can be
obtained via the Pd_session::native_pd accessor function. The
kernel-specific interfaces are named Nova_native_pd, Foc_native_pd, and
Linux_native_pd.
The latter change allowed for to deduplication of the
pd_session_component code among the various base platforms.
To retain API compatibility, we keep the 'Cap_session' and
'Cap_connection' around. But those classes have become mere wrappers
around the PD session interface.
Issue #1841
This patch removes the SIGNAL service from core and moves its
functionality to the PD session. Furthermore, it unifies the PD service
implementation and terminology across the various base platforms.
Issue #1841
This is the default optimization level in the original seL4 SDK. By
adapting to O3, we work around a bug [1] in version 2.1.0 that only
shows on low optimization levels.
[1] https://github.com/seL4/seL4/issues/20
Previously, ports that were needed for a scenario and that were not
prepared or outdated, triggered one assertion each during the second
build stage. The commit slots a mechanism in ahead that gathers all
these ports during the first build stage and reports them in form of a
list before the second build stage is entered. This list can be used
directly as argument for tool/ports/prepare_port to prepare respectively
update the ports. If, however, this mechanism is not available, for
example because a target is build without the first build stage, the old
assertion still prevents the target from running into troubles with a
missing port.
Fixes#1872
This patch updates seL4 from the experimental branch of one year ago to
the master branch of version 2.1. The transition has the following
implications.
In contrast to the experimental branch, the master branch has no way to
manually define the allocation of kernel objects within untyped memory
ranges. Instead, the kernel maintains a built-in allocation policy. This
policy rules out the deallocation of once-used parts of untyped memory.
The only way to reuse memory is to revoke the entire untyped memory
range. Consequently, we cannot share a large untyped memory range for
kernel objects of different protection domains. In order to reuse memory
at a reasonably fine granularity, we need to split the initial untyped
memory ranges into small chunks that can be individually revoked. Those
chunks are called "untyped pages". An untyped page is a 4 KiB untyped
memory region.
The bootstrapping of core has to employ a two-stage allocation approach
now. For creating the initial kernel objects for core, which remain
static during the entire lifetime of the system, kernel objects are
created directly out of the initial untyped memory regions as reported
by the kernel. The so-called "initial untyped pool" keeps track of the
consumption of those untyped memory ranges by mimicking the kernel's
internal allocation policy. Kernel objects created this way can be of
any size. For example the phys CNode, which is used to store page-frame
capabilities is 16 MiB in size. Also, core's CSpace uses a relatively
large CNode.
After the initial setup phase, all remaining untyped memory is turned
into untyped pages. From this point on, new created kernel objects
cannot exceed 4 KiB in size because one kernel object cannot span
multiple untyped memory regions. The capability selectors for untyped
pages are organized similarly to those of page-frame capabilities. There
is a new 2nd-level CNode (UNTYPED_CORE_CNODE) that is dimensioned
according to the maximum amount of physical memory (1M entries, each
entry representing 4 KiB). The CNode is organized such that an index
into the CNode directly corresponds to the physical frame number of the
underlying memory. This way, we can easily determine a untyped page
selector for any physical addresses, i.e., for revoking the kernel
objects allocated at a specific physical page. The downside is the need
for another 16 MiB chunk of meta data. Also, we need to keep in mind
that this approach won't scale to 64-bit systems. We will eventually
need to replace the PHYS_CORE_CNODE and UNTYPED_CORE_CNODE by CNode
hierarchies to model a sparsely populated CNode.
The size constrain of kernel objects has the immediate implication that
the VM CSpaces of protection domains must be organized via several
levels of CNodes. I.e., as the top-level CNode of core has a size of
2^12, the remaining 20 PD-specific CSpace address bits are organized as
a 2nd-level 2^4 padding CNode, a 3rd-level 2^8 CNode, and several
4th-level 2^8 leaf CNodes. The latter contain the actual selectors for
the page tables and page-table entries of the respective PD.
As another slight difference from the experimental branch, the master
branch requires the explicit assignment of page directories to an ASID
pool.
Besides the adjustment to the new seL4 version, the patch introduces a
dedicated type for capability selectors. Previously, we just used to
represent them as unsigned integer values, which became increasingly
confusing. The new type 'Cap_sel' is a PD-local capability selector. The
type 'Cnode_index' is an index into a CNode (which is not generally not
the entire CSpace of the PD).
Fixes#1887
* Move the Synced_interface from os -> base
* Align the naming of "synchronized" helpers to "Synced_*"
* Move Synced_range_allocator to core's private headers
* Remove the raw() and lock() members from Synced_allocator and
Synced_range_allocator, and re-use the Synced_interface for them
* Make core's Mapped_mem_allocator a friend class of Synced_range_allocator
to enable the needed "unsafe" access of its physical and virtual allocators
Fix#1697
Instead of holding SPEC-variable dependent files and directories inline
within the repository structure, move them into 'spec' subdirectories
at the corresponding levels, e.g.:
repos/base/include/spec
repos/base/mk/spec
repos/base/lib/mk/spec
repos/base/src/core/spec
...
Moreover, this commit removes the 'platform' directories. That term was
used in an overloaded sense. All SPEC-relative 'platform' directories are
now named 'spec'. Other files, like for instance those related to the
kernel/architecture specific startup library, where moved from 'platform'
directories to explicit, more meaningful places like e.g.: 'src/lib/startup'.
Fix#1673
Instead of returning pointers to locked objects via a lookup function,
the new object pool implementation restricts object access to
functors resp. lambda expressions that are applied to the objects
within the pool itself.
Fix#884Fix#1658
For most platforms except of NOVA a distinction between pager entrypoint
and pager activation is not needed, and only exists due to historical
reasons. Moreover, the pager thread's execution path is almost identical
between most platforms excluding NOVA, HW, and Fisco.OC. Therefore,
this commit unifies the pager loop for the other platforms, and removes
the pager activation class.
This commit eliminates the mutual interlaced taking of destruction lock,
list lock and weak pointer locks that could lead to a dead-lock situation
when a lock pointer was tried to construct while a weak object is in
destruction progress.
Now, all weak pointers are invalidated and dequeued at the very
beginning of the weak object's destruction. Moreover, before a weak pointer
gets invalidated during destruction of a weak object, it gets dequeued, and
the list lock is freed again to avoid the former dead-lock.
Fix#1607
This patch enable clients of core's TRACE service to obtain the
execution times of trace subjects (i.e., threads). The execution time is
delivered as part of the 'Subject_info' structure.
Right now, the feature is available solely on NOVA. On all other base
platforms, the returned execution times are 0.
Issue #813
This patch installs the parent endpoint selector and the PD's CNode into
a PD at its creation time. Furthermore, it initializes the IPC buffer
for the main thread of the new component.
This allows us to see debug messages printed at the eary initialization
of init (before init is able to obtain the regular LOG session). This
will be reverted as soon as the initialziation of the non-core base
environment works.
To build core and other Genode components, we will need to extend the
base-common.mk library with additions that conflict with the
minimalistic root-task environment of test/sel4. To preserve the
minimalistic root task, we need to decouple it from the base-common
library.
