Commit Graph

2462 Commits

Author SHA1 Message Date
Norman Feske
b723b11b30 Modularize Sculpt OS image creation
This patch equips Sculpt with the ability to customize the system image
in very flexible ways.

All customizable aspects of the image have been relocated from the
former sculpt.run script and the accompanied gems/run/sculpt/ directory
to a new location - the sculpt/ directory - which can exist in any
repository. The directory at repos/gems/sculpt/ serves as reference.

The sculpt directory can host any number of <name>-<board>.sculpt files,
each containing a list of ingredients to be incorporated into the
Sculpt system image. The <name> can be specified to the sculpt.run
script. E.g., the following command refers to the 'default-pc.sculpt'
file:

  make run/sculpt KERNEL=nova BOARD=pc SCULPT=default

If no 'SCULPT' argument is supplied, the value 'default' is used.

A .sculpt file refers to a selection of files found at various
subdirectries named after their respective purpose. In particular, There
exists a subdirectory for each file in Sculpt's config fs, like
nitpicker, drivers... The .sculpt file selects the alternative to use
by a simple tag-value notation.

  drivers: pc

The supported tags are as follows.

*Optional* selection of /config files. If not specified, those files are
omitted, which prompts Sculpt to manage those configurations
automatically or via the Leitzentrale GUI:

  fonts
  nic_router
  event_filter
  wifi
  runtime
  gpu_drv

Selection of mandatory /config files. If not specified, the respective
'default' alternative will be used.

  nitpicker
  deploy
  fb_drv
  clipboard
  drivers
  numlock_remap
  leitzentrale
  usb
  system
  ram_fs

Furthermore, the .sculpt file supports the optional selection of
supplemental content such as a set of launchers.

  launches: nano3d system_shell

Another type of content are the set of blessed pubkey/download files
used for installing and verifying software on target.

With the new version, it has become possible to supply a depot with the
the system image. The depot content is assembled according to the 'pkg'
attributes found in launcher files and the selected deploy config.
The resulting depot is incorporated into the system image as 'depot.tar'
archive. It can be supplied to the Sculpt system by mounting it into the
ram fs as done by the 'ram_fs/depot' configuration for the ram fs.

It is possible to add additional boot modules to the system image. There
are two options.

  build: <list of targets>

This tag prompts the sculpt.run script to build the specified targets
directly using the Genode build system and add the created artifacts
into the system image as boot modules.

  import: <list of depot src or pkg archives>

This tag instructs Sculpt to supply the specifid depot-archive content
as boot modules to the system image. This change eliminates the need for
board-specific pkg/sculpt-<board> archives. The board-specific
specializations can now be placed directly into the respective .sculpt
files by using 'import:'.

To make the use of Sculpt as testbed during development more convenient,
the log output of the drivers, leitzentrale, and runtime subsystems
can be redirected to core using the optional 'LOG=core' argument, e.g.,

  make run/sculpt KERNEL=linux BOARD=linux LOG=core

The former pkg/sculpt-installation and pkg/sculpt-installation-pc
archives have been replaced by pkg/sculpt_distribution-pc, which
references the generic pkg/sculpt_distribution archive. Those pkgs are
solely used for publishing / distribution purposes.

Fixes #4369
2022-01-19 12:38:11 +01:00
Christian Prochaska
8e252f79f8 Remove mutex from 'Genode::Trace_output'
Fixes #4356
2022-01-19 12:35:49 +01:00
Alexander Boettcher
4ac553e5ff cpu_balancer.run: disable testing on foc
Due to some kernel issue the test will not finish, see #4357

Fixes #4357
2022-01-19 12:35:49 +01:00
Piotr Tworek
d610f9f4f1 Fix some signed/unsigned implicit conversions.
Fix some trivial cases where the signedness of the constant value does
not match the signedness of type the code expects to see. GCC can be
asked to warn about those by passing Wsign-covnersion flag.

Issue #4354
2022-01-19 12:35:48 +01:00
Piotr Tworek
f2dab083f6 os: Properly initialize File_system::Watch_handle.
This comes up when building the code with clang 13. It happens due to
recently enabled Wconversion warning, which in case of clang also
enables implicit-int-conversion warning. The warning reads:

  fs_file_system.h:937:44: error: higher order bits are zeroes after
  implicit conversion [-Werror,-Wimplicit-int-conversion]
                        ::File_system::Watch_handle fs_handle { -1U };
                                                    ~~~~~~~~~   ^~~

This can be fixed by properly specifying fs_handle value to be of
unsigned long type.

Issue #4354
2022-01-19 12:35:48 +01:00
Alexander Boettcher
507e5ac776 vmm_x86: fix sporadic test failures on x86_32 nova
Explicitly configure wanted exits instead of using the "random" primary
ctrl value.
2022-01-19 12:35:48 +01:00
Sebastian Sumpf
79e69a5549 nic_loopback: Adjust to packet allocator changes
Remove '_expected_offset' check on round trip test from RX packet,
because allocators between RX/TX can have different allocation
strategies. Rely on 'pattern' check for RX packets only.

In loopback server alloc size must match actual packet size.

issue #4312
2022-01-19 12:35:48 +01:00
Sebastian Sumpf
08c1e69d71 nic/packet_allocator: align allocations to 2 bytes
Override 'try_alloc/free' because ethernet frame headers are 14 bytes
(src/dst mac (12) + ethertype (2)) causing the IP header to be 2 byte
aligned, leading to problems on platforms that require load/store
operations to be naturally aligned when reading, for example, 4 byte IP
addresses. Therefore, we align the allocation to 2 bytes, so the IP
header is aligned to 4.

issue #4312
2022-01-19 12:35:48 +01:00
Norman Feske
5611fd2355 depot: update recipe hashes 2021-12-17 15:06:38 +01:00
Norman Feske
d658648a09 test/block_cache: tweak RAM quota for base-nova 2021-12-17 15:06:38 +01:00
Norman Feske
3cc6df3116 base: tighten affinity handling
This patch improves the robustness of the CPU-affinity handling.

- The types in base/affinity.h received the accessors
  'Location::within(space)' and 'Affinity::valid', which alleviates
  the fiddling with coordinates when sanity checking the values,
  in init or core.

- The 'Affinity::Location::valid' method got removed because its
  meaning was too vague. For sanity checks of affinity configurations,
  the new 'within' method is approriate. In cases where only the x,y
  values are used for selecting a physical CPU (during thread creation),
  the validity check (width*height > 0) was not meaningful anyway.

- The 'Affinity::Location::from_xml' requires a 'Affinity::Space'
  as argument because a location always relates to the bounds of
  a specific space. This function now implements the selection of
  whole rows or columns, which has previously a feature of the
  sandbox library only.

- Whenever the sandbox library (init) encounters an invalid affinity
  configuration, it prints a warning message as a diagnostic aid.

- A new 'Affinity::unrestricted' function constructs an affinity that
  covers the whole affinity space. The named functions clarifies
  the meaning over the previous use of the default constructor.

- Core's CPU service denies session requests with an invalid
  affinity parameter. Previously, it would fall back to an
  unrestricted affinity.

Issue #4300
2021-12-17 15:06:38 +01:00
Christian Helmuth
a80a22c819 recipes: don't declare implemented API in used_apis
Prevents "*** detected duplicates in REPOSITORIES" on recipe build.
2021-12-17 15:06:38 +01:00
Norman Feske
d370f56a77 Remove obsolete Trace::Session::subject_info RPC
Issue #3610
Fixes #4349
2021-12-17 15:04:49 +01:00
Stefan Kalkowski
6dbdb966fc virtio: remove bitfield in Index template
Eliminates note of GCC about bitfield as parameter semantic change.

Ref #4344
2021-12-17 15:04:48 +01:00
Piotr Tworek
1a9fad4022 virtio_nic: Tweak default buffer sizes.
After VirtIO::Queue refactoring buffers no longer share the same
dataspace as VirtIO rings. This makes optimal buffer calculations a lot
easier. In this case 64 buffers 2kB each will need precisely 128kB of
RAM. Previous value of 2016 will just waste 768b.

Fixed #4347
2021-12-17 15:04:48 +01:00
Piotr Tworek
643747b912 os: Refactor VirtIO::Queue implementation.
The key changes in this patch are:
* Buffer allocation is moved into a separate Buffer_pool helper. The
  implementation of the buffer allocation strategy does not change.
  The helper allocates a single RAM dataspace and splits it in multiple,
  equally sized chunks.
* Management of main descriptor ring is enacapsulated in Descriptor_ring
  helper class.
* Use separate RAM dataspaces for descriptor rings and buffers.
  Previously both of them were packed into a single dataspace. This
  might have been more RAM efficient, but IMO it made the code uglier and
  harder to understand.
* All of the VirtIO::Queue members are now initialized on the class member
  initializer list. This is possible due to previously listed changes.
* Since all VirtIO::Queue members are initalized on member initalizer
  list, some additional ones can be marked as const, ex _avail, _used ring
  pointers.
* Move descriptor writing code into a common method used by both
  write_data and write_data_read_reply members. This avoids some code
  duplication between those methods.
* Get rid of request_irq argument that most public VirtIO::Queue methods
  accept. None of the existing drivers use it and I doubt this will
  change any time soon.
* Use Genode namespace by default in Virtio.

This patch also fixes at least one bug that I discovered while working
on VirtIO block device driver. Namely, when chaining descriptors only the
first descriptor in the chain should be exposed in the available ring.

Issue #4347
2021-12-17 15:04:48 +01:00
Norman Feske
04cf6ea3ab os: avoid implicit conversions
Issue #23
2021-12-17 15:04:45 +01:00
Norman Feske
03047009b1 base: avoid implicit conversions
This patch is a prerequisite for compiling the code with
the warnings -Wconversion enabled.

Issue #23
2021-12-17 15:04:44 +01:00
Johannes Schlatow
b1bbd72e84 net: fix tail size calculation in Ethernet_frame
The const-variant of the data() method contained an erroneous
calculation of the tail size. This led to the size guard throwing
exceptions when trying to parse TCP packets that only contained the
TCP header.

Fixes genodelabs/genode#4340
2021-12-17 15:04:43 +01:00
Norman Feske
162ddb1fdb virtio: fix len calculation
Thanks to Piotr Tworek for the fix and his explanation as follows:

The basic idea is to try to fit payload data into the descriptor used to
send the header. If there is no payload, or the payload fits exactly
into the remaining space in the header decriptor, len should be 0 and
only one descriptor should be used. In such case the "next" and "flags"
members of the descriptor structure should be set to 0.

In case there is some extra payload data to send, but its size is
bigger than the remaining free space in the descriptor used to send the
header, len should contain the remaining size of the payload that
can't be sent via the header descriptor. The code will then chain
additional descriptors to handle this remainder.

With the len variable shadowing, the code will never queue the remaining
data.

Issue #4327
2021-12-17 15:04:43 +01:00
Stefan Kalkowski
6cc6047962 platform_drv: introduce clocks, power, reset
Share datastructures for clock, power and reset related configurations
per device. In the generic platform driver component these structures
are kept empty. Driver derivates can fill the clocks settings, power and
reset switches with life. The former Driver::Env gets removed.

Fix #4338
2021-12-17 15:04:39 +01:00
Stefan Kalkowski
f11d2b5b54 autopilot: automated stress test for platform_drv
Fix #4330
2021-12-17 14:56:53 +01:00
Stefan Kalkowski
132e4fe815 platform_drv(arm): robust re-configuration support
This change of the inner working of the platform driver for ARM allows
clients to have permanent open sessions, as long as a policy node matches
the client. If devices disappear from the policy resp. from the set of
available devices (hotplug), the devices ROM of the session gets updated,
and a corresponding device session gets closed. If the device remains
untouched in the configuration but other devices appeared/disappeared, the
device session is not affected.

Ref #4330
2021-12-17 14:56:46 +01:00
Christian Helmuth
92b7be4d11 depot: update recipe hashes 2021-11-30 12:08:54 +01:00
Stefan Kalkowski
14de84fae6 base: be more restrictive with 'managing_system'
* Only give managing_system permission when all parent nodes of the
  corresponding component agree in doing so.
* Move the physical memory constrains heuristic from sandbox library to core

Fix #4335
2021-11-29 15:11:53 +01:00
Johannes Schlatow
916683b6d6 platform_drv(arm): fix cap and memory leaks
* Track all caps and ram quotas of the sub-sessions properly
* Release DMA buffers, it is not done implicitely when destroying
  the Constrained_ram_allocator
* Do not replenish quota before really releasing memory from
  the allocator

Issue #4330
2021-11-29 15:11:53 +01:00
Johannes Schlatow
e256969489 platform_drv(arm): fix destruction order
The Session_component must be destroyed before updating the device
model because the Session_component must also release all previously
acquired devices. If the device model is updated before, the devices
might have been removed.

Issue #4330
2021-11-29 15:11:53 +01:00
Alexander Boettcher
45cebd774d cpu_balancer: avoid dynamic policy allocation
Pre-allocate all possible type of policy objects as part of the thread meta
state to avoid increased memory consumption due to different policy object
sizes. The cpu_balancer accounts the memory per client and can't forward
potentially occurring out-of-ram exceptions during config-ROM update phases.

Fixes #4333
2021-11-29 15:11:53 +01:00
Alexander Boettcher
1f58b05255 cpu_balancer: limit mem increase on config update
The commits avoids reading in and allocating memory for all potentially
threads, which are potentially currently not existent (but configured in the
policy beforehand). Instead the policy is read in and evaluated when a thread
is created and policy changes are solely applied to existing/running threads.
By this the commit avoids the increase of memory consumption during the
evaluation of policies during config ROM updates.

Issue #4333
2021-11-29 15:11:53 +01:00
Piotr Tworek
30b3ad218f virt_qemu: Use VirtIO fb driver instead of ramfb.
This driver should be better suited for VirtIO based board like
qemu_virt.
2021-11-29 15:11:53 +01:00
Piotr Tworek
7c1888644a os: Add VirtIO framebuffer driver.
This implements the necessary bits to provide 2D framebuffer support on
top of VirtIO GPU device as implemented in Qemu. I don't know if any
other implementation of this specific device exists.

Compared to the ramfb driver which already exists in Genode Virtio FB driver
has one major benefit. It allows Qemu window to be dynamically resized at
runtime. The driver will treat this as resolution change and act accordingly.
Ramfb driver can currently only use the hardcoded 1024x768 screen size. Changing
screen resolution might not sound like a big deal, but it is rather useful to
run Genode on Qemu in full screen mode.
2021-11-29 15:11:53 +01:00
Piotr Tworek
2ec9e69fd4 os: Add support for reading VirtIO responses.
Some more advanced devices like VirtIO GPU do expect they can receive
responses to VirtIO commands they issue via VirtIO queue. Such responses
are not sent via a separate device writeable queue. Instead the driver
is expected to queue some additional descriptors and buffers which the
device can then use to provide the reply.

This patch adds support for such write-data-read-response opeartion to
Genode VirtIO::Queue implementation. The implementation is pretty simple
and does not support any fancy features like receiving the response
asynchronously. Instead the operation will use caller provided callback
to wait for the device to process the command. Once this callback
returns the write-data-read-response VirtIO::Queue function will invoke
another callback passing received response as argument.
2021-11-29 15:11:53 +01:00
Josef Söntgen
645e51dc1d gpu: add information for etnaviv driver
Mesa queries information about the underlying device and this header
denotes the layout of the information. It is also used by the driver
itself to populate the 'info_dataspace'.

Issue #4329.
2021-11-29 15:11:52 +01:00
Norman Feske
133d21ad38 event_filter: add touch-click filter
This filter bridges the gap between a touchscreen driver, which
generates raw touch events and traditional GUI applications that expect
a pointer (absolute motion, press/release of the left mouse button).

Fixes #4332
2021-11-29 15:11:52 +01:00
Norman Feske
09d020508c depot: recipes for pin-control/state session APIs
Issue #4315
2021-11-29 15:11:52 +01:00
Norman Feske
dc39a8db62 base: introduce Allocator::try_alloc
This patch changes the 'Allocator' interface to the use of 'Attempt'
return values instead of using exceptions for propagating errors.

To largely uphold compatibility with components using the original
exception-based interface - in particluar use cases where an 'Allocator'
is passed to the 'new' operator - the traditional 'alloc' is still
supported. But it existes merely as a wrapper around the new
'try_alloc'.

Issue #4324
2021-11-29 15:11:52 +01:00
Johannes Schlatow
a65807ff08 rom_filter: optionally skip top-level node
Since the top-level node of the output ROM is always generated by the
rom_filter, there is no way to pass-through the content of an input ROM
without wrapping in an addition XML node.

genodelabs/genode#4326
2021-11-29 15:10:53 +01:00
Norman Feske
3b456e0037 test/resource_request: ignore spurious requests
This patch makes the test less dependent on the rate of state updates by
calculating the upgraded quota from the values found in the state report
instead of simply increasing the '_ram_quota' for each incoming report.
2021-11-29 15:10:53 +01:00
Norman Feske
231ac187fe base: introduce Ram_allocator::try_alloc
This patch replaces the 'Ram_allocator::alloc' RPC function by a
'try_alloc' function, which reflects errors as 'Attempt' return value
instead of an exception.

Issue #4322
Issue #3612
2021-11-29 15:10:53 +01:00
Norman Feske
293d545b97 sandbox: use Genode::update_list_model_from_xml
This patch replaces the former local implementation by the function
provided in 'util/list_model.h'.

Issue #4317
2021-11-29 15:10:52 +01:00
Norman Feske
f2a627c107 os: Pin_control and Pin_state session interfaces
The new interfaces are meant to gradually replace the existing
Gpio_session interface.

- Each session refers to a single pin.
- The session types distiguish the direction of the signal as input or
  output.
- Pin coordinates can be selected via session labels.
- GPIO interrupts are covered by the regular IRQ session interface.

The interfaces are accompanied by framework utilities and interfaces:

- os/pin_driver.h
- pin_control_session/component.h
- pin_state_session/component.h

These headers relieve GPIO drivers from implementing boilerplate code by
providing device-agnostic portions. The A64 pio driver serves as
reference for using those utilities.

  https://github.com/nfeske/genode-allwinner/tree/master/src/drivers/pin/a64

Fixes #4315
2021-11-29 15:10:52 +01:00
Alexander Boettcher
533015b93e nova: support transfer of IA32_TSC_AUX MSR for vCPUs
The 32-bit MSR is returned by rdtscp in ecx register and used to detect
the CPU ID the timestamp was taken on.

Issue #4314
2021-11-29 15:10:52 +01:00
Stefan Kalkowski
b12b0ed93d imx8q_evk: remove board support from repository
Moved to separate repo at https://github.com/skalk/genode-imx/

Fix #4301
2021-11-29 15:10:51 +01:00
Johannes Schlatow
7917c5d9ec Remove zynq_qemu platform and zynq nic driver
Moved to separate repo at https://github.com/jschlatow/genode-zynq

Fixes genodelabs/genode#4280
2021-11-29 15:10:51 +01:00
Stefan Kalkowski
6d991313b1 nvme_drv: get I/O resources from platform API only
Do not use the I/O memory service from core directly,
but instead use the I/O resources the platform session
provides to the driver anyway.

Fix #4309
2021-11-29 15:10:51 +01:00
Stefan Kalkowski
43ef9b5a3a platform_drv: do not account IOMEM as ram quota
Accidentally, the size of IOMEM dataspace got accounted within the
implementation of the platform driver for ARM. Instead we should
only account a bit for the metadata and paging of the I/O memory.

Fix #4307
2021-11-29 15:10:51 +01:00
Johannes Schlatow
87bb81cd66 nic: add _custom_conn_tx_ack_avail_handler()
The custom ack avail handler is required for zero-copy nic drivers (e.g.
the zynq nic driver), which must release the corresponding DMA buffers.

Fixes genodelabs/genode#4277
2021-11-29 15:10:51 +01:00
Piotr Tworek
4f1e9fd256 os: Use VirtIO input drivers for virt_qemu.
This makes Genode's interactive test scenarios run on virt_qemu board
actually interactive.

Issue #4282
2021-11-29 15:10:51 +01:00
Piotr Tworek
60f1a1a554 os: Add VirtIO input driver.
This component can service Qemu VirtIO mouse, keyboard and tablet
devices. The implementation is based on VirtIO 1.1 device spec. Its
described in section 5.8 "Input Device".

Issue #4282
2021-11-29 15:10:51 +01:00
Christian Helmuth
abc9a2f232 depot: update recipe hashes 2021-10-14 13:46:25 +02:00