genode/doc/release_notes-20-08.txt

              ===============================================
              Release notes for the Genode OS Framework 20.08
              ===============================================

                               Genode Labs



There are two overarching themes of Genode version 20.08: Increasing the
weight of native work loads, and strengthening the system's resilience against
driver failures.

With native work loads, we are speaking of software executed directly on
Genode without relying on virtual machines. Compared to static systems or
hypervisor scenarios, such work loads are brutally unforgiving when it comes
to the quality of the POSIX runtime, the performance of Genode's protocol
stacks, and the economics of the porting of software. By bringing the
*Chromium* web engine alive, we expose Genode to one of the most heavy-weight
commodity software stacks in existence. We are thrilled to report in Section
[Improved Qt5 integration and work flows] that the Chromium-based Falkon web
browser can be hosted on Sculpt OS now. The collateral effects of this work
are at least as valuable as the particular application: Improved work flows
for porting large software projects, and covering many formerly hidden corner
cases of the C and C++ runtimes.

With [https://genode.org/download/sculpt - Sculpt OS], the resilience of
Genode in the event of failing graphics or input drivers came into focus.
Section [The GUI stack, restacked] describes a complex surgery that puts
the *low-level GUI stack* upside down, paving the ground for exciting features
like swapping out or updating drivers on the fly without reboot.

Besides these main topics, the current release features the continuation of of
two long-term projects, namely the CBE block encrypter and the profound
support of 64-bit ARM devices.
Section [Consistent Block Encrypter] describes the CBE's new *pluggable
crypto* and trust anchor facilities.
Section [Device drivers] goes into detail about the steadily emerging *driver
landscape on 64-bit ARM*.


The GUI stack, restacked
########################

The current release breaks free from several limitations of Genode's low-level
GUI stack. This architectural change was preceded by a two-years planning and
exploration phase, spinning the following goals in our heads:

* Achieving resilience against failing device drivers. Graphical applications
  on Genode shall be able to survive driver failures.
  We strive to keep overly complex device drivers out of the trusted computing
  base of security-critical applications.

* Enabling the principal support for screen capturing and remote desktop
  scenarios without compromising the security and privacy of the user.

* Paving the ground for sophisticated multi-head scenarios in the future,
  e.g., using more than one graphics card.

* Starting, killing, and updating drivers at runtime without rebooting the
  system.

* Optimizing the throughput of the pixel data path and the input latency,
  aiming at a highly responsive interactive experience without tearing
  artifacts.

The key idea and plan for this change has been laid out in the dedicated article
[https://genodians.org/nfeske/2020-06-23-gui-stack - Restacking the GUI stack].
We executed the steps outlined in the article just in time for the current
release.


Consistent naming
=================

The starting point of Genode's GUI stack was the nitpicker GUI server,
introduced in 2006 and based on prior research of the Genode developers
([https://www.genode-labs.com/publications/secure-gui-2009.pdf - PDF]).
Back then, the name of the implementation somehow proliferated throughout
Genode from the API level, over the configuration level, up to the naming of
related components. Over the years, the jargon crept into the project without
much thought. This had to stop and now was the right time. The renaming
affects the following areas.

:Source tree:

  The _include/nitpicker_session_ headers moved to _include/gui_session_.
  Those headers are not tied to the specific nitpicker implementation but are
  much more general. For example, the window manager provides an alternative
  implementation of the same interface.

  The change also affects the corresponding depot API archive accordingly.

:Configuration and run scripts:

  The "Nitpicker" service is now called "Gui" service. The upper/lower casing
  follows the convention for services provided outside of Genode's core
  component.

:Packaged runtime definitions:

  The '<nitpicker>' nodes in runtime files are now named '<gui>'.

:API level:

  The C++ namespace 'Nitpicker' has been renamed to 'Gui'.

:Component names:

  The nit_fb server has become the gui_fb server.
  The nit_fader has become the gui_fader.


Streamlined pixel format
========================

Genode's framebuffer session interface was traditionally based on the RGB565
pixel format. We have now changed the format to 32-bit XRGB where the X part
is ignored. We updated all affected libraries, graphical applications, and
device drivers accordingly.

This change also affects the users of the drivers_interactive packages. Now,
we generally assign 64 MiB RAM and 1500 caps to the drivers subsystem, which
is sufficient for covering high resolutions at 32 bits per pixel and to
accommodate multi-component USB HID input stacks.


New "Capture" and "Event" session interfaces
============================================

Traditionally, Genode's driver model for framebuffer and input drivers
followed a layered architecture. Each framebuffer driver naturally provided a
"Framebuffer" service and each user-input driver provided an "Input" service.
At the next layer, the nitpicker GUI server would use those services as a
client and, in turn, provide a higher-level "Nitpicker" service (which has now
been renamed to "Gui"). In this architecture, higher levels naturally depend
on all lower levels, specifically the framebuffer and input drivers at the
bottom. The interplay of the components looks as follows. The arrows denote
client-server relationships where the arrow points from the client to the
server. The drivers and the nitpicker GUI server are displayed in red because
they are servers. The GUI applications at the right are nitpicker clients.
Nitpicker, in turn, is a client of the framebuffer and input drivers.

[image nitpicker_orig]

This architecture is flawed because it makes the trusted low-complexity GUI
server (nitpicker) depend on high-complexity and potentially flaky device
drivers. If a driver fails, the entire GUI stack - up to the applications -
suffers. To put the risk into perspective: The binary of the nitpicker GUI
server incorporates about 11,000 lines of code, which includes the Genode API.
The actual nitpicker code is about 3000 lines of plain C++.
In contrast, the Intel framebuffer driver comprises more than 122,000 lines of
code (sloccount). Code complexity equals risk. The dependency of nitpicker on
the Intel framebuffer driver is indefensible. For input devices like USB HID,
the situation looks very similar.

To solve this dilemma, we had to turn the client-server relationship between
the GUI server and the drivers upside down. This approach is enabled by two
new service interfaces. The new "Event" session replaces the former input
session. The new "Capture" session replaces the former framebuffer session.
The information that is flowing through those interfaces remains the same -
input events and pixels - but the direction is reversed. An event client
propagates input events to the server. A capture client obtains pixel data
from the server.

[image nitpicker_restacked]

In the new architecture, nitpicker remains as the only server, providing the
"Event", "Capture", and "Gui" interfaces. It does no longer depend on the
well-being of any driver. This is reflected by the <start> node of nitpicker
when integrating the GUI server into an init configuration:

! <provides>
!   <service name="Gui"/> <service name="Capture"/> <service name="Event"/>
! /provides>

To retain support of cascaded usage scenarios of the GUI server, nitpicker can
still be instructed to request a framebuffer session or input session by
specifying the '<config>' attributes 'request_framebuffer="yes"' and
'request_input="yes"'. If specified, nitpicker requests sessions for an
"Input" and a "Framebuffer" service upon startup and uses those as
input/output back end. Those services are usually provided by the gui_fb
component. Note that this facility may eventually be replaced by a
'request_gui' attribute (requesting a GUI session), eliminating the notion of
framebuffer and input services altogether.


Turning all framebuffer and input drivers into nitpicker clients
================================================================

Following the introduction of the new capture-session interface, we replaced
the use of the "Framebuffer" session interface by the new "Capture" session
interface in all framebuffer drivers. The new versions of all drivers had to
be tested on the respective hardware. Generally, the drivers have become
simpler.

Analogously to the graphics drivers, all input drivers had to be reworked to
operate as event clients instead of input servers. This came down to adjusting
and testing the variety of PS/2 drivers and USB HID drivers on all supported
platforms. As a minor side note, the server/acpi_input pseudo driver has
turned into the app/acpi_event application. Similar to the framebuffer
drivers, the transition from input servers to event clients generally made the
drivers simpler.

The drivers_interactive packages for various boards have changed their public
interface. The drivers subsystem no longer provides "Framebuffer" and "Input"
services but needs a valid route to the "Capture" and "Event" services
provided by nitpicker. It goes without saying that this change affected all
run scripts that make use of nitpicker.


Replacing the input filter with an event filter
===============================================

The transition from the "Input" session to the "Event" session is not limited
to the drivers only but also affected Genode's input-filtering mechanism. The
functionality of the input_filter is now provided by the event_filter. The
event filter requests only one "Event" session as destination for the filter
results, which is usually routed to the nitpicker GUI server. It provides an
"Event" service to which any number of event sources can connect.

The configuration of the filter chain remains almost the same. Only the
declaration of the '<input>' nodes is no longer needed. Instead, the
configuration must specify '<policy>' nodes, which define the mapping of
"Event" clients (event sources) to the inputs used in the filter chain.


Improved Qt5 integration and work flows
#######################################

Streamlined build-system integration
====================================

Up to now, Qt libraries and applications for Genode had been built using the
Genode build system. The Qt library build files in the _libports/lib/mk_
directory were in part generated with a shell script from the GNU make files
which the _qmake_ tool had generated when building Qt for Linux. Also there
was some rudimentary support for building Qt applications from qmake project
files by just interpreting the qmake project files as GNU make files and
translating the relevant qmake variables to corresponding Genode build-system
variables.

But this approach was not feasible anymore when building the 'qtwebengine'
module (the successor of the deprecated 'qtwebkit' module), which is based on
the Chromium web engine source code and mostly built with the Ninja build
system on officially supported platforms instead of qmake.

We also wanted to make it easier for Genode users to build Qt applications, in
particular by using the [https://github.com/nfeske/goa - goa] tool. It would
be better to be able to actually process qmake project files with the qmake
tool instead of interpreting them as GNU make files and aborting when a
project file contains qmake-specific code not understood by GNU make.

For these reasons, we reworked the Qt build process such that it now uses
qmake with a tailored Genode platform configuration for building both Qt
libraries and applications.

If you have already built a Qt application for Genode and want to continue
using it with the new release and you cannot use the goa tool yet (for example
because additional Qt libraries cannot be built with goa or because other
architectures than x86_64 are not supported by goa currently), you need to
update the 'target.mk' file (and any run scripts) of your project. For the
details, please have a look at the updated Qt example projects in the
'libports' repository and feel free to ask on the Genode mailing list if any
problems arise with the conversion. It is also necessary to rebuild the Qt
tool chain with the 'tool/tool_chain_qt5' script to get the qmake tool built
and installed.

While converting the run scripts of the Qt example projects, we switched to
usage of the 'drivers_interactive' package, which is the preferred way to
start drivers for interactive scenarios nowadays. A downside might be that
network or storage drivers are not loaded this way. If these features are
needed for a Qt project, the suggested solution is to use goa or to run the
scenario with Sculpt OS instead, if possible.

Another Qt-related feature in this release is that the 'qt5_component' library
now extracts command line arguments and environment variables from the
'config' ROM, like the 'posix' library already did for non-Qt applications.


Chromium engine and Falkon web browser
======================================

Since the previously ported 'Arora' web browser has not been maintained
upstream for years and the 'qtwebkit' Qt module it depended on has also been
deprecated in the meantime, it was eventually time to try to port the newer
Chromium-based 'qtwebengine' module to Genode as well as to find a new and
maintained qtwebengine-based web browser which could replace Arora. At first
glance, porting qtwebengine to Genode appeared to be quite a challenging task,
since officially only Linux, Windows, and macOS are supported and most of the
qtwebengine code, which also contains a huge amount of third-party libraries,
is not built with qmake for the most part like other Qt modules, but with the
Ninja build system.

Fortunately, the situation turned out to be less problematic than expected, at
least with regard to the initial goal of getting a lightweight web browser
running and displaying regular web sites comparably to Arora. One reason for
that is that there already existed a qtwebengine port for FreeBSD, which
resolved most of the possible libc compatibility problems and thus served as
the basis for the Genode port. There were still some operating system
specifics in the code, which had to be dealt with, though, like supporting
shared memory with 'mmap' or cache maintenance for JIT-compiled JavaScript
code. Advanced web-engine features like multimedia support are disabled at the
moment, also some security-related features like the verification of server
certificates (requires a port of libnss), multi-processing, or sandboxing
remain unused at the current time.

The build system issue could eventually be resolved by adding a Genode
platform back end for qmake and passing the correct compiler options to the
Chromium build system.

As qtwebengine-based web browser, we chose the [https://www.falkon.org - Falkon]
browser, which was previously known as 'QupZilla'. Even though the current
performance and memory footprint clearly call for further optimization work on
our side, it looks quite promising so far:

[image falkon_screenshot]

For the upcoming release of the a Sculpt OS in September, Falkon will be
offered as a ready-to-use package.

The Arora web browser and qtwebkit module are no longer available with this
release.


Base framework and OS-level infrastructure
##########################################

Base API/ABI changes
====================

Synchronization primitives
~~~~~~~~~~~~~~~~~~~~~~~~~~

With the current release, we wrap up the
[https://genode.org/documentation/release-notes/20.05#Migration_from__Lock__to__Mutex__and__Blockade_ - migration]
of the code base from the 'Lock' to the 'Mutex' and 'Blockade' types to
improve diagnostics and clear the path towards future performance
optimizations.


Moved atexit handling from base library to C runtime
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Traditionally, Genode's base library took care of the execution of 'atexit'
handlers as this mechanism was regarded as fundamental to both C and C++. In
particular, the C++ compiler automatically generates calls to the
'__cxa_atexit' function of the C++ ABI for objects instantiated as local
static variables so that the destructors of such objects are called at the
exit of the program.

With the transition to Genode's modern component API (introduced in version
[https://genode.org/documentation/release-notes/16.05#New_component_API - 16.05]),
this mechanism became irrelevant for plain Genode components. But for
higher-level components that rely on the C runtime, in particular POSIX
applications, this mechanism remained crucial. Hence, we did not question the
presence of the atexit mechanism in Genode's base library for a long time.

However, we ultimately realized that the atexit functionality must be
moved from the base library into the C runtime to fully adhere the
[https://genode.org/documentation/release-notes/17.02#New_execution_model_of_the_C_runtime - execution model]
of the C runtime. Atexit handlers may perform I/O such as the closing and
syncing of files after all.

This change has the positive side effect of reducing the complexity of the
base library. Furthermore, it overcomes the limitations of the formerly
statically dimensioned array of atexit handlers, which was wasteful for plain
Genode components, yet insufficient for complex POSIX applications.

Note that this change deliberately renders the atexit mechanism for plain
Genode components (without C runtime) ineffective. Genode components that used
to rely on the 'genode_exit' function for calling cleanup code must be changed
to explicitly call cleanup code.

This change also removes the implicit call of 'genode_exit(1)' by 'abort',
which is triggered by, i.e., uncaught exceptions. Scenarios that relied on
the former behavior for component monitoring should be changed to use Genode's
[https://genode.org/documentation/release-notes/18.11#Component_health_monitoring - heartbeat monitoring]
mechanism instead.

As another minor but still noteworthy detail, shared-library destructors are
no longer called via the atexit mechanism but explicitly by the dynamic
linker. This slightly changes the call order of destructors: Functions marked
as destructors are called after the libc-managed atexit handlers now.


Removal of deprecated interfaces
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Unsafe 'Xml_node' methods
-------------------------

The unsafe 'Xml_node' methods that were marked as deprecated in the
[https://genode.org/documentation/release-notes/20.05#Deprecation_of_unsafe__Xml_node__methods - previous release]
are removed now.


Cancel-blocking interface
-------------------------

Closely related to the revised synchronization primitives mentioned above, we
removed the outdated 'Cancelable_lock' API along with the underlying
cancel-blocking mechanism, which had become obsolete by now. In past times,
this mechanism was employed to cleanly wind down threads that are blocked at
their destruction time. The move of Genode's API design away from blocking RPC
to asynchronously working components marginalized the problem to the point
where it ceased to exist.


Delayed dispatch of 'Rpc_entrypoint'
------------------------------------

Similar to the cancel-blocking mechanism, the ability to delay the dispatching
of RPC requests until an explicit call of 'Rpc_entrypoint::activate' is an API
relic from a time when each RPC server was multi-threaded. The modern flow of
execution has become much simpler, eliminating the need for this feature.


C runtime
=========

Serialized VFS access
~~~~~~~~~~~~~~~~~~~~~

Unsurprisingly, the VFS is an integral part of the C runtime as _everything is
a file_ in the UNIX philosophy. The basis for the libc VFS is the VFS library
and its versatile plugin concept. As the libc (and other users of the library)
may employ multiple threads, which could use the VFS, the integrity of data
structures, esp. those of more complex plugins like the network stack in
_vfs_lwip_, must be protected from concurrent access. From the beginning, we
planned to move the serialization of accesses out of the plugins into code
that uses the VFS library but missed to completely achieve this goal with the
libc execution model. With Genode 20.08, we transform the lower parts of the
libc into an execution monitor that ensures the aspired serialized processing
of VFS operations. This paves the way towards the removal of synchronization
measures in all VFS plugins with the upcoming release.


Removal of libc_pipe LibC plugin
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The VFS pipe plugin was introduced in
[https://genode.org/documentation/release-notes/19.11#VFS_plugin_for_emulating_POSIX_pipes - Genode 19.11]
and matured into a suitable replacement for the aged libc_pipe plugin library.
Now, we finalize this process by removing the old libc plugin from Genode and
adapt all run scripts (most prominently those for Qt5 examples). Users can
migrate to the VFS plugin with the following changes to their component
configurations.

* The libc must be configured to access the pipe mechanism from a VFS
  path by the 'pipe="..."' configuration attribute like follows.
  ! <libc ... pipe="/pipe"/>

* The pipe plugin must be instantiated in the VFS at the desired path.
  ! <vfs>...
  !  <dir name="pipe"> <pipe/> </dir>
  ! </vfs>

* Run scripts no longer need to integrate _libc_pipe.lib.so_ into the
  boot modules but _vfs_pipe.lib.so_.


Placing pthreads on CPUs
~~~~~~~~~~~~~~~~~~~~~~~~

Up to now, pthreads experienced no affinity configuration on creation time and
ran implicitly on the first CPU available to the component only. With this
release, we add a placement strategy and configuration option to the C runtime
to enable the tuning of CPU placement for pthreads of libc components. The
default placement strategy is 'all-cpus', which means pthreads are assigned to
the available CPUs. round-robin.

The new '<pthread placement="manual">' libc configuration sub-node can be used
to manually place pthreads according to their 'id' on 'cpu' as follows.

! <config>
!   <libc ...>
!     <pthread placement="manual">
!     .   <thread id="0" cpu="0"/> <!-- pthread.0 placed on CPU 0 -->
!     .   <thread id="1" cpu="1"/> <!-- pthread.1 placed on CPU 1 -->
!     .   <thread id="2" cpu="2"/> <!-- pthread.2 placed on CPU 2 -->
!     .   <thread id="3" cpu="2"/> <!-- pthread.3 placed on CPU 2 -->
!     </pthread>
!  </libc>

Note, the current placement-strategy implementation is limited to 32 pthreads.
If more pthreads are running simultaneously further threads are placed on CPU
0.


Standard C++ library
====================

We enabled support for thread-related features like 'std::thread' or
'std::mutex' in the stdcxx library.


Ada/SPARK support
=================

With the development of the [Consistent Block Encrypter], Genode's Ada/SPARK
support was used in the context of a rather complex software project for the
first time. Therefore, we felt the necessity to make up our minds about a
generally accepted Ada/SPARK coding-style as long as this step still doesn't
entail the painstaking adaption of large amounts of code.

Fortunately, the GNAT compiler flags assist developers a lot in this affair.
That said, we added the consideration of two new environment Make-variables to
the Genode tool chain: 'CC_ADA_WARN' and 'CC_ADA_WARN_STRICT'. Analogously to
the C++ pendants ('CC_CXX_WARN' and 'CC_CXX_WARN_STRICT'), the variable
'CC_ADA_WARN' denotes the common warning-configuration of the compiler for
Ada/SPARK whereas 'CC_ADA_WARN_STRICT' applies additional configuration for a
much higher level of strictness by default. This strict mode might be
deactivated locally for code which is not subject to the Genode coding style -
e.g., ported third party code - by adding the following line to the
corresponding build description files.

! CC_ADA_WARN_STRICT =

The common, less strict warning mode for Ada/SPARK merely enables all warning
messages with the '-wa' flag. The strict mode, in addition to it, causes GNAT
to treat warnings like errors and to refuse compiling affected code. In
addition to that, the strict mode activates a broad variety of checks for
potential pitfalls and style consistency using the configuration
'-gnatyyBdSux', which is compatible to the GNAT standard style.
A comprehensive description of all the checks would be beyond the scope of
this document. Please have a look at the
[https://gcc.gnu.org/onlinedocs/gnat_ugn/Style-Checking.html - GNAT style-checking reference]
instead if you're interested.

Besides the style enforcement through compiler flags, we decided to generally
promote "Camel-Snake Casing" for Ada/SPARK code. The only exceptions from that
are abbreviations of initials that are completely upper case and language
keywords that are completely lower case. This would be a conform snippet of
code:

! function T1_Node_XML_Tag_Open (
!    Node     : Type_1_Node_Type;
!    Node_Idx : Type_1_Node_Block_Index_Type;
!    Show_Hsh : Boolean;
!    VBA      : Virtual_Block_Address_Type)
! return String;

Another minor improvement of Genode's Ada/SPARK runtime consists in the added
support for the exponential operator on integers.


Operating-system components
###########################

Consistent Block Encrypter
==========================

With the Genode 20.05 release, our
[https://genode.org/documentation/release-notes/20.05#Feature-completeness_of_the_consistent_block_encrypter - Consistent Block Encrypter]
(CBE) reached feature completeness by adding support for
[https://genodians.org/m-stein/2020-06-07-cbe-rekeying - online rekeying] and
[https://genodians.org/m-stein/2020-06-08-cbe-resizing - online resizing].
Furthermore, we were able to run a Linux virtual machine using a CBE device as
system partition. During the last three months, we mainly reviewed and
consolidated the code but also improved the integration in a notable way.

One important step forward is that we implemented a new _reference_ trust
anchor. The trust anchor (TA) is a separate entity (might be a smartcard, a
USB dongle, or a piece of software) that the CBE relies on for key management
and for storing superblock hashes. In contrast to the dummy TA we used for
testing so far, the reference TA stores private key and superblock hashes
persistently through a file-system back-end and requires the user to enter a
passphrase to be unlocked. This enables the CBE to actually verify that the
CBE device is in the same state that it was left in the last time and that
only authorized users can access its data.

Furthermore, both the crypto and the TA back end of the CBE are now integrated
through VFS plugins. This change gives the user of the CBE more flexibility in
two ways. On one hand, one can switch between integrating those entities
locally (inside the CBE driver) or remotely (in a VFS server instance isolated
from the CBE driver) by merely altering the system configuration.

[image cbe_integration] Three different ways to integrate the CBE

On the other hand, the implementation of the crypto entity and the trust
anchor can now be swapped out through the means of configuration alone.
For instance, let us take a look at the VFS server in the CBE test scenario
_repos/cbe/run/vfs_cbe.run:_

! <start name="vfs_cbe" caps="120">
!   <config>
!     <vfs>
!       <cbe_crypto_aes_cbc name="cbe_crypto"/>
!       ...
!     </vfs>
!     ...
!   </config>
!   ...
! </start>

We may replace the plugin employing an AES-CBC ESSIV block-cipher in this test
with one using plain memcopy (in other words: disabling block encryption) by
changing one line only:

! <start name="vfs_cbe" caps="120">
!   <config>
!     <vfs>
!       <cbe_crypto_memcopy name="cbe_crypto"/>
!       ...
!     </vfs>
!     ...
!   </config>
!   ...
! </start>

The memcopy example may in fact be useful for contexts where only the
abilities of the CBE to manage snapshots and to ensure consistency are of
interest and encryption would only generate unnecessary overhead. However, the
primary motivation for this high modularity is the easy integration of other
cipher algorithms in the future.

In order to make things even more approachable, the VFS-plugin-related code
between CBE, crypto entity, and trust anchor relies on generic interfaces.
This simplifies supporting other existing cryptographic algorithms and TA
types as enabling them boils down to the implementation of a handful of well
defined hook functions.

As for the CBE demo scenario that we published with the previous Genode
release: The corresponding packages moved from the package source 'cnuke' to
'mstein'. Furthermore, the CBE demo is now based on the unaltered Genode
master branch and the new master branch in the CBE repository itself. Besides
that, for the most part, the procedure for reproducing the demo described in
the [https://genodians.org/m-stein/2020-06-12-cbe-linux-vm - demo article] and
the [https://genodians.org/m-stein/2020-06-17-cbe-linux-vm-video - demo video]
remained the same. But be prepared to find minor discrepancies.


Virtual file system (VFS)
=========================

We extended the 'zero' VFS plugin to limit the amount of data - the number of
zero bytes - it will provide on plain sequential reads. The file size can be
determined by setting the 'size' configuration attribute limiting readable
seek offsets to the configured size and resulting in EOF beyond that offset.
The attribute is parsed as 'Number_of_bytes', which caps the amount to the
maximum of the addressable memory on the platform.

This addition comes in handy when needing to provide a synthetic empty file
with a fixed size:

! <vfs>
!   <ram/>
!   <import>
!     <zero name="block_file" size="32M"/>
!   </import>
! </vfs>


Furthermore, we added an option to the 'terminal' VFS plugin to ignore control
characters. Usually this plugin is used in interactive configurations where
reacting to control characters, e.g. ^C, is vital. That being said, the plugin
lends itself as general transport mechanism for low-traffic data transfers.
Those transfers, however, may contain arbitrary data that could very well be
interpreted as control characters. Therefore, we introduced the 'raw'
configuration attribute. Setting the attribute instructs the plugin to pass
all data through as-is without further interpretation.

! <vfs>
!   <dir name="/dev">
!     <terminal name="io" raw="yes"/>
!   </dir>
! </vfs>


Block server backed by VFS content
==================================

This release adds a new component that provides access to a VFS file via the
block-session interface. Block requests will be translated to VFS requests
operating directly on the file.

The client access is configured by setting the attributes of the 'policy' node
appropriately. The assumed block size of the underlying file must be specified
via the 'block_size' attribute. It defaults to 512 bytes.

The block count is determined by querying the file and dividing its size by
the block size. Pseudo file systems, which do not return a proper size in
their 'stat' implementation will therefore not work. The 'writeable' attribute
denotes if the block session is allowed to perform write requests. However,
should the underlying file be read-only, such requests will nonetheless fail.
The default value is 'no'.

The following snippet shows a configuration where the 'vfs_block' component
grants the component 'the_client' writeable access to the 'ext2_disk.img'
file. In return, this file is accessed via a file-system session.

! <start name="vfs_block">
!   <resource name="RAM" quantum="3M"/>
!    <provides> <service name="Block"/> </provides>
!      <config>
!
!        <vfs>
!          <fs buffer_size="2M" label="backend"/>
!        </vfs>
!
!        <policy label="the_client" file="/ext2_disk.img" block_size="512"
!                writeable="yes"/>
!      </config>
!      <route>
!        <service name="File_system" label="backend">
!          <child name="fs_provider"/> </service>
!        <any-service> <parent/> </any-service>
!      </route>
! </start>

Since the 'vfs_block' is - by using the VFS library - flexible enough to cover
the use-cases of components like 'ram_block' and 'rom_block', we were able to
remove those components.

For example a typical 'rom_block' use-case is implemented by using the
following configuration:

! <start name="vfs_block">
!   <resource name="RAM" quantum="2M"/>
!    <provides> <service name="Block"/> </provides>
!      <config>
!
!        <vfs>
!          <rom name="genode.iso"/>
!        </vfs>
!
!        <default-policy file="/genode.iso" block_size="2048"/>
!      </config>
!      <route>
!        <any-service> <parent/> </any-service>
!      </route>
! </start>

On a technical note, the server currently allows a single session only and
processes requests sequentially.


Device drivers
##############

System management privilege
===========================

The PD session interface of the base library got extended by a
system-management call:

! Managing_system_state managing_system(Managing_system_state const & state)

It is used to call privileged system-management functionality, like power
management or clock settings that cannot be implemented in user-land
components. The transferred state is equivalent to the common general-purpose
CPU registers. Currently, the function is implemented by base-hw's core
component on ARMv8 to call the ARM Trusted Firmware on behalf of the PD
session client. All other kernel and architectures leave the 'managing_system'
call empty.

A special privilege is necessary to enable a component to use the new call.
Therefore, the init component's configuration provides a new attribute in the
start node:

! <start name="platform_drv" managing_system="yes">
!   ...

The new call and privilege got introduced in order to enable a platform driver
to implement power management on modern ARM SoC.

The formerly existent, special privilege 'constrain_phys', which was used by
the platform driver to constrain physical memory to be allocated for DMA to a
specific region, is now implicit part of the 'managing_system' privilege.
Configurations must be adapted accordingly as the 'constrain_phys' attribute
is no longer supported.

That means configurations that so far used the "constrain_phys" attribute in
the RAM resource description, like the following:

! <start name="platform_drv">
!   <resource name="RAM" quantum="2M" constrain_phys="yes"/>
!   ...
! </start>

have to be changed to the following to be effective:

! <start name="platform_drv" managing_system="true">
!   <resource name="RAM"/>
!   ...
! </start>

Platform driver
===============

Configuration changes
~~~~~~~~~~~~~~~~~~~~~

The new platform driver API that was introduced with the
[https://genode.org/documentation/release-notes/20.05#New_platform_driver_for_the_ARM_universe - previous release]
for the ARM architecture received a minor subsequent improvement, namely the
device resource description within the configuration of the platform driver:

! <device name"foo">
!   <resource name="IRQ"    ... />
!   <resource name="IO_MEM" ... />
! </device>

is now shortened to the following:

! <device name="foo">
!   <irq    ... />
!   <io_mem ... />
! </device>


Power and clocks for i.MX 8M Quad SoC
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To drive peripherals of the i.MX 8M Quad Evaluation Kit and the
[https://www.crowdsupply.com/mnt/reform - MNT Reform 2] appropriately, we
introduce a new platform driver specific for the i.MX 8M Quad SoC used by both
platforms.

The new driver supplements the API of the generic ARM platform driver
introduced in release 20.05 with power and clock management specifically for
this SoC. It comprises all power and clock settings available in the SoC.
Single power domains and clocks can be assigned to devices. Whenever the
client of the platform driver acquires a device, its power domains and clocks
are enabled implicitly, and all clocks are set accordingly. When the device
gets released, all power-domains and clocks are disabled again.

The following USB host-controller example for the i.MX 8M Quad EVK illustrates
the additional configuration options:

! <device name="usb_host_2">
!   <io_mem       address="0x38200000" size="0x10000"/>
!   <irq          number="73"/>
!   <power-domain name="usb_otg_2"/>
!   <clock        name="usb_phy_ref_clk_root"
!                 driver_name="usb_phy_root_clk"
!                 parent="system_pll1_div8"
!                 rate="100000000"/>
!   <clock        name="usb_core_ref_clk_root"
!                 parent="system_pll1_div8"
!                 rate="100000000"/>
!   <clock        name="usb_bus_clk_root"
!                 parent="system_pll2_div2"
!                 rate="500000000"/>
!   <clock        name="usb_ctrl2_gate"/>
!   <clock        name="usb_phy2_gate"/>
!   <property     name="compatible" value="snps,dwc3"/>
!   <property     name="dr_mode" value="host"/>
! </device>

Most clock settings can be derived directly by looking at device tree sources
for the corresponding board and peripheral, e.g., in the Linux kernel tree.
But be aware that the clock names are aligned to the i.MX 8M Quad Applications
Processors Reference Manual and not the Linux code.

As a side effect of the clock settings work, the CPU cores of the i.MX 8M Quad
SoC are now running with the full-rate of 1.5 GHz.


New platform driver for Raspberry Pi
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

As some driver components like the USB host-controller driver share one code
base across a lot of different boards, SoCs, and architectures. We like to
unify the API they are dependent on, including the platform driver's API.
Therefore, the already existent platform driver for Raspberry Pi 1 got
modernized to take advantage of the new platform driver API for ARM introduced
with the previous release.

However, we did not achieve to change all drivers at once. Therefore, we kept
the old platform driver still used by some drivers for now. To prevent name
clashes, the new driver is named "rpi_new_platform_drv" until the legacy
driver "rpi_platform_drv" vanishes.

During this work, we unified the formerly different enumeration of interrupts
between the Fiasco.OC and base-hw kernels.


USB host-controller driver
==========================

The USB host-controller driver got modified to iterate over all device
resources the platform driver provides to it, and registers those in
corresponding Linux kernel structures of the ported driver code.
This board-specific information was hard-coded beforehand. Now, the driver
uses the platform driver for x86, as well as all ARM architectures.


Interactive drivers support for Arndale and Panda boards
========================================================

In the Genode world repository the almost abandoned support for Pandaboard and
the Arndale board got revived by drivers_interactive depot packages for both
platforms. Therewith, it is now easy again to use interactive run-script
scenarios on these platforms.


Libraries and applications
##########################

Migration of ported software to Genode-World
============================================

In the previous releases, we continuously relieved the Genode main repository
from features that receive no continuous maintenance. Rather than dropping
such features, they find a new home at the
[https://github.com/genodelabs/genode-world - Genode-World] repository.
We foster this distinction to avoid wrong expectations. Whereas the code of
the Genode main repository is subjected to rigid quality assurance at Genode
Labs, the content of the World repository is tested and maintained in a rather
ad-hoc fashion.

The current release moves the Solo5 runtime (originally added in version
[https://genode.org/documentation/release-notes/18.11#Genode_as_a_platform_for_Mirage-OS_unikernels - 18.11])
and the stubby DNS tool to the Genode-World repository.


New ported 3rd-party software in Genode-World
=============================================

Port of SDL 2.0.14
------------------

As more and more software requires SDL2, it became necessary to support it as
well in addition to the already existing support for SDL1. We therefore ported
the up-to-date SDL version to Genode.

For the time being, its functionality is limited when compared to the SDL1
port. At the current time, only input and non-accelerated video are supported.

Even in its limited state, it suffices for
[https://www.genodians.org/alex-ab/2020-07-02-scrcpy - Streaming Android to Genode using Scrcpy].


Port of gRPC
------------

_This work is a contribution by Pirmin Duss of_
_[https://gapfruit.com - Gapfruit]. Thanks for this welcome update._

At gapfruit, we wanted to provide an efficient way to combine a Genode system
with existing cloud environments. For that, we ported the gRPC/protobuf
libraries and tool chain to Genode. For this release, we simplified the tool
chain so it does not depend on any packages of your Linux distribution.
Additionally, we fixed a memory leak and updated the port to gRPC version
1.28.1 and protobuf version 3.11.2.


Removal of orphaned 3rd-party ports
===================================

The dde_zircon device-driver environment was added in
[https://genode.org/documentation/release-notes/18.08#Experimental_runtime_for_Zircon-based_drivers - 18.08]
as an experiment for executing device drivers of Google's Fuchsia OS as Genode
components. However, since the initial import of the feature, it remained
unused and received no further development. As Fuchsia largely diverged from
the APIs implemented by dde_zircon in the meantime, we decided to remove the
feature.


Sculpt OS improvements
######################

The Genode-based [https://genode.org/download/sculpt - Sculpt] operating
system has been adapted to the new Genode version without major user-visible
changes. Given the fact that this release turned the low-level GUI stack
upside down (Section [The GUI stack, restacked]), this must be read as *good*
news. The GUI-stack changes have several nice side effects:

* The elimination of color-space conversions in the pixel-data path has
  noticeably improved the interactive latency. Sculpt OS has never been
  as snappy as now.

* As Sculpt's base system no longer depends on the successful initialization
  of the framebuffer and user-input devices, the administrative user interface
  can be started even before the low-level drivers are ready, increasing the
  parallelism at boot time. Yes, Sculpt OS boots even faster now.

* The now ubiquitous use of 32-bit pixels throughout the whole GUI stack has
  largely eliminated the need for dithering, thereby improving the visual
  clarity of the user interface. Dithering remains in use in some situations
  though. In particular to reduce quantization artifacts for the alpha channel
  of the leitzentrale overlay. But that is a feature now, not a limitation.


Interactive CPU affinity configuration
--------------------------------------

The component-configuration dialog gained the new ability to restrict the
component to a subset of CPUs. This option is available under the new
"Resource assignment" sub menu.

[image sculpt_affinity]

The configuration dialog shows a matrix of CPU cores where the x-axis denotes
the physical cores and the y-axis the hyperthreads. By default, all available
CPU cores are selected. The user can toggle the nodes by clicking on them.
The dialog ensures that the selection is always a rectangular area. So one
click may affect nodes other than only the clicked one.


Fail-safe handling of unsupported partition tables
--------------------------------------------------

Sculpt now uses Genode's heartbeat-monitoring mechanism to detect the failure
of part_block instances during storage discovery. This situation can happen
when a storage device contains an unsupported partitioning-table format.
In this case or other error cases, the discovery would fail to complete.
The new version of Sculpt responds to this situation by removing the
unresponsive part_block instance automatically, releasing the corresponding
device. This leaves the device available for alternative uses, e.g., for
assigning it to a virtual machine.