=============================================== Release notes for the Genode OS Framework 24.02 =============================================== Genode Labs Version 24.02 focuses on developer experience and framework infrastructure. Genode's Goa SDK has reached prominence in the past few releases. It largely streamlines the porting, development, and publishing of software targeting Genode and Sculpt OS in particular. With the current release, Goa has become able to conveniently use Sculpt OS as a remote test target. Regardless of whether targeting a PC or the PinePhone, either can be turned into a test target in seconds and the developer's compile-test cycle looks exactly the same (Section [Sculpt OS as remote test target for the Goa SDK]). A long anticipated infrastructure topic is the rework of Genode's audio stack to accommodate latency-sensitive scenarios, using flexible sample rates, and making audio drivers pluggable. Section [Revised audio infrastructure] gives an overview of the taken architectural approach, the interfaces, and a low-complexity mixer modelled as self-sufficient resource multiplexer. Speaking of infrastructure, we are excited to report to have wrapped up the transition to our modern Linux device-driver environment based on Linux 6.x. The last piece of the puzzle was the TCP/IP stack that was still based on code originating from Linux 4.4.3. Section [TCP/IP stack based on DDE-Linux version 6.1.20] details the new TCP/IP stack. According to our [https://genode.org/about/road-map - road map], we plan to add suspend/resume as feature to Sculpt OS 24.04. As a crucial stepping stone towards this goal, all drivers that cannot be easily restarted must become suspend/resume aware. Section [Suspend/resume awareness of GPU, AHCI, and NVMe drivers] explains this achievement for the AHCI, NVMe, and Intel GPU drivers. Further highlights of the release are the much improved handling of HID events including the generalized calibration of motion events, API safety improvements, the prospect of de-privileged tracing in Sculpt OS, and multi-client support for Vivante GPUs. On our road map, we had scheduled two further topics that are notably absent, namely USB and SMS. But don't fret. Even though the large rework of our USB infrastructure for fine-grained and dynamic USB access has been completed just in time, we felt that this far-reaching change should better not be rushed into the release. It will be merged shortly after, and settle into the upcoming Sculpt OS version 24.04 just fine. The second topic not covered is SMS support for the PinePhone, which is a topic actively [https://github.com/genodelabs/genode/issues/5127 - worked on] but with no user-visible effect until its integration in Sculpt OS in April. Revised audio infrastructure ############################ After first introduced in version [https://genode.org/documentation/release-notes/10.05#Device-class_interfaces_for_NIC_and_Audio-out - 10.05], Genode's [https://genode.org/documentation/genode-foundations/23.05/components/Common_session_interfaces.html#Audio_output - audio support] slowly evolved over the years, covering audio mixing in version [https://genode.org/documentation/release-notes/10.11#Audio_mixer - 10.11], leveraging OpenBSD's audio driver since version [https://genode.org/documentation/release-notes/15.05#Audio_drivers_ported_from_OpenBSD - 15.05] and offering the OSS interface as VFS plugin since version [https://genode.org/documentation/release-notes/20.11#Streamlined_ioctl_handling_in_the_C_runtime___VFS - 20.11]. With our recent focus on use cases like [https://genodians.org/jws/2023-11-16-sip-client-for-genode - VoIP on the PinePhone] or [https://genode.org/documentation/release-notes/21.05#Webcam_support - video conferencing], however, we identified limitations that cannot be overcome without an architectural revision. First, in the name of simplicity, we used to tie the inter-component audio interfaces to a fixed sample rate of 44100 Hz. This has recently become a problem because some audio drivers tend to support only 48000 Hz. Second, in latency-sensitive scenarios, we observed that the existing interfaces were prone to effects caused by the drifting of time between the producer and consumer of audio data. One effect are buffer under-runs, which produce audible noise. The other is the slow accumulation of buffered sample data, which increases latency over time (affecting the effectiveness of acoustic echo cancellation) and yields an audible buffer overrun after a while. Third, the mixer is a single client of the audio driver, which makes the mixer dependent on the liveliness of the driver. Therefore, the driver cannot be restarted without also restarting the mixer and - transitively - each client of the mixer. The rigid relation between the audio driver and the mixer also stands in the way of routing audio between different audio devices operated by separate drivers. After having successfully introduced the concept of _pluggable drivers_ for graphics in version [https://genode.org/documentation/release-notes/20.08#The_GUI_stack__restacked - 20.08] and applying the same idea to networking in version [https://genode.org/documentation/release-notes/21.02#Pluggable_network_device_drivers - 21.02], the time was ripe for turning the audio infrastructure upside down. [image audio_vs_recordplay] original layered architecture (left) compared to the new pluggable architecture (right) The new architecture as shown on the right turns the mixer into a self-sufficient resource multiplexer, which offers a service for playing audio and a service for recording audio. Both audio drivers as well as audio applications are becoming mere clients of the mixer. With this architecture, the dynamic starting, removal, and restarting of the driver, of even multiple drivers, is trivially solved. To bridge the gap between audio clients operating at different sample rates, the mixer automatically detects and converts sample rates as needed. Both play and record clients are expected to operate periodically. The number of samples produced per period is up to each client and does not need to be constant over time. The mixer infers the used sample rates and periods by observing the behavior of the clients. It measures the jitter of clients to automatically adjust buffering parameters to attain continuous playback while trying to optimize for low latency. Those runtime-measurements can be augmented by explicit configuration values. Multi-channel playing and recording are realized by one session per channel whereas one channel is used to drive the time allocation while all further channels merely enqueue/obtain data into/from their respective sessions without any synchronous interplay with the mixer. The mixer routes and mixes audio signals produced by play clients to record clients according to its configuration. Typical play clients are an audio player or a microphone driver whereas typical record clients are an audio recorder or an audio-output driver. A simple mixer configuration looks as follows: ! ! ! ! ! ! ! ! ! This configuration defines two signals "left" and "right" that are mixed from the audio input of the matching clients. In the example, each play session labeled as "left" is mixed into the "left" signal. Each node can host an arbitrary number of nodes. The same policy can appear at multiple nodes. A node assigns a signal to a record client. In the example, a record client labeled "left" is connected to the signal "left". The mixer allows for the cascading of nodes. For example, the following signal "lefty" is a mix of the two signals "left" and "right", weighted by respective volume attributes. ! ! ! ! [image mixed_waveforms] Example of the mixer output for a sine wave as the "left" signal (top), a signal mixed 70:30, a signal mixed 30:70, and a square wave as the "right" signal (bottom). The operation and configuration of the mixer is described in more detail by the accompanied README at _os/src/record_play_mixer/_. The inter-component interfaces are located at _os/include/play_session/_ and _os/include/record_session/_. The _gems/run/waveform_player.run_ script illustrates the integration of the mixer by using a waveform generator as multi-channel play client and an oscilloscope as record client. Current state and next steps ---------------------------- The new infrastructure is ready to be exercised by the synthetic example mentioned above as well as by the _audio_out.run_ and _audio_in.run_ scripts located at _repos/dde_bsd/run/_. The OpenBSD-based audio driver can be operated in either of two modes. By default, it is compatible to the old audio in/out interfaces. The new record/play mode can be enabled by setting the 'record_play="yes"' config attribute. Over the next release cycle, we will successively convert the other pieces of the audio stack, in particular the other drivers and the OSS VFS plugin, to the new record and play interfaces. Following this transition, the original audio in/out interfaces will be removed. Sculpt OS as remote test target for the Goa SDK ############################################### The run-stage generalization from [https://genode.org/documentation/release-notes/23.08#Run-stage_generalization - release 23.08], paved the way for the new run-target "sculpt" that allows using Sculpt OS as a remote test target for 'goa run'. Since Goa already placed all the required files for running a scenario into a _var/run_ directory, adding this target merely involved coming up with a solution for synchronizing the run directory with Sculpt OS and getting a hold of the log output. The implementation in Goa is accompanied by a _goa_testbed_ package that starts a remotely-controlled subsystem on Sculpt OS. It particularly hosts a _lighttpd_ and _tcp_terminal_ component. The former is used for run-directory synchronization based on HTTP PUT. The latter provides the log output of the test scenario via telnet. For more details, you may take a look at the corresponding [https://genodians.org/jschlatow/2024-01-29-goa-sculpt - blog post on genodians.org]. In order to integrate support for this mechanism into Sculpt OS, we supplemented the NIC router configuration with a _http_ and a _telnet_ domain. Each of these domains is intended to accommodate a single client. Ports 80 and 23 of the _uplink_ domain are then forwarded to the clients in the _http_ and _telnet_ domain respectively. This is complemented by the _goa_testbed_ preset added to the PC and PinePhone version of Sculpt OS that turns the system into a ready-to-use remote test target. You can see this feature in action in our [https://genodians.org/nfeske/2024-02-15-fosdem-aftermath - FOSDEM talks]. When implementing the Sculpt target in Goa, we also had to come up with a way to supply Goa with the IP address of the remote test target. Goa's modularity w.r.t. custom run stages motivated us to implement a generic mechanism for target-specific options. For this purpose, we added the config variable 'target_opt' that is defined as a Tcl array. The Sculpt target evaluates the array elements 'sculpt-server', 'sculpt-port-http' and 'sculpt-port-telnet'. We further augmented Goa's command-line parsing such that individual elements of the 'target_opt' as well as the 'version' config variables, which are both arrays, can be supplied as command-line arguments. The corresponding arguments follow the pattern '--target-opt-