From cffe847778602dc04f8cafd1e2f4e9865f7279e0 Mon Sep 17 00:00:00 2001 From: Norman Feske Date: Wed, 13 Mar 2019 18:24:03 +0100 Subject: [PATCH] Update documentation for Sculpt CE --- repos/gems/recipes/pkg/sculpt/README | 919 ++++++++++++++++++--------- 1 file changed, 612 insertions(+), 307 deletions(-) diff --git a/repos/gems/recipes/pkg/sculpt/README b/repos/gems/recipes/pkg/sculpt/README index b31219284a..63309eafe9 100644 --- a/repos/gems/recipes/pkg/sculpt/README +++ b/repos/gems/recipes/pkg/sculpt/README @@ -1,8 +1,8 @@ - =================================== - Sculpt with Visual Composition (VC) - =================================== + ===================================== + Sculpt as a community experience (CE) + ===================================== Norman Feske @@ -20,13 +20,24 @@ idea of crafting, molding, and tweaking the system interactively. Starting from a fairly minimalistic and generic base system, this tour through the Sculpt system will cover the following topics: -* A boot image that is a live system, rescue system, and bootstrap system - all in one, -* Ways to tweak and introspect the system, -* Formatting a hard disk or USB stick and storing files on the file system, +* A boot image that is a live system, rescue system, and bootstrap system all + in one, * Connecting to a wired or wireless network, -* Installing and deploying software, and -* Running a guest operating system inside a virtual machine. +* Installing and deploying software, +* Ways to tweak and introspect the system, and +* Managing and accessing storage devices. + + +Community +--------- + +The best place to learn more about using and tweaking Sculpt, to follow the +work of the developers, and to get hold of announcements of new software and +features is the federated Genodians blog: + +:Genodians.org community blog: + + [https://genodians.org] Feedback and contact @@ -47,51 +58,20 @@ Your feedback is appreciated! [https://www.genode-labs.com] A printable PDF version of this document is available at -[https://genode.org/documentation/sculpt-vc.pdf]. - - -Prerequisites -############# - -Sculpt with Visual Composition (VC) is the third of four revisions planned for -2018 with a successively increased ease of use. It features a graphical user -interface for performing fundamental tasks like connecting to a wireless -network, or installing and running software from packages. However, the full -power of the system is accessible only via a textual interface. - - -Vim skills recommended -====================== - -Sculpt VC leverages (a subset of) GNU coreutils, bash, and Vim as the user -interface for sculpting the system. If you are not yet familiar with using -Vim, you may take Sculpt VC as a welcome chance to get your toes wet. To -enjoy the experience, you should be comfortable with the following -operations: - -* Opening and navigating within a text file (moving the cursor, - using '/' to search), -* Using the insert mode to make modifications, -* Reverting accidental modifications ('u' undo), -* Saving a modified file (':w'), -* Opening a file in a secondary buffer (':e'), -* Switching between buffers (':bn' for next, ':bp' for previous), -* Copy and paste ('v' start selection, 'V' start line selection, - 'y' remember selection, 'p' paste remembered selection), -* Exiting Vim (':x' save and exit, ':q!' discard changes). +[https://genode.org/documentation/sculpt-ce.pdf]. Hardware requirements and preparations -====================================== +###################################### -Sculpt VC should be compatible with recent Intel-based PC hardware -featuring Intel graphics, E1000 networking, Intel wireless, and AHCI. +Sculpt CE should be compatible with recent Intel-based PC hardware featuring +Intel graphics, E1000 networking, Intel wireless, and AHCI/NVMe. -It is tested best on laptops of the Lenovo X and T series (X220, X250, -X260, T430, T460, T470). For experimenting with Sculpt, we recommend getting a -refurbished version of one of these. You may also find the unofficial -hardware compatibility list [https://usr.sysret.de/jws/genode/hcl.html] -helpful for finding Genode-compatible hardware. +It is tested best on laptops of the Lenovo X and T series (X220, X250, X260, +T430, T460, T470). For experimenting with Sculpt, we recommend getting a +refurbished version of one of these. You may also find the unofficial hardware +compatibility list [https://usr.sysret.de/jws/genode/hcl.html] helpful for +finding Genode-compatible hardware. Sculpt has been tested with screen resolutions up to 2560 x 1440. Displays with a higher resolution are not expected to work. The sweet spot is a full-HD @@ -105,6 +85,11 @@ Please revisit the BIOS settings of your machine in the following respects: :VT-x enabled: Hardware-assisted virtualization is needed to run VirtualBox on top of Sculpt. +:Execution prevention enabled: The standard Sculpt package is provided + for x86-64 and expects your platform to support data execution + prevention (abbreviated DEP or NX). If this feature is disabled your + PC will just reboot on startup. + :Boot from USB enabled: Sculpt is usually booted from a USB stick. :UEFI boot enabled: Sculpt boots via UEFI by default. The boot image @@ -125,121 +110,176 @@ Getting a first impression Sculpt is best explored by first booting the prebuilt disk image downloadable from [https://genode.org/download/sculpt]. -Right after booting the system, Sculpt's system-management user interface +Right after system boot, Sculpt's system-management user interface ("Leitzentrale") appears. The menu on the left provides convenient access to the connected storage devices and the network configuration. The center -displays a live graph (runtime view) of the running components and their -relationships. On the right, diagnostic messages are presented. +displays a live graph (runtime view) of components running and their +relationships. On the right side, diagnostic messages are presented. Consider +the following steps as a warm-up with Sculpt OS. -# Select the in-memory file system as default storage location by clicking - on the "ram" item of the "Storage" dialog and enabling the "Use" button. - This instructs Sculpt that installed software is stored in memory without - accessing any real storage device. +Select the in-memory file system as *default storage location* by clicking +on the "ram" item of the "Storage" dialog and pressing the "Use" button. +This way, software will be installed solely into memory without accessing any +real storage device. -# Enable networking in the "Network" dialog by selecting the "Wired" or - "Wifi" option. In the latter case, select an access point and enter the - corresponding passphrase (if needed). The successful network connection is - indicated by the IP address displayed at the bottom of the network dialog. +Enable *networking* in the "Network" dialog by selecting the "Wired" or +"Wifi" option. In the latter case, select an access point and enter the +corresponding passphrase (if needed). A successful network connection is +indicated by the IP address displayed at the bottom of the network dialog. -# With a storage location selected and network connectivity, it is - time to install and start additional components by clicking on the '+' - button of the runtime view and selecting a component from the - context menu. When started for the first time, the ingredients of the - selected subsystem are downloaded to the "used" storage location. - Once the download is complete, the subsystem is started. As a - first try, select the "backdrop" item. You can follow the progress of the - installation procedure in the "Runtime" dialog. Once the installation is - complete, you should notice a slight visual change. +With a storage location selected and established network connectivity, it is +time to *install and start* additional components by clicking on the '+' +button of the runtime view. Select "Depot ..." from the menu. -# Press F12 to toggle between the Leitzentrale and the actual runtime. - Now, the backdrop should become visible in full glory. +[image sculpt_ce_menu 40%] -# Try adding additional components by selecting items in the "+" context - menu of the runtime view. Most components expect the presence of a - window manager. Hence, you should first select the "wm", "window_layouter", - and "motif_decorator" components to make the window-management functionality - available. The distinct roles of the three components are described below. +The depot contains software packages, which can be obtained by different +independent software providers. The selection of software providers is +completely up to the user and can be defined in the "Selection ..." sub menu. - Please pay attention to diagnostic messages given in the runtime dialog on - the left. Whenever a component depends on another one, a corresponding - message appears. +[image sculpt_ce_select 40%] -# You may click on any component in the runtime view to reveal additional - information such as its memory usage. For components that you started - manually, a remove button is displayed. +Select "genodelabs" to download the directory of software officially +provided by Genode Labs. Note that the other options are not necessarily +affiliated with Genode Labs. Think of each option as a different individual. +If you don't trust one particular software provider, you can still install and +use the provided software without risk as long as you don't explicitly grant +their components access to sensitive parts of your system. The judgement +of trust is entirely yours. -The following example subsystems are available from the "+" menu: +When now going back to the depot menu, a new menu item "genodelabs ..." +appears. It leads to a catalogue of software browsable via hierarchically +structured menus. As a starter, let's add a desktop background. +In the "GUI ..." sub menu, a click on the first item named "sticks blue +backdrop" reveals the option to install the package. -:'fonts_fs': A file-system server that transforms TrueType fonts into - glyph images, which become thereby accessible as virtual files. - This provides a hook for customizing the font size of any component that - uses the font server, and relieves components from depending on a specific - font-rendering library. According to the '' information, its - configuration is taken from _/config/managed/fonts_. The 'fonts_fs' - is used by the graphical terminal of the noux subsystem and the 'top_view' - application. +[image sculpt_ce_install_backdrop 40%] -:'wm': A component that allows for the creation of windowed graphical - user interfaces. It must be combined with a window layouter and a window - decorator (see below). +A click on the "Install" button triggers the download of the package and its +dependencies. Once the download is complete, the menu presents a configuration +dialog that allows you to define the interplay of the new component with the +system. In this particular case, you have to decide for a GUI service to be +used by the backdrop. -:'window_layouter': A component that defines the behavior and layout policy - of the window manager's windows. By default, each application is hosted in a - floating window that can be moved, resized, and re-stacked with the mouse. +[image sculpt_ce_backdrop_routes 40%] -:'motif_decorator': A window decorator component that defines how window - decorations look. It is inspired by the simplistic look of traditional - Motif-based graphical user interfaces. +The first option "system GUI server" would grant direct access to the system's +low-level GUI server, which is normally not used by applications but by +higher-level GUI servers like a window manager. The second option would +connect the component to the special "desktop background" GUI session, which +appears as a layer behind all other applications. The third option "keyboard +focus" is preserved for a single component that controls the keyboard focus. +In our case, "desktop background" is the correct choice. Once the +configuration is complete, a new button for adding the component to the system +appears. -:'themed_decorator: A modern looking window layouter that can be used as - an alternative to the 'motif_decorator'. +[image sculpt_ce_add_backdrop 40%] -:'backdrop': A wallpaper that adjusts itself to any screen size. +After pressing the button, you should notice a slight visual change. *Press* +*F12* to toggle between the Leitzentrale and the desktop. Now, the backdrop +should become visible in full glory. In the runtime graph, the new component +appears connected to the "GUI". A click on the component reveals further +information along with the option to remove it from the system. -:'nano3d': A simple software-rendering demo, which can be adjusted at runtime - by modifying its configuration via the textual interface described in - Section [Runtime management]. For example, by adding a custom config node - directly inside the '' node, the appearance can be changed on the fly: - ! +[image sculpt_ce_backdrop_selected 40%] -:'noux-system': A noux instance with a graphical terminal, similar to the - inspect window of the Leitzentrale. Note the routing of the various - file-system sessions when selecting the component in the runtime view. +As a next step, let us add a window system. In the '+' menu, you can find +a readily packaged window system at _genodelabs_ -> _GUI_ -> _themed wm_. +After installing the package, you are asked to take three decisions: -:'shared_fs': A file-system server that provides the _/shared_ sub directory - of the Sculpt file system as a new file system. A client of this server - will not see any other parts of the file system. +The _GUI (focus)_ should be assigned to "keyboard focus" to put the +window manager in charge of controlling the keyboard focus, which is part +of its job after all. +The _GUI_ should be assigned to "system GUI server" as the basic mechanism +to be used for graphical output and user input for the windowed applications. +Finally, by assigning _Report_ to _system reports_, we allow the window +manager to report information about mouse-pointer shapes. +By adding the component, the "themed wm" will appear in the runtime view. +To give the window system a quick try, add the small demo you can find at +_genodelabs_ -> _Demos_ -> _nano3d_ and assign its _GUI_ to our "themed wm". -:'usb_devices_rom': A hook for assigning USB devices to a virtual machine, - explained in Section [Updating the USB boot device from within VirtualBox]. +[image sculpt_ce_nano3d 40%] -:'vm_fs': A file-system server that provides the _/vm/debian/_ sub directory - of the Sculpt file system as a new file system. It is explained in Section - [Hosting a guest operating system]. +Next, let us add a *small Unix-like subsystem* called _noux_ hosted in a +window. This noux runtime will be presented in a terminal window. The font +used by the terminal is obtained from a font server. To create the font +server, install and add the package _genodelabs_ -> _GUI_ -> _fonts fs_, and +assign the _system font configuration_ to its _ROM (config)_. "ROM" means +read-only memory. So we grant the font server read-only access to the system's +default font configuration (which is generated automatically according to the +screen resolution). -:'top_view': An application that shows the CPU load, similar to 'top'. +For starting the actual noux runtime, select and install +_genodelabs_ -> _Tools_ -> _noux-system_ from the menu. The configuration +dialog is a bit more elaborate this time. -:'2048': A _Threes!_ inspired puzzle game running in a native Libretro runtime. +:GUI: defines the GUI server that should host the terminal. + Select "themed wm". -:'vbox5-tc-browser': A throw-away virtual machine for running Firefox on - TinyCore Linux. It uses VirtualBox as virtual-machine monitor. +:File system (config): defines which file system should be mounted at + _/config/_ inside the instance. There exist a number of options. + By selecting _writeable system configuration_, we grant control over + the whole system. It goes without saying that this should not be done + lightheartedly. However, since we trust the "noux-system" package from + Genode Labs, let's do it. -:'seoul-tc-browser': The same virtual machine as 'vbox5-tc-browser' but - executed inside the light-weight Seoul virtual-machine monitor. +:File system (report): defines the file system to be mounted at _/report/_ + inside the instance. By selecting "read-only system reports", we allow + the instance to look at the state of various parts of the system. -:'config_editor': Qt5-based text editor that is explicitly granted access to - the config file system. +:File system (target): defines a file system to be mounted at _/rw/_. + This can be any file system you'd like to work with or explore, for + example the "ram fs". -:'arora': Qt5-based web browser, which does not touch any persistent file - system. +:File system (fonts): defines the place where to obtain the font used by + the terminal. Select the "fonts fs" component we have started earlier. -:'acpica': ACPI driver, which reports power-management state to - _/report/runtime/acpica/_ and responds to changes of the _/config/system_ - state. +:ROM (vimrc): defines the configuration for the vim text editor used + within the instance. Select "default vim configuration" to grant + read-only access to this information. -:'report_dump': A subsystem that periodically copies the content of the - report file system to the default file system. Please refer to Section - [Sculpt as a hardware-probing instrument] for more information. +With those decisions taken, a fresh noux runtime can be started, which appears +in a window. + +[image sculpt_ce_noux 60%] + +When selecting the "noux-system" component in the runtime view, the +relationship to the other components of the system is presented. This provides +a convenient way to reveal the _trusted computing base_ of the selected +component. For example, since there is no connection from _noux-system_ to the +_nic_router_ we know that this component is isolated from the network. The +network-related components are outside the trusted computing base of the +noux instance. + +[image sculpt_ce_noux_selected 50%] + + +Further exploration +------------------- + +Of course, there are many more components to explore and to combine. +For inspiration, please follow the postings at +[https://genodians.org], for example: + +:Use GNU/Linux inside a virtual machine on top of Sculpt: + + There is a ready-to-use package for downloading Debian for the use inside + a virtual machine along with the ability to use VirtualBox guest + additions: + + [https://genodians.org/jws/2019-03-01-download_debian-guest-additions] + + For configuring and starting the virtual machine, you may find the + following guide useful: + + [https://genodians.org/m-stein/2019-03-07-vm-with-sculpt-ce-preview] + +:Disposable Firefox VMs: + + Use a minimalistic Tinycore-Linux system to run Firefox in memory without + access to any persistent storage: + + [https://genodians.org/alex-ab/2019-03-06-disposal-browser-vm] Base system @@ -369,18 +409,13 @@ The Leitzentrale can be toggled at any time by pressing F12 and will be enabled right after boot. It presents itself with a minimalistic GUI for accessing the storage devices attached to your machine and for configuring your network connectivity. Most importantly, however, it allows the user to spawn an -interactive shell for manual _config_ and _report_ file -systems access. To spawn this command-line interface, click on the "ram" item from -the menu and select "Inspect". -While inspecting file systems, the inspect window replaces the runtime view. -However, both views can be toggled by clicking on the title of the storage dialog -for the inspect window, or any other dialog for the runtime view. +interactive shell for manual _config_ and _report_ file systems access. To +spawn this command-line interface, click on the "ram" item from the menu and +select "Inspect". While inspecting file systems, the inspect window replaces +the runtime view. -[image noux 45%] - Noux runtime environment for executing Unix tools - -The inspect window hosts a small Unix runtime called noux (Figure [noux]) -as user interface. Don't let the presence of a Unix shell mislead you. +The inspect window hosts a small Unix-like runtime called noux as user +interface. Don't let the presence of a Unix shell mislead you. Sculpt is not a Unix system. It merely uses Unix subsystems in the form of noux instances as convenient tools for managing and editing files. Within the inspect window, you can interact with both the report and @@ -399,7 +434,28 @@ Tweaking and inspecting the system ================================== The Leitzentrale subsystem empowers you to interactively inspect and tweak -the running system. Let's take a walk next. +the running system using a command-line interface and the Vim text editor. + + +Vim skills recommended +---------------------- + +Sculpt CE leverages (a subset of) GNU coreutils, bash, and Vim as the user +interface for sculpting the system. If you are not yet familiar with using +Vim, you may take Sculpt CE as a welcome chance to get your toes wet. To +enjoy the experience, you should be comfortable with the following +operations: + +* Opening and navigating within a text file (moving the cursor, + using '/' to search), +* Using the insert mode to make modifications, +* Reverting accidental modifications ('u' undo), +* Saving a modified file (':w'), +* Opening a file in a secondary buffer (':e'), +* Switching between buffers (':bn' for next, ':bp' for previous), +* Copy and paste ('v' start selection, 'V' start line selection, + 'y' remember selection, 'p' paste remembered selection), +* Exiting Vim (':x' save and exit, ':q!' discard changes). Adjusting the user-input handling @@ -491,14 +547,352 @@ the following tweaks: * Change the font size of the 'log_terminal' component from "10" to "18". -You may also enjoy tinkering with the configuration of the nitpicker GUI +You may also enjoy tinkering with the configuration of the Nitpicker GUI server, which is located at _/config/nitpicker_. For example, you may change the background color or the labeling color of the "default" domain. +System resources +================ + +Whenever adding a new component via the '+' menu, one has to define how to +connect the component with the rest of the system. It is important to know +what the presented options mean to take educated decisions. + +[image sculpt_ce_noux_routing 40%] + +Each choice represents a connection to a system resource of a particular type. +Initially, the presented options are resources that are built-in into Sculpt's +base system. Once new components enter the picture, their services also appear +as options. + + + Resource type | Interface | Built-in options + ---------------------------------------------------------------------------- + ---------------------------------------------------------------------------- + GUI | 'Nitpicker' | keyboard focus + ---------------------------------------------------------------------------- + | | desktop background + ---------------------------------------------------------------------------- + | | system GUI server + ---------------------------------------------------------------------------- + ROM | 'ROM' | global capslock state + ---------------------------------------------------------------------------- + | | default vim configuration + ---------------------------------------------------------------------------- + | | system font configuration + ---------------------------------------------------------------------------- + | | platform information + ---------------------------------------------------------------------------- + | | system status + ---------------------------------------------------------------------------- + Report | 'Report' | system reports + ---------------------------------------------------------------------------- + File system | 'File_system' | writeable system configuration + ---------------------------------------------------------------------------- + | | read-only system reports + ---------------------------------------------------------------------------- + Real-time clock | 'Rtc' | system clock + ---------------------------------------------------------------------------- + Block device | 'Block' | direct block-device access + ---------------------------------------------------------------------------- + USB | 'Usb' | direct USB-device access + ---------------------------------------------------------------------------- + Device access | 'Platform' | wifi hardware + ---------------------------------------------------------------------------- + | | network hardware + ---------------------------------------------------------------------------- + | | audio hardware + ---------------------------------------------------------------------------- + | | ACPI + ---------------------------------------------------------------------------- + Region maps | 'RM' | custom virtual memory objects + ---------------------------------------------------------------------------- + Direct memory-mapped I/O | 'IO_MEM' | raw hardware access + ---------------------------------------------------------------------------- + Direct port I/O | 'IO_PORT' | raw hardware access + ---------------------------------------------------------------------------- + Direct device interrupts | 'IRQ' | raw hardware access + ---------------------------------------------------------------------------- + Tracing | 'TRACE' | system-global tracing + ---------------------------------------------------------------------------- + Hardware virtualization | 'VM' | virtualization hardware + ---------------------------------------------------------------------------- + Network | 'Nic' | + ---------------------------------------------------------------------------- + Terminal | 'Terminal' | + ---------------------------------------------------------------------------- + Audio input | 'Audio_in' | + ---------------------------------------------------------------------------- + Audio output | 'Audio_out' | + +[table resources] + Overview of system resources + +Table [resources] gives an overview of the different built-in +resources and their types. The names given in the interface column correspond +to the Genode session types as found in the routing rules of the deploy +configuration or launcher definitions (Section [Runtime management]). +Let's look into each type in more detail. + + +GUI +~~~ + +The GUI service interface is provided by the low-level system GUI server +(named Nitpicker) as well as the higher-level window manager. It entails both +the ability to perform graphical output and the reception of user input. Note +that the low-level GUI server keeps its client applications isolated by +default. One application cannot see the output of other applications nor can +it sniff user input globally. One can connect multiple applications - +trusted and untrusted alike - to the low-level GUI server without fear. + +However, in typical scenarios, applications don't use the bare-bone system GUI +server directly but rather employ a window manager that sits in-between the +system GUI server and the applications, and equips the system with the +notion of windows. + +The base system provides three different GUI options. + +:keyboard focus: grants control over the keyboard focus. It should be + assigned to only one component, typically a window manager. However, + in principle, another component like _noux-system_ can be connected + to it and thereby becomes able to receive keyboard input. + +:desktop background: allows a component to present its graphical output + in a dedicated layer behind all other applications. The desktop background + cannot receive keyboard focus. But it can respond to pointer events + (mouse clicks and motion). + +:system GUI server: allows a component to perform graphical output and + handle pointer events, but no keyboard input. It is designated as a + base mechanism for the window manager, or for implementing GUI features + like global overlays or status displays. + +By the way, the configuration of the low-level GUI server can be found at +_/config/nitpicker_ and can be modified on the fly. + + +ROM +~~~ + +ROM stands for read-only memory. A ROM service reveals information to +its clients but a client cannot change the information. Note that the +provided information does not need to be static. It can potentially +change over time. Whenever that happens, the ROM service informs its +clients about the availability of a new version. The base system provides +the following built-in ROM resources: + +:global capslock state: the system-global state of the capslock key. + It can be handed out to components like virtual machines to keep the + capslock state of guest operation systems consistent with the host. + +:default vim configuration: the configuration of the Vim text + editor as used in the Leitzentrale's inspect window and managed at + _/config/vimrc_. It allows you to customize one vim configuration at a + central place and use this configuration consistently across Sculpt's + inspect window and manually deployed components. + +:system font configuration: is the default font configuration managed + by Sculpt according to the current screen resolution. It is of course + customizable by the user. By using this configuration, components can + foster a consistent user experience regarding the display of text. + +:platform information: provides details about the underlying kernel and + hardware. Some drivers and virtual machine monitors need this information to + take platform intrinsics like the concrete flavor of virtualization hardware + into account. Normal applications should never need this information. + +:system status: reflects the system-global power state. It is used by the + optional ACPICA driver to respond to requests for a system reset or + power-down. + + +Report +~~~~~~ + +Report services play the counterpart of ROM services. They allow clients +to report information in a fire-and-forget fashion, but not to retrieve +information. As explained in Section [System overview], the base system +includes a report service that aggregates incoming reports into the in-memory +report file system. The incoming reports are organized according to their +origin (their session labels). By granting a component access to the _system_ +_reports_, the component can contribute to this knowledge base. However, +keep in mind that Sculpt's built-in report file system is limited in size. +A misbehaving component may put the system in jeopardy by producing overly +sized reports. + +Reports labeled with 'shape' play a special role. They are routed to the mouse +pointer and thereby enable graphical applications to suggest context-specific +pointer shapes. The application-provided shape is shown whenever the +corresponding application is hovered. + + +File system +~~~~~~~~~~~ + +A file-system service offers the storage and retrieval of data organized in +a hierarchical directory structure. Access to a file system can be restricted +to be read only. The distinction between read-only and read-writable does +not exist per file but for the entire file-system resource. + +Of course, many use cases call for finer-grained access control. This is +where intermediate _chroot_ file-system components come into play. Such +a component reduces the view on an existing file system to a specific +sub directory, which then appears as a new file system to the application. +For examples, please review the launchers at _/config/launcher/_ such as the +_shared_fs_. + +The Sculpt base system has two built-in file systems. + +:writeable system configuration: corresponds to the config file system + described in Section [System overview]. The ability to access this file + system equals to total control over the system. Hence, never assign + this file system to components that you don't fully trust. + +:read-only system reports: allows the client to inspect the global state of + the system. The reports found in this file system are organized in a + directory structure that corresponds to the system structure. For example, + all reports generated by the runtime sub system reside within the _runtime/_ + directory. Note that this global state may contain sensible information. + For this reason, the system reports should not be handed out to components + that are suspected of information leakage. + +The two built-in file systems reside in memory. In order to access persistent +storage, additional file-system services can be started as regular components +within the runtime subsystem. Those components, in turn, need to be connected to +the corresponding block devices. + + +Block device +~~~~~~~~~~~~ + +A block-device resource allows for the block-level storage and retrieval +of data whereas a block-device can be read-only or read-writable. The base +system's built-in block service hands out the devices listed at +_/report/drivers/block_devices_ according to the label of the client's session. +The label must correspond to the name of the block device. Typically, the +built-in block service is used via manually created launchers by using a route +like this: + +! +! +! + + +Device access +~~~~~~~~~~~~~ + +With Sculpt, device drivers can be installed and used like any regular +component. In contrast to plain applications, however, device drivers need +to access the corresponding device hardware. This access is guarded by the +so-called platform driver hosted in the _drivers_ subsystem. The platform +driver has two purposes. +First, it uses the IOMMU to isolate devices from each other and to restrict +the reach of each device to the memory explicitly assigned to the device +(DMA buffers). +Second, it arbitrates the access of device-driver components to devices. + +Regarding the latter, the platform driver differentiates four categories +of drivers, namely wifi, network, audio, and ACPI. When assigning one +of those resources to a driver component, the driver will see a virtual +PCI bus with only the devices that fall in the chosen category. + +Note that USB, AHCI, NVMe, and PS/2 are handled differently as those +devices are driven by Sculpt's _drivers_ subsystem. + + +USB +~~~ + +By connecting a component to USB, the component becomes able to access +individual USB devices. The device is selected by the component by specifying +a session label that corresponds to a name of a device listed at +_/report/drivers/usb_devices_. The most prominent use case for the USB +resource is the direct assignment of USB devices to virtual machines +as described in Section +[Updating the USB boot device from within VirtualBox]. + + + +Real-time clock +~~~~~~~~~~~~~~~ + +The real-time clock enables a component to know what time it is. + + +Region maps +~~~~~~~~~~~ + +The region-map service of the base system gives components a flexible way to +manage their virtual address spaces manually. This mechanism is used by a few +advanced components only, most specifically virtual machine monitors. Access +to the region-map service is not security critical. But as it is rarely +needed, it is not granted be default to limit the potential (attack) surface +of the base system as far as possible by default. + + +Direct memory-mapped I/O, port I/O, and device interrupts +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +These low-level services are provided by Genode's core component. They +should never be needed by any regular component. Even device drivers +don't use those services directly but rather rely on the higher-level +device-access service described in Section [Device access]. + +However, a few special use cases demand for such low-level access. In +particular the use of ACPI functionality. + +As a rule of thumb, never grant access of those resources to any component +except you know exactly what you are doing and you completely trust the +provider of the component. + + +Tracing +~~~~~~~ + +The low-level tracing interface allows a component to observe and to +manipulate all activities in the system. It should only be granted to +components that are fully trusted. + + +Hardware virtualization +~~~~~~~~~~~~~~~~~~~~~~~ + +The hardware-virtualization service allows virtual machine monitors to +leverage virtualization technology (Intel VT). + + +Network +~~~~~~~ + +Network services provide an interface for sending and receiving network +packets. Note that the base system does not provide such a service. However, +Sculpt's Leitzentrale conveniently manages drivers for wireless (wifi drv) and +wired (nic drv) networking as well as the user-level network routing component +(nic router). So you usually see those options. The drivers should not be +used directly while the NIC router is running because they only accept +one client at a time. However, the NIC router multiplexes the network access +and multiple network applications can be connected to the NIC router to +reach the network. + + +Terminal, audio input, and audio output +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Terminal services allow for the input and output of streams of text. Examples +are a graphical terminal, or an UART driver. The base system has no built-in +terminal service. + +As the names suggest, audio input and output enable components to record +and play audio. The base system does not provide such services. Instead, +audio infrastructure like drivers and a mixer can be installed as regular +components. + + Runtime management -################## +================== [image sculpt_runtime_highlighted] @@ -525,37 +919,37 @@ in the menu. For the start, it is best to select the "ram" file system as storage location. Once you are comfortable with Sculpt, you may make the installation and customizations permanent by using a real storage device instead. + The selection of a "used" file system has two immediate effects. First, any -files present at _config//_ at the selected file system are copied to +files present at _config//_ at the selected file system are copied to the config file system. As the RAM file system is empty, no files are copied. Second, the so-called _depot/_ is initialized at the selected file system. The depot is the designated place for the installation of software packages. By default, the depot is initialized such that the Sculpt system accepts software -published by Genode's core developers. You may inspect the content of -_/ram/depot_ using the inspect window. +from Genode Labs as well as from a few members of the Genode community. You +may inspect the content of _/ram/depot_ using the inspect window. With a file system and an Internet connection selected, additional software -can be installed and run. The primary interface for software installation and -deployment is the _/config/deploy_ file and the so-called launchers located at -_/config/launcher/_. The deploy file contains a number of commented-out -template snippets for various subsystems. As a first test, uncomment the -'' entries for the _fonts_fs_, _wm_, and _backdrop_. When saving the -file, the Sculpt manager will automatically kick off the download of the -selected packages and its dependencies and thereby populate the depot. Once -the download has completed, the packages are started. +can be installed and run. The most convenient way is the interactive use of +the '+' menu to browse the catalogues of packages provided by software +providers and to configure new component instances. -Each '' node refers a launcher according to the 'name' attribute. It is -possible to explicitly refer to a differently named launcher by specifying a -'launcher' attribute. This way, one launcher can be instantiated multiple -times. Pay special attention to the '' definitions in the launchers. -They define how the respective subsystem is connected to other parts of the -system. For example, by default, the launcher for the backdrop connects the -component directly to the nitpicker GUI server of the base system (parent). By -changing the route from '' to '' the backdrop -subsystem will be connected to the window manager instead. +Additionally, the deployment can be controlled +by the _/config/deploy_ file and the so-called launchers located at +_/config/launcher/_. +The _deploy_ file contains a '' node for each running component. +Such a node specifies the package, the assigned resources, +and the rules of how the component is connected with other components. + +Alternatively, a node may refer to a launcher that encapsulates +this policy as a separate file at _/config/launcher/_. +By default, the launcher corresponds to the 'name' attribute of the start node. +It is possible to explicitly refer to a differently named launcher by +specifying a 'launcher' attribute. This way, one launcher can be instantiated +multiple times. The files at _/config/launcher/_ are monitored by Sculpt and therefore can be -edited on the fly. This is especially useful for editing '' nodes. - +edited on the fly. This is especially useful for editing '' nodes +of already running components. A '' node within a launcher - when present - overrides the one provided by the package. In turn, a '' node within a node of the deploy config overrides any other '' node. Both the launcher and a @@ -564,14 +958,25 @@ deploy config overrides any other '' node. Both the launcher and a way, the routing of a launcher can be parameterized at the deploy configuration. -Under the hood, the deployment is not directly controlled by _/config/deploy_. -Instead, Sculpt incorporates the user interaction with the runtime view and -the information provided by _/config/deploy_ into the actually used deploy -configuration at _/config/managed/deploy_. Note that any modification of -_/config/deploy_ resets _/config/managed/deploy_ to the state defined -in _/config/deploy_. To preserve interactive changes, you may copy -_/config/managed/deploy_ to _/config/deploy_ before tweaking _/config/deploy_ -manually. +Each time the _/config/deploy_ file or a launcher file is written, the change +takes immediate effect. In particular, the Sculpt manager will automatically +kick off the download of the referenced packages and its dependencies and +thereby populates the depot. Once the download has completed, the packages are +started. + + +Interplay of the deploy configuration and interactive changes +------------------------------------------------------------- + +The content of _/config/deploy_ is taken as the starting point for the +interactive use via the '+' menu. All interactive changes to the deployment +are reflected in the _/config/managed/deploy_ file. +Note that any modification of _/config/deploy_ resets _/config/managed/deploy_ +to the state defined in _/config/deploy_. + +Launchers appear at the top level of the '+' menu. So they are a convenient +mechanism to quickly add often-used components with specific policies to the +system. Storage device access and preparation @@ -585,7 +990,8 @@ the drivers subsystem. A click on a device reveals possible operations or - if a partition table is present - more details about the device structure. Depending on the operation selected by the user, the Sculpt manager will -automatically reshape the runtime subsystem to perform the selected operation. +automatically spawn helper components in the runtime to perform the selected +operation. For example, by selecting the "Format device" operation, the Sculpt manager will create a noux instance with the selected block device mounted at '/dev/block' and e2fsprogs mounted at '/'. The noux instance runs @@ -617,12 +1023,16 @@ Making customizations permanent It is possible to make any customization of the config file system permanent by copying the modified files to a directory named -'config/' on a persistent file system where '' corresponds -to the Sculpt version number as found in the '/VERSION' file. -Each time, this file system is selected for "Use", those files will be -automatically copied to the in-memory config file system. Note that this -mechanism works even for the '/config/deploy' file, which allows one to -restore a once sculpted system composition directly at boot time. +_config/_ on a persistent file system where __ corresponds +to the Sculpt version number as found in the _/VERSION_ file. +Each time this file system is selected for "Use", those files will be +automatically copied to the in-memory config file system. + +The most important customization is the system composition, usually created +via the '+' menu. To make it permanent, copy the current state of +_/config/managed/deploy_ to _//config//deploy_ where __ +corresponds to your Sculpt partition. This deploy configuration will take +effect whenever the Sculpt partition is selected for "Use". To streamline the boot procedure into a customized Sculpt system even more, it is possible to mark one file system as default. At boot time - when the @@ -640,84 +1050,6 @@ the last partition of the Sculpt USB stick can be marked as default or non-default using this button. -Hosting a guest operating system -################################ - -The default deploy configuration found at _/config/deploy_ and the launcher -at _/config/launcher/vm_ contain all the pieces needed to host a virtual -machine on top of Sculpt. A virtual machine (VM) is a convenient stop-gap -solution for running programs that are not yet available natively on Genode. -It ultimately enables us Genode developers to use Sculpt as day-to-day OS. - -By convention, we host the content of each VM in a dedicated -directory _/vm//_ at the file system. The VM directory contains -a virtual disk image(s) as well as the VM configuration. -To install the ingredients for running Debian in the VM, -you may start the 'download_debian' subsystem, which will automatically -download the ISO image of the Debian installer and install a reasonable -VM configuration. The subsystem requests a file-system session -that points to the target directory. To pass the _/vm/debian_ directory -to the subsystem, the file-system session is routed to the 'vm_fs' component. -Please make sure to uncomment this component along with the 'download_debian' -subsystem. - -Please review and adjust the _machine.vbox_ file as needed, in particular -you may reconsider the amount of RAM by changing the 'RAMSize' attribute. -To start the VM, remove the comments around the following snippets within -_/config/deploy_: - -# "wm" - the window manager that will host a window of the VM, -# "vm_fs" - the location of the virtual-disk image and VM configuration, -# "shared_fs" - the location for sharing files between the guest OS and - other parts of the Sculpt system, -# "usb_devices_rom" - a configuration that contains a list of USB devices - passed to the VM, -# "vm" - the virtual machine. - -After saving the file, VirtualBox should appear, starting the Debian -installer. - -After the installation is finished and the guest system was rebooted, it is -time to install the guest additions of VirtualBox. To do that, the apt(1) -configuration has to be adjusted. Edit the file - -! # vi /etc/apt/sources.list - -and add the line - -! deb http://ftp.debian.org/debian stretch-backports main contrib non-free - -Afterwards update the package cache - -! # apt update - -and upgrade the packages - -! # apt upgrade - -and install the Linux kernel headers - -! # apt install linux-headers-amd64 - -Just to be sure that the guest additions will use the newest kernel, reboot -the guest system. Next, install all needed packages for the guest -additions: - -! # apt install virtualbox-guest-dkms virtualbox-guest-x11 - -Having the Linux-header package is mandatory as the needed modules will not -be built without it. After the packages are installed and the modules have -been built, certain features like the dynamic mode-setting and shared -folders can be used. - -The example _machine.vbox_ file already provides a configured shared folder -called 'shared'. By executing - -! # mount -t vboxsf shared /mnt/ - -it can be mounted and accessed via '/mnt'. - - Advanced usage ############## @@ -735,12 +1067,12 @@ work flows and configuration tweaks are largely automated. For example, This convenience comes at the price of built-in policy, which may stand in the way of sophisticated scenarios. For this reason, almost all policies -of the Sculpt manager can be manually overriden by manually managing -configuration files. +of the Sculpt manager can be overridden by manually managed configuration +files. The Sculpt manager interacts with the rest of the system solely by monitoring reports aggregated in the report file system, and writing -configuration files into the config file systems. All files generated +configuration files into the config file system. All files generated by the Sculpt manager are located at _/config/managed/_. By manually creating a same-named file at _/config/_, one can supply a custom managed configuration to the Sculpt manager. A useful practice is taking a snapshot of the @@ -787,18 +1119,18 @@ a free download at [https://genode.org]. ! git clone https://github.com/genodelabs/genode.git ! cd genode - ! git checkout -b sculpt_vc sculpt_vc + ! git checkout -b sculpt_ce sculpt_ce # Download the support for the NOVA microkernel - ! ./tool/depot/download genodelabs/bin/x86_64/base-nova/2018-09-19 + ! ./tool/depot/download genodelabs/bin/x86_64/base-nova/2019-03-17 The content is downloaded to the _public/_ directory and extracted to the _depot/_ directory. # Download all ingredients for the Sculpt boot image - ! ./tool/depot/download genodelabs/pkg/x86_64/sculpt/2018-09-21 + ! ./tool/depot/download genodelabs/pkg/x86_64/sculpt/2019-03-19 # Create a build directory @@ -837,7 +1169,7 @@ in particular for customizing the system. Before building the packages, various ports of 3rd-party software need to be prepared. The following command prepares all of them at once: -! /tool/ports/prepare_port \ +! /tool/ports/prepare_port \ ! bash coreutils curl dde_ipxe dde_linux \ ! dde_rump e2fsprogs gnupg grub2 jitterentropy \ ! libarchive libc libgcrypt libiconv libssh \ @@ -854,18 +1186,18 @@ The _repos/gems/run/sculpt.run_ script can be executed to build a boot image. By default, the boot image refers to 'genodelabs/pkg/sculpt' and to 'genodelabs/pkg/sculpt-installation' for the runtime-installed software. You may want to install your version of these packages instead by changing the -package provider from 'genodelabs' to '' by adding the line +package provider from 'genodelabs' to '' by adding the line -! RUN_OPT += --depot-user +! RUN_OPT += --depot-user to your _/etc/build.conf_. To build the packages for the boot image: -! /tool/depot/create \ +! /tool/depot/create \ ! UPDATE_VERSIONS=1 FORCE=1 REBUILD= \ -! /pkg/x86_64/sculpt \ -! /bin/x86_64/base-nova +! /pkg/x86_64/sculpt \ +! /bin/x86_64/base-nova The 'FORCE=1' argument ensures that source archives are re-created and checked for the consistency with their versions. If the source code of any @@ -879,9 +1211,9 @@ to parallelize the build process where '' is the level of parallelism. Building the 'sculpt-installation' package works analogously to the 'sculpt' package. -! /tool/depot/create \ +! /tool/depot/create \ ! UPDATE_VERSIONS=1 FORCE=1 REBUILD= \ -! /pkg/x86_64/sculpt-installation +! /pkg/x86_64/sculpt-installation To make the 'sculpt-installation' available for download from within the boot image, you must publish it. This involves the archiving, signing, @@ -889,7 +1221,7 @@ and uploading of the content. The former two steps are covered by the _tool/depot/publish_ tool, which expects one to specify a concrete version. The current version of the 'sculpt-installation' can be obtained via -! cat /repos/gems/recipes/pkg/sculpt-installation/hash +! cat /repos/gems/recipes/pkg/sculpt-installation/hash The first part is the version. The second part is the content hash of the version. For more information about working with the depot tool, refer to @@ -898,7 +1230,7 @@ version. For more information about working with the depot tool, refer to The launchers integrated in the boot image are defined at _gems/run/sculpt/launcher/_. Each file contains a node with a mandatory pkg attribute. If the attribute value contains one or more '/' characters, it is -assumed to be a complete pkg path of the form '/pkg//'. +assumed to be a complete pkg path of the form '/pkg//'. Otherwise it is assumed to be just the pkg name and is replaced by the current version of the current depot user's pkg at system-integration time. @@ -924,33 +1256,6 @@ Sculpt ISO image to the device by following the steps described in Section [Building the boot image]. -Sculpt as a hardware-probing instrument -======================================= - -Sculpt can be used as a convenient tool for probing Genode's compatibility -with new hardware via the so-called 'report_dump' subsystem, which -periodically copies the content of Sculpt's report file system to the default -file system. - -First, a USB stick with a fresh Sculpt image is booted on a fully supported -machine. The user then customizes the USB stick within the running system by -expanding the USB stick's Genode partition, setting it as the default -storage location, and deploying the 'report_dump' subsystem. The last step -triggers the installation of the 'report_dump' package onto the USB stick. -Finally, the user copies the deploy configuration from the in-memory config -file system (_/config/deploy_) to the USB stick -(_/usb-/config//deploy_). When booting this prepared USB stick, -this deployment configuration becomes active automatically. At this point, the -Sculpt system will copy a snapshot of the report file system to the Genode -partition of the USB stick every 10 seconds. The snapshots captured on -the USB stick can later be analyzed on another machine. - -The snapshots not only contain all log messages (_/report/log_) but also the -reports generated by various components of the drivers subsystem and any other -deployed components. For example, with 'acpica' present in the deploy configuration, -the battery state is captured as well. - - Credits #######