mirror of
https://github.com/genodelabs/genode.git
synced 2024-12-27 09:12:32 +00:00
7bed3967ae
This patch makes the build-system integration of noux packages usable for non-Noux targets. It moves the GNU build system wrapper to ports/mk/gnu_build.mk, which is now included by noux.mk. This way, non-noux applications can use the same build-system wrapper. So the porting of individual applications becomes easier. This change removes the 'NOUX_' prefix use the various build variables used by the noux-pkg's target.mk files to steer the behaviour of the GNU build system. E.g., NOUX_CONFIGURE_ARGS is now called CONFIGURE_ARGS. Note that there is a single exception to this pattern: The formerly named NOUX_LIBS is now called LDLIBS because the plain LIBS variable is used by the Genode build system. Fixes #2094
1452 lines
57 KiB
Plaintext
1452 lines
57 KiB
Plaintext
====================
|
||
Genode Porting Guide
|
||
====================
|
||
|
||
Genode Labs GmbH
|
||
|
||
|
||
Overview
|
||
########
|
||
|
||
This document describes the basic workflows for porting applications, libraries,
|
||
and device drivers to the Genode framework. It consists of the following
|
||
sections:
|
||
|
||
:[http:porting_applications - Porting third-party code to Genode]:
|
||
Overview of the general steps needed to use 3rd-party code on Genode.
|
||
|
||
:[http:porting_dosbox - Porting a program to natively run on Genode]:
|
||
Step-by-step description of applying the steps described in the first
|
||
section to port an application, using DosBox as an example.
|
||
|
||
:[http:porting_libraries - Native Genode port of a library]:
|
||
Many 3rd-party applications have library dependencies. This section shows
|
||
how to port a library using SDL_net (needed by DosBox) as an example.
|
||
|
||
:[http:porting_noux_packages - Porting an application to Genode's Noux runtime]:
|
||
On Genode, there exists an environment specially tailored to execute
|
||
command-line based Unix software, the so-called Noux runtime. This section
|
||
demonstrates how to port and execute the tar program within Noux.
|
||
|
||
:[http:porting_device_drivers - Porting devices drivers]:
|
||
This chapter describes the concepts of how to port a device driver to the
|
||
Genode framework. It requires the basic knowledge introduced in the previous
|
||
chapters and should be read last.
|
||
|
||
Before reading this guide, it is strongly advised to read the "The Genode
|
||
Build System" documentation:
|
||
|
||
:Build-system manual:
|
||
|
||
[http://genode.org/documentation/developer-resources/build_system]
|
||
|
||
|
||
Porting third-party code to Genode
|
||
##################################
|
||
|
||
Porting an existing program or library to Genode is for the most part a
|
||
straight-forward task and depends mainly on the complexity of the program
|
||
itself. Genode provides a fairly complete libc based on FreeBSD's libc whose
|
||
functionality can be extended by so-called libc plugins. If the program one
|
||
wants to port solely uses standard libc functions, porting becomes easy. Every
|
||
porting task involves usually the same steps which are outlined below.
|
||
|
||
|
||
Steps in porting applications to Genode
|
||
=======================================
|
||
|
||
# Check requirements/dependencies (e.g. on Linux)
|
||
|
||
The first step is gathering information about the application,
|
||
e.g. what functionality needs to be provided by the target system and
|
||
which libraries does it use.
|
||
|
||
# Create a port file
|
||
|
||
Prepare the source code of the application for the use within Genode. The
|
||
Genode build-system infrastructure uses fetch rules, so called port files,
|
||
which declare where the source is obtained from, what patches are applied
|
||
to the source code, and where the source code will be stored and
|
||
configured.
|
||
|
||
# Check platform dependent code and create stub code
|
||
|
||
This step may require changes to the original source code
|
||
of the application to be compilable for Genode. At this point, it
|
||
is not necessary to provide a working implementation for required
|
||
functions. Just creating stubs of the various functions is fine.
|
||
|
||
# Create build-description file
|
||
|
||
To compile the application we need build rules. Within these rules
|
||
we also declare all dependencies (e.g. libraries) that are needed
|
||
by it. The location of these rules depends on the type
|
||
of the application. Normal programs on one hand use a _target.mk_ file,
|
||
which is located in the program directory (e.g. _src/app/foobar_)
|
||
within a given Genode repository. Libraries on the other hand use
|
||
one or more _<library-name>.mk_ files that are placed in the _lib/mk_
|
||
directory of a Genode repository. In addition, libraries have to
|
||
provide _import-<library-name>.mk_ files. Amongst other things, these
|
||
files are used by applications to find the associated header files
|
||
of a library. The import files are placed in the _lib/import_
|
||
directory.
|
||
|
||
# Create a run script to ease testing
|
||
|
||
To ease the testing of applications, it is reasonable to write a run script
|
||
that creates a test scenario for the application. This run script is used
|
||
to automatically build all components of the Genode OS framework that are
|
||
needed to run the application as well as the application itself. Testing
|
||
the application on any of the kernels supported by Genode becomes just a
|
||
matter of executing the run script.
|
||
|
||
# Compile the application
|
||
|
||
The ported application is compiled from within the respective build
|
||
directory like any other application or component of Genode. The build
|
||
system of Genode uses the build rules created in the fourth step.
|
||
|
||
# Run the application
|
||
|
||
While porting an application, easy testing is crucial. By using the run script
|
||
that was written in the fifth step we reduce the effort.
|
||
|
||
# Debug the application
|
||
|
||
In most cases, a ported application does not work right away. We have to
|
||
debug misbehaviour and implement certain functionality in the platform-depending
|
||
parts of the application so that is can run on Genode. There are
|
||
several facilities available on Genode that help in the process. These are
|
||
different on each Genode platform but basically break down to using either a
|
||
kernel debugger (e.g., JDB on Fiasco.OC) or 'gdb(1)'. The reader of this guide
|
||
is advised to take a look at the "User-level debugging on Genode via GDB"
|
||
documentation.
|
||
|
||
_The order of step 1-4 is not mandatory but is somewhat natural._
|
||
|
||
|
||
Porting a program to natively run on Genode
|
||
###########################################
|
||
|
||
As an example on how to create a native port of a program for Genode, we will
|
||
describe the porting of DosBox more closely. Hereby, each of the steps
|
||
outlined in the previous section will be discussed in detail.
|
||
|
||
|
||
Check requirements/dependencies
|
||
===============================
|
||
|
||
In the first step, we build DosBox for Linux/x86 to obtain needed information.
|
||
Nowadays, most applications use a build-tool like Autotools or something
|
||
similar that will generate certain files (e.g., _config.h_). These files are
|
||
needed to successfully compile the program. Naturally they are required on
|
||
Genode as well. Since Genode does not use the original build tool of the
|
||
program for native ports, it is appropriate to copy those generated files
|
||
and adjust them later on to match Genode's settings.
|
||
|
||
We start by checking out the source code of DosBox from its subversion repository:
|
||
|
||
! $ svn export http://svn.code.sf.net/p/dosbox/code-0/dosbox/trunk@3837 dosbox-svn-3837
|
||
! $ cd dosbox-svn-3837
|
||
|
||
At this point, it is helpful to disable certain options that are not
|
||
available or used on Genode just to keep the noise down:
|
||
|
||
! $ ./configure --disable-opengl
|
||
! $ make > build.log 2>&1
|
||
|
||
After the DosBox binary is successfully built, we have a log file
|
||
(build.log) of the whole build process at our disposal. This log file will
|
||
be helpful later on when the _target.mk_ file needs to be created. In
|
||
addition, we will inspect the DosBox binary:
|
||
|
||
! $ readelf -d -t src/dosbox|grep NEEDED
|
||
! 0x0000000000000001 (NEEDED) Shared library: [libasound.so.2]
|
||
! 0x0000000000000001 (NEEDED) Shared library: [libdl.so.2]
|
||
! 0x0000000000000001 (NEEDED) Shared library: [libpthread.so.0]
|
||
! 0x0000000000000001 (NEEDED) Shared library: [libSDL-1.2.so.0]
|
||
! 0x0000000000000001 (NEEDED) Shared library: [libpng12.so.0]
|
||
! 0x0000000000000001 (NEEDED) Shared library: [libz.so.1]
|
||
! 0x0000000000000001 (NEEDED) Shared library: [libSDL_net-1.2.so.0]
|
||
! 0x0000000000000001 (NEEDED) Shared library: [libX11.so.6]
|
||
! 0x0000000000000001 (NEEDED) Shared library: [libstdc++.so.6]
|
||
! 0x0000000000000001 (NEEDED) Shared library: [libm.so.6]
|
||
! 0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1]
|
||
! 0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
|
||
|
||
Using _readelf_ on the binary shows all direct dependencies. We now know
|
||
that at least libSDL, libSDL_net, libstdc++, libpng, libz, and
|
||
libm are required by DosBox. The remaining libraries are mostly
|
||
mandatory on Linux and do not matter on Genode. Luckily all of these
|
||
libraries are already available on Genode. For now all we have to do is to
|
||
keep them in mind.
|
||
|
||
|
||
Creating the port file
|
||
======================
|
||
|
||
Since DosBox is an application, which depends on several ported
|
||
libraries (e.g., libSDL), the _ports_ repository within the Genode
|
||
source tree is a natural fit. On that account, the port file
|
||
_ports/ports/dosbox.port_ is created.
|
||
|
||
For DosBox the _dosbox.port_ looks as follows:
|
||
|
||
! LICENSE := GPLv2
|
||
! VERSION := svn
|
||
! DOWNLOADS := dosbox.svn
|
||
!
|
||
! URL(dosbox) := http://svn.code.sf.net/p/dosbox/code-0/dosbox/trunk
|
||
! DIR(dosbox) := src/app/dosbox
|
||
! REV(dosbox) := 3837
|
||
|
||
First, we define the license, the version and the type of the source code
|
||
origin. In case of DosBox, we checkout the source code from a Subversion
|
||
repository. This is denoted by the '.svn' suffix of the item specified in
|
||
the 'DOWNLOADS' declaration. Other valid types are 'file' (a plain file),
|
||
'archive' (an archive of the types tar.gz, tar.xz, tgz, tar.bz2, or zip)
|
||
or 'git' (a Git repository).
|
||
To checkout the source code out from the Subversion repository, we also need
|
||
its URL, the revision we want to check out and the destination directory
|
||
that will contain the sources afterwards. These declarations are mandatory and
|
||
must always be specified. Otherwise the preparation of the port will fail.
|
||
|
||
! PATCHES := $(addprefix src/app/dosbox/patches/,\
|
||
! $(notdir $(wildcard $(REP_DIR)/src/app/dosbox/patches/*.patch)))
|
||
!
|
||
! PATCH_OPT := -p2 -d src/app/dosbox
|
||
|
||
As next step, we declare all patches that are needed for the DosBox port.
|
||
Since in this case, the patches are using a different path format, we have
|
||
to override the default patch settings by defining the _PATCH_OPT_ variable.
|
||
|
||
Each port file comes along with a hash file. This hash is generated by taking
|
||
several sources into account. For one, the port file, each patch and the
|
||
port preparation tool (_tool/ports/prepare_port_) are the ingredients for
|
||
the hash value. If any of these files is changed, a new hash will be generated,
|
||
For now, we just write "dummy" in the '_ports/ports/dosbox.hash_ file.
|
||
|
||
The DosBox port can now be prepared by executing
|
||
|
||
! $ <genode-dir>/tool/ports/prepare_port dosbox
|
||
|
||
However, we get the following error message:
|
||
|
||
! Error: <rep-dir>/ports/dosbox.port is out of date, expected <fingerprint>
|
||
|
||
We get this message because we had specified the "dummy" hash value in
|
||
the _dosbox.hash_ file. The prepare_port tool computes a fingerprint
|
||
of the actual version of the port and compares this fingerprint with the
|
||
hash value specified in _dosbox.hash_. The computed fingerprint can
|
||
be found at _<genode-dir>/contrib/dosbox-dummy/dosbox.hash_. In the final
|
||
step of the port, we will replace the dummy fingerprint with the actual
|
||
fingerprint of the port. But before finalizing the porting work, it is
|
||
practical to keep using the dummy hash and suppress the fingerprint check.
|
||
This can be done by adding 'CHECK_HASH=no' as argument to the prepare_port tool:
|
||
|
||
! $ <genode-dir>/tool/ports/prepare-port dosbox CHECK_HASH=no
|
||
|
||
|
||
Check platform-dependent code
|
||
=============================
|
||
|
||
At this point, it is important to spot platform-dependent source files or
|
||
rather certain functions that are not yet available on Genode. These source
|
||
files should be omitted. Of course they may be used as a guidance when
|
||
implementing the functionality for Genode later on, when creating the
|
||
_target.mk_ file. In particular the various 'cdrom_ioctl_*.cpp' files are such
|
||
candidates in this example.
|
||
|
||
|
||
Creating the build Makefile
|
||
===========================
|
||
|
||
Now it is time to write the build rules into the _target.mk_, which will be
|
||
placed in _ports/src/app/dosbox_.
|
||
|
||
Armed with the _build.log_ that we created while building DosBox on Linux,
|
||
we assemble a list of needed source files. If an application just
|
||
uses a simple Makefile and not a build tool, it might be easier to just
|
||
reuse the contents of this Makefile instead.
|
||
|
||
First of all, we create a shortcut for the source directory of DosBox by calling
|
||
the 'select_from_ports' function:
|
||
|
||
! DOSBOX_DIR := $(call select_from_ports,dosbox)/src/app/dosbox
|
||
|
||
Under the hood, the 'select_from_ports' function looks up the
|
||
fingerprint of the specified port by reading the corresponding
|
||
<port-name>.hash file. It then uses this hash value to construct the
|
||
directory path within the _<genode-dir>contrib/_ directory that belongs to
|
||
the matching version of the port. If there is no hash file that matches the
|
||
port name, or if the port directory does not exist, the build system
|
||
will back out with an error message.
|
||
|
||
Examining the log file leaves us with the following list of source files:
|
||
|
||
! SRC_CC_cpu = $(notdir $(wildcard $(DOSBOX_DIR)/src/cpu/*.cpp))
|
||
! SRC_CC_debug = $(notdir $(wildcard $(DOSBOX_DIR)/src/debug/*.cpp))
|
||
! FILTER_OUT_dos = cdrom_aspi_win32.cpp cdrom_ioctl_linux.cpp cdrom_ioctl_os2.cpp \
|
||
! cdrom_ioctl_win32.cpp
|
||
! SRC_CC_dos = $(filter-out $(FILTER_OUT_dos), \
|
||
! $(notdir $(wildcard $(DOSBOX_DIR)/src/*.cpp)))
|
||
! […]
|
||
! SRC_CC = $(DOSBOX_DIR)/src/dosbox.cpp
|
||
! SRC_CC += $(SRC_CC_cpu) $(SRC_CC_debug) $(SRC_CC_dos) $(SRC_CC_fpu) \
|
||
! $(SRC_CC_gui) $(SRC_CC_hw) $(SRC_CC_hw_ser) $(SRC_CC_ints) \
|
||
! $(SRC_CC_ints) $(SRC_CC_misc) $(SRC_CC_shell)
|
||
!
|
||
! vpath %.cpp $(DOSBOX_DIR)/src
|
||
! vpath %.cpp $(DOSBOX_DIR)/src/cpu
|
||
! […]
|
||
|
||
_The only variable here that is actually evaluated by Genode's build-system is_
|
||
'SRC_CC'. _The rest of the variables are little helpers that make our_
|
||
_life more comfortable._
|
||
|
||
In this case, it is mandatory to use GNUMake's 'notdir' file name function
|
||
because otherwise the compiled object files would be stored within
|
||
the _contrib_ directories. Genode runs on multiple platforms with varying
|
||
architectures and mixing object files is considered harmful, which can happen
|
||
easily if the application is build from the original source directory. That's
|
||
why you have to use a build directory for each platform. The Genode build
|
||
system will create the needed directory hierarchy within the build directory
|
||
automatically. By combining GNUMake's 'notdir' and 'wildcard' function, we
|
||
can assemble a list of all needed source files without much effort. We then
|
||
use 'vpath' to point GNUMake to the right source file within the dosbox
|
||
directory.
|
||
|
||
The remaining thing to do now is setting the right include directories and proper
|
||
compiler flags:
|
||
|
||
! INC_DIR += $(PRG_DIR)
|
||
! INC_DIR += $(DOSBOX_DIR)/include
|
||
! INC_DIR += $(addprefix $(DOSBOX_DIR)/src, cpu debug dos fpu gui hardware \
|
||
! hardware/serialport ints misc shell)
|
||
|
||
'PRG_DIR' _is a special variable of Genode's build-system_
|
||
_and its value is always the absolute path to the directory containing_
|
||
_the 'target.mk' file._
|
||
|
||
We copy the _config.h_ file, which was generated in the first step to this
|
||
directory and change certain parts of it to better match Genode's
|
||
environment. Below is a skimmed diff of these changes:
|
||
|
||
! --- config.h.orig 2013-10-21 15:27:45.185719517 +0200
|
||
! +++ config.h 2013-10-21 15:36:48.525727975 +0200
|
||
! @@ -25,7 +25,8 @@
|
||
! /* #undef AC_APPLE_UNIVERSAL_BUILD */
|
||
!
|
||
! /* Compiling on BSD */
|
||
! -/* #undef BSD */
|
||
! +/* Genode's libc is based on FreeBSD 8.2 */
|
||
! +#define BSD 1
|
||
!
|
||
! […]
|
||
!
|
||
! /* The type of cpu this target has */
|
||
! -#define C_TARGETCPU X86_64
|
||
! +/* we define it ourself */
|
||
! +/* #undef C_TARGETCPU */
|
||
!
|
||
! […]
|
||
|
||
Thereafter, we specify the compiler flags:
|
||
|
||
! CC_OPT = -DHAVE_CONFIG_H -D_GNU_SOURCE=1 -D_REENTRANT
|
||
! ifeq ($(filter-out $(SPECS),x86_32),)
|
||
! INC_DIR += $(PRG_DIR)/x86_32
|
||
! CC_OPT += -DC_TARGETCPU=X86
|
||
! else ifeq ($(filter-out $(SPECS),x86_64),)
|
||
! INC_DIR += $(PRG_DIR)/x86_64
|
||
! CC_OPT += -DC_TARGETCPU=X86_64
|
||
! endif
|
||
!
|
||
! CC_WARN = -Wall
|
||
! #CC_WARN += -Wno-unused-variable -Wno-unused-function -Wno-switch \
|
||
! -Wunused-value -Wno-unused-but-set-variable
|
||
|
||
As noted in the commentary seen in the diff we define 'C_TARGETCPU'
|
||
and adjust the include directories ourselves according to the target
|
||
architecture.
|
||
|
||
While debugging, compiler warnings for 3rd-party code are really helpful but
|
||
tend to be annoying after the porting work is finished, we can
|
||
remove the hashmark to keep the compiler from complaining too
|
||
much.
|
||
|
||
Lastly, we need to add the required libraries, which we acquired in step 1:
|
||
|
||
! LIBS += libc libm libpng sdl stdcxx zlib
|
||
! LIBS += libc_lwip_nic_dhcp config_args
|
||
|
||
In addition to the required libraries, a few Genode specific
|
||
libraries are also needed. These libraries implement certain
|
||
functions in the libc via the libc's plugin mechanism.
|
||
libc_lwip_nic_dhcp, for example, is used to connect the BSD socket interface
|
||
to a NIC service such as a network device driver.
|
||
|
||
|
||
Creating the run script
|
||
=======================
|
||
|
||
To ease compiling, running and debugging DosBox, we create a run script
|
||
at _ports/run/dosbox.run_.
|
||
|
||
First, we specify the components that need to be built
|
||
|
||
! set build_components {
|
||
! core init drivers/audio drivers/framebuffer drivers/input
|
||
! drivers/pci drivers/timer app/dosbox
|
||
! }
|
||
! build $build_components
|
||
|
||
and instruct _tool/run_ to create the boot directory that hosts
|
||
all binaries and files which belong to the DosBox scenario.
|
||
|
||
As the name 'build_components' suggests, you only have to declare
|
||
the components of Genode, which are needed in this scenario. All
|
||
dependencies of DosBox (e.g. libSDL) will be built before DosBox
|
||
itself.
|
||
|
||
Nextm we provide the scenario's configuration 'config':
|
||
|
||
! append config {
|
||
! <config>
|
||
! <parent-provides>
|
||
! <service name="ROM"/>
|
||
! <service name="RAM"/>
|
||
! <service name="IRQ"/>
|
||
! <service name="IO_MEM"/>
|
||
! <service name="IO_PORT"/>
|
||
! <service name="CAP"/>
|
||
! <service name="PD"/>
|
||
! <service name="RM"/>
|
||
! <service name="CPU"/>
|
||
! <service name="LOG"/>
|
||
! <service name="SIGNAL"/>
|
||
! </parent-provides>
|
||
! <default-route>
|
||
! <any-service> <parent/> <any-child/> </any-service>
|
||
! </default-route>
|
||
! <start name="audio_drv">
|
||
! <resource name="RAM" quantum="6M"/>}
|
||
! <provides><service name="Audio_out"/></provides>
|
||
! </start>
|
||
! <start name="fb_drv">
|
||
! <resource name="RAM" quantum="4M"/>
|
||
! <provides><service name="Framebuffer"/></provides>
|
||
! </start>
|
||
! <start name="ps2_drv">
|
||
! <resource name="RAM" quantum="1M"/>
|
||
! <provides><service name="Input"/></provides>
|
||
! </start>
|
||
! <start name="timer">
|
||
! <resource name="RAM" quantum="1M"/>
|
||
! <provides><service name="Timer"/></provides>
|
||
! </start>
|
||
! <start name="dosbox">
|
||
! <resource name="RAM" quantum="128M"/>
|
||
! <config>
|
||
! <sdl_audio_volume value="100"/>
|
||
! <libc stdout="/dev/log" stderr="/dev/log">
|
||
! <vfs>
|
||
! <tar name="dosbox.tar"/>
|
||
! <dir name="dev"> <log/> </dir>
|
||
! </vfs>
|
||
! </libc>
|
||
! </config>
|
||
! </start>
|
||
! </config>}
|
||
! install_config $config
|
||
|
||
The _config_ file will be used by the init program to start all
|
||
components and application of the scenario, including DosBox.
|
||
|
||
Thereafter we declare all boot modules:
|
||
|
||
! set boot_modules {
|
||
! core init timer audio_drv fb_drv ps2_drv ld.lib.so
|
||
! libc.lib.so libm.lib.so
|
||
! lwip.lib.so libpng.lib.so stdcxx.lib.so sdl.lib.so
|
||
! pthread.lib.so zlib.lib.so dosbox dosbox.tar
|
||
! }
|
||
! build_boot_image $boot_modules
|
||
|
||
The boot modules comprise all binaries and other files like
|
||
the tar archive that contains DosBox' configuration file _dosbox.conf_
|
||
that are needed for this scenario to run sucessfully.
|
||
|
||
Finally, we set certain options, which are used when Genode is executed
|
||
in Qemu and instruct _tool/run_ to keep the scenario executing as long
|
||
as it is not manually stopped:
|
||
|
||
! append qemu_args " -m 256 -soundhw ac97 "
|
||
! run_genode_until forever
|
||
|
||
_It is reasonable to write the run script in a way that makes it possible_
|
||
_to use it for multiple Genode platforms. Debugging is often done on_
|
||
_Genode/Linux or on another Genode platform running in Qemu but testing_
|
||
_is normally done using actual hardware._
|
||
|
||
|
||
Compiling the program
|
||
=====================
|
||
|
||
To compile DosBox and all libraries it depends on, we execute
|
||
|
||
! $ make app/dosbox
|
||
|
||
from within Genode's build directory.
|
||
|
||
_We could also use the run script that we created in the previous step but_
|
||
_that would build all components that are needed to actually run_ DosBox
|
||
_and at this point our goal is just to get_ DosBox _compiled._
|
||
|
||
At the first attempt, the compilation stopped because g++ could not find
|
||
the header file _sys/timeb.h_:
|
||
|
||
! /src/genode/ports/contrib/dosbox-svn-3837/src/ints/bios.cpp:35:23: fatal error:
|
||
! sys/timeb.h: No such file or directory
|
||
|
||
This header is part of the libc but until now there was no program, which
|
||
actually used this header. So nobody noticed that it was missing. This
|
||
can happen all time when porting a new application to Genode because most
|
||
functionality is enabled or rather added on demand. Someone who is
|
||
porting applications to Genode has to be aware of the fact that it might be
|
||
necessary to extend Genode functionality by enabling so far disabled
|
||
bits or implementing certain functionality needed by the
|
||
application that is ported.
|
||
|
||
Since 'ftime(3)' is a deprecated function anyway we change the code of
|
||
DosBox to use 'gettimeofday(2)'.
|
||
|
||
After this was fixed, we face another problem:
|
||
|
||
! /src/genode/ports/contrib/dosbox-svn-3837/src/ints/int10_vesa.cpp:48:33: error:
|
||
! unable to find string literal operator ‘operator"" VERSION’
|
||
|
||
The fix is quite simple and the compile error was due to the fact
|
||
that Genode uses C++11 by now. It often happens that 3rd party code
|
||
is not well tested with a C++11 enabled compiler. In any case, a patch file
|
||
should be created which will be applied when preparing the port.
|
||
|
||
Furthermore it would be reasonable to report the bug to the DosBox
|
||
developers so it can be fixed upstream. We can then get rid of our
|
||
local patch.
|
||
|
||
The next show stoppers are missing symbols in Genode's SDL library port.
|
||
As it turns out, we never actually compiled and linked in the cdrom dummy
|
||
code which is provided by SDL.
|
||
|
||
|
||
Running the application
|
||
=======================
|
||
|
||
DosBox was compiled successfully. Now it is time to execute the binary
|
||
on Genode. Hence we use the run script we created in step 5:
|
||
|
||
! $ make run/dosbox
|
||
|
||
This may take some time because all other components of the Genode OS
|
||
Framework that are needed for this scenario have to be built.
|
||
|
||
|
||
Debugging the application
|
||
=========================
|
||
|
||
DosBox was successfully compiled but unfortunately it did not run.
|
||
To be honest that was expected and here the fun begins.
|
||
|
||
At this point, there are several options to chose from. By running
|
||
Genode/Fiasco.OC within Qemu, we can use the kernel debugger (JDB)
|
||
to take a deeper look at what went wrong (e.g., backtraces of the
|
||
running processes, memory dumps of the faulted DosBox process etc.).
|
||
Doing this can be quite taxing but fortunately Genode runs on multiple
|
||
kernels and often problems on one kernel can be reproduced on another
|
||
kernel. For this reason, we choose Genode/Linux where we can use all
|
||
the normal debugging tools like 'gdb(1)', 'valgrind(1)' and so on. Luckily
|
||
for us, DosBox also fails to run on Genode/Linux. The debugging steps
|
||
are naturally dependent on the ported software. In the case of DosBox,
|
||
the remaining stumbling blocks were a few places where DosBox assumed
|
||
Linux as a host platform.
|
||
|
||
For the sake of completeness here is a list of all files that were created by
|
||
porting DosBox to Genode:
|
||
|
||
! ports/ports/dosbox.hash
|
||
! ports/ports/dosbox.port
|
||
! ports/run/dosbox.run
|
||
! ports/src/app/dosbox/config.h
|
||
! ports/src/app/dosbox/patches/bios.patch
|
||
! ports/src/app/dosbox/patches/int10_vesa.patch
|
||
! ports/src/app/dosbox/target.mk
|
||
! ports/src/app/dosbox/x86_32/size_defs.h
|
||
! ports/src/app/dosbox/x86_64/size_defs.h
|
||
|
||
[image dosbox]
|
||
DosBox ported to Genode
|
||
|
||
Finally, after having tested that both the preparation-step and the
|
||
build of DosBox work as expected, it is time to
|
||
finalize the fingerprint stored in the _<genode-dir>/ports/ports/dosbox.hash_
|
||
file. This can be done by copying the content of the
|
||
_<genode-dir>/contrib/dosbox-dummy/dosbox.hash file_.
|
||
Alternatively, you may invoke the _tool/ports/update_hash_ tool with the
|
||
port name "dosbox" as argument. The next time, you
|
||
invoke the prepare_port tool, do not specify the 'CHECK_HASH=no' argument.
|
||
So the fingerprint check will validate that the _dosbox.hash_ file
|
||
corresponds to your _dosbox.port_ file. From now on, the
|
||
_<genode-dir>/contrib/dosbox-dummy_ directory will no longer be used because
|
||
the _dosbox.hash_ file points to the port directory named after the real
|
||
fingerprint.
|
||
|
||
|
||
Native Genode port of a library
|
||
###############################
|
||
|
||
Porting a library to be used natively on Genode is similar to porting
|
||
an application to run natively on Genode. The source codes have to be
|
||
obtained and, if needed, patched to run on Genode.
|
||
As an example on how to port a library to natively run on Genode, we
|
||
will describe the porting of SDL_net in more detail. Ported libraries
|
||
are placed in the _libports_ repository of Genode. But this is just a
|
||
convention. Feel free to host your library port in a custom repository
|
||
of your's.
|
||
|
||
|
||
Checking requirements/dependencies
|
||
==================================
|
||
|
||
We will proceed as we did when we ported DosBox to run natively on Genode.
|
||
First we build SDL_net on Linux to obtain a log file of the whole build
|
||
process:
|
||
|
||
! $ wget http://www.libsdl.org/projects/SDL_net/release/SDL_net-1.2.8.tar.gz
|
||
! $ tar xvzf SDL_net-1.2.8.tar.gz
|
||
! $ cd SDL_net-1.2.8
|
||
! $ ./configure
|
||
! $ make > build.log 2>&1
|
||
|
||
|
||
Creating the port file
|
||
======================
|
||
|
||
We start by creating _<genode-dir>/libports/ports/sdl_net.port:
|
||
|
||
! LICENSE := BSD
|
||
! VERSION := 1.2.8
|
||
! DOWNLOADS := sdl_net.archive
|
||
!
|
||
! URL(sdl_net) := http://www.libsdl.org/projects/SDL_net/release/SDL_net-$(VERSION).tar.gz
|
||
! SHA(sdl_net) := fd393059fef8d9925dc20662baa3b25e02b8405d
|
||
! DIR(sdl_net) := src/lib/sdl_net
|
||
!
|
||
! PATCHES := src/lib/sdl_net/SDLnet.patch src/lib/sdl_net/SDL_net.h.patch
|
||
|
||
In addition to the URL the SHA1 checksum of the SDL_net archive needs to
|
||
specified because _tool/prepare_port_ validates the downloaded archive
|
||
by using this hash.
|
||
|
||
Applications that want to use SDL_net have to include the 'SDL_net.h' header
|
||
file. Hence it is necessary to make this file visible to applications. This is
|
||
done by populating the _<genode-dir>/contrib/sdl-<hash>/include_ directory:
|
||
|
||
! DIRS := include/SDL
|
||
! DIR_CONTENT(include/SDL) := src/lib/sdl_net/SDL_net.h
|
||
|
||
For now, we also use a dummy hash in the _sdl_net.hash_ file like it was done
|
||
while porting DosBox. We will replace the dummy hash with the proper one at
|
||
the end.
|
||
|
||
|
||
Creating the build Makefile
|
||
===========================
|
||
|
||
We create the build rules in _libports/lib/mk/sdl_net.mk_:
|
||
|
||
! SDL_NET_DIR := $(call select_from_ports,sdl_net)/src/lib/sdl_net
|
||
!
|
||
! SRC_C = $(notdir $(wildcard $(SDL_NET_DIR)/SDLnet*.c))
|
||
!
|
||
! vpath %.c $(SDL_NET_DIR)
|
||
!
|
||
! INC_DIR += $(SDL_NET_DIR)
|
||
!
|
||
! LIBS += libc sdl
|
||
|
||
'SDL_net' should be used as shared library. To achieve this, we
|
||
have to add the following statement to the 'mk' file:
|
||
|
||
! SHARED_LIB = yes
|
||
|
||
_If we omit this statement, Genode's build system will automatically_
|
||
_build SDL_net as a static library called_ 'sdl_net.lib.a' _that_
|
||
_is linked directly into the application._
|
||
|
||
It is reasonable to create a dummy application that uses the
|
||
library because it is only possible to build libraries automatically
|
||
as a dependency of an application.
|
||
|
||
Therefore we create
|
||
_libports/src/test/libports/sdl_net/target.mk_ with the following content:
|
||
|
||
! TARGET = test-sdl_net
|
||
! LIBS = libc sdl_net
|
||
! SRC_CC = main.cc
|
||
|
||
! vpath main.cc $(PRG_DIR)/..
|
||
|
||
At this point we also create _lib/import/import-sdl_net.mk_
|
||
with the following content:
|
||
|
||
! SDL_NET_PORT_DIR := $(call select_from_ports,sdl_net)
|
||
! INC_DIR += $(SDL_NET_PORT_DIR)/include $(SDL_NET_PORT_DIR)/include/SDL
|
||
|
||
Each port that depends on SDL_net and has added it to its LIBS variable
|
||
will automatically include the _import-sdl_net.mk_ file and therefore
|
||
will use the specified include directory to find the _SDL_net.h_ header.
|
||
|
||
|
||
Compiling the library
|
||
=====================
|
||
|
||
We compile the SDL_net library as a side effect of building our dummy test
|
||
program by executing
|
||
|
||
! $ make test/libports/sdl_net
|
||
|
||
All source files are compiled fine but unfortunately the linking of the
|
||
library does not succeed:
|
||
|
||
! /src/genodebuild/foc_x86_32/var/libcache/sdl_net/sdl_net.lib.so:
|
||
! undefined reference to `gethostbyaddr'
|
||
|
||
The symbol 'gethostbyaddr' is missing, which is often a clear sign
|
||
of a missing dependency. In this case however 'gethostbyaddr(3)' is
|
||
missing because this function does not exist in Genode's libc _(*)_.
|
||
But 'getaddrinfo(3)' exists. We are now facing the choice of implementing
|
||
'gethostbyaddr(3)' or changing the code of SDL_net to use 'getaddrinfo(3)'.
|
||
Porting applications or libraries to Genode always may involve this kind of
|
||
choice. Which way is the best has to be decided by closely examining the
|
||
matter at hand. Sometimes it is better to implement the missing functions
|
||
and sometimes it is more beneficial to change the contributed code.
|
||
In this case, we opt for changing SDL_net because the former function is
|
||
obsolete anyway and implementing 'gethostbyaddr(3)' involves changes to
|
||
several libraries in Genode, namely libc and the network related
|
||
libc plugin. Although we have to keep in mind that it is likely to encounter
|
||
another application or library that also uses this function in the future.
|
||
|
||
With this change in place, SDL_net compiles fine.
|
||
|
||
_(*) Actually this function is implemented in the Genode's_ libc _but is_
|
||
_only available by using libc_resolv which we did not do for the sake of_
|
||
_this example._
|
||
|
||
|
||
Testing the library
|
||
===================
|
||
|
||
The freshly ported library is best tested with the application, which was the
|
||
reason the library was ported in the first place, since it is unlikely that
|
||
we port a library just for fun and no profit. Therefore, it is not necessary to
|
||
write a run script for a library alone.
|
||
|
||
For the records, here is a list of all files that were created by
|
||
porting SDL_net to Genode:
|
||
|
||
! libports/lib/mk/sdl_net.mk
|
||
! libports/lib/mk/import/import-sdl_net.mk
|
||
! libports/ports/sdl_net.hash
|
||
! libports/ports/sdl_net.port
|
||
! libports/src/lib/sdl_net/SDLnet.patch
|
||
! libports/test/libports/sdl_net/target.mk
|
||
|
||
|
||
Porting an application to Genode's Noux runtime
|
||
###############################################
|
||
|
||
Porting an application to Genode's Noux runtime is basically the same as
|
||
porting a program to natively run on Genode. The source code has to be
|
||
prepared and, if needed, patched to run in Noux. However in contrast to
|
||
this, there are Noux build rules (_ports/mk/noux.mk_) that enable us to use
|
||
the original build-tool if it is based upon Autotools. Building the
|
||
application is done within a cross-compile environment. In this environment
|
||
all needed variables like 'CC', 'LD', 'CFLAGS' and so on are set to their
|
||
proper values. In addition to these precautions, using _noux.mk_ simplifies certain things.
|
||
The system-call handling/functions is/are implemented in the libc plugin
|
||
_libc_noux_ (the source code is found in _ports/src/lib/libc_noux_). All
|
||
applications running in Noux have to be linked against this library which is
|
||
done implicitly by using the build rules of Noux.
|
||
|
||
As an example on how to port an application to Genode's Noux runtime, we
|
||
will describe the porting of GNU's 'tar' tool in more detail. A ported
|
||
application is normally referred to as a Noux package.
|
||
|
||
Checking requirements/dependencies
|
||
==================================
|
||
|
||
As usual, we first build GNU tar on Linux/x86 and capture the build
|
||
process:
|
||
|
||
! $ wget http://ftp.gnu.org/gnu/tar/tar-1.27.tar.xz
|
||
! $ tar xJf tar-1.27.tar.xz
|
||
! $ cd tar-1.27
|
||
! $ ./configure
|
||
! $ make > build.log 2>&1
|
||
|
||
|
||
Creating the port file
|
||
======================
|
||
|
||
We start by creating the port Makefile _ports/ports/tar.mk_:
|
||
|
||
! LICENSE := GPLv3
|
||
! VERSION := 1.27
|
||
! DOWNLOADS := tar.archive
|
||
!
|
||
! URL(tar) := http://ftp.gnu.org/gnu/tar/tar-$(VERSION).tar.xz
|
||
! SHA(tar) := 790cf784589a9fcc1ced33517e71051e3642642f
|
||
! SIG(tar) := ${URL(tar)}.sig
|
||
! KEY(tar) := GNU
|
||
! DIR(tar) := src/noux-pkg/tar
|
||
|
||
_As of version 14.05, Genode does not check the signature specified via_
|
||
_the SIG and KEY declaration but relies the SHA checksum only. However,_
|
||
_as signature checks are planned in the future, we use to include the_
|
||
_respective declarations if signature files are available._
|
||
|
||
While porting GNU tar we will use a dummy hash as well.
|
||
|
||
|
||
Creating the build rule
|
||
=======================
|
||
|
||
Build rules for Noux packages are located in _<genode-dir>/ports/src/noux-pkgs_.
|
||
|
||
The _tar/target.mk_ corresponding to GNU tar looks like this:
|
||
|
||
! CONFIGURE_ARGS = --bindir=/bin \
|
||
! --libexecdir=/libexec
|
||
!
|
||
! include $(REP_DIR)/mk/noux.mk
|
||
|
||
The variable 'CONFIGURE_ARGS' contains the options that are
|
||
passed on to Autoconf's configure script. The Noux specific build
|
||
rules in _noux.mk_ always have to be included last.
|
||
|
||
The build rules for GNU tar are quite short and therefore at the end
|
||
of this chapter we take a look at a much more extensive example.
|
||
|
||
|
||
Creating a run script
|
||
=====================
|
||
|
||
Creating a run script to test Noux packages is the same as it is
|
||
with running natively ported applications. Therefore we will only focus
|
||
on the Noux-specific parts of the run script and omit the rest.
|
||
|
||
First, we add the desired Noux packages to 'build_components':
|
||
|
||
! set noux_pkgs "bash coreutils tar"
|
||
!
|
||
! foreach pkg $noux_pkgs {
|
||
! lappend_if [expr ![file exists bin/$pkg]] build_components noux-pkg/$pkg }
|
||
!
|
||
! build $build_components
|
||
|
||
Since each Noux package is, like every other Genode binary, installed to the
|
||
_<build-dir>/bin_ directory, we create a tar archive of each package from
|
||
each directory:
|
||
|
||
! foreach pkg $noux_pkgs {
|
||
! exec tar cfv bin/$pkg.tar -h -C bin/$pkg . }
|
||
|
||
_Using noux.mk makes sure that each package is always installed to_
|
||
_<build-dir>/bin/<package-name>._
|
||
|
||
Later on, we will use these tar archives to assemble the file system
|
||
hierarchy within Noux.
|
||
|
||
Most applications ported to Noux want to read and write files. On that
|
||
matter, it is reasonable to provide a file-system service and the easiest
|
||
way to do this is to use the ram_fs server. This server provides a RAM-backed
|
||
file system, which is perfect for testing Noux applications. With
|
||
the help of the session label we can route multiple directories to the
|
||
file system in Noux:
|
||
|
||
! append config {
|
||
! <config>
|
||
! […]
|
||
! <start name="ram_fs">
|
||
! <resource name="RAM" quantum="32M"/>
|
||
! <provides><service name="File_system"/></provides>
|
||
! <config>
|
||
! <content>
|
||
! <dir name="tmp"> </dir>
|
||
! <dir name="home"> </dir>
|
||
! </content>
|
||
! <policy label="noux -> root" root="/" />
|
||
! <policy label="noux -> home" root="/home" writeable="yes" />
|
||
! <policy label="noux -> tmp" root="/tmp" writeable="yes" />
|
||
! </config>
|
||
! </start>
|
||
! […]
|
||
|
||
The file system Noux presents to the running applications is constructed
|
||
out of several stacked file systems. These file systems have to be
|
||
registered in the 'fstab' node in the configuration node of Noux:
|
||
|
||
! <start name="noux">
|
||
! <resource name="RAM" quantum="256M" />
|
||
! <config>
|
||
! <fstab>}
|
||
|
||
Each Noux package is added
|
||
|
||
! foreach pkg $noux_pkgs {
|
||
! append config {
|
||
! <tar name=\"$pkg.tar\" />" }}
|
||
|
||
and the routes to the ram_fs file system are configured:
|
||
|
||
! append config {
|
||
! <dir name="home"> <fs label="home" /> </dir>
|
||
! <dir name="ram"> <fs label="root" /> </dir>
|
||
! <dir name="tmp"> <fs label="tmp" /> </dir>
|
||
! </fstab>
|
||
! <start name="/bin/bash">
|
||
! <env name="TERM" value="linux" />
|
||
! </start>
|
||
! </config>
|
||
! </start>}
|
||
|
||
In this example we save the run script as _ports/run/noux_tar.run_.
|
||
|
||
|
||
Compiling the Noux package
|
||
==========================
|
||
|
||
Now we can trigger the compilation of tar by executing
|
||
|
||
! $ make VERBOSE= noux-pkg/tar
|
||
|
||
_At least on the first compilation attempt, it is wise to unset_ 'VERBOSE'
|
||
_because it enables us to see the whole output of the_ 'configure' _process._
|
||
|
||
By now, Genode provides almost all libc header files that are used by
|
||
typical POSIX programs. In most cases, it is rather a matter of enabling
|
||
the right definitions and compilation flags. It might be worth to take a
|
||
look at FreeBSD's ports tree because Genode's libc is based upon the one
|
||
of FreeBSD 8.2.0 and if certain changes to the contributed code are needed,
|
||
they are normally already done in the ports tree.
|
||
|
||
The script _noux_env.sh_ that is used to create the cross-compile
|
||
environment as well as the famous _config.log_ are found
|
||
in _<build-dir>/noux-pkg/<package-name>_.
|
||
|
||
|
||
Running the Noux package
|
||
========================
|
||
|
||
We use the previously written run script to start the scenario, in which we
|
||
can execute and test the Noux package by issuing:
|
||
|
||
! $ make run/noux_tar
|
||
|
||
After the system has booted and Noux is running, we first create some test
|
||
files from within the running bash process:
|
||
|
||
! bash-4.1$ mkdir /tmp/foo
|
||
! bash-4.1$ echo 'foobar' > /tmp/foo/bar
|
||
|
||
Following this we try to create a ".tar" archive of the directory _/tmp/foo_
|
||
|
||
! bash-4.1$ cd /tmp
|
||
! bash-4.1$ tar cvf foo.tar foo/
|
||
! tar: /tmp/foo: Cannot stat: Function not implemented
|
||
! tar: Exiting with failure status due to previous errors
|
||
! bash-4.1$
|
||
|
||
Well, this does not look too good but at least we have a useful error message
|
||
that leads (hopefully) us into the right direction.
|
||
|
||
|
||
Debugging an application that uses the Noux runtime
|
||
===================================================
|
||
|
||
Since the Noux service is basically the kernel part of our POSIX runtime
|
||
environment, we can ask Noux to show us the system calls executed by tar.
|
||
We change its configuration in the run script to trace all system calls:
|
||
|
||
! […]
|
||
! <start name="noux">
|
||
! <config trace_syscalls="yes">
|
||
! […]
|
||
|
||
We start the runscript again, create the test files and try to create a
|
||
".tar" archive. It still fails but now we have a trace of all system calls
|
||
and know at least what is going in Noux itself:
|
||
|
||
! […]
|
||
! [init -> noux] PID 0 -> SYSCALL FORK
|
||
! [init -> noux] PID 0 -> SYSCALL WAIT4
|
||
! [init -> noux] PID 5 -> SYSCALL STAT
|
||
! [init -> noux] PID 5 -> SYSCALL EXECVE
|
||
! [init -> noux] PID 5 -> SYSCALL STAT
|
||
! [init -> noux] PID 5 -> SYSCALL GETTIMEOFDAY
|
||
! [init -> noux] PID 5 -> SYSCALL STAT
|
||
! [init -> noux] PID 5 -> SYSCALL OPEN
|
||
! [init -> noux] PID 5 -> SYSCALL FTRUNCATE
|
||
! [init -> noux] PID 5 -> SYSCALL FSTAT
|
||
! [init -> noux] PID 5 -> SYSCALL GETTIMEOFDAY
|
||
! [init -> noux] PID 5 -> SYSCALL FCNTL
|
||
! [init -> noux] PID 5 -> SYSCALL WRITE
|
||
! [init -> noux -> /bin/tar] DUMMY fstatat(): fstatat called, not implemented
|
||
! [init -> noux] PID 5 -> SYSCALL FCNTL
|
||
! [init -> noux] PID 5 -> SYSCALL FCNTL
|
||
! [init -> noux] PID 5 -> SYSCALL WRITE
|
||
! [init -> noux] PID 5 -> SYSCALL FCNTL
|
||
! [init -> noux] PID 5 -> SYSCALL WRITE
|
||
! [init -> noux] PID 5 -> SYSCALL GETTIMEOFDAY
|
||
! [init -> noux] PID 5 -> SYSCALL CLOSE
|
||
! [init -> noux] PID 5 -> SYSCALL FCNTL
|
||
! [init -> noux] PID 5 -> SYSCALL WRITE
|
||
! [init -> noux] PID 5 -> SYSCALL CLOSE
|
||
! [init -> noux] child /bin/tar exited with exit value 2
|
||
! […]
|
||
|
||
_The trace log was shortened to only contain the important information._
|
||
|
||
We now see at which point something went wrong. To be honest, we see the
|
||
'DUMMY' message even without enabling the tracing of system calls. But
|
||
there are situations where a application is actually stuck in a (blocking)
|
||
system call and it is difficult to see in which.
|
||
|
||
Anyhow, 'fstatat' is not properly implemented. At this point, we either have
|
||
to add this function to the Genode's libc or rather add it to libc_noux.
|
||
If we add it to the libc, not only applications running in Noux will
|
||
benefit but all applications using the libc. Implementing it in
|
||
libc_noux is the preferred way if there are special circumstances because
|
||
we have to treat the function differently when used in Noux (e.g. 'fork').
|
||
|
||
For the sake of completeness here is a list of all files that were created by
|
||
porting GNU tar to Genode's Noux runtime:
|
||
|
||
! ports/ports/tar.hash
|
||
! ports/ports/tar.port
|
||
! ports/run/noux_tar.run
|
||
! ports/src/noux-pkg/tar/target.mk
|
||
|
||
|
||
Extensive build rules example
|
||
=============================
|
||
|
||
The build rules for OpenSSH are much more extensive than the ones in
|
||
the previous example. Let us take a quick look at those build rules to
|
||
get a better understanding of possible challenges one may encounter while
|
||
porting a program to Noux:
|
||
|
||
! # This prefix 'magic' is needed because OpenSSH uses $exec_prefix
|
||
! # while compiling (e.g. -DSSH_PATH) and in the end the $prefix and
|
||
! # $exec_prefix path differ.
|
||
!
|
||
! CONFIGURE_ARGS += --disable-ip6 \
|
||
! […]
|
||
! --exec-prefix= \
|
||
! --bindir=/bin \
|
||
! --sbindir=/bin \
|
||
! --libexecdir=/bin
|
||
|
||
In addition to the normal configure options, we have to also define the
|
||
path prefixes. The OpenSSH build system embeds certain paths in the
|
||
ssh binary, which need to be changed for Noux.
|
||
|
||
! INSTALL_TARGET = install
|
||
|
||
Normally the Noux build rules (_noux.mk_) execute 'make install-strip' to
|
||
explicitly install binaries that are stripped of their debug symbols. The
|
||
generated Makefile of OpenSSH does not use this target. It automatically
|
||
strips the binaries when executing 'make install'. Therefore, we set the
|
||
variable 'INSTALL_TARGET' to override the default behaviour of the
|
||
Noux build rules.
|
||
|
||
! LIBS += libcrypto libssl zlib libc_resolv
|
||
|
||
As OpenSSH depends on several libraries, we need to include these in the
|
||
build Makefile. These libraries are runtime dependencies and need to be
|
||
present when running OpenSSH in Noux.
|
||
|
||
Sometimes it is needed to patch the original build system. One way to do
|
||
this is by applying a patch while preparing the source code. The other
|
||
way is to do it before building the Noux package:
|
||
|
||
! noux_built.tag: Makefile Makefile_patch
|
||
!
|
||
! Makefile_patch: Makefile
|
||
! @#
|
||
! @# Our $(LDFLAGS) contain options which are usable by gcc(1)
|
||
! @# only. So instead of using ld(1) to link the binary, we have
|
||
! @# to use gcc(1).
|
||
! @#
|
||
! $(VERBOSE)sed -i 's|^LD=.*|LD=$(CC)|' Makefile
|
||
! @#
|
||
! @# We do not want to generate host-keys because we are crosscompiling
|
||
! @# and we can not run Genode binaries on the build system.
|
||
! @#
|
||
! $(VERBOSE)sed -i 's|^install:.*||' Makefile
|
||
! $(VERBOSE)sed -i 's|^install-nokeys:|install:|' Makefile
|
||
! @#
|
||
! @# The path of ssh(1) is hardcoded to $(bindir)/ssh which in our
|
||
! @# case is insufficient.
|
||
! @#
|
||
! $(VERBOSE)sed -i 's|^SSH_PROGRAM=.*|SSH_PROGRAM=/bin/ssh|' Makefile
|
||
|
||
The target _noux_built.tag_ is a special target defined by the Noux build
|
||
rules. It will be used by the build rules when building the Noux package.
|
||
We add the 'Makefile_patch' target as a dependency to it. So after configure
|
||
is executed, the generated Makefile will be patched.
|
||
|
||
Autoconf's configure script checks if all requirements are fulfilled and
|
||
therefore, tests if all required libraries are installed on the host system.
|
||
This is done by linking a small test program against the particular library.
|
||
Since these libraries are only build-time dependencies, we fool the configure
|
||
script by providing dummy libraries:
|
||
|
||
! #
|
||
! # Make the zlib linking test succeed
|
||
! #
|
||
! Makefile: dummy_libs
|
||
!
|
||
! LDFLAGS += -L$(PWD)
|
||
!
|
||
! dummy_libs: libz.a libcrypto.a libssl.a
|
||
!
|
||
! libcrypto.a:
|
||
! $(VERBOSE)$(AR) -rc $@
|
||
! libssl.a:
|
||
! $(VERBOSE)$(AR) -rc $@
|
||
! libz.a:
|
||
! $(VERBOSE)$(AR) -rc $@
|
||
|
||
|
||
Porting devices drivers
|
||
#######################
|
||
|
||
Even though Genode encourages writing native device drivers, this task sometimes
|
||
becomes infeasible. Especially if there is no documentation available for a
|
||
certain device or if there are not enough programming resources at hand to
|
||
implement a fully fledged driver. Examples of ported drivers can be found in
|
||
the 'dde_linux', 'dde_bsd', and 'dde_ipxe' repositories.
|
||
|
||
In this chapter we will exemplary discuss how to port a Linux driver for an ARM
|
||
based SoC to Genode. The goal is to execute driver code in user land directly on
|
||
Genode while making the driver believe it is running within the Linux kernel.
|
||
Traditionally there have been two approaches to reach this goal in Genode. In
|
||
the past, Genode provided a Linux environment, called 'dde_linux26', with the
|
||
purpose to offer just enough infrastructure to easily port drivers. However,
|
||
after adding more drivers it became clear that this repository grew extensively,
|
||
making it hard to maintain. Also updating the environment to support newer
|
||
Linux-kernel versions became a huge effort which let the repository to be neglected
|
||
over time.
|
||
|
||
Therefore we choose the path to write a customized environment for each driver,
|
||
which provides a specially tailored infrastructure. We found that the
|
||
support code usually is not larger than a couple of thousand lines of code,
|
||
while upgrading to newer driver versions, as we did with the USB drivers, is
|
||
feasible.
|
||
|
||
|
||
Basic driver structure
|
||
======================
|
||
|
||
The first step in porting a driver is to identify the driver code that has to be
|
||
ported. Once the code is located, we usually create a new Genode repository and
|
||
write a port file to download and extract the code. It is good practice to name
|
||
the port and the hash file like the new repository, e.g. _dde_linux.port_ if
|
||
the repository directory is called _<genode-dir>/repos/dde_linux_.
|
||
Having the source code ready, there are three main tasks the environment must
|
||
implement. The first is the driver back end, which is responsible for raw device
|
||
access using Genode primitives, the actual environment that emulates Linux
|
||
function calls the driver code is using, and the front end, which exposes for
|
||
example some Genode-session interface (like NIC or block session) that client
|
||
applications can connect to.
|
||
|
||
|
||
Further preparations
|
||
====================
|
||
|
||
Having the code ready, the next step is to create an _*.mk_ file that actually
|
||
compiles the code. For a driver library _lib/mk/<driver name>.mk_ has to be
|
||
created and for a stand-alone program _src/<driver name>/target.mk_ is created
|
||
within the repository. With the _*.mk_ file in place, we can now start the
|
||
actual compilation. Of course this will cause a whole lot of errors and
|
||
warnings. Most of the messages will deal with implicit declarations of functions
|
||
and unknown data types. What we have to do now is to go through each warning and
|
||
error message and either add the header file containing the desired function or
|
||
data type to the list of files that will be extracted to the _contrib_ directory
|
||
or create our own prototype or data definition.
|
||
|
||
When creating our own prototypes, we put them in a file called _lx_emul.h_. To
|
||
actually get this file included in all driver files we use the following code in
|
||
the _*.mk_ file:
|
||
|
||
! CC_C_OPT += -include $(INC_DIR)/lx_emul.h
|
||
|
||
where 'INC_DIR' points to the include path of _lx_emul.h_.
|
||
|
||
The hard part is to decide which of the two ways to go for a specific function
|
||
or data type, since adding header files also adds more dependencies and often
|
||
more errors and warnings. As a rule of thumb, try adding as few headers as
|
||
possible.
|
||
|
||
The compiler will also complain about a lot of missing header files. Since we do
|
||
not want to create all these header files, we use a trick in our _*.mk_ file that
|
||
extracts all header file includes from the driver code and creates symbolic
|
||
links that correspond to the file name and links to _lx_emul.h_. You can put the
|
||
following code snippet in your _*.mk_ file which does the trick:
|
||
|
||
!#
|
||
!# Determine the header files included by the contrib code. For each
|
||
!# of these header files we create a symlink to _lx_emul.h_.
|
||
!#
|
||
!GEN_INCLUDES := $(shell grep -rh "^\#include .*\/" $(DRIVER_CONTRIB_DIR) |\
|
||
! sed "s/^\#include [^<\"]*[<\"]\([^>\"]*\)[>\"].*/\1/" | \
|
||
! sort | uniq)
|
||
!
|
||
!#
|
||
!# Filter out original Linux headers that exist in the contrib directory
|
||
!#
|
||
!NO_GEN_INCLUDES := $(shell cd $(DRIVER_CONTRIB_DIR); find -name "*.h" | sed "s/.\///" | \
|
||
! sed "s/.*include\///")
|
||
!GEN_INCLUDES := $(filter-out $(NO_GEN_INCLUDES),$(GEN_INCLUDES))
|
||
!
|
||
!#
|
||
!# Put Linux headers in 'GEN_INC' dir, since some include use "../../" paths use
|
||
!# three level include hierarchy
|
||
!#
|
||
!GEN_INC := $(shell pwd)/include/include/include
|
||
!
|
||
!$(shell mkdir -p $(GEN_INC))
|
||
!
|
||
!GEN_INCLUDES := $(addprefix $(GEN_INC)/,$(GEN_INCLUDES))
|
||
!INC_DIR += $(GEN_INC)
|
||
!
|
||
!#
|
||
!# Make sure to create the header symlinks prior building
|
||
!#
|
||
!$(SRC_C:.c=.o) $(SRC_CC:.cc=.o): $(GEN_INCLUDES)
|
||
!
|
||
!$(GEN_INCLUDES):
|
||
! $(VERBOSE)mkdir -p $(dir $@)
|
||
! $(VERBOSE)ln -s $(LX_INC_DIR)/lx_emul.h $@
|
||
|
||
Make sure 'LX_INC_DIR' is the directory containing the _lx_emul.h_ file. Note
|
||
that 'GEN_INC' is added to your 'INC_DIR' variable.
|
||
|
||
The 'DRIVER_CONTRIB_DIR' variable is defined by calling the _select_from_port_
|
||
function at the beginning of a Makefile or a include file, which is used by
|
||
all other Makefiles:
|
||
|
||
! DRIVER_CONTRIB_DIR := $(call select_from_ports,driver_repo)/src/lib/driver_repo
|
||
|
||
The process of function definition and type declaration continues until the code
|
||
compiles. This process can be quite tiresome. When the driver code finally compiles, the
|
||
next stage is linking. This will of course lead to another whole set of errors
|
||
that complain about undefined references. To actually obtain a linked binary we
|
||
create a _dummies.cc_ file. To ease things up we suggest to create a macro called
|
||
'DUMMY' and implement functions as in the example below:
|
||
|
||
! /*
|
||
! * Do not include 'lx_emul.h', since the implementation will most likely clash
|
||
! * with the prototype
|
||
! */
|
||
!
|
||
!#define DUMMY(retval, name) \
|
||
! DUMMY name(void) { \
|
||
! PDBG( #name " called (from %p) not implemented", __builtin_return_address(0)); \
|
||
! return retval; \
|
||
!}
|
||
!
|
||
! DUMMY(-1, kmalloc)
|
||
! DUMMY(-1, memcpy)
|
||
! ...
|
||
|
||
Create a 'DUMMY' for each undefined reference until the binary links. We now
|
||
have a linked binary with a dummy environment.
|
||
|
||
|
||
Debugging
|
||
=========
|
||
|
||
From here on, we will actually start executing code, but before we do that, let us
|
||
have a look at the debugging options for device drivers. Since drivers have to
|
||
be tested on the target platform, there are not as many debugging options
|
||
available as for higher level applications, like running applications on the
|
||
Linux version of Genode while using GDB for debugging. Having these
|
||
restrictions, debugging is almost completely performed over the serial line and
|
||
on rare occasions with an hardware debugger using JTAG.
|
||
|
||
For basic Linux driver debugging it is useful to implement the 'printk'
|
||
function (use 'dde_kit_printf' or something similar) first. This way, the driver
|
||
code can output something and additions for debugging can be made. The
|
||
'__builtin_return_address' function is also useful in order to determine where a
|
||
specific function was called from. 'printk' may become a problem with devices
|
||
that require certain time constrains because serial line output is very slow. This is
|
||
why we port most drivers by running them on top of the Fiasco.OC version of
|
||
Genode. There you can take advantage of Fiasco's debugger (JDB) and trace buffer
|
||
facility.
|
||
|
||
The trace buffer can be used to log data and is much faster than 'printk' over
|
||
serial line. Please inspect the 'ktrace.h' file (at
|
||
_base-foc/contrib/l4/pkg/l4sys/include/ARCH-*/ktrace.h_)
|
||
that describes the complete interface. A very handy function there is
|
||
|
||
!fiasco_tbuf_log_3val("My message", variable1, variable2, variable3);
|
||
|
||
which stores a message and three variables in the trace buffer. The trace buffer
|
||
can be inspected from within JDB by pressing 'T'.
|
||
|
||
JDB can be accessed at any time by pressing the 'ESC' key. It can be used to
|
||
inspect the state of all running threads and address spaces on the system. There
|
||
is no recent JDB documentation available, but
|
||
|
||
:Fiasco kernel debugger manual:
|
||
|
||
[http://os.inf.tu-dresden.de/fiasco/doc/jdb.pdf]
|
||
|
||
should be a good starting point. It is also possible to enter the debugger at
|
||
any time from your program calling the 'enter_kdebug("My breakpoint")' function
|
||
from within your code. The complete JDB interface can be found in
|
||
_base-foc/contrib/l4/pkg/l4sys/include/ARCH-*/kdebug.h_.
|
||
|
||
Note that the backtrace ('bt') command does not work out of the box on ARM
|
||
platforms. We have a small patch for that in our Fiasco.OC development branch
|
||
located at GitHub: [http://github.com/ssumpf/foc/tree/dev]
|
||
|
||
|
||
The back end
|
||
============
|
||
|
||
To ease up the porting of drivers and interfacing Genode from C code, Genode offers a
|
||
library called DDE kit. DDE kit provides access to common functions required
|
||
by drivers like device memory, virtual memory with physical-address lookup,
|
||
interrupt handling, timers, etc. Please inspect _os/include/dde_kit_ to see the
|
||
complete interface description. You can also use 'grep -r dde_kit_ *' to see
|
||
usage of the interface in Genode.
|
||
|
||
As an example for using DDE kit we implement the 'kmalloc' call:
|
||
|
||
!void *kmalloc(size_t size, gfp_t flags)
|
||
!{
|
||
! return dde_kit_simple_malloc(size);
|
||
!}
|
||
|
||
It is also possible to directly use Genode primitives from C++ files, the
|
||
functions only have to be declared as 'extern "C"' so they can be called from C
|
||
code.
|
||
|
||
|
||
The environment
|
||
===============
|
||
|
||
Having a dummy environment we may now begin to actually execute driver code.
|
||
|
||
Driver initialization
|
||
~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
Most Linux drivers will have an initialization routine to register itself within
|
||
the Linux kernel and do other initializations if necessary. In order to be
|
||
initialized, the driver will register a function using the 'module_init' call.
|
||
This registered function must be called before the driver is actually used. To
|
||
be able to call the registered function from Genode, we define the 'module_init'
|
||
macro in _lx_emul.h_ as follows:
|
||
|
||
! #define module_init(fn) void module_##fn(void) { fn(); }
|
||
|
||
when a driver now registers a function like
|
||
|
||
! module_init(ehci_hcd_init);
|
||
|
||
we would have to call
|
||
|
||
! module_ehci_hcd_init();
|
||
|
||
during driver startup. Having implemented the above, it is now time to start our
|
||
ported driver on the target platform and check if the initialization function is
|
||
successful. Any important dummy functions that are called must be implemented
|
||
now. A dummy function that does not do device related things, like Linux book
|
||
keeping, may not be implemented. Sometimes Linux checks the return values of
|
||
functions we might not want to implement, in this case it is sufficient to simply
|
||
adjust the return value of the affected function.
|
||
|
||
Device probing
|
||
~~~~~~~~~~~~~~
|
||
Having the driver initialized, we will give the driver access to the device
|
||
resources. This is performed in two steps. In the case of ARM SoC's we have to
|
||
check in which state the boot loader (usually U-Boot) left the device. Sometimes
|
||
devices are already setup by the boot loader and only a simple device reset is
|
||
necessary to proceed. If the boot loader did not touch the device, we most
|
||
likely have to check and setup all the necessary clocks on the platform and may
|
||
have to perform other low level initializations like PHY setup.
|
||
|
||
If the device is successfully (low level) initialized, we can hand it over to
|
||
the driver by calling the 'probe' function of the driver. For ARM platforms the
|
||
'probe' function takes a 'struct platform_device' as an argument and all
|
||
important fields, like device resources and interrupt numbers, should be set to
|
||
the correct values before calling 'probe'. During 'probe' the driver will most
|
||
likely map and access device memory, request interrupts, and reset the device.
|
||
All dummy functions that are related to these tasks should be implemented or
|
||
ported at this point.
|
||
|
||
When 'probe' returns successful, you may either test other driver functions by
|
||
hand or start building the front-end.
|
||
|
||
|
||
The front end
|
||
=============
|
||
|
||
An important design question is how the front end is attached to the driver. In
|
||
some cases the front end may not use the driver directly, but other Linux
|
||
subsystems that are ported or emulated by the environment. For example, the USB
|
||
storage driver implements parts of the SCSI subsystem, which in turn is used
|
||
by the front end. The whole decision depends on the kind of driver that is
|
||
ported and on how much additional infrastructure is needed to actually make use
|
||
of the data. Again an USB example: For USB HID, we needed to port the USB controller
|
||
driver, the hub driver, the USB HID driver, and the generic HID driver in order
|
||
to retrieve keyboard and mouse events from the HID driver.
|
||
|
||
The last step in porting a device driver is to make it accessible to other
|
||
Genode applications. Typically this is achieved by implementing one of Genode's
|
||
session interfaces, like a NIC session for network adapters or a block session for
|
||
block devices. You may also define your own session interfaces. The
|
||
implementation of the session interface will most likely trigger driver calls,
|
||
so you have to have to keep an eye on the dummy functions. Also make sure that calls to the
|
||
driver actually do what they are supposed to, for example, some wrong return value
|
||
of a dummy function may cause a function to return without performing any work.
|
||
|
||
|
||
Notes on synchronization
|
||
========================
|
||
|
||
After some experiences with Linux drivers and multi-threading, we lately
|
||
choose to have all Linux driver code executed by a single thread only. This way no Linux
|
||
synchronization primitives have to be implemented and we simply don't have to
|
||
worry about subtle pre- and postconditions of many functions (like "this
|
||
function has to be called with lock 'x' being held").
|
||
|
||
Unfortunately we cannot get rid of all threads within a device-driver server,
|
||
there is at least one waiting for interrupts and one for the entry point that
|
||
waits for client session requests. In order to synchronize these threads, we use
|
||
Genode's signalling framework. So when, for example, the IRQ thread receives an
|
||
interrupt it will send a signal. The Linux driver thread will at certain points
|
||
wait for these signals (e.g., functions like 'schedule_timeout' or
|
||
'wait_for_completion') and execute the right code depending on the kind of
|
||
signal delivered or firmly speaking the signal context. For this to work, we use
|
||
a class called 'Signal_dispatcher' (_base/include/base/signal.h_) which inherits
|
||
from 'Signal_context'. More than one dispatcher can be bound to a signal
|
||
receiver, while each dispatcher might do different work, like calling the
|
||
Linux interrupt handler in the IRQ example.
|
||
|
||
|