======================= The Genode build system ======================= Norman Feske Abstract ######## The Genode OS Framework comes with a custom build system that is designed for the creation of highly modular and portable systems software. Understanding its basic concepts is pivotal for using the full potential of the framework. This document introduces those concepts and the best practises of putting them to good use. Beside building software components from source code, common and repetitive development tasks are the testing of individual components and the integration of those components into complex system scenarios. To streamline such tasks, the build system is accompanied with special tooling support. This document introduces those tools. Build directories and repositories ################################## The build system is supposed to never touch the source tree. The procedure of building components and integrating them into system scenarios is done at a distinct build directory. One build directory targets a specific platform, i.e., a kernel and hardware architecture. Because the source tree is decoupled from the build directory, one source tree can have many different build directories associated, each targeted at another platform. The recommended way for creating a build directory is the use of the 'create_builddir' tool located at '/tool/'. By starting the tool without arguments, its usage information will be printed. For creating a new build directory, one of the listed target platforms must be specified. Furthermore, the location of the new build directory has to be specified via the 'BUILD_DIR=' argument. For example: ! cd ! ./tool/create_builddir linux_x86 BUILD_DIR=/tmp/build.linux_x86 This command will create a new build directory for the Linux/x86 platform at _/tmp/build.linux_x86/_. Build-directory configuration via 'build.conf' ============================================== The fresh build directory will contain a 'Makefile', which is a symlink to _tool/builddir/build.mk_. This makefile is the front end of the build system and not supposed to be edited. Beside the makefile, there is a _etc/_ subdirectory that contains the build-directory configuration. For most platforms, there is only a single _build.conf_ file, which defines the parts of the Genode source tree incorporated in the build process. Those parts are called _repositories_. The repository concept allows for keeping the source code well separated for different concerns. For example, the platform-specific code for each target platform is located in a dedicated _base-_ repository. Also, different abstraction levels and features of the system are residing in different repositories. The _etc/build.conf_ file defines the set of repositories to consider in the build process. At build time, the build system overlays the directory structures of all repositories specified via the 'REPOSITORIES' declaration to form a single logical source tree. By changing the list of 'REPOSITORIES', the view of the build system on the source tree can be altered. The _etc/build.conf_ as found in a fresh created build directory will list the _base-_ repository of the platform selected at the 'create_builddir' command line as well as the 'base', 'os', and 'demo' repositories needed for compiling Genode's default demonstration scenario. Furthermore, there are a number of commented-out lines that can be uncommented for enabling additional repositories. Note that the order of the repositories listed in the 'REPOSITORIES' declaration is important. Front-most repositories shadow subsequent repositories. This makes the repository mechanism a powerful tool for tweaking existing repositories: By adding a custom repository in front of another one, customized versions of single files (e.g., header files or target description files) can be supplied to the build system without changing the original repository. Building targets ================ To build all targets contained in the list of 'REPOSITORIES' as defined in _etc/build.conf_, simply issue 'make'. This way, all components that are compatible with the build directory's base platform will be built. In practice, however, only some of those components may be of interest. Hence, the build can be tailored to those components which are of actual interest by specifying source-code subtrees. For example, using the following command ! make core server/nitpicker the build system builds all targets found in the 'core' and 'server/nitpicker' source directories. You may specify any number of subtrees to the build system. As indicated by the build output, the build system revisits each library that is used by each target found in the specified subtrees. This is very handy for developing libraries because instead of re-building your library and then your library-using program, you just build your program and that's it. This concept even works recursively, which means that libraries may depend on other libraries. In practice, you won't ever need to build the _whole tree_ but only the targets that you are interested in. Cleaning the build directory ============================ To remove all but kernel-related generated files, use ! make clean To remove all generated files, use ! make cleanall Both 'clean' and 'cleanall' won't remove any files from the _bin/_ subdirectory. This makes the _bin/_ a safe place for files that are unrelated to the build process, yet required for the integration stage, e.g., binary data. Controlling the verbosity of the build process ============================================== To understand the inner workings of the build process in more detail, you can tell the build system to display each directory change by specifying ! make VERBOSE_DIR= If you are interested in the arguments that are passed to each invocation of 'make', you can make them visible via ! make VERBOSE_MK= Furthermore, you can observe each single shell-command invocation by specifying ! make VERBOSE= Of course, you can combine these verboseness toggles for maximizing the noise. Enabling parallel builds ======================== To utilize multiple CPU cores during the build process, you may invoke 'make' with the '-j' argument. If manually specifying this argument becomes an inconvenience, you may add the following line to your _etc/build.conf_ file: ! MAKE += -j This way, the build system will always use '' CPUs for building. Caching inter-library dependencies ================================== The build system allows to repeat the last build without performing any library-dependency checks by using: ! make again The use of this feature can significantly improve the work flow during development because in contrast to source-codes, library dependencies rarely change. So the time needed for re-creating inter-library dependencies at each build can be saved. Repository directory layout ########################### Each Genode repository has the following layout: Directory | Description ------------------------------------------------------------ 'doc/' | Documentation, specific for the repository ------------------------------------------------------------ 'etc/' | Default configuration of the build process ------------------------------------------------------------ 'mk/' | The build system ------------------------------------------------------------ 'include/' | Globally visible header files ------------------------------------------------------------ 'src/' | Source codes and target build descriptions ------------------------------------------------------------ 'lib/mk/' | Library build descriptions Creating targets and libraries ############################## Target descriptions =================== A good starting point is to look at the init target. The source code of init is located at _os/src/init/_. In this directory, you will find a target description file named _target.mk_. This file contains the building instructions and it is usually very simple. The build process is controlled by defining the following variables. Build variables to be defined by you ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ :'TARGET': is the name of the binary to be created. This is the only *mandatory variable* to be defined in a _target.mk_ file. :'REQUIRES': expresses the requirements that must be satisfied in order to build the target. You find more details about the underlying mechanism in Section [Specializations]. :'LIBS': is the list of libraries that are used by the target. :'SRC_CC': contains the list of '.cc' source files. The default search location for source codes is the directory, where the _target.mk_ file resides. :'SRC_C': contains the list of '.c' source files. :'SRC_S': contains the list of assembly '.s' source files. :'SRC_BIN': contains binary data files to be linked to the target. :'INC_DIR': is the list of include search locations. Directories should always be appended by using +=. Never use an assignment! :'EXT_OBJECTS': is a list of Genode-external objects or libraries. This variable is mostly used for interfacing Genode with legacy software components. Rarely used variables --------------------- :'CC_OPT': contains additional compiler options to be used for '.c' as well as for '.cc' files. :'CC_CXX_OPT': contains additional compiler options to be used for the C++ compiler only. :'CC_C_OPT': contains additional compiler options to be used for the C compiler only. Specifying search locations ~~~~~~~~~~~~~~~~~~~~~~~~~~~ When specifying search locations for header files via the 'INC_DIR' variable or for source files via 'vpath', relative pathnames are illegal to use. Instead, you can use the following variables to reference locations within the source-code repository, where your target lives: :'REP_DIR': is the base directory of the current source-code repository. Normally, specifying locations relative to the base of the repository is never used by _target.mk_ files but needed by library descriptions. :'PRG_DIR': is the directory, where your _target.mk_ file resides. This variable is always to be used when specifying a relative path. Library descriptions ==================== In contrast to target descriptions that are scattered across the whole source tree, library descriptions are located at the central place _lib/mk_. Each library corresponds to a _.mk_ file. The base of the description file is the name of the library. Therefore, no 'TARGET' variable needs to be set. The source-code locations are expressed as '$(REP_DIR)'-relative 'vpath' commands. Library-description files support the following additional declarations: :'SHARED_LIB = yes': declares that the library should be built as a shared object rather than a static library. The resulting object will be called _.lib.so_. Specializations =============== Building components for different platforms likely implicates portions of code that are tied to certain aspects of the target platform. For example, a target platform may be characterized by * A kernel API such as L4v2, Linux, L4.sec, * A hardware architecture such as x86, ARM, Coldfire, * A certain hardware facility such as a custom device, or * Other properties such as software license requirements. Each of these attributes express a specialization of the build process. The build system provides a generic mechanism to handle such specializations. The _programmer_ of a software component knows the properties on which his software relies and thus, specifies these requirements in his build description file. The _user/customer/builder_ decides to build software for a specific platform and defines the platform specifics via the 'SPECS' variable per build directory in _etc/specs.conf_. In addition to an (optional) _etc/specs.conf_ file within the build directory, the build system incorporates the first _etc/specs.conf_ file found in the repositories as configured for the build directory. For example, for a 'linux_x86' build directory, the _base-linux/etc/specs.conf_ file is used by default. The build directory's 'specs.conf' file can still be used to extend the 'SPECS' declarations, for example to enable special features. Each '' in the 'SPECS' variable instructs the build system to * Include the 'make'-rules of a corresponding _base/mk/spec-.mk_ file. This enables the customization of the build process for each platform. * Search for _.mk_ files in the _lib/mk//_ subdirectory. This way, we can provide alternative implementations of one and the same library interface for different platforms. Before a target or library gets built, the build system checks if the 'REQUIRES' entries of the build description file are satisfied by entries of the 'SPECS' variable. The compilation is executed only if each entry in the 'REQUIRES' variable is present in the 'SPECS' variable as supplied by the build directory configuration. Building tools to be executed on the host platform =================================================== Sometimes, software requires custom tools that are used to generate source code or other ingredients for the build process, for example IDL compilers. Such tools won't be executed on top of Genode but on the host platform during the build process. Hence, they must be compiled with the tool chain installed on the host, not the Genode tool chain. The Genode build system accommodates the building of such host tools as a side effect of building a library or a target. Even though it is possible to add the tool compilation step to a regular build description file, it is recommended to introduce a dedicated pseudo library for building such tools. This way, the rules for building host tools are kept separate from rules that refer to Genode programs. By convention, the pseudo library should be named __host_tools_ and the host tools should be built at _/tool//_. With __, we refer to the name of the software package the tool belongs to, e.g., qt5 or mupdf. To build a tool named __, the pseudo library contains a custom make rule like the following: ! $(BUILD_BASE_DIR)/tool//: ! $(MSG_BUILD)$(notdir $@) ! $(VERBOSE)mkdir -p $(dir $@) ! $(VERBOSE)...build commands... To let the build system trigger the rule, add the custom target to the 'HOST_TOOLS' variable: ! HOST_TOOLS += $(BUILD_BASE_DIR)/tool// Once the pseudo library for building the host tools is in place, it can be referenced by each target or library that relies on the respective tools via the 'LIBS' declaration. The tool can be invoked by referring to '$(BUILD_BASE_DIR)/tool//tool'. For an example of using custom host tools, please refer to the mupdf package found within the libports repository. During the build of the mupdf library, two custom tools fontdump and cmapdump are invoked. The tools are built via the _lib/mk/mupdf_host_tools.mk_ library description file. The actual mupdf library (_lib/mk/mupdf.mk_) has the pseudo library 'mupdf_host_tools' listed in its 'LIBS' declaration and refers to the tools relative to '$(BUILD_BASE_DIR)'. Building additional custom targets accompanying library or program ================================================================== There are cases when it is important to build additional targets besides standard files built for library or program. Of course there is no problem with writing specific make rules for commands that generate those target files but for them to be built a proper dependency must be specified. To achieve it those additional targets should be added to 'CUSTOM_TARGET_DEPS' variable like e.g. in iwl_firmware library from dde_linux repository: ! CUSTOM_TARGET_DEPS += $(addprefix $(BIN_DIR)/,$(IMAGES)) Automated integration and testing ################################# Genode's cross-kernel portability is one of the prime features of the framework. However, each kernel takes a different route when it comes to configuring, integrating, and booting the system. Hence, for using a particular kernel, profound knowledge about the boot concept and the kernel-specific tools is required. To streamline the testing of Genode-based systems across the many different supported kernels, the framework comes equipped with tools that relieve you from these peculiarities. Run scripts =========== Using so-called run scripts, complete Genode systems can be described in a concise and kernel-independent way. Once created, a run script can be used to integrate and test-drive a system scenario directly from the build directory. The best way to get acquainted with the concept is reviewing the run script for the 'hello_tutorial' located at _hello_tutorial/run/hello.run_. Let's revisit each step expressed in the _hello.run_ script: * Building the components needed for the system using the 'build' command. This command instructs the build system to compile the targets listed in the brace block. It has the same effect as manually invoking 'make' with the specified argument from within the build directory. * Creating a new boot directory using the 'create_boot_directory' command. The integration of the scenario is performed in a dedicated directory at _/var/run//_. When the run script is finished, this directory will contain all components of the final system. In the following, we will refer to this directory as run directory. * Installing the Genode 'config' file into the run directory using the 'install_config' command. The argument to this command will be written to a file called 'config' at the run directory picked up by Genode's init process. * Creating a bootable system image using the 'build_boot_image' command. This command copies the specified list of files from the _/bin/_ directory to the run directory and executes the platform-specific steps needed to transform the content of the run directory into a bootable form. This form depends on the actual base platform and may be an ISO image or a bootable ELF image. * Executing the system image using the 'run_genode_until' command. Depending on the base platform, the system image will be executed using an emulator. For most platforms, Qemu is the tool of choice used by default. On Linux, the scenario is executed by starting 'core' directly from the run directory. The 'run_genode_until' command takes a regular expression as argument. If the log output of the scenario matches the specified pattern, the 'run_genode_until' command returns. If specifying 'forever' as argument (as done in 'hello.run'), this command will never return. If a regular expression is specified, an additional argument determines a timeout in seconds. If the regular expression does not match until the timeout is reached, the run script will abort. Please note that the _hello.run_ script does not contain kernel-specific information. Therefore it can be executed from the build directory of any base platform by using: ! make run/hello When invoking 'make' with an argument of the form 'run/*', the build system will look in all repositories for a run script with the specified name. The run script must be located in one of the repositories 'run/' subdirectories and have the file extension '.run'. For a more comprehensive run script, _os/run/demo.run_ serves as a good example. This run script describes Genode's default demo scenario. As seen in 'demo.run', parts of init's configuration can be made dependent on the platform's properties expressed as spec values. For example, the PCI driver gets included in init's configuration only on platforms with a PCI bus. For appending conditional snippets to the _config_ file, there exists the 'append_if' command, which takes a condition as first and the snippet as second argument. To test for a SPEC value, the command '[have_spec ]' is used as condition. Analogously to how 'append_if' appends strings, there exists 'lappend_if' to append list items. The latter command is used to conditionally include binaries to the list of boot modules passed to the 'build_boot_image' command. The run mechanism explained =========================== Under the hood, run scripts are executed by an expect interpreter. When the user invokes a run script via _make run/_, the build system invokes the run tool at _/tool/run_ with the run script as argument. The run tool is an expect script that has no other purpose than defining several commands used by run scripts, including a platform-specific script snippet called run environment ('env'), and finally including the actual run script. Whereas _tool/run_ provides the implementations of generic and largely platform-independent commands, the _env_ snippet included from the platform's respective _base-/run/env_ file contains all platform-specific commands. For reference, the most simplistic run environment is the one at _base-linux/run/env_, which implements the 'create_boot_directory', 'install_config', 'build_boot_image', and 'run_genode_until' commands for Linux as base platform. For the other platforms, the run environments are far more elaborative and document precisely how the integration and boot concept works on each platform. Hence, the _base-/run/env_ files are not only necessary parts of Genode's tooling support but serve as resource for peculiarities of using each kernel. Using run script to implement test cases ======================================== Because run scripts are actually expect scripts, the whole arsenal of language features of the Tcl scripting language is available to them. This turns run scripts into powerful tools for the automated execution of test cases. A good example is the run script at _libports/run/lwip.run_, which tests the lwIP stack by running a simple Genode-based HTTP server on Qemu. It fetches and validates a HTML page from this server. The run script makes use of a regular expression as argument to the 'run_genode_until' command to detect the state when the web server becomes ready, subsequently executes the 'lynx' shell command to fetch the web site, and employs Tcl's support for regular expressions to validate the result. The run script works across base platforms that use Qemu as execution environment. To get the most out of the run mechanism, a basic understanding of the Tcl scripting language is required. Furthermore the functions provided by _tool/run_ and _base-/run/env_ should be studied. Automated testing across base platforms ======================================= To execute one or multiple test cases on more than one base platform, there exists a dedicated tool at _tool/autopilot_. Its primary purpose is the nightly execution of test cases. The tool takes a list of platforms and of run scripts as arguments and executes each run script on each platform. The build directory for each platform is created at _/tmp/autopilot./_ and the output of each run script is written to a file called _..log_. On stderr, autopilot prints the statistics about whether or not each run script executed successfully on each platform. If at least one run script failed, autopilot returns a non-zero exit code, which makes it straight forward to include autopilot into an automated build-and-test environment.