* fixes two places, where the free tree module used to continue to process a
request after actually having determined that the request fails
* moves the functionality of checking the hash of a read block and decoding it
to a dedicated method in order to improve readability
Ref #5077
Adds a new command attribute "uninitialized_data" to the Tresor Tester
configuration. If a <request op="read"> command has this attribute set to "yes"
it assumes the read blocks to be uninitialized and therefore contain only 0's.
Note, that a command that has "uninitialized_data" set to "yes" cannot have the
attribute "salt".
Ref #5077
Snapshots must only be removed when securing the superblock. Otherwise, the
last secured superblock might get corrupted. The Free Tree allocation algorithm
would not consider the deleted snapshots anymore although they are still active
in the secured superblock and re-use their blocks. This would render the tresor
container unusable if the superblock with the deleted snapshots is not secured
in the end (driver crash, power down, ...).
Ref #5077
Superblock_control::Initialize used to decode a read superblock before checking
its hash. This is not necessary but may cause the operation to end up in a
decoding error on a superblock that is not the desired one anyway.
Ref #5077
Instead of iterating over all superblocks and checking each valid one,
check only the one whose hash matches the hash stored in the trust anchor.
I.e., the last one that was secured to the trust anchor. We must assume that
the other superblocks were corrupted in the meantime by operating the Tresor
container and, anyway, these Superblocks are not used anymore.
Ref #5077
The request of extending a tree used to halt when it found that
it could not add more levels to the tree because the maximum level index was
reached. Now, the library simply marks the request as failed, leaving it to
the user to handle the error condition.
Ref #5077
* differentiates request types that where merged formerly per module;
e.g. instead of type Superblock_control::Request, there are now types
* Superblock_control::Read_vbas
* Superblock_control::Write_vbas
* Superblock_control::Rekey
* Superblock_control::Initialize
* ...
each holding only the state and functionality that is required for exactly
that request
* removes all classes of the Tresor module framework and adapts all
Tresor- and File-Vault- related libs, apps, and tests accordingly
* the former "channel" state is merged into the new request types, meaning, a
request manages no longer only the "call" to a functionality but
also the execution of that functionality; every request has a lifetime
equal to the "call" and an execute method to be driven forward
* state that is used by a request but has a longer lifetime (e.g. VFS file
handles in Tresor::Crypto) is managed by the top level
of the user and handed over via the execute arguments; however, the
synchronization of multiple requests on this state is done by the module
(e.g. Tresor::Crypto)
* requests are now driven explicitly as first argument of the (overloaded)
execute method of their module; the module can, however, stall a request
by returning false without doing anything (used for synchronization on
resources)
* introduces Request_helper, Generated_request and Generatable_request in the
Tresor namespace in order to avoid the redundancy of sub-request generation
and execution
* moves access to Client-Data pointers up to Tresor::Virtual_block_device in
order to simplify Tresor::Block_io and Tresor::Crypto
* removes Tresor::Client_data and introduces pure interface
Client_data_interface in order to remove Tresor::Client_data and
move management of Client Data to the top level of a Tresor user
* introduces pure interface Crypto_files_interface in order to move management
of Crypto files to the top level of a Tresor user
* moves management of Block-IO and Trust-Anchor files to the top level of a
Tresor user
* adapts all execute methods, so, that they return the progress state
instead of modifying a reference argument
* removes Tresor::Request_and Tresor:Request and instead implements
scheduling at the top level of the Tresor user
* the Tresor Tester uses a list as schedule that holds Command objects; this
list ensures, that commands are started in the order of configuration
the Command type is a merge of the state of all possible commands that can
be configured at the Tresor Tester; the actual Tresor requests (if any) are
then allocated on-demand only
* the Tresor VFS plugin does not use a dynamic data structure for scheduling;
the plugin has 5 members that each reflect a distinct type of operation:
* initialize operation
* deinitialize operation
* data operation
* extend operation
* rekey operation
consequently, of each type, there can be only one operation in-flight at a
time; at the user front-end each operation (except "initialize") can be
controlled through a dedicated VFS file; for each of these files, the VFS
expects only one handle to be open at a time and only one file operation
(read, write, sync) active at a time; once an operation gets started it is
finished without preemtion (except of the interleaving at rekey and
extend); when multiple operations are waiting to be started the plugin
follows a static priority scheme:
init op > deinit op > data op > extend op > rekey op
there are some operation-specific details
* the initialize operation is started only by the plugin itself on startup
and will be driven as side effect by subsequent user calls to file
operations
* the data file is the only contiguous file in the front end and the file
operations work as on usual data files
* the other 3 files are transactional files and the user is expected to
follow this scheme when operating on them
1) stat (to determine file size)
2) seek to offset 0
3) read entire file once (this will be queued until there is no operation
of this type pending anymore and return the last result:
"none" | "failed" | "succeeded"; used primarily for synchronization)
4) write operation parameters (this returns immediately and marks the
operation as "requested")
5) read entire file once (the same as above but this time in order to
determine the operation result)
* the rekey op and deinitialize op are requested by writing "true"
* the extend op is requested by writing "tree=[TREE], blocks=[BLOCKS]"
where TREE is either "vbd" or "ft" and BLOCKS is the number of physical
4K blocks by which the physical range of the tresor container expands
(the physical range always starts at block address 0 and is always
expanded upwards)
* replaces the former <trust-anchor op="initialize"> command at the Tresor
Tester with <initialize-trust-achor> as there are no other trust anchor
operations that can be requested through the Tester config anyway
* removes the "sync" attribute from all commands at the Tresor Tester except
from <request op="rekey">, <request "extend_ft">, <request op="extend_vbd">;
as the Tester controls scheduling now, requests are generally synchronous;
at the rekeying and extension commands, the "sync" attribute determines
wether subsequent commands are interleaved with the execution of these
commands (if possible)
* removes "debug" config attribute from Tresor VFS plugin and reworks "verbose"
attribute to generate more sensible output
* removes NONCOPYABLE macro and instead uses Genode::Noncopyable and in-place
Constructors deletion
* introduces types Attr and Execute_attr where a constructor or execute method
have many arguments in order to raise readability
* renames the "hashsum" file that is provided by the Tresor Trust-Anchor VFS
plugin to "hash" in order to become conformant with the wording in the Tresor
lib
* makes the VFS Tresor test an automated test by merging in the functionality
of vfs_tresor_init.run and removing the interactive front end; removes
vfs_tresor_init.run as it is not needed anymore; adds consideration for
autopilot file structure in the Test and adds it to autopilot.list
* removes all snapshot controls and the progress files for rekeying and
extending from the Tresor VFS plugin; both functionalities were tested
only rudimentary by the VFS Tresor test and are not supported with the only
real user, the File Vault
* use /* .. */ instead of // ..
* use (..) instead of { .. } in init lists
Ref #5148
The virtual block device module used to hand over the wrong VBA as
parameter "rekeying VBA" to the Free Tree when allocating PBAs for data
access during rekeying. In certain constellations, this caused the Free
Tree to alloc PBAs that were still in use. The Free Tree PBA selection
algorithm, however, is just fine. When fixing the call parameter, it works
as desired. This re-enables the async rekeying test.
Ref #5075
The script tests the use of an encrypted file system that is created and
provided via the File Vault.
Furthermore the script can be used for test-driving existing File-Vault
containers (created with potentially older File-Vault versions) under the
current File-Vault version. This is done via the "LX_FS_DIR_TEMPLATE"
env variable.
Ref #5062
During one of the many re-factorization steps that were applied to the Tresor
library and its predecessor, the CBE library, one of the main features of the
project, the integrity check, accidentally received a grave regression. The
most recent version of the Tresor still used to check all hashes of meta-data
blocks but ignored the hashes of the actual data blocks.
With this commit, the hashes of all but yet uninitialized data blocks get
checked. The reason for ignoring uninitialized blocks is that they are not
actually read from disc but simply generated as an all-zeros block in the
driver in order to prevent having to initialize them all to zero in
Tresor-Init. That said, the integrity of these blocks cannot be compomised.
The according hashes in the meta data remain unset until the data block gets
written for the first time.
Ref #5062
The request classes Block_io::Read_client_data and Block_io::Write_client_data
used to receive a block reference for no reason. This commit removes these
args.
Ref #5062
The tresor_check tool became outdated back when the Tresor project was created
by re-writing its predecessor, the CBE, in C++. At this time, the check tool
was merely renamed but not updated. As there was also no autopilot test for the
tool, the tool remained outdated.
This commit rewrites the tool for the most recent Tresor version and adds an
autopilot test.
Ref #5062
* Make command pool a proper module
* The command pool used to be kind of a module but it was driven via custom
tresor-tester specific code. Now, it becomes a proper module that
is driven by the module framework instead.
* Move the code for creating and handling the module-execution progress flag
into Module_composition::execute_modules as the function is always used with
this code surrounding it.
* Reorganize files, remove deprecated files
* A new class Module_channel is introduced in the module framework and all
channel classes inherit from it. With that class in place, the formerly
module-specific implementations of the following methods are replaced by
new generic implementations in the Module framework:
* ready_to_submit_request
* submit_request
* _peek_completed_request
* _drop_completed_request
* _peek_generated_request
* _drop_generated_request
* generated_request_complete
* Module requests are now held for the duration of their lifetime at the
module they originate from and not, like before, at their target module. As
a result, modules can generate new requests inline (without having to wait
for the target module), making code much simpler to read, reducing the amount
of channel state, and allowing for non-copyable request types.
* Introduce a sub-state-machine for securing a superblock in the
superblock_control module in order to reduce redundancy.
* Some modules, like free_tree, were completely re-designed in order to make
them more readable.
* Replace all conditional exceptions by using the macros in
tresor/assertion.h .
* Move methods that are used in multiple modules but that were implemented
redundantly in each module to tresor/types.h.
* Remove verbosity node and all that was related to it from tresor tester
config as the targeted verbosity can be achieved with the
VERBOSE_MODULE_COMMUNICATION flag in tresor/verbosity.h .
* Extract the aspect of translating the byte-granular I/O-requests to
tresor-block requests from the tresor VFS-plugin and move it to a new module
called splitter.
* Rename the files and interface of the hashing back-end to not reflect the used
hashing algorithm/config anymore, while at the same time making the hashing
interface strict regarding the used types.
* Introduce the NONCOPYABLE macro that makes marking a class noncopyable short
and clear.
* Replace the former tresor/vfs_utilities.h/.cc with a new tresor/file.h
that contains the classes Read_write_file and Write_only_file. These classes
significantly simplify the modules crypto, block_io, and trust_anchor by
moving the details of file access to a sub-state machine.
* The former, rather trivial block allocator module is replaced by a normal
object of type Pba_allocator that must be provided by the client of the
Sb_initializer (reference in the Sb_initializer_request).
Ref #5062
tresor: read uninitialized vbas as all zeroes
Virtual addresses in a Tresor container that were not yet written by the user
should always return a data block that is all-zeroes. This was the concept
right from the beginning of the project. However, somehow this aspect either
never got implement or got lost along the way.
Some context for understanding the commit: The Tresor doesn't initialize the
payload data blocks of a container when creating a new container as this would
be rather expensive. Instead, it marks the leaf metadata nodes of the
virtual-block-device tree (those that reference the payload data blocks in
physical address space) with generation 0.
Now, this commit ensures that, whenever the virtual-block-device module reads
such a generation-0 leaf, instead of asking the block_io and crypto to deliver
data from disc, it directly provides the user with 4K of zeroes.
Ref #5062
The order of execution inside the Tresor lib slightly changed compared to the
previous CBE lib. AFAICT, this is nothing to worry about and related to the
now cleaner structuring. However, it can produce higher peak requirements
regarding the allocation pool in the Free Tree. Therefor, this commit extends
the dimensions of the Free Tree used in the test.
Ref #4971
* Implement requests "create snapshot" and "discard snapshot" in tresor lib.
* Adapt tresor tester in order to test the new feature.
* Remove temporary code from tresor tester that skipped such requests with
the hint that they were not supported yet.
* Add mandatory "id" attribute to <request op="create_snapshot"/> and
<request op="discard_snapshot"/> tag. A "discard snapshot" command always
refers to the snapshot created by the "create snapshot" command with the
same "id" value.
* Clean-up command pool a bit.
Fix#4971
The re-keying state machine in the VBD module would use block data of the wrong
block for the hash update of an inner node in a certain circumstance.
On re-keying, the VBD iterates for a given VBA over all snapshots, beginning
with the newest and re-keys the VBA in each of the snapshots. At each snapshot
it therefore loads the branch of the VBA top-down, and then updates the branch
bottom-up. However, if loading a certain level of the branch of a certain
snapshot runs into the same physical block as with the last snapshot on this
level, the algorithm turns around and updates the branch from this point
upwards instead of going further down the whole way to the leaf. This is
because everything below this point has already been re-keyed in the course of
a newer snapshot.
The case where this turning around is not right above the leaf (i.e., the first
shared physical block is a metadata block) that's were the bug was located. In
this situation, we have to re-encode the highest shared metadata block into a
buffer again before starting to update. The update code acts as if the
mentioned block was just written back (which is true when going down all the
way to the leaf before updating) and consequently is present in the encoded
buffer.
Ref #4971
Until now, it was possible to use bad Free-Tree/VBD configurations with the
<initialize/> command. The tresor tester didn't complaining about it but the
tresor lib crashed or, worse, corrupted the tresor container. Now, the tresor
tester checks things, like for instance, that "nr_of_children" must be a power
of 2.
Ref #4971
The Superblock Control module now issues a snapshot garbage collection on each
incoming request. In return for that, the commit removes all calls to the
garbage collection from other modules.
Ref #4971
The Virtual Block Device module used to create a local copy of the Snapshots
array respectively Snapshot root it received with an incoming request. After
finishing the VBD operation on the copy, the source module of the request
used to back-copy the resulting Snapshot array resp. Snapshot root. This is
not only less efficient than referencing but also allowed a bug to sneak into
the new C++ implementation.
In contrast to the old Ada/SPARK implementation (CBE), the new design doesn't
allow for global objects that can be accessed by any module without receiving a
reference in a module request. Therefore, the Free Tree module has to receive a
reference to a Snapshots array with each request in order to be able to use it.
In our case, these requests are allocations for a "Write" operation from the
VBD. However, the VBD itself receives only the one Snapshot required for
writing and therefore causes the Free Tree to make bad decisions on whether or
not a block can be re-allocated or not.
With this commit, the VBD always receive a reference to the whole Snapshots
array and also propagates it this way to the Free Tree.
Ref #4971
This is function gets called by some libssh applications using vms_lxip.
For the dummy implementation I looked at the old port.
Issue genodelabs#5161
Issue gapfruit#1976
- always assign apps/overlay to targets (visible=true/false) to
prevent 0x0 geometry, which is interpreted as close
- add QMenu as exampel to panel button
- use usb-tablet on Qemu
Per default, windows assigned to targets are visible, which can be
changed with the new boolean "visible" attribute. Thus, window can be
hidden without changing their geometry.
Before, the current back-most window was not restacked if it was part of
the already, which lead to partially inconsistent view of the window
stack between decorator and nitpicker.
The added hook 'OBJ_POSTPROC_SRC' gives us a way to post-process object
files for generating supplemental code. By using this hook, the
initcall_table.c generated by import-lx_emul_common.inc gets reliably
executed after all object files are built.
Fixes#5159
The option is used during the generation of initcall_table.c.
However, it happens to strip the first argument following the option.
The long option --defined-only works as expected.
Issue #5155
Due to a bug in the original implementation, the size of the MMIO
range covering the 'Request_sense_response' data was set too large
during the MMIO boundary change. This rendered devices that were not
yet ready and required an 'Request_sense' command unusable.
The commit also adapts all other commands where the MMIO size does
not match the expected one.
Fixes#5133.
The commit adds support to throttle the rate of the RX IRQs to a specified
value. The effect is, that no RX IRQs below the time threshold will fire and
therefore the CPU load gets reduced on the host. Trade-off gaming between
cpu load, throughput, overload.
Modular Sculpt 23.10 on S938 as testcase. In brackets the CPU affinity is
denoted.
ipxe (0,0) -> nic_router (1,0) -> Debian VM vbox6 (3,0) and (3,1)
VM: iperf -C X.X.X.X -t 60 -R
iperf server X.X.X.X is outside Sculpt and sends data due to '-R' to VM
Non representative measure points:
cpu load - ipxe - nic_router - iperf throughput
--------------------------------------------------
w/o patch - ~80% - ~50% - ~706 MBit/s - 0 -> throttling off by default on S938
patch 651 - ~20% - ~35% - ~763 MBit/s - 651 -> 0.166ms throttle RX IRQ
patch 5580 - ~15% - ~25% - ~650 MBit/s - 5580 -> 1.4ms throttle RX IRQ
Issue #5149
A bunch of transmit requests received by the Uplink server (nic_router)
are currently added one by one to the ring buffer and every time the hardware
is notified to process each single request.
Instead, add as many as possible transmit requests in the ring buffer of
the hardware and when done trigger the hardware to process the ring.
Additionally, don't receive an "processed" TX IRQ for each element in the
ring, which causes high CPU load.
With this commit the TX IRQs in the ipxe driver for a
iperf -c X.X.X.X -t 60
from within a VM to the outside iperf server is reduced from about
~2'600'000 IRQs to about ~200'000. The overall CPU load for the driver
(when executed alone on CPU 0) is reduced from ~85 percent load to ~45 percent
load.
Issue #5149
during receive the nic_ep may block as long as the guest does not provide
another receive network descriptor. In the meantime, all Genode signals
regarding the network interface, e.g. tx, will be postponed, which may
effect the throughput.
Instead use the nic_ep for rx packets unblocking. Add an notification mechanism
to the e1000 vbox network model, to notify us as soon as the guest added new
receive descriptors in the model.
Issue #5146
For pbxa9, Qemu is started with only 256 MiB for foc but with 768 MiB
for base-hw. By reducing the RAM quota for all start nodes within the
remote scenario, each component gets enough RAM quota to breathe.
