ExternalVendorCode/heads
1
0
Fork 0
mirror of https://github.com/linuxboot/heads.git synced 2024-12-18 20:47:55 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge pull request #1855 from tlaurion/docker_helpers

Browse Source 
add 3 Docker users/build helpers : local_dev, repro and latest, update README.md to simplify usage
This commit is contained in:
Thierry Laurion 2024-11-21 11:44:49 -05:00 committed by GitHub
commit ee8d1d9ae8
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
4 changed files with 207 additions and 12 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

28
README.md
View File

@ -55,10 +55,10 @@ Build docker from nix develop layer locally
#### Build image #### Build image
* Have docker and Nix installed
* Build nix developer local environment with flakes locked to specified versions * Build nix developer local environment with flakes locked to specified versions
* `nix --print-build-logs --verbose develop --ignore-environment --command true` * `./docker_local_dev.sh`
* Build docker image with current develop created environment (this will take a while and create "linuxboot/heads:dev-env" local docker image):
* `nix --print-build-logs --verbose build .#dockerImage && docker load < result`
On some hardened OSes, you may encounter problems with ptrace. On some hardened OSes, you may encounter problems with ptrace.
``` ```
@ -75,12 +75,16 @@ sudo sysctl -w kernel.yama.ptrace_scope=1 #setup the value to let nix+docker run
Done! Done!
Your local docker image "linuxboot/heads:dev-env" is ready to use, reproducible for the specific Heads commit used and will produce ROMs reproducible for that Heads commit ID. Your local docker image "linuxboot/heads:dev-env" is ready to use, reproducible for the specific Heads commit used to build it, and will produce ROMs reproducible for that Heads commit ID.
Jump into nix develop created docker image for interactive workflow Jump into nix develop created docker image for interactive workflow
==== ====
`docker run -e DISPLAY=$DISPLAY --network host --rm -ti -v $(pwd):$(pwd) -w $(pwd) linuxboot/heads:dev-env` There is 3 helpers:
- `./docker_local_dev.sh`: for developers wanting to customize docker image built from flake.nix(nix devenv creation) and flake.lock (pinned versions used by flake.nix)
- `./docker_latest.sh`: for Heads developers, wanting to use latest published docker images to develop Heads
- `./docker_repro.sh`: versioned docker image used under CircleCI to produce reproducivle builds, both locally and under CircleCI. **Use this one if in doubt**
ie: `./docker_repro.sh` will jump into CircleCI used versioned docker image for that Heads commit id to build images reproducibly if git repo is clean (not dirty).
From there you can use the docker image interactively. From there you can use the docker image interactively.
@ -92,22 +96,22 @@ Please refer to [qemu documentation](targets/qemu.md) for more information.
Eg: Eg:
``` ```
make BOARD=qemu-coreboot-fbwhiptail-tpm2 # Build rom, export public key to emulated usb storage from qemu runtime ./docker_repro.sh make BOARD=qemu-coreboot-fbwhiptail-tpm2 # Build rom, export public key to emulated usb storage from qemu runtime
make BOARD=qemu-coreboot-fbwhiptail-tpm2 PUBKEY_ASC=~/pubkey.asc inject_gpg # Inject pubkey into rom image ./docker_repro.sh make BOARD=qemu-coreboot-fbwhiptail-tpm2 PUBKEY_ASC=~/pubkey.asc inject_gpg # Inject pubkey into rom image
make BOARD=qemu-coreboot-fbwhiptail-tpm2 USB_TOKEN=Nitrokey3NFC PUBKEY_ASC=~/pubkey.asc ROOT_DISK_IMG=~/qemu-disks/debian-9.cow2 INSTALL_IMG=~/Downloads/debian-9.13.0-amd64-xfce-CD-1.iso run # Install ./docker_repro.sh make BOARD=qemu-coreboot-fbwhiptail-tpm2 USB_TOKEN=Nitrokey3NFC PUBKEY_ASC=~/pubkey.asc ROOT_DISK_IMG=~/qemu-disks/debian-9.cow2 INSTALL_IMG=~/Downloads/debian-9.13.0-amd64-xfce-CD-1.iso run # Install
``` ```
Alternatively, you can use locally built docker image to build a board ROM image in a single call. Alternatively, you can use locally built docker image to build a board ROM image in a single call **but do not expect reproducible builds if not using versioned docker images as per CircleCI as per usage of `./docker_repro.sh`**
Eg: Eg:
`docker run -e DISPLAY=$DISPLAY --network host --rm -ti -v $(pwd):$(pwd) -w $(pwd) linuxboot/heads:dev-env -- make BOARD=nitropad-nv41` `./docker_local_dev.sh make BOARD=nitropad-nv41`
Pull docker hub image to prepare reproducible ROMs as CircleCI in one call Pull docker hub image to prepare reproducible ROMs as CircleCI in one call
==== ====
``` ```
docker run -e DISPLAY=$DISPLAY --network host --rm -ti -v $(pwd):$(pwd) -w $(pwd) tlaurion/heads-dev-env:latest -- make BOARD=x230-hotp-maximized ./docker_repro.sh make BOARD=x230-hotp-maximized
docker run -e DISPLAY=$DISPLAY --network host --rm -ti -v $(pwd):$(pwd) -w $(pwd) tlaurion/heads-dev-env:latest -- make BOARD=nitropad-nv41 ./docker_repro.sh make BOARD=nitropad-nv41
``` ```
Maintenance notes on docker image Maintenance notes on docker image

49
docker_latest.sh Executable file
View File

@ -0,0 +1,49 @@
#!/bin/bash
# Inform the user that the latest published Docker image is being used
echo "Using the latest Docker image: tlaurion/heads-dev-env:latest"
# Function to display usage information
usage() {
echo "Usage: $0 [OPTIONS] -- [COMMAND]"
echo "Options:"
echo " CPUS=N Set the number of CPUs"
echo " V=1 Enable verbose mode"
echo "Command:"
echo " The command to run inside the Docker container, e.g., make BOARD=BOARD_NAME"
}
# Function to kill GPG toolstack related processes using USB devices
kill_usb_processes() {
echo "Killing any GPG toolstack related processes on host currently using USB devices..."
sudo lsof /dev/bus/usb/00*/0* 2>/dev/null | awk 'NR>1 {print $2}' | xargs -r ps -p | grep -E 'scdaemon|pcscd' | awk '{print $1}' | xargs -r sudo kill -9
if [ $? -ne 0 ]; then
echo "Failed to kill GPG toolstack related processes using USB devices. Please run the following command manually:"
echo "sudo lsof /dev/bus/usb/00*/0* | awk 'NR>1 {print \$2}' | xargs -r ps -p | grep -E 'scdaemon|pcscd' | awk '{print \$1}' | xargs -r sudo kill -9"
exit 1
fi
}
# Handle Ctrl-C (SIGINT) to exit gracefully
trap "echo 'Script interrupted. Exiting...'; exit 1" SIGINT
# Check if --help or -h is provided
for arg in "$@"; do
if [[ "$arg" == "--help" || "$arg" == "-h" ]]; then
usage
exit 0
fi
done
# Kill processes using USB devices
kill_usb_processes
# Inform the user about entering the Docker container
echo "----"
echo "Usage reminder: The minimal command is 'make BOARD=XYZ', where additional options, including 'V=1' or 'CPUS=N' are optional."
echo "For more advanced QEMU testing options, refer to targets/qemu.md and boards/qemu-*/*.config."
echo "----"
echo "Entering the Docker container. Type 'exit' to return to the host shell."
# Execute the docker run command with the provided parameters
docker run --device=/dev/bus/usb:/dev/bus/usb -e DISPLAY=$DISPLAY --network host --rm -ti -v $(pwd):$(pwd) -w $(pwd) tlaurion/heads-dev-env:latest -- "$@"

84
docker_local_dev.sh Executable file
View File

@ -0,0 +1,84 @@
#!/bin/bash
# Check if Nix is installed
if ! command -v nix &> /dev/null; then
echo "Nix is not installed or not in the PATH. Please install Nix before running this script."
echo "Refer to the README.md at the root of the repository for installation instructions."
exit 1
fi
# Check if Docker is installed
if ! command -v docker &> /dev/null; then
echo "Docker is not installed or not in the PATH. Please install Docker before running this script."
echo "Refer to the README.md at the root of the repository for installation instructions."
exit 1
fi
# Inform the user about the Docker image being used
echo "** This ./docker_local_dev.sh script is for developers usage only. **"
echo ""
echo "Using the last locally produced Docker image: linuxboot/heads:dev-env"
echo "Warning: Using anything other than the published Docker image might lead to non-reproducible builds."
echo ""
echo "For using the latest published Docker image, refer to ./docker_latest.sh."
echo "For producing reproducible builds as CircleCI, refer to ./docker_repro.sh."
echo ""
echo "---"
# Function to display usage information
usage() {
echo "Usage: $0 [OPTIONS] -- [COMMAND]"
echo "Options:"
echo " CPUS=N Set the number of CPUs"
echo " V=1 Enable verbose mode"
echo "Command:"
echo " The command to run inside the Docker container, e.g., make BOARD=BOARD_NAME"
}
# Function to kill GPG toolstack related processes using USB devices
kill_usb_processes() {
echo "Killing any GPG toolstack related processes on host currently using USB devices..."
sudo lsof /dev/bus/usb/00*/0* 2>/dev/null | awk 'NR>1 {print $2}' | xargs -r ps -p | grep -E 'scdaemon|pcscd' | awk '{print $1}' | xargs -r sudo kill -9
if [ $? -ne 0 ]; then
echo "Failed to kill GPG toolstack related processes using USB devices. Please run the following command manually:"
echo "sudo lsof /dev/bus/usb/00*/0* | awk 'NR>1 {print \$2}' | xargs -r ps -p | grep -E 'scdaemon|pcscd' | awk '{print \$1}' | xargs -r sudo kill -9"
exit 1
fi
}
# Handle Ctrl-C (SIGINT) to exit gracefully
trap "echo 'Script interrupted. Exiting...'; exit 1" SIGINT
# Check if --help or -h is provided
for arg in "$@"; do
if [[ "$arg" == "--help" || "$arg" == "-h" ]]; then
usage
exit 0
fi
done
# Check if the git repository is dirty and if flake.nix or flake.lock are part of the uncommitted changes
if [ -n "$(git status --porcelain | grep -E 'flake\.nix|flake\.lock')" ]; then
echo "Warning: Uncommitted changes detected in flake.nix or flake.lock. The Docker image will be rebuilt."
echo "If this was not intended, please commit your changes and rerun the script."
echo "Building the Docker image from flake.nix..."
nix --print-build-logs --verbose develop --ignore-environment --command true
nix --print-build-logs --verbose build .#dockerImage && docker load < result
else
echo "Git repository is clean. Using the previously built Docker image."
echo "---"
sleep 1
fi
# Kill processes using USB devices
kill_usb_processes
# Inform the user about entering the Docker container
echo "----"
echo "Usage reminder: The minimal command is 'make BOARD=XYZ', where additional options, including 'V=1' or 'CPUS=N' are optional."
echo "For more advanced QEMU testing options, refer to targets/qemu.md and boards/qemu-*/*.config."
echo "----"
echo "Entering the Docker container. Type 'exit' to return to the host shell."
# Execute the docker run command with the provided parameters
docker run --device=/dev/bus/usb:/dev/bus/usb -e DISPLAY=$DISPLAY --network host --rm -ti -v $(pwd):$(pwd) -w $(pwd) linuxboot/heads:dev-env -- "$@"

58
docker_repro.sh Executable file
View File

@ -0,0 +1,58 @@
#!/bin/bash
# Extract the Docker image version from the CircleCI config file
DOCKER_IMAGE=$(grep -oP '^\s*-?\s*image:\s*\K(tlaurion/heads-dev-env:[^\s]+)' .circleci/config.yml | head -n 1)
# Check if the Docker image was found
if [ -z "$DOCKER_IMAGE" ]; then
echo "Error: Docker image not found in .circleci/config.yml"
exit 1
fi
# Inform the user about the versioned CircleCI Docker image being used
echo "Using CircleCI Docker image: $DOCKER_IMAGE"
# Function to display usage information
usage() {
echo "Usage: $0 [OPTIONS] -- [COMMAND]"
echo "Options:"
echo " CPUS=N Set the number of CPUs"
echo " V=1 Enable verbose mode"
echo "Command:"
echo " The command to run inside the Docker container, e.g., make BOARD=BOARD_NAME"
}
# Function to kill GPG toolstack related processes using USB devices
kill_usb_processes() {
echo "Killing any GPG toolstack related processes on host currently using USB devices..."
sudo lsof /dev/bus/usb/00*/0* 2>/dev/null | awk 'NR>1 {print $2}' | xargs -r ps -p | grep -E 'scdaemon|pcscd' | awk '{print $1}' | xargs -r sudo kill -9
if [ $? -ne 0 ]; then
echo "Failed to kill GPG toolstack related processes using USB devices. Please run the following command manually:"
echo "sudo lsof /dev/bus/usb/00*/0* | awk 'NR>1 {print \$2}' | xargs -r ps -p | grep -E 'scdaemon|pcscd' | awk '{print \$1}' | xargs -r sudo kill -9"
exit 1
fi
}
# Handle Ctrl-C (SIGINT) to exit gracefully
trap "echo 'Script interrupted. Exiting...'; exit 1" SIGINT
# Check if --help or -h is provided
for arg in "$@"; do
if [[ "$arg" == "--help" || "$arg" == "-h" ]]; then
usage
exit 0
fi
done
# Kill processes using USB devices
kill_usb_processes
# Inform the user about entering the Docker container
echo "----"
echo "Usage reminder: The minimal command is 'make BOARD=XYZ', where additional options, including 'V=1' or 'CPUS=N' are optional."
echo "For more advanced QEMU testing options, refer to targets/qemu.md and boards/qemu-*/*.config."
echo "----"
echo "Entering the Docker container. Type 'exit' to return to the host shell."
# Execute the docker run command with the provided parameters
docker run --device=/dev/bus/usb:/dev/bus/usb -e DISPLAY=$DISPLAY --network host --rm -ti -v $(pwd):$(pwd) -w $(pwd) $DOCKER_IMAGE -- "$@"