ExternalVendorCode/heads
1
0
Fork 0
mirror of https://github.com/linuxboot/heads.git synced 2024-12-18 20:47:55 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

README.md Makefile: address comments in PR review for daily/non-daily Nix users, remove NIX_REPRO_NOTES, Makefile dev helpers self-explain themselves

Browse Source 
Signed-off-by: Thierry Laurion <insurgo@riseup.net>
This commit is contained in:
Thierry Laurion 2024-05-10 11:37:05 -04:00
parent c7f652bf89
commit 181ce621bb
No known key found for this signature in database
GPG Key ID: 9A53E1BB3FF00461
3 changed files with 44 additions and 100 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

36
Makefile
View File

@ -819,6 +819,8 @@ $(board_build)/$(CB_OUTPUT_BASENAME)-gpg-injected.rom: $(board_build)/$(CB_OUTPU
./bin/inject_gpg_key.sh --cbfstool "$(build)/$(coreboot_dir)/cbfstool" \
"$(board_build)/$(CB_OUTPUT_FILE_GPG_INJ)" "$(PUBKEY_ASC)"
#Dev cycles helpers:
real.clean:
for dir in \
$(module_dirs) \
@ -830,20 +832,28 @@ real.clean:
done
cd install && rm -rf -- *
real.gitclean:
#Use git ignore file as a base to wipe everything not in tree. Keeps coreboot forks downloaded since detected as git repos, wipes the rest.
@echo "Cleaning the repository using Git ignore file as a base..."
@echo "This will wipe everything not in the Git tree, but keep downloaded coreboot forks (detected as Git repos)."
git clean -fxd
real.gitclean_keep_packages:
#Same as above but keep the packages downloaded to save bandwidth
@echo "Cleaning the repository using Git ignore file as a base..."
@echo "This will wipe everything not in the Git tree, but keep the 'packages' directory."
git clean -fxd -e "packages"
real.remove_canary_files-extract_patch_rebuild_what_changed:
#Another approach is to remove the "canary" files
# This forces Heads to restart building a board config by checking packages integrity, extracting them, redoing patching on files and rebuilding what needs to be rebuilt
# reinstalling what is needed under ./install as well which is what we normally want on a development cycle.
#Limitations: if for whatever reason, a patch creates a file in an extracted package dir, this approach will fail without further manual actions
# This is not so bad though: git patch apply tells you exactly which file couldn't be created as expected. Just delete those files and relaunch the build and it will succeed.
#This approach economizes a lot of time since most of the build artifacts do not need to be rebuilt since the dates of the files should be the same as when you originally built them.
# So only a minimal time is needed to rebuild, and this is also good for your SSD.
#**** USE THIS APPROACH FIRST ***
find ./build/ -type f -name ".canary" | xargs rm || echo "All .carnary files already deleted"
find ./install/*/* | xargs rm -rf || echo "All install/ARCH/* dirs and files already deleted"
echo "you can now call make BOARD=desired_board, and if any patch fails to apply because file exists; just rm that build/path_to_file and continue testing!"
@echo "Removing 'canary' files to force Heads to restart building board configurations..."
@echo "This will check package integrity, extract them, redo patching on files, and rebuild what needs to be rebuilt."
@echo "It will also reinstall the necessary files under './install'."
@echo "Limitations: If a patch creates a file in an extracted package directory, this approach may fail without further manual actions."
@echo "In such cases, Git will inform you about the file that couldn't be created as expected. Simply delete those files and relaunch the build."
@echo "This approach economizes time since most build artifacts do not need to be rebuilt, as the file dates should be the same as when you originally built them."
@echo "Only a minimal time is needed for rebuilding, which is also good for your SSD."
@echo "*** USE THIS APPROACH FIRST ***"
find ./build/ -type f -name ".canary" -print -delete
find ./install/*/* -print -exec rm -rf {} +
real.gitclean_keep_packages_and_build:
@echo "Cleaning the repository using Git ignore file as a base..."
@echo "This will wipe everything not in the Git tree, but keep the 'packages' and 'build' directories."
git clean -fxd -e "packages" -e "build"

85
NIX_REPRO_NOTES
View File

@ -1,85 +0,0 @@
Notes to repro on top of QubesOS debian-12-xfce template based qube
- Clone a debian-12-xfce on top of Q4.2.1 to be specialized
- Deploy QubesOS nix requirements in template: https://dataswamp.org/~solene/2023-05-15-qubes-os-install-nix.html
- Deploy QubesOS requirements for docker: https://gist.github.com/tlaurion/9113983bbdead492735c8438cd14d6cd
- Create a qube based on the cloned template above
- Open qube
- Clone whatever repo is most recent between https://github.com/mmlb/osresearch-heads/tree/wip-nix-for-build or https://github.com/tlaurion/heads/tree/wip-nix-for-build
- git clone Above_repo
- cd /home/user/heads or cd /home/user/osresearch-heads
- git checkout wip-nix-for-build
- git reset --hard
Now, replication traces
#Install nix cleanly:
sudo rm -rf /nix/* || echo "cannot delete /nix" && sh <(curl -L https://nixos.org/nix/install) --no-daemon
#Source nix prior of creating nix cache and jumping in it
. /home/user/.nix-profile/etc/profile.d/nix.sh
#Configure nix for local builds until we have proper docker image instead:
mkdir -p ~/.config/nix
echo 'experimental-features = nix-command flakes' >~/.config/nix/nix.conf
#Build nix developer local env with flakes locks to specified versions and jump into it:
nix --print-build-logs --verbose develop --ignore-environment
#Build a board
make BOARD=nitropad-nv41
In case you need to rebuild from packages+ patches + a clean ./install (where libraries and headers are found):
make real.remove_canary_files-extract_patch_rebuild_what_changed
IF that doesn;t help you, look at the end of the global Helper for more helper. Iterate.
build docker out of nix develop environement and jump into it:
nix build .#dockerImage && docker load < result && docker run --network host --rm -ti -v $(pwd):$(pwd) -w $(pwd) linuxboot/heads:dev-env
From there, rebuild with
make real.remove_canary_files-extract_patch_rebuild_chat_changed
make BOARD=nitropad-nv41
Make changes until taht part works in both nix layer on top of OS and withing docker image.
......
SUCCESSSSS?!?!?!?! CELEBRATION!
Tell @mmlb and @insurgo on matrix channel!!!!
-------
Some raw and incomplete notes on how to push that docker to dockerhub
docker tag linuxboot/heads:dev-env tlaurion/heads-dev-env:v0.0.3
docker push tlaurion/heads-dev-env:v0.0.3
(to be continued)
-------------------
#To create the docker image, using garbage collector didn't help and made the size of the image grow bigger?! not using it reverting to what worked
# produced image without garbage collector was 990 Mb image, with it (and stripping which is already done by nix builder...) its 1006Mb. Over my head
# TODO:
# - revisit later, should not block merge of pr
# - do doc under heads's README.md and then heads-wiki
sh <(curl -L https://nixos.org/nix/install) --no-daemon
. /home/user/.nix-profile/etc/profile.d/nix.sh
#And then build derivates
nix build .#dockerImage && docker load < result && docker run --network host --rm -ti -v $(pwd):$(pwd) -w $(pwd) linuxboot/heads:dev-env
#Use it
docker run -e DISPLAY=$DISPLAY --network host --rm -ti -v $(pwd):$(pwd) -w $(pwd) linuxboot/heads:dev-env -- make BOARD=qemu-coreboot-whiptail-tpm2
docker run -e DISPLAY=$DISPLAY --network host --rm -ti -v $(pwd):$(pwd) -w $(pwd) linuxboot/heads:dev-env -- make BOARD=qemu-coreboot-whiptail-tpm2 run
#push it
docker login
docker tag linuxboot/heads:dev-env tlaurion/heads-dev-env:v0.1.1
docker push tlaurion/heads-dev-env:v0.1.1
# For end-users, CircleCI
docker run -e DISPLAY=$DISPLAY --network host --rm -ti -v $(pwd):$(pwd) -w $(pwd) laurion/heads-dev-env:v0.1.1 -- make BOARD=qemu-coreboot-whiptail-tpm2
docker run -e DISPLAY=$DISPLAY --network host --rm -ti -v $(pwd):$(pwd) -w $(pwd) laurion/heads-dev-env:v0.1.1 -- make BOARD=qemu-coreboot-whiptail-tpm2 run

23
README.md
View File

@ -39,6 +39,7 @@ Under QubesOS?
Build docker from nix develop layer locally
====
If you do not use Nix on a daily basis:
```
# DANGER: remove /nix store and recreates a fresh one. Skip if you use Nix already:
sudo rm -rf /nix/* || echo "cannot delete /nix" && sh <(curl -L https://nixos.org/nix/install) --no-daemon
@ -47,14 +48,31 @@ mkdir -p ~/.config/nix
echo 'experimental-features = nix-command flakes' >~/.config/nix/nix.conf
# Source nix prior of anything else:
. /home/user/.nix-profile/etc/profile.d/nix.sh
# END OF DANGER SECTION TO BE REVIEWED
# [...]
```
If you use Nix on a daily basis:
```
# Make sure your nix setup supports both nix-command and flakes experimental features:
mkdir -p ~/.config/nix
echo 'experimental-features = nix-command flakes' >> ~/.config/nix/nix.conf
# Review ~/.config/nix/nix.conf for inconsistencies in your favorite editor (vim, vi, gedit etc)
# Build nix developer local env with flakes locks to specified versions and exits just running "true" command:
nix --print-build-logs --verbose develop --ignore-environment --command true
# Build docker image with current develop created environment (this will take a while and create "linuxboot/heads:dev-env" local docker image:
nix build .#dockerImage && docker load < result
```
Common steps to follow to build local doscker image from nix develop environment:
```
# Build nix developer local env with flakes locks to specified versions and exits just running "true" command:
nix --print-build-logs --verbose develop --ignore-environment --command true
# Build docker image with current develop created environment (this will take a while and create "linuxboot/heads:dev-env" local docker image:
nix build .#dockerImage && docker load < result
```
Done!
Your local docker image "linuxboot/heads:dev-env" is ready to use, reproducible for the specific Heads commit used and will produce ROMs reproducible for that Heads commit ID.
Jump into nix develop created docker image for interactive workflow
=====
@ -81,6 +99,7 @@ Alternatively, you can use locally built docker image to build a board ROM image
Eg:
`docker run -e DISPLAY=$DISPLAY --network host --rm -ti -v $(pwd):$(pwd) -w $(pwd) linuxboot/heads:dev-env -- make BOARD=nitropad-nv41`
Pull docker hub image to prepare reproducible ROMs as CircleCI in one call
====
```