Non-impactful action, first step for #1421 based on participation in testing of #1398 and prior non-tested PRs.
EDIT: last minute readd of x220-maximized boards (x220-maximized and x220-hotp-maximized boards).
x220 is still UNTESTED (legacy, manually extracting ifs, me and gbe).
EDIT: last minute readd of t440p-maximized boards (t440p-maximized and t440p-hotp-maximized boards).
Thanks to @srgrint for lat minute report that t440p and x220 were tested
----
Traces of commands used:
ls qemu-linuxboot* leopard* r630* s2600wf* tioga* winterfell* t420* t520* t440p* w530* kgpe* p8z77* x220* x230-maximized-fhd_edp* | grep ":" | awk -F ":" {'print $1'}| while read board; do mv $board/$board.config $board/UNTESTED_$board.config; done
ls qemu-linuxboot* leopard* r630* s2600wf* tioga* winterfell* t420* t520* t440p* w530* kgpe* p8z77* x220* x230-maximized-fhd_edp* | grep ":" | awk -F ":" {'print $1'}| while read dir; do mv $dir UNTESTED_$dir; done
ls UNTESTED* | grep ":" | awk -F ":" {'print $1'}| awk -F "UNTESTED_" {'print $2'} | while read line; do sed 's/'"$line"'/UNTESTED_'"$line"'/g' ../.circleci/config.yml -i ; done
quick fix of circleci:
sed -i 's/UNTESTED_UNTESTED/UNTESTED/g' ../.circleci/config.yml
sed -i 's/UNTESTED_UNTESTED/UNTESTED/g' ../.circleci/config.yml
sed -i 's/UNTESTED_UNTESTED/UNTESTED/g' ../.circleci/config.yml
Modify p8z77-m_pro-tpm1 hotp board config to include to their maximized counterpart
- Based on initial server board
- Uses whiptail as opposed to fbwhiptail (was slow and output fuzzy)
- Simple fix to have dual KVM(BMC) and vga output for consoles
Reasoning for dropping fbwhiptail support is that:
- it is impossible to output framebuffer content through remote BMC console.
- A workstation board config could output to fbwhiptail for VGA and give remote recovery shell access through BMC
- If someone shows interest for that, qemu-coreboot-tpm boards can be used as reference.
- slowness/fuzzyness of fbwhiptail output through AST would still need to be fixed in kernel drivers. Not a priority here.
Limitation:
- Since whiptail is sent to both consoles:
- If one console goes to recovery shell, recovery shell access invalidate TPM PCR4 measurements.
- The other console won't be aware that TPM measurements were invalidated, and will consequently:
- not be able to unseal TOTP if refreshed
- not be able to unseal TPM disk unlock key on default boot
- A reboot will fix this.
This change will improve build times by allowing the T440p to share the
Coreboot 4.17 cache with the Librem boards. Once we update the other
ThinkPad boards to use Coreboot 4.19, we'll make the T440p depend on the
X230 again.
Co-authored-by: Thierry Laurion <insurgo@riseup.net>
- Add 4.19 under modules/coreboot
- point all 4.13 boards to 4.19
- adapt x230 FHD/EDP patch under patches/coreboot-4.19/0001-x230-fhd-variant.patch (poked upstream to fix patch under https://review.coreboot.org/c/coreboot/+/28950)
- correct versioning info under .circleci/config/yml
- add x230-maximized-fhd_edp and x230-hotp-maximized-fhd_edp board configs
- add/rework coreboot patch for x230 fhd variant to be applied on top of 4.13
- add coreboot config to point to x230-edp variant, fixing path to vbt file since default path is wrong under. Comment made upstream https://review.coreboot.org/c/coreboot/+/28950/22#message-4904ce82f01ba0505b391e072e4537b6a9f1a229
- remove no gfx init and replace with libgfxinit(defonfig default), set internal display as default
- add x230-hotp-maximized-fhd_edp and x230-maximized-fhd_edp to CircleCI builds
- One single shared coreboot config between boards/x230-hotp-maximized-fhd_edp/x230-hotp-maximized-fhd_edp.config and boards/x230-maximized-fhd_edp/x230-maximized-fhd_edp.config
- Coreboot 4.13 patch from coreboot at patches/coreboot-4.13/0002-x230-fhd-variant.patch
- config/coreboot-x230-maximized-fhd_edp.config points to seperate coreboot config per patch (CONFIG_BOARD_LENOVO_X230_EDP)
This also involves splitting workspaces based on target architecture to
avoid severely degrading performance of CI.
Signed-off-by: Sergii Dmytruk <sergii.dmytruk@3mdeb.com>
Global Makefile is the most effective modifier of builds.
As soon as the global Makefile change, so should not be reused caches having measured a different Makefile
Coreboot 4.11 boards are not properly building as of now.
coreboot.pre fails to depend on .car.data because of a race condition that can only be mitigated by single threading CPUS=
This is unrelated to other changes.
KGPE-D16 will soon enough depend on dasharo coreboot and be ported upstream later on.
Heads buildstystem:
Makefile logic will download modules packages under ./packages, check itheir integrity, then extract it and patch extraction directory ONLY if no corresponding .*_verify files are found under ./packages directory. They are extracted under build/modulename-ver/ where patches are applied prior of building them.
build/module* .configured is written when packages are configured under build/modulename-ver/.configured
build/modules* .build is written when packages are built under build/modulename-ver/.build
CircleCI caching subsystem notes:
A cache name tag is calculated in the prep_env stage early at each beginning of a workflow, and consists of a cache name, appended by a calculated digest signature (which is the final hash of hashed files (the hash of a digest).
Look for the following under .circleci/config.yml:
"Creating .... digest statements" : they are basically files passed under sha256sum to create a digest.
restore_cache keys: they are basically a string concatenating: name + checksum of digest + CACHE_VERSION. Only the first cache is extracted following declared order.
save_cache keys: same as above, only saving non-existing caches. That is, skipping existing ones and creating missing ones.
A cache is extracted at the beginning of a workflow if an archive matches an archive name, which consists of a name tag + digest hash + CACHE_VERSION
A cache is created only at the end of a workflow ("Saving cache...").
Caches are specialized. Caches are linked to checkumming of some content. And the largest available cache is extracted on next workflow, only extracting the directories/files that were contained in that cache.
A workspace cache ("Attaching workspace..."), as opposed to a end workflow cache, is passed along steps that depends on prior workflow, as specified under CirclecI config. The current CircleCI config creates a workspace cache for:
make + gawk + musl-cross-make (passed along next)
the most massive board config for each coreboot version (passed along next)
which is finally leading to the workflow cache, specialized for different content that should not change across builds.
That is 3 caches
musl-cross-make and bootstrapping tools (builds make and gawk locally) as long as musl-cross module has same checksum
a coreboot cache, containing all coreboot building directories, as long as coreboot module and patches are having the same hashes
a global cache containing alla builds artifacts (build dir, install dir, musl-cross dir etc)
Consequently, a workspace cache contains all the files under a path that is specified. For heads running under CircleCI, this is ~/project, which is basically "heads" checked out GitHub project, and everything being built under it.
When a workflow is successful, save_cache is ran, constructing caches for digest hashes that are not yet saved (which corresponds to a hash matching muslc-cross module hash, coreboot+patches digest hash and another one for all modules and patches digest hash.
On next workspace iteration, pre_env step will include a "Restore cache" step, which will use the largest cache available and extract it prior of passing it as workspace caches. This is why there is no such different in build time when building on a clean build (the workspace caches layers are smaller, and passed along. This means saving it, passing it. next workspace downloads extracts and builds on top of those smaller layers), as opposed to a workspace reusing and repassing the bigger workspaces containing the whole cache (bigger initial cache extract, then compressing and saving it to be passed as a workspace layer that is then downloaded, extracted, building on top, compressing and saving which then passed as a workspace cache to the next layer depending on it).
And finally, the caching system (save_cache, restore_cache) is based on a CircleCI environment variable named CACHE_VERSION which is appended at the end of the checkum fingerprint of a named cache. It can at any moment be changed to wipe actually used cache, if for some reason it is broken.
Consequently:
CircleCI cache should include packages cache (so that packages are downloaded and verified only once.)
Heads Makefile only downloads, checks and extracts packages and then patch extracted directory content if packages/.module-version_verify doesn't exist. This was missing, causing coreboot tarballs to be redownloaded (not present under packages) and reextracted and repatched (since _verify file was not present under packages/*_verify)
- Add kgpe-d16 patch to remove HID for PCI devices (successful build on top of #1101 and #1012 per https://app.circleci.com/pipelines/github/tlaurion/heads/937/workflows/de49bea0-3f58-4a91-8891-87622f5a0eed)
- CircleCI modified to build for coreboot 4.11 kgpe-d16_workstation on top of 4.15 passed workspace
- CircleCI modified so that we still archive all the logs in artifacts for the current build even if failing. We now exit 1 after having archived all the log files under build/
- Add xx30 vbios extract scripts to test. Expecting musl-cross target to fail since make and gawk aren't built
- CircleCI: gawk was not installed in apt statements under Debian. Installing
- Makefile: seperate and fix local make and gawk building pror of using. Otherwise, impossible to build musl-cross target seperatly.
- Also give some debugging info at start of Heads builds to tell which local gawk and make are used, also telling which make call will be propagated in the rest of the builds
- Fix gawk version checking, reporting bad version even if 4.2.1 as expected on debian-10 (debian-10 OS deploys gawk and make in version 4.2.1)
- CircleCI: Changing musl-cross taget to bootstrap (gawk+make) and musl-cross-make (bootstrap_musl-cross-make) for clarity
- Add kgpe-d16 patch to remove HID for PCI devices (successful build on top of #1101 and #1012 per https://app.circleci.com/pipelines/github/tlaurion/heads/937/workflows/de49bea0-3f58-4a91-8891-87622f5a0eed)
- CircleCI modified to build for coreboot 4.11 kgpe-d16_workstation on top of 4.15 passed workspace
- CircleCI modified so that we still archive all the logs in artifacts for the current build even if failing. We now exit 1 after having archived all the log files under build/
-CircleCI addition.
-Removal of t530-flash, w530-flash boards, flash scripts and associated coreboot configs (no more legacy boards additions)
This is a merger of #1071, #1072 and #1073 so that test builds are available over CircleCI until osresearch/master CircleCI gets unlocked.
CircleCI: We currently drop coreboot 4.11 builds.
- There is a file missing in the builds. Not sure why/how this is happening
src/soc/intel/fsp_broadwell_de/romstage/romstage.c:41:10: fatal error: build.h: No such file or directory
Example:https://app.circleci.com/pipelines/github/tlaurion/heads/877/workflows/7d0248d2-459c-42ad-b741-8fd56a75d527/jobs/2487
- kgpe-d16_workstation building for all GPUs is unfortunately taking too much time to build (40 minutes).
- Not sure why, but it seems that the kernel build paralellization is not working for 4.11 while it works for 4.13
Makefile: Uncomment MAKE_JOBS which passes the number of jobs to numbers cores by default and --max-load of 16
CircleCI: Remove CPUS statement to use Makefile default
modules/newt: force build with one make job, otherwise there is a race condition in module which fails randomly expecting build modules. (TODO: FIX)
Interestingly, building all coreboot 4.13 boards is happening on a clean commit just above 1h limit.
More details:
- CircleCI changed job build time to a maximum of 1h each.
- CircleCI now permits parallelization of 30 jobs
- 6000 build minutes a month.
- Still waiting for osresearch/heads CircleCI project to be unlocked (currently not recognized as open source project?!)
Readd https://github.com/osresearch/heads/pull/984 without cache
Add kgpe-d16 musl-cross target prior of having kgpe-d16 depend on musl-cross target (To try to have musl-cross step successfull under 1h CircleCI new limit)
CircleCI: add a subcommand that can follow a target (to build musl-cross-make now and coreboot version specific musl-cross later)
Output of hashes is now optional
29/11/2021 CircleCI public information available states parallelization of up to 30 jobs at a time. Let's play
- We first build heads musl-cross-make and persist (passing musl-cross-make into next job)
- We then build per coreboot version board with coreboot make statement only and persist (passing musl-cross-make + coreboot's musl-cross buildstack)
- We then build per coreboot version board (reusing past build musl-cross-make and coreboot's version musl-cross buildstack)
Remove 4.11 boards for the moment to test only build time and parallelization
- me_cleaner downloaded from 43612a630c/me_cleaner.py
- placed under xx30 blobs dir
- CircleCI uses it locally without downloading it everytime (me_cleaner hasn<t changed since 2018)
- xx30 legacy boards (x230, x230-flash, t430, t430-flash) now rely also on coreboot 4.13
- DOWNSIDE: x230 and t430 legacy boards now rely on WHIPTAIL (NOT FBWhiptail) to have enough space to fit under 7mb)
- xx20 boards moved to 4.13 (no need of xx20-flash boards here since single SPI boards with 7.5mb useable since blobs scripts are required)
- DOWNSIDE: all xx20 boards now have dropbear deactivated, while still having ethernet driver in.
- qemu-coreboot and qemu-coreboot-fbwhiptail switched to coreboot 4.13 WITHOUT TPM SUPPORT (with cryptsetup 2.x support)
- DOWNSIDE:
- coreboot-qemu board CBFS_SIZE=0x700000 -> 0x750000
- coreboot-qemu-fbwhiptail CBFS_SIZE=0x750000 -> 0x780000
- CircleCi build recipe removes 4.8.1 boards altogether
- KGPE-D16 workstation is used as new base build to save workspace layer (we removed one workspace layer)
- Removing one workspace layer will save approx 2 hours of build time on fresh builds
- Removing one coreboot version will save us approx 2 hours of build time on fresh builds
- KGPE-D16 will stay to coreboot 4.11 until forward notice.
- All other board configs SHOULD be built on latest coreboot versions
* Bump CircleCI config version to 2.1.
* Use commands and parameters to get rid of repeated commands. New boards can be added with just 5 lines at the bottom of the config.
* Made use of some parallelisation. Currently a single board from each Coreboot version is built. Afterwards all remaining boards are built in parallel.
The idea here is a cache to restore from (musl-cross from coreboot version bound crosscomipler, from which coreboot is built)
1- Reuse existing cache for all modules and patches created digest's hash past build matching cache.
(If a single module or patch changes, we have cache miss.)
2- Reuse existing coreboot and musl-cross-make created digest's hash past build's matching cache
(If a patch was added to current coreboot, or new coreboot version was added in coreboot module definition, we have a cache miss.)
3- Reuse existing musl-cross-make created digest's hash past build matching cache
(If musl-cross-make module didn't change, we don't rebuild it.)
Per https://github.com/osresearch/heads/pull/947#issuecomment-753507412 proposition
* xx30-*-maximized: update flashrom options removing --ifd bios option, keeping whole flash of rom internally. WARNING: ifd needs to be initially unlocked through ifdtool -u on 8mb bottom SPI backup. YOU CANNOT COME FROM 1VYRAIN. IF COMING FROM SKULLS, YOU MUST HAVE RAN OPTIONAL -u OPTION FROM SKULLS. PLEASE UPGRADE ONLY AFTER HAVING A PHYSICAL BACKUP OF BOTH SPI FLASH CHIPS. MORE INFORMATION UNDER https://github.com/osresearch/heads/pull/703. This will guarantee that future flash of produced rom will reflash the ROM totally, where heads make sure of adding users customizations (public key, /etc/config.user) when internally flashed. Unfortunately, if you flash externally, you will have to reinject your public key and readd /etc/config customizations.
* Adding generated bincfg coreboot 4.8.1 patch (merged under coreboot 4.13 and backported here to 4.8.1), resulting in gbe.bin under blobs/xx30/gbe.bin and instructions to replicate in README prior of automation (under repo). Note that MAC under gbe.bin is fixed to DE:AD:C0:FF:EE unless extract.sh script is ran on external backup to keep current user's MAC (Thanks to @Thrilleratplay's contribution!)
* xx30 blobs: add two blobs management scripts for xx30: extract from local backup/download+neuter ME
extract.sh: extract from external backup: gbe.bin, neuter under me.bin and maximize BIOS+reduce ME regions under unlocked ifd.bin.
download_clean_me.sh: download and verify Lenovo latest ME version from website, and drop me.bin in place.
Note: me.bin is 98kb, containing only BUP and ROMP partitions which cannot be modified nor deleted else computer won't boot. As a result, BIOS region is maximized in ifd.bin to 11.5mb and coreboot config takes advantage of that freed space.
* CircleCI: xx30-*-maximized additional step to call download_clean_me.sh prior of building boards so that me.bin is dopped in place. This should be done by users prior of building xx30-*-maximized boards locally, which is imitated in CircleCI builds (look at .circleci/config.yaml for innoextract host added dependency and board buildings. Results on github for each commit).
