Readd https://github.com/osresearch/heads/pull/984 without cache
Add kgpe-d16 musl-cross target prior of having kgpe-d16 depend on musl-cross target (To try to have musl-cross step successfull under 1h CircleCI new limit)
CircleCI: add a subcommand that can follow a target (to build musl-cross-make now and coreboot version specific musl-cross later)
Output of hashes is now optional
29/11/2021 CircleCI public information available states parallelization of up to 30 jobs at a time. Let's play
- We first build heads musl-cross-make and persist (passing musl-cross-make into next job)
- We then build per coreboot version board with coreboot make statement only and persist (passing musl-cross-make + coreboot's musl-cross buildstack)
- We then build per coreboot version board (reusing past build musl-cross-make and coreboot's version musl-cross buildstack)
Remove 4.11 boards for the moment to test only build time and parallelization
- me_cleaner downloaded from 43612a630c/me_cleaner.py
- placed under xx30 blobs dir
- CircleCI uses it locally without downloading it everytime (me_cleaner hasn<t changed since 2018)
- xx30 legacy boards (x230, x230-flash, t430, t430-flash) now rely also on coreboot 4.13
- DOWNSIDE: x230 and t430 legacy boards now rely on WHIPTAIL (NOT FBWhiptail) to have enough space to fit under 7mb)
- xx20 boards moved to 4.13 (no need of xx20-flash boards here since single SPI boards with 7.5mb useable since blobs scripts are required)
- DOWNSIDE: all xx20 boards now have dropbear deactivated, while still having ethernet driver in.
- qemu-coreboot and qemu-coreboot-fbwhiptail switched to coreboot 4.13 WITHOUT TPM SUPPORT (with cryptsetup 2.x support)
- DOWNSIDE:
- coreboot-qemu board CBFS_SIZE=0x700000 -> 0x750000
- coreboot-qemu-fbwhiptail CBFS_SIZE=0x750000 -> 0x780000
- CircleCi build recipe removes 4.8.1 boards altogether
- KGPE-D16 workstation is used as new base build to save workspace layer (we removed one workspace layer)
- Removing one workspace layer will save approx 2 hours of build time on fresh builds
- Removing one coreboot version will save us approx 2 hours of build time on fresh builds
- KGPE-D16 will stay to coreboot 4.11 until forward notice.
- All other board configs SHOULD be built on latest coreboot versions
* Bump CircleCI config version to 2.1.
* Use commands and parameters to get rid of repeated commands. New boards can be added with just 5 lines at the bottom of the config.
* Made use of some parallelisation. Currently a single board from each Coreboot version is built. Afterwards all remaining boards are built in parallel.
The idea here is a cache to restore from (musl-cross from coreboot version bound crosscomipler, from which coreboot is built)
1- Reuse existing cache for all modules and patches created digest's hash past build matching cache.
(If a single module or patch changes, we have cache miss.)
2- Reuse existing coreboot and musl-cross-make created digest's hash past build's matching cache
(If a patch was added to current coreboot, or new coreboot version was added in coreboot module definition, we have a cache miss.)
3- Reuse existing musl-cross-make created digest's hash past build matching cache
(If musl-cross-make module didn't change, we don't rebuild it.)
Per https://github.com/osresearch/heads/pull/947#issuecomment-753507412 proposition
* xx30-*-maximized: update flashrom options removing --ifd bios option, keeping whole flash of rom internally. WARNING: ifd needs to be initially unlocked through ifdtool -u on 8mb bottom SPI backup. YOU CANNOT COME FROM 1VYRAIN. IF COMING FROM SKULLS, YOU MUST HAVE RAN OPTIONAL -u OPTION FROM SKULLS. PLEASE UPGRADE ONLY AFTER HAVING A PHYSICAL BACKUP OF BOTH SPI FLASH CHIPS. MORE INFORMATION UNDER https://github.com/osresearch/heads/pull/703. This will guarantee that future flash of produced rom will reflash the ROM totally, where heads make sure of adding users customizations (public key, /etc/config.user) when internally flashed. Unfortunately, if you flash externally, you will have to reinject your public key and readd /etc/config customizations.
* Adding generated bincfg coreboot 4.8.1 patch (merged under coreboot 4.13 and backported here to 4.8.1), resulting in gbe.bin under blobs/xx30/gbe.bin and instructions to replicate in README prior of automation (under repo). Note that MAC under gbe.bin is fixed to DE:AD:C0:FF:EE unless extract.sh script is ran on external backup to keep current user's MAC (Thanks to @Thrilleratplay's contribution!)
* xx30 blobs: add two blobs management scripts for xx30: extract from local backup/download+neuter ME
extract.sh: extract from external backup: gbe.bin, neuter under me.bin and maximize BIOS+reduce ME regions under unlocked ifd.bin.
download_clean_me.sh: download and verify Lenovo latest ME version from website, and drop me.bin in place.
Note: me.bin is 98kb, containing only BUP and ROMP partitions which cannot be modified nor deleted else computer won't boot. As a result, BIOS region is maximized in ifd.bin to 11.5mb and coreboot config takes advantage of that freed space.
* CircleCI: xx30-*-maximized additional step to call download_clean_me.sh prior of building boards so that me.bin is dopped in place. This should be done by users prior of building xx30-*-maximized boards locally, which is imitated in CircleCI builds (look at .circleci/config.yaml for innoextract host added dependency and board buildings. Results on github for each commit).
Changing CONFIG_USB_BOOT_DEV to sdc1, adding back CONFIG_BOOT_STATIC_IP to 192.168.2.3, adding dual console to OpenBMC and tty0 in attempt to have QubesOS graphic installer which complains with no networking when attempting to start VNC
Adding dual console to OpenBmc and tty0
putting kgpe-d16-coreboot.conf in defconfig format
NO_HZ wasn't included in kernel config. Adding it.
Wasn't able to have both console firing up QubesOS gui installer, complaining about hvc1 console errors. Splitting up Workstation and server config. This one works for Worstation
Removing serial configuration and static IP stuff since we have a workstation here.
Seperate Workstation and Server board configurations until dual console truely works through QubesOS gui installation. kgpe-d16 board config removed until then.
Placing files in good directories
Corrrect flashrom options for kgpe-d16 server and workstation boards
kgpe-d16 linux: NO_HZ_IDLE instead of NO_HZ
kgpe-d16: seperate board for workstation to be AST and gui-init based, while kgpe-d16-> kgpe-d16_server
kgpe-d16_server: boots, shows ASpeed text on VGA, controllable through BMC via SSH.
kgpe-d16_workstation on ASpeed console. WIP. (Includes CIs configs to build server and workstation)
kgpe-d16_workstation in defconfig format
kgpe-d16 boards: pass from GPG to GPG2 board definitions
kgpe-d16_workstation : Adding Cairo and FbWhpitail in board config for gui-init to work in FB mode
kgpe-d16: removing plymouth.ignore-serial-consoles to fix server terminal output
kgpe-d16: bring par with staging branch https://gitlab.com/tlaurion/heads/commits/kgpe-d16_staging
kgpe-d16 : expressively export CONFIG_TPM=n
kgpe-d16_wokstation gui-init variables were missing
kgpe-d16 boards: add CONFIG_LINUX_USB_COMPANION_CONTROLLER so that usb is recognized
linux-kgpe-d16*: add support for Pike
kgpe-d16_workstation-usb_keyboard board support addition
kgpe-d16_server-whiptail: Add board and dependencies to have gui-init in whiptail (console mode, not FbWhiptail based
GitlabCI: kgpe-d16 fixes and upstream merge of change
kgpe-d16* board: add statement to fixate coreboot version to 4.8.1 for the moment
kgpe-d16: add missing config/linux-kgpe-d16_server-whiptail.config file
KGPE-D16: community work migration to coreboot 4.11 to fix issue #740
KGPE-D16 boards: Adding VBOOT+measured boot, musl-cross patch and 4.11 patch brought up per https://github.com/osresearch/heads/pull/709
kgpe-d16* boards: add VBOOT Kconfig patch per @miczyg1 recommendation under https://github.com/osresearch/heads/pull/795#issuecomment-671214637
KGPE-D16* coreboot configs: Add S3NV as a Runtime data whitelist (so that it is not measured at term) per @miczyg1 recommendation under https://github.com/osresearch/heads/pull/795#issuecomment-671214637
kgpe-d16 coreboot 4.11: add https://review.coreboot.org/c/coreboot/+/36908 patch
kgpe-d16 boards: add Linux kernel version where missing.
CircleCI: Add debug output on fail for kgpe-d16 board builds to bring par with upstream after rebasing on master
coreboot module: typo correction (tabs vs spaces)
CircleCI: trying to address "g++: fatal error: Killed signal terminated program cc1plus." happening under coreboot 4.11 and coreboot 4.12 builds
CircleCI: remove past addition to test recommendation from CircleCI: "resource_class: large"
CircleCi: Ok.... lets output dmesg content prior of other logs.... I'm out of ideas. Next step, ask CircleCI for support
At this stage:
- job's "make --load" is supposed to guarantee that the number of thread doesn't exhaust pass of a load of 2 (medium, free class, CircleCI has 32 cores so possibility of a load of 32)
- "--max_old_space_size=4096" in CircleCI environement is supposed to limit memory consumption to 4096Mb of memory, the max of a medium class free tier CircleCI node
CircleCI: remove verbose build (no more V=1), in case of failed build, find all logs modified in last minute and output each of them on console.
coreboot module: implement load average respect inside of problematic CI build for coreboot 4.11+ being killed in the action (32 cores with 4Gb ram get gcc OOM)
coreboot module: replace nproc by number of Gb actually available as number of CPUs, since each thread is expected to have 1Gb of ram.
CircleCI & coreboot config: fix merge conflict rebasing on master
coreboot 4.11 kgpe-d16 vboot patches addendum, credits goes to @Tonux599
Fix merge conflicts and make sure all boards are inside of CircleCI builds. PoC build for #867
* modules/linux: add support for building with kernel 5.4.69
Add support to module, port patches from 4.19.139.
Needed for newer platforms not supported by 4.19 kernel.
Signed-off-by: Matt DeVillier <matt.devillier@puri.sm>
* CircleCI: add rysnc dependency for building kernel 5.x
Signed-off-by: Matt DeVillier <matt.devillier@puri.sm>
* Migrate all Librem boards to kernel 5.x, common config
Update linux-librem_common.config from 4.x to 5.x, and add
CONFIG items needed to support the librem_l1um (AST DRM drivers,
serial port output).
Tested on Librem 13v4, Librem Mini, and Librem Server L1UM.
Signed-off-by: Matt DeVillier <matt.devillier@puri.sm>
* add x230-nkstorecli board;
* add modules: nkstorecli, libnk, libhidapi-libusb
* version bump nkstorecli; related minor in libnk
* upd. libnk module version bump to 3.6; remove 3.5 patch
* modules/coreboot: add option to use coreboot 4.11
Port patches from coreboot 4.8.1 to 4.11:
* 0000-measure-boot -> 0001
* 0010-cross-compiler-support
All other patches for coreboot 4.8.1 have either already been
integrated, or are for platforms which do not need to be migrated
to coreboot 4.11 (they will move to 4.12 or newer).
Signed-off-by: Matt DeVillier <matt.devillier@puri.sm>
* patches/coreboot-4.11: Add Broadwell-DE platform patch
Add a patch for FSP Broadwell-DE to make use of Heads' measured boot.
Signed-off-by: Matt DeVillier <matt.devillier@puri.sm>
* patches/coreboot-4.11: Add patch to read serial # from CBFS
Will be used by multiple Librem boards.
Signed-off-by: Matt DeVillier <matt.devillier@puri.sm>
* patches/coreboot-4.11: add board support for Librem Server L1UM
Signed-off-by: Matt DeVillier <matt.devillier@puri.sm>
* Librem Server L1UM: add new board
Add board config, coreboot config, kernel config files.
Add conditional purism-blobs dependency to coreboot-4.11 module.
Signed-off-by: Matt DeVillier <matt.devillier@puri.sm>
* flash.sh: add special handling for librem_l1um board
Add support for persisting PCIe config via PCHSTRP9 in flash descriptor.
This is needed to support multiple variants of the L1UM server which
use the same firmware but differ in PCIe lane configuration via the
PCH straps configuration in the flash descriptor.
Signed-off-by: Matt DeVillier <matt.devillier@puri.sm>
* patches/coreboot-4.11: Add 'Use PRIxPTR to print uintptr_t' patch
Cherry-picked from upstream coreboot (post-4.11), fixes compilation issue.
Signed-off-by: Matt DeVillier <matt.devillier@puri.sm>
* CircleCI: add target to build board librem_l1um
Signed-off-by: Matt DeVillier <matt.devillier@puri.sm>
* musl-cross: remove old patch artifact of the musl-cross era
* CircleCI: do not produce hash digest for musl-cross-make patches (artifact was for musl-cross, not musl-cross-make)
There's only one Librem Mini board, it doesn't use a TPM,
no reason to unnecesarily lengthen the board name.
Signed-off-by: Matt DeVillier <matt.devillier@puri.sm>
* CircleCI: readd librem_mini while making sure that if a board build fails, all logfile modified in the last minute are outputted on the CircleCI console prior to really failing and exiting
* librem_mini-NoTPM: addition of board config, distinctive coreboot config (required per Heads build system) to construct a ROM without TPM requirement.
* librem_mini: deletion of board and coreboot relative config, keeping librem_mini-NoTPM and coreboot config only. Removed librem_mini board build under CircleCI, keeping only librem_mini-NoTPM
* patches/coreboot-4.12: Add patch for Cannonlake ME status
Add patch print ME status regardless of enablement state
Signed-off-by: Matt DeVillier <matt.devillier@puri.sm>
* modules: add purism-blobs module
Rather than require users to manually run a script to download the required
blobs to build Purism Librem boards, automate it so the correct version
is automatically downloaded/extracted. Restrict to coreboot 4.12 for now
since 4.8.1 still needs FSP blobs, which are not in module.
Signed-off-by: Matt DeVillier <matt.devillier@puri.sm>
* configs/linux-librem13v2: unset CONFIG_RETPOLINE
Fixes compilation issue with newer kernels, ignored by older ones
which don't need it
Signed-off-by: Matt DeVillier <matt.devillier@puri.sm>
* Add new board: Librem Mini
Add Librem Mini board patch for coreboot 4.12, board config and
coreboot config. Continue reusing existing librem13v2 Linux config,
same as all other Librem boards currently. Use new purism-blobs module.
Signed-off-by: Matt DeVillier <matt.devillier@puri.sm>
* board/librem*: rename for consistency
Use 'librem_<board>' notation for consistency across all models.
Rename linux config file since used by multiple Librem models.
Signed-off-by: Matt DeVillier <matt.devillier@puri.sm>
* CircleCI: add librem_mini board to test
Signed-off-by: Matt DeVillier <matt.devillier@puri.sm>
* CircleCI: debian:10 docker based. Give possitility to override CACHE_VERSION through CircleCI when needed
* Makefile: fix#799 with implementation of @osresearch's recommended https://github.com/osresearch/heads/issues/799#issuecomment-673059028
* modules/coreboot : indentation fix and putting version hashes together to facilitate future maintainership.
CircleCI: two cache save/restore mechanisms. One bound to musl-cross+patches, one for modules + patches.
* Replacing the generic cache bound to CircleCI user to have two caches levels. One for musl-cross and its patchsets, one for all modules and their patchset being the same. So if modules changed, we use the cache for musl-cross as a fallback to economize one hour of precious build time out of two, while most of Heads changes are on the scripts and can be built on top of packages+patches cache
