CircleCI: Add coreboot+musl-cross cache

The idea here is a cache to restore from (musl-cross from coreboot version bound crosscomipler, from which coreboot is built)

1- Reuse existing cache for all modules and patches created digest's hash past build matching cache.
(If a single module or patch changes, we have cache miss.)
2- Reuse existing coreboot and musl-cross-make created digest's hash past build's matching cache
(If a patch was added to current coreboot, or new coreboot version was added in coreboot module definition, we have a cache miss.)
3- Reuse existing musl-cross-make created digest's hash past build matching cache
(If musl-cross-make module didn't change, we don't rebuild it.)

Per https://github.com/osresearch/heads/pull/947#issuecomment-753507412 proposition
This commit is contained in:
Thierry Laurion 2021-01-02 15:09:52 -05:00
parent 69075fa738
commit 62a90ed3be
No known key found for this signature in database
GPG Key ID: E7B4A71658E36A93

View File

@ -17,23 +17,32 @@ jobs:
git reset --hard "$CIRCLE_SHA1" \ git reset --hard "$CIRCLE_SHA1" \
- run: - run:
name: Creating all modules and patches digest name: Creating all modules and patches digest (All modules cache digest)
command: | command: |
find ./patches/ ./modules/ -type f | sort -h |xargs sha256sum > /tmp/all_modules_and_patches.sha256sums \ find ./patches/ ./modules/ -type f | sort -h |xargs sha256sum > /tmp/all_modules_and_patches.sha256sums \
- run: - run:
name: Creating musl-cross-make and musl-cross-make patches digest name: Creating coreboot (and associated patches) and musl-cross-make modules digest (musl-cross-make and coreboot cache digest)
command: | command: |
find modules/musl-cross* -type f | sort -h | xargs sha256sum > /tmp/musl-cross_module_and_patches.sha256sums \ find ./modules/coreboot ./modules/musl-cross* ./patches/coreboot* -type f | sort -h | xargs sha256sum > /tmp/coreboot_musl-cross.sha256sums \
- run:
name: Creating musl-cross-make and musl-cross-make patches digest (musl-cross-make cache digest)
command: |
find modules/musl-cross* -type f | sort -h | xargs sha256sum > /tmp/musl-cross.sha256sums \
- restore_cache: - restore_cache:
keys: keys:
#Restore existing cache for modules checksums validated to be exactly the same as in github current commit #Restore existing cache for matching modules digest, validated to be exactly the same as in github current commit.
#This cache was made on top of below caches, if previously existing. If no module definition changed, we reuse this one. Otherwise...
- heads-modules-and-patches-{{ checksum "/tmp/all_modules_and_patches.sha256sums" }}{{ .Environment.CACHE_VERSION }} - heads-modules-and-patches-{{ checksum "/tmp/all_modules_and_patches.sha256sums" }}{{ .Environment.CACHE_VERSION }}
#If precedent fails. Restore cache for musl-cross module checksum validated to be exactly the same as in github current commit #If precedent cache not found, restore cache for coreboot module (and patches) and musl-cross-make digests (bi-yearly modified)
- heads-cross-musl-{{ checksum "/tmp/musl-cross_module_and_patches.sha256sums" }}{{ .Environment.CACHE_VERSION }} #Otehrwise....
- heads-coreboot-musl-cross-{{ checksum "/tmp/coreboot_musl-cross.sha256sums" }}{{ .Environment.CACHE_VERSION }}
#If precedent cache not found. Restore cache for musl-cross-make module digest (rarely modified).
#Otherwise, we build cleanly.
- heads-musl-cross-{{ checksum "/tmp/musl-cross.sha256sums" }}{{ .Environment.CACHE_VERSION }}
# linuxboot steps need something to pass in the kernel header path # linuxboot steps need something to pass in the kernel header path
# skipping for now # skipping for now
# - run: # - run:
@ -578,11 +587,17 @@ jobs:
- save_cache: - save_cache:
#Generate cache for the same musl-cross module definition if hash is not previously existing #Generate cache for the same musl-cross module definition if hash is not previously existing
key: heads-cross-musl-{{ checksum "/tmp/musl-cross_module_and_patches.sha256sums" }}{{ .Environment.CACHE_VERSION }} key: heads-musl-cross-{{ checksum "/tmp/musl-cross.sha256sums" }}{{ .Environment.CACHE_VERSION }}
paths: paths:
- crossgcc - crossgcc
- build/musl-cross-* - build/musl-cross-*
- save_cache:
#Generate cache for the same coreboot mnd musl-cross-make modules definition if hash is not previously existing
key: heads-coreboot-musl-cross-{{ checksum "/tmp/coreboot_musl-cross.sha256sums" }}{{ .Environment.CACHE_VERSION }}
paths:
- build/coreboot-*
- crossgcc
- build/musl-cross-*
- save_cache: - save_cache:
#Generate cache for the exact same modules definitions if hash is not previously existing #Generate cache for the exact same modules definitions if hash is not previously existing
key: heads-modules-and-patches-{{ checksum "/tmp/all_modules_and_patches.sha256sums" }}{{ .Environment.CACHE_VERSION }} key: heads-modules-and-patches-{{ checksum "/tmp/all_modules_and_patches.sha256sums" }}{{ .Environment.CACHE_VERSION }}
@ -590,6 +605,7 @@ jobs:
- packages - packages
- crossgcc - crossgcc
- build - build
- install
workflows: workflows:
version: 2 version: 2