Introduce `sha256_unsigned` which is a checksum of the image _before_ a
signature is attached. This is helpful to compare image reproducibility.
Since the `.sha256sum` file is located in the $(KDIR) folder, switch
$(BIN_DIR) with $(KDIR) to simplify the code. The value of $(BIN_DIR)
itself is not stored inside the resulting JSON file, so it can be
replaced.
Signed-off-by: Paul Spooren <mail@aparcar.org>
(cherry picked from commit 8822a8d850)
Drop the -processors argument from the mksquashfs4 call, so it will use
all available processors. This dramatically reduces the time to create
squashfs filesystems.
The times below are observed when building an image for my main router,
the WatchGuard Firebox M300 (qoriq target):
Before:
real 4m45,973s
After:
real 0m23,497s
With this commit `mksquashfs` may use more cores than defined via `-j`.
This is the same behaviour as for archive creation of ImageBuilder, SDK
or toolchain. There is no trivial way to limit `mksquashfs` CPU core
usage to the amount of "free" make jobs since two running `mksquashfs`
instances would each run with the total allowed number (-j) of threads.
Signed-off-by: Stijn Tintel <stijn@linux-ipv6.be>
[extended reasoning in commit message]
Signed-off-by: Paul Spooren <mail@aparcar.org>
(cherry picked from commit df2ae8826c)
DEVICE_DTS_DELIMITER needs to be in the DEFAULT_DEVICE_VARS
list to work as expected. This was missing from the original
version and got overlooked.
Fixes: fd67908647 ("scripts: mkits.sh: Allow legacy @ mode for dts creation")
Signed-off-by: Christian Lamparter <chunkeey@gmail.com>
Multiple profiles create artifacts, these should be stored in the JSON
file as well, allowing downstream tooling to show those files, too.
Artifacts don't have specific filesystems so only the fields `name`,
`type` and `sha256` are available.
Rename env variable names from IMAGE_ to FILE_ prefixes to reflect that
images, kernels and artifacts are added with the same command.
Signed-off-by: Paul Spooren <mail@aparcar.org>
The option '-xattr' for mksquashfs4 should be '-xattrs' which lead to
build failure with SELinux enabled. Add the missing 's'.
Fixes: 4baf47b9a8 ("images: squashfs: xattrs should not depend on buld host")
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Enable xattr for the generated squashfs only if needed for SELinux.
This eliminates warnings during boot on target when building
(non-SELinux) OpenWrt on SELinux-enabled hosts like Fedora.
Reported-by: fda77 <fda77@users.noreply.github.com>
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Don't attempt to copy initramfs images for devices which do not output
an initramfs image.
This was breaking builds for mpc85xx-p1010 since mid-march.
Signed-off-by: David Bauer <mail@david-bauer.net>
The variable was missing in the definition of DEFAULT_DEVICE_VARS which
caused it to contain wrong values, messing up the resulting JSON files.
This patch adds the variable DEVICE_PACKAGES to DEFAULT_DEVICE_VARS.
Suggested-by: Baptiste Jonglez <git@bitsofnetworks.org>
Signed-off-by: Paul Spooren <mail@aparcar.org>
Before this commit, it was assumed that mkhash is in the PATH. While
this was fine for the normal build workflow, this led to some issues if
make TOPDIR="$(pwd)" -C "$pkgdir" compile
was called manually. In most of the cases, I just saw warnings like this:
make: Entering directory '/home/.../package/gluon-status-page'
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
[...]
While these were only warnings and the package still compiled sucessfully,
I also observed that some package even fail to build because of this.
After applying this commit, the variable $(MKHASH) is introduced. This
variable points to $(STAGING_DIR_HOST)/bin/mkhash, which is always the
correct path.
Signed-off-by: Leonardo Mörlein <me@irrelefant.net>
In case CONFIG_TARGET_MULTI_PROFILE is set, IMG_PREFIX cannot be
expanded. Use DEVICE_IMG_PREFIX instead and make sure it's defined.
Fixes: 8f89b1ab0f ("image: add 'append-image' build command")
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Add possibility to use images and initramfs in artifacts.
Signed-off-by: Oskari Lemmela <oskari@lemmela.net>
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Add new target feature 'dt-overlay' which makes DTC keep the symbol
names in the generated dtb.
Make sure additional DT overlay sources specified by the new device
variable DEVICE_DTS_OVERLAY get compiled together with the main DTS
(currently overlays got to be in the same folder). Let Build/fit pass
the generated DT overlay blobs to mkits.sh.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
U-boot will reject all nodes with @ since commit:
79af75f777
This will cause the OpenWrt images to fail booting,
to rectify use the config-1 as default.
Signed-off-by: Robert Marko <robert.marko@sartura.hr>
We so far had two variables IMG_PREFIX and IMAGE_PREFIX with
different content. Since these names are obviously quite
confusing, this patch renames the latter to DEVICE_IMG_PREFIX,
as it's a device-dependent variable, while IMG_PREFIX is only
(sub)target-dependent.
For consistency, also rename IMAGE_NAME to DEVICE_IMG_NAME, as
that's a device-dependent variable as well.
Cc: Paul Spooren <mail@aparcar.org>
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
Previously, build would fail for targets containing devices with not
initramfs image (such as mpc85xx-p1010). Only generate the JSON image
info for the initramfs image when we have one to avoid breaking the
builds.
Fixes commit d3140d0529 ("build/json: generate json file for initramfs")
Signed-off-by: David Bauer <mail@david-bauer.net>
The initramfs images are missing from the profiles.json files.
Signed-off-by: Moritz Warning <moritzwarning@web.de>
[fix code by exporting device variables]
Signed-off-by: Paul Spooren <mail@aparcar.org>
Use approach suggested by Adrian Schmutzler instead of introducing
another device variable.
Also revert the unnecessary white-space changes accidentally introduced
by the previous commit.
Fixed: c067b1e79b ("mediatek: move out-of-tree DTS files to dedicated dts folder")
Suggested-by: Adrian Schmutzler <mail@adrianschmutzler.de>
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Use dedicated dts folder like on ramips to store device tree source
files for boards not already supported in vanilla Linux.
Doing so instead of having them in files-* has several advantages:
* we don't need to duplicate them for several kernel versions
* changes to a device tree don't trigger a complete kernel rebuild
* the files are more obvious to find
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
The license folder is a core part of OpenWrt and all GPL-2.0 licensed.
Use SPDX license tags to allow machines to check licenses.
Signed-off-by: Paul Spooren <mail@aparcar.org>
[rebase, keep some Copyright lines, sharpen commit message]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
The majority of our targets provide a default value for the variable
SUPPORTED_DEVICES, which is used in images to check against the
compatible on a running device:
SUPPORTED_DEVICES := $(subst _,$(comma),$(1))
At the moment, this is implemented in the Device/Default block of
the individual targets or even subtargets. However, since we
standardized device names and compatible in the recent past, almost
all targets are following the same scheme now:
device/image name: vendor_model
compatible: vendor,model
The equal redundant definitions are a symptom of this process.
Consequently, this patch moves the definition to image.mk making it
a global default. For the few targets not using the scheme above,
SUPPORTED_DEVICES will be defined to a different value in
Device/Default anyway, overwriting the default. In other words:
This change is supposed to be cosmetic.
This can be used as a global measure to get the current compatible
with: $(firstword $(SUPPORTED_DEVICES))
(Though this is not precisely an achievement of this commit.)
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
Some images are created using different filesystems, most popular
squashfs and ext4. To allow downstream projects to distinguesh between
those, add the `filesystem` information to created json files.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Allow a device recipe to specify a custom UIMAGE_MAGIC value, as used by
OpenWrt's -M flag for mkimage. This allows to automatically customize
the magic bytes in all calls to Build/uImage for this device, similar to
the behaviour of UIMAGE_NAME. Since the -M argument is inserted before
the user arguments, it can be overriden.
The following example would use 0x87654321 for the KERNEL image, but
0x12345678 for the KERNEL_INITRAMFS image:
define Device/MyDevice
UIMAGE_MAGIC := 0x87654321
KERNEL := ... | uImage lzma
KERNEL_INITRAMFS := ... | uImage lzma -M 0x12345678
...
endef
Fixes: df8e6be59a ("rtl838x: add new architecture")
[UIMAGE_MAGIC was not declared as a device variable]
Signed-off-by: Sander Vanheule <sander@svanheule.net>
[rebase, improve formatting of "Fixes"]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
The source date epoch is the only reproducible date close to the actual
build date. It can be used for tooling like the firmware wizard to show
the image age.
Signed-off-by: Paul Spooren <mail@aparcar.org>
This removes switches dependent on kernel version 4.19 as well as
several packages/modules selected only for that version.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
LegacyDevice is not used anymore in our tree, so let's drop it
together with the relevant definitions and recipes.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
This drops unused legacy recipes Image/Build/SysupgradeNAND and
Image/Build/UbinizeImage.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
Instead of hardcoding 'targeted' policy, evaluate /etc/selinux/config
in rootfs to choose according to which policy files in the rootfs got
to be labeled.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
By installing policycoreutils to host/bin it is also available within
the ImageBuilder and SDK, allowing to correctly label both filesystems
and packages.
Signed-off-by: Paul Spooren <mail@aparcar.org>
'setfiles' and others should be installed to $(STAGING_DIR_HOSTPKG)/bin
rather than $(...)/sbin which isn't in PATH.
Also using -Wl,-rpath to set library search location instead of setting
LD_LIBRARY_PATH when calling setfiles in image.mk.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Some bootloaders are really keen on just one special
fdt in a multi-image fit image. This is a problem, because
currently this is fixed to "fdt@1".
This patch introduces a new device variable:
DEVICE_FDT_NUM that allows to specify the right
fdt number.
If the value is absent "1" will be chosen.
Signed-off-by: Christian Lamparter <chunkeey@gmail.com>
Using fakeroot without passing the paths to libfakeroot.sh and faked
causes havoc. Use the $(FAKEROOT) Make variable which includes them.
Fixes: 353ce2e521 ("build: ipkg-build use fakeroot with PKG_FILE_MODES")
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
This allows the build process to prepare a squashfs filesystem for use
with SELinux.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[rebase, add commit message]
Signed-off-by: W. Michael Petullo <mike@flyn.org>
We regularly encounter the situation that devices are subject to
changes that will make them incompatible to previous versions.
Removing SUPPORTED_DEVICES will not really be helpful in most of these
cases, as this only helps after a rename.
To solve this situation, this patchset introduces a compatibility
version for devices. It will be implemented via a per-device
Make variable DEVICE_COMPAT_VERSION, which will be set to 1.0
globally by default and then can be overwritten as needed.
Furthermore, a variable DEVICE_COMPAT_MESSAGE is added, where
a message to be displayed during sysupgrade may be specified
optionally.
This patch only implements the build variables and adds them
to the sysupgrade metadata, the evaluation will be addressed
in a subsequent patch.
To set it, one would just need to add the following to a device node:
define Device/somedevice
...
DEVICE_COMPAT_VERSION := 1.1
DEVICE_COMPAT_MESSAGE := Config cannot be migrated from swconfig to DSA
endef
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
By specifying "BROKEN := 1" or "BROKEN := y" for a device, it will be
hidden (and deselected) by default. By that, it provides a stronger
option to "disable" a device beyond just using DEFAULT := n.
To make these devices visible, just enable the BROKEN option in
developer settings as already implemented for targets and packages.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
With this commit the `profiles.json` contain both the target specific
`default_packages` as well as the device specific `device_packages` as a
array of strings.
This information is required for downstream projects like the various
web-based interactive firmware generators.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Signed-off-by: Paul Spooren <mail@aparcar.org>
JSON info files contain machine readable information of built profiles
and resulting images. These files were added in commit 881ed09ee6
("build: create JSON files containing image info").
They are useful for firmware wizards and script checking for
reproducibility.
Currently all JSON files are stored next to the built images, resulting
in up to 168 individual files for the ath79/generic target.
This patch refactors the JSON creation to store individual per image
(not per profile) files in $(BUILD_DIR)/json_info_files and create an
single overview file called `profiles.json` in the target directory.
Storing per image files and not per profile solves the problem of
parallel file writes. If a profiles sysupgrade and factory image are
finished at the same time both processes would write to the same JSON
file, resulting in randomly broken outputs.
Some target like x86/64 do not use the image code yet, resulting in
missing JSON files. If no JSON info files were created, no
`profiles.json` files is created as it would be empty anyway.
As before, this creation is enabled by default only if `BUILDBOT` is set.
Tested via buildroot & ImageBuilder on ath79/generic, imx6 and x86/64.
Signed-off-by: Paul Spooren <mail@aparcar.org>
[json_info_files dir handling in Make, if case refactoring]
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Add EFI platform bootable images for x86 platforms. These images can
also boot from legacy BIOS platform.
EFI System Partition need to be fat12/fat16/fat32 (not need to load
filesystem drivers), so the first partition of EFI images are not ext4
filesystem any more.
GPT partition table has an alternate partition table, we did not
generate it. This may cause problems when use these images as qemu disk
(kernel can not find rootfs), we pad enough sectors will be ok.
Signed-off-by: 李国 <uxgood.org@gmail.com>
[part_magic_* refactoring, removed genisoimage checks]
Signed-off-by: Petr Štetiar <ynezz@true.cz>
This commit introduces few related changes which need to be done in
single commit to keep images buildable between git revisions. In result
it retains all previous image creation possibilities with slight name
change of generated images. Brief summary of the commit:
* Split up image generation recipe to smaller chunks to make it more
generic and reusable.
* Make iso images x86 specific and drop their definition as root
filesystem.
* Convert image creation process to generic code specified in image.mk.
* Make geode subtarget inherit features from the main target instead of
redefining them.
* For subtargets create device definitions with basic packages set.
Signed-off-by: Tomasz Maciej Nowak <tomek_n@o2.pl>
[rebased]
Signed-off-by: Paul Spooren <mail@aparcar.org>
IMAGE_SIZE is widely used in many targets. Declare it in the default template to
clean up redundant code. This also prevents deriving IMAGE_SIZE unintentionally
from the previously defined device.
While at it, remove duplicate KERNEL_SIZE declaration.
Signed-off-by: Sungbo Eo <mans0n@gorani.run>
Some targets like kirkwood or omap don't use a subtarget which results
in a malformed JSON info file.
Instead of having a valid value like `"target": "ath79/tiny"` for these
targets the value is `"target": "kirkwood/"`.
This patch uses the same if condition to use `generic` if the subtarget
is empty.
Tested for the kirkwood target.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Most of the kernel version switches below 4.14 were removed in commit
97940f8766 ("kernel: remove obsolete kernel version switches"),
but some of them still remained. Remove them now.
Signed-off-by: Sungbo Eo <mans0n@gorani.run>
This creates the device variable SOC and adds it to DEFAULT_DEVICE_VARS.
It is supposed to replace target-specific SOC variables like ATH_SOC or
MTK_SOC and thus unify variable names across targets.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
Replace 2 instances of non posix use of 'head' with posix compliant
'cut'.
'head -c n' cuts 'n' bytes from the passed string and happens to work on
Linux & Mac OS X even though '-c' is not posix.
'head --bytes n' does the same thing and happens to work on linux but
not on Mac OS X and is also not posix.
'cut -b1-8' cuts the first 8 bytes from the passed string and is posix
compliant, hence works on Linux & Mac OS X.
Our usage of 'head --bytes' was particularly unfortunate since it was
used to calculated the RootFS UUID passed to grub - the net result being
a non-functioning system waiting for the root file system to appear.
Thanks to karlp, ynezz & others for pointers on solving this.
Signed-off-by: Kevin Darbyshire-Bryant <ldir@darbyshire-bryant.me.uk>
x86, mvebu and tegra targets are currently using more or less same
SIGNATURE variable which provides unique partition ID/signature, so it
makes sense to refactor it out into common variable which could be
reused by all targets. While at it, make the content of the variable
reproducible.
Ref: http://lists.infradead.org/pipermail/openwrt-devel/2019-March/016148.html
Signed-off-by: Paul Spooren <mail@aparcar.org>
Suggested-by: Jo-Philipp Wich <jo@mein.io>
[renamed to IMG_PARTSIGNATURE, reworked with epoch+vermagic hash]
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Commit 881ed09ee6 ("build: create JSON files containing image info")
has removed the crucial empty new line from the image copy step
resulting in the following errors during make function expansion:
GZ_SUFFIX :=
bash: GZ_SUFFIX: command not found
Makefile:86: recipe for target 'openwrt-ath79-generic-tplink_archer-c7-v5-squashfs-sysupgrade.bin' failed
Fixes: 881ed09ee6 ("build: create JSON files containing image info")
Signed-off-by: Petr Štetiar <ynezz@true.cz>
UUID of ext4 volumes generated by make_ext4fs are determined by volume
label and it will all be 57f8f4bc-abf4-655f-bf67-946fc0f9f25b when label
is empty
Labeling them does not make them unique but tools like block command
from fstools have a better chance differentiating them
Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
The JSON info files contain details about the created firmware images
per device and are stored next to the created images.
The JSON files are stored as "$(IMAGE_PREFIX).json" and contain some
device/image meta data as well as a list of created firmware images.
An example of openwrt-ramips-rt305x-aztech_hw550-3g.json
{
"id": "aztech_hw550-3g",
"image_prefix": "openwrt-ramips-rt305x-aztech_hw550-3g",
"images": [
{
"name": "openwrt-ramips-rt305x-aztech_hw550-3g-squashfs-sysupgrade.bin",
"sha256": "db2b34b0ec4a83d9bf612cf66fab0dc3722b191cb9bedf111e5627a4298baf20",
"type": "sysupgrade"
}
],
"metadata_version": 1,
"supported_devices": [
"aztech,hw550-3g",
"hw550-3g"
],
"target": "ramips/rt305x",
"titles": [
{
"model": "HW550-3G",
"vendor": "Aztech"
},
{
"model": "ALL0239-3G",
"vendor": "Allnet"
}
],
"version_commit": "r10920+123-0cc87b3bac",
"version_number": "SNAPSHOT"
}
Signed-off-by: Paul Spooren <mail@aparcar.org>
