Some images are created using different filesystems, most popular
squashfs and ext4. To allow downstream projects to distinguesh between
those, add the `filesystem` information to created json files.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Allow a device recipe to specify a custom UIMAGE_MAGIC value, as used by
OpenWrt's -M flag for mkimage. This allows to automatically customize
the magic bytes in all calls to Build/uImage for this device, similar to
the behaviour of UIMAGE_NAME. Since the -M argument is inserted before
the user arguments, it can be overriden.
The following example would use 0x87654321 for the KERNEL image, but
0x12345678 for the KERNEL_INITRAMFS image:
define Device/MyDevice
UIMAGE_MAGIC := 0x87654321
KERNEL := ... | uImage lzma
KERNEL_INITRAMFS := ... | uImage lzma -M 0x12345678
...
endef
Fixes: df8e6be59a ("rtl838x: add new architecture")
[UIMAGE_MAGIC was not declared as a device variable]
Signed-off-by: Sander Vanheule <sander@svanheule.net>
[rebase, improve formatting of "Fixes"]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
The source date epoch is the only reproducible date close to the actual
build date. It can be used for tooling like the firmware wizard to show
the image age.
Signed-off-by: Paul Spooren <mail@aparcar.org>
This removes switches dependent on kernel version 4.19 as well as
several packages/modules selected only for that version.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
LegacyDevice is not used anymore in our tree, so let's drop it
together with the relevant definitions and recipes.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
This drops unused legacy recipes Image/Build/SysupgradeNAND and
Image/Build/UbinizeImage.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
Instead of hardcoding 'targeted' policy, evaluate /etc/selinux/config
in rootfs to choose according to which policy files in the rootfs got
to be labeled.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
By installing policycoreutils to host/bin it is also available within
the ImageBuilder and SDK, allowing to correctly label both filesystems
and packages.
Signed-off-by: Paul Spooren <mail@aparcar.org>
'setfiles' and others should be installed to $(STAGING_DIR_HOSTPKG)/bin
rather than $(...)/sbin which isn't in PATH.
Also using -Wl,-rpath to set library search location instead of setting
LD_LIBRARY_PATH when calling setfiles in image.mk.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Some bootloaders are really keen on just one special
fdt in a multi-image fit image. This is a problem, because
currently this is fixed to "fdt@1".
This patch introduces a new device variable:
DEVICE_FDT_NUM that allows to specify the right
fdt number.
If the value is absent "1" will be chosen.
Signed-off-by: Christian Lamparter <chunkeey@gmail.com>
Using fakeroot without passing the paths to libfakeroot.sh and faked
causes havoc. Use the $(FAKEROOT) Make variable which includes them.
Fixes: 353ce2e521 ("build: ipkg-build use fakeroot with PKG_FILE_MODES")
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
This allows the build process to prepare a squashfs filesystem for use
with SELinux.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[rebase, add commit message]
Signed-off-by: W. Michael Petullo <mike@flyn.org>
We regularly encounter the situation that devices are subject to
changes that will make them incompatible to previous versions.
Removing SUPPORTED_DEVICES will not really be helpful in most of these
cases, as this only helps after a rename.
To solve this situation, this patchset introduces a compatibility
version for devices. It will be implemented via a per-device
Make variable DEVICE_COMPAT_VERSION, which will be set to 1.0
globally by default and then can be overwritten as needed.
Furthermore, a variable DEVICE_COMPAT_MESSAGE is added, where
a message to be displayed during sysupgrade may be specified
optionally.
This patch only implements the build variables and adds them
to the sysupgrade metadata, the evaluation will be addressed
in a subsequent patch.
To set it, one would just need to add the following to a device node:
define Device/somedevice
...
DEVICE_COMPAT_VERSION := 1.1
DEVICE_COMPAT_MESSAGE := Config cannot be migrated from swconfig to DSA
endef
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
By specifying "BROKEN := 1" or "BROKEN := y" for a device, it will be
hidden (and deselected) by default. By that, it provides a stronger
option to "disable" a device beyond just using DEFAULT := n.
To make these devices visible, just enable the BROKEN option in
developer settings as already implemented for targets and packages.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
With this commit the `profiles.json` contain both the target specific
`default_packages` as well as the device specific `device_packages` as a
array of strings.
This information is required for downstream projects like the various
web-based interactive firmware generators.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Signed-off-by: Paul Spooren <mail@aparcar.org>
JSON info files contain machine readable information of built profiles
and resulting images. These files were added in commit 881ed09ee6
("build: create JSON files containing image info").
They are useful for firmware wizards and script checking for
reproducibility.
Currently all JSON files are stored next to the built images, resulting
in up to 168 individual files for the ath79/generic target.
This patch refactors the JSON creation to store individual per image
(not per profile) files in $(BUILD_DIR)/json_info_files and create an
single overview file called `profiles.json` in the target directory.
Storing per image files and not per profile solves the problem of
parallel file writes. If a profiles sysupgrade and factory image are
finished at the same time both processes would write to the same JSON
file, resulting in randomly broken outputs.
Some target like x86/64 do not use the image code yet, resulting in
missing JSON files. If no JSON info files were created, no
`profiles.json` files is created as it would be empty anyway.
As before, this creation is enabled by default only if `BUILDBOT` is set.
Tested via buildroot & ImageBuilder on ath79/generic, imx6 and x86/64.
Signed-off-by: Paul Spooren <mail@aparcar.org>
[json_info_files dir handling in Make, if case refactoring]
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Add EFI platform bootable images for x86 platforms. These images can
also boot from legacy BIOS platform.
EFI System Partition need to be fat12/fat16/fat32 (not need to load
filesystem drivers), so the first partition of EFI images are not ext4
filesystem any more.
GPT partition table has an alternate partition table, we did not
generate it. This may cause problems when use these images as qemu disk
(kernel can not find rootfs), we pad enough sectors will be ok.
Signed-off-by: 李国 <uxgood.org@gmail.com>
[part_magic_* refactoring, removed genisoimage checks]
Signed-off-by: Petr Štetiar <ynezz@true.cz>
This commit introduces few related changes which need to be done in
single commit to keep images buildable between git revisions. In result
it retains all previous image creation possibilities with slight name
change of generated images. Brief summary of the commit:
* Split up image generation recipe to smaller chunks to make it more
generic and reusable.
* Make iso images x86 specific and drop their definition as root
filesystem.
* Convert image creation process to generic code specified in image.mk.
* Make geode subtarget inherit features from the main target instead of
redefining them.
* For subtargets create device definitions with basic packages set.
Signed-off-by: Tomasz Maciej Nowak <tomek_n@o2.pl>
[rebased]
Signed-off-by: Paul Spooren <mail@aparcar.org>
IMAGE_SIZE is widely used in many targets. Declare it in the default template to
clean up redundant code. This also prevents deriving IMAGE_SIZE unintentionally
from the previously defined device.
While at it, remove duplicate KERNEL_SIZE declaration.
Signed-off-by: Sungbo Eo <mans0n@gorani.run>
Some targets like kirkwood or omap don't use a subtarget which results
in a malformed JSON info file.
Instead of having a valid value like `"target": "ath79/tiny"` for these
targets the value is `"target": "kirkwood/"`.
This patch uses the same if condition to use `generic` if the subtarget
is empty.
Tested for the kirkwood target.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Most of the kernel version switches below 4.14 were removed in commit
97940f8766 ("kernel: remove obsolete kernel version switches"),
but some of them still remained. Remove them now.
Signed-off-by: Sungbo Eo <mans0n@gorani.run>
This creates the device variable SOC and adds it to DEFAULT_DEVICE_VARS.
It is supposed to replace target-specific SOC variables like ATH_SOC or
MTK_SOC and thus unify variable names across targets.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
Replace 2 instances of non posix use of 'head' with posix compliant
'cut'.
'head -c n' cuts 'n' bytes from the passed string and happens to work on
Linux & Mac OS X even though '-c' is not posix.
'head --bytes n' does the same thing and happens to work on linux but
not on Mac OS X and is also not posix.
'cut -b1-8' cuts the first 8 bytes from the passed string and is posix
compliant, hence works on Linux & Mac OS X.
Our usage of 'head --bytes' was particularly unfortunate since it was
used to calculated the RootFS UUID passed to grub - the net result being
a non-functioning system waiting for the root file system to appear.
Thanks to karlp, ynezz & others for pointers on solving this.
Signed-off-by: Kevin Darbyshire-Bryant <ldir@darbyshire-bryant.me.uk>
x86, mvebu and tegra targets are currently using more or less same
SIGNATURE variable which provides unique partition ID/signature, so it
makes sense to refactor it out into common variable which could be
reused by all targets. While at it, make the content of the variable
reproducible.
Ref: http://lists.infradead.org/pipermail/openwrt-devel/2019-March/016148.html
Signed-off-by: Paul Spooren <mail@aparcar.org>
Suggested-by: Jo-Philipp Wich <jo@mein.io>
[renamed to IMG_PARTSIGNATURE, reworked with epoch+vermagic hash]
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Commit 881ed09ee6 ("build: create JSON files containing image info")
has removed the crucial empty new line from the image copy step
resulting in the following errors during make function expansion:
GZ_SUFFIX :=
bash: GZ_SUFFIX: command not found
Makefile:86: recipe for target 'openwrt-ath79-generic-tplink_archer-c7-v5-squashfs-sysupgrade.bin' failed
Fixes: 881ed09ee6 ("build: create JSON files containing image info")
Signed-off-by: Petr Štetiar <ynezz@true.cz>
UUID of ext4 volumes generated by make_ext4fs are determined by volume
label and it will all be 57f8f4bc-abf4-655f-bf67-946fc0f9f25b when label
is empty
Labeling them does not make them unique but tools like block command
from fstools have a better chance differentiating them
Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
The JSON info files contain details about the created firmware images
per device and are stored next to the created images.
The JSON files are stored as "$(IMAGE_PREFIX).json" and contain some
device/image meta data as well as a list of created firmware images.
An example of openwrt-ramips-rt305x-aztech_hw550-3g.json
{
"id": "aztech_hw550-3g",
"image_prefix": "openwrt-ramips-rt305x-aztech_hw550-3g",
"images": [
{
"name": "openwrt-ramips-rt305x-aztech_hw550-3g-squashfs-sysupgrade.bin",
"sha256": "db2b34b0ec4a83d9bf612cf66fab0dc3722b191cb9bedf111e5627a4298baf20",
"type": "sysupgrade"
}
],
"metadata_version": 1,
"supported_devices": [
"aztech,hw550-3g",
"hw550-3g"
],
"target": "ramips/rt305x",
"titles": [
{
"model": "HW550-3G",
"vendor": "Aztech"
},
{
"model": "ALL0239-3G",
"vendor": "Allnet"
}
],
"version_commit": "r10920+123-0cc87b3bac",
"version_number": "SNAPSHOT"
}
Signed-off-by: Paul Spooren <mail@aparcar.org>
This reverts commit 1c0290c5cc.
Dropping the nopad can make the padding overflow into the next erase
block on devices using a non-aligned rootfs start. This breaks the jffs2
overlay partition with the following messages:
[ 30.343877] jffs2_scan_eraseblock(): End of filesystem marker found at 0x10000
[ 30.376512] jffs2: Cowardly refusing to erase blocks on filesystem with no valid JFFS2 nodes
[ 30.385253] jffs2: empty_blocks 196, bad_blocks 0, c->nr_blocks 197
Signed-off-by: Jonas Gorski <jonas.gorski@gmail.com>
While the -nopad option prevents mksquashfs from padding the
image to an arbitrary 4k. It does not take into consideration
that squashfs is programmed to have this 4k padding when it's
being used on on a block device... which is its main "use-case".
Now, after a week long discussion on the ML that included a
back-and-forth between some of the possible options.
But this is likely the best KISS patch to deal with the issue
right away given the limited resources.
From squashfs code point of view, be warned. The 4k padding is
not enough when dealing with devices that have a PAGE_SIZE
bigger than 4k.
if it turns out to be affecting you, then please look-up either:
"FS#2460 - kernel panic reading squashfs from ubi volume" bug
Or the discussion on the OpenWrt-Devel ML in
"amp821xx: use newly added pad-squashfs for Meraki MR24" and
"Squashfs breakage lottery with UBI..."
before making an educated guess.
Note: This will not affect the "tiny"/small flash devices as
much as it seems at first. This is because the the rootfs_data
partition that follows uses jffs2. And it requires to be aligned
to the flash block-size in order to work at all.
So either the involved FSes will meet in the middle as before,
or not at all. But in that latter case the image was already
hoping for the "undefined behaviour" gamble to turn out in its
favour and this is probably why this was unnoticed for so long.
Fixes: FS#2460
Reported-by: Russell Senior <russell@personaltelco.net>
Signed-off-by: Christian Lamparter <chunkeey@gmail.com>
Underscores don't harm in filenames, however the sanitize function from
version.mk replaces them, causing inconsistent filenames.
A fix tried to solve this previously via
dfe99645db however I did not look close
enough:
openwrt-ath79-generic-8dev-carambola2.manifest # current
openwrt-ath79-generic_8dev-carambola2.manifest # patched
---------------------^
Eventually the sanitization of PROFILES *could* be removed as more and
more profiles follow the device tree approach of vendor_model, neither
containing upper case letters nor spaces.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Some devices are produced and sold under different names. To debloat
the buildroot but keeping it complete, new variables are introduced to
handle different namings. Below an example taken from a recent PR[0]
DEVICE_VENDOR := Arcadyan
DEVICE_MODEL := ARV4520PW
DEVICE_ALT0_VENDOR := Vodafone
DEVICE_ALT0_MODEL := Easybox 800
DEVICE_ALT1_VENDOR := Airties
DEVICE_ALT1_MODEL := WAV-281
With this commit the buildroot is extended to take care of up to three
alternative namings. The primary title plus alternatives names (if
defined) are shown in the `make menuconfig` dialog. Selecting on of
devices automatically selects all alternative names as they share the
same profile.
A list of the newly introduced variables:
DEVICE_ALT0_VENDOR :=
DEVICE_ALT0_MODEL :=
DEVICE_ALT0_VARIANT :=
DEVICE_ALT1_VENDOR :=
DEVICE_ALT1_MODEL :=
DEVICE_ALT1_VARIANT :=
DEVICE_ALT2_VENDOR :=
DEVICE_ALT2_MODEL :=
DEVICE_ALT2_VARIANT :=
[0]: https://github.com/openwrt/openwrt/pull/2229/files#diff-b436f01932a18876c27800ba183d95f6R140
Signed-off-by: Paul Spooren <mail@aparcar.org>
Allow overriding the default selection state for Devices, similar to
setting a default for packages.
E.g. by setting DEFAULT to n, they won't be selected by default anymore
when enabling all device in the multi device profile.
This allows preventing images being built by the default config for
known broken devices, devices without enough RAM/flash, or devices not
working with a certain kernel versions.
This does not prevent the devices from being manually selected or images
being built by the ImageBuilder. These devices often still have worth
with a reduced package-set, or as a device for regression testing, when
no better device is available.
Signed-off-by: Jonas Gorski <jonas.gorski@gmail.com>
The manifest file is based on IMG_PREFIX and PROFILE_SANITIZED, whereas
the latter takes a string like DEVICE_8dev_carambola2 and sanitizes it.
This behaviour results in a useless "device_" profile-prefix in the
device manifest filename. Now uses *subst* to remove that.
Therefore this patch results more consistent device file names:
openwrt-ath79-generic-8dev_carambola2-initramfs-kernel.bin
openwrt-ath79-generic-8dev-carambola2.manifest
openwrt-ath79-generic-8dev_carambola2-squashfs-sysupgrade.bin
instead of a single file being called
openwrt-ath79-generic-device_8dev-carambola2.manifest
Signed-off-by: Paul Spooren <mail@aparcar.org>
This allows the definition of DEVICE_VENDOR, DEVICE_MODEL and
DEVICE_VARIANT. All three are merged together to look the same as the
current DEVICE_TITLE. Also, if DEVICE_TITLE is set it's directly used as
a *fallback* for devices which weren't upgraded yet.
This is based on the work of @sudhanshu16.
The motivation is to create JSON files based on provided metadata,
needing clear differentiation between vendor, model and revision.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Image padding with dd needs to be done in an OS independent way as not
all dd understand capital letter unit specifiers, and not all understand
lowercase either.
Fixes: 97833a57ef ("build: image: add pad-to and pad-rootfs-squashfs helpers")
Reported-by: Kevin Darbyshire-Bryant <ldir@darbyshire-bryant.me.uk>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
This common code could be shared by at least 3 targets (malta, armvirt
and x86) so let's factor it out to separate variable.
Signed-off-by: Petr Štetiar <ynezz@true.cz>
The helper shared Build/append-uboot in include/image-commands.mk
uses it, so include this variable in DEFAULT_DEVICE_VARS.
Signed-off-by: Christian Lamparter <chunkeey@gmail.com>
Otherwise tar will keep the sgid bit when running
from a sgid-set directory, resulting in a different
file being generated.
Signed-off-by: Eneas U de Queiroz <cote2004-github@yahoo.com>
Signed-off-by: Christian Lamparter <chunkeey@gmail.com>
[reworked commit message, removed DMARC]
It was reported to me on IRC today, that my change causes issues with
kernel versions between 4.14 and 4.19.
It's because I've wrongly used `git describe` in order to get kernel
version where we should disable noisy DTC checks, but I should've used
`git tag --contains` instead.
Fixes: cbbef976e2 ("build: dtc: Disable noisy warnings by default")
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Having image metadata (and signature) appended is a condition for
semi-automated sysupgrade, hence IB needs to be able to tell which
images will end up with metadata.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
squashfskit is a fork of the squashfs-tools.
squashfskit creates reproducible filesystems and includes
many of the distro patches.
Signed-off-by: Alexander Couzens <lynxis@fe80.eu>
Tested-by: Paul Spooren <mail@aparcar.org>
BIN_DIR can be set to overwrite the output path for new images. This is an
advertised feature for the imagebuilder and is used by systems like
LibreMesh's chef.
The legacy images are build using a new sub-make which doesn't receive the
variable overwrites of the parent make process. As result, the BIN_DIR is
automatically defined to the default value from rules.mk. The images will
therefore not be placed in the output path which was selected by the user.
Providing BIN_DIR as an explicit variable override to the sub-make works
around this problem.
Fixes: 26c771452c ("image.mk: add LegacyDevice wrapper to allow legacy image building code to be used for device profiles")
Reported-by: Paul Spooren <mail@aparcar.org>
Signed-off-by: Sven Eckelmann <sven@narfation.org>
The KERNEL_ENTRY was missing from the DEFAULT_DEVICE_VARS.
This bug was discovered while preparing alternative images
for the mpc85xx's TP-Link WDR4900-V1, which all failed to
boot due to this:
|## Booting kernel from Legacy Image at 02000000 ...
| Image Name: POWERPC OpenWrt Linux-4.14.96
| Image Type: PowerPC Linux Kernel Image (uncompressed)
| Data Size: 2056568 Bytes = 2 MiB
| Load Address: 01000000
| Entry Point: 00000000
| Verifying Checksum ... OK
Signed-off-by: Christian Lamparter <chunkeey@gmail.com>
It was reported to me today on IRC, that building of artifacts doesn't
work properly if the concat_cmd references DEVICE_NAME variable. I've
found out, that it's due to missing call of Device/Export in artifacts
building code path, so this patch adds the missing Device/Export call
which in turn exports DEFAULT_DEVICE_VARS into the artifacts
environment.
Fixes: 493c9a3551 ("build: Introduce building of artifacts")
Tested-by: Oskari Lemmela <oskari@lemmela.net>
Reported-by: Oskari Lemmela <oskari@lemmela.net>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
While helping with review and build testing of a few 4.19 pull requests,
I've noticed, that dtc compiler in OpenWrt uses different options then
upstream kernel, which is leading to a very noisy output[1]. It wouldn't
be that bad per se, but a lot of such warnings aren't easily fixable so
I think, that we should follow what upstream does and simply ignore^W
silence those noisy warnings.
So this patch tries to syncs dtc compiler flags with upstream kernel
till version 4.19.13, disabling those warnings as they were added in
upstream kernel:
v4.6-rc1-2-gbc55398 dtc: turn off dtc unit address warnings by default
The newly added dtc warning to check DT unit-address without reg
property and vice-versa generates lots of warnings. Turn off the check
unless building with W=1 or W=2.
v4.11-rc2-11-g8654cb8 dtc: update warning settings for new bus and node/property
dtc gained new warnings checking PCI and simple buses, unit address
formatting, and stricter node and property name checking. Disable the
new dtc warnings by default as there are 1000s. As before, warnings are
enabled with W=1 or W=2. The strict node and property name checks are a
bit subjective, so they are only enabled for W=2.
v4.16-rc3-9-g4fd98e3 scripts: turn off some new dtc warnings by default
The latest dtc update adds some new noisy warnings, so turn them off by
default. Disable 'avoid_unnecessary_addr_size' and 'alias_paths'. They
can be re-enabled by building with 'W=1'.
v4.17-rc1-27-g74656b6 kbuild: disable new dtc graph and unit-address warnings
dtc gained some new warnings for OF graphs and unique unit addresses,
but they are currently much too noisy. So turn off
'graph_child_address', 'graph_port', and 'unique_unit_address' warnings
by default. They can be enabled by building dtbs with W=1.
Build tested on imx6 and ath79 with 4.14 and 4.19.
1. https://github.com/openwrt/openwrt/pull/1694#issuecomment-450864335
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Add the ABI_VERSION source makefile variable to the binary package basename
and resolve source dependencies on packages with ABI_VERSION set to such
expanded names.
If for example a package specifies DEPENDS:=libopenssl while the OpenSSL
Makefile specifies ABI_VERSION:=1.0.0, the resulting ipk control data
dependency will be "Depends: libopenssl1.0.0" and the libopenssl ipk file
will be called "libopenssl1.0.0_<version>_<arch>.ipk".
The next time a library such as OpenSSL is updated to an incompatible
version, the ABI_VERSION shall be changed accordingly to prevent opkg from
simply upgrading to an incompatible library without considering the
dependencies of already installed packages.
Also introduce another "SourceName" control field which is required by
the newly introduced "scritps/ipkg-remove" to determine the proper related
.ipk files to delete upon buildroot package clean operations.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
