Commit Graph

370 Commits

Author SHA1 Message Date
Adrian Schmutzler
1d5260cf72 build: add option to mark devices as BROKEN
By specifying "BROKEN := 1" or "BROKEN := y" for a device, it will be
hidden (and deselected) by default. By that, it provides a stronger
option to "disable" a device beyond just using DEFAULT := n.

To make these devices visible, just enable the BROKEN option in
developer settings as already implemented for targets and packages.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2020-07-30 21:49:07 +02:00
Paul Spooren
263f7e5bbd build: store default/device packages in JSON
With this commit the `profiles.json` contain both the target specific
`default_packages` as well as the device specific `device_packages` as a
array of strings.

This information is required for downstream projects like the various
web-based interactive firmware generators.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-06-30 23:21:10 +01:00
Kevin Darbyshire-Bryant
c14c6902f5 treewide: convert sed -r to posix -E
Signed-off-by: Kevin Darbyshire-Bryant <ldir@darbyshire-bryant.me.uk>
2020-04-04 11:28:15 +01:00
Paul Spooren
07449f692c build: refactor JSON info files to profiles.json
JSON info files contain machine readable information of built profiles
and resulting images. These files were added in commit 881ed09ee6
("build: create JSON files containing image info").

They are useful for firmware wizards and script checking for
reproducibility.

Currently all JSON files are stored next to the built images, resulting
in up to 168 individual files for the ath79/generic target.

This patch refactors the JSON creation to store individual per image
(not per profile) files in $(BUILD_DIR)/json_info_files and create an
single overview file called `profiles.json` in the target directory.

Storing per image files and not per profile solves the problem of
parallel file writes. If a profiles sysupgrade and factory image are
finished at the same time both processes would write to the same JSON
file, resulting in randomly broken outputs.

Some target like x86/64 do not use the image code yet, resulting in
missing JSON files. If no JSON info files were created, no
`profiles.json` files is created as it would be empty anyway.

As before, this creation is enabled by default only if `BUILDBOT` is set.

Tested via buildroot & ImageBuilder on ath79/generic, imx6 and x86/64.

Signed-off-by: Paul Spooren <mail@aparcar.org>
[json_info_files dir handling in Make, if case refactoring]
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-04-03 12:17:45 +02:00
李国
a6b7c3e672 x86: generate EFI platform bootable images
Add EFI platform bootable images for x86 platforms. These images can
also boot from legacy BIOS platform.

EFI System Partition need to be fat12/fat16/fat32 (not need to load
filesystem drivers), so the first partition of EFI images are not ext4
filesystem any more.

GPT partition table has an alternate partition table, we did not
generate it. This may cause problems when use these images as qemu disk
(kernel can not find rootfs), we pad enough sectors will be ok.

Signed-off-by: 李国 <uxgood.org@gmail.com>
[part_magic_* refactoring, removed genisoimage checks]
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-03-31 16:20:47 +02:00
Paul Spooren
cb007a7bf6 x86: switch image generation to new code
This commit introduces few related changes which need to be done in
single commit to keep images buildable between git revisions. In result
it retains all previous image creation possibilities with slight name
change of generated images. Brief summary of the commit:

* Split up image generation recipe to smaller chunks to make it more
  generic and reusable.

* Make iso images x86 specific and drop their definition as root
  filesystem.

* Convert image creation process to generic code specified in image.mk.

* Make geode subtarget inherit features from the main target instead of
  redefining them.

* For subtargets create device definitions with basic packages set.

Signed-off-by: Tomasz Maciej Nowak <tomek_n@o2.pl>
[rebased]
Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-03-21 10:36:00 +00:00
Sungbo Eo
90daff4cf8 build: image: move IMAGE_SIZE to image.mk
IMAGE_SIZE is widely used in many targets. Declare it in the default template to
clean up redundant code. This also prevents deriving IMAGE_SIZE unintentionally
from the previously defined device.

While at it, remove duplicate KERNEL_SIZE declaration.

Signed-off-by: Sungbo Eo <mans0n@gorani.run>
2020-03-11 18:25:06 +01:00
Paul Spooren
73f3ad1df2 build: fix empty SUBTARGET in json files
Some targets like kirkwood or omap don't use a subtarget which results
in a malformed JSON info file.

Instead of having a valid value like `"target": "ath79/tiny"` for these
targets the value is `"target": "kirkwood/"`.

This patch uses the same if condition to use `generic` if the subtarget
is empty.

Tested for the kirkwood target.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-02-13 17:45:46 +01:00
Sungbo Eo
c26b687e31 kernel: remove further obsolete kernel version switches
Most of the kernel version switches below 4.14 were removed in commit
97940f8766 ("kernel: remove obsolete kernel version switches"),
but some of them still remained. Remove them now.

Signed-off-by: Sungbo Eo <mans0n@gorani.run>
2020-01-18 19:39:05 +01:00
Adrian Schmutzler
bc3783d221 build: image: add SOC device variable
This creates the device variable SOC and adds it to DEFAULT_DEVICE_VARS.

It is supposed to replace target-specific SOC variables like ATH_SOC or
MTK_SOC and thus unify variable names across targets.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2019-12-20 01:07:25 +01:00
Kevin Darbyshire-Bryant
dfd8c45a98 build: image: posix compatibility cut v head
Replace 2 instances of non posix use of 'head' with posix compliant
'cut'.

'head -c n' cuts 'n' bytes from the passed string and happens to work on
Linux & Mac OS X even though '-c' is not posix.

'head --bytes n' does the same thing and happens to work on linux but
not on Mac OS X and is also not posix.

'cut -b1-8' cuts the first 8 bytes from the passed string and is posix
compliant, hence works on Linux & Mac OS X.

Our usage of 'head --bytes' was particularly unfortunate since it was
used to calculated the RootFS UUID passed to grub - the net result being
a non-functioning system waiting for the root file system to appear.

Thanks to karlp, ynezz & others for pointers on solving this.

Signed-off-by: Kevin Darbyshire-Bryant <ldir@darbyshire-bryant.me.uk>
2019-11-12 15:43:14 +00:00
Paul Spooren
98d1c7d834 build: image: add common and reproducible IMG_PART_SIGNATURE variable
x86, mvebu and tegra targets are currently using more or less same
SIGNATURE variable which provides unique partition ID/signature, so it
makes sense to refactor it out into common variable which could be
reused by all targets.  While at it, make the content of the variable
reproducible.

Ref: http://lists.infradead.org/pipermail/openwrt-devel/2019-March/016148.html
Signed-off-by: Paul Spooren <mail@aparcar.org>
Suggested-by: Jo-Philipp Wich <jo@mein.io>
[renamed to IMG_PARTSIGNATURE, reworked with epoch+vermagic hash]
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-11-11 16:37:13 +01:00
Petr Štetiar
0fb23d67f0 build: image: fix build breakage of some images
Commit 881ed09ee6 ("build: create JSON files containing image info")
has removed the crucial empty new line from the image copy step
resulting in the following errors during make function expansion:

 GZ_SUFFIX :=
 bash: GZ_SUFFIX: command not found
 Makefile:86: recipe for target 'openwrt-ath79-generic-tplink_archer-c7-v5-squashfs-sysupgrade.bin' failed

Fixes: 881ed09ee6 ("build: create JSON files containing image info")
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-11-08 11:02:18 +01:00
Yousong Zhou
66ebca7103 build: label kernel and rootfs ext4 volumes
UUID of ext4 volumes generated by make_ext4fs are determined by volume
label and it will all be 57f8f4bc-abf4-655f-bf67-946fc0f9f25b when label
is empty

Labeling them does not make them unique but tools like block command
from fstools have a better chance differentiating them

Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
2019-11-03 04:25:40 +00:00
Paul Spooren
881ed09ee6 build: create JSON files containing image info
The JSON info files contain details about the created firmware images
per device and are stored next to the created images.

The JSON files are stored as "$(IMAGE_PREFIX).json" and contain some
device/image meta data as well as a list of created firmware images.

An example of openwrt-ramips-rt305x-aztech_hw550-3g.json

    {
      "id": "aztech_hw550-3g",
      "image_prefix": "openwrt-ramips-rt305x-aztech_hw550-3g",
      "images": [
        {
          "name": "openwrt-ramips-rt305x-aztech_hw550-3g-squashfs-sysupgrade.bin",
          "sha256": "db2b34b0ec4a83d9bf612cf66fab0dc3722b191cb9bedf111e5627a4298baf20",
          "type": "sysupgrade"
        }
      ],
      "metadata_version": 1,
      "supported_devices": [
        "aztech,hw550-3g",
        "hw550-3g"
      ],
      "target": "ramips/rt305x",
      "titles": [
        {
          "model": "HW550-3G",
          "vendor": "Aztech"
        },
        {
          "model": "ALL0239-3G",
          "vendor": "Allnet"
        }
      ],
      "version_commit": "r10920+123-0cc87b3bac",
      "version_number": "SNAPSHOT"
    }

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-09-29 13:51:28 +02:00
Jonas Gorski
f11d90a76b Revert "build: remove harmful -nopad option from mksquashfs"
This reverts commit 1c0290c5cc.

Dropping the nopad can make the padding overflow into the next erase
block on devices using a non-aligned rootfs start. This breaks the jffs2
overlay partition with the following messages:

[   30.343877] jffs2_scan_eraseblock(): End of filesystem marker found at 0x10000
[   30.376512] jffs2: Cowardly refusing to erase blocks on filesystem with no valid JFFS2 nodes
[   30.385253] jffs2: empty_blocks 196, bad_blocks 0, c->nr_blocks 197

Signed-off-by: Jonas Gorski <jonas.gorski@gmail.com>
2019-09-07 14:45:32 +02:00
Christian Lamparter
1c0290c5cc build: remove harmful -nopad option from mksquashfs
While the -nopad option prevents mksquashfs from padding the
image to an arbitrary 4k. It does not take into consideration
that squashfs is programmed to have this 4k padding when it's
being used on on a block device... which is its main "use-case".

Now, after a week long discussion on the ML that included a
back-and-forth between some of the possible options.
But this is likely the best KISS patch to deal with the issue
right away given the limited resources.

From squashfs code point of view, be warned. The 4k padding is
not enough when dealing with devices that have a PAGE_SIZE
bigger than 4k.

if it turns out to be affecting you, then please look-up either:
"FS#2460 - kernel panic reading squashfs from ubi volume" bug
Or the discussion on the OpenWrt-Devel ML in
"amp821xx: use newly added pad-squashfs for Meraki MR24" and
"Squashfs breakage lottery with UBI..."
before making an educated guess.

Note: This will not affect the "tiny"/small flash devices as
much as it seems at first. This is because the the rootfs_data
partition that follows uses jffs2. And it requires to be aligned
to the flash block-size in order to work at all.

So either the involved FSes will meet in the middle as before,
or not at all. But in that latter case the image was already
hoping for the "undefined behaviour" gamble to turn out in its
favour and this is probably why this was unnoticed for so long.

Fixes: FS#2460
Reported-by: Russell Senior <russell@personaltelco.net>
Signed-off-by: Christian Lamparter <chunkeey@gmail.com>
2019-08-30 20:45:40 +02:00
Paul Spooren
58aab73797 image.mk: keep underscores when sanitize PROFILE
Underscores don't harm in filenames, however the sanitize function from
version.mk replaces them, causing inconsistent filenames.

A fix tried to solve this previously via
dfe99645db however I did not look close
enough:

openwrt-ath79-generic-8dev-carambola2.manifest # current
openwrt-ath79-generic_8dev-carambola2.manifest # patched
---------------------^

Eventually the sanitization of PROFILES *could* be removed as more and
more profiles follow the device tree approach of vendor_model, neither
containing upper case letters nor spaces.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-08-22 12:57:34 +02:00
Paul Spooren
4ee3cf2b5a build: introduce ALT vendor/model/variant
Some devices are produced and sold under different names. To debloat
the buildroot but keeping it complete, new variables are introduced to
handle different namings. Below an example taken from a recent PR[0]

    DEVICE_VENDOR := Arcadyan
    DEVICE_MODEL := ARV4520PW
    DEVICE_ALT0_VENDOR := Vodafone
    DEVICE_ALT0_MODEL := Easybox 800
    DEVICE_ALT1_VENDOR := Airties
    DEVICE_ALT1_MODEL := WAV-281

With this commit the buildroot is extended to take care of up to three
alternative namings. The primary title plus alternatives names (if
defined) are shown in the `make menuconfig` dialog. Selecting on of
devices automatically selects all alternative names as they share the
same profile.

A list of the newly introduced variables:

    DEVICE_ALT0_VENDOR :=
    DEVICE_ALT0_MODEL :=
    DEVICE_ALT0_VARIANT :=
    DEVICE_ALT1_VENDOR :=
    DEVICE_ALT1_MODEL :=
    DEVICE_ALT1_VARIANT :=
    DEVICE_ALT2_VENDOR :=
    DEVICE_ALT2_MODEL :=
    DEVICE_ALT2_VARIANT :=

[0]: https://github.com/openwrt/openwrt/pull/2229/files#diff-b436f01932a18876c27800ba183d95f6R140

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-08-22 12:50:41 +02:00
Jonas Gorski
7546be6007 build: allow overriding default selection state for devices
Allow overriding the default selection state for Devices, similar to
setting a default for packages.

E.g. by setting DEFAULT to n, they won't be selected by default anymore
when enabling all device in the multi device profile.

This allows preventing images being built by the default config for
known broken devices, devices without enough RAM/flash, or devices not
working with a certain kernel versions.

This does not prevent the devices from being manually selected or images
being built by the ImageBuilder. These devices often still have worth
with a reduced package-set, or as a device for regression testing, when
no better device is available.

Signed-off-by: Jonas Gorski <jonas.gorski@gmail.com>
2019-08-13 22:13:18 +02:00
Paul Spooren
dfe99645db image.mk: remove device_ from manifest filename
The manifest file is based on IMG_PREFIX and PROFILE_SANITIZED, whereas
the latter takes a string like DEVICE_8dev_carambola2 and sanitizes it.
This behaviour results in a useless "device_" profile-prefix in the
device manifest filename. Now uses *subst* to remove that.

Therefore this patch results more consistent device file names:

openwrt-ath79-generic-8dev_carambola2-initramfs-kernel.bin
openwrt-ath79-generic-8dev-carambola2.manifest
openwrt-ath79-generic-8dev_carambola2-squashfs-sysupgrade.bin

instead of a single file being called

openwrt-ath79-generic-device_8dev-carambola2.manifest

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-08-13 10:40:36 +02:00
Paul Spooren
8c3dbaf8b4 ath79: split DEVICE_TITLE in multiple variables
This allows the definition of DEVICE_VENDOR, DEVICE_MODEL and
DEVICE_VARIANT. All three are merged together to look the same as the
current DEVICE_TITLE. Also, if DEVICE_TITLE is set it's directly used as
a *fallback* for devices which weren't upgraded yet.

This is based on the work of @sudhanshu16.

The motivation is to create JSON files based on provided metadata,
needing clear differentiation between vendor, model and revision.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-07-04 15:50:17 +02:00
Petr Štetiar
ef5b7aa6c3 build: image: make image padding OS agnostic
Image padding with dd needs to be done in an OS independent way as not
all dd understand capital letter unit specifiers, and not all understand
lowercase either.

Fixes: 97833a57ef ("build: image: add pad-to and pad-rootfs-squashfs helpers")
Reported-by: Kevin Darbyshire-Bryant <ldir@darbyshire-bryant.me.uk>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-06-27 17:15:37 +02:00
Petr Štetiar
62105e5c9a build: image: add IMG_ROOTFS and IMG_COMBINED variables
Adding those two variables in order to share them across the tree.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-06-25 14:12:22 +00:00
Petr Štetiar
83d9fb0a99 build: image: add variable for gzip-ext4-padded-squashfs
This common code could be shared by at least 3 targets (malta, armvirt
and x86) so let's factor it out to separate variable.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-06-25 14:12:22 +00:00
Petr Štetiar
97833a57ef build: image: add pad-to and pad-rootfs-squashfs helpers
For better reusability.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-06-25 14:12:22 +00:00
Christian Lamparter
80f861ef75 build: add UBOOT_PATH to DEFAULT_DEVICE_VARS and set a default
The helper shared Build/append-uboot in include/image-commands.mk
uses it, so include this variable in DEFAULT_DEVICE_VARS.

Signed-off-by: Christian Lamparter <chunkeey@gmail.com>
2019-05-18 16:37:30 +02:00
Eneas U de Queiroz
8252511dc0 build: remove sgid permission from tar
Otherwise tar will keep the sgid bit when running
from a sgid-set directory, resulting in a different
file being generated.

Signed-off-by: Eneas U de Queiroz <cote2004-github@yahoo.com>
Signed-off-by: Christian Lamparter <chunkeey@gmail.com>
[reworked commit message, removed DMARC]
2019-04-06 19:14:06 +02:00
Petr Štetiar
6027e49093 build: image: Fix off-by-one in DTC kernel version checks
It was reported to me on IRC today, that my change causes issues with
kernel versions between 4.14 and 4.19.

It's because I've wrongly used `git describe` in order to get kernel
version where we should disable noisy DTC checks, but I should've used
`git tag --contains` instead.

Fixes: cbbef976e2 ("build: dtc: Disable noisy warnings by default")
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-04-01 13:53:31 +02:00
Daniel Golle
d6fa04a437 IB: include SUPPORTED_DEVICES in 'make info' output
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2019-03-11 05:32:29 +01:00
Daniel Golle
13c379e5c6 ib: display whether profile comes with image metadata
Having image metadata (and signature) appended is a condition for
semi-automated sysupgrade, hence IB needs to be able to tell which
images will end up with metadata.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2019-03-06 14:50:42 +01:00
Alexander Couzens
889b6423b7
tools: migrate from squashfs4 to squashfskit4
squashfskit is a fork of the squashfs-tools.
squashfskit creates reproducible filesystems and includes
many of the distro patches.

Signed-off-by: Alexander Couzens <lynxis@fe80.eu>
Tested-by: Paul Spooren <mail@aparcar.org>
2019-02-28 01:38:46 +01:00
Sven Eckelmann
9a5a10eb69 build: Accept BIN_DIR parameter for legacy-images
BIN_DIR can be set to overwrite the output path for new images. This is an
advertised feature for the imagebuilder and is used by systems like
LibreMesh's chef.

The legacy images are build using a new sub-make which doesn't receive the
variable overwrites of the parent make process. As result, the BIN_DIR is
automatically defined to the default value from rules.mk. The images will
therefore not be placed in the output path which was selected by the user.

Providing BIN_DIR as an explicit variable override to the sub-make works
around this problem.

Fixes: 26c771452c ("image.mk: add LegacyDevice wrapper to allow legacy image building code to be used for device profiles")
Reported-by: Paul Spooren <mail@aparcar.org>
Signed-off-by: Sven Eckelmann <sven@narfation.org>
2019-02-17 19:22:39 +01:00
Christian Lamparter
0dc48905cb build: add KERNEL_ENTRY and sort DEFAULT_DEVICE_VARS
The KERNEL_ENTRY was missing from the DEFAULT_DEVICE_VARS.

This bug was discovered while preparing alternative images
for the mpc85xx's TP-Link WDR4900-V1, which all failed to
boot due to this:
|## Booting kernel from Legacy Image at 02000000 ...
|   Image Name:   POWERPC OpenWrt Linux-4.14.96
|   Image Type:   PowerPC Linux Kernel Image (uncompressed)
|   Data Size:    2056568 Bytes = 2 MiB
|   Load Address: 01000000
|   Entry Point:  00000000
|   Verifying Checksum ... OK

Signed-off-by: Christian Lamparter <chunkeey@gmail.com>
2019-02-11 19:01:50 +01:00
Petr Štetiar
bb23cb1bf2 build: Fix missing device variables for artifacts
It was reported to me today on IRC, that building of artifacts doesn't
work properly if the concat_cmd references DEVICE_NAME variable. I've
found out, that it's due to missing call of Device/Export in artifacts
building code path, so this patch adds the missing Device/Export call
which in turn exports DEFAULT_DEVICE_VARS into the artifacts
environment.

Fixes: 493c9a3551 ("build: Introduce building of artifacts")
Tested-by: Oskari Lemmela <oskari@lemmela.net>
Reported-by: Oskari Lemmela <oskari@lemmela.net>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-02-05 19:37:30 +01:00
Petr Štetiar
cbbef976e2 build: dtc: Disable noisy warnings by default
While helping with review and build testing of a few 4.19 pull requests,
I've noticed, that dtc compiler in OpenWrt uses different options then
upstream kernel, which is leading to a very noisy output[1]. It wouldn't
be that bad per se, but a lot of such warnings aren't easily fixable so
I think, that we should follow what upstream does and simply ignore^W
silence those noisy warnings.

So this patch tries to syncs dtc compiler flags with upstream kernel
till version 4.19.13, disabling those warnings as they were added in
upstream kernel:

 v4.6-rc1-2-gbc55398 dtc: turn off dtc unit address warnings by default

  The newly added dtc warning to check DT unit-address without reg
  property and vice-versa generates lots of warnings. Turn off the check
  unless building with W=1 or W=2.

 v4.11-rc2-11-g8654cb8 dtc: update warning settings for new bus and node/property

  dtc gained new warnings checking PCI and simple buses, unit address
  formatting, and stricter node and property name checking. Disable the
  new dtc warnings by default as there are 1000s. As before, warnings are
  enabled with W=1 or W=2. The strict node and property name checks are a
  bit subjective, so they are only enabled for W=2.

 v4.16-rc3-9-g4fd98e3 scripts: turn off some new dtc warnings by default

  The latest dtc update adds some new noisy warnings, so turn them off by
  default. Disable 'avoid_unnecessary_addr_size' and 'alias_paths'. They
  can be re-enabled by building with 'W=1'.

 v4.17-rc1-27-g74656b6 kbuild: disable new dtc graph and unit-address warnings

  dtc gained some new warnings for OF graphs and unique unit addresses,
  but they are currently much too noisy. So turn off
  'graph_child_address', 'graph_port', and 'unique_unit_address' warnings
  by default. They can be enabled by building dtbs with W=1.

Build tested on imx6 and ath79 with 4.14 and 4.19.

1. https://github.com/openwrt/openwrt/pull/1694#issuecomment-450864335

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-01-27 00:16:14 +01:00
Jo-Philipp Wich
e6bcf1e4ac build: add ABI_VERSION to binary package names
Add the ABI_VERSION source makefile variable to the binary package basename
and resolve source dependencies on packages with ABI_VERSION set to such
expanded names.

If for example a package specifies DEPENDS:=libopenssl while the OpenSSL
Makefile specifies ABI_VERSION:=1.0.0, the resulting ipk control data
dependency will be "Depends: libopenssl1.0.0" and the libopenssl ipk file
will be called "libopenssl1.0.0_<version>_<arch>.ipk".

The next time a library such as OpenSSL is updated to an incompatible
version, the ABI_VERSION shall be changed accordingly to prevent opkg from
simply upgrading to an incompatible library without considering the
dependencies of already installed packages.

Also introduce another "SourceName" control field which is required by
the newly introduced "scritps/ipkg-remove" to determine the proper related
.ipk files to delete upon buildroot package clean operations.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2019-01-19 14:32:12 +01:00
Petr Štetiar
493c9a3551 build: Introduce building of artifacts
We currently could (ab)use IMAGES for this task, but the downside is,
that the filenames has filesystem tied to the filename, which might be
confusing as the artifact itself don't has to be used with that specific
filesystem. Another downside is, that the artifacts built with IMAGES
target are build for every FILESYSTEMS filesystem.

Consider following use case:

 define Device/apalis
   ...
   FILESYSTEMS := ext4 squashfs
   IMAGES := spl-uboot.bin recovery.scr
   IMAGE/spl-uboot.bin := append-uboot-spl | pad-to 68k | append-uboot
   IMAGE/recovery.scr := recovery-scr
 endef

Where we would get target binaries with following filenames:

 openwrt-imx6-apalis-squashfs.recovery.scr
 openwrt-imx6-apalis-squashfs.spl-uboot.bin
 openwrt-imx6-apalis-ext4.recovery.scr
 openwrt-imx6-apalis-ext4.spl-uboot.bin

With proposed patch, we could now just do:

 define Device/apalis
   ...
   ARTIFACTS := spl-uboot.bin recovery.scr
   ARTIFACT/spl-uboot.bin := append-uboot-spl | pad-to 68k | append-uboot
   ARTIFACT/recovery.scr := recovery-scr
 endef

Which would produce target binaries with following filenames:

 openwrt-imx6-apalis-recovery.scr
 openwrt-imx6-apalis-spl-uboot.bin

Signed-off-by: Petr Štetiar <ynezz@true.cz>
Acked-by: Hauke Mehrtens <hauke@hauke-m.de>
2018-11-26 09:39:20 +01:00
Felix Fietkau
516d995d6a build: ensure that iwinfo is selected when building for multiple devices
extra_packages needs to be added there, like on profiles and target
packages lists

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2018-07-02 14:27:04 +02:00
Sven Eckelmann
1b773a46c2 build: Allow to change the FIT config section name
Some devices only boot when a special config is found in the image and
completely ignore the default entry during the selection. These devices can
now use the variable DEVICE_DTS_CONFIG in their device image definition.

Signed-off-by: Sven Eckelmann <sven@narfation.org>
2018-03-17 08:09:04 +01:00
Christian Lamparter
5862f01ef7 build: revert "make image target wait for initramfs"
This reverts commit 43be5087a9.

The change is incompatible with the image builder code.
Luckily the RT-AC58U is no longer depending on the initramfs
being available for the target's image generation rules.

Reported-by: Venitex Aveon <aveon@aenote.com>
Signed-off-by: Christian Lamparter <chunkeey@gmail.com>
2018-03-15 22:27:37 +01:00
Christian Lamparter
43be5087a9 build: make image target wait for initramfs
The image production rules does not have the initramfs-image
as a dependency. So, from make’s perspective initramfs
creation can run independently/in parallel with the image
generation code in the target's Makefile.

This is a problem for devices that have to use the initramfs
for the image creation and can lead to broken images.

Signed-off-by: Christian Lamparter <chunkeey@gmail.com>
2018-03-14 19:04:51 +01:00
Matthias Schiffer
643850f568
inclue/image.mk: fix package installation for per-device rootfs
Fixes FS#1421.

Fixes: 2fbf669730 ("imagebuilder: reuse rootfs preparation from rootfs.mk")
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-03-09 08:18:54 +01:00
Matthias Schiffer
f505fb07f3
include/image.mk: base package-list manifest on unprepared rootfs
With CONFIG_CLEAN_IPKG, the prepared rootfs does not have a full opkg
status file.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-03-07 09:59:08 +01:00
Matthias Schiffer
cf1c7c0f17
include/rootfs.mk: pass additional files dir to prepare_rootfs as an argument
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-03-07 09:59:07 +01:00
Felix Fietkau
cde71a543c build: replace uses of OpenWrt with $(VERSION_DIST)
This makes the distribution name more configurable.

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2018-02-05 10:15:53 +01:00
Jo-Philipp Wich
ab1785b1b2 build: fix restoring /etc/opkg with PER_DEVICE_ROOTFS
When generating per-device rootfs directories, the ./etc/opkg/ directory
is moved away prior to calling opkg install, opkg remove and rootfs_prepare.
After the opkg invocations and the rootfs_prepare macro call, the saved opkg
config directory is supposed to be moved back to its previous ./etc/opkg
location.

The mv command however can fail to properly restore the directory under
certain circumstances, e.g. when the prior opkg or files/ overlay copy
operations caused a new ./etc/opkg/ directory to be created.

In this case, the backed up directory (named target-dir-$hash.opkg) will be
moved into the preexisting ./etc/opkg/ directory instead, causing the opkg
configuration to be located in a wrong path on the final rootfs, e.g. in
/etc/opkg/target-dir-$hash.opkg/distfeeds.conf instead of
/etc/opkg/distfeeds.conf.

Solve this problem by replacing the naive "mv" command with a recursive
"cp -T" invocation which causes the backed up directory tree to get merged
with the destination directory in case it already exists.

Also perform the rootfs_prepare macro call after restoring the opkg
configuration, to allow users to override it again by using the files/
overlay mechanism.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2018-01-11 18:20:39 +01:00
Zoltan HERPAI
7b5c989ab9 merge: targets: update image generation and targets
Signed-off-by: Zoltan HERPAI <wigyori@uid0.hu>
2017-12-08 19:41:18 +01:00
Felix Fietkau
b0e6284879 build: fix generating dtb with / in DEVICE_DTS
Fixes layerscape build error

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-11-09 14:40:32 +01:00
Felix Fietkau
c6f71ad30b build: fix another regression in append-dtb fix
Filter out duplicate generated code for the same dts files

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-11-03 16:21:24 +01:00
Felix Fietkau
fae5f02912 build: fix regression in append-dtb fix
Strip whitespace from DTS directory
Put the dependency earlier in the chain to avoid make errors

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-11-03 11:00:20 +01:00
Felix Fietkau
986d9deb3b build: allow calling append-dtb from image build commands
mpc85xx uses this for firmware image files, since the dtb data is not
directly part of the kernel image. This causes build failures in the
image builder.

Fix this by adding a separate build step that runs this call earlier,
reusing the generated file for any calls from kernel or image build
commands.

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-11-02 16:01:18 +01:00
Florian Fainelli
818f36aa14 include: Include new location for DT bindings
Starting with commit d5d332d3f7e8 ("devicetree: Move include prefixes
from arch to separate directory") included in 4.12 and newer relocated
the dt-bindings directory, so account for that while passing CPPFLAGS
before DTC runs.

Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
2017-10-23 18:39:29 -07:00
Michal Sojka
aa8e91a1e4 image.mk: Generate cpiogz with root-owned files
Some files (e.g. /etc/dropbear) need to be owned by root. Add cpio
option to ensure that.

Other image types (at least targz and squashfs) already have this.

Signed-off-by: Michal Sojka <sojkam1@fel.cvut.cz>
2017-05-14 21:43:17 +02:00
John Crispin
003e15221a include/image.mk: allow passing a compat string to the NAND image template
Signed-off-by: John Crispin <john@phrozen.org>
2017-04-24 11:11:52 +02:00
John Crispin
eb3ac8281b include: add KERNEL_LOAD_ADDR to TARGET_VARS
This will allow us to override the variable from within a Device template.

Signed-off-by: John Crispin <john@phrozen.org>
2017-03-22 09:45:18 +01:00
Felix Fietkau
0d6494daf5 image.mk: force kernel rebuild on every run
DTS dependencies are not processed correctly so makes it safer against
poentially stale builds

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-03-13 13:33:50 +01:00
Felix Fietkau
9467ce42da build: get rid of host.mk
Defined required host related variables in toplevel.mk instead

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-02-26 13:31:44 +01:00
Felix Fietkau
8af5e5751d image.mk: add generic function for gzipping images if enabled
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-13 16:59:28 +01:00
Felix Fietkau
87b668765e image: when using the new image build code, gzip ext4 images by default
This reduces the amount of hacks in the makefile code.

Remove the apm821xx code to do the same - it was broken and left both
compressed and uncompressed images in $(BIN_DIR)

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-13 16:59:28 +01:00
Ian Pozella
859693509f image.mk: use LINUX_KARCH rather than ARCH for mkits
The generated 'its' is passed to mkimage which expects linux arch
strings rather than the full arch (e.g. mips not mipsel).

It currently works in some cases where LINUX_KARCH == ARCH but
otherwise you get an unknown arch build error.

Signed-off-by: Ian Pozella <Ian.Pozella@imgtec.com>
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-13 14:54:11 +01:00
Felix Fietkau
1e1d735e52 build: remove obsolete parallel build related options
Always use the main make jobserver, which has been the default for ages

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-10 12:10:20 +01:00
Felix Fietkau
5903c46ef8 build: fix build of ubifs images
--force-compr was added by the xz compression patch, which is gone now.

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-09 16:31:39 +01:00
Felix Fietkau
84bd74057f build: use mkhash to replace various quirky md5sum/openssl calls
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-05 11:09:12 +01:00
Jo-Philipp Wich
9791fb2ac2 build: support adding version code to file names (FS#323)
Now that the VERSION_NUMBER variable holds the human friendly name and not
the commit ID anymore, we need to support adding the revision ID as well.

Introduce a new config variable CONFIG_VERSION_CODE_FILENAMES which, if set,
causes the resulting file names to contain a commit ID designation as printed
by scripts/getver.sh.

Also sanitize the input variables to ensure that the resulting strings are
lowercased and no not contain spaces.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2016-12-09 17:16:43 +01:00
Felix Fietkau
77265e00c7 build: add support code for appending metadata to images
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-11-19 11:24:10 +01:00
Jo-Philipp Wich
d1ae4c4958 config: ext4: drop option to set maximum number of inodes
There is very little practical use to limit the number of available inodes on
an ext4 filesystem and the make_ext4fs utility is able to calculate useful
defaults by itself.

Drop the option to make resulting ext4 filesystems more flexible by default.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Acked-by: Michael Heimpold <mhei@heimpold.de>
2016-10-27 19:24:38 +02:00
Jo-Philipp Wich
244955de16 include: image.mk: make ext4 reserved blocks percentage optional
Allow CONFIG_TARGET_EXT4_RESERVED_PCT to be empty as make_ext4fs is usually
able to figure out a suitable default.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Reviewed-by: Michael Heimpold <mhei@heimpold.de>
2016-10-27 19:24:38 +02:00
Matthias Schiffer
021b96d7c5
rootfs: remove unnecessary and potentially harmful force flags from opkg call
Especially --force-overwrite and --force-depends will often lead to broken
images; it's better to fail the build in such cases than to silently ignore
the errors.

Instead, ignore errors in the per-device rootfs opkg remove command, so
the build doesn't break when packages can't be removed because of
dependencies.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2016-09-26 17:57:58 +02:00
Matthias Schiffer
593dfac909
image: per-device rootfs: first remove, then install packages
Some DEVICE_PACKAGES definitions replace one package variant with another
(e.g. wpad-mini is replaced with wpad). To avoid file conflicts, first
remove, then install packages.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2016-09-26 17:57:58 +02:00
Matthias Schiffer
a16a8814ea
image: don't modify file permissions before rootfs generation
Modifying the file permissions can be harmful, as it would make files
world-readable even if they weren't in the ipk packages. The
Image/mkfs/prepare step is removed completely, as it is redundant now (/tmp
and /overlay are already provided by base-files with the correct
permissions).

It has been verified that this change does not affect any permissions of
files in the default package set except /etc/ppp/chap-secrets, which was
world-readable before. All packages not in the default set are more likely
to be installed via opkg than being part of a base image and thus were
usually not affected by the permission modification anyways.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2016-09-26 17:57:56 +02:00
Matthias Schiffer
663145e419
image: fix CONFIG_CLEAN_IPKG with CONFIG_TARGET_PER_DEVICE_ROOTFS
Running prepare_rootfs on TARGET_DIR deletes the opkg state when
CONFIG_CLEAN_IPKG is enabled, making the per-device rootfs package install
fail.

To avoid this, create a copy of the TARGET_DIR before prepare_rootfs is run
and use this as basis for per-device rootfs generation.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2016-09-25 09:30:55 +02:00
Matthias Schiffer
3675e657ed
image: per-device rootfs: don't fail without opkg
Ignore errors caused by /etc/opkg not existing (i.e. when opkg is not
selected).

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2016-09-24 18:41:37 +02:00
Matthias Schiffer
e916579340
image: allow specifying additional packages for device-specific rootfs
Add a new option to each device in multi-profile mode, allowing to provide
a list of packages to add or remove. In case of added packages, the user
must take care that these are selected to be built.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2016-09-24 18:25:25 +02:00
Matthias Schiffer
c15d70c6d6
image: don't override opkg list directory in per-device rootfs mode
opkg's -l option is always interpreted relative to the installation root.
This leads to very weird paths inside the rootfs (containing the whole path
to the LEDE tree on the build machine) and causes the subsequent deletion
of the list directory to fail (cluttering the resulting images).

Instead, use the default list directory and remove its contents in
prepare_rootfs.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2016-09-24 03:37:08 +02:00
Mathias Kresin
14b40d61e1 image: use check-size from new image build code
Use the new image build code and deduplicate the code.

Signed-off-by: Mathias Kresin <dev@kresin.me>
2016-09-13 19:36:41 +02:00
Mathias Kresin
1cd0a4c688 image: add KERNEL_SIZE to the default device vars
This reverts commit ec37a56587 and fixes
the underlying issue.
2016-09-13 19:36:41 +02:00
Felix Fietkau
d27bce8d28 build: drop UBI EOF marker from images by default
Only add them where they are actually required.
Should help with compatibility issues with stock U-Boot images that
access UBI

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-09-12 18:21:52 +02:00
Pushpal Sidhu
4fd043b95b image.mk: Create a manifest file of installed packages as a build artifact
A few linux BSP's create a manifest file of installed packages for a given
target in order to help them understand exactly what's on their images. Create
one here as well as a build artifact since many users have an affinity to
prune down on packages to save valuable flash space.

Signed-off-by: Pushpal Sidhu <psidhu@gateworks.com>
Signed-off-by: Tim Harvey <tharvey@gateworks.com>
2016-09-08 13:40:02 +02:00
Felix Fietkau
a84a74f618 scripts/ubinize-image.sh: add support for adding custom partitions
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-08-31 13:05:19 +02:00
Felix Fietkau
2180b715c1 image: fix per-device rootfs build error when not all opkg package files are found
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-08-04 11:19:41 +02:00
John Crispin
4df2011794 include/image.mk: allow image code to override uImage name
Signed-off-by: John Crispin <john@phrozen.org>
2016-08-03 15:30:13 +02:00
Jo-Philipp Wich
83175687c8 build: remove image specific checksum code
Now that we globally calculate sha256sums over the bin/ directory we can remove
the target image specific checksum handling.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2016-08-01 18:11:26 +02:00
Felix Fietkau
905e50d2fb image: use the merged package directory to resolve dependencies for per-device rootfs
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-08-01 11:13:38 +02:00
Felix Fietkau
baeda5d31d image.mk: add CMDLINE to DEVICE_VARS
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-07-30 14:10:33 +02:00
Felix Fietkau
84718d8736 image: add support for overriding kernel/rootfs images in sysupgrade-tar template
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-07-30 14:10:33 +02:00
Felix Fietkau
c5a9a08f1e image.mk: remove leftover variable from a previous rework
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-07-29 17:13:48 +02:00
Felix Fietkau
76341cfc5f build: add support for per-device rootfs based on device profile packges
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-07-29 10:18:26 +02:00
Felix Fietkau
c0dceae4bb build: minor cleanup of redundant code
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-07-29 10:18:26 +02:00
Felix Fietkau
c792c7512c build: add target_params variable for getting root filesystem image parameters
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-07-29 10:18:26 +02:00
Felix Fietkau
c5ca181d12 image: add wrapper variable to get the target dir for mkfs commands
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-07-29 10:18:26 +02:00
Felix Fietkau
4fed7a60f9 image: make mkfs template output to $@
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-07-29 10:18:26 +02:00
Felix Fietkau
b47f438d98 build: remove image prefix from kernel files in KDIR
This allows the image builder to change EXTRA_IMAGE_NAME at a later
point in time without breaking the build

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-07-25 12:44:30 +02:00
Felix Fietkau
9945a1dca5 build: remove cpio.gz and tar.gz from regular filesystem types
They are not used to produce regular firmware images anyway. Instead,
call their build templates directly if enabled in the config

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-07-25 12:43:39 +02:00
Felix Fietkau
11fc0cd1b1 image.mk: create default ubifs before calling legacy image build code
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-07-22 13:58:58 +02:00
Felix Fietkau
8265fdcc4d imagebuilder: strip DEVICE_ prefix from profiles (FS#55)
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-07-21 13:41:20 +02:00
Felix Fietkau
b5b2425cba image.mk: initialize BOARD_NAME and IMAGES, add it to DEVICE_VARS
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-07-20 10:13:50 +02:00
Felix Fietkau
9a50a213d2 image.mk: add support for specifying the VID header offset for UBI
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-07-20 10:13:50 +02:00
Felix Fietkau
9497a23ecb build: add support for parameter passing to mkfs from devices
Will be used to support different ubifs parameters per device

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-07-20 10:13:50 +02:00
Felix Fietkau
4added6692 build: split legacy image mkfs.ubifs command from the one used for the new image building code
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-07-20 10:13:50 +02:00