Initramfs images were using a common rootfs (TARGET_DIR) for all
devices, ignoring TARGET_PER_DEVICE_ROOTFS. If a single device required
a package to build a functional initramfs image, it should be included
by default for all devices or that device should be isolated into a new
subtarget. Now the initramfs will be built using the target-specific
Implementing Per Device Rootfs for Initramfs is not trivial as the
rootfs needs to be embedded in the kernel image. The kernel supports an
option to define the initramfs location and the image generation for the
kernel can't be run in parallel as other checks are done to config and
other arch dependent files.
To handle this, we prepare a config for each rootfs and we generate the
images under lock to prevent problem with parallel execution.
Signed-off-by: Luiz Angelo Daros de Luca <luizluca@gmail.com>
[ rework implementation for locking support ]
Link: https://github.com/openwrt/openwrt/pull/12959
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
Simplify SetInitramfs compression ALGO config setup by using Makefile
foreach.
While at it also make it more readable.
Link: https://github.com/openwrt/openwrt/pull/12959
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
Rework SetInitramfs functions to take a second arg to define the
location of the .config. This is needed in preparation for PER_ROOTFS
Initramfs support as we will prepare .config in dedicated directory and
use them only later when the image is actually built.
Link: https://github.com/openwrt/openwrt/pull/12959
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
Allow Kernel/CompileImage/Initramfs to use a different rootfs location.
If the additional arg is not defined, TARGET_DIR is used by default.
This allows the caller to customize the kernel initramfs for different
rootfs.
Signed-off-by: Luiz Angelo Daros de Luca <luizluca@gmail.com>
[ simplify commit and rework commit description ]
Link: https://github.com/openwrt/openwrt/pull/12959
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
Kernel module packages compiling is not cached (e.g. mac80211)
even with CONFIG_CCACHE on.
CC should be set to KERNEL_CC in KERNEL_MAKE_FLAGS at kernel.mk
to allow kernel module packages using ccache.
Signed-off-by: Zeyu Dong <dzy201415@gmail.com>
New kernel version 6.1 introduced new INITRAMFS option. Add them to the
Initramfs functions to correctly compile initramfs images.
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
Commit 54070a1 was added to allow building proper SDKs with kernels <
5.10. Now that all targets use at least kernel 5.10 it can be reverted.
Signed-off-by: Sebastian Kemper <sebastian_ml@gmx.net>
Requires: tools/libdeflate
fix consistency of executable to use
`$(STAGING_DIR_HOST)/bin/libdeflate-gzip`, and not system-installed ones
from the usual environment `PATH`;
this affects option `CONFIG_KERNEL_INITRAMFS_COMPRESSION_GZIP`
this may have worked in the past but only via side effect of having the
binaries on the host system (and whatever unpredictable version or
patchset those might be), and did not use the improved but totally
compatible libdeflate-gzip
Fixes: 330bd380e8 ("image: allow building FIT and uImage with ramdisk")
Signed-off-by: Tony Butler <spudz76@gmail.com>
Requires: tools/bzip2
fix consistency of executable to use `$(STAGING_DIR_HOST)/bin/bzip2`, and
not system-installed ones from the usual environment `PATH`;
this affects option `CONFIG_KERNEL_INITRAMFS_COMPRESSION_BZIP2`
this may have worked in the past but only via side effect of having the
binaries on the host system (and whatever unpredictable version or
patchset those might be)
Fixes: 330bd380e8 ("image: allow building FIT and uImage with ramdisk")
Signed-off-by: Tony Butler <spudz76@gmail.com>
libdeflate decompresses much faster than gzip.
Example:
~/d/openwrt> time gzip -dc dl/cmake-3.25.1.tar.gz > /dev/null
________________________________________________________
Executed in 1.01 secs fish external
usr time 912.61 millis 1.67 millis 910.94 millis
sys time 32.21 millis 0.25 millis 31.96 millis
~/d/openwrt> time libdeflate-gzip -dc dl/cmake-3.25.1.tar.gz > /dev/null
________________________________________________________
Executed in 523.04 millis fish external
usr time 415.48 millis 1.07 millis 414.41 millis
sys time 107.74 millis 0.15 millis 107.59 millis
~/d/openwrt> time gzip -dc dl/tessdata-4.1.0.tar.gz > /dev/null
________________________________________________________
Executed in 8.99 secs fish external
usr time 8.90 secs 530.00 micros 8.90 secs
sys time 0.07 secs 63.00 micros 0.07 secs
~/d/openwrt> time libdeflate-gzip -dc dl/tessdata-4.1.0.tar.gz > /dev/null
________________________________________________________
Executed in 2.74 secs fish external
usr time 2.38 secs 537.00 micros 2.38 secs
sys time 0.35 secs 66.00 micros 0.35 secs
Place libdeflate into tools-core as it is needed to decompress other
archives.
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Requires: tools/lz4, tools/lzop
complete the wiring so that these options work:
* `CONFIG_KERNEL_INITRAMFS_COMPRESSION_LZO`
* `CONFIG_KERNEL_INITRAMFS_COMPRESSION_LZ4`
Signed-off-by: Tony Butler <spudz76@gmail.com>
[remove blocking dependencies for separate ramdisk, fix lzop options]
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
When building UML this showed up as of late (2022-09-01):
| Makefile:1335: *** Headers not exportable for UML. Stop.
This message popped up because of an upstream patch that
fixed this error not showing up sooner. For more information see:
<https://lore.kernel.org/lkml/20220901011252.550830-1-masahiroy@kernel.org/T/>
Now, I don't think we can just skip the kernel headers.
Some userspace applications are going to need these. So,
The next best alternative I can think of, is to go with
the headers by the host arch the UML is compiled for.
(Technically, ARCH= should be safe for all other targets
as well... But let's not poke a hornet's nest)
Signed-off-by: Christian Lamparter <chunkeey@gmail.com>
Make sure the timestamp of the root directory of the initramfs is set
to SOURCE_DATE_EPOCH as well.
Fixes: 29d7461d11 ("kernel: set options to make external initramfs reproducible")
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Make sure xz uses at least 2 threads so compression always runs in
multi-threaded mode as the resulting file in single-threaded mode
differs.
Fixes: 29d7461d11 ("kernel: set options to make external initramfs reproducible")
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
CONFIG_EXTERNAL_CPIO is a string variable, hence testing for 'y'
doesn't make much sense here.
Fixes: 330bd380e8 ("image: allow building FIT and uImage with ramdisk")
Reported-by: Huangbin Zhan <zhanhb88@gmail.com>
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
The $(LINUX_DIR)/.config timesptamp changes between runs of
make target/compile and make target/install (which builds the image).
Kernel-dependent packages and out of tree modules are built in between
those runs, and they check the .config timestamp to decide if they need
to be rebuilt.
Save the target/compile .config to use its timestamp if the file does
not change between runs. That way the subsequent kernel packages are
not unnecessarily rebuilt when you run 'make' back to back.
Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
This fixes a regression introduced with commit
5ed1e5140a ("build: build kernel image
before building modules/packages").
Before this commit the make target would always include "modules",
resulting in a MODPOST and a complete Module.symvers file. Since this
commit a MODPOST of the kernel modules is not guaranteed for kernels <
5.10. This results in some broken SDKs in which external packages that
depend on exported symbols from kernel modules fail to compile.
Adding "modules" back to the calls to the CompileImage defines fixes the
regression. For kernels > 5.10 this is not needed, but it doesn't cause
any harm either.
Tested with kernels 5.4.x and 5.10.x.
Signed-off-by: Sebastian Kemper <sebastian_ml@gmx.net>
Use an 'if' so the absence of $(LINUX_DIR)/user_headers doesn't make the
line evaluate to false and cause the build to fail.
Signed-off-by: Matt Merhar <mattmerhar@protonmail.com>
Before this commit, it was assumed that mkhash is in the PATH. While
this was fine for the normal build workflow, this led to some issues if
make TOPDIR="$(pwd)" -C "$pkgdir" compile
was called manually. In most of the cases, I just saw warnings like this:
make: Entering directory '/home/.../package/gluon-status-page'
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
[...]
While these were only warnings and the package still compiled sucessfully,
I also observed that some package even fail to build because of this.
After applying this commit, the variable $(MKHASH) is introduced. This
variable points to $(STAGING_DIR_HOST)/bin/mkhash, which is always the
correct path.
Signed-off-by: Leonardo Mörlein <me@irrelefant.net>
Instead of embedding the initrd cpio archive into the kernel, allow
for having an external ramdisk added to the FIT or uImage.
This is useful to overcome kernel size limitations present in many
stock bootloaders, as the ramdisk is then loaded seperately and doesn't
add to the kernel size. Hence we can have larger ramdisks to host ie.
installers with all binaries to flash included (or a web-based
firmware selector).
In terms of performance and total size the differences are neglectible.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
The license folder is a core part of OpenWrt and all GPL-2.0 licensed.
Use SPDX license tags to allow machines to check licenses.
Signed-off-by: Paul Spooren <mail@aparcar.org>
[rebase, keep some Copyright lines, sharpen commit message]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
Linux 5.9 introduces support for ZSTD ramdisk and initrd compression,
make sure we enable/disable the relevant options when building an
initramfs enabled kernel.
Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
When we use an external kernel tree which may not have been fully
cleaned, there may be user_headers left which do not match the target
architecture, leading to build failures for packages that do an explicit
inclusion of user_headers (such as iproute2 or iptables). Make sure we
delete them while preparing the directory.
Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
On non-GNU systems, zcat often does not handle gzip decompression.
Use gzip -dc like the regular unpack command
Signed-off-by: Felix Fietkau <nbd@nbd.name>
This is a follow up of 8cb13f4e6d which sets the right timestamp for the
/init file in initramfs. The previous patch doesn't cover it as the
files appear to come from a later step during compilation.
CC: Alexander Couzens <lynxis@fe80.eu>
Signed-off-by: Paul Spooren <mail@aparcar.org>
They are no longer stored in the "testing" subdirectory and are not
available as .tar.xz archives. If -rc is detected download it from the
git.kernel.org and use .tar.gz.
Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
In commit fce35bce0f ("config: support new symbol intro'd in kernel
4.12")
I forgot to remove the initial debug test line.
This clearly is wrong as the same symbol is defined conditionally in the
line below as it should be.
I looked over it as I just checked if the symbol was present now upon
testing it.
Fixes: fce35bce0f ("config: support new symbol intro'd in kernel
4.12")
Signed-off-by: Koen Vandeputte <koen.vandeputte@ncentric.com>
Symbol CONFIG_INITRAMFS_FORCE allows to ignore the value passed by the
bootloader.
By default, all symbols containing INITRAMFS are wiped from the final
config and then re-added conditionally.
Add support for this symbol, as the build will stop otherwise
questioning the user about this option:
* Restart config...
*
*
* General setup
*
Cross-compiler tool prefix (CROSS_COMPILE) []
Compile also drivers which will not load (COMPILE_TEST) [N/y/?] n
...
Initial RAM filesystem and RAM disk (initramfs/initrd) support
(BLK_DEV_INITRD) [Y/n/?] y
Initramfs source file(s) (INITRAMFS_SOURCE) []
Ignore the initramfs passed by the bootloader (INITRAMFS_FORCE)
[N/y/?] (NEW)
Signed-off-by: Koen Vandeputte <koen.vandeputte@ncentric.com>
This allows packages to use kernel make options without the forced
-C $(LINUX_DIR). It also makes it more clear that it to be called from
kernel module packages directly.
Signed-off-by: Felix Fietkau <nbd@nbd.name>
In case we use external and/or git cloned kernels, let the kernel
determine the appropriate KERNELRELEASE. We cannot used
LINUX_UNAME_VERSION because that one gets determined at a later time,
when the kernel is already built proper.
Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
This reverts commit 0df2c6563a since it
gets in the way of identifying properly which kernel we are running.
This is particularly important if LEDE is using external kernels/git
cloned kernels. We want to make sure we only load modules from that
specific kernel.
Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
Keeping it in base-files was resulting in adding it to the base-files
package. This file is meant to be included manually for initramfs
images only.
Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
KERNELRELEASE contains a $(shell) call which is evaluated over and over
again.
The call to checksyscalls.sh is unnecessary for LEDE and also takes a
few seconds to complete.
Signed-off-by: Felix Fietkau <nbd@nbd.name>
Utilize the existing git download logic from include/download.mk and migrate
the kernel download over to it. This avoids repeatedly cloning kernel sources
after a make target/linux/clean for instance.
Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
Signed-off-by: Felix Fietkau <nbd@nbd.name> [fix build error]
When the kernel build picks up a localversion file in the source tree,
that string is unconditionally appended to LOCALVERSION and affects the
uname string.
Make sure to delete any such file.
Signed-off-by: Josua Mayer <josua.mayer97@gmail.com>
When building the kernel from a git repository, the kernel build appends
either a + or a short commit hash to localversion.
This behaviour can be prevented by passing the empty LOCALVERSION variable
to make.
Signed-off-by: Josua Mayer <josua.mayer97@gmail.com>
