While experimenting with the AUTOREMOVE option in search of a way to use
prebuilt host tools in different buildroot, it was discovered that the
md5 generated by find_md5 in depends.mk is not reproducible.
Currently the hash is generated by the path of the file in addition to
the file mod time. Out of confusion, probably, there was an idea that
such command was used on the package build_dir. Reality is that this
command is run on the package files. (Makefile, patches, src)
This is problematic because the package Makefile (for example) change at
each git clone and base the hash on the Makefile mtime doesn't really
reflect if the Makefile actually changes across a buildroot or not.
A better approach is to generate an hash of each file and then generate
an hash on the sort hash list. This way we remove the problem of git
clone setting a wrong mtime while keeping the integrity of checking if a
file changed for the package as any change will result in a different
hash.
Introduce a new kind of find_md5 function, find_md5_reproducible that
apply this new logic and limit it only with AUTOREMOVE option set to
prevent any kind of slowdown due to additional hash generation.
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
Some OS may have the true bin to a different location than /bin/true.
BSD based system and macos have true on /usr/bin/true.
Fix this by checking both location and take the one available in the
system.
Reported-by: Sergey V. Lobanov <sergey@lobanov.in>
Suggested-by: Huangbin Zhan <zhanhb88@gmail.com>
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
When building UML this showed up as of late (2022-09-01):
| Makefile:1335: *** Headers not exportable for UML. Stop.
This message popped up because of an upstream patch that
fixed this error not showing up sooner. For more information see:
<https://lore.kernel.org/lkml/20220901011252.550830-1-masahiroy@kernel.org/T/>
Now, I don't think we can just skip the kernel headers.
Some userspace applications are going to need these. So,
The next best alternative I can think of, is to go with
the headers by the host arch the UML is compiled for.
(Technically, ARCH= should be safe for all other targets
as well... But let's not poke a hornet's nest)
Signed-off-by: Christian Lamparter <chunkeey@gmail.com>
One way to solve the python3 dependency check is to install
python3-minimal instead of python3 on Debian based systems.
Unfortunately, this results in a fairly unusable python.
Added check for ntpath, which is how the issue originally presented
itself.
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Debian and by extension Ubuntu packages distutils in a suboptimal way
where import distutils works but none of the methods do.
This alternative check verifies that distutils is actually usable.
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Package with whitespace in their build directory are not correctly
removed when CONFIG_AUTOREMOVE is enabled. This is caused by xargs that
use whitespace as delimiters. To handle this use \0 as the delimiter and
set find to use \0 as the delimiter.
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
Just like later in the file,
the precursor to <stampfile>_check
<stampfile>_check.1 is supposed to be moved to
<stampfile>_check before it is touched.
This line would error if it was ever run.
Signed-off-by: Michael Pratt <mcpratt@pm.me>
Building images usally stores them in KDIR_TMP and then copies them over
to BIN_DIR. This is fine as rebuilding of images overwrites existing
images. When using the EXTRA_IMAGE_NAME variable frequently this fills
up the ImageBuilder KDIR_TMP folder since every built image is stored
forever.
This commit clears the KDIR_TMP folder before building a new image.
Below an example how sysupgrade.openwrt.org filled up after the release
of 22.03.0 where every created image contains a hash of the package
selction in the filename:
aparcar@asu-01:~/asu/worker1/cache/22.03.0$ du -d 1 -h
400M ./kirkwood
260M ./gemini
2.0G ./ipq806x
1.7G ./ipq40xx
8.1G ./ramips
4.0K ./octeon
495M ./sunxi
728M ./lantiq
1.8G ./rockchip
3.7G ./mediatek
4.0K ./realtek
5.4G ./mvebu
8.9G ./ath79
3.0G ./bcm47xx
14G ./bcm27xx
11G ./x86
4.0K ./bcm63xx
312M ./mpc85xx
600M ./apm821xx
5.4G ./bcm53xx
66G .
Signed-off-by: Paul Spooren <mail@aparcar.org>
Dependency on xxd was added in commit c4dd2441e7 ("tools: add xxd
(from vim)") as U-Boot requires xxd to create the default environment
from an external file.
Later in commit 2b94aac7a1 ("tools: xxd: use more convenient source
tarball"), xxd from another source was used instead, but that source is
currently unavailable, so let's fix it by using simple xxdi.pl Perl
script instead.
Fixes: #10555
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Instead of relying on dtc being provided by the build host use the
dtc from $(LINUX_DIR) similar to how it's done also in u-boot.mk.
For this to work kernel.mk now needs to be included before
trusted-firmware-a.mk, add this include to all affected packages.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Removed following upstreamed patch:
* bcm53xx: 081-next-ARM_dts_BCM53015-add-mr26.patch
All other patches automagically rebased.
Signed-off-by: Petr Štetiar <ynezz@true.cz>
python3.11 beta is out but fails to run the makefile currently
this supports python versions from 3.6 to 3.99 with the python3 binary
it also checks specifically for 3.11 as it is the latest version out
Signed-off-by: Oscar Molnar <oscar@tymscar.com>
The ulog iptables target was removed with kernel 3.17, remove the kernel
and also the iptables package in OpenWrt too.
Signed-off-by: Hauke Mehrtens <hauke@hauke-m.de>
The nft NAT packages for IPv4 and IPv6 were merged into the common
packages with kernel 5.1. The kmod-nft-nat6 package was empty in our
build, remove it.
Multiple kernel configuration options were also removed, remove them
from our generic kernel configuration too.
Signed-off-by: Hauke Mehrtens <hauke@hauke-m.de>
Netgear encrypted image is used in various devices including WAX202,
WAX206, and EX6400v3. This image format also requires a dummy squashfs4
image which is added here as well.
References in WAX202 GPL source:
https://www.downloads.netgear.com/files/GPL/WAX202_V1.0.5.1_Source.rar
* openwrt/bootloader/u-boot-mt7621-2018.09-gitb178829-20200526/board/ralink/common/dual_image.c
Bootloader code that verifies the presence of a squashfs4 image, thus
a dummy image is added here.
* openwrt/tools/imgencoder/src/gj_enc.c
Contains code that generates the encrypted image. There is support for
adding an RSA signature, but it does not look like the signature is
verified by the stock firmware or bootloader.
* openwrt/tools/imgencoder/src/imagekey.h
Contains the encryption key and IV. It appears the same key/IV is used
for other Netgear devices including WAX206 and EX6400v3.
Signed-off-by: Wenli Looi <wlooi@ucalgary.ca>
gettext (libintl-stub) was removed in commit [1], so the libintl-stub
lib and include directories aren't existing anymore. This commit cleans
up the INTL flags for the BUILD_NLS=n case.
[1] e6f569406f
Signed-off-by: Sebastian Kemper <sebastian_ml@gmx.net>
Reviewed-by: Rosen Penev <rosenp@gmail.com>
Manual rebase by Marty Jones:
bcm27xx/patches-5.15/950-0078-BCM2708-Add-core-Device-Tree-support.patch
All other patches automatically rebased.
Signed-off-by: John Audia <therealgraysky@proton.me>
Signed-off-by: Marty Jones <mj8263788@gmail.com>
[Apply same changes to new dts entry in modified file]
Signed-off-by: Hauke Mehrtens <hauke@hauke-m.de>
The ImageBuilder does not need git or rsync since it only glues files
together, packages are downloaded via wget and not rsync.
Signed-off-by: Paul Spooren <mail@aparcar.org>
[ solve conflict with additional git prereq test ]
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
This avoids a dangling symlink to self when reinstalling the toolchain:
./staging_dir/toolchain-mips_24kc_gcc-9.3.0_musl/lib/lib -> lib
This is caused by the fact that in the toolchain dir we have
'lib64 -> lib'
and on executing
'ln lib ./staging_dir/toolchain-mips_24kc_gcc-9.3.0_musl/lib64'
ln dereference the symbolic link 'lib64' to 'lib' so the REAL command is
'ln lib ./staging_dir/toolchain-mips_24kc_gcc-9.3.0_musl/lib'
this results in the dangling symlink to self.
Signed-off-by: Sven Wegener <sven.wegener@stealer.net>
[ add more details to the commit description and fix title ]
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
No patches rebased, just checksum update for this refresh.
Build system: x86_64
Build-tested: ipq806x/R7800
Signed-off-by: John Audia <therealgraysky@proton.me>
No patches required a rebase, just updated checksum.
Build system: x86_64
Build-tested: ipq806x/R7800
Signed-off-by: John Audia <therealgraysky@proton.me>
This updates prereq-build.mk to find a suitable realpath utility, and
adds another place to look for a suitable getopt utility.
realpath has been used most notably by scripts/ipkg-build since
commit bb95be9265 ("scripts,ipkg-build: use realpath for pkg_dir")
and there are assorted other uses of it during a build.
It is ordinarily provided by GNU coreutils. This adds a SetupHostCommand
to locate it either under its own name or under grealpath, the name that
it will be available as under MacPorts or Homebrew, which use
--program-prefix=g.
Similarly, update the SetupHostCommand for getopt to be able to locate a
util-linux getopt at the default path used by MacPorts, in the same
fashion that
commit cc16f5d73e ("build: try to find getopt in macOS homebrew's standard location")'
did for Homebrew. As there is no standard alternative --program-prefix
for util-linux utilities in the way that GNU packages often use a "g"
prefix, this path-based approach is required in case a non-util-linux
getopt (such as one provided by an OS) shadows the util-linux getopt
in the PATH.
Signed-off-by: Mark Mentovai <mark@moxienet.com>
Compiler option -no-plt will break kernel builds on some architectures
eg. (x86) Filter this option from the recently introduced handling of
KCFLAGS vs EXTRA_OPTIMISATION
Fixes: 1d42af720c ("kernel: use KCFLAGS for passing EXTRA_OPTIMIZATION flags")
Suggested-by: Felix Fietkau <nbd@nbd.name>
Signed-off-by: Kevin Darbyshire-Bryant <ldir@darbyshire-bryant.me.uk>
This uses kernel's generic variable and doesn't require patching it with
a custom Makefile change. It's expected *not* to change any behaviour.
Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
With some OS (Guix) the git submodule command is wrapped in
a script. Current logic parse the git submodule script directly.
If it's wrapped the prereq check wrongly fails while 'git submodule
--recursive' is actually available.
Add an additional check that try to directly use the 'git submodule'
command to check if the prereq is satisfied.
Fixes: #9986
Reported-by: Attila Lendvai <attila@lendvai.name>
Suggested-by: Attila Lendvai <attila@lendvai.name>
Signed-off-by: Christian 'Ansuel' Marangi <ansuelsmth@gmail.com>
Delete the crypto-lib-blake2s kmod package, as BLAKE2s is now built-in.
Patches automatically rebased.
Signed-off-by: Rui Salvaterra <rsalvaterra@gmail.com>
Delete the crypto-lib-blake2s kmod package, as BLAKE2s is now built-in.
Patches automatically rebased.
Build system: x86_64
Build-tested: ipq806x/R7800, x86/64
Signed-off-by: John Audia <therealgraysky@proton.me>
If you change SCAN_EXTRA variable with "-path target/linux/xxxx" in
include/toplevel.mk for speed up scan, find will warn with:
find: warning: you have specified the global option -maxdepth after
the argument -path, but global options are not positional, i.e.,
-maxdepth affects tests specified before it as well as those specified
after it. Please specify global options before other arguments.
The find option -mindepth -maxdepth are global options and must be
before any path option. Change order of $(SCAN_EXTRA) after -mindepth
and -maxdepth to fix this.
Signed-off-by: Leo Chung <gewalalb@gmail.com>
[capitalize Description, Author and Sob and minor description tweak]
Signed-off-by: Christian 'Ansuel' Marangi <ansuelsmth@gmail.com>
According to the GNU make manual, specifying library paths should be
done in LDFLAGS rather than LDLIBS. Replace KBUILD_HOSTLDLIBS with
KBUILD_HOSTLDFLAGS to pass the host lib directory.
Signed-off-by: Stijn Tintel <stijn@linux-ipv6.be>
HOST_LOADLIBES was renamed to KBUILD_HOSTLDLIBS in kernel 4.19. As the
oldest kernel version we support is 5.10, cleanup HOST_LOADLIBES use.
Signed-off-by: Stijn Tintel <stijn@linux-ipv6.be>
Image metadata and signature is of no use for images which are included
inside other artifacts (like an SD-card image). Strip them off before
using images in artifacts or stashing them for the ImageBuilder as the
contained signature breaks reproducibility.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
