Commit Graph

289 Commits

Author SHA1 Message Date
Michael Pratt
fe33d5ba13
tools: add gnulib source
By having a local copy of gnulib, we can:
import the latest macro fixes into any package,
get rid of some statically stored macros that were otherwise missing,
bootstrap GNU tools with the latest relevant source
without having to wait for a release or rely on git submodules,
and possibly more...

The patch assists in bootstrapping by ignoring
the building of po files using gettext,
and also to allow a user-defined path to a program
to include parameters.

Signed-off-by: Michael Pratt <mcpratt@pm.me>
2023-05-04 06:07:27 +02:00
Rosen Penev
291b137778
tools/lz4: build with meson
Simplifies Makefile and speeds up compilation.

Before:
Executed in   68.00 secs    fish           external
   usr time  104.17 secs    1.33 millis  104.17 secs
   sys time    4.29 secs    0.35 millis    4.29 secs

After:
Executed in   27.98 secs    fish           external
   usr time   25.18 secs    1.32 millis   25.18 secs
   sys time    3.31 secs    0.35 millis    3.31 secs

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2023-04-20 14:21:46 +02:00
Tony Butler
36150ff6ff
tools/bzip2: add bzip2 binaries
`bzip2` is the standard executable for bzip2 compression

this includes development includes and both static and shared libs
(libbz2) which can be used by other packages

the initramfs generator offers the BZIP2 option but there was no
executable to support it, and worked only via side effect of having a
system-installed version of bzip2, which could be less predictable

Signed-off-by: Tony Butler <spudz76@gmail.com>
[ remove unintended change ]
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-04-18 15:25:21 +02:00
Christian Marangi
b691362d1d
Revert "tools/bzip2: add bzip2 binaries"
This reverts commit 394d7134ec.

The commit has unintentded change that cause compilation error with SDK
or LZO compression.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-04-18 15:22:48 +02:00
Tony Butler
394d7134ec
tools/bzip2: add bzip2 binaries
`bzip2` is the standard executable for bzip2 compression

this includes development includes and both static and shared libs
(libbz2) which can be used by other packages

the initramfs generator offers the BZIP2 option but there was no
executable to support it, and worked only via side effect of having a
system-installed version of bzip2, which could be less predictable

Signed-off-by: Tony Butler <spudz76@gmail.com>
2023-04-18 13:57:47 +02:00
Christian Marangi
30f2d516ba
tools/squashfs4: add new tool for squashfs4 images
squashfs tool is finally reborn and correctly maintained.

Introduce the new version as a replacement for squasfs4kit as it was a
fork and also abandoned.

Add additional patch to add the missing feature present in squashfskit4
but still missing on this new project.

Backport each required patch that fix compilation error on macos.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-02-20 07:34:03 +01:00
Christian Marangi
01262c921c
tools/squashfs: rename to squashfs3-lzma
The name of squashfs is confusing since in reality it's a really old
version using an old lzma library. This tools is used for old ath79
netgear target and to produde a fake squasfs3 image needed for some
specific bootloader from some OEM (AVM for example)

Rename squashfs tool to squasfs3-lzma to better describe it.
Rename the installed bin from mksquashfs-lzma to mksquashfs3-lzma.
Use tar transform to migrate the root directory in tar to the new
naming.
Drop redundant PKG_CAT variable not needed anymore.
Also update any user of this tool.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-02-18 21:11:36 +01:00
Olliver Schinagl
62e150949f
libdeflate: Avoid circular dependencies
CMake depends on (libdeflate-)gunzip, libdeflate depends on Cmake, so we
can't win.

Luckily libdeflate is _very_ easy to build, without any build system, so
lets just manually compile it and be done with it.

Signed-off-by: Olliver Schinagl <oliver@schinagl.nl>
Signed-off-by: Rosen Penev <rosenp@gmail.com>
2023-01-17 21:01:54 +01:00
Olliver Schinagl
92feae64eb
tools: Improve diffability/maintainability
Like with commit ae614fb397 ("tools: Improve diffability/maintainability")
we also want tools-core to be easy to maintain. While a smaller target,
it's still usefull and makes things nice and consistent.

To avoid duplicating any tools in the comment, simplify the comment
instead.

Signed-off-by: Olliver Schinagl <oliver@schinagl.nl>
2023-01-17 20:54:05 +01:00
Tony Butler
32e99ed069 tools/lzop: add lzop binaries
Depends: `tools/liblzo`

`lzop` is the standard executable for LZO compression

the initramfs generator offers the LZO option but there was no
executable to support it actually working

Signed-off-by: Tony Butler <spudz76@gmail.com>
2023-01-16 23:51:10 +00:00
Tony Butler
9a8509c2c5 tools/liblzo: add liblzo library
prerequisite of upcoming `tools/lzop` addition, and subsequent initramfs
and squashfs cleanups

same as `packages/lzo` modified to be a HOST/tools type build,
and should always be the same version and sources when either one is
bumped

because this (and `packages/lzo`) only provide liblzo and no
executables, use the clearer name `tools/liblzo`

Signed-off-by: Tony Butler <spudz76@gmail.com>
2023-01-16 23:51:10 +00:00
Tony Butler
8b12966d20 tools/lz4: add lz4 binaries
same as `packages/liblz4` modified to be a HOST/tools type build with
unified liblz4 (this is also the dev package for liblz4)

the image initramfs generator offers the LZ4 option but there was no
executable to support it actually working

Signed-off-by: Tony Butler <spudz76@gmail.com>
2023-01-16 23:51:10 +00:00
Markus Stockhausen
a51ca085bf tools: Add gzip-libdeflate advanced compressor
Several devices provide U-Boot versions with only gzip compressed kernel
support (e.g. Realtek switches). This compression method produces larger
images than lzma. To save space on flash and avoid going the hard way with
lzma-loader we can make use of enhanced gzip tool based on libdeflate
compression library from https://github.com/ebiggers/libdeflate. It
keeps 100% deflate/gzip compatibility while improving compression ratio.
The image can be unpacked by the default inflate routines inside U-Boot.

Signed-off-by: Markus Stockhausen <markus.stockhausen@gmx.de>
[Switched to v1.15 and made it work with cmake]
Signed-off-by: Olliver Schinagl <oliver@schinagl.nl>
Reviewed-by: Robert Marko <robimarko@gmail.com>
Reviewed-by: Rosen Penev <rosenp@gmail.com>
Reviewed-by: Sander Vanheule <sander@svanheule.net>
2023-01-02 10:15:37 +01:00
Olliver Schinagl
ae614fb397
tools: Improve diffability/maintainability
There's no purpose to squish multiple tools into a single line (and
spread those out over multiple lines). It might look 'nice' in certain
conditions, but it's annoying to maintain.

For example, but not limited to:
* adding/removing tools, causes hard to read diffs
* Duplicates are harder to spot
* Sorting can not be (easily?) automated

With this proposed change, the above annoyances go away. Inserting a new
tool can be done with a single line-change-diff, sorting can be done by
any editor (in vi, select, :sort for example) and dupes are much easier
to spot.

Signed-off-by: Olliver Schinagl <oliver@schinagl.nl>
2022-12-24 11:54:10 +01:00
Felix Fietkau
a06e023b4e mac80211: remove old legacy legacy drivers
Get rid of drivers that are either limited to 802.11b/g or don't even support
cfg80211/mac80211. Most of these are either limited to boards that we don't even
support anymore because of firmware size, or were only used for custom hacks by
a really small number of users in the past.
Let's get rid of those to reduce the maintenance effort and the number of useless
packages

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2022-11-09 20:44:28 +01:00
Hauke Mehrtens
2c603ecb55 tools/genext2fs: Add dependency to libtool
This fixes the following build error:
  ./autogen.sh: line 13: aclocal: command not found

Fixes: b6d29af947 ("tools/genext2fs: update to 1.5.0")
Signed-off-by: Hauke Mehrtens <hauke@hauke-m.de>
2022-10-23 01:45:52 +02:00
Rosen Penev
3268f7b9fa
tools: remove implicit dependencies
Small cleanup.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2022-10-20 00:33:25 +02:00
Christian Marangi
ebabdff401
tools: add option BUILD_ALL_HOST_TOOLS to compile all host tools
Add option to compile all host tools even if not needed.
This can be useful to prepare a universal precompiled host tools
archive to use in another buildroot and speedup compilation.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-09-30 22:26:54 +02:00
Rosen Penev
3848cf458e
tools/cmake: Build without some included libs
Saves a little bit of time when compiling cmake.

Added patches to fix searching liblzma and zlib. The issue is that
because pkgconfig is not used, the system libraries get used.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2022-09-29 19:33:17 +02:00
Rosen Penev
89df3589e6
tools/expat: build with autotools again
Allows to set expat as a dependency to cmake and save on compilation
time.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2022-09-29 19:33:17 +02:00
Rosen Penev
d602e7a969
tools/zlib: switch to configure script
A future commit will make tools/cmake use this.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2022-09-29 19:33:13 +02:00
Rosen Penev
b71affaf8b tools: fix firmware-utils depends
When firmware-utils was converted to use cmake, the dependency was not
updated.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2022-09-27 15:35:41 -07:00
Petr Štetiar
88c9056a70 tools: remove xxd package
It shouldn't be needed anymore as we've now `scripts/xxdi.pl`, which
should be self contained and fully compatible `xxd -i` replacement.

Fixes: #10555
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2022-09-06 08:04:53 +02:00
Jan Hoffmann
5fcc6f0f19 tools: add 7z host package
Add the 7zr command line tool, which is a version of the 7z application
that only supports 7z archives.

7z is one of the two compression formats supported in H3C firmware
images (the alternative would be ARJ).

(Alternatively, the 7zr command line tool could also be built from a
current version of the public-domain LZMA SDK. That would require
repackaging the source package, as it is only provided in 7z format.)

Signed-off-by: Jan Hoffmann <jan@3e8.eu>
2022-07-28 14:08:56 +02:00
Stijn Tintel
16e9ccd5fa tools/elfutils: depend on m4
Some buildbots fail to build elfutils due to m4 being missing. Add m4 as
a dependency for elfutils to fix this.

Fixes: ad79b92719 ("elfutils: move host build to tools")
Signed-off-by: Stijn Tintel <stijn@linux-ipv6.be>
2022-05-18 16:17:35 +03:00
Tony Ambardar
02850d7c9c tools/dwarves: add host package
dwarves is a set of tools that use the debugging information inserted in
ELF binaries by compilers such as GCC. Utilities in the dwarves suite
include pahole, which can be used to find alignment holes in structs and
classes, and also extracts other information such as CPU cacheline
alignment, helping pack those structures to achieve more cache hits.

These tools are also used to encode and read the BTF type information
format used with the bpf syscall, making this a Linux build dependency
when using kernel BTF information.

Signed-off-by: Tony Ambardar <itugrok@yahoo.com>
Signed-off-by: Felix Fietkau <nbd@nbd.name>
[bump to 1.23, add elfutils dep, drop host lib usage, drop cmake release
target, use RM macro]
Signed-off-by: Stijn Tintel <stijn@linux-ipv6.be>
2022-05-18 13:32:06 +03:00
Stijn Tintel
ad79b92719 elfutils: move host build to tools
The upcoming dwarves host package requires elfutils. As dependencies for
tools must exist in tools, we need to move elfutils host build there.

As there is at least one package that depends on this, and there is no
proper way to create such dependency in the build system, build it
unconditionally when not building on macOS.

Signed-off-by: Stijn Tintel <stijn@linux-ipv6.be>
2022-05-18 13:32:06 +03:00
Huangbin Zhan
4a19cf3bc7 tools/mkimage: update to 2022.01
- enable dot config
- enable openwrt verbose
- add bison as dependency to avoid failure
```
  bison -oscripts/kconfig/zconf.tab.c -t -l scripts/kconfig/zconf.y
bison: /builder/shared-workdir/build/staging_dir/host/share/bison/m4sugar/m4sugar.m4: cannot open: No such file or directory
```

Signed-off-by: Huangbin Zhan <zhanhb88@gmail.com>
2022-03-01 00:08:08 +01:00
Rosen Penev
4e13229dd1 tools/expat: update to 2.4.6
Switched to CMake for faster compilation and greater parallel
friendliness.

Added CMake options from the packages feed.

This release fixes various CVEs.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2022-03-01 00:08:08 +01:00
Rosen Penev
94dd68ff73 tools/zstd: update to 1.5.2
Switched to building with meson as it's faster and does not need a
dependency on cmake, which takes a long time to build.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2022-03-01 00:08:08 +01:00
Rosen Penev
2d5f03205a tools/ccache: add cmake dependency
This will be needed for the next commit as ccache's cmake dependency is
satisfied by zstd currenly.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2022-03-01 00:08:08 +01:00
Jo-Philipp Wich
af79853c73 Revert "tools/zstd: update to 1.5.2"
This reverts commit 8de901ccf7.

Apparently this update breaks tools building.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2022-02-07 09:03:19 +01:00
Rosen Penev
8de901ccf7 tools/zstd: update to 1.5.2
Switched to building with meson as it's faster and does not need a
dependency on cmake, which takes a long time to build.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2022-02-07 00:03:27 +01:00
Felix Fietkau
3869ccbcc8 tools: build bash on macOS and use it for ipkg-build
On macOS, system binaries silently drop the environment variables for injecting
extra shared libraries (used by fakeroot). This is done for security reasons.
Work around this by building bash from source, so that it gets an ad-hoc signature
and does not have these restrictions

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2022-01-27 13:38:48 +01:00
Felix Fietkau
340c2ed2ef tools/llvm-bpf: move tarball packing to target/llvm-bpf
This ensures that the tarball is regenerated after make clean or after switching
to a different target

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2021-11-22 12:00:40 +01:00
Felix Fietkau
848499c1cf tools: include the value of CONFIG_SDK_LLVM_BPF in the stampfile
tools/llvm-bpf needs to be checked if the value changes

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2021-11-21 13:47:18 +01:00
Felix Fietkau
10cbce3b52 tools: ensure that BIN_DIR is created early
Fixes llvm-bpf buildbot failure

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2021-11-04 19:05:17 +01:00
Felix Fietkau
6738820bf6 build: fix bpf toolchain dependency for qosify
Add hidden symbols to fix defaults with CONFIG_DEVEL unset

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2021-11-01 18:41:20 +01:00
Felix Fietkau
5ad663b765 config: add choice for selecting the LLVM toolchain type
This is global for all BPF based packages

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2021-11-01 16:38:40 +01:00
Felix Fietkau
faa6a9a04b tools/llvm-bpf: add llvm+clang build suitable for compiling code to eBPF
Preparation for building packages that ship eBPF code

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2021-10-20 11:35:34 +02:00
Rosen Penev
aa344bcfa8 tools/pkgconf: update to 1.8.0
Switch to compiling with meson for faster and more reliable compilation.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2021-09-16 16:31:28 -10:00
Rosen Penev
c4dfdde2ea tools: add meson
meson is a next generation build system designed to have good defaults,
simpler build files, and fast compilation.

It is built upon python and uses ninja for compilation. The latter
provides fast by default (parallel) and problem free compilation.

There are over 40 packages already successfully using meson. The next
commit will convert pkgconf to use meson compilation.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2021-09-16 16:31:28 -10:00
Rosen Penev
4b898afdd2 tools/mm-macros: remove
ccaaab1c04 says that this is in the
codebase because of libsigc++, which is not in the codebase anymore.
Neither in base nor in packages. It doesn't seem to be needed by
anything else either. GNOME packages have transitioned to using meson,
which does not use m4 files.

Tested local compile with CONFIG_ALL. No problems seen.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2021-09-16 16:27:32 -10:00
Felix Fietkau
fa0aea53b2 tools: build ninja before ccache to fix the build order
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2021-06-12 14:33:57 +02:00
Felix Fietkau
d45baa860f tools: fix dependencies of cmake packages
Make cmake depend on ninja, so that other cmake based tools also depend on it

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2021-06-12 12:40:27 +02:00
Felix Fietkau
97258f5363 build: add ninja build tool and make it available for cmake
ninja is faster at building cmake packages than make, and according to reports
also more reliable at handling parallel builds
This commit includes a patch that adds GNU make jobserver support, in order to
allow more precise control over the number of parallel tasks

Enable parallel build by default for packages using ninja

Signed-off-by: Felix Fietkau <nbd@nbd.name>
Signed-off-by: Rosen Penev <rosenp@gmail.com>
2021-06-12 10:46:39 +02:00
Tony Ambardar
bf4aa0c6a2 tools/libelf: remove unneeded host library
This old ELF library dating to 2009 used to be necessary on MacOS but
is not required for building the kernel or tools since [1]. On Linux
systems, libelf is already an OpenWRT build-system prerequisite [2].

Presence of the older library can mask or conflict with the system libelf
and lead to build errors, as seen compiling Linux kernels since v5.8 or
host tools such as dwarves (e.g. pahole).

Remove the unnecessary tools/libelf library and avoid the related issues.

[1] 5f8e587240 ("build: force disable stack validation during kernel build
                 on non-linux systems")
[2] https://openwrt.org/docs/guide-developer/build-system/install-buildsystem#prerequisites

Tested-by: Rosen Penev <rosenp@gmail.com> (Linux)
Tested-by: Georgi Valkov <gvalkov@abv.bg> (MacOS)
Signed-off-by: Tony Ambardar <itugrok@yahoo.com>
2021-03-13 22:20:02 +00:00
Daniel Golle
c4dd2441e7
tools: add xxd (from vim)
U-Boot requires xxd to create the default environment from an external
file as done in uboot-mediatek.
Build xxd (only, not the rest of vim) as part of tools to make sure it
is present on the buildhost.

Reported-by: David Bauer <mail@david-bauer.net>
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2021-02-28 19:27:47 +00:00
David Bauer
ad54e32651 tools: add cpio
mediatek-mt7622 as well as mediatek-mt7623 require CPIO to create their
initramfs images. So build CPIO as part of the host toolchain.

Signed-off-by: David Bauer <mail@david-bauer.net>
2021-02-28 00:09:09 +00:00
Rosen Penev
ec0c6c1143 tools/zstd: compile with cmake
It's faster and more reliable.

Removed ccache cmake build dependency as it's now implicit.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2021-01-30 18:14:50 -10:00