Commit Graph

113 Commits

Author SHA1 Message Date
Petr Štetiar
21e5db97c4
build: add CycloneDX SBOM JSON support
CycloneDX is an open source standard developed by the OWASP foundation.
It supports a wide range of development ecosystems, a comprehensive set
of use cases, and focuses on automation, ease of adoption, and
progressive enhancement of SBOMs (Software Bill Of Materials) throughout
build pipelines.

So lets add support for CycloneDX SBOM for packages and images
manifests.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
(cherry picked from commit d604a07225)
2023-11-02 14:44:47 +00:00
Paul Spooren
218ce40cd7 build: generate index.json
The index.json file lies next to Packages index files and contains a
json dict with the package architecture and a dict of package names and
versions.

This can be used for downstream project to know what packages in which
versions are available.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2023-05-09 23:56:49 +02:00
Alexander Egorenkov
1854aeec4d build: fix opkg install step for large package selection
When the list of packages to be installed in a built image exceeds a certain
number, then 'opkg install' executed for target '$(curdir)/install' in
package/Makefile fails with: /usr/bin/env: Argument list too long.

On Linux, the length of a command-line parameter is limited by
MAX_ARG_STRLEN to max 128 kB.

* https://elixir.bootlin.com/linux/latest/source/include/uapi/linux/binfmts.h#L15
* https://www.in-ulm.de/~mascheck/various/argmax/

To solve the problem, store the package list being passed to 'opkg install'
in a temporary file and use the shell command substitution to pass the
content of the file to 'opkg install'. This guarantees that the length of
the command-line parameters passed to the bash shell is short.

The following bash script demonstrates the problem:
----------------------------------------------------------------------------
count=${1:-1000}

FILES=""
a_file="/home/egorenar/Repositories/openwrt-rel/bin/targets/alpine/generic/packages/base-files_1414-r16464+19-e887049fbb_arm_cortex-a15_neon-vfpv4.ipk"

for i in $(seq 1 $count); do
	FILES="$FILES $a_file"
done

env bash -c "echo $FILES >/dev/null"
echo "$FILES" | wc -c
----------------------------------------------------------------------------

Test run:
----------------------------------------------------------------------------
$ ./test.sh 916
130989
$ ./test.sh 917
./test.sh: line 14: /bin/env: Argument list too long
131132
----------------------------------------------------------------------------

Signed-off-by: Alexander Egorenkov <egorenar-dev@posteo.net>
[reword commit subject]
Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-05-12 11:13:53 +02:00
Felix Fietkau
953435795d build: always build package/kernel/linux
build: always build package/kernel/linux

If no in-tree module packages are selected, the build system does not process
package/kernel/linux. This package is required for building the virtual
'kernel' package, which is specified as a dependency for all kernel packages.

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2020-10-15 13:25:58 +02:00
Paul Spooren
62d5ec7306 build: store SourceDateEpoch in manifest
The usage of granular `SOURCE_DATE_EPOCH` for packages is an
incrementing integer which could be useful for downstream tooling,
therefore add it to the packages manifest.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-08-31 11:18:06 +01:00
Jo-Philipp Wich
e1f588e446 packages: apply usign padding workarounds to package indexes if needed
Since usign miscalculates SHA-512 digests for input sizes of exactly
64 + N * 128 + 110 or 64 + N * 128 + 111 bytes, we need to apply some
white space padding to avoid triggering the hashing edge case.

While usign itself has been fixed already, there is still many firmwares
in the wild which use broken usign versions to verify current package
indexes so we'll need to carry this workaround in the forseeable future.

Ref: https://forum.openwrt.org/t/signature-check-failed/41945
Ref: https://git.openwrt.org/5a52b379902471cef495687547c7b568142f66d2
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2019-08-07 07:15:07 +02:00
Jo-Philipp Wich
e6bcf1e4ac build: add ABI_VERSION to binary package names
Add the ABI_VERSION source makefile variable to the binary package basename
and resolve source dependencies on packages with ABI_VERSION set to such
expanded names.

If for example a package specifies DEPENDS:=libopenssl while the OpenSSL
Makefile specifies ABI_VERSION:=1.0.0, the resulting ipk control data
dependency will be "Depends: libopenssl1.0.0" and the libopenssl ipk file
will be called "libopenssl1.0.0_<version>_<arch>.ipk".

The next time a library such as OpenSSL is updated to an incompatible
version, the ABI_VERSION shall be changed accordingly to prevent opkg from
simply upgrading to an incompatible library without considering the
dependencies of already installed packages.

Also introduce another "SourceName" control field which is required by
the newly introduced "scritps/ipkg-remove" to determine the proper related
.ipk files to delete upon buildroot package clean operations.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2019-01-19 14:32:12 +01:00
Matthias Schiffer
2fbf669730
imagebuilder: reuse rootfs preparation from rootfs.mk
In addition to removing redundant code, this fixes various issues in
IB-generated images that have been fixed in prepare_rootfs before,
including better handling of CONFIG_CLEAN_IPKG and enabling of initscripts
from FILES.

We also reuse the opkg macro and remove --force-... flags that have been
removed from rootfs.mk as well.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-03-07 09:59:08 +01:00
Matthias Schiffer
cf1c7c0f17
include/rootfs.mk: pass additional files dir to prepare_rootfs as an argument
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-03-07 09:59:07 +01:00
Matthias Schiffer
3abf663c22
build: remove package preconfig feature
This feature has been unused for years, and its scope is too limited to be
actually useful.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-01-13 19:54:44 +01:00
Yousong Zhou
dac629f710 build: cleanup tmp/ dir of target rootfs
Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
2017-05-02 22:10:50 +08:00
Felix Fietkau
0aa46bf76a build: skip opkg host dependency within the SDK
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-02-21 20:43:40 +01:00
Felix Fietkau
adeae0e02a build: move opkg host dependency from package/install to package/compile
Improves parallel build behavior, since it allows opkg to be built at
the same time as other packages

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-02-21 13:03:20 +01:00
Felix Fietkau
d826af2cbb build: make <subdir>/install opt-in, use it for target/ only
Fixes buildbot errors on running make target/install or
toolchain/install

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-02-09 13:51:35 +01:00
Felix Fietkau
664918d891 Revert "build: always run package/cleanup before package/compile"
This reverts commit 2990a21058.
This introduces a race condition, let's fix this in buildbot instead.

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-22 13:47:40 +01:00
Felix Fietkau
2990a21058 build: always run package/cleanup before package/compile
Remove unnecessary stampfile indirection
Fixes an issue with the command sequence used by buildbot

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-22 13:31:54 +01:00
Jo-Philipp Wich
1fb673ee12 build: fix triggering opkg/host compilation
Commit 131db36 "build: remove separate /install step for host builds" dropped
the package/*/host/install targets in favor to performing the install steps
within the compile target instead.

Adjust package/Makefile accordingly in order to prevent a missing
staging_dir/host/bin/opkg when staging package archives into the rootfs.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2017-01-19 02:44:08 +01:00
Felix Fietkau
7a523569f7 build: add support for automatically removing build dir contents during build
This is used to save space on buildbot instances.
If any part of a package needs to be rebuild, the whole package is
rebuilt from scratch. Stamp files are preserved to allow dependency
checks to work

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-18 23:57:08 +01:00
Felix Fietkau
9cdf852ae0 opkg: drop S/MIME support
It has never been used by default (due to being too bloated), and it is
properly replaced by usign (which has been the default for a long time
now).

Remove this feature to simplify the build system

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-10 13:44:25 +01:00
Alberto Bursi
c24172cad1 package/Makefile & ipkg-make-index.sh: add full package data list
The external script used to generate the package lists for the
LEDE wiki's table of packages [1] and package indexes [2] requires
a "Source:" field in the package lists to find package makefiles.
The package makefiles are used to read the package's Category and Submenu.

The "Source:" field was removed in commit
b4aa3c899c
to reduce package list sizes and lessen opkg issues in low ram devices.

Add a separate package list file with full data to be used by the wiki's script.
It's called Packages.manifest and isn't compressed as it's not necessary.

1. https://lede-project.org/packages/start
2. https://lede-project.org/packages/index/start

Signed-off-by: Alberto Bursi <alberto.bursi@outlook.it>
2016-12-26 11:17:33 +01:00
Felix Fietkau
180e93ba8b build: add CHECK_ALL variable to allow make download/check to include not selected packages
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-12-17 10:36:52 +01:00
Felix Fietkau
7a315b0b5d build: implement make check and make package/X/check
This is intended to be used for a wide array of package sanity checks.

The first check that is implemented is for the hash of downloaded files.
It checks:
  - Missing hash
  - Use of SHA256 instead of MD5
  - dl/<file> hash not matching hash in makefile
  - deprecated MD5SUM variable

The deprecated MD5SUM variable check is skipped for feeds/ until OpenWrt
is updated as well

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-12-17 10:36:25 +01:00
Matthias Schiffer
663145e419
image: fix CONFIG_CLEAN_IPKG with CONFIG_TARGET_PER_DEVICE_ROOTFS
Running prepare_rootfs on TARGET_DIR deletes the opkg state when
CONFIG_CLEAN_IPKG is enabled, making the per-device rootfs package install
fail.

To avoid this, create a copy of the TARGET_DIR before prepare_rootfs is run
and use this as basis for per-device rootfs generation.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2016-09-25 09:30:55 +02:00
Felix Fietkau
180465c38f build: create a package feed directory containing all packages
Needed for proper dependency handling for per-device rootfs

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-08-01 11:13:38 +02:00
Felix Fietkau
731b166528 build: add template for getting opkg package files from package names
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-07-29 10:18:26 +02:00
Felix Fietkau
5d30bf8303 build: rework opkg command invocation
Drop included $(XARGS), add support for passing target dir via parameter

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-07-29 10:18:26 +02:00
Felix Fietkau
7dffc32ffa build: rework prepare_rootfs to pass target dir via parameter
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-07-29 10:18:26 +02:00
Felix Fietkau
973e6e1d71 build: move rootfs processing code to include/rootfs.mk so it can be reused later
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-07-29 10:18:26 +02:00
Jo-Philipp Wich
5170393f8c include: choose package output directory based on repository info
Use the new repository metadata field to choose the output directory of the
final package archives.

Non-sharable packages will be placed in the per-target package directory
while the rest will be placed in a per-repository sub directory within the
$OUTPUT_DIR/packages/$CPU_TYPE/ prefix.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2016-04-06 21:55:44 +02:00
Hauke Mehrtens
ca77367dae package: remove .git files from images
If you have your ./files/ directory stored in a git repository,
the .git will be included into images using precious space.
This patch removes .git directories before packing images.

Signed-off-by: Joerg Jungermann <jj@borkum.net>
Signed-off-by: Hauke Mehrtens <hauke@hauke-m.de>

SVN-Revision: 49180
2016-04-17 12:50:30 +00:00
Jo-Philipp Wich
d201f01d5d buildroot: apply IGNORE_ERRORS to host builds
Apply the error ignore mechanism to host builds as well in order to skip over
broken feed packages.

Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>

SVN-Revision: 48918
2016-03-04 11:20:11 +00:00
Felix Fietkau
ad8169546c package/Makefile: override opkg installation time when SOURCE_DATE_EPOCH set
Signed-off-by: Alexander Couzens <lynxis@fe80.eu>
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 48585
2016-01-31 23:29:12 +00:00
Jo-Philipp Wich
0333da8943 include: group kmod ipk files into a "kernel" subdirectory
This is useful to just use the kmods from an official build while supplying
base packages from a custom feed or the other way around; for just overriding
the kmods with a local repo while using official repos for the rest.

Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>

SVN-Revision: 48475
2016-01-24 15:36:05 +00:00
Felix Fietkau
ce6c1c6c84 build: add opkg host dependency
fixes a missing host opkg
found by: unselect base-files, opkg

Signed-off-by: Dirk Neukirchen <dirkneukirchen@web.de>

SVN-Revision: 47885
2015-12-13 17:03:40 +00:00
John Crispin
096b3759a9 build: Prevent more gzip timestamps
To improve reproducibility, prevent the inclusion of timestamps
in the gzip header.

Signed-off-by: Reiner Herrmann <reiner@reiner-h.de>

SVN-Revision: 46361
2015-07-14 09:57:45 +00:00
Matteo Croce
c37d7a7e8f fix mklibs with musl
crate a relative symlink to libc.so because
make wildcard function ignores broken symlinks

SVN-Revision: 46123
2015-06-25 10:14:22 +00:00
Felix Fietkau
b35846974e build: do another init script enabling run, fixes init scripts added via files/ (#19857)
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 45964
2015-06-14 17:42:05 +00:00
Jo-Philipp Wich
edbb32897a build: trigger pacakge index creation for all feeds if required
When CONFIG_PER_FEED_REPO_ADD_DISABLED is set, trigger index creation for
any available feed, regardless of whether there where binaries built or not.

Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>

SVN-Revision: 45765
2015-05-26 11:37:05 +00:00
Felix Fietkau
ecb14f4a5d build: allow creating empty package feeds
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 45757
2015-05-26 09:09:07 +00:00
Felix Fietkau
beca028bd6 build: add integration for managing opkg package feed keys
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 45286
2015-04-06 19:39:51 +00:00
John Crispin
44249f71a8 postinst trigger: the new postinst trigger broke IB
Signed-off-by: John Crispin <blogic@openwrt.org>

SVN-Revision: 42571
2014-09-16 11:56:19 +00:00
John Crispin
d6dbc22bbc ipkg: add a default postinst/prerm script
the postinst script enables/starts the init.d scripts upon package installation
and installs the users required by the package.

the prerm script stops and disables the init.d scripts.

Signed-off-by: John Crispin <blogic@openwrt.org>

SVN-Revision: 42470
2014-09-11 12:27:49 +00:00
Jo-Philipp Wich
aebbc47c2a build: consider sub-directories when deploying packages to image with CONFIG_PER_FEED_REPO (#17430)
Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>

SVN-Revision: 42041
2014-08-07 15:35:12 +00:00
Jo-Philipp Wich
a720dd209f build: introduce per feed repository support
This changeset implements a new menuconfig option to generate separate
repositories for each enabled package feed instead of one monolithic one.

Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>

SVN-Revision: 42002
2014-08-05 11:24:24 +00:00
John Crispin
3936ed4028 package/Makefile: Substituted deprecated -perm +mode with -perm /mode
The find(1) -perm +mode is no longer supported and has been deprecated since
2005.

This could resolve #13855 ticket.

Signed-off-by: Rocco Folino <lordzen87@gmail.com>

SVN-Revision: 40927
2014-06-02 12:44:13 +00:00
Jo-Philipp Wich
7490659254 package: ignore failed downloads with IGNORE_ERRORS
When IGNORE_ERROR is set, also skip broken downloads, this is required
to make a full "make download" run even if some upstream sources are
unreachable.

Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>

SVN-Revision: 40768
2014-05-14 15:49:22 +00:00
Felix Fietkau
ee4110aff0 target/sdk: use .config instead of unconditionally enabling all build dirs
Call make defconfig on every build to catch newly added packages

Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 39913
2014-03-13 12:12:58 +00:00
Felix Fietkau
d1c64ab221 build: fix installation of default-variant packages when the source package has multiple variants (#15237)
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 39899
2014-03-12 19:53:13 +00:00
Steven Barth
03b0c3b5a6 package/index: fix index creating when building without signing
Signed-off-by: Alexander Couzens <lynxis@fe80.eu>

SVN-Revision: 38287
2013-10-02 16:38:28 +00:00
Steven Barth
cbdd346b11 Add package signing infrastructure
Add package signing key and certificate configuration options to the
"Image configuration" submenu. If enabled, the Packages.gz list will
be signed as file Packages.sig. The passphrase for the signing key can
be sourced from a file or entered by the user. The signing certificate
is automatically added to the firmware image if opkg-smime is selected.

Signed-off-by: Evan Hunt <each@isc.org>
Signed-off-by: Steven Barth <steven@midlink.org>

SVN-Revision: 38284
2013-10-02 12:12:10 +00:00