This commit solves multiple issues. First of just install the three
special packages base-files, libc and kernel directly from the index. In
upstream indexes, those will never appear to prevent accidental upgrades
may breaking the system.
Next, enable caching for the ImageBuilder, which speeds up consecutive
builds from ~33 seconds to ~5 seconds. Using cache however makes APK
create the folder `/var/cache/apk/` which conflicts with the base-files
installation, which ships a symlink from `/var` to `/tmp`, so specify
`--no-cache` for the rootfs initialization.
Lastly, drop the use of `apk update` since APK automatically does that.
Signed-off-by: Paul Spooren <mail@aparcar.org>
The same that is done in `ipkg-make-index.sh` should happen with APK.
If the pseudo packages, only added to add dependency constraints, are
added to the index, APK happily "upgrades" them and installs updated
kmods, too. However, the Kernel itself is never installed via a regular
package.
Fixes: https://github.com/openwrt/openwrt/issues/16808
Signed-off-by: Paul Spooren <mail@aparcar.org>
Prior to this commit keys would only be generated if `make` is called
alone, but not for something like `make package/busybox/compile`.
The exact reasons are in the depth of make magic, so this is sheer luck!
Signed-off-by: Paul Spooren <mail@aparcar.org>
To better support imagebuilder declaring --repositories-file on calling
apk macro, detach this and --repository from rootfs.mk macro and move it
to package Makefile and image.mk where they are used to permit a more
generic usage.
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
The keys are created differently compared to the old OPKG keys. Instead
of being part of base-files/configure, they are created as a Makefile
requirement of `package/compile`, which is a cleaner solution.
This requirement would only be added to non SDK environments, however
APK always requires keys to be available. Add an `else` case for the SDK
and create keys.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Currently the build with USE_APK=y fails in package/libs/toolchain:
staging_dir/host/bin/fakeroot: line 182: staging_dir/host/bin/apk: No such file or directory
make[2]: *** [Makefile:758: bin/targets/mediatek/filogic/packages/libgcc1-13.2.0-r4.apk] Error 127
as commit d788ab376f ("build: add APK package build capabilities") added
dependency on apk in packaging step, but there is no host build
dependency defined, thus apk binary is missing when libgcc1 apk package is being
created. So lets fix it by adding explicit apk/host dependency to all
targets in the subdirectories.
Fixes: d788ab376f ("build: add APK package build capabilities")
Signed-off-by: Petr Štetiar <ynezz@true.cz>
[ rework logic to be more self contained ]
Link: https://github.com/openwrt/openwrt/pull/15543
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
Other than OPKG which only uses signed package list, APK uses
individually signed packages in addition to signed package lists. Hence,
in order to be able to generate package, the private key needs to be
generated before compiling packages. Express that dependency and
generate the private key before building any packages instead of doing
so as part of the base-files package build.
Fixes: d788ab376f ("build: add APK package build capabilities")
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
A new option called `USE_APK` is added which generated APK packages
(.apk) instead of OPKG packages (.ipk).
Some features like fstools `snapshot` command are not yet ported
Signed-off-by: Paul Spooren <mail@aparcar.org>
The dependency can't be satisfied when building using the SDK, breaking
package builds. As the staging and bin dirs are distributed with the SDK
archive, ignoring the dependency is fine when SDK is set.
Fixes: fbb924abff ("build: add $(STAGING_DIR) and $(BIN_DIR) ...")
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
In a pristine build, these directories are created as dependencies of
the tools subdir compile, however this step never runs when the tools
compile stamp already exists. Since commit ed6ba2801c ("tools: keep
stamp file in $(STAGING_DIR_HOST)"), this will happen after `make clean`:
$(STAGING_DIR) has been deleted, but the tools stamp still exists, so
the next build will fail because $(STAGING_DIR) has not been set up
correctly.
Fix builds after `make clean` by adding the preparation as dependencies
for the target and package directories as well.
Fixes: ed6ba2801c ("tools: keep stamp file in $(STAGING_DIR_HOST)")
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
CycloneDX is an open source standard developed by the OWASP foundation.
It supports a wide range of development ecosystems, a comprehensive set
of use cases, and focuses on automation, ease of adoption, and
progressive enhancement of SBOMs (Software Bill Of Materials) throughout
build pipelines.
So lets add support for CycloneDX SBOM for packages and images
manifests.
Signed-off-by: Petr Štetiar <ynezz@true.cz>
The index.json file lies next to Packages index files and contains a
json dict with the package architecture and a dict of package names and
versions.
This can be used for downstream project to know what packages in which
versions are available.
Signed-off-by: Paul Spooren <mail@aparcar.org>
When the list of packages to be installed in a built image exceeds a certain
number, then 'opkg install' executed for target '$(curdir)/install' in
package/Makefile fails with: /usr/bin/env: Argument list too long.
On Linux, the length of a command-line parameter is limited by
MAX_ARG_STRLEN to max 128 kB.
* https://elixir.bootlin.com/linux/latest/source/include/uapi/linux/binfmts.h#L15
* https://www.in-ulm.de/~mascheck/various/argmax/
To solve the problem, store the package list being passed to 'opkg install'
in a temporary file and use the shell command substitution to pass the
content of the file to 'opkg install'. This guarantees that the length of
the command-line parameters passed to the bash shell is short.
The following bash script demonstrates the problem:
----------------------------------------------------------------------------
count=${1:-1000}
FILES=""
a_file="/home/egorenar/Repositories/openwrt-rel/bin/targets/alpine/generic/packages/base-files_1414-r16464+19-e887049fbb_arm_cortex-a15_neon-vfpv4.ipk"
for i in $(seq 1 $count); do
FILES="$FILES $a_file"
done
env bash -c "echo $FILES >/dev/null"
echo "$FILES" | wc -c
----------------------------------------------------------------------------
Test run:
----------------------------------------------------------------------------
$ ./test.sh 916
130989
$ ./test.sh 917
./test.sh: line 14: /bin/env: Argument list too long
131132
----------------------------------------------------------------------------
Signed-off-by: Alexander Egorenkov <egorenar-dev@posteo.net>
[reword commit subject]
Signed-off-by: Paul Spooren <mail@aparcar.org>
build: always build package/kernel/linux
If no in-tree module packages are selected, the build system does not process
package/kernel/linux. This package is required for building the virtual
'kernel' package, which is specified as a dependency for all kernel packages.
Signed-off-by: Felix Fietkau <nbd@nbd.name>
The usage of granular `SOURCE_DATE_EPOCH` for packages is an
incrementing integer which could be useful for downstream tooling,
therefore add it to the packages manifest.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Since usign miscalculates SHA-512 digests for input sizes of exactly
64 + N * 128 + 110 or 64 + N * 128 + 111 bytes, we need to apply some
white space padding to avoid triggering the hashing edge case.
While usign itself has been fixed already, there is still many firmwares
in the wild which use broken usign versions to verify current package
indexes so we'll need to carry this workaround in the forseeable future.
Ref: https://forum.openwrt.org/t/signature-check-failed/41945
Ref: https://git.openwrt.org/5a52b379902471cef495687547c7b568142f66d2
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Add the ABI_VERSION source makefile variable to the binary package basename
and resolve source dependencies on packages with ABI_VERSION set to such
expanded names.
If for example a package specifies DEPENDS:=libopenssl while the OpenSSL
Makefile specifies ABI_VERSION:=1.0.0, the resulting ipk control data
dependency will be "Depends: libopenssl1.0.0" and the libopenssl ipk file
will be called "libopenssl1.0.0_<version>_<arch>.ipk".
The next time a library such as OpenSSL is updated to an incompatible
version, the ABI_VERSION shall be changed accordingly to prevent opkg from
simply upgrading to an incompatible library without considering the
dependencies of already installed packages.
Also introduce another "SourceName" control field which is required by
the newly introduced "scritps/ipkg-remove" to determine the proper related
.ipk files to delete upon buildroot package clean operations.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
In addition to removing redundant code, this fixes various issues in
IB-generated images that have been fixed in prepare_rootfs before,
including better handling of CONFIG_CLEAN_IPKG and enabling of initscripts
from FILES.
We also reuse the opkg macro and remove --force-... flags that have been
removed from rootfs.mk as well.
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
This feature has been unused for years, and its scope is too limited to be
actually useful.
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
Commit 131db36 "build: remove separate /install step for host builds" dropped
the package/*/host/install targets in favor to performing the install steps
within the compile target instead.
Adjust package/Makefile accordingly in order to prevent a missing
staging_dir/host/bin/opkg when staging package archives into the rootfs.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
This is used to save space on buildbot instances.
If any part of a package needs to be rebuild, the whole package is
rebuilt from scratch. Stamp files are preserved to allow dependency
checks to work
Signed-off-by: Felix Fietkau <nbd@nbd.name>
It has never been used by default (due to being too bloated), and it is
properly replaced by usign (which has been the default for a long time
now).
Remove this feature to simplify the build system
Signed-off-by: Felix Fietkau <nbd@nbd.name>
The external script used to generate the package lists for the
LEDE wiki's table of packages [1] and package indexes [2] requires
a "Source:" field in the package lists to find package makefiles.
The package makefiles are used to read the package's Category and Submenu.
The "Source:" field was removed in commit
b4aa3c899c
to reduce package list sizes and lessen opkg issues in low ram devices.
Add a separate package list file with full data to be used by the wiki's script.
It's called Packages.manifest and isn't compressed as it's not necessary.
1. https://lede-project.org/packages/start
2. https://lede-project.org/packages/index/start
Signed-off-by: Alberto Bursi <alberto.bursi@outlook.it>
This is intended to be used for a wide array of package sanity checks.
The first check that is implemented is for the hash of downloaded files.
It checks:
- Missing hash
- Use of SHA256 instead of MD5
- dl/<file> hash not matching hash in makefile
- deprecated MD5SUM variable
The deprecated MD5SUM variable check is skipped for feeds/ until OpenWrt
is updated as well
Signed-off-by: Felix Fietkau <nbd@nbd.name>
Running prepare_rootfs on TARGET_DIR deletes the opkg state when
CONFIG_CLEAN_IPKG is enabled, making the per-device rootfs package install
fail.
To avoid this, create a copy of the TARGET_DIR before prepare_rootfs is run
and use this as basis for per-device rootfs generation.
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
Use the new repository metadata field to choose the output directory of the
final package archives.
Non-sharable packages will be placed in the per-target package directory
while the rest will be placed in a per-repository sub directory within the
$OUTPUT_DIR/packages/$CPU_TYPE/ prefix.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
If you have your ./files/ directory stored in a git repository,
the .git will be included into images using precious space.
This patch removes .git directories before packing images.
Signed-off-by: Joerg Jungermann <jj@borkum.net>
Signed-off-by: Hauke Mehrtens <hauke@hauke-m.de>
SVN-Revision: 49180
Apply the error ignore mechanism to host builds as well in order to skip over
broken feed packages.
Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>
SVN-Revision: 48918
This is useful to just use the kmods from an official build while supplying
base packages from a custom feed or the other way around; for just overriding
the kmods with a local repo while using official repos for the rest.
Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>
SVN-Revision: 48475
To improve reproducibility, prevent the inclusion of timestamps
in the gzip header.
Signed-off-by: Reiner Herrmann <reiner@reiner-h.de>
SVN-Revision: 46361
When CONFIG_PER_FEED_REPO_ADD_DISABLED is set, trigger index creation for
any available feed, regardless of whether there where binaries built or not.
Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>
SVN-Revision: 45765
