Commit Graph

209 Commits

Author SHA1 Message Date
Sven Wegener
e9fb9b79f6 build: drop ccache variables from specific targets
With commit 2ca084cc ("build: improve ccache support") these variables
are being set globally and we don't need them for specific targets.

Signed-off-by: Sven Wegener <sven.wegener@stealer.net>
2021-01-01 10:29:07 -10:00
Petr Štetiar
4e19cbc553 download: handle possibly invalid local tarballs
Currently it's assumed, that already downloaded tarballs are always
fine, so no checksum checking is performed and the tarball is used even
if it might be corrupted.

From now on, we're going to always check the downloaded tarballs before
considering them valid.

Steps to reproduce:

 1. Remove cached tarball

   rm dl/libubox-2020-08-06-9e52171d.tar.xz

 2. Download valid tarball again

   make package/libubox/download

 3. Invalidate the tarball

   sed -i 's/PKG_MIRROR_HASH:=../PKG_MIRROR_HASH:=ff/' package/libs/libubox/Makefile

 4. Now compile with corrupt tarball source

   make package/libubox/{clean,compile}

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-11-27 14:46:13 +01:00
Felix Fietkau
7716a43092 build: fix extreme build system slowdown caused by SOURCE_DATE_EPOCH changes
Adding inline shell invocations in per-target variables causes them to be
executed over and over again, which causes a significant slowdown.
Fix this by evaluating it only once per package directory

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2020-09-05 11:26:57 +02:00
Roman Yeryomin
2ca084ccaa build: improve ccache support
Set CCACHE_DIR to $(TOPDIR)/.ccache and CCACHE_BASEDIR to $(TOPDIR).
This allows to do clean and dirclean. Cache hit rate for test build
after dirclean is ~65%.
If CCACHE is enabled stats are printed out at the end of building process.
CCACHE_DIR config variable allows to override default, which could be useful
when sharing cache with many builds.
cacheclean make target allows to clean the cache.

Changes from v1:
- remove ccache directory using CCACHE_DIR variable
- remove ccache leftovers from sdk and toolchain make files
- introduce CONFIG_CCACHE_DIR variable
- introduce cacheclean make target

Signed-off-by: Roman Yeryomin <roman@advem.lv>
2020-07-11 15:19:53 +02:00
Jeffery To
dddcff2550 build: Remove STAGING_DIR_HOST references for InstallDev/UninstallDev
Build/InstallDev no longer places a file list in
$(STAGING_DIR_HOST)/packages; this change removes the creation of
$(STAGING_DIR_HOST)/packages and the attempted removal of a
STAGING_DIR_HOST file list during package clean.

This also changes the host directory passed to Build/UninstallDev from
$(STAGING_DIR_HOST) to $(STAGING_DIR)/host, to match the directory
passed to Build/InstallDev.

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
2020-03-01 21:35:59 +01:00
Felix Fietkau
7ec092e641 Revert faulty tree push
Revert "mac80211: add new minstrel_ht patches to improve probing on mt76x2" (9861050b85)
Revert "kernel: use bulk free in kfree_skb_list to improve performance" (98b654de2e)
Revert "ramips: add preliminary support for WIO ONE" (085141dc5b)
Revert "ramips: add preliminary support for SGE AP-MTKH7-0006 developer board" (b1db6d0539)
Revert "build: use config.site generated by autoconf-lean, drop hardcoded sitefiles" (363ce4329d)
Revert "toolchain: add autoconf-lean" (fdb30eed03)
Revert "build: allow overriding the filename on the remote server when downloading" (6fa0e07758)

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2019-08-12 12:27:06 +02:00
Felix Fietkau
363ce4329d build: use config.site generated by autoconf-lean, drop hardcoded sitefiles
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2019-08-12 11:43:39 +02:00
Jeffery To
e545fac8d9 build: include BUILD_VARIANT in PKG_BUILD_DIR
This changes the default PKG_BUILD_DIR to take BUILD_VARIANT into
account (if set), so that packages do not need to manually override
PKG_BUILD_DIR just to handle variants.

This also updates most base packages with variants to use the updated
default PKG_BUILD_DIR.

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
2019-08-05 23:22:26 +02:00
John Crispin
9e57b62783 include/package.mk: Add support for src-checkout/ folder
This feature is similar to the SRC_TREE_OVERRIDE. However instead of having
to manually create a symlink inside the package folder, the buildsystem
will check if $(TOP_DIR)/src-checkout/$(PKG_NAME)/.git is present and use
this instead.

Signed-off-by: John Crispin <john@phrozen.org>
2019-07-10 14:00:19 +02:00
Jo-Philipp Wich
27a4a71c24 metadata: handle ABI version rebuild tracking for transient dependencies
Extend the packageauxvars database to keep a list of possible package
dependencies for each provider, then utilize this information in buildroot
to resolve the ABI version dependencies of dependent packages up to five
levels deep.

This should properly trigger rebuilds for packages indirectly depending
on other packages whose ABI_VERSION changed.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2019-06-21 12:03:43 +02:00
Jeffery To
a117093679 build: fix STAGING_DIR cleaning for packages
This fixes two issues with cleaning package files from STAGING_DIR:

* CleanStaging currently can only remove files and not directories. This
  changes CleanStaging to use clean-package.sh, which does remove
  directories.

* Because of the way directories are ordered in the staging files list,
  clean-package.sh currently tries (and fails) to remove parent
  directories before removing subdirectories. This changes
  clean-package.sh to process the staging files list in reverse, so that
  subdirectories are removed first.

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
2019-01-30 12:32:07 +01:00
Jo-Philipp Wich
991c7a4f69 build: do not override CCACHE_DIR when ccache is disabled
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2018-07-24 15:23:05 +02:00
Jo-Philipp Wich
b287c82bed build: prevent spurious package rebuilds under CONFIG_AUTOREMOVE
When CONFIG_AUTOREMOVE is enabled, the build system touches an additional
".autoremove" stamp file in the cleaned build directory.

Since the autoremove stamp file is touched after the compile process
completed, it ends up being the most recent file of the package build
directory, causing the timestamp.pl check of depends.mk to erroneously
declare the ".built" and ".prepared" stamp files as stale, triggering
a forced clean-build submake process.

Fix the problem by using the ".built" stamp file as modification time
reference when touching the ".autoremove" stamp file.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2018-05-22 09:27:38 +02:00
Felix Fietkau
6fa88be486 build: add support for git submodules with CONFIG_SRC_TREE_OVERRIDE
Also work around an issue where git would store the modified workdir in
the submodule git config files

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2018-04-27 15:19:19 +02:00
Matthias Schiffer
307b29032f
include/package.mk: remove old configured stamps before attempting configuration
Some packages, e.g. busybox, explicitly remove old .configured stamps
before attempting configuration, rather than after the actual configuration
step. This seems like a good idea, as there will be no stamp left if
configuration fails. Change generic rules to work like this, so package-
specific rules can be dropped.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2017-12-28 12:24:25 +01:00
Felix Fietkau
905bbc96ef build: allow PKG_PREPARED_DEPENDS and PKG_CONFIG_DEPENDS to be changed after including package.mk
Reverts commit a9c96ef0ac and replaces it
with a different approach

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-12-12 12:45:28 +01:00
Felix Fietkau
4b8a7c9b48 build: fix QUILT related overrides
They need to be defined before including quilt.mk

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-05-25 20:42:45 +02:00
Felix Fietkau
b044bd5921 build: remove package makefile overlay functionality
Recent attempts to use it have shown that it does not work properly
except for a few undocumented cases. It's better to remove this now to
avoid having more people fall into the same trap

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-04-12 09:53:06 +02:00
Felix Fietkau
89118da865 build: fix STAMP_PREPARED with quilt
quilt.mk needs to be included first, to ensure that STAMP_PREPARED does
not include the hash if quilt is used.

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-03-22 20:17:20 +01:00
Felix Fietkau
a9c96ef0ac build: improve performance by avoiding lazy-eval for make shell calls
Avoids lots of redundant calls to mkhash on things like
package/kernel/linux

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-03-16 19:13:47 +01:00
Felix Fietkau
1f22957247 build: clean up redundant touching of the package install info file
Instead of using a separate .clean stamp file, remove the install info
file on compile, then append the install package list afterwards

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-02-26 13:31:44 +01:00
Felix Fietkau
9467ce42da build: get rid of host.mk
Defined required host related variables in toplevel.mk instead

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-02-26 13:31:44 +01:00
Felix Fietkau
d86802acd8 build: keep recursive dependency check stampfiles up to date
Since the main stamp file depends on the _check stampfile, the _check
stampfile needs to be created on the first run as well.

Fixes spurious rebuilds with CONFIG_AUTOREBUILD=y

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-02-01 14:45:13 +01:00
Felix Fietkau
807461e11c build: make the touch .autoremove call non-fatal
Fixes build issues when there is nothing to do on the compile step (e.g.
with unselected packages).

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-24 13:55:55 +01:00
Felix Fietkau
053dc3b77a build: do not auto-clean packages where the autoremove step has not run
This avoids repeatedly unpacking and rebuilding packages that are
failing the build. Re-running the failing step should be much faster.

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-23 22:04:17 +01:00
Felix Fietkau
0e22d63775 build: fix CONFIG_AUTOREMOVE for packages with multiple variants
Calling the clean target removes all .ipk files and un-stages the
package. Add a new target just for clearing the build dir and call that
one instead of the full clean target

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-22 17:40:31 +01:00
Felix Fietkau
5aa03e16a7 build: disable CONFIG_AUTOREMOVE for packages/tools where QUILT is used
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-19 13:30:22 +01:00
Felix Fietkau
86a72765b9 build: include CONFIG_AUTOREMOVE in STAMP_PREPARED
Fixes build issues when changing the CONFIG_AUTOREMOVE setting

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-19 11:37:19 +01:00
Felix Fietkau
7a523569f7 build: add support for automatically removing build dir contents during build
This is used to save space on buildbot instances.
If any part of a package needs to be rebuild, the whole package is
rebuilt from scratch. Stamp files are preserved to allow dependency
checks to work

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-18 23:57:08 +01:00
Felix Fietkau
9228d1c066 build: introduce extra targets that contain only proper dependencies
This can be used to check if targets like prepare or compile are up to date

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-18 23:57:08 +01:00
Felix Fietkau
6cf067d084 build: define common subdir targets in rules.mk
Reduce build system clutter and enable further rework

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-18 23:57:07 +01:00
Matthias Schiffer
3d71786154
build: move STAGING_DIR_HOSTPKG and BUILD_DIR_HOST back to a common directory for all targets
Using a single host package staging dir (and build dir) significantly speeds up
builds when multiple targets are built in succession, especially for large host
packages like NodeJS.

$(STAGING_DIR)/host is kept in addition to $(STAGING_DIR_HOSTPKG) in most
places; it is still used as destination for host files in Build/InstallDev.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2017-01-18 19:47:36 +01:00
Felix Fietkau
1e1d735e52 build: remove obsolete parallel build related options
Always use the main make jobserver, which has been the default for ages

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-10 12:10:20 +01:00
Felix Fietkau
ffb0181a87 build: add defaults for PKG_SOURCE, PKG_SOURCE_SUBDIR, PKG_VERSION
This makes it easier to unify versioning of git based package downloads.
PKG_SOURCE_DATE along with an 8-character abbreviation of the git hash
is used as PKG_VERSION, PKG_RELEASE should be used like normal packages.

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-12-22 16:42:20 +01:00
Felix Fietkau
7a315b0b5d build: implement make check and make package/X/check
This is intended to be used for a wide array of package sanity checks.

The first check that is implemented is for the hash of downloaded files.
It checks:
  - Missing hash
  - Use of SHA256 instead of MD5
  - dl/<file> hash not matching hash in makefile
  - deprecated MD5SUM variable

The deprecated MD5SUM variable check is skipped for feeds/ until OpenWrt
is updated as well

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-12-17 10:36:25 +01:00
Felix Fietkau
6a5cc2d085 include/package.mk: sync default value for hash fallback with mirror hash
Simplifies dealing with PKG_HASH vs PKG_MD5SUM

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-12-16 15:40:07 +01:00
Felix Fietkau
881c5b47ec build: remove duplicate Download/default definition from include/host-build.mk
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-12-16 15:35:21 +01:00
Felix Fietkau
7416d2e046 build: replace MD5SUM variables with HASH
Since we've switched to preferring SHA256 over MD5, the old variable
name is misleading. Packages using the old name remain compatible.

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-12-16 13:49:24 +01:00
Yousong Zhou
4eb371e363 build: fix cleaning configured stamp file
We have packages with their own parts appended to standard STAMP_CONFIGURED
(mostly with an underscore character).  This will render the current
STAMP_CONFIGURED_WILD setting invalid and the build system may miss a rebuild
on config change

 1. Build with config A
 2. Build with config B, yet .configured_A_xx did not get cleaned
 3. Return to config A, but rebuild will not happen because stamp file
    of config A still exists

Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
2016-10-15 11:36:51 +02:00
Matthias Schiffer
6fb212f293
build: don't add -j for parallel builds with Make 4.2+
Make usually passes -j and jobserver arguments as part of MAKEFLAGS. LEDE
removes MAKEFLAGS to have better control of the build, and re-adds the
jobserver arguments with -j to the Make commandline where desired.

Make 4.2+ behave differently with these arguments passed on the commandline
than in MAKEFLAGS: -j will override the jobserver argument and the job
count will be unlimited.

Moving the flags to MAKEFLAGS will need many packages to be changed and
tested; therefore, we opt for a less invasive change for now and just
remove -j for Make 4.2+, as the jobserver argument alone is enough to
enable parallel builds for these Make versions.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2016-06-15 19:26:17 +02:00
Felix Fietkau
852aaf6b2c build: add support for specifying extra package dependencies for prepared, configured and built
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-05-05 19:43:23 +01:00
Jo-Philipp Wich
5170393f8c include: choose package output directory based on repository info
Use the new repository metadata field to choose the output directory of the
final package archives.

Non-sharable packages will be placed in the per-target package directory
while the rest will be placed in a per-repository sub directory within the
$OUTPUT_DIR/packages/$CPU_TYPE/ prefix.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2016-04-06 21:55:44 +02:00
Felix Fietkau
b737686b2c build: use -iremap feature to replace __FILE__ absolute path names in packages with ones relative to $(BUILD_DIR)
improves reproducible builds

Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 48489
2016-01-25 15:49:31 +00:00
Felix Fietkau
b1986d8fd6 build: use directory name instead of PKG_NAME for staging files and package metadata
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 48110
2016-01-04 01:33:32 +00:00
Steven Barth
fc5f02410f buildroot: move hardening options into separate file
Signed-off-by: Steven Barth <steven@midlink.org>

SVN-Revision: 46070
2015-06-20 17:37:18 +00:00
Steven Barth
ca96d1ab3f package.mk: replicate SSP and relro in both CFLAGS and LDFLAGS
Signed-off-by: Steven Barth <steven@midlink.org>

SVN-Revision: 46035
2015-06-18 10:48:28 +00:00
Jo-Philipp Wich
a621b8c284 include: clean package staging dir files before configure
This is required to allow packages to ship libraries with a different API.
Previously the linker would favor the old shared libraries in STAGING_DIR
instead of the just compiled ones in PKG_BUILD_DIR.

Avoid that problem by clearing the package staging files right after
prepare.

Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>

SVN-Revision: 45759
2015-05-26 09:17:13 +00:00
Felix Fietkau
d42be88915 build: include variant name in STAMP_INSTALLED
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 45253
2015-04-03 18:33:25 +00:00
Luka Perkov
7002f6326e include: remove trailing whitespaces
Signed-off-by: Luka Perkov <luka@openwrt.org>

SVN-Revision: 45127
2015-03-29 07:29:18 +00:00
Felix Fietkau
74ca7e9af3 build: clean up sitefiles, make them per-arch
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 45090
2015-03-28 13:38:34 +00:00