While experimenting with the AUTOREMOVE option in search of a way to use
prebuilt host tools in different buildroot, it was discovered that the
md5 generated by find_md5 in depends.mk is not reproducible.
Currently the hash is generated by the path of the file in addition to
the file mod time. Out of confusion, probably, there was an idea that
such command was used on the package build_dir. Reality is that this
command is run on the package files. (Makefile, patches, src)
This is problematic because the package Makefile (for example) change at
each git clone and base the hash on the Makefile mtime doesn't really
reflect if the Makefile actually changes across a buildroot or not.
A better approach is to generate an hash of each file and then generate
an hash on the sort hash list. This way we remove the problem of git
clone setting a wrong mtime while keeping the integrity of checking if a
file changed for the package as any change will result in a different
hash.
Introduce a new kind of find_md5 function, find_md5_reproducible that
apply this new logic and limit it only with AUTOREMOVE option set to
prevent any kind of slowdown due to additional hash generation.
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
(cherry picked from commit 53a08e3743)
Before this commit, it was assumed that mkhash is in the PATH. While
this was fine for the normal build workflow, this led to some issues if
make TOPDIR="$(pwd)" -C "$pkgdir" compile
was called manually. In most of the cases, I just saw warnings like this:
make: Entering directory '/home/.../package/gluon-status-page'
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
[...]
While these were only warnings and the package still compiled sucessfully,
I also observed that some package even fail to build because of this.
After applying this commit, the variable $(MKHASH) is introduced. This
variable points to $(STAGING_DIR_HOST)/bin/mkhash, which is always the
correct path.
Signed-off-by: Leonardo Mörlein <me@irrelefant.net>
The license folder is a core part of OpenWrt and all GPL-2.0 licensed.
Use SPDX license tags to allow machines to check licenses.
Signed-off-by: Paul Spooren <mail@aparcar.org>
[rebase, keep some Copyright lines, sharpen commit message]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
It was observed that the MD5 would not change after source files had been
modified, looking deeper into the build process it was discovered that
find_md5 build function makes a list of the files being built and then
passes the list to a summing utility on stdin. The resultant MD5 is of
the file list, not the contents of the files.
The MD5 would change if the ordering of the list changed, or items were
removed or deleted.
The proposed fix is to add the modification time after the filename and
then sort the list to prevent find returning files in a different order
falsely re-triggering a rebuild. The MD5 will now change when a file is
modified or files are added/removed from the list.
Using 'T@' to show time in epoch for timezone independent behaviour.
Signed-off-by: John Beckett <john.beckett@net2edge.com>
This makes the build slightly more efficient by avoiding the need to
re-stage packages on every full build run.
It is also necessary for the upcoming CONFIG_AUTOREMOVE feature
Signed-off-by: Felix Fietkau <nbd@nbd.name>
Targets like malta can have no patches/ directory available and this
commit tries quash "no such file or directory" messages from `find`
Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
