It seems like after a build the /dl dir seems to now contain a .hash
file for each source file due to inproper cleanup so fix it by removing
those intermediate files before leaving the download action.
Fixes: 4e19cbc553 ("download: handle possibly invalid local tarballs")
Reported-by: Hannu Nyman <hannu.nyman@iki.fi>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Currently it's assumed, that already downloaded tarballs are always
fine, so no checksum checking is performed and the tarball is used even
if it might be corrupted.
From now on, we're going to always check the downloaded tarballs before
considering them valid.
Steps to reproduce:
1. Remove cached tarball
rm dl/libubox-2020-08-06-9e52171d.tar.xz
2. Download valid tarball again
make package/libubox/download
3. Invalidate the tarball
sed -i 's/PKG_MIRROR_HASH:=../PKG_MIRROR_HASH:=ff/' package/libs/libubox/Makefile
4. Now compile with corrupt tarball source
make package/libubox/{clean,compile}
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Silence the warning in git 2.27 about undefined fast-forward style
in git pull. Define "ff-only" as the style.
Signed-off-by: Hannu Nyman <hannu.nyman@iki.fi>
As package size changes are a continuous topic on the mailing list this
scripts helps developers to compare their local package modifications
against latest upstream.
The script downloads the latest package indexes based on env variables
or the `.config` file. The script compares the actual installed size
(data.tar.gz) or the IPK package size.
An example output is found below:
```
user@dawn:~/src/openwrt/openwrt$ ./scripts/size_compare.sh
Compare packages of ath79/tiny/mips_24kc:
dropbear busybox iw ubus
Checking configuration difference
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 554 100 554 0 0 336 0 0:00:01 0:00:01 --:--:-- 336
--- start config diff ---
--- /tmp/config.DDjwVh-LOCAL 2020-11-23 09:08:28.913203068 -1000
+++ /tmp/config.DDjwVh-UPSTREAM 2020-11-23 09:08:36.369240887 -1000
@@ -1,5 +1,9 @@
+CONFIG_ALL_KMODS=y
+CONFIG_ALL_NONSHARED=y
CONFIG_AUTOREBUILD=y
+CONFIG_AUTOREMOVE=y
--- 8< ---
CONFIG_BINARY_FOLDER=""
+CONFIG_BUILDBOT=y
+CONFIG_TARGET_ALL_PROFILES=y
CONFIG_TARGET_ROOTFS_DIR=""
CONFIG_USE_SSTRIP=y
CONFIG_USE_UCLIBCXX=y
--- end config diff ---
Checking installed size
Fetching latest package indexes...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 80634 100 80634 0 0 33499 0 0:00:02 0:00:02 --:--:-- 33485
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 54082 100 54082 0 0 24252 0 0:00:02 0:00:02 --:--:-- 24252
Comparing package sizes...
Change Local Remote Package
+271 51386 51115 base-files
+123 705241 705118 bnx2-firmware
+86 17209 17123 fstools
+22 47989 47967 procd
+21 208311 208290 busybox
+19 67181 67162 netifd
```
I plan to integrate this script into the CI so we have a summary how
sizes change over different architectures.
Signed-off-by: Paul Spooren <mail@aparcar.org>
With this commit, the download script will try downloading source files
using the filename instead of the url-filename in case the previous
download attempt using the url-filename failed.
This is required, as the OpenWrt sources mirrors serve files using the
filename files might be renamed to after downloading. If the original
mirror for a file where url-filename and filename do not match goes
down, the download failed prior to this patch.
Further improvement can be done by performing this only for the
OpenWrt sources mirrors.
Signed-off-by: David Bauer <mail@david-bauer.net>
Kernel has separated the structs that are reported to be const in
checkpatch.pl into a file of its own, const_structs.checkpatch.
This file has been missing after the recent update of checkpatch.pl,
leading to the following message:
No structs that should be const will be found - file
'/data/openwrt/scripts/const_structs.checkpatch': No such file
or directory
This commit adds the relevant file from v5.10-rc4.
Fixes: 086ee09bbc ("scripts: Update checkpatch.pl to 2020-06-11")
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
The kernel expects changes to MAINTAINERS for all removed or added
files, printing warnings like:
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#828:
deleted file mode 100644
Since this does not apply to "our" files in OpenWrt repo, this
warning should be disabled.
This can be achieved easiest by setting $reported_maintainer_file
to 1. While this is a hack that tricks the script into believing
the proper MAINTAINERS changes have been made, it's the easiest
solution as it does not require to touch any other code.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
Running the updated checkpatch version with spelling.txt show that
spelling mistakes happen to everyone.
> /target/linux$ fd .*.patch | xargs ../../scripts/checkpatch.pl | rg spell
WARNING: 'usefull' may be misspelled - perhaps 'useful'?
WARNING: 'afecting' may be misspelled - perhaps 'affecting'?
WARNING: 'usefull' may be misspelled - perhaps 'useful'?
WARNING: 'afecting' may be misspelled - perhaps 'affecting'?
WARNING: 'begining' may be misspelled - perhaps 'beginning'?
WARNING: 'superflous' may be misspelled - perhaps 'superfluous'?
WARNING: 'multipe' may be misspelled - perhaps 'multiple'?
WARNING: 'recieves' may be misspelled - perhaps 'receives'?
WARNING: 'retreive' may be misspelled - perhaps 'retrieve'?
WARNING: 'tranfer' may be misspelled - perhaps 'transfer'?
WARNING: 'additonal' may be misspelled - perhaps 'additional'?
WARNING: 'accomodate' may be misspelled - perhaps 'accommodate'?
[...]
Signed-off-by: Paul Spooren <mail@aparcar.org>
Remove checks for device tree documentation as the OpenWrt tree comes
withouth the ./Documentation folder.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Checkpatch grown in functionallity and we should make use of that. If
OpenWrt patches should be upstream material they should also be checked
based on upstream checkpatch.pl instead of 2013.
Signed-off-by: Paul Spooren <mail@aparcar.org>
The source date epoch is the only reproducible date close to the actual
build date. It can be used for tooling like the firmware wizard to show
the image age.
Signed-off-by: Paul Spooren <mail@aparcar.org>
mkhash currently returns the hash of an empty input when trying to hash
a folder. This can be missleading in caseswhere e.g. an env variable is
undefined which should contain a filename. `mkhash ./path/to/$FILE`
would exit with code 0 and return a legit looking checksum.
A better behaviour would be to fail with exit code 1, which imitates the
behaviour of `md5sum` and `sha256sum`.
To avoid hashing of folders the `stat()` is checked.
Hashing empty inputs result in the following checksums:
md5: d41d8cd98f00b204e9800998ecf8427e
sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Signed-off-by: Paul Spooren <mail@aparcar.org>
If hashing a file fails mkhash shouldn't just silently fail. Now check
after each call of `hash_file()` the return and exit early in case of
errors. The return value which was previously ignored and would always
return 0.
Signed-off-by: Paul Spooren <mail@aparcar.org>
The -n option prints the filename of hashed files next to the calculated
checksum. Reflect that in the usage message.
user@dawn:~/src/openwrt/openwrt$ ./a.out md5 -n .config
eb06db36e7b6751cb18801945e46bf5d .config
Signed-off-by: Paul Spooren <mail@aparcar.org>
Recent versions (> 5.33) of `file` report liblto_plugin.so as
executable:
$ file liblto_plugin.so
liblto_plugin.so.0.0.0: ELF 64-bit LSB pie executable ...
Which then leads to improper packaging of the plugin, resulting in the
broken compiler:
configure: checking whether the C compiler works
mips-openwrt-linux-musl/bin/ld: liblto_plugin.so: error loading plugin: liblto_plugin.so: invalid ELF header
As the LTO compiler plugin library is incorrectly packaged as SDK
executable:
$ head -1 ~/staging_dir/toolchain...libexec/gcc/.../liblto_plugin.so
#!/usr/bin/env bash
Fix this by filtering out shared libraries from the patching.
Ref: https://bugzilla.redhat.com/show_bug.cgi?id=1296868
Acked-by: Jo-Philipp Wich <jo@mein.io>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Shallow copies are quite common on CI platforms nowadays, making REBOOT
tag unavailable, thus producing following confusing errors in the build
logs:
fatal: Invalid revision range ee53a240ac902dc83209008a2671e7fdcf55957a..HEAD
fatal: Invalid revision range ee53a240ac902dc83209008a2671e7fdcf55957a..0493d57e04774d47921a7d2014b567455d5dc16b
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Explicitly set the QEMU cpu type and support the MIPS R2 ISA, for both
64-bit and 32-bit targets. The later previously supported MIPS R2 by
implicit default.
This is needed after commit 93608697f3 ("malta: update MIPS64 ISA to R2"),
otherwise booting malta images with scripts/qemustart will hang.
Signed-off-by: Tony Ambardar <itugrok@yahoo.com>
As the PWM has its own sub-system in the Linux kernel,
I think it should be handled in the same way as GPIO, RTC, PCI...
This patch introduces a specific feature flag "pwm" and the
"leds-pwm" kernel module as the first customer.
Signed-off-by: Christian Lamparter <chunkeey@gmail.com>
Some bootloaders are really keen on just one special
fdt in a multi-image fit image. This is a problem, because
currently this is fixed to "fdt@1".
This patch introduces a new device variable:
DEVICE_FDT_NUM that allows to specify the right
fdt number.
If the value is absent "1" will be chosen.
Signed-off-by: Christian Lamparter <chunkeey@gmail.com>
SPDX moved from GPL-2.0 to GPL-2.0-only and from GPL-2.0+ to
GPL-2.0-or-later. Reflect that in the SPDX license headers.
Signed-off-by: Paul Spooren <mail@aparcar.org>
[adjust commit title, update remaining files]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
Use the prepared .packageusergroup file to lookup user and group names
when processing the passed file mode.
Also replace the various subshell/cut invocations with a sequence of
standard variable interpolations which fixes paths with embedded colons
as a side-effect.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
With the introduction of `./tmp/userids` the `ipkg-build` script can now
resolve values of "PKG_FILE_MODES", allowing users to set names rather
than numeric values.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Since the introduction of fakeroot support, wrapped SDK executables might
be invoked from a shell that has libfakeroot.so preloaded.
Since we're using preloading as well in order to mangle argv[0] when
invoking the shipped ELF interpreter directly, we must take care of
preloading the already preloaded libraries as well, to avoid invoked
programs losing their fakeroot capabilities.
Extend the bundle-libraries.sh script to take any existing $LD_PRELOAD
into account when invoking the target ELF executable with a preloaded
runas.so library.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
With the new `SOURCE` argument of `get_source_date_epoch` it is possible
to set package timestamps based on actual package changes rather thane
$TOPDIR changes.
This commit adds a new variable PKG_SOURCE_DATE_EPOCH which is used by
the `ipkg` build script. As a fallback the existing SOURCE_DATE_EPOCH is
used or as last resort the current time.
The redundant checks for `.git/` and `.svn/` are removed.
Signed-off-by: Paul Spooren <mail@aparcar.org>
The SOURCE_DATE_EPOCH variable is used to make builds reproducible even
if rebuild at different times. Instead of using the current timestamp,
the time of the last source change is used.
Created packages are `touch`ed with a specific timestamp so resulting
packages have the same checksums.
The `get_source_date_epoch.sh` script tries multiple ways (file, git,
hg) to determine the correct timestamp.
Until now the script would only consider the $TOPDIR instead of package
specific changes. Resulting in packages with same versions but different
timestamps, as $TOPDIR (openwrt.git) received changes not affecting
package versions. This results in warning/erros in `opkg` as the package
versions stay the same but checksums changed.
This commit adds an optional argument to get the `SOURCE_DATE_EPOCH` of
a specific path (e.g. package SOURCE) rather than the $TOPDIR. As a
consequence this allows granular but still reproducible timestamps.
As packages might be distributed over multiple repositories the check
for `.git/` becomes unfeasible. Instead tell `git` and `hg` to change
their working directories and automatically traverse the repo folder.
Signed-off-by: Paul Spooren <mail@aparcar.org>
If a `cd` to `TOPDIR` fails the script should quit.
Also unify `try_mtime` function by storing it in a variable.
Signed-off-by: Paul Spooren <mail@aparcar.org>
The `ipkg-build` script converts a folder into a `opkg` installable
package. Until now it would use root:root for all packages and try to
preserve file modes.
This has the two drawbacks of packages want to add non-root files or add
SUID files, like the `sudo` package does.
To give more flexibility regarding file modes and avoid init script
hacks, a new variable called `PKG_FILE_MODES`. The variable contains a
list of files modes in the format `path:owner:group:mode`.
An example for the `sudo` package below:
```
PKG_FILE_MODES:=\
/usr/bin/sudo:root:root:4755 \
/etc/sudoers:root:root:0440
```
The `ipkg-build` now runs within a fakeroot environment to set any mode
and directly store it in the resulting `ipk` package archive.
Both options `-o` and `-g` are no longer required due to the introduction
of the more flexible `-m` options, which takes the `PKG_FILE_MODES` as
input.
Lastly the option `-c` is removed as it's unused within the script.
Signed-off-by: Paul Spooren <mail@aparcar.org>
This reverts commit c737a9ee6a.
The source CDN has been discontinued in its current form and will take a
while to be reestablished. Even then it makes little sense to put a CDN
before other CDNs such as kernel.org, apache.org, sourceforge etc.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Fix shellcheck SC2230
> which is non-standard. Use builtin 'command -v' instead.
Using `command -v` is POSIX compliant while `which` is not. Also to
mention, `command -v` is a shell builtin whereas `which` is a separate
busybox applet.
Once applied to everything concerning OpenWrt we can disable the busybox
feature `which` and save 3.8kB.
Acked-by: Stijn Tintel <stijn@linux-ipv6.be>
Signed-off-by: Paul Spooren <mail@aparcar.org>
[also replace cases in zram-swap]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
That file was added accidentally in v3 of the patch and I haven't
properly reviewed that patch before pushing it.
Fixes: 656b562aff ("scripts: Add Buildbot dump-target-info.pl script")
Signed-off-by: Petr Štetiar <ynezz@true.cz>
The script comes from buildbot.git[0] and is used to print available
targets and architectures, which are then build.
As the buildbot clones openwrt.git anyway, the script might as well live
here to be used for other cases as well, e.g. determining what
architectures are available when building Docker containers or show
developers an overview which architectures are used by which target.
It's called with either the parameter `architectures` or `targets`,
showing architectures followed by supported targets or targets, followed
by the supported architectures:
$ ./scripts/dump-target-info.pl architectures
aarch64_cortex-a53 bcm27xx/bcm2710 mediatek/mt7622 mvebu/cortexa53 sunxi/cortexa53
aarch64_cortex-a72 bcm27xx/bcm2711 mvebu/cortexa72
...
$ ./scripts/dump-target-info.pl targets
apm821xx/nand powerpc_464fp
apm821xx/sata powerpc_464fp
...
In the future the the script could be removed from the buildbot
repository and maintained only here.
Rename `dumpinfo.pl` to `dump-target-info.pl` to improve verbosity of
filename.
[0]: https://git.openwrt.org/?p=buildbot.git;a=blob;f=scripts/dumpinfo.pl;h=aa97f8d60379076a41b968402e9337cea824ece5;hb=HEAD
Signed-off-by: Paul Spooren <mail@aparcar.org>
checkpatch.pl uses a list of files to detect the root OpenWrt
directory. This includes README, which has been renamed to
README.md in the previous commit.
Update the file name in checkpatch.pl to prevent errors like the
following when running the script:
Must be run from the top-level dir. of a OpenWrt tree
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
By specifying "BROKEN := 1" or "BROKEN := y" for a device, it will be
hidden (and deselected) by default. By that, it provides a stronger
option to "disable" a device beyond just using DEFAULT := n.
To make these devices visible, just enable the BROKEN option in
developer settings as already implemented for targets and packages.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
The f-string feature was introduced in Python 3.6. As Buildbots may run
on Debian 9, which comes per default with Python 3.5, this would cause
an issue. Instead of f-strings use the *legacy* `.format()` function.
Signed-off-by: Paul Spooren <mail@aparcar.org>
We don't need to see how git will be executed and it produces non silent
output on 'scripts/env diff' commands when there are no differences
unlike before.
Re-introduce original silent behaviour.
Signed-off-by: Kevin Darbyshire-Bryant <ldir@darbyshire-bryant.me.uk>
Due to a line break, printf was accidentally called with three
arguments instead of two, causing a different output than before.
Fix it by splitting the printf command into two lines.
Fixes: 907053193a ("scripts/mkits.sh: replace echo -e with printf")
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
Fixes following shellcheck warning:
In scripts/mkits.sh line 19:
"-k kernel [-D name -d dtb] -o its_file" "$(basename $0)"
^-- SC2086: Double quote to prevent globbing and word splitting.
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Fixes following shellcheck warning:
In scripts/env line 25:
exit ${1:-1}
^-----^ SC2086: Double quote to prevent globbing and word splitting.
Signed-off-by: Petr Štetiar <ynezz@true.cz>
On a system python3 is linked to python3.6, fail to perform json_overview_image_info
and got `TypeError: __init__() got an unexpected keyword argument 'capture_output'`.
This patch emulate the behaviour on python 3.7+.
Signed-off-by: Huangbin Zhan <zhanhb88@gmail.com>
This updates mconf.c readme message to maintain less changes with
upstream and consistency with nconf.c
Signed-off-by: Sergio E. Nemirowski <sergio@outerface.net>
Add support for make target nconfig (ncurses)
Reviewed-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
Signed-off-by: Sergio E. Nemirowski <sergio@outerface.net>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Only collect arch_packages if actually generating any output.
Fixes: commit f09b9319 ("build,json: store arch_packages in profiles.json"(
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Signed-off-by: Paul Spooren <mail@aparcar.org>
The `arch_packages` contains the supported package architecture.
Previously it was necessary to parse the `Packages` index for the line
`Architecture:`, requiring both an additional parser and file download.
Signed-off-by: Paul Spooren <mail@aparcar.org>
With this commit the `profiles.json` contain both the target specific
`default_packages` as well as the device specific `device_packages` as a
array of strings.
This information is required for downstream projects like the various
web-based interactive firmware generators.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Signed-off-by: Paul Spooren <mail@aparcar.org>
Sercomm firmwares are encrypted with AES 256 CBC.
The key is generated with a custom algorithm from the firmware tag:
char key[32];
char version[32];
char iv[32];
char random[32];
char size[32];
Key must be generated with Sercomm's algorithm. However, the rest of the
header can be empty. IV and random are set to 0 on purpose.
Signed-off-by: Álvaro Fernández Rojas <noltari@gmail.com>
Signed-off-by: Rosen Penev <rosenp@gmail.com>
[cut out of patch with different subject]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
These are in if statements (where they should be), and echo output that is
not piped to anything. These should be safe.
Found with shellcheck.
Signed-off-by: Rosen Penev <rosenp@gmail.com>
This is a temporary commit to have 'make config-clean' remove
temporary files from the previous scripts/config version.
The .gitignore file is updated to deal with the old files as well.
Cc: Kevin Darbyshire-Bryant <ldir@darbyshire-bryant.me.uk>
Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
Major changes include:
- Much more readable reverse dependencies separated in groups
- Improved recursive dependency report
- More readable .config files: add comments to signal end of menus
- More warnings for incorrect Config.in entries, such as a 'choice'
default not contained in the 'choice'
- Hability to properly display pseudographics with non-latin locales
- Recursive dependencies can optionally be treated as errors
Changes from failed dcf3e63a35 attempt:
- Recursive dependencies are treated as warnings by default
- The option to treat them as errors is implemented as a command-line
flag to scripts/config/conf instead of a compile-time definition
- fixed handling of select with umnet dependencies
Cc: Petr Štetiar <ynezz@true.cz>
Cc: Jo-Philip Wich <jow@mein.io>
Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
OpenWrt now has a CDN for sources at sources.cdn.openwrt.org which
mirrors sources.openwrt.org.
Downloading sources outside Europe or US (mainland) could
result in low throughput, extremely slowing down the first compilation of
the build system.
This patch adds sources.cdn.openwrt.org as the first mirror to offer
worldwide fast download speeds by default. If the CDN goes down for
whatever reason, the script jumps to the next available mirror and
downloads requested files as before (in regional varying speed).
Signed-off-by: Paul Spooren <mail@aparcar.org>
Acked-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
The JSON `WORK_DIR` ($(KDIR)/json_info_files) is only created if the new
image generation methods from `image.mk` are used. However some targets
like `armvirt` do not use it yet, so the folder is never created.
The `json_overview_image_info.py` script used to raise an error if the
given `WORK_DIR` isn't a folder, however it should just notify about
missing JSON files.
This patch removes the Python assert and exists with code 0 even if no
JSON files were found, as this is not necessarily an error but simply
not yet implemented. Using `glob` on an not existing `Path` results in
an empty list, therefore the for loop won't run.
Signed-off-by: Paul Spooren <mail@aparcar.org>
CC: Petr Štetiar <ynezz@true.cz>
This reverts commit dcf3e63a35.
The kconfig update requires further testing and refinement until it can
remain in tree. Main problems are:
- Recursive deps are now fatal instead of a warning
- Previously legal syntax now leads to hard failures
- It fails all package builds since multiple days
The updated kconfig implementation needs to cope with the current status
quo in the various package feeds before we can reconsider it for master.
It is not desirable that single broken packages can hard-fail the entire
build pipeline.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
This reverts commit 3204430e38.
Reverting this commit in preparation for reverting
dcf3e63a35 ("build: scripts/config - update to kconfig-v5.6") which
introduces various unaddressed build breakages.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
This reverts commit 8514b6b42c.
Reverting this commit in preparation for reverting
dcf3e63a35 ("build: scripts/config - update to kconfig-v5.6") which
introduces various unaddressed build breakages.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
kconfig-v5.6 disallowed a bool symbol to select another symbol that
'depends on m' (i.e. can be only 'm' on 'n'). It is, in fact, an unmet
dependency to have set to 'y'. However, openwrt depends on the previous
behavior, to be able to build a package that can be a module or built-in
depending on a bool config. This restores the previous behavior.
Ref: https://forum.openwrt.org/t/wireless-fails-on-snapshot-r12900-kernel-5-4-on-c2600-ipq806x
Tested-by: Hannu Nyman <hannu.nyman@iki.fi>
Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
Signed-off-by: Petr Štetiar <ynezz@true.cz> [added forum ref]
This addes the option to treat recursive dependencies as warnings
instead of errors, by running make with WARN_RECURSIVE_DEP=1.
Note that the script/config targets will not get rebuilt when you add or
remove WARN_RECURSIVE_DEP while running make. One must run
'make config-clean' before building config with a different setting.
Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
Major changes include:
- Much more readable reverse dependencies separated in groups
- Improved recursive dependency report
- More readable .config files: add comments to signal end of menus
- More warnings for incorrect Config.in entries, such as a 'choice'
default not contained in the 'choice'
- Hability to properly display pseudographics with non-latin locales
- Recursive dependencies are now treated as errors - this should make
it harder for them to creep in.
Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
Instead of passing pkg-config location through a variable when building
qconf (make xconfig), prepend its parent directory to the PATH, as it is
being done for other conf targets.
Use a Makefile pattern rule to group all 'scripts/config/%onf'
(currently conf, mconf, qconf) targets in a single rule. Add -O2 to
CFLAGS when building them as well.
Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
JSON info files contain machine readable information of built profiles
and resulting images. These files were added in commit 881ed09ee6
("build: create JSON files containing image info").
They are useful for firmware wizards and script checking for
reproducibility.
Currently all JSON files are stored next to the built images, resulting
in up to 168 individual files for the ath79/generic target.
This patch refactors the JSON creation to store individual per image
(not per profile) files in $(BUILD_DIR)/json_info_files and create an
single overview file called `profiles.json` in the target directory.
Storing per image files and not per profile solves the problem of
parallel file writes. If a profiles sysupgrade and factory image are
finished at the same time both processes would write to the same JSON
file, resulting in randomly broken outputs.
Some target like x86/64 do not use the image code yet, resulting in
missing JSON files. If no JSON info files were created, no
`profiles.json` files is created as it would be empty anyway.
As before, this creation is enabled by default only if `BUILDBOT` is set.
Tested via buildroot & ImageBuilder on ath79/generic, imx6 and x86/64.
Signed-off-by: Paul Spooren <mail@aparcar.org>
[json_info_files dir handling in Make, if case refactoring]
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Add EFI platform bootable images for x86 platforms. These images can
also boot from legacy BIOS platform.
EFI System Partition need to be fat12/fat16/fat32 (not need to load
filesystem drivers), so the first partition of EFI images are not ext4
filesystem any more.
GPT partition table has an alternate partition table, we did not
generate it. This may cause problems when use these images as qemu disk
(kernel can not find rootfs), we pad enough sectors will be ok.
Signed-off-by: 李国 <uxgood.org@gmail.com>
[part_magic_* refactoring, removed genisoimage checks]
Signed-off-by: Petr Štetiar <ynezz@true.cz>
qemustart allows easy testing of created images via `qemu`. The script
automatically selects created images and can setup e.g. networks.
As the x86 target now uses the generic image.mk the profile appears also
in the image name, this is *generic*.
Add the profile name to the qemustart script so it still finds the file.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Currently, a symbolic link whose target is a directory will not be
removed when cleaning packages from STAGING_DIR.
In the first cleaning pass in scripts/clean-package.sh, the -f test for
a directory symlink returns false (because the link target is a
directory) and so the symlink is not removed.
In the second pass, the -d test returns true for a directory symlink,
but the symlink is not removed by rmdir because rmdir only removes
(real) directories.
This updates clean-package.sh to remove all non-directories (including
symbolic links) in the first pass.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This replaces deprecated backticks by more versatile $(...) syntax.
Found with shellcheck.
Signed-off-by: Rosen Penev <rosenp@gmail.com>
[adjust commit title and message]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
This replaces deprecated backticks by more versatile $(...) syntax.
Found with shellcheck.
Signed-off-by: Rosen Penev <rosenp@gmail.com>
[slightly adjusted commit title, added commit message]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
This replaces deprecated backticks by more versatile $(...) syntax.
Found with shellcheck.
Signed-off-by: Rosen Penev <rosenp@gmail.com>
[adjust commit title and message]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
apache mirrors holds only latest releases, to download
older releases, one must use archive.apache.org to get
them.
Signed-off-by: Jiri Kastner <cz172638@gmail.com>
UUID of ext4 volumes generated by make_ext4fs are determined by volume
label and it will all be 57f8f4bc-abf4-655f-bf67-946fc0f9f25b when label
is empty
Labeling them does not make them unique but tools like block command
from fstools have a better chance differentiating them
Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
Set the image sequence number to SOURCE_DATE_EPOCH.
If not set, ubinize will use rand() as image sequence.
Signed-off-by: Alexander Couzens <lynxis@fe80.eu>
This script allows image signing indipendend of the actual build
process, to run on a master server after receiving freshly backed
images. Idea is to avoid storying private keys on third party builders
while still beeing to be able to sign packages.
Run ./scripts/sign_images.sh with the following env vars:
* TOP_DIR where to search for sysupgrade.bin images
* BUILD_KEY place of key-build{,.pub,.ucert}
* REMOVE_OTHER_SIGNATURES removes signatures added by e.g. buildbots
Only sysupgrade.bin files are touched as factory.bin signatures wouldn't
be evaluated on stock from.
Signed-off-by: Paul Spooren <mail@aparcar.org>
The JSON info files contain details about the created firmware images
per device and are stored next to the created images.
The JSON files are stored as "$(IMAGE_PREFIX).json" and contain some
device/image meta data as well as a list of created firmware images.
An example of openwrt-ramips-rt305x-aztech_hw550-3g.json
{
"id": "aztech_hw550-3g",
"image_prefix": "openwrt-ramips-rt305x-aztech_hw550-3g",
"images": [
{
"name": "openwrt-ramips-rt305x-aztech_hw550-3g-squashfs-sysupgrade.bin",
"sha256": "db2b34b0ec4a83d9bf612cf66fab0dc3722b191cb9bedf111e5627a4298baf20",
"type": "sysupgrade"
}
],
"metadata_version": 1,
"supported_devices": [
"aztech,hw550-3g",
"hw550-3g"
],
"target": "ramips/rt305x",
"titles": [
{
"model": "HW550-3G",
"vendor": "Aztech"
},
{
"model": "ALL0239-3G",
"vendor": "Allnet"
}
],
"version_commit": "r10920+123-0cc87b3bac",
"version_number": "SNAPSHOT"
}
Signed-off-by: Paul Spooren <mail@aparcar.org>
`make xconfig` fails with following linking error of qconf binary:
g++ -lQt5Widgets -lQt5Gui -lQt5Core -o qconf qconf.o zconf.tab.o
/usr/bin/ld: qconf.o: in function ConfigList::metaObject() const': qconf.cc:(.text+0x3eb): undefined reference to QObjectData::dynamicMetaObject() const'
/usr/bin/ld: qconf.o: in function `ConfigList::qt_metacast(char const*)': link error.
which is caused by the wrong order of the linked objects/libraries so
this patch reorders the linker's arguments which makes the qconf compile
again.
Signed-off-by: leo chung <gewalalb@gmail.com>
[commit subject and message tweaks, whitespace fix]
Signed-off-by: Petr Štetiar <ynezz@true.cz>
On a recent Gentoo Linux installation, invoking `make menuconfig`, `make
kernel_menuconfig` or `make kernel_nconfig` in the build system fails,
whereas for example `make menuconfig` in the kernel tree alone works as
expected.
This is happening because STAGING_PREFIX is not defined when kernel's
{menu,n}config target calls pkg-config from the toolchain/host and thus
pkg-config returns an empty value, and the fallback values in the kernel
config script are applied but those are off and the linking fails.
Solution is to use system's pkg-config for all ncurses based menu config
targets in order to provide proper compiler/linker flags.
Ref: FS#2423
Cc: Thomas Albers <thomas.gameiro@gmail.com>
Tested-by: Stijn Tintel <stijn@linux-ipv6.be>
Tested-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
If no device tree is given there is no node generated, but
the configuration does still include the name of the missing node.
This will result in a successful build fit image, but bootm does
throw a error message if we want to boot the bad configuration.
Signed-off-by: Johann Neuhauser <jneuhauser@dh-electronics.com>
SquashFS has a minimum block size of at least 1k, so we need to make
sure the last data block is also at least that big.
This is not an issue on NOR or SD CARD devices, since their rootfs
partitions go all the way to the end of the usable space.
But on NAND with ubiblock, the rootfs partition will be the exact space,
rounded up to LEB size. Unfortunately, some NAND chips with small sub
pages have a LEB size of x.5 kiB. This can cause the the last data block
to be less than 1k, which will cause the last block to be inaccessible,
causing boot failures as seen on MR24:
[ 1.532960] block ubiblock0_3: created from ubi0:3(rootfs)
[ 1.538457] ubiblock: device ubiblock0_3 (rootfs) set to be root filesystem
[ 1.552847] SQUASHFS error: squashfs_read_data failed to read block 0x621472
[ 1.559896] squashfs: SQUASHFS error: unable to read id index table
[ 1.566474] VFS: Cannot open root device "(null)" or unknown-block(254,0): error -5
Since on most NOR devices, the start of the squashfs partition is not
aligned. Since the start of the rootfs_data partition there is dependend
on the SquashFS size, we cannot just always pad it, as the padding could
creep into the rootfs_data partition, breaking jffs2.
So fix this by ensuring a squashfs rootfs is always a multiple of 1k
only for UBI and NAND sysupgrade images.
Fixes#2460 without affecting NOR devices.
Tested-by: Russell Senior <russell@personaltelco.net>
Signed-off-by: Jonas Gorski <jonas.gorski@gmail.com>
Commit 775b70f8d5 renamed parse_file() parameters without
updating the recursive call. This broke parsing of any feeds.conf
using 'src-include'.
$ scripts/feeds update -a
Can't use string ("defaults") as a HASH ref while "strict refs" in use at scripts/feeds line 63, <$fh> line 1.
Fixes: 775b70f8d5 ("scripts/feeds: allow adding parameters to feeds")
Signed-off-by: Bjørn Mork <bjorn@mork.no>
When CONFIG_BUILD_SUFFIX is enabled, the target-* folders in build_dir
and staging_dir have this suffix in the name, but not the
toolchain directories. When detecting the names for "arch" and "libc",
also accept the suffix and do not use it for the toolchain path.
Signed-off-by: Thomas Langer <thomas.langer@intel.com>
this allows adding "--" prefixed parameters inside feeds.conf between the
target and name. The first parameter is --force which has the same effect
as using -f when installing any of the packages. This allows creating
feeds that will override base packages by default.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Allow overriding the default selection state for Devices, similar to
setting a default for packages.
E.g. by setting DEFAULT to n, they won't be selected by default anymore
when enabling all device in the multi device profile.
This allows preventing images being built by the default config for
known broken devices, devices without enough RAM/flash, or devices not
working with a certain kernel versions.
This does not prevent the devices from being manually selected or images
being built by the ImageBuilder. These devices often still have worth
with a reduced package-set, or as a device for regression testing, when
no better device is available.
Signed-off-by: Jonas Gorski <jonas.gorski@gmail.com>
New Docker based buildslaves install just bare minimum of packages, thus
not having bsdmainutils package installed which provides `hexdump`
utility, leading to the following build breakage on buildbots:
ubinize-image.sh: 12: /builder/scripts/ubinize-image.sh: hexdump: not found
So this patch simply replaces `hexdump` with `od` utility provided by
coreutils package, which should be likely available.
Co-authored-by: Yousong Zhou <yszhou4tech@gmail.com>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Having the build time written on stderr make it appear with V=w
although it is not an error or warning. Just write the time on stdout
to have it part of the build log like all the rest, but not clutter
the output when only warnings and errors should be shown.
Signed-off-by: Alban Bedel <albeu@free.fr>
The src-include method allows recursive inclusion of feeds.conf snippets.
This can for example be used for adding static local feeds to
feeds.conf.default without ever having to update the local feeds.conf:
src-include defaults feeds.conf.default
src-link custom /usr/local/src/lede/custom
Signed-off-by: Bjørn Mork <bjorn@mork.no>
Extend the packageauxvars database to keep a list of possible package
dependencies for each provider, then utilize this information in buildroot
to resolve the ABI version dependencies of dependent packages up to five
levels deep.
This should properly trigger rebuilds for packages indirectly depending
on other packages whose ABI_VERSION changed.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
The Makefile was missing dependencies on *c_shipped, so changes never
triggered a rebuild. Add these as optional dependencies so their absence
isn't treated as an error.
In addition, fix a typo preventing the zconf.lex.o from being removed on
clean.
Fixes: 9d5510a500 ("build: add new menuconfig code based on linux 3.9")
Signed-off-by: Jonas Gorski <jonas.gorski@gmail.com>
Acked-by: Jo-Philipp Wich <jo@mein.io>
When there are multiple packages providing a meta-package, it is
possible to to create a config where a package is selected as =y, but
all of its dependency providers are just selected as =m. This is due to
the selection statement being just
config PACKAGE_foo
select PACKAGE_bar if !PACKAGE_baz
which is already fulfilled by PACKAGE_bar=m. Fix this by properly
comparing the selection states:
config PACKAGE_foo
select PACKAGE_bar if PACKAGE_baz<PACKAGE_foo
Also invert the select conditions to improve readability.
Signed-off-by: Jonas Gorski <jonas.gorski@gmail.com>
[slightly reword commit message]
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Since commit 31847b67bec0 ("kconfig: allow use of relations other than
(in)equality") it is possible to use relational operators in Kconfig
statements. However, those operators give unexpected results when
applied to bool/tristate values:
(n < y) = y (correct)
(m < y) = y (correct)
(n < m) = n (wrong)
This happens because relational operators process bool and tristate
symbols as strings and m sorts before n. It makes little sense to do a
lexicographical compare on bool and tristate values though.
Documentation/kbuild/kconfig-language.txt states that expression can have
a value of 'n', 'm' or 'y' (or 0, 1, 2 respectively for calculations).
Let's make it so for relational comparisons with bool/tristate
expressions as well and document them. If at least one symbol is an
actual string then the lexicographical compare works just as before.
Signed-off-by: Nicolas Pitre <nico@linaro.org>
Acked-by: Randy Dunlap <rdunlap@infradead.org>
Tested-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
[rebased against OpenWrt kconfig, slightly reword commit message]
(backported from upstream 9059a3493efea6492451430c7e2fa0af799a2abb)
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
The pregenerated `zconf.hash.c` and `zconf.lex.c` files have not been
kept in sync with their respective `*.y` and `*.l` sources in the past
causing our kconfig implementation to not recognize important kconfig
grammer elements such as relational `<`, `<=`, `>` and `>=` operators.
Fixes: 2d7e602381 ("scripts/config: sync with latest linux upstream")
Signed-off-by: Jonas Gorski <jonas.gorski@gmail.com>
[reword commit message]
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Most of the patches are copied over from GCC 8.3.
The following patches are backported from the GCC 9.X development branch:
toolchain/gcc/patches/9.1.0/970-recompute-dom-fast-queries-before-vn.patch
toolchain/gcc/patches/9.1.0/975-g++-ICE-with-generic-lambda.patch
The specs file changed with gcc 9, now it contains "%@{L*}" instead of
"%{L*}" in older GCC versions.
Signed-off-by: Joseph Benden <joe@benden.us>
Adds support to && operand in DEPENDS. Also, fixes generation of ||
dependencies by scripts/package-metadata.pl.
The precedence order from higher to lower is && then ||. Use of
parentheses to change the order is not supported. As before, they are
silently ignored. Use them for readability only.
Signed-off-by: Eneas U de Queiroz <cote2004-github@yahoo.com>
Signed-off-by: Christian Lamparter <chunkeey@gmail.com> [DMARC removal]
When looping through a package's STAGING_FILES_LIST (a list of
file/directory paths delimited by newlines), if the path contains
spaces, then the path will be split by the while loops, and the
file/directory will not be deleted/removed.
This sets the internal field separator to the newline only so that the
entire path is considered when deleting/removing.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
If the target supports a newer kernel version that is not used by default
yet, it can be enabled with this option
Signed-off-by: Felix Fietkau <nbd@nbd.name>
We only enabled pcnet driver for malta machine, and x86/legacy target
was supposed to only support e1000 model
Cc: Petr Štetiar <ynezz@true.cz>
Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
This should make it easier to set up and possibly provide a "work out of
the box" experience for most test usage. Typical wan&lan networking
setup is only two characters argument away.
Cc: Petr Štetiar <ynezz@true.cz>
Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
Acked-by: Hans Dedecker <dedeckeh@gmail.com>
Commit 8bf500eb79 added support for qemu usage without networking
support but broke networking support as -n and -z do not work with
unquoted argument; fix this by quoting the arguments.
Fixes 8bf500eb79
Signed-off-by: Hans Dedecker <dedeckeh@gmail.com>
ar71xx is in the process of being deprecated as a target accepting new
devices. The replacement target for the same hardware is DTS
based ath79.
Switch the default build target selection from ar71xx to ath79.
This is intended to encourage DTS takeup & support for ath79 and longer
term will also aid kernel upstream support.
Signed-off-by: Kevin Darbyshire-Bryant <ldir@darbyshire-bryant.me.uk>
Currently it's not possible to test boot squashfs root images, so this
patch now allows this use case as well.
Cc: Yousong Zhou <yszhou4tech@gmail.com>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Currently it's not possible to test boot squashfs root images, so this
patch now allows this use case as well.
Cc: Yousong Zhou <yszhou4tech@gmail.com>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
For basic tests it's not necessary to have the networking setup and this
allows testing as a normal user as well, without root privileges.
So this patch adds `--no-network` long option or `-n` short option,
which allows starting QEMU without network.
Cc: Yousong Zhou <yszhou4tech@gmail.com>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
The script always exits with value 0, even if some of the commands fail.
This can potentially create broken, unbootable images, e.g. when
make_ext4fs fails due to TARGET_KERNEL_PARTSIZE being too small for the
kernel. Avoid this by failing the script when any command fails.
Acked-by: Jo-Philipp Wich <jo@mein.io>
Acked-by: Petr Štetiar <ynezz@true.cz>
Signed-off-by: Stijn Tintel <stijn@linux-ipv6.be>
Having image metadata (and signature) appended is a condition for
semi-automated sysupgrade, hence IB needs to be able to tell which
images will end up with metadata.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
The current eva_ramboot.py script is currently only compatible with
Lantiq based AVM devices. For IPQ40xx devices, the offset needs to be
changed. Also an alignment is not necessary here.
Adjust the script to be able to pass an optional offset to load the
image to. In case no offset is provided, the script behaves exactly as
before this commit.
Signed-off-by: David Bauer <mail@david-bauer.net>
Qualcomm SDK (QSDK) sysupgrade compatible images for IPQ40xx, IPQ806x
and IPQ807x use FIT format together with 'dumpimage' tool from U-Boot
for verifying and extracting them. Based on 'images' sections names,
corresponding mtd partitions are flashed. For example, in case of
NOR-only boards, below mapping is used (section name -> mtd name):
hlos* -> 0:HLOS
rootfs* -> rootfs
And for boards with NAND (kernel inside UBI):
ubi* -> rootfs
Above mappings come from unmodified QSDK sources and might be wrong for
boards running custom or modified QSDK-based firmware. Some of vendors
adjust them to meet their modified mtd layout or features like recovery
or dual-image support.
This adds simple script 'mkits-qsdk-ipq-image.sh' (based on 'mkits.sh')
for generating FIT images tree source files, compatible with the QSDK
sysupgrade format. Resulting images can be used for initial (factory ->
OpenWrt) installation and would work both in CLI and GUI.
The script is universal in a way it allows to include as many sections
as needed. To make use of it, two generic/basic build recipes for NOR
and NAND based boards are also included in 'image-commands.mk':
Build/qsdk-ipq-factory-nand
Build/qsdk-ipq-factory-nor
Example usage for board with UBI in NAND:
IMAGE/nand-factory.bin := append-ubi | qsdk-ipq-factory-nand
Signed-off-by: Piotr Dymacz <pepe2k@gmail.com>
This can be used to set machine options like highmem=off for running old
armvirt/32 kernel lacking LPAE support with QEMU version 3.0 or later
[1] Armv7 guest fails to boot with qemu-3.0.0-1,
https://bugzilla.redhat.com/show_bug.cgi?id=1633328#c5
Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
Use `stat -L` instead of `ls -l` to follow symbolic links when obtaining
the file size of .ipk archives.
Without this change, the size of the symlink, not the size of the target
file is encoded in the package index file.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
This fixes two issues with cleaning package files from STAGING_DIR:
* CleanStaging currently can only remove files and not directories. This
changes CleanStaging to use clean-package.sh, which does remove
directories.
* Because of the way directories are ordered in the staging files list,
clean-package.sh currently tries (and fails) to remove parent
directories before removing subdirectories. This changes
clean-package.sh to process the staging files list in reverse, so that
subdirectories are removed first.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This commit adds the 'Build/linksys-image' rule and the
'linksys-image.sh' script to the build system.
This change is needed for generating factory images for the Linksys
EA6350v3 device. Without this patch, only valid sysupgrade images can be
generated. With this patch, users can flash the device without the
need of physical access or disassembly.
Signed-off-by: Ryan Pannell <ryan@osukl.com>
Signed-off-by: Oever González <notengobattery@gmail.com>
When a library package specifies additional provides, e.g. libncurses
which provides libncursesw, we should also append the abi version
suffix to each provide, since there may be more than one package
providing the virtual library.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Package archives built before commit e6bcf1e4ac
("build: add ABI_VERSION to binary package names") lack the SourceName
control file field which caused ipkg-remove to skip such archives.
Add fallback code that matches the files by their basename followed by
an underscore, similar to how the old cleanup code worked.
Fixes: #2067
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Add the ABI_VERSION source makefile variable to the binary package basename
and resolve source dependencies on packages with ABI_VERSION set to such
expanded names.
If for example a package specifies DEPENDS:=libopenssl while the OpenSSL
Makefile specifies ABI_VERSION:=1.0.0, the resulting ipk control data
dependency will be "Depends: libopenssl1.0.0" and the libopenssl ipk file
will be called "libopenssl1.0.0_<version>_<arch>.ipk".
The next time a library such as OpenSSL is updated to an incompatible
version, the ABI_VERSION shall be changed accordingly to prevent opkg from
simply upgrading to an incompatible library without considering the
dependencies of already installed packages.
Also introduce another "SourceName" control field which is required by
the newly introduced "scritps/ipkg-remove" to determine the proper related
.ipk files to delete upon buildroot package clean operations.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Subdequent commits need this information to resolve the ABI version when
computing binary ipk dependencies.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Subsequent commits will put more auxiliary information into this file,
such as the per-package ABI version, so rename the metadata script
subcommand and file names accordingly.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Tar will automatically record the user/group ID and name for
files/directories. This reduces the reproducibility of the sysupgrade.tar
because most of the people don't use the same username as the OpenWrt
buildserver.
Signed-off-by: Sven Eckelmann <sven@narfation.org>
The file ordering in the current sysupgrade depends on the order of file in
a filesystem. This is often already in a sane (alphabetical order) but this
is not always the case. For example, the OpenWrt build servers return a
different ordering.
This breaks the reproducibility of the sysupgrade tarballs significantly
and also resulted in images which cannot be used for upgrades on devices
like the OpenMesh A42/A62.
Signed-off-by: Sven Eckelmann <sven@narfation.org>
2efe776 introduces rpath checks with the commit message:
> Remove all rpath entries which do not point to a location below /lib or
> /usr/lib and which do not begin with '$ORIGIN'.
However the implementation failed to support rpath entries with only the
'$ORIGIN' token and no trailing slash, so allow these as well.
Signed-off-by: Yorkie Liu <yazhong.liu@rokid.com>
[reword commit message]
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
When the SOURCE_DATE_EPOCH environment variable is set, use it to
override the timestamps of .ipk archive contents.
This ensures that .ipk archives built in environments without SCM
metadata (mainly the SDK) are reproducible between different runs.
Ref: https://github.com/openwrt/packages/issues/6954
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
This updates these two files to commit 2fa97a8a0ed3 ("config.guess
(amd64:CYGWIN*:*:*, x86_64:CYGWIN*:*:*): Set master") which is the
current master of
https://git.savannah.gnu.org/gitweb/?p=config.git;a=summary
This contains updates for multiple architectures and will unbreak the
build on the x32 ABI.
Signed-off-by: Hauke Mehrtens <hauke@hauke-m.de>
The OM2P(-HS)v4 got a variant which uses a slightly different flash. The
standard versions used a flash with 256KB blocks which is no longer
available. The replacement flash uses a flash with 64K blocks.
The padding for the image rootfs is already for 64K and 256K and thus can
be flashed on the device without any problems. Unfortunately, the
bootloader will check $rootfs_size (rounded down to the nearest 64k block)
minus 1x 64k. But it is now possible that the new JFFS2 rootfs_data starts
even earlier and modifies the checked region. The check will then fail and
the backup image (when available) will be booted.
Just setting it to the same number of skipped blocks as other 64K models
avoids this problem.
Signed-off-by: Sven Eckelmann <sven.eckelmann@openmesh.com>
When Package/xx/conffiles only contains directories that are empty at
package time, conffiles.resolved will be missing and the following
error messages will appear in the build log.
/home/yousong/git-repo/openwrt/openwrt/scripts/ipkg-build -c -o 0 -g 0 /home/yousong/git-repo/openwrt/openwrt/build_dir/target-mips_24kc_musl/linux-malta_be/openvswitch-2.10.0/ipkg-mips_24kc/openvswitch-common /home/yousong/git-repo/openwrt/openwrt/bin/packages/mips_24kc/packages
mv: cannot stat 'CONTROL/conffiles.resolved': No such file or directory
chmod: cannot access 'CONTROL/conffiles': No such file or directory
It will not break the ipkg-build process. The change is merely cosmetic
to not cause confusion when reading logs
Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
The need arises from building Open vSwitch kernel datapath modules, e.g.
- kmod-openvswitch from Linux upstream
- kmod-openvswitch-intree from openvswitch source code
where both provides virtual package "kmod-openvswitch" for userspace
packages to select and depend on
Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
Previous refactoring of the script moved the LDSO detection into a
file-not-exists condition, causing onyl the very first executable to
get bundled.
Solve the problem by unconditionally checking for LDSO again.
Fixes: 9030a78a71 ("scripts: bundle-libraries: prevent loading host locales")
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
The current make-ras.sh image generation script for the ZyXEL NBG6617
has portability issues with bash. Because of this, factory images are
currently not built correctly by the OpenWRT buildbots.
This commit replaces the make-ras.sh by C-written mkrasimage.
The new mkrasimage is also compatible with other ZyXEL devices using
the ras image-format.
This is not tested with the NBG6616 but it correctly builds the
header for ZyXEL factory image.
Signed-off-by: David Bauer <mail@david-bauer.net>
Binary patch the bundled glibc library to inhibit loading of host locale
archives in order to avoid triggering internal libc assertions when
invoking shipped, bundled executables.
The problem has been solved with upstream Glibc commit
0062ace229 ("Gracefully handle incompatible locale data") but we still
need to deal with older Glibc binaries for some time to come.
Fixes FS#1803
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Use Time::HiRes when available and fallback to raw syscall interface
when not. If that fails too, simply report 0, 0 as real time.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Replace the GNU time program invocation with a simple Perl script reporting
the timing values. Since we require Perl anyway for the build system, we can
as well use that instead of requiring a random GNU utility rarely installed
by default.
Fixes: ff6e62b288 ("build: log time taken by each packages/steps")
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Compiler is producing the warning:
warning: format not a string literal and no format arguments
[-Wformat-security]
This patch makes the format a literal string in printf statements.
This with: gcc version 7.3.0 (Ubuntu 7.3.0-16ubuntu3) in Ubuntu 18.04
Signed-off-by: Paul Schulz <paul@mawsonlakes.org>
Allow enabling/commenting/disabling each feed individually by using a
tristate config symbol.
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
The src-dummy method does not actually obtain any feed, but it can be used
to insert addtional entries into the opkg distfeeds.conf. This is useful to
make package feeds available to users without requiring the corresponding
source feeds to be available during build.
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
- Make the code more GitHub-specific
- Requires mirror hash to work with .gitattributes
- Use different API depending on whether PKG_SOURCE_VERSION is a
complete commit id or other ref types like tags
- Fix removing symbolic link
- pre-clean dir_untar for possible leftovers from previous run
Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
Move SCAN_DEPS to scan.mk to eliminate redundancy with scripts/feeds
Add image/*.mk to SCAN_DEPS for targets to pick up newly added devices
Signed-off-by: Felix Fietkau <nbd@nbd.name>
A new python script scripts/download.py is added to fetch tarballs using
GitHub archive API [1], then repack in a reproducible way same as the
current DownloadMethod/git
GitHub imposes a 60 reqs/hour rate limit on unauthenticated API
access[2]. This affects fetching commit date for feeding tar --mtime=
argument. However, observation indicates that archive download is NOT
subject to this limit at the moment. In the rare cases where download
fails because of this, we will falback to using DownloadMethod/git
The missing piece in the GitHub API is that it cannot provide in the
tarball dependent submodules's source code. In that case, the
implementation will also fallback to using DownloadMethod/git
[1] Get archive link, https://developer.github.com/v3/repos/contents/#get-archive-link
[2] Rate limiting, https://developer.github.com/v3/#rate-limiting
v2 <- v1:
- allow passing multiple urls with --urls argument
- add commit ts cache. can be helpful on retry
Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
This patch adds support for ZyXEL NBG6617
Hardware highlights:
SOC: IPQ4018 / QCA Dakota
CPU: Quad-Core ARMv7 Processor rev 5 (v7l) Cortex-A7
DRAM: 256 MiB DDR3L-1600/1866 Nanya NT5CC128M16IP-DI @ 537 MHz
NOR: 32 MiB Macronix MX25L25635F
ETH: Qualcomm Atheros QCA8075 Gigabit Switch (4 x LAN, 1 x WAN)
USB: 1 x 3.0 (via Synopsys DesignWare DWC3 controller in the SoC)
WLAN1: Qualcomm Atheros QCA4018 2.4GHz 802.11bgn 2:2x2
WLAN2: Qualcomm Atheros QCA4018 5GHz 802.11a/n/ac 2:2x2
INPUT: RESET Button, WIFI/Rfkill Togglebutton, WPS Button
LEDS: Power, WAN, LAN 1-4, WLAN 2.4GHz, WLAN 5GHz, USB, WPS
Serial:
WARNING: The serial port needs a TTL/RS-232 3.3v level converter!
The Serial setting is 115200-8-N-1. The 1x4 .1" header comes
pre-soldered. Pinout:
1. 3v3 (Label printed on the PCB), 2. RX, 3. GND, 4. TX
first install / debricking / restore stock:
0. Have a PC running a tftp-server @ 192.168.1.99/24
1. connect the PC to any LAN-Ports
2. put the openwrt...-factory.bin (or V1.00(ABCT.X).bin for stock) file
into the tftp-server root directory and rename it to just "ras.bin".
3. power-cycle the router and hold down the the WPS button (for 30sek)
4. Wait (for a long time - the serial console provides some progress
reports. The u-boot says it best: "Please be patient".
5. Once the power LED starts to flashes slowly and the USB + WPS LEDs
flashes fast at the same time. You have to reboot the device and
it should then come right up.
Installation via Web-UI:
0. Connect a PC to the powered-on router. It will assign your PC a
IP-address via DHCP
1. Access the Web-UI at 192.168.1.1 (Default Passwort: 1234)
2. Go to the "Expert Mode"
3. Under "Maintenance", select "Firmware-Upgrade"
4. Upload the OpenWRT factory image
5. Wait for the Device to finish.
It will reboot into OpenWRT without any additional actions needed.
To open the ZyXEL NBG6617:
0. remove the four rubber feet glued on the backside
1. remove the four philips screws and pry open the top cover
(by applying force between the plastic top housing from the
backside/lan-port side)
Access the real u-boot shell:
ZyXEL uses a proprietary loader/shell on top of u-boot: "ZyXEL zloader v2.02"
When the device is starting up, the user can enter the the loader shell
by simply pressing a key within the 3 seconds once the following string
appears on the serial console:
| Hit any key to stop autoboot: 3
The user is then dropped to a locked shell.
|NBG6617> HELP
|ATEN x[,y] set BootExtension Debug Flag (y=password)
|ATSE x show the seed of password generator
|ATSH dump manufacturer related data in ROM
|ATRT [x,y,z,u] RAM read/write test (x=level, y=start addr, z=end addr, u=iterations)
|ATGO boot up whole system
|ATUR x upgrade RAS image (filename)
|NBG6617>
In order to escape/unlock a password challenge has to be passed.
Note: the value is dynamic! you have to calculate your own!
First use ATSE $MODELNAME (MODELNAME is the hostname in u-boot env)
to get the challange value/seed.
|NBG6617> ATSE NBG6617
|012345678901
This seed/value can be converted to the password with the help of this
bash script (Thanks to http://www.adslayuda.com/Zyxel650-9.html authors):
- tool.sh -
ror32() {
echo $(( ($1 >> $2) | (($1 << (32 - $2) & (2**32-1)) ) ))
}
v="0x$1"
a="0x${v:2:6}"
b=$(( $a + 0x10F0A563))
c=$(( 0x${v:12:14} & 7 ))
p=$(( $(ror32 $b $c) ^ $a ))
printf "ATEN 1,%X\n" $p
- end of tool.sh -
|# bash ./tool.sh 012345678901
|
|ATEN 1,879C711
copy and paste the result into the shell to unlock zloader.
|NBG6617> ATEN 1,0046B0017430
If the entered code was correct the shell will change to
use the ATGU command to enter the real u-boot shell.
|NBG6617> ATGU
|NBG6617#
Co-authored-by: David Bauer <mail@david-bauer.net>
Signed-off-by: Christian Lamparter <chunkeey@googlemail.com>
Signed-off-by: David Bauer <mail@david-bauer.net>
When sending script/env diff's output to a file or pipeline, you
don't want escape characters for ANSI color sequences fouling the
byte stream.
Signed-off-by: Philip Prindeville <philipp@redfish-solutions.com>
SourceForge has supported HTTPS for its downloads for a long time now.
I have not been able to see any failures resulting from this change.
Signed-off-by: Rosen Penev <rosenp@gmail.com>
This allegedly fixes compilation of the library bundler preload library on
Apple OS X. The resulting executables have not been runtime tested due to a
lack of suitable test hardware.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
* QCA IPQ4019
* 256 MB of RAM
* 32 MB of SPI NOR flash (s25fl256s1)
- 2x 15 MB available; but one of the 15 MB regions is the recovery image
* 2T2R 2.4 GHz
- QCA4019 hw1.0 (SoC)
- requires special BDF in QCA4019/hw1.0/board-2.bin with
bus=ahb,bmi-chip-id=0,bmi-board-id=20,variant=OM-A62
* 2T2R 5 GHz (channel 36-64)
- QCA9888 hw2.0 (PCI)
- requires special BDF in QCA9888/hw2.0/board-2.bin
bus=pci,bmi-chip-id=0,bmi-board-id=16,variant=OM-A62
* 2T2R 5 GHz (channel 100-165)
- QCA4019 hw1.0 (SoC)
- requires special BDF in QCA4019/hw1.0/board-2.bin with
bus=ahb,bmi-chip-id=0,bmi-board-id=21,variant=OM-A62
* multi-color LED (controlled via red/green/blue GPIOs)
* 1x button (reset; kmod-input-gpio-keys compatible)
* external watchdog
- triggered GPIO
* 1x USB (xHCI)
* TTL pins are on board (arrow points to VCC, then follows: GND, TX, RX)
* 2x gigabit ethernet
- phy@mdio3:
+ Label: Ethernet 1
+ gmac0 (ethaddr) in original firmware
+ 802.3at POE+
- phy@mdio4:
+ Label: Ethernet 2
+ gmac1 (eth1addr) in original firmware
+ 18-24V passive POE (mode B)
* powered only via POE
The tool ap51-flash (https://github.com/ap51-flash/ap51-flash) should be
used to transfer the factory image to the u-boot when the device boots up.
The initramfs image can be started using
setenv bootargs 'loglevel=8 earlycon=msm_serial_dm,0x78af000 console=ttyMSM0,115200 mtdparts=spi0.0:256k(0:SBL1),128k(0:MIBIB),384k(0:QSEE),64k(0:CDT),64k(0:DDRPARAMS),64k(0:APPSBLENV),512k(0:APPSBL),64k(0:ART),64k(0:custom),64k(0:KEYS),15552k(inactive),15552k(inactive2)'
tftpboot 0x84000000 openwrt-ipq40xx-openmesh_a62-initramfs-fit-uImage.itb
set fdt_high 0x85000000
bootm 0x84000000
Signed-off-by: Sven Eckelmann <sven.eckelmann@openmesh.com>
Use the EVA bootloader to load a small linux system into the ram and boot
it from there:
./scripts/flashing/eva_ramboot.py 192.168.178.1 path/to/initramfs-kernel.bin
Signed-off-by: Valentin Spreckels <Valentin.Spreckels@Informatik.Uni-Oldenburg.DE>
Acked-by: John Crispin <john@phrozen.org>
[reworded commit message]
Signed-off-by: Mathias Kresin <dev@kresin.me>
Some devices only boot when a special config is found in the image and
completely ignore the default entry during the selection. These devices can
now use the variable DEVICE_DTS_CONFIG in their device image definition.
Signed-off-by: Sven Eckelmann <sven@narfation.org>
/lib/functions.sh can deal with Require-User specifications that only
contain a group, but no user. Adjust metadata.pm to allow such
specifications as well.
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
``make xconfig`` toplevel target will invoke ``make qconf`` inside
./scripts/config directory, which results a ``qconf`` executable.
This commit removes leftover ``qconf`` executable during ``make
config-clean``.
Signed-off-by: Alif M. Ahmad <alive4ever@live.com>
Remove references to /etc/, /lib/ and /usr/ from the bundled ld.so
interpreter using simple binary patching.
This is needed to prevent loading host system libraries such as
libnss_compat.so.2 on foreign systems, which may result in ld.so
inconsistency assertions.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Prevent emitting broken dependency statements when resolving references
to non existing packages to an empty provider list.
Fixes: 47d6b05ad3 ("metadata: always resolve dependencies through provides list")
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
The logic for choice between source and binary packages was reversed.
Fixes: 52719c2b67 "metadata: scripts/feeds: distinguish between source
and binary packages, resolve virtual dependencies"
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
Package "features" seem to be unused for some time. In any case, custom
Config.in snippets and package PROVIDES are a much more flexible way to
express similar options.
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
Properly resolve build depends to source packages and runtime depends to
binary packages. Dependencies on virtual packages are resolved to the first
provider now.
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
Instead of adding virtual packages to the normal package list, keep a
separate list for provides, make each package provide itself, and resolve
all dependencies through this list. This allows to use PROVIDES to replace
existing packages.
Fixes FS#837.
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
Runtime depends cannot have a buildtype suffix, and they never refer to
source package names. In addition, this adds warnings about unsatisfiable
dependencies.
Furthermore, this change fixes the generation of conditional build
dependencies for virtual packages provided by different source packages.
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
Target build depends are similar to host build depends in that they refer
to source packages rather than binary packages. Therefore, it makes sense
to handle them together, rather than putting them in a list together with
runtime depends and trying to figure out if the entries refer to source or
to binary packages afterwards.
This does lead to PKG_BUILD_DEPENDS entries referring to binary package
names not working anymore, which requires some fixes in the package
repositories.
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
This feature has been unused for years, and its scope is too limited to be
actually useful.
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
Every single reference to subdir was concatenated with the source package
name, so it makes sense to store the concatenated value instead.
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
