Commit Graph

799 Commits

Author SHA1 Message Date
Paul Spooren
2cb6d471b7 scripts: fixup qemustart for new x86 image names
qemustart allows easy testing of created images via `qemu`. The script
automatically selects created images and can setup e.g. networks.

As the x86 target now uses the generic image.mk the profile appears also
in the image name, this is *generic*.

Add the profile name to the qemustart script so it still finds the file.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-03-21 10:36:00 +00:00
Jeffery To
f96cfe019a build: Fix directory symlinks not removed when cleaning STAGING_DIR
Currently, a symbolic link whose target is a directory will not be
removed when cleaning packages from STAGING_DIR.

In the first cleaning pass in scripts/clean-package.sh, the -f test for
a directory symlink returns false (because the link target is a
directory) and so the symlink is not removed.

In the second pass, the -d test returns true for a directory symlink,
but the symlink is not removed by rmdir because rmdir only removes
(real) directories.

This updates clean-package.sh to remove all non-directories (including
symbolic links) in the first pass.

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
2020-03-01 21:35:59 +01:00
Rosen Penev
e37e2f0cf1 scripts/arm-magic.sh: switch to /bin/sh
This does not use any special bash stuff.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2020-01-02 23:59:20 +01:00
Rosen Penev
995378a2a5 scripts/env: replace -a and -o with &&/||
The former are not well defined.

Found with shellcheck.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2020-01-02 23:56:19 +01:00
Rosen Penev
2fe5319d58 scripts/env: use explicit find location
Some find binaries do not imply the current directory.

Found with shellcheck.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2020-01-02 23:51:45 +01:00
Rosen Penev
33f87d2ef6 scripts/gen-dependencies.sh: use /bin/sh
This uses no special bash stuff.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2020-01-01 17:01:02 +01:00
Rosen Penev
dec46a9ea0 scripts/gen-dependencies.sh: replace backticks with $()
Found with shellcheck.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2020-01-01 17:00:38 +01:00
Rosen Penev
9a8ba44f9c scripts/mkits.sh: replace legacy backticks with $()
This replaces deprecated backticks by more versatile $(...) syntax.

Found with shellcheck.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
[adjust commit title and message]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2019-12-31 11:44:04 +01:00
Rosen Penev
543d945f07 scripts/symlink-tree.sh: use /bin/sh
This uses nothing bash specific.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2019-12-31 11:43:15 +01:00
Adrian Schmutzler
ec0fb23a41 scripts/strip-kmod.sh: harmonize leading whitespaces
Convert leading spaces to tabs for consistency in the file.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2019-12-31 11:41:07 +01:00
Rosen Penev
f4b812dac1 scripts/strip-kmod.sh: use sh instead of bash
There's nothing bash specific here.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2019-12-31 11:40:24 +01:00
Rosen Penev
eae889b8f2 scripts/strip-kmod.sh: replace legacy backticks with $()
This replaces deprecated backticks by more versatile $(...) syntax.

Found with shellcheck.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
[slightly adjusted commit title, added commit message]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2019-12-31 11:38:06 +01:00
Rosen Penev
3f65fe5d62 scripts/gen_image_generic.sh: replace deprecated backticks with $()
This replaces deprecated backticks by more versatile $(...) syntax.

Found with shellcheck.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
[adjust commit title and message]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2019-12-31 01:06:20 +01:00
Rosen Penev
2e14dd23c9 scripts/gen_image_generic.sh: remove $ in arithmetic
Fixes shellcheck warning:

SC2004: $/${} is unnecessary on arithmetic variables.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2019-12-31 01:05:27 +01:00
Jiri Kastner
dc34c695c4 scripts/dowload.pl: add archive.apache.org to apache mirror list
apache mirrors holds only latest releases, to download
older releases, one must use archive.apache.org to get
them.

Signed-off-by: Jiri Kastner <cz172638@gmail.com>
2019-12-23 00:21:46 +01:00
Yousong Zhou
d26738bc76 scripts/dl_github_archive.py: fix python3 str, bytes confusion
Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
2019-11-04 11:11:19 +00:00
Yousong Zhou
66ebca7103 build: label kernel and rootfs ext4 volumes
UUID of ext4 volumes generated by make_ext4fs are determined by volume
label and it will all be 57f8f4bc-abf4-655f-bf67-946fc0f9f25b when label
is empty

Labeling them does not make them unique but tools like block command
from fstools have a better chance differentiating them

Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
2019-11-03 04:25:40 +00:00
Alexander Couzens
5625a02e77
ubinize-image: set image sequence from SOURCE_DATE_EPOCH
Set the image sequence number to SOURCE_DATE_EPOCH.
If not set, ubinize will use rand() as image sequence.

Signed-off-by: Alexander Couzens <lynxis@fe80.eu>
2019-11-03 04:20:21 +01:00
Paul Spooren
2ae5100d70 build: add script to sign packages
This script allows image signing indipendend of the actual build
process, to run on a master server after receiving freshly backed
images. Idea is to avoid storying private keys on third party builders
while still beeing to be able to sign packages.

Run ./scripts/sign_images.sh with the following env vars:

* TOP_DIR where to search for sysupgrade.bin images
* BUILD_KEY place of key-build{,.pub,.ucert}
* REMOVE_OTHER_SIGNATURES removes signatures added by e.g. buildbots

Only sysupgrade.bin files are touched as factory.bin signatures wouldn't
be evaluated on stock from.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-10-21 14:07:08 +02:00
Paul Spooren
881ed09ee6 build: create JSON files containing image info
The JSON info files contain details about the created firmware images
per device and are stored next to the created images.

The JSON files are stored as "$(IMAGE_PREFIX).json" and contain some
device/image meta data as well as a list of created firmware images.

An example of openwrt-ramips-rt305x-aztech_hw550-3g.json

    {
      "id": "aztech_hw550-3g",
      "image_prefix": "openwrt-ramips-rt305x-aztech_hw550-3g",
      "images": [
        {
          "name": "openwrt-ramips-rt305x-aztech_hw550-3g-squashfs-sysupgrade.bin",
          "sha256": "db2b34b0ec4a83d9bf612cf66fab0dc3722b191cb9bedf111e5627a4298baf20",
          "type": "sysupgrade"
        }
      ],
      "metadata_version": 1,
      "supported_devices": [
        "aztech,hw550-3g",
        "hw550-3g"
      ],
      "target": "ramips/rt305x",
      "titles": [
        {
          "model": "HW550-3G",
          "vendor": "Aztech"
        },
        {
          "model": "ALL0239-3G",
          "vendor": "Allnet"
        }
      ],
      "version_commit": "r10920+123-0cc87b3bac",
      "version_number": "SNAPSHOT"
    }

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-09-29 13:51:28 +02:00
leo chung
b2c55d50f8 build: fix xconfig target
`make xconfig` fails with following linking error of qconf binary:

 g++ -lQt5Widgets -lQt5Gui -lQt5Core -o qconf qconf.o zconf.tab.o
 /usr/bin/ld: qconf.o: in function ConfigList::metaObject() const': qconf.cc:(.text+0x3eb): undefined reference to QObjectData::dynamicMetaObject() const'
 /usr/bin/ld: qconf.o: in function `ConfigList::qt_metacast(char const*)': link error.

which is caused by the wrong order of the linked objects/libraries so
this patch reorders the linker's arguments which makes the qconf compile
again.

Signed-off-by: leo chung <gewalalb@gmail.com>
[commit subject and message tweaks, whitespace fix]
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-09-25 12:50:24 +02:00
Petr Štetiar
965f341aa9 build: fix host menu config targets using ncurses
On a recent Gentoo Linux installation, invoking `make menuconfig`, `make
kernel_menuconfig` or `make kernel_nconfig` in the build system fails,
whereas for example `make menuconfig` in the kernel tree alone works as
expected.

This is happening because STAGING_PREFIX is not defined when kernel's
{menu,n}config target calls pkg-config from the toolchain/host and thus
pkg-config returns an empty value, and the fallback values in the kernel
config script are applied but those are off and the linking fails.

Solution is to use system's pkg-config for all ncurses based menu config
targets in order to provide proper compiler/linker flags.

Ref: FS#2423
Cc: Thomas Albers <thomas.gameiro@gmail.com>
Tested-by: Stijn Tintel <stijn@linux-ipv6.be>
Tested-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-09-25 12:50:24 +02:00
Johann Neuhauser
205e0939f0 build: make device tree arg really optional in mkits.sh
If no device tree is given there is no node generated, but
the configuration does still include the name of the missing node.
This will result in a successful build fit image, but bootm does
throw a error message if we want to boot the bad configuration.

Signed-off-by: Johann Neuhauser <jneuhauser@dh-electronics.com>
2019-09-19 23:43:27 +02:00
Yousong Zhou
18dc533172 download.dl: fix @KERNEL mirror urls
- Prepend "/pub" for mirror.rackspace.com
 - Use https for download.xs4all.nl and mirrors.mit.edu

Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
2019-09-16 08:11:59 +00:00
Felix Fietkau
d5cd80a1e8 scripts/feeds: fix accepting "-" in feed type string
Fixes a syntax error in processing the type src-git-full

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2019-09-15 19:54:03 +02:00
Jonas Gorski
ee76bd11bb images: fix boot failures on NAND with small sub pages
SquashFS has a minimum block size of at least 1k, so we need to make
sure the last data block is also at least that big.

This is not an issue on NOR or SD CARD devices, since their rootfs
partitions go all the way to the end of the usable space.

But on NAND with ubiblock, the rootfs partition will be the exact space,
rounded up to LEB size. Unfortunately, some NAND chips with small sub
pages have a LEB size of x.5 kiB. This can cause the the last data block
to be less than 1k, which will cause the last block to be inaccessible,
causing boot failures as seen on MR24:

[    1.532960] block ubiblock0_3: created from ubi0:3(rootfs)
[    1.538457] ubiblock: device ubiblock0_3 (rootfs) set to be root filesystem
[    1.552847] SQUASHFS error: squashfs_read_data failed to read block 0x621472
[    1.559896] squashfs: SQUASHFS error: unable to read id index table
[    1.566474] VFS: Cannot open root device "(null)" or unknown-block(254,0): error -5

Since on most NOR devices, the start of the squashfs partition is not
aligned. Since the start of the rootfs_data partition there is dependend
on the SquashFS size, we cannot just always pad it, as the padding could
creep into the rootfs_data partition, breaking jffs2.

So fix this by ensuring a squashfs rootfs is always a multiple of 1k
only for UBI and NAND sysupgrade images.

Fixes #2460 without affecting NOR devices.

Tested-by: Russell Senior <russell@personaltelco.net>
Signed-off-by: Jonas Gorski <jonas.gorski@gmail.com>
2019-09-14 11:43:19 +02:00
Bjørn Mork
a21b70be31 scripts/feeds: fix 'src-include' directive
Commit 775b70f8d5 renamed parse_file() parameters without
updating the recursive call. This broke parsing of any feeds.conf
using 'src-include'.

 $ scripts/feeds update -a
 Can't use string ("defaults") as a HASH ref while "strict refs" in use at scripts/feeds line 63, <$fh> line 1.

Fixes: 775b70f8d5 ("scripts/feeds: allow adding parameters to feeds")
Signed-off-by: Bjørn Mork <bjorn@mork.no>
2019-09-02 16:48:40 +02:00
Thomas Langer
035906fd05 Fix handling of BUILD_SUFFIX in remote-gdb script
When CONFIG_BUILD_SUFFIX is enabled, the target-* folders in build_dir
and staging_dir have this suffix in the name, but not the
toolchain directories. When detecting the names for "arch" and "libc",
also accept the suffix and do not use it for the toolchain path.

Signed-off-by: Thomas Langer <thomas.langer@intel.com>
2019-09-01 18:38:05 +02:00
Jo-Philipp Wich
775b70f8d5 scripts/feeds: allow adding parameters to feeds
this allows adding "--" prefixed parameters inside feeds.conf between the
target and name. The first parameter is --force which has the same effect
as using -f when installing any of the packages. This allows creating
feeds that will override base packages by default.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2019-08-28 16:10:41 +02:00
Yousong Zhou
164037983d scripts/dl_github_archive.py: fix python3 transition
Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
2019-08-20 13:34:58 +00:00
Yousong Zhou
0f3667864d download.pl: use https://source.openwrt.org
https://sources.lede-openwrt.org now redirects to there

https://downloads.openwrt.org/sources returns 404, so remove it here

Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
2019-08-18 15:19:22 +00:00
Jonas Gorski
7546be6007 build: allow overriding default selection state for devices
Allow overriding the default selection state for Devices, similar to
setting a default for packages.

E.g. by setting DEFAULT to n, they won't be selected by default anymore
when enabling all device in the multi device profile.

This allows preventing images being built by the default config for
known broken devices, devices without enough RAM/flash, or devices not
working with a certain kernel versions.

This does not prevent the devices from being manually selected or images
being built by the ImageBuilder. These devices often still have worth
with a reduced package-set, or as a device for regression testing, when
no better device is available.

Signed-off-by: Jonas Gorski <jonas.gorski@gmail.com>
2019-08-13 22:13:18 +02:00
Petr Štetiar
c6d41c320c scripts/ubinize-image.sh: fix buildbot breakage
New Docker based buildslaves install just bare minimum of packages, thus
not having bsdmainutils package installed which provides `hexdump`
utility, leading to the following build breakage on buildbots:

 ubinize-image.sh: 12: /builder/scripts/ubinize-image.sh: hexdump: not found

So this patch simply replaces `hexdump` with `od` utility provided by
coreutils package, which should be likely available.

Co-authored-by: Yousong Zhou <yszhou4tech@gmail.com>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-07-30 10:16:16 +02:00
Petr Štetiar
57bb89b57f scripts/jungo-image: convert to Python 3 with 2-to-3
Let's convert the script to Python 3.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-07-26 08:09:16 +02:00
Petr Štetiar
e785da815b scripts/dl_github_archive.py: convert to Python 3 with 2-to-3
Let's convert the script to Python 3.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-07-26 08:09:16 +02:00
Alban Bedel
340df72e07 scripts: time.pl: Don't print the time on stderr
Having the build time written on stderr make it appear with V=w
although it is not an error or warning. Just write the time on stdout
to have it part of the build log like all the rest, but not clutter
the output when only warnings and errors should be shown.

Signed-off-by: Alban Bedel <albeu@free.fr>
2019-07-03 07:45:00 +02:00
Bjørn Mork
7a1b575ac4 scripts/feeds: add src-include method
The src-include method allows recursive inclusion of feeds.conf snippets.

This can for example be used for adding static local feeds to
feeds.conf.default without ever having to update the local feeds.conf:

 src-include defaults feeds.conf.default
 src-link custom /usr/local/src/lede/custom

Signed-off-by: Bjørn Mork <bjorn@mork.no>
2019-07-01 16:18:44 +02:00
Jo-Philipp Wich
27a4a71c24 metadata: handle ABI version rebuild tracking for transient dependencies
Extend the packageauxvars database to keep a list of possible package
dependencies for each provider, then utilize this information in buildroot
to resolve the ABI version dependencies of dependent packages up to five
levels deep.

This should properly trigger rebuilds for packages indirectly depending
on other packages whose ABI_VERSION changed.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2019-06-21 12:03:43 +02:00
Jonas Gorski
0096a1cf00 scripts/config: fix *c_shipped build depency tracking
The Makefile was missing dependencies on *c_shipped, so changes never
triggered a rebuild. Add these as optional dependencies so their absence
isn't treated as an error.

In addition, fix a typo preventing the zconf.lex.o from being removed on
clean.

Fixes: 9d5510a500 ("build: add new menuconfig code based on linux 3.9")
Signed-off-by: Jonas Gorski <jonas.gorski@gmail.com>
Acked-by: Jo-Philipp Wich <jo@mein.io>
2019-06-20 17:11:07 +02:00
Jonas Gorski
1fd50531ca metadata: ensure one dependency provider to be y if a package is y
When there are multiple packages providing a meta-package, it is
possible to to create a config where a package is selected as =y, but
all of its dependency providers are just selected as =m. This is due to
the selection statement being just

  config PACKAGE_foo
    select PACKAGE_bar if !PACKAGE_baz

which is already fulfilled by PACKAGE_bar=m. Fix this by properly
comparing the selection states:

  config PACKAGE_foo
    select PACKAGE_bar if PACKAGE_baz<PACKAGE_foo

Also invert the select conditions to improve readability.

Signed-off-by: Jonas Gorski <jonas.gorski@gmail.com>
[slightly reword commit message]
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2019-06-20 14:18:25 +02:00
Nicolas Pitre
75dcaf3d23 config: fix relational operators for bool and tristate symbols
Since commit 31847b67bec0 ("kconfig: allow use of relations other than
(in)equality") it is possible to use relational operators in Kconfig
statements. However, those operators give unexpected results when
applied to bool/tristate values:

	(n < y) = y (correct)
	(m < y) = y (correct)
	(n < m) = n (wrong)

This happens because relational operators process bool and tristate
symbols as strings and m sorts before n. It makes little sense to do a
lexicographical compare on bool and tristate values though.

Documentation/kbuild/kconfig-language.txt states that expression can have
a value of 'n', 'm' or 'y' (or 0, 1, 2 respectively for calculations).
Let's make it so for relational comparisons with bool/tristate
expressions as well and document them. If at least one symbol is an
actual string then the lexicographical compare works just as before.

Signed-off-by: Nicolas Pitre <nico@linaro.org>
Acked-by: Randy Dunlap <rdunlap@infradead.org>
Tested-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
[rebased against OpenWrt kconfig, slightly reword commit message]
(backported from upstream 9059a3493efea6492451430c7e2fa0af799a2abb)
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2019-06-20 14:14:16 +02:00
Jonas Gorski
972123f1e0 config: regenerate *_shipped sources
The pregenerated `zconf.hash.c` and `zconf.lex.c` files have not been
kept in sync with their respective `*.y` and `*.l` sources in the past
causing our kconfig implementation to not recognize important kconfig
grammer elements such as relational `<`, `<=`, `>` and `>=` operators.

Fixes: 2d7e602381 ("scripts/config: sync with latest linux upstream")
Signed-off-by: Jonas Gorski <jonas.gorski@gmail.com>
[reword commit message]
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2019-06-20 14:14:16 +02:00
Joseph Benden
88c07c6552 toolchain: Add GCC 9.1.0 release
Most of the patches are copied over from GCC 8.3.

The following patches are backported from the GCC 9.X development branch:
toolchain/gcc/patches/9.1.0/970-recompute-dom-fast-queries-before-vn.patch
toolchain/gcc/patches/9.1.0/975-g++-ICE-with-generic-lambda.patch

The specs file changed with gcc 9, now it contains "%@{L*}" instead of
"%{L*}" in older GCC versions.

Signed-off-by: Joseph Benden <joe@benden.us>
2019-06-16 16:40:08 +02:00
Eneas U de Queiroz
a41f474d17 build: add support to && in DEPENDS
Adds support to && operand in DEPENDS.  Also, fixes generation of ||
dependencies by scripts/package-metadata.pl.

The precedence order from higher to lower is && then ||.  Use of
parentheses to change the order is not supported. As before, they are
silently ignored.  Use them for readability only.

Signed-off-by: Eneas U de Queiroz <cote2004-github@yahoo.com>
Signed-off-by: Christian Lamparter <chunkeey@gmail.com> [DMARC removal]
2019-05-31 11:21:22 +02:00
Jeffery To
3fcb709275 build: fix STAGING_DIR cleaning when filenames contain spaces
When looping through a package's STAGING_FILES_LIST (a list of
file/directory paths delimited by newlines), if the path contains
spaces, then the path will be split by the while loops, and the
file/directory will not be deleted/removed.

This sets the internal field separator to the newline only so that the
entire path is considered when deleting/removing.

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
2019-05-17 21:41:43 +02:00
Felix Fietkau
85017c40f4 build: add a config option for enabling a testing version of the target kernel
If the target supports a newer kernel version that is not used by default
yet, it can be enabled with this option

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2019-05-11 11:37:10 +02:00
Yousong Zhou
c2fee615ce scripts/qemustart: add notes on adding new options
Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
2019-04-26 13:09:33 +00:00
Yousong Zhou
d726878546 scripts/qemustart: fix network device emulation
We only enabled pcnet driver for malta machine, and x86/legacy target
was supposed to only support e1000 model

Cc: Petr Štetiar <ynezz@true.cz>
Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
2019-04-26 13:09:33 +00:00
Yousong Zhou
647356a3d2 scripts/qemustart: no network by default and enable with -n
This should make it easier to set up and possibly provide a "work out of
the box" experience for most test usage.  Typical wan&lan networking
setup is only two characters argument away.

Cc: Petr Štetiar <ynezz@true.cz>
Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
Acked-by: Hans Dedecker <dedeckeh@gmail.com>
2019-04-26 13:08:38 +00:00
Hans Dedecker
9424b6f998 scripts/qemustart: fix usage with networking support
Commit 8bf500eb79 added support for qemu usage without networking
support but broke networking support as -n and -z do not work with
unquoted argument; fix this by quoting the arguments.

Fixes 8bf500eb79

Signed-off-by: Hans Dedecker <dedeckeh@gmail.com>
2019-04-24 20:02:54 +02:00