Commit Graph

1047 Commits

Author SHA1 Message Date
Felix Fietkau
0bdf8d1206 scripts/feeds: fix installing targets without explicitly specifying the feed
Add similar code to what is done on packages

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2022-01-24 13:29:05 +01:00
Ansuel Smith
88204bfa82 treewide: drop use of which
Ubuntu started to flag which as deprecated and it
seems which is not really standard and may vary
across Distro.
Drop the use of which and use the standard 'command -v'
for this simple task.
Which is still present in the prereq if some package/script
still use which.
A utility script called command_all.sh is implemented that
will just mimic the output of which -a.

Signed-off-by: Ansuel Smith <ansuelsmth@gmail.com>
2022-01-17 09:14:26 +01:00
Paul Spooren
02de391b08 scripts: dump-target-info print kernel versions
This commits adds the ability to print Kernel versions of all
targets/subtargets. If a testing Kernel is set print that version as
well.

Example output:

    apm821xx/nand 5.10
    apm821xx/sata 5.10
    arc770/generic 5.4
    archs38/generic 5.4
    armvirt/32 5.10
    armvirt/64 5.10
    at91/sam9x 5.10
    at91/sama5 5.10
    ath25/generic 5.4
    ath79/generic 5.4 5.10
    ath79/mikrotik 5.4 5.10
    --- %< ---

This should help to get a quick update on the state of Kernels.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-12-29 14:06:10 +01:00
Enrico Mioso
57c1f3f9c5
base-files: add eMMC sysupgrade support
Adds generic support for sysupgrading on eMMC-based devices.

Provide function emmc_do_upgrade and emmc_copy_config to be used in
/lib/upgrade/platform.sh instead of redundantly implementing the same
logic over and over again.
Similar to generic sysupgrade on NAND, use environment variables
CI_KERNPART, CI_ROOTPART and newly introduce CI_DATAPART to indicate
GPT partition names to be used. On devices with more than one MMC
block device, CI_ROOTDEV can be used to specify the MMC device for
partition name lookups.

Also allow to select block devices directly using EMMC_KERN_DEV,
EMMC_ROOT_DEV and EMMC_DATA_DEV, as using GPT partition names is not
always an option (e.g. when forced to use MBR).

To easily handle writing kernel and rootfs make use of sysupgrade.tar
format convention which is also already used for generic NAND support.

Signed-off-by: Enrico Mioso <mrkiko.rs@gmail.com>
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
CC: Li Zhang <li.zhang@gl-inet.com>
CC: TruongSinh Tran-Nguyen <i@truongsinh.pro>
2021-12-02 20:42:58 +00:00
Robert Marko
5a305e429f scripts: ubinize-image: make rootfs optional
Currently ubinize-image script always expects the
rootfs image to be passed and a volume for it created.

So, to allow only ubinizing a kernel for example which
the MikroTik hAP ac3 and other new NAND devices from
MikroTik require make rootfs an optional parameter like
kernel.

Signed-off-by: Robert Marko <robimarko@gmail.com>
2021-11-28 17:17:22 +01:00
Felix Fietkau
db34b93331 toolchain: add a version that can be bumped to force toolchain/target rebuild
This can be used to ensure that the compiled code is up to date, when
something important changes in the toolchain.
A recent example of this is the gcc 11 fix for a code miscompilation issue

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2021-11-16 22:02:45 +01:00
Arne Zachlod
b5ab6d0bc5 scripts/env: fix env for git conf init.defaultBranch not set to "master"
Since version 2.28, git has a config option init.defaultBranch to set the name
of the first branch created with git init. The env script expects this name to
be "master". This commit sets the initial branch name to "master"
instead of using the git configured one.

Signed-off-by: Arne Zachlod <arne@nerdkeller.org>
2021-11-13 10:29:41 -10:00
Josh Soref
08622de7d6 scripts: fix various typos
This only affects typos in comments or user-facing output.

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
[only picks changes to scripts, drop "commandline" replacement,
 fix case for "arbitrary", improve commit message]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2021-10-31 21:24:47 +01:00
Jan-Niklas Burfeind
88a2ea41da scripts: eva_ramboot.py: remove unused import
concludes:
commit e7bc8984d9 ("scripts: make eva_ramboot.py offset configurable")

Signed-off-by: Jan-Niklas Burfeind <git@aiyionpri.me>
2021-10-30 15:00:22 +02:00
Daniel Golle
b2aca61360
base-files, metadata: support additional group membership
Some packages may require additional group membership for the system
user added by that package. Allow defining additional groups as third
member of the ':'-separated tuple, allowing to specify multiple
','-separated groups with optional GID.

Example:
USERID:=foouser=1000:foogroup=1000:addg1=1001,addg2=1002,addg3

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2021-10-28 16:27:21 +01:00
Hans Dedecker
abc7a97e9c Revert "ethtool: update to v5.14"
This reverts commit 7630001427

Signed-off-by: Hans Dedecker <dedeckeh@gmail.com>
2021-10-23 21:39:00 +02:00
Hans Dedecker
7630001427 ethtool: update to v5.14
Update to newly released version 5.14

Signed-off-by: Hans Dedecker <dedeckeh@gmail.com>
2021-10-23 21:35:38 +02:00
Felix Fietkau
b45ff2d023 scripts/rstrip.sh: do not strip .o files with STRIP_KMOD
Fixes packaging of eBPF modules

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2021-10-11 11:49:14 +02:00
Paul Spooren
3128dfc18a scripts: package-metadata add pkgmanifestjson call
The new `pkgmanifestjson` call prints all package manifest of a feed in
JSON format. This function can be used to print an overview of packages
information used for downstream tooling.

The script is entirely based on Petrs work on dependency visualisation.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-09-23 21:57:22 -10:00
Paul Spooren
108901fffb scripts: store maintainer in package metadata
The maintainer could be usable for downstream tooling, so start storing
it in the metadata.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-09-23 21:38:59 -10:00
Damien Mascord
fd67908647 scripts: mkits.sh: Allow legacy @ mode for dts creation
commit 5ec60cbe9d ("scripts: mkits.sh: replace @ with - in nodes")
broke support for Meraki MR32 and this patch makes the replacement
configurable allowing for specifying the @ or - or whatever character
that is desired to retain backwards compatibility with existing devices.

For example, this patch includes the fix for the Meraki MR32 in
target/linux/bcm53xx/image for meraki_mr32:

  DEVICE_DTS_DELIMITER := @
  DEVICE_DTS_CONFIG := config@1

Fixes: 5ec60cbe9d ("scripts: mkits.sh: replace @ with - in nodes")
Signed-off-by: Damien Mascord <tusker@tusker.org>
[Added tags, checkpatch.pl fixes, noted that this is for old stuff]
Signed-off-by: Christian Lamparter <chunkeey@gmail.com>
2021-09-23 20:49:32 +02:00
Ansuel Smith
93f488fc37 scripts: handle gcc and linux in dl_cleanup script
Handle gcc and linux with a special regex that set their progname with
their major version. This way every minor version can be cleared. The
build cleanup logic can be tweaked later to clean the entire toolchain
and target dir with a different gcc version.

Signed-off-by: Ansuel Smith <ansuelsmth@gmail.com>
[reformat commit message]
Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-09-22 22:24:39 -10:00
Ansuel Smith
ff875876da scripts: format dl_cleanup to black format python style
Forma dl_cleanup python script to black style.

Signed-off-by: Ansuel Smith <ansuelsmth@gmail.com>
2021-09-22 19:54:00 -10:00
Ansuel Smith
4eb4c3c469 scripts: add missing regex for dl_cleanup script
Regex xxx-YYYY-MM-DD-GIT_SHASUM was missing. Add the new regex to improve
and better find outdated package. This also fix a bug where some bug were
incorrectly detected as packagename-yyyy-mm-dd instead of packagename due
to them be parsed by the wrong parser

Example:

    openwrt-keyring-2021-02-20-49283916.tar.xz

Signed-off-by: Ansuel Smith <ansuelsmth@gmail.com>
[added example in commit message]
Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-09-22 19:51:44 -10:00
Paul Spooren
5ef4608c02 build: store artifacts in JSON
Multiple profiles create artifacts, these should be stored in the JSON
file as well, allowing downstream tooling to show those files, too.

Artifacts don't have specific filesystems so only the fields `name`,
`type` and `sha256` are available.

Rename env variable names from IMAGE_ to FILE_ prefixes to reflect that
images, kernels and artifacts are added with the same command.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-09-14 10:42:38 -10:00
Daniel Golle
98bccdafd7
base-files: rename 'sdcard' to 'legacy-sdcard'
While an image layout based on MBR and 'bootfs' partition may be easy
to understand for users who are very used to the IBM PC and always have
the option to access the SD card outside of the device (and hence don't
really depend on other recovery methods or dual-boot), in my opinion
it's a dead end for many desirable features on embedded systems,
especially when managed remotely (and hence without an easy option to
access the SD card using another device in case things go wrong, for
example).

Let me explain:

* using a MSDOS/VFAT filesystem to store kernel(s) is problematic, as a
  single corruption of the bootfs can render the system into a state
  that it no longer boots at all. This makes dual-boot useless, or at
  least very tedious to setup with then 2 independent boot partitions
  to avoid the single point of failure on a "hot" block (the FAT index
  of the boot partition, written every time a file is changed in
  bootfs). And well: most targets even store the bootloader environment
  in a file in that very same FAT filesystem, hence it cannot be used
  to script a reliable dual-boot method (as loading the environment
  itself will already fail if the filesystem is corrupted).

* loading the kernel uImage from bootfs and using rootfs inside an
  additional partition means the bootloader can only validate the
  kernel -- if rootfs is broken or corrupted, this can lead to a reboot
  loop, which is often a quite costly thing to happen in terms of
  hardware lifetime.

* imitating MBR-boot behavior with a FAT-formatted bootfs partition
  (like IBM PC in the 80s and 90s) is just one of many choices on
  embedded targets. There are much better options with modern U-Boot
  (which is what we use and build from source for all targets booting
  off SD cards), see examples in mediatek/mt7622 and mediatek/mt7623.

Hence rename the 'sdcard' feature to 'legacy-sdcard', and prefix
functions with 'legacy_sdcard_' instead of 'sdcard_'.

Tested-by: Stijn Tintel <stijn@linux-ipv6.be>
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2021-08-16 12:22:17 +01:00
Stijn Tintel
8347d8b4fb base-files: add generic sdcard upgrade method
Add a generic sdcard upgrade method instead of duplicating code in yet
another target, and add a feature flag to only install this upgrade
method in targets that set this flag. Copied from mvebu.

Signed-off-by: Stijn Tintel <stijn@linux-ipv6.be>
2021-08-07 01:34:40 +03:00
Stijn Tintel
84c9570aaa scripts/target-metadata.pl: order features alphabetically
We generaly do this in many other places, so let's do it here also.

Signed-off-by: Stijn Tintel <stijn@linux-ipv6.be>
2021-08-07 01:30:37 +03:00
Yonghyu Ban
7270d768da scripts/mkits.sh: Fix the hash algorithm paramter
The mkits.sh script help message states hash algorithm can be
specified using the -H command-line option, but it does not work
currently due to a bug in the script.

This patch fixes this problem by changing the option from -S to
-H and specify getopts parameter after it

Signed-off-by: Yonghyu Ban <yonghyu@empo.im>
2021-07-18 19:37:30 +01:00
Ansuel Smith
64ce35bfaf scripts: check if dl directory exist in dl_cleanup script
Check if the provided dl directory exist and return on error.

Signed-off-by: Ansuel Smith <ansuelsmth@gmail.com>
2021-07-05 14:15:58 -10:00
Paul Spooren
fd0d9909bf build,json: fix generation with empty profiles
If the image generation doesn't add any profiles to the output the
*profile merge* will fail. To avoid that set an empty profile as
fallback.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-06-20 22:42:29 -10:00
Moritz Warning
a463b96241 build: preserve profiles.json between builds
Keep other profiles.json content if the data belongs to the current
build version.

Also useful for the ImageBuilder, which builds for a single model each
time. Without this commit the profiles.json would only contain the
latest build profile information.

Signed-off-by: Moritz Warning <moritzwarning@web.de>
[improve commit message]
Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-06-16 21:28:53 -10:00
Karel Kočí
1aa80ce393 scripts/feeds: generate index after all feeds are updated
This separates index update from feed update. The result is that all
requested feeds are first updated and only then indexed.

The reason for this change is to prevent errors being reported and
potentially invalid index being generated thanks to cross feeds
dependency.
The feeds script pulls in default all feeds as they come and on install
prefers packages from first feeds (unless special feed is requested).
Thus order of feeds in some way specifies preferences. This is handy for
downstream distributions as they can simply override any package from
upstream feeds by placing their feed before them. This removes need to
patch or fork upstream feeds.
The problem is that such feed most likely depends in some way also on
subsequent feeds. The most likely feeds are 'packages' or 'luci'. The
example would be Python package that needs 'python.mk' from 'packages'
feed. Ordering custom feed after dependent feeds is sometimes just not
possible because of preference requirement described before.
The solution is to just first pull all feeds and generate indexes only
after that. In the end this ensures that index is generated correctly at
first try without any error.

In terms of code this removes 'perform_update' argument from
'update_feed' as with index update removal the update is the only action
performed in that subroutine. Thus this moves condition to 'update'
subroutine.

Signed-off-by: Karel Kočí <karel.koci@nic.cz>
2021-06-07 12:44:50 -10:00
Paul Spooren
ec4852f731 scripts: config.guess: update to 2021-05-24
This script hasn't seen an update in multiple years, update it to the
latest version provided upstream. Both `config.guess` and `config.sub`
are copied from upstream[1] and not modified.

The full changelog is available within the upstream repository[1].

[1]: https://git.savannah.gnu.org/git/config.git

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-05-29 13:03:07 +02:00
Leonardo Mörlein
b993b68b6c build: introduce $(MKHASH)
Before this commit, it was assumed that mkhash is in the PATH. While
this was fine for the normal build workflow, this led to some issues if

    make TOPDIR="$(pwd)" -C "$pkgdir" compile

was called manually. In most of the cases, I just saw warnings like this:

    make: Entering directory '/home/.../package/gluon-status-page'
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    [...]

While these were only warnings and the package still compiled sucessfully,
I also observed that some package even fail to build because of this.

After applying this commit, the variable $(MKHASH) is introduced. This
variable points to $(STAGING_DIR_HOST)/bin/mkhash, which is always the
correct path.

Signed-off-by: Leonardo Mörlein <me@irrelefant.net>
2021-05-13 15:13:15 +02:00
Paul Spooren
7880a64848 build,json: 3rd fixup of default_packages
This became a bit of a tragedy, caused by a corner cases which wasn't
put into account during testing. DEFAULT_PACKAGES are defined in
target/linux/<target>/Makefile but also in
target/linux/<target>/<subtarget>/target.mk.

The latter was no longer imported when using DUMP=1, however not using
DUMP=1 while running the Makefile in target/linux/<target>/ caused duplicate
packages in the list.

As a solution, which should have been used from day 0, `make` runs in
target/linux/ without DUMP=1, resulting in no duplicate packages and all
inclusions from include/target.mk, linux/target/<target>/{Makefile,
<subtarget>/target.mk}

While at it, sort the list of default packages.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-03-25 10:25:54 -10:00
Paul Spooren
b36068d35d build,json: fixup fixup of arch_packages
The commit "1bf2b3fe90 build,json: fixup missing arch_packages" fixes
the missing package architecture locally but runs $(TOPDIR)/Makefile
rather than a target specific one. While this works on local builds just
fine, it causes the buildbots to add garbage to the `arch_packages`
variable:

    cd \"/builder/shared-workdir/build\"; git log --format=%h -1
    toolchain > /builder/shared-workdir/build/tmp/.ver_check\ncmp -s
    /builder/shared-workdir/build/tmp/.ver_check
    /builder/shared-workdir/build/staging_dir/toolchain-x86_64_gcc-8.4.0_musl/stamp/.ver_check
    || { \\\n\trm -rf
    /builder/shared-workdir/build/build_dir/target-x86_64_musl
    /builder/shared-workdir/build/staging_dir/target-x86_64_musl
    /builder/shared-workdir/build/staging_dir/toolchain-x86_64_gcc-8.4.0_musl
    /builder/shared-workdir/build/build_dir/toolchain-x86_64_gcc-8.4.0_musl;
    \\\n\tmkdir -p
    /builder/shared-workdir/build/staging_dir/toolchain-x86_64_gcc-8.4.0_musl/stamp;
    \\\n\tmv /builder/shared-workdir/build/tmp/.ver_check
    /builder/shared-workdir/build/staging_dir/toolchain-x86_64_gcc-8.4.0_musl/stamp/.ver_check;
    \\\n}\nx86_64

Only the last line contains the desired string.

Future investigation should check why the build system prints this to
stdout rather than stderr.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-03-24 19:03:24 -10:00
Paul Spooren
1bf2b3fe90 build,json: fixup missing arch_packages
Fix 7f4c2b1 "build,json: fix duplicates in default_packages" which
removed duplicate default packages but also removed the package
architecture from the profiles.json.

If DUMP=1 is set, the `ARCH_PACKAGES` is no longer exported and
therefore empty. Fix this by running make twice, once with DUMP=1 and
once without.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-03-24 13:41:34 -10:00
Paul Spooren
7f4c2b1a4f build,json: fix duplicates in default_packages
Calling without the DUMP=1 argument causes the target specific Makefile
to be "included" again which adds the target specific packages twice,
once on the actual run and once included from `include/target.mk`.

This led to duplicate package entries, causing confusion in downstream
projects using the generated JSON files.

While at it, apply `black` style to Python script.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-03-20 23:58:52 -10:00
Daniel Golle
6890f6fe13 include/image*: add support for device-tree overlays
Add new target feature 'dt-overlay' which makes DTC keep the symbol
names in the generated dtb.
Make sure additional DT overlay sources specified by the new device
variable DEVICE_DTS_OVERLAY get compiled together with the main DTS
(currently overlays got to be in the same folder). Let Build/fit pass
the generated DT overlay blobs to mkits.sh.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2021-03-17 17:45:34 +00:00
Daniel Golle
4e6de4f093 scripts/mkits.sh: add support for adding DT overlay blobs to image
Allow adding multiple device tree overlay blobs to an image and
generate configurations for each of them.
This is useful on boards with modern U-Boot which allow e.g. user-
configurable peripherals ("shields") in that way.
Note that currently, each generated configuration adds exactly one
overlay on top of the base image, ie. adding multiple overlays at the
same time is not yet supported.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2021-03-17 17:41:24 +00:00
Paul Spooren
bb95be9265 scripts,ipkg-build: use realpath for pkg_dir
This allows manual execution of the ipkg-build script even with
releative path.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-03-14 19:32:30 -10:00
Paul Spooren
72f481b157 scripts,ipkg-build: apply shellcheck
This commit cleans the `ipkg-build` script via changes suggested by
shellcheck. These are mostly word splitting issues.

Remove the definition of GZIP, this adds three "lookups" of the `gzip`
binary but the rest of the build system doesn't seem to use such
improvements neither.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-03-14 19:31:43 -10:00
Robert Marko
5ec60cbe9d scripts: mkits.sh: replace @ with - in nodes
U-boot will reject the nodes with @ for the address since
commit:
79af75f777

This in turn will cause the failure to boot with OpenWrt
generated images.

So, to rectify that simply replace @ with -.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
(cover also newly added rootfs@1 and initrd@1 nodes)
Signed-off-by: Robert Marko <robert.marko@sartura.hr>
2021-03-05 15:45:33 +00:00
Daniel Golle
ebcb4f1d0a
treewide: fix spelling 'seperate' -> 'separate'
This popular spelling mistake was also introduced by myself lately.
Fix it everywhere.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2021-02-28 23:59:21 +00:00
David Bauer
9a9cf40dd9 download: add mirror alias for Debian
Add an alias for Debian packages and download them from the Debian
mirror redirector.

Signed-off-by: David Bauer <mail@david-bauer.net>
2021-02-26 20:41:00 +01:00
David Bauer
01c01d9861 download: use mirror redirector for GNOME downloads
Use the GNOME mirror redirector as the primary download source for GNOME
packages.

Signed-off-by: David Bauer <mail@david-bauer.net>
2021-02-26 20:41:00 +01:00
Adrian Schmutzler
ef2cb8572b treewide: rename IMAGE_PREFIX/IMAGE_NAME to DEVICE_IMG_*
We so far had two variables IMG_PREFIX and IMAGE_PREFIX with
different content. Since these names are obviously quite
confusing, this patch renames the latter to DEVICE_IMG_PREFIX,
as it's a device-dependent variable, while IMG_PREFIX is only
(sub)target-dependent.

For consistency, also rename IMAGE_NAME to DEVICE_IMG_NAME, as
that's a device-dependent variable as well.

Cc: Paul Spooren <mail@aparcar.org>
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2021-02-25 18:11:06 +01:00
Daniel Golle
330bd380e8 image: allow building FIT and uImage with ramdisk
Instead of embedding the initrd cpio archive into the kernel, allow
for having an external ramdisk added to the FIT or uImage.
This is useful to overcome kernel size limitations present in many
stock bootloaders, as the ramdisk is then loaded seperately and doesn't
add to the kernel size. Hence we can have larger ramdisks to host ie.
installers with all binaries to flash included (or a web-based
firmware selector).
In terms of performance and total size the differences are neglectible.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2021-02-24 01:35:20 +00:00
Daniel Golle
e6aac8d98f image: add support for building FIT image with filesystem
Allow for single (external-data) FIT image to hold kernel, dtb and
squashfs. In that way, the bootloader verifies the system integrity
including the rootfs, because what's the point of checking that the
hash of the kernel is correct if it won't boot in case of squashfs
being corrupted? Better allow bootloader to check everything needed
to make it at least up to failsafe mode. As a positive side effect
this change also makes the sysupgrade process on nand potentially
much easier as it is now.
In short: mkimage has a parameter '-E' which allows generating FIT
images with 'external' data rather than embedding the data into the
device-tree blob itself. In this way, the FIT structure itself remains
small and can be parsed easily (rather than having to page around
megabytes of image content). This patch makes use of that and adds
support for adding sub-images of type 'filesystem' which are used to
store the squashfs. Now U-Boot can verify the whole OS and the new
partition parsers added in the Linux kernel can detect the filesystem
sub-images, create partitions for them, and select the active rootfs
volume based on the configuration in FIT (passing configuration via
device tree could be implemented easily at a later stage).

This new FIT partition parser works for NOR flash (on top of mtdblock),
NAND flash (on top of ubiblock) as well as classic block devices
(ie. eMMC, SDcard, SATA, NVME, ...).
It could even be used to mount such FIT images via `losetup -P` on a
user PC if this patch gets included in Linux upstream one day ;)

Signed-off-by: John Crispin <john@phrozen.org>
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2021-02-24 01:35:20 +00:00
David Bauer
f04e007491 download: remove broken mirrors
These mirrors don't exist anymore. Remove them.

Signed-off-by: David Bauer <mail@david-bauer.net>
2021-02-22 00:57:04 +01:00
Felix Fietkau
075fa4cd9a Mostly revert "build: add support for fixing up library soname"
This reverts commit b12288fa69.
The patchelf approach is too fragile, and the only users of this have been
converted to make patching unnecessary
Leave the abi_version_str variable in place in rules.mk

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2021-02-15 18:47:21 +01:00
Felix Fietkau
c921650382 build: drop ABI version from metadata
Preparation for supporting dynamic ABI versions that depend on the runtime
configuration. Read the suffix from the staging dir pkginfo version files.

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2021-02-14 19:41:52 +01:00
Felix Fietkau
b12288fa69 build: add support for fixing up library soname
This makes it possible to declare a package ABI_VERSION independent from the
upstream soname by setting PKG_ABI_VERSION in the package makefile.
The library filename is fixed up for files installed to packages and to the
staging dir. References to the original from executables within the same
package are also fixed up

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2021-02-14 19:41:51 +01:00
Paul Spooren
6d5425976f scripts: fix checkpatch.pl for changed license dir
As multiple LICENSES are shipped and no longer just LICENSE, modify the
OpenWrt tree detection in checkpatch.pl.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-02-14 19:21:38 +01:00
Paul Spooren
8286f3a3d3 treewide: unify OpenWrt hosted source via @OPENWRT
Multiple sources are hosted on OpenWrts source server only. The source
URLs to point to the server vary based on different epochs in OpenWrts
history.

Replace all by @OPENWRT which is an "empty" mirror, therefore using the
fallback servers sources.cdn.openwrt.org and sources.openwrt.org.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-02-05 12:00:24 -10:00
Paul Spooren
7521aedff4 scripts: sources CDN as fallback in download.pl
In case the default sources for a package fail use the CDN rather than
our own mirror. In case the CDN fails, fallback to our mirror.

Also remove mirror1 which isn't available anymore.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-01-27 22:46:08 -10:00
Paul Spooren
4dad324429 scripts: target-metadata don't add PROFILES twice
Since 4ee3cf2b5a profiles with alternative vendor names may appear
multiple times in `tmp/.targetinfo` or `.targetinfo` (for
ImageBuilders).

The `target-metadata.pl` script adds these profiles then twice to
`PROFILE_NAMES` and the ImageBuilder show the profile twice when running
`make info`.

This patch removes duplicate profile IDs and only adds them once to
`.profiles.mk`.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-01-14 22:42:21 -10:00
Paul Spooren
a323a653f0 scripts/feed: no warn on toolchain/linux overwrite
The recent 7f285d "scripts/feeds: warn when skipping core package
override" floods SDK output with warning of overwriting "linux" and
"toolchain" core packages. This should be ignored as these are not
regular packages added via feeds.

While at it slightly improve the warning string.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-01-06 14:04:34 -10:00
Kevin Darbyshire-Bryant
e2180e3cbe build: drop clang wrapper
clang's gcc emulation does the right thing with -print-file-name now,
drop the wrapper

Signed-off-by: Kevin Darbyshire-Bryant <ldir@darbyshire-bryant.me.uk>
2021-01-05 08:59:59 +00:00
Paul Spooren
1b002434f0 scripts/qemustart: use squashfs instead of ext4
The qemustart script currently picks the ext4 filesystem rather than
squashfs, while the latter is default for nearly all OpenWrt targets.

Change the default behaviour of qemustart to be in line with the rest.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-01-01 17:49:05 -10:00
Marek Lindner
4871fd2616 ipq40xx: add support for Plasma Cloud PA2200
Device specifications:

* QCA IPQ4019
* 256 MB of RAM
* 32 MB of SPI NOR flash (w25q256)
  - 2x 15 MB available; but one of the 15 MB regions is the recovery image
* 2T2R 2.4 GHz
  - QCA4019 hw1.0 (SoC)
  - requires special BDF in QCA4019/hw1.0/board-2.bin with
    bus=ahb,bmi-chip-id=0,bmi-board-id=20,variant=PlasmaCloud-PA2200
* 2T2R 5 GHz (channel 36-64)
  - QCA9888 hw2.0 (PCI)
  - requires special BDF in QCA9888/hw2.0/board-2.bin
    bus=pci,bmi-chip-id=0,bmi-board-id=16,variant=PlasmaCloud-PA2200
* 2T2R 5 GHz (channel 100-165)
  - QCA4019 hw1.0 (SoC)
  - requires special BDF in QCA4019/hw1.0/board-2.bin with
    bus=ahb,bmi-chip-id=0,bmi-board-id=21,variant=PlasmaCloud-PA2200
* GPIO-LEDs for 2.4GHz, 5GHz-SoC and 5GHz-PCIE
* GPIO-LEDs for power (orange) and status (blue)
* 1x GPIO-button (reset)
* TTL pins are on board (arrow points to VCC, then follows: GND, TX, RX)
* 2x gigabit ethernet
  - phy@mdio3:
    + Label: Ethernet 1
    + gmac0 (ethaddr) in original firmware
    + used as LAN interface
  - phy@mdio4:
    + Label: Ethernet 2
    + gmac1 (eth1addr) in original firmware
    + 802.3at POE+
    + used as WAN interface
* 12V 2A DC

Flashing instructions:

The tool ap51-flash (https://github.com/ap51-flash/ap51-flash) should be
used to transfer the factory image to the u-boot when the device boots up.

Signed-off-by: Marek Lindner <marek.lindner@kaiwoo.ai>
[sven@narfation.org: prepare commit message, rebase, use all LEDs, switch
to dualboot_datachk upgrade script, use eth1 as designated WAN interface]
Signed-off-by: Sven Eckelmann <sven@narfation.org>
2020-12-22 19:11:50 +01:00
Marek Lindner
ea5bb6bbfe ipq40xx: add support for Plasma Cloud PA1200
Device specifications:

* QCA IPQ4018
* 256 MB of RAM
* 32 MB of SPI NOR flash (w25q256)
  - 2x 15 MB available; but one of the 15 MB regions is the recovery image
* 2T2R 2.4 GHz
  - QCA4019 hw1.0 (SoC)
  - requires special BDF in QCA4019/hw1.0/board-2.bin with
    bus=ahb,bmi-chip-id=0,bmi-board-id=16,variant=PlasmaCloud-PA1200
* 2T2R 5 GHz
  - QCA4019 hw1.0 (SoC)
  - requires special BDF in QCA4019/hw1.0/board-2.bin with
    bus=ahb,bmi-chip-id=0,bmi-board-id=17,variant=PlasmaCloud-PA1200
* 3x GPIO-LEDs for status (cyan, purple, yellow)
* 1x GPIO-button (reset)
* 1x USB (xHCI)
* TTL pins are on board (arrow points to VCC, then follows: GND, TX, RX)
* 2x gigabit ethernet
  - phy@mdio4:
    + Label: Ethernet 1
    + gmac0 (ethaddr) in original firmware
    + used as LAN interface
  - phy@mdio3:
    + Label: Ethernet 2
    + gmac1 (eth1addr) in original firmware
    + 802.3af/at POE(+)
    + used as WAN interface
* 12V/24V 1A DC

Flashing instructions:

The tool ap51-flash (https://github.com/ap51-flash/ap51-flash) should be
used to transfer the factory image to the u-boot when the device boots up.

Signed-off-by: Marek Lindner <marek.lindner@kaiwoo.ai>
[sven@narfation.org: prepare commit message, rebase, use all LEDs, switch
to dualboot_datachk upgrade script, use eth1 as designated WAN interface]
Signed-off-by: Sven Eckelmann <sven@narfation.org>
2020-12-22 19:11:50 +01:00
Sven Eckelmann
5fc28ef479 ath79: Add support for Plasma Cloud PA300
Device specifications:

* Qualcomm/Atheros QCA9533 v2
* 650/600/217 MHz (CPU/DDR/AHB)
* 64 MB of RAM
* 16 MB of SPI NOR flash (mx25l12805d)
  - 2x 7 MB available; but one of the 7 MB regions is the recovery image
* 2x 10/100 Mbps Ethernet
* 2T2R 2.4 GHz Wi-Fi
* multi-color LED (controlled via red/green/blue GPIOs)
* 1x GPIO-button (reset)
* external h/w watchdog (enabled by default)
* TTL pins are on board (arrow points to VCC, then follows: GND, TX, RX)
* 2x fast ethernet
  - eth0
    + Label: Ethernet 1
    + 24V passive POE (mode B)
    + used as WAN interface
  - eth1
    + Label: Ethernet 2
    + 802.3af POE
    + builtin switch port 2
    + used as LAN interface
* 12-24V 1A DC
* internal antennas

Flashing instructions:

The tool ap51-flash (https://github.com/ap51-flash/ap51-flash) should be
used to transfer the factory image to the u-boot when the device boots up.

Signed-off-by: Sven Eckelmann <sven@narfation.org>
2020-12-22 19:11:50 +01:00
Sven Eckelmann
3599ea5263 images: Fix sysupgrade.tar for devices with NOR flash
The NOR flash rootfs images stored in a sysupgrade.tar must end with the
JFFS2 marker. Otherwise, devices like OpenMesh A42/A62 are not able to
calculate the md5sum of the fixed squashfs part and store it inside the
u-boot-env.

But the commit ee76bd11bb ("images: fix boot failures on NAND with small
sub pages") adds up to 1020 0x00 bytes after the 0xdead0de EOF marker. The
calculated md5sum will be wrong due do this change and u-boot will fail to
boot the newly flashed device with a message like:

  Validating MD5Sum of 'vmlinux'...
  Passed!
  Validating MD5Sum of 'rootfs'...
  Failed!
      583a1b7b54b8601efa64ade42742459b != 8850ee812dfd7638e94083329d5d2781

  Data validation failed!

and boot the old image again.

Since the original change should not change the behavior of NOR images,
just check for the deadc0de marker at the end of the squashfs-jffs2 image
do avoid the problematic behavior for these images.

Fixes: ee76bd11bb ("images: fix boot failures on NAND with small sub pages")
Signed-off-by: Sven Eckelmann <sven@narfation.org>
2020-12-22 19:11:50 +01:00
Sven Eckelmann
a4c30d9399 scripts/om-fwupgradecfg-gen.sh: Generate checksum over whole squashfs
The rootfs is padded to the full block size by padjffs2 and a 4 byte magic
value ("deadc0de") is added to the end. On first boot, the JFFS2 is
replacing the "deadc0de" marker when the rootfs_data is initialized.

The static part of the rootfs is therefore $rootfs_size - 4 and not
$rootfs_size - 262144 - 4.

Signed-off-by: Sven Eckelmann <sven@narfation.org>
2020-12-22 19:11:50 +01:00
Sven Eckelmann
7c75eaadfc scripts/om-fwupgradecfg-gen.sh: Drop block alignment code
The padding and block alignment is handled by the image build script and
doesn't need to be duplicated in the fwupgrade.cfg build script.

Signed-off-by: Sven Eckelmann <sven@narfation.org>
2020-12-22 19:11:50 +01:00
INAGAKI Hiroshi
fdcdbdfdef scripts: add -N option to mkhash for printing without newline
Added "-N" option, it allow printing hash(es) without newline.

Signed-off-by: INAGAKI Hiroshi <musashino.open@gmail.com>
2020-12-22 19:11:50 +01:00
Paul Spooren
bc0ffff36a build/json: add filesystem information
Some images are created using different filesystems, most popular
squashfs and ext4. To allow downstream projects to distinguesh between
those, add the `filesystem` information to created json files.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-12-13 22:07:52 -10:00
Karel Kočí
a3e5b24d43 scripts/feeds: fix preference of package install
The previous behavior prefered same feed for dependent packages as
initial package. This caused inconsitency in installation of packages.
The difference was if two feeds provide same package (different version)
there was different result if you executed install for that specific
version compared to install for package depending on it from different
feed.

This ensures that preferred feed is propagated without change and
selected feed is used only really for package it was selected for.

Signed-off-by: Karel Kočí <karel.koci@nic.cz>
2020-12-09 23:25:05 -10:00
Brian Norris
7f285d0b43 scripts/feeds: warn when skipping core package override
Otherwise, a n00b like myself can get quite confused when moving a
package from core to feeds, for example.

(Hint: one *really* needs to clear out the tmp/info/.packageinfo...
entries for the stale package, but '-f' works as well.)

Signed-off-by: Brian Norris <computersforpeace@gmail.com>
2020-12-09 11:30:07 -10:00
Piotr Stefaniak
1db9b5b6d6 build: mkhash on FreeBSD
Apply patch from
https://bugs.openwrt.org/index.php?do=details&task_id=971
in order to make it easier to build OpenWRT on FreeBSD.

Signed-off-by: Piotr Stefaniak <pstef@freebsd.org>
2020-11-27 18:01:22 -10:00
Petr Štetiar
52a5d0d27f download.pl: properly cleanup intermediate .hash file
It seems like after a build the /dl dir seems to now contain a .hash
file for each source file due to inproper cleanup so fix it by removing
those intermediate files before leaving the download action.

Fixes: 4e19cbc553 ("download: handle possibly invalid local tarballs")
Reported-by: Hannu Nyman <hannu.nyman@iki.fi>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-11-27 22:27:08 +01:00
Petr Štetiar
4e19cbc553 download: handle possibly invalid local tarballs
Currently it's assumed, that already downloaded tarballs are always
fine, so no checksum checking is performed and the tarball is used even
if it might be corrupted.

From now on, we're going to always check the downloaded tarballs before
considering them valid.

Steps to reproduce:

 1. Remove cached tarball

   rm dl/libubox-2020-08-06-9e52171d.tar.xz

 2. Download valid tarball again

   make package/libubox/download

 3. Invalidate the tarball

   sed -i 's/PKG_MIRROR_HASH:=../PKG_MIRROR_HASH:=ff/' package/libs/libubox/Makefile

 4. Now compile with corrupt tarball source

   make package/libubox/{clean,compile}

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-11-27 14:46:13 +01:00
Hannu Nyman
2b1d92f134 scripts/feeds: silence git warning by selecting pull style
Silence the warning in git 2.27 about undefined fast-forward style
in git pull. Define "ff-only" as the style.

Signed-off-by: Hannu Nyman <hannu.nyman@iki.fi>
2020-11-26 19:29:18 -10:00
Paul Spooren
0c87304d2b scripts: add size_compare.sh
As package size changes are a continuous topic on the mailing list this
scripts helps developers to compare their local package modifications
against latest upstream.

The script downloads the latest package indexes based on env variables
or the `.config` file. The script compares the actual installed size
(data.tar.gz) or the IPK package size.

An example output is found below:

```
user@dawn:~/src/openwrt/openwrt$ ./scripts/size_compare.sh
Compare packages of ath79/tiny/mips_24kc:
dropbear busybox iw ubus

Checking configuration difference
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   554  100   554    0     0    336      0  0:00:01  0:00:01 --:--:--   336

 --- start config diff ---
--- /tmp/config.DDjwVh-LOCAL	2020-11-23 09:08:28.913203068 -1000
+++ /tmp/config.DDjwVh-UPSTREAM	2020-11-23 09:08:36.369240887 -1000
@@ -1,5 +1,9 @@
+CONFIG_ALL_KMODS=y
+CONFIG_ALL_NONSHARED=y
 CONFIG_AUTOREBUILD=y
+CONFIG_AUTOREMOVE=y
--- 8< ---
 CONFIG_BINARY_FOLDER=""
+CONFIG_BUILDBOT=y
+CONFIG_TARGET_ALL_PROFILES=y
 CONFIG_TARGET_ROOTFS_DIR=""
 CONFIG_USE_SSTRIP=y
 CONFIG_USE_UCLIBCXX=y
 --- end config diff ---

Checking installed size

Fetching latest package indexes...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 80634  100 80634    0     0  33499      0  0:00:02  0:00:02 --:--:-- 33485
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 54082  100 54082    0     0  24252      0  0:00:02  0:00:02 --:--:-- 24252

Comparing package sizes...
Change 	Local	Remote 	Package
+271	51386	51115	base-files
+123	705241	705118	bnx2-firmware
+86	17209	17123	fstools
+22	47989	47967	procd
+21	208311	208290	busybox
+19	67181	67162	netifd

```

I plan to integrate this script into the CI so we have a summary how
sizes change over different architectures.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-11-24 18:18:48 -10:00
David Bauer
d369993898 scripts: download.pl: retry download using filename
With this commit, the download script will try downloading source files
using the filename instead of the url-filename in case the previous
download attempt using the url-filename failed.

This is required, as the OpenWrt sources mirrors serve files using the
filename files might be renamed to after downloading. If the original
mirror for a file where url-filename and filename do not match goes
down, the download failed prior to this patch.

Further improvement can be done by performing this only for the
OpenWrt sources mirrors.

Signed-off-by: David Bauer <mail@david-bauer.net>
2020-11-20 02:35:43 +01:00
Adrian Schmutzler
7e7470b517 scripts: add const_structs.checkpatch for checkpatch.pl
Kernel has separated the structs that are reported to be const in
checkpatch.pl into a file of its own, const_structs.checkpatch.

This file has been missing after the recent update of checkpatch.pl,
leading to the following message:

  No structs that should be const will be found - file
  '/data/openwrt/scripts/const_structs.checkpatch': No such file
  or directory

This commit adds the relevant file from v5.10-rc4.

Fixes: 086ee09bbc ("scripts: Update checkpatch.pl to 2020-06-11")

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2020-11-18 21:50:58 +01:00
Adrian Schmutzler
6fdd4187e0 scripts: checkpatch.pl: suppress warnings about MAINTAINERS
The kernel expects changes to MAINTAINERS for all removed or added
files, printing warnings like:

  WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
  #828:
  deleted file mode 100644

Since this does not apply to "our" files in OpenWrt repo, this
warning should be disabled.

This can be achieved easiest by setting $reported_maintainer_file
to 1. While this is a hack that tricks the script into believing
the proper MAINTAINERS changes have been made, it's the easiest
solution as it does not require to touch any other code.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2020-11-18 21:50:53 +01:00
Felix Fietkau
184d735707 scripts/kconfig.pl: allow regex syntax in filtering out config entries
This will be used to filter out some autogenerated config values from
the kernel config files

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2020-11-13 13:17:53 +01:00
Paul Spooren
3c0357e7b4 scripts: add spelling.txt for checkpatch.pl
Running the updated checkpatch version with spelling.txt show that
spelling mistakes happen to everyone.

> /target/linux$ fd .*.patch | xargs ../../scripts/checkpatch.pl | rg spell
WARNING: 'usefull' may be misspelled - perhaps 'useful'?
WARNING: 'afecting' may be misspelled - perhaps 'affecting'?
WARNING: 'usefull' may be misspelled - perhaps 'useful'?
WARNING: 'afecting' may be misspelled - perhaps 'affecting'?
WARNING: 'begining' may be misspelled - perhaps 'beginning'?
WARNING: 'superflous' may be misspelled - perhaps 'superfluous'?
WARNING: 'multipe' may be misspelled - perhaps 'multiple'?
WARNING: 'recieves' may be misspelled - perhaps 'receives'?
WARNING: 'retreive' may be misspelled - perhaps 'retrieve'?
WARNING: 'tranfer' may be misspelled - perhaps 'transfer'?
WARNING: 'additonal' may be misspelled - perhaps 'additional'?
WARNING: 'accomodate' may be misspelled - perhaps 'accommodate'?
[...]

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-11-12 18:21:55 +01:00
Paul Spooren
66e87a1f38 scripts: checkpatch remove device-tree doc check
Remove checks for device tree documentation as the OpenWrt tree comes
withouth the ./Documentation folder.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-11-12 18:21:55 +01:00
Paul Spooren
086ee09bbc scripts: Update checkpatch.pl to 2020-06-11
Checkpatch grown in functionallity and we should make use of that. If
OpenWrt patches should be upstream material they should also be checked
based on upstream checkpatch.pl instead of 2013.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-11-12 18:21:55 +01:00
Paul Spooren
165f0b00cd build: store SOURCE_DATE_EPOCH in JSON info files
The source date epoch is the only reproducible date close to the actual
build date. It can be used for tooling like the firmware wizard to show
the image age.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-11-12 18:19:44 +01:00
Paul Spooren
fcd3e13779 scripts: mkhash fail on hashing a folder
mkhash currently returns the hash of an empty input when trying to hash
a folder. This can be missleading in caseswhere e.g. an env variable is
undefined which should contain a filename. `mkhash ./path/to/$FILE`
would exit with code 0 and return a legit looking checksum.

A better behaviour would be to fail with exit code 1, which imitates the
behaviour of `md5sum` and `sha256sum`.

To avoid hashing of folders the `stat()` is checked.

Hashing empty inputs result in the following checksums:
md5: d41d8cd98f00b204e9800998ecf8427e
sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-11-09 10:54:29 +00:00
Paul Spooren
fac98f1c54 scripts: mkhash fix return code handling
If hashing a file fails mkhash shouldn't just silently fail. Now check
after each call of `hash_file()` the return and exit early in case of
errors. The return value which was previously ignored and would always
return 0.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-11-09 10:54:29 +00:00
Paul Spooren
91a0dc5161 scripts: mkhash show -n option in usage
The -n option prints the filename of hashed files next to the calculated
checksum. Reflect that in the usage message.

user@dawn:~/src/openwrt/openwrt$ ./a.out md5 -n .config
eb06db36e7b6751cb18801945e46bf5d .config

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-11-09 10:52:19 +00:00
Petr Štetiar
4b7165722c scripts: bundle-libraries.sh: fix broken SDK compiler
Recent versions (> 5.33) of `file` report liblto_plugin.so as
executable:

 $ file liblto_plugin.so
 liblto_plugin.so.0.0.0: ELF 64-bit LSB pie executable ...

Which then leads to improper packaging of the plugin, resulting in the
broken compiler:

 configure: checking whether the C compiler works
 mips-openwrt-linux-musl/bin/ld: liblto_plugin.so: error loading plugin: liblto_plugin.so: invalid ELF header

As the LTO compiler plugin library is incorrectly packaged as SDK
executable:

 $ head -1 ~/staging_dir/toolchain...libexec/gcc/.../liblto_plugin.so
 #!/usr/bin/env bash

Fix this by filtering out shared libraries from the patching.

Ref: https://bugzilla.redhat.com/show_bug.cgi?id=1296868
Acked-by: Jo-Philipp Wich <jo@mein.io>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-11-02 08:39:49 +01:00
Petr Štetiar
f52aee0455 scripts/getver.sh: silence rev-list errors
Shallow copies are quite common on CI platforms nowadays, making REBOOT
tag unavailable, thus producing following confusing errors in the build
logs:

 fatal: Invalid revision range ee53a240ac902dc83209008a2671e7fdcf55957a..HEAD
 fatal: Invalid revision range ee53a240ac902dc83209008a2671e7fdcf55957a..0493d57e04774d47921a7d2014b567455d5dc16b

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-11-02 08:39:49 +01:00
Tony Ambardar
5b3ff53ebd scripts/qemustart: update malta to use MIPS64 R2 cpu
Explicitly set the QEMU cpu type and support the MIPS R2 ISA, for both
64-bit and 32-bit targets. The later previously supported MIPS R2 by
implicit default.

This is needed after commit 93608697f3 ("malta: update MIPS64 ISA to R2"),
otherwise booting malta images with scripts/qemustart will hang.

Signed-off-by: Tony Ambardar <itugrok@yahoo.com>
2020-10-11 16:16:24 +02:00
Christian Lamparter
f6fbc39706 build: define PWM_SUPPORT arch feature flag
As the PWM has its own sub-system in the Linux kernel,
I think it should be handled in the same way as GPIO, RTC, PCI...

This patch introduces a specific feature flag "pwm" and the
"leds-pwm" kernel module as the first customer.

Signed-off-by: Christian Lamparter <chunkeey@gmail.com>
2020-09-25 19:32:33 +02:00
Christian Lamparter
fed1788689 scripts: mkits.sh make it possible to specify fdt@#
Some bootloaders are really keen on just one special
fdt in a multi-image fit image. This is a problem, because
currently this is fixed to "fdt@1".

This patch introduces a new device variable:
DEVICE_FDT_NUM that allows to specify the right
fdt number.

If the value is absent "1" will be chosen.

Signed-off-by: Christian Lamparter <chunkeey@gmail.com>
2020-09-25 19:32:33 +02:00
Paul Spooren
8be2eb3b9b scripts: update SPDX license names
SPDX moved from GPL-2.0 to GPL-2.0-only and from GPL-2.0+ to
GPL-2.0-or-later. Reflect that in the SPDX license headers.

Signed-off-by: Paul Spooren <mail@aparcar.org>
[adjust commit title, update remaining files]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2020-09-22 20:36:04 +02:00
Jo-Philipp Wich
4038c031cb scripts: ipkg-build: simplify uid/gid resolving
Use the prepared .packageusergroup file to lookup user and group names
when processing the passed file mode.

Also replace the various subshell/cut invocations with a sequence of
standard variable interpolations which fixes paths with embedded colons
as a side-effect.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2020-09-14 17:11:04 +02:00
Paul Spooren
51ec51871f build: add user/group ID resolve function
With the introduction of `./tmp/userids` the `ipkg-build` script can now
resolve values of "PKG_FILE_MODES", allowing users to set names rather
than numeric values.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-09-14 10:54:52 +01:00
David Bauer
fe82ea049e scripts: download.pl: fix indentation
Signed-off-by: David Bauer <mail@david-bauer.net>
2020-09-11 17:35:19 +02:00
Jo-Philipp Wich
10dbdeccfa scripts: bundle-libraries.sh: retain preloaded libraries
Since the introduction of fakeroot support, wrapped SDK executables might
be invoked from a shell that has libfakeroot.so preloaded.

Since we're using preloading as well in order to mangle argv[0] when
invoking the shipped ELF interpreter directly, we must take care of
preloading the already preloaded libraries as well, to avoid invoked
programs losing their fakeroot capabilities.

Extend the bundle-libraries.sh script to take any existing $LD_PRELOAD
into account when invoking the target ELF executable with a preloaded
runas.so library.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2020-09-10 14:24:50 +02:00
Paul Spooren
fd29f2bcd9 build: store granular timestamps in packages
With the new `SOURCE` argument of `get_source_date_epoch` it is possible
to set package timestamps based on actual package changes rather thane
$TOPDIR changes.

This commit adds a new variable PKG_SOURCE_DATE_EPOCH which is used by
the `ipkg` build script. As a fallback the existing SOURCE_DATE_EPOCH is
used or as last resort the current time.

The redundant checks for `.git/` and `.svn/` are removed.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-08-31 11:18:06 +01:00
Paul Spooren
a83b2af47f build: get_source_date_epoch allow external repos
The SOURCE_DATE_EPOCH variable is used to make builds reproducible even
if rebuild at different times. Instead of using the current timestamp,
the time of the last source change is used.

Created packages are `touch`ed with a specific timestamp so resulting
packages have the same checksums.

The `get_source_date_epoch.sh` script tries multiple ways (file, git,
hg) to determine the correct timestamp.

Until now the script would only consider the $TOPDIR instead of package
specific changes. Resulting in packages with same versions but different
timestamps, as $TOPDIR (openwrt.git) received changes not affecting
package versions. This results in warning/erros in `opkg` as the package
versions stay the same but checksums changed.

This commit adds an optional argument to get the `SOURCE_DATE_EPOCH` of
a specific path (e.g. package SOURCE) rather than the $TOPDIR. As a
consequence this allows granular but still reproducible timestamps.

As packages might be distributed over multiple repositories the check
for `.git/` becomes unfeasible. Instead tell `git` and `hg` to change
their working directories and automatically traverse the repo folder.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-08-31 11:18:06 +01:00
Paul Spooren
ae87e53e33 build: Fix Shellcheck for get_source_date_epoch.sh
If a `cd` to `TOPDIR` fails the script should quit.

Also unify `try_mtime` function by storing it in a variable.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-08-31 11:18:06 +01:00
Paul Spooren
353ce2e521 build: ipkg-build use fakeroot with PKG_FILE_MODES
The `ipkg-build` script converts a folder into a `opkg` installable
package. Until now it would use root:root for all packages and try to
preserve file modes.

This has the two drawbacks of packages want to add non-root files or add
SUID files, like the `sudo` package does.

To give more flexibility regarding file modes and avoid init script
hacks, a new variable called `PKG_FILE_MODES`. The variable contains a
list of files modes in the format `path:owner:group:mode`.

An example for the `sudo` package below:

```
PKG_FILE_MODES:=\
        /usr/bin/sudo:root:root:4755 \
        /etc/sudoers:root:root:0440
```

The `ipkg-build` now runs within a fakeroot environment to set any mode
and directly store it in the resulting `ipk` package archive.

Both options `-o` and `-g` are no longer required due to the introduction
of the more flexible `-m` options, which takes the `PKG_FILE_MODES` as
input.

Lastly the option `-c` is removed as it's unused within the script.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-08-31 11:13:12 +01:00
Jo-Philipp Wich
bf96eb55c8 Revert "scripts/download: add sources CDN as first mirror"
This reverts commit c737a9ee6a.

The source CDN has been discontinued in its current form and will take a
while to be reestablished. Even then it makes little sense to put a CDN
before other CDNs such as kernel.org, apache.org, sourceforge etc.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2020-08-24 15:55:14 +02:00
Paul Spooren
1fdf6b745c treewide: replace which with command -v
Fix shellcheck SC2230
> which is non-standard. Use builtin 'command -v' instead.

Using `command -v` is POSIX compliant while `which` is not.  Also to
mention, `command -v` is a shell builtin whereas `which` is a separate
busybox applet.

Once applied to everything concerning OpenWrt we can disable the busybox
feature `which` and save 3.8kB.

Acked-by: Stijn Tintel <stijn@linux-ipv6.be>
Signed-off-by: Paul Spooren <mail@aparcar.org>
[also replace cases in zram-swap]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2020-08-12 11:08:05 +02:00
Petr Štetiar
80d360202f scripts: remove checkpatch.sh
That file was added accidentally in v3 of the patch and I haven't
properly reviewed that patch before pushing it.

Fixes: 656b562aff ("scripts: Add Buildbot dump-target-info.pl script")
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-08-04 14:25:44 +02:00
Paul Spooren
656b562aff scripts: Add Buildbot dump-target-info.pl script
The script comes from buildbot.git[0] and is used to print available
targets and architectures, which are then build.

As the buildbot clones openwrt.git anyway, the script might as well live
here to be used for other cases as well, e.g. determining what
architectures are available when building Docker containers or show
developers an overview which architectures are used by which target.

It's called with either the parameter `architectures` or `targets`,
showing architectures followed by supported targets or targets, followed
by the supported architectures:

$ ./scripts/dump-target-info.pl architectures
aarch64_cortex-a53 bcm27xx/bcm2710 mediatek/mt7622 mvebu/cortexa53 sunxi/cortexa53
aarch64_cortex-a72 bcm27xx/bcm2711 mvebu/cortexa72
...

$ ./scripts/dump-target-info.pl targets
apm821xx/nand powerpc_464fp
apm821xx/sata powerpc_464fp
...

In the future the the script could be removed from the buildbot
repository and maintained only here.

Rename `dumpinfo.pl` to `dump-target-info.pl` to improve verbosity of
filename.

[0]: https://git.openwrt.org/?p=buildbot.git;a=blob;f=scripts/dumpinfo.pl;h=aa97f8d60379076a41b968402e9337cea824ece5;hb=HEAD

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-08-04 09:28:39 +02:00
Adrian Schmutzler
b99623329c scripts/checkpatch.pl: fix README.md file name after rename
checkpatch.pl uses a list of files to detect the root OpenWrt
directory. This includes README, which has been renamed to
README.md in the previous commit.

Update the file name in checkpatch.pl to prevent errors like the
following when running the script:

   Must be run from the top-level dir. of a OpenWrt tree

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2020-08-02 15:51:39 +02:00
Adrian Schmutzler
1d5260cf72 build: add option to mark devices as BROKEN
By specifying "BROKEN := 1" or "BROKEN := y" for a device, it will be
hidden (and deselected) by default. By that, it provides a stronger
option to "disable" a device beyond just using DEFAULT := n.

To make these devices visible, just enable the BROKEN option in
developer settings as already implemented for targets and packages.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2020-07-30 21:49:07 +02:00
Paul Spooren
5b8b42c78b build,json: fix compatibility with Python 3.5
The f-string feature was introduced in Python 3.6. As Buildbots may run
on Debian 9, which comes per default with Python 3.5, this would cause
an issue. Instead of f-strings use the *legacy* `.format()` function.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-07-13 22:45:04 +02:00
Kevin Darbyshire-Bryant
8242c6de29 scripts/env: Fix 56f813674a scripts/env: use command -v instead of which
We don't need to see how git will be executed and it produces non silent
output on 'scripts/env diff' commands when there are no differences
unlike before.

Re-introduce original silent behaviour.

Signed-off-by: Kevin Darbyshire-Bryant <ldir@darbyshire-bryant.me.uk>
2020-07-12 13:29:28 +01:00
Adrian Schmutzler
52cdd6185e scripts/mkits.sh: fix use of printf
Due to a line break, printf was accidentally called with three
arguments instead of two, causing a different output than before.

Fix it by splitting the printf command into two lines.

Fixes: 907053193a ("scripts/mkits.sh: replace echo -e with printf")

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2020-07-11 18:46:53 +02:00
Petr Štetiar
dec9ae6d9e scripts/mkits.sh: fix remaining shellcheck warning
Fixes following shellcheck warning:

 In scripts/mkits.sh line 19:
 		 "-k kernel [-D name -d dtb] -o its_file" "$(basename $0)"
                                                                       ^-- SC2086: Double quote to prevent globbing and word splitting.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-07-11 14:42:32 +02:00
Rosen Penev
3d418dd51e scripts/mkits.sh: switch from bash to sh
This no longer needs bash.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2020-07-11 14:42:32 +02:00
Rosen Penev
7aefc85cab scripts/mkits.sh: fix improper string and array concatenation
Found with shellcheck.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2020-07-11 14:42:32 +02:00
Rosen Penev
765858f7ad scripts/mkits.sh: add missing quotes
Found with shellcheck.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2020-07-11 14:42:32 +02:00
Rosen Penev
907053193a scripts/mkits.sh: replace echo -e with printf
echo flags are not POSIX. printf does the same with added \n.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2020-07-11 14:42:32 +02:00
Petr Štetiar
92d7cedcad scripts/gen_image_generic.sh: fix more shellcheck warnings
Fixes following shellcheck warnings:

 In scripts/gen_image_generic.sh line 20:
 cyl=$(( (KERNELSIZE + ROOTFSSIZE) * 1024 * 1024 / (head * sect * 512)))
 ^-^ SC2034: cyl appears unused. Verify use (or export if used externally).
 --
 In scripts/gen_image_generic.sh line 34:
     [ -n "$PADDING" ] && dd if=/dev/zero of="$OUTPUT" bs=512 seek="$(($ROOTFSOFFSET + $ROOTFSSIZE))" conv=notrunc count="$sect"
                                                                       ^-----------^ SC2004: $/${} is unnecessary on arithmetic variables.
 --
 In scripts/gen_image_generic.sh line 35:
     mkfs.fat -n kernel -C "$OUTPUT.kernel" -S 512 "$(($KERNELSIZE / 1024))"
                                                       ^---------^ SC2004: $/${} is unnecessary on arithmetic variables.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-07-11 14:42:32 +02:00
Rosen Penev
aee169b06d scripts/gen_image_generic.sh: use /bin/sh
This has nothing bash specific.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2020-07-11 13:33:28 +02:00
Rosen Penev
3819337cdf scripts/gen_image_generic.sh: replace -o with if/&&
-o is not well defined.

Found with shellcheck.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2020-07-11 13:33:28 +02:00
Petr Štetiar
227e47b3ca scripts/env: fix remaining shellcheck warning
Fixes following shellcheck warning:

 In scripts/env line 25:
 	exit ${1:-1}
             ^-----^ SC2086: Double quote to prevent globbing and word splitting.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-07-11 13:33:28 +02:00
Rosen Penev
5b7cace9e3 scripts/env: use read -r instead of read
read mangles backslashes.

Found with shellcheck.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2020-07-11 13:33:28 +02:00
Rosen Penev
f80a540dbf scripts/env: exit in case of failure to cd
Found with shellcheck.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2020-07-11 13:33:28 +02:00
Rosen Penev
e68810ba53 scripts/env: replace \! with !
The latter is more standard. The former throws an error under
shellcheck.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2020-07-11 13:33:28 +02:00
Rosen Penev
56f813674a scripts/env: use command -v instead of which
Simpler and built in to the shell.

Found with shellcheck.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2020-07-11 13:33:28 +02:00
Huangbin Zhan
3caad5150c build: fix compatibility with python 3.6
On a system python3 is linked to python3.6, fail to perform json_overview_image_info
 and got `TypeError: __init__() got an unexpected keyword argument 'capture_output'`.
This patch emulate the behaviour on python 3.7+.

Signed-off-by: Huangbin Zhan <zhanhb88@gmail.com>
2020-07-07 19:47:24 +02:00
Sergio E. Nemirowski
ab26022cc1 build: mconf readme update
This updates mconf.c readme message to maintain less changes with
upstream and consistency with nconf.c

Signed-off-by: Sergio E. Nemirowski <sergio@outerface.net>
2020-07-07 19:47:24 +02:00
Sergio E. Nemirowski
8fb0b15efd build: add nconfig
Add support for make target nconfig (ncurses)

Reviewed-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
Signed-off-by: Sergio E. Nemirowski <sergio@outerface.net>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-07-07 19:47:24 +02:00
Daniel Golle
3b0f698760 build,json: fix build failure in case no data is found
Only collect arch_packages if actually generating any output.

Fixes: commit f09b9319 ("build,json: store arch_packages in profiles.json"(
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-07-03 22:09:24 +01:00
Paul Spooren
f09b9319c6 build,json: store arch_packages in profiles.json
The `arch_packages` contains the supported package architecture.

Previously it was necessary to parse the `Packages` index for the line
`Architecture:`, requiring both an additional parser and file download.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-07-03 15:28:05 +01:00
Paul Spooren
263f7e5bbd build: store default/device packages in JSON
With this commit the `profiles.json` contain both the target specific
`default_packages` as well as the device specific `device_packages` as a
array of strings.

This information is required for downstream projects like the various
web-based interactive firmware generators.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-06-30 23:21:10 +01:00
Álvaro Fernández Rojas
3f8e163e01 scripts: config: remove accidentally added file
Also ignore the file.

Signed-off-by: Álvaro Fernández Rojas <noltari@gmail.com>
2020-06-02 18:17:31 +02:00
Álvaro Fernández Rojas
35d7f92c60 scripts: support Sercomm load tags
Header consists in Sercomm PID bytes, followed by a SHA256 hash of the
input binary.

Signed-off-by: Álvaro Fernández Rojas <noltari@gmail.com>
2020-06-02 13:06:42 +02:00
Álvaro Fernández Rojas
7f1250a7fe scripts: support Sercomm crypto
Sercomm firmwares are encrypted with AES 256 CBC.
The key is generated with a custom algorithm from the firmware tag:
	char key[32];
	char version[32];
	char iv[32];
	char random[32];
	char size[32];
Key must be generated with Sercomm's algorithm. However, the rest of the
header can be empty. IV and random are set to 0 on purpose.

Signed-off-by: Álvaro Fernández Rojas <noltari@gmail.com>
2020-06-02 08:37:54 +02:00
Álvaro Fernández Rojas
58bdbe7321 scripts: support Sercomm partition tags
Sercomm uses a custom layout for partition tags:
	char part_name[32];
	char size[32];
	char part_version[32];
	char reserved[32];
	char rootfs_version[32];

Signed-off-by: Álvaro Fernández Rojas <noltari@gmail.com>
2020-06-02 08:33:11 +02:00
Rosen Penev
69f8983af4 scripts/flashing/flash.sh: remove trailing whitespaces
Signed-off-by: Rosen Penev <rosenp@gmail.com>
[cut out of patch with different subject]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2020-05-24 14:44:00 +02:00
Rosen Penev
34a2780395 scripts/flashing/flash.sh: Add missing quotes
These are in if statements (where they should be), and echo output that is
not piped to anything. These should be safe.

Found with shellcheck.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2020-05-24 14:44:00 +02:00
Álvaro Fernández Rojas
d90828411a scripts: add CFE Partition Tags support
Some BCM63xx NAND devices require a specific JFFS2 partition tag to verify
the JFFS2 partition validity:
	u32 part_id;
	u32 part_size;
	u16 flags;
	char part_name[33];
	char part_version[21];
	u32 part_crc32;

Signed-off-by: Álvaro Fernández Rojas <noltari@gmail.com>
2020-05-21 19:11:24 +02:00
Eneas U de Queiroz
80b350f528 build: have config-clean deal with old temp files
This is a temporary commit to have 'make config-clean' remove
temporary files from the previous scripts/config version.

The .gitignore file is updated to deal with the old files as well.

Cc: Kevin Darbyshire-Bryant <ldir@darbyshire-bryant.me.uk>
Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
2020-05-20 17:03:45 +02:00
Eneas U de Queiroz
9f843b1d43 build: scripts/config - update to kconfig-v5.6
Major changes include:
 - Much more readable reverse dependencies separated in groups
 - Improved recursive dependency report
 - More readable .config files: add comments to signal end of menus
 - More warnings for incorrect Config.in entries, such as a 'choice'
   default not contained in the 'choice'
 - Hability to properly display pseudographics with non-latin locales
 - Recursive dependencies can optionally be treated as errors

Changes from failed dcf3e63a35 attempt:
 - Recursive dependencies are treated as warnings by default
 - The option to treat them as errors is implemented as a command-line
   flag to scripts/config/conf instead of a compile-time definition
 - fixed handling of select with umnet dependencies

Cc: Petr Štetiar <ynezz@true.cz>
Cc: Jo-Philip Wich <jow@mein.io>
Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
2020-05-20 17:03:45 +02:00
Álvaro Fernández Rojas
258cb7f169 scripts: cfe-wfi-tag: fix shebang
Fixes: 72985233a6 ("scripts: support CFE WFI images")
Signed-off-by: Álvaro Fernández Rojas <noltari@gmail.com>
2020-05-19 08:06:29 +02:00
Álvaro Fernández Rojas
30fcb62840 scripts: cfe-bin-header: fix shebang
Fixes: e575a7f777 ("scripts: support tags for CFE binaries")
Signed-off-by: Álvaro Fernández Rojas <noltari@gmail.com>
2020-05-19 08:05:22 +02:00
Álvaro Fernández Rojas
72985233a6 scripts: support CFE WFI images
CFE WFI (Whole Flash Image) consists in 20 byte footer:
- u32: CRC32
- u32: WFI Version
- u32: Chip ID
- u32: Flash Type
- u32: Flags

Signed-off-by: Álvaro Fernández Rojas <noltari@gmail.com>
2020-05-18 18:24:06 +02:00
Álvaro Fernández Rojas
e575a7f777 scripts: support tags for CFE binaries
CFE loads binaries based on a 12 byte header which corresponds to:
- u32: Load Address
- u32: Entry Address
- u32: Size

Signed-off-by: Álvaro Fernández Rojas <noltari@gmail.com>
2020-05-18 18:24:06 +02:00
Paul Spooren
c737a9ee6a scripts/download: add sources CDN as first mirror
OpenWrt now has a CDN for sources at sources.cdn.openwrt.org which
mirrors sources.openwrt.org.

Downloading sources outside Europe or US (mainland) could
result in low throughput, extremely slowing down the first compilation of
the build system.

This patch adds sources.cdn.openwrt.org as the first mirror to offer
worldwide fast download speeds by default. If the CDN goes down for
whatever reason, the script jumps to the next available mirror and
downloads requested files as before (in regional varying speed).

Signed-off-by: Paul Spooren <mail@aparcar.org>
Acked-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
2020-04-14 23:16:55 +01:00
Paul Spooren
14cbd8fb2d scripts: JSON merge don't crash if no JSON found
The JSON `WORK_DIR` ($(KDIR)/json_info_files) is only created if the new
image generation methods from `image.mk` are used. However some targets
like `armvirt` do not use it yet, so the folder is never created.

The `json_overview_image_info.py` script used to raise an error if the
given `WORK_DIR` isn't a folder, however it should just notify about
missing JSON files.

This patch removes the Python assert and exists with code 0 even if no
JSON files were found, as this is not necessarily an error but simply
not yet implemented. Using `glob` on an not existing `Path` results in
an empty list, therefore the for loop won't run.

Signed-off-by: Paul Spooren <mail@aparcar.org>
CC: Petr Štetiar <ynezz@true.cz>
2020-04-14 23:16:55 +01:00
Jo-Philipp Wich
7b1d809a8d Revert "build: scripts/config - update to kconfig-v5.6"
This reverts commit dcf3e63a35.

The kconfig update requires further testing and refinement until it can
remain in tree. Main problems are:

 - Recursive deps are now fatal instead of a warning
 - Previously legal syntax now leads to hard failures
 - It fails all package builds since multiple days

The updated kconfig implementation needs to cope with the current status
quo in the various package feeds before we can reconsider it for master.

It is not desirable that single broken packages can hard-fail the entire
build pipeline.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2020-04-12 00:10:47 +02:00
Jo-Philipp Wich
6c16d64b2a Revert "build: add option to warn on recursive dependency"
This reverts commit 3204430e38.

Reverting this commit in preparation for reverting
dcf3e63a35 ("build: scripts/config - update to kconfig-v5.6") which
introduces various unaddressed build breakages.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2020-04-12 00:10:30 +02:00
Jo-Philipp Wich
fbc01f2a4c Revert "build: config: allow bool to select a module pkg"
This reverts commit 8514b6b42c.

Reverting this commit in preparation for reverting
dcf3e63a35 ("build: scripts/config - update to kconfig-v5.6") which
introduces various unaddressed build breakages.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2020-04-12 00:10:16 +02:00
Eneas U de Queiroz
8514b6b42c build: config: allow bool to select a module pkg
kconfig-v5.6 disallowed a bool symbol to select another symbol that
'depends on m' (i.e. can be only 'm' on 'n').  It is, in fact, an unmet
dependency to have set to 'y'.  However, openwrt depends on the previous
behavior, to be able to build a package that can be a module or built-in
depending on a bool config.  This restores the previous behavior.

Ref: https://forum.openwrt.org/t/wireless-fails-on-snapshot-r12900-kernel-5-4-on-c2600-ipq806x
Tested-by: Hannu Nyman <hannu.nyman@iki.fi>
Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
Signed-off-by: Petr Štetiar <ynezz@true.cz> [added forum ref]
2020-04-11 19:06:26 +02:00
Eneas U de Queiroz
3204430e38 build: add option to warn on recursive dependency
This addes the option to treat recursive dependencies as warnings
instead of errors, by running make with WARN_RECURSIVE_DEP=1.

Note that the script/config targets will not get rebuilt when you add or
remove WARN_RECURSIVE_DEP while running make.  One must run
'make config-clean' before building config with a different setting.

Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
2020-04-09 12:51:10 +02:00
Eneas U de Queiroz
dcf3e63a35 build: scripts/config - update to kconfig-v5.6
Major changes include:
 - Much more readable reverse dependencies separated in groups
 - Improved recursive dependency report
 - More readable .config files: add comments to signal end of menus
 - More warnings for incorrect Config.in entries, such as a 'choice'
   default not contained in the 'choice'
 - Hability to properly display pseudographics with non-latin locales
 - Recursive dependencies are now treated as errors - this should make
   it harder for them to creep in.

Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
2020-04-09 12:51:10 +02:00
Eneas U de Queiroz
8636a17214 build: simplify building *config targets
Instead of passing pkg-config location through a variable when building
qconf (make xconfig), prepend its parent directory to the PATH, as it is
being done for other conf targets.

Use a Makefile pattern rule to group all 'scripts/config/%onf'
(currently conf, mconf, qconf) targets in a single rule.  Add -O2 to
CFLAGS when building them as well.

Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
2020-04-09 12:51:10 +02:00
Paul Spooren
07449f692c build: refactor JSON info files to profiles.json
JSON info files contain machine readable information of built profiles
and resulting images. These files were added in commit 881ed09ee6
("build: create JSON files containing image info").

They are useful for firmware wizards and script checking for
reproducibility.

Currently all JSON files are stored next to the built images, resulting
in up to 168 individual files for the ath79/generic target.

This patch refactors the JSON creation to store individual per image
(not per profile) files in $(BUILD_DIR)/json_info_files and create an
single overview file called `profiles.json` in the target directory.

Storing per image files and not per profile solves the problem of
parallel file writes. If a profiles sysupgrade and factory image are
finished at the same time both processes would write to the same JSON
file, resulting in randomly broken outputs.

Some target like x86/64 do not use the image code yet, resulting in
missing JSON files. If no JSON info files were created, no
`profiles.json` files is created as it would be empty anyway.

As before, this creation is enabled by default only if `BUILDBOT` is set.

Tested via buildroot & ImageBuilder on ath79/generic, imx6 and x86/64.

Signed-off-by: Paul Spooren <mail@aparcar.org>
[json_info_files dir handling in Make, if case refactoring]
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-04-03 12:17:45 +02:00
李国
a6b7c3e672 x86: generate EFI platform bootable images
Add EFI platform bootable images for x86 platforms. These images can
also boot from legacy BIOS platform.

EFI System Partition need to be fat12/fat16/fat32 (not need to load
filesystem drivers), so the first partition of EFI images are not ext4
filesystem any more.

GPT partition table has an alternate partition table, we did not
generate it. This may cause problems when use these images as qemu disk
(kernel can not find rootfs), we pad enough sectors will be ok.

Signed-off-by: 李国 <uxgood.org@gmail.com>
[part_magic_* refactoring, removed genisoimage checks]
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-03-31 16:20:47 +02:00
Paul Spooren
2cb6d471b7 scripts: fixup qemustart for new x86 image names
qemustart allows easy testing of created images via `qemu`. The script
automatically selects created images and can setup e.g. networks.

As the x86 target now uses the generic image.mk the profile appears also
in the image name, this is *generic*.

Add the profile name to the qemustart script so it still finds the file.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-03-21 10:36:00 +00:00
Jeffery To
f96cfe019a build: Fix directory symlinks not removed when cleaning STAGING_DIR
Currently, a symbolic link whose target is a directory will not be
removed when cleaning packages from STAGING_DIR.

In the first cleaning pass in scripts/clean-package.sh, the -f test for
a directory symlink returns false (because the link target is a
directory) and so the symlink is not removed.

In the second pass, the -d test returns true for a directory symlink,
but the symlink is not removed by rmdir because rmdir only removes
(real) directories.

This updates clean-package.sh to remove all non-directories (including
symbolic links) in the first pass.

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
2020-03-01 21:35:59 +01:00