Commit Graph

740 Commits

Author SHA1 Message Date
Matthias Schiffer
de0ccd3532
metadata: improve code readability by using (s)printf instead of string interpolation
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-01-13 19:54:45 +01:00
Matthias Schiffer
4bb54ed15e
build: remove obsolete "package feature" feature
Package "features" seem to be unused for some time. In any case, custom
Config.in snippets and package PROVIDES are a much more flexible way to
express similar options.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-01-13 19:54:45 +01:00
Matthias Schiffer
52719c2b67
metadata: scripts/feeds: distinguish between source and binary packages, resolve virtual dependencies
Properly resolve build depends to source packages and runtime depends to
binary packages. Dependencies on virtual packages are resolved to the first
provider now.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-01-13 19:54:45 +01:00
Matthias Schiffer
47d6b05ad3
metadata: always resolve dependencies through provides list
Instead of adding virtual packages to the normal package list, keep a
separate list for provides, make each package provide itself, and resolve
all dependencies through this list. This allows to use PROVIDES to replace
existing packages.

Fixes FS#837.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-01-13 19:54:44 +01:00
Matthias Schiffer
565de8b8a8
metadata: simplify generation of build depends from runtime depends
Runtime depends cannot have a buildtype suffix, and they never refer to
source package names. In addition, this adds warnings about unsatisfiable
dependencies.

Furthermore, this change fixes the generation of conditional build
dependencies for virtual packages provided by different source packages.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-01-13 19:54:44 +01:00
Matthias Schiffer
5b479f7dcf
metadata: handle target build depends together with host build depends
Target build depends are similar to host build depends in that they refer
to source packages rather than binary packages. Therefore, it makes sense
to handle them together, rather than putting them in a list together with
runtime depends and trying to figure out if the entries refer to source or
to binary packages afterwards.

This does lead to PKG_BUILD_DEPENDS entries referring to binary package
names not working anymore, which requires some fixes in the package
repositories.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-01-13 19:54:44 +01:00
Matthias Schiffer
3abf663c22
build: remove package preconfig feature
This feature has been unused for years, and its scope is too limited to be
actually useful.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-01-13 19:54:44 +01:00
Matthias Schiffer
423ec18116
metadata: remove redundant fields from package hash
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-01-13 19:54:44 +01:00
Matthias Schiffer
f0c702f124
metadata: replace %subdir hash with a path field in source packages
Every single reference to subdir was concatenated with the source package
name, so it makes sense to store the concatenated value instead.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-01-13 19:54:44 +01:00
Matthias Schiffer
504cb913eb
metadata: change pkg->{src} field to hold a reference
We often want to access fields of a source packages through pkg->{src}.
Allow accessing them directly instead of resolving the source hash through
srcpackages.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-01-13 19:54:44 +01:00
Matthias Schiffer
47f292ff78
metadata: interate over source packages when generating Makefile
All build dependencies are between source packages. Interating over source
rather than binary packages simplifies parts of the code and prepares
further improvement.

As a side effect, this changes the implicit default variant of a few
packages (the first defined is used now instead of the lexicographically
first).

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-01-13 19:54:44 +01:00
Matthias Schiffer
b2ab9c9d33
metadata: move 'builddepends' from binary to source packages
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-01-13 19:54:43 +01:00
Matthias Schiffer
39be6e7f19
metadata: move 'buildtypes' from binary to source packages
Build types are a property of source rather than binary packages. This is a
preparation for followup cleanup.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-01-13 19:54:43 +01:00
Matthias Schiffer
19577582e4
metadata: make srcpackage extensible
Turn the srcpackage values into hashes to allow storing more information
than just binary package names.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-01-13 19:54:43 +01:00
Matthias Schiffer
1a2b5eb7ac
metadata: remove 'base-files' special case
Nothing explicitly depends on base-files, and even if it would, it would
not cause any problems. Remove the unused special case.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-01-13 19:54:43 +01:00
Sven Eckelmann
b02fa3336b om-fwupgradecfg-gen.sh: add support for the A42
Signed-off-by: Sven Eckelmann <sven.eckelmann@open-mesh.com>
2018-01-13 07:58:49 +01:00
Sven Eckelmann
1a384ac0b5 combined-ext-image.sh: generate image in temp dir
The new build commands operate on the input image and use it again as
output image. This conflicts with the way combined-ext-image.sh was
operating. It required that input and output files are different files and
and that it can write freely to the output file.

This can be avoided when all intermediate build steps by
combined-ext-image.sh are done in a temporary directory. The output file is
then only overwritten in the last step.

Signed-off-by: Sven Eckelmann <sven.eckelmann@open-mesh.com>
2018-01-13 07:58:46 +01:00
Matthias Schiffer
0b28cc56d4
scripts/package-metadata.pl: remove broken detection of self-dependencies
A self-dependency is not an error worth a warning; rather, it is very
common: whenever there are dependencies between different binary packages
originating from the same source package, such dependencies occur. Not
actually generating dependency rules is correct, but already handled a few
lines below.

A typo prevented this redundant rule from working, which is the reason the
warning was not actually printed.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-01-04 17:18:16 +01:00
Jo-Philipp Wich
f4b9d9d6f1 base-files: allow skipping of hash verification
When calling a download target, hash verification is now completely
skipped if we set PKG_HASH=skip.

This allows to easily bump package version:

$ make package/<mypackage>/download PKG_HASH=skip V=s
$ make package/<mypackage>/check FIXUP=1 V=s

This will download the new version of the package, and then automatically
update PKG_HASH with the hash of the new version.  Of course, it is still
the responsibility of the packager to ensure that the new tarball is
legitimate, because it is downloaded from a possibly untrusted source.

Fixes: b30ba14e ("scripts/download.pl: fail loudly if provided hash is unsupported")
Signed-off-by: Baptiste Jonglez <git@bitsofnetworks.org>
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Acked-by: Stijn Tintel <stijn@linux-ipv6.be>
Signed-off-by: John Crispin <john@phrozen.org>
2017-12-14 09:29:31 +01:00
Zoltan HERPAI
7b5c989ab9 merge: targets: update image generation and targets
Signed-off-by: Zoltan HERPAI <wigyori@uid0.hu>
2017-12-08 19:41:18 +01:00
Zoltan HERPAI
34bbbbf9c3 merge: base: update base-files and basic config
Signed-off-by: Zoltan HERPAI <wigyori@uid0.hu>
2017-12-08 19:41:18 +01:00
Alif M. Ahmad
b1501a684b scripts/config: add qconf files to .gitignore
This commit adds qconf related files to .gitignore.
The files to be tracked are qconf.cc, qconf.h, and images.c.
The files to be ignored are qconf*.o, qconf*, qconf.moc, and
.tmp_qtcheck.

Signed-off-by: Alif M. Ahmad <alive4ever@live.com>
2017-11-02 15:58:45 +01:00
Alif M. Ahmad
ff43c03c14 scripts/config: Add qconf target to Makefile
qconf is kconfig UI utilizing QT toolkit. This makes it possible to use
graphical interface interaction to configure LEDE build target.

This commit adds qconf target to ./script/config/Makefile to be used by
toplevel ``make xconfig`` later.

Signed-off-by: Alif M. Ahmad <alive4ever@live.com>
2017-11-02 15:58:45 +01:00
Felix Fietkau
349bb6e663 scripts/config: add -DKBUILD_NO_NLS to CXXFLAGS
Preparation for adding C++ code which needs the same flag

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-11-02 15:58:45 +01:00
Alif M. Ahmad
96eb96c3f0 scripts/config: import qconf
Import qconf.cc, qconf.h, and images.c from linux kernel 4.9.13 archive.

Signed-off-by: Alif M. Ahmad <alive4ever@live.com>
2017-11-02 15:58:45 +01:00
Lorenzo Santina
fd84ecda7d treewide: fix shellscript syntax errors/typos
Fix multiple syntax errors in shelscripts (of packages only)
These errors were causing many conditions to not working properly

Signed-off-by: Lorenzo Santina <lorenzo.santina@edu.unito.it>
[increase PKG_RELEASE, drop command substitution from directip.sh]
Signed-off-by: Mathias Kresin <dev@kresin.em>
2017-09-13 08:07:54 +02:00
Baptiste Jonglez
b30ba14e2a scripts/download.pl: fail loudly if provided hash is unsupported
Currently, if the provided hash is unsupported (length different from 32
or 64 bytes), we happily download the requested file without any kind of
checksum verification.

This is quite dangerous and may provide a false sense of security, because
a single typo in the hash (e.g. one character deleted by mistake) may skip
checksum verification entirely.

Instead, fail immediately if we don't support the provided hash.
In particular, if an external package repository decides to change the
hash algorithm one day, we will now fail loudly instead of skipping
checksum verification without complaints.

Note: if some users of scripts/download.pl knowingly provide an empty hash
because they don't need checksum verification, this change will break
them.  This does not seem to be the case currently, but if this feature is
ever needed, an option should be added to download.pl instead of relying
on the hash being empty.

Fixes: eaa4eba10a ("scripts/download.pl: add SHA-256 support")

Signed-off-by: Baptiste Jonglez <git@bitsofnetworks.org>
2017-09-11 17:13:02 +02:00
Zoltan Gyarmati
0a761fe8ef scripts/dowload.pl: use glob to expand target dir
If CONFIG_DOWNLOAD_FOLDER is set to for example "~/dl", the download
script fails to create the .hash and .dl files with the following
errors:
 Cannot create file ~/dl/dropbear-2017.75.tar.bz2.dl: No such file or directory
 sh: 1: cannot create ~/dl/dropbear-2017.75.tar.bz2.hash: Directory nonexistent

If the tarball already exists in the ~/dl dir, it's properly found and
used, so this issue only affects the download.pl script.
 This patch calls glob() on the target dir parameter, which will expand `~`.

Signed-off-by: Zoltan Gyarmati <mr.zoltan.gyarmati@gmail.com>
2017-08-30 18:12:36 +02:00
Jo-Philipp Wich
bf5d32af2a scripts/package-metadata.pl: inhibit compile deps on missing build types
When a package declares a PKG_BUILD_DEPENDENCY or HOST_BUILD_DEPENDENCY on
a not existing build type, the metadata script will emit a reference to an
unresolvable build target in tmp/.packagedeps, causing the make process to
fail hard in a way not catchable by the IGNORE_ERRORS mechanism.

In a situation where a package "test-a" declares a build dependency
"PKG_BUILD_DEPENDS:=test-b/host" while the Makefile of "test-b" does not
implement a HostBuild, make fails with an unrecoverable error in the form:

    make[1]: Entering directory '...'
    make[1]: *** No rule to make target 'package/test-b/host/compile',
                 needed by 'package/test-a/compile'.  Stop.
    make[1]: Leaving directory '...'
    .../toplevel.mk:200: recipe for target 'package/test-a/compile' failed
    make: *** [package/test-a/compile] Error 2

Extend the metadata generation script to catch such unresolved references
and emit a visable warning upon detection.

After this change, the script will emit a warning similar to:

    WARNING: Makefile "package/test-a/Makefile" has a build dependency on
    "test-b/host" but "package/test-b/Makefile" does not implement a
    "host" build type

Fixes a global build cluster outage which occured after the "python-cffi"
feed package removed its HostBuild which the "python-cryptography" package
build-depended on.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2017-07-27 00:37:05 +02:00
Jo-Philipp Wich
9e64874fb2 build: bundle-libraries.sh: do not override argv[0] in inner exec calls
Only mangle argv[0] of the first executed process and leave the argument
vector of subsequent invocations as-is to allow child programs to properly
discover resources relative to their binary locations.

Fixes "cc1" discovery when executing the host gcc through the bundled
"ccache" executable.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2017-07-24 13:39:21 +02:00
Daniel Engberg
99c4295126 scripts/download.pl: Adjust URLs
Internet2 isn't considered a trusted issuer meaning that https links to
rit.edu will fail.

The host mirror.csclub.uwaterloo.ca has a trusted SSL cert and peering
is good so it can replace rit.edu without performance issues.

Signed-off-by: Daniel Engberg <daniel.engberg.lists@pyret.net>
[Jo-Philipp Wich: rewrapped commit message]
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2017-07-21 08:09:45 +02:00
Jo-Philipp Wich
ef1cafa736 build: fix invocation of bundled ld.so in SDK and Imagebuilder
Commit 72d751cba9 "build: rework library bundling" introduced a new helper
binary "runas" whose sole purpose was mangling the argv vector passed to
the actual called ELF image so that the renamed executable could obtain the
proper name from argv[0].

This approach, however totally defeated the purpose of calling bundled ELF
executables through the shipped ld.so loader since the execv() invocation
performed by "runas" would cause the kernel the interprete the final program
image through the system ELF loader again.

To solve the problem, use an alternative approach of shipping a shared object
"runas.so" which uses an ELF ".init_array" function pointer to obtain the
argv[] vector of the to-be-executed main() function and mangle it in-place.

The actual argv[0] value to use is communicated out-of-band using an
environment variable "RUNAS_ARG0" by the shell wrapper script. The wrapper
script also takes care of setting LD_PRELOAD to instruct the shipped ELF
loader to preload the actual ELF program image with the "runas.so" helper
library.

Fixes FS#909.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2017-07-21 08:09:45 +02:00
Stijn Tintel
6a8cbc08c0 scripts/gen_image_generic.sh: drop NOGRUB variable
It's not used since commit 816a9b30f6.

Signed-off-by: Stijn Tintel <stijn@linux-ipv6.be>
2017-07-14 04:09:16 +02:00
Stijn Tintel
a576ed5437 scripts/gen_image_generic.sh: move from x86/image
The script can also be useful for other targets.

Signed-off-by: Stijn Tintel <stijn@linux-ipv6.be>
2017-07-14 04:09:16 +02:00
Stijn Tintel
c2a11bc8e4 scripts: make all scripts executable
Signed-off-by: Stijn Tintel <stijn@linux-ipv6.be>
2017-07-14 04:09:16 +02:00
Stijn Tintel
cf96453ac9 scripts/openbsd.sh: remove script
Commit 6f5f328003 removed freebsd.sh
because it was outdated and bad practice. Let's be consistent and remove
openbsd.sh as well.

Signed-off-by: Stijn Tintel <stijn@linux-ipv6.be>
2017-07-14 04:09:16 +02:00
Yousong Zhou
80d9ec5d3d scripts/package-metadata.pl: parse and validate field Require-User
The script will now detect uid/gid collision and can generate a table of
current allocation

    ./scripts/package-metadata.pl usergroup tmp/.packageinfo \
	| sort -k 1,1r -k 3,3n \
	| column -t

This should ensure that no collision will happen for each single build

Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
2017-06-18 10:39:35 +08:00
Felix Fietkau
cbe0a7ecc0 scripts/download.pl: print the command used to download files
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-05-25 19:01:07 +02:00
Florian Fainelli
6704410b15 scripts: Probe external toolchains for libthread-db
libthread-db is a package that can be configured for external
toolchains, so let's have the script probe for it.

Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
2017-05-11 13:43:01 -07:00
Florian Fainelli
08fb61d316 scripts: Probe external toolchains for libgomp
Now that libgomp can be a package, probe external toolchains for that
library.

Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
2017-05-11 13:42:58 -07:00
Bastian Köcher
5378c85677 build: remove absolute path to perl and replace with /usr/bin/env perl
Signed-off-by: Bastian Köcher <git@kchr.de>
2017-05-02 14:33:58 +02:00
Yousong Zhou
50fcfd4b65 scripts/qemustart: initial version
The script can be used to quickly spin up QEMU virtual machines with lan
and wan network.  Please read the initial part of the script for
instructions about how to configure host machine

Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
2017-04-23 09:19:34 +08:00
Felix Fietkau
2fd1503378 scripts/download.pl: fix error message on hash mismatch
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-04-04 12:33:37 +02:00
Hauke Mehrtens
d5a770f2c1 scripts: only generate config from feature flag if fully match
Without this change the code checked if the string was contained in the
feature option and not if the string matches the complete word. This only
removes the nand option from the omap24xx target, the other changes are
only removing options which were added twice.

Signed-off-by: Hauke Mehrtens <hauke.mehrtens@intel.com>
2017-03-20 22:04:41 +01:00
Michal Sojka
b1f102b03e scripts/feeds: Reuse TOPDIR if defined in environment
The feeds script sets value of TOPDIR in a way that is inconsistent
with how toplevel Makefile sets it. The inconsistency manifests when I
use a "build directory" with symlinks to LEDE source (see below).

When make is invoked in such a directory, make's TOPDIR variable is
set to that directory, whereas scripts/feeds sets TOPDIR to the top of
LEDE source, which results in creating feeds directory inside the LEDE
source instead of in the build directory.

This patch changes the script so that it reuses the TOPDIR value form
the environment if it exists. The result is that 'make
package/symlinks' correctly fetches feeds to the build directory
instead in the source.

I use the following commands to create the build directory:

    ln -s $SRC/config config
    ln -s $SRC/Config.in Config.in
    ln -s $SRC/feeds.conf.default feeds.conf.default
    ln -s $SRC/include include
    ln -s $SRC/Makefile Makefile
    mkdir package
    ln -s $SRC/package/base-files package/base-files
    ln -s $SRC/package/boot package/boot
    ln -s $SRC/package/devel package/devel
    ln -s $SRC/package/firmware package/firmware
    ln -s $SRC/package/kernel package/kernel
    ln -s $SRC/package/libs package/libs
    ln -s $SRC/package/Makefile package/Makefile
    ln -s $SRC/package/network package/network
    ln -s $SRC/package/system package/system
    ln -s $SRC/package/utils package/utils
    ln -s $SRC/rules.mk rules.mk
    ln -s $SRC/scripts scripts
    ln -s $SRC/target target
    ln -s $SRC/toolchain toolchain
    ln -s $SRC/tools tools

This allows me to easily test changes in LEDE on multiple targets.

Signed-off-by: Michal Sojka <sojkam1@fel.cvut.cz>
2017-02-21 13:03:20 +01:00
Felix Fietkau
663f2a0864 gen-dependencies.sh: fix handling variations in "file" output
On some systems, file adds ", with debug info" after "not stripped"

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-02-19 16:56:18 +01:00
Felix Fietkau
b9b681c067 rstrip.sh: fix handling variations in "file" output
On some systems, file adds ", with debug info" after "not stripped"

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-02-19 16:56:17 +01:00
Jo-Philipp Wich
3bfcd21e4b feeds: add option to force feed update despite modified files
Implement a new flag "-f" for the feeds update command which causes the
script to fall back to a more agressive git update strategy in case there
are locally modified files in the feeds directory.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2017-02-15 10:07:00 +01:00
Felix Fietkau
9dcb921d90 build: add buildbot specific config option for setting defaults
This can be used to tweak the buildbot behavior without having to change
buildbot's configuration.
It will also allow us to add more aggressive clean steps (e.g. on
toolchain changes), which would break developers' workflows if enable
by default.

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-02-14 14:18:35 +01:00
Sven Eckelmann
8cc4d1e27b scripts/om-fwupgradecfg-gen.sh: add support for the A60
Signed-off-by: Sven Eckelmann <sven.eckelmann@open-mesh.com>
2017-02-14 12:18:05 +01:00
Felix Fietkau
f55c29e4c7 build: remove libc version suffix from build/staging directories
Our supported libcs are ABI compatible across version upgrades

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-02-07 17:18:15 +01:00
Jo-Philipp Wich
68bef935d5 scripts: get_source_date_epoch.sh: fix mercurial support, add mtime fallback
Add a fallback case to get_source_date_epoch.sh which reports the modification
time of the script itself in case there is no SCM information available, e.g.
when downloading .tar.gz or .zip tarballs produced by Github.

Also fix the mercurial case while we're at it.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2017-02-02 00:13:50 +01:00
Magnus Kroken
289f2a8ce3 scripts/getver.sh: append short git hash based on upstream commit
The short git hash suffix printed by getver.sh is taken from the
latest local commit, change this to use the hash from latest
upstream commit if available. This is considered the intended
behavior based on commit message a642a11fac,
introducing getver.sh.

Signed-off-by: Magnus Kroken <mkroken@gmail.com>
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-02-01 17:48:45 +01:00
Felix Fietkau
74c09e714e download.pl: fix detecting download errors with curl
Avoid treating error pages as successfully downloaded content

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-31 11:24:19 +01:00
Felix Fietkau
cfd83555fc scripts/package-metadata.pl: fix overriding conditional dependencies with conditional select
Conditional dependencies use the '(!cond) || dep' syntax, whereas
conditional select uses 'dep if cond'.
Add an extra check to suppress emitting a conditional if an equal
conditional select already exists.

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-11 18:24:39 +01:00
Jo-Philipp Wich
72d751cba9 build: rework library bundling
Rework the bundle-libraries.sh implementation to use a more robust approach
for executing host binaries through the shipped ELF loader and libraries.

The previous approach relied on symlinks pointing to a wrapper script which
caused various issues, especially with multicall binaries as the original
argv[0] name was not preserved through the ld.so invocation. Another down-
side was the fact that the actual binaries got moved into another directory
which caused executables to fail looking up resources with paths relative
to the executable location.

The new library wrapper implements the following improvements:

 - Instead of symlinks pointing to a common wrapper, each ELF executable
   is now replaced by a unqiue shell script which retains the original
   program name getting called

 - Instead of letting ld.so invoke the ELF executable directly, launch
   the final ELF binary through a helper program which fixes up the argv[0]
   argument for the target program

 - Support sharing a common location for the bundled libraries instead of
   having one copy in each directory containing wrapped binaries

Finally modify the SDK build to wrap the staging_dir and toolchain binaries
which allows to use the SDK on systems with a different glibc version.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2017-01-10 12:27:28 +01:00
Felix Fietkau
84bd74057f build: use mkhash to replace various quirky md5sum/openssl calls
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-05 11:09:12 +01:00
Felix Fietkau
dad48c6438 build: add a small standalone utility for calculating md5/sha256 hash
This will be used to simplify the build system code for checking hashes.
Instead of using various variants of md5sum / openssl, use one simple
utility for all of them

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-05 11:09:12 +01:00
Brian J. Murrell
fc6b6f4583 download.pl: use curl in preference to wget
Because wget doesn't know how to do Negotiate authentication with a proxy
and curl does, use curl if it's present. The user is expected to have a
~/.curlrc that sets the options necessary for any proxy authentication.

A ~/.curlrc is completely optional however and curl will work in exactly
the same manner as wget without one.

Signed-off-by: Brian J. Murrell <brian@interlinx.bc.ca>
[Jo-Philipp Wich: Rework code to detect curl usability by checking --version,
                  Use vararg style open() to bypass the shell when downloading,
                  Use Text::ParseWords to decompose env vars into arguments]
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2017-01-03 14:26:41 +01:00
Alberto Bursi
c24172cad1 package/Makefile & ipkg-make-index.sh: add full package data list
The external script used to generate the package lists for the
LEDE wiki's table of packages [1] and package indexes [2] requires
a "Source:" field in the package lists to find package makefiles.
The package makefiles are used to read the package's Category and Submenu.

The "Source:" field was removed in commit
b4aa3c899c
to reduce package list sizes and lessen opkg issues in low ram devices.

Add a separate package list file with full data to be used by the wiki's script.
It's called Packages.manifest and isn't compressed as it's not necessary.

1. https://lede-project.org/packages/start
2. https://lede-project.org/packages/index/start

Signed-off-by: Alberto Bursi <alberto.bursi@outlook.it>
2016-12-26 11:17:33 +01:00
Felix Fietkau
78a1b6e880 scripts/update-package-md5sum: remove file, it is obsoleted by make check FIXUP=1
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-12-17 10:37:00 +01:00
Felix Fietkau
d49059693b build: add FIXUP option for make check
This will attempt to automatically fix common mistakes like using MD5
instead of SHA256, using the MD5SUM variable instead of HASH, or even a
missing mirror file hash.

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-12-17 10:36:45 +01:00
Felix Fietkau
651bc94df4 download.pl: check for existing file before the first download attempt
In the build system, flock will prevent multiple concurrent downloads
for the same file. However, if one download request for the same file is
waiting for another one to finish, it will result in downloading the
same file twice consecutively.

Prevent this issue by exiting immediately if the file has already been
downloaded

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-12-16 15:16:47 +01:00
Felix Fietkau
b4aa3c899c ipkg-make-index.sh: drop a few non-essential fields
Saves some more space on the package index

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-12-14 12:13:14 +01:00
Felix Fietkau
34c2b3de66 ipkg-make-index.sh: drop md5sum from package index
We have switched opkg to sha256 a long time ago, and shrinking package
lists is useful for systems that are running low on RAM

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-12-14 12:13:14 +01:00
Jo-Philipp Wich
a642a11fac scripts: getver.sh: append Git short hash to revision
Change getver.sh to append a short Git commit hash to the end of the artifical
revision number. This way we still have order- and comparable commit numbers
but also a direct relation to the Git commit.

The new output format will look like "r2400+2-882472e" for dirty trees or like
"r2402-882472e" for clean ones.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2016-12-02 16:38:55 +01:00
Mathias Kresin
59261cbf38 docs: remove all refrences in Makefiles/scripts
The SDK Makefile still trys to copy the docs folder which was removed
with 882f4d2d63. This causes an SDK build
error.

All other removals are just cleanup.

Signed-off-by: Mathias Kresin <dev@kresin.me>
2016-11-30 10:13:14 +01:00
Rafał Miłecki
bf3d92f0cd scripts/getver.sh: treat all commits as local if can't find upstream
If something goes wrong and script can't find upstream revision it will
return something like:
r2220
which looks like a valid upstream revision 2220. We cant' distinguish it
from e.g. 2200 upstream commits and 20 local ones.

The new format still provides revision number but also points clearly
that is may be not the upstream one:
r0+2220

Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
Acked-by: John Crispin < john@phrozen.org>
2016-11-21 14:36:04 +01:00
Rafał Miłecki
0a6439154a scripts/feeds: use git rev-parse for getting revision
It provides simpler output so we don't need extra head and cut commands.

Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
2016-11-08 11:17:11 +01:00
Rafał Miłecki
4f7947dab8 scripts/feeds: display "X" as revision of uninitialized feeds
So far we were displaying "local" which could be misinterpreted. It
wasn't possible e.g. to say if src-link feed was initialized or not.
Hopefully "X" makes (a bit) more sense.

Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
2016-11-08 11:17:11 +01:00
Rafał Miłecki
a2944a0c09 scripts/feeds: use 10 chars for feed name column width
It's always hard to find a reasonable width that will make everyone
happy. This one at least makes "telephony" (one of default feeds) name
fit the column and hopefully isn't too big.

Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
2016-11-03 08:34:38 +01:00
Jo-Philipp Wich
12d15fa8a5 scripts/package-metadata.pl: honour DEFAULT_VARIANT
So far, package-metadata.pl always considered the first provider of a virtual
package to be the default variant which might deviate from what buildroot
considers to be the default.

Change the Kconfig dependency / select code generation for virtual package
providers to consider the DEFAULT_VARIANT to be the primary provider and only
fall back to the first provider if no default variant was explicitely tagged.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2016-11-02 02:02:44 +01:00
Jo-Philipp Wich
f64360c7ca scripts/package-metadata.pl: fix handling of virtual (PROVIDES) depends
Currently the code emitting dependencies for provide candidates is overwriting
the specification calculated by the previous conditional dependency handling
code, rendering dependencies on virtual PROVIDES packages in conjunction with
conditional dependencies unusable.

Instead of overwriting, append the PROVIDES dependency spec in order to fix
using DEPENDS on virtual provider packages in conjunction with conditions.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2016-11-02 01:01:34 +01:00
Daniel Engberg
6f5f328003 scripts/freebsd.sh: Remove script
This is outdated and bad practice,  general dependencies should be documented and leave the rest up to the user.

Signed-off-by: Daniel Engberg <daniel.engberg.lists@pyret.net>
2016-10-15 11:36:53 +02:00
Karl Palsson
c511795f47 scripts: case insensitive sort device names
When selecting devices from the Target Devices menu, the brand choices
of naming makes it confusing to find particular devices by name, as the
sorting is case sensitve.  AirTight came after ALFA, and devolo and
jjPlus both came after Zyxel.

This does _not_ apply to the Target Profile list, as that includes
"Default - all profiles" inside the profile list.

Signed-off-by: Karl Palsson <karlp@etactica.com>
2016-10-15 11:36:51 +02:00
Karl Palsson
5dc56b4123 scripts: add help text for some generated KConfigs
TARGET_MULTI_PROFILE and TARGET_PER_DEVICE_ROOTFS get some help text to
try and clarify their behaviour.

Signed-off-by: Karl Palsson <karlp@etactica.com>
2016-10-15 11:36:51 +02:00
Zefir Kurtisi
cf458de382 scripts: fix build warning when overriding packages
If core packages are overridden, CONFIG_OVERRIDE_PKGS is set
based on the scan order of packages, which eventually causes
that config value to be modified on each build and with
that causes the build process to warn for configuration
being out of sync.

This commit changes the CONFIG_OVERRIDE_PKGS to be sorted
and prevents that false warning.

Signed-off-by: Zefir Kurtisi <zefir.kurtisi@neratec.com>
2016-10-15 11:36:51 +02:00
Mathias Kresin
82b8eaa89f scripts/diffconfig.sh: fix output if TARGET_PER_DEVICE_ROOTFS is set
diffconfig show packages selected as dependencies of
TARGET_PER_DEVICE_ROOTFS as individual selected.

Signed-off-by: Mathias Kresin <dev@kresin.me>
2016-10-06 22:00:10 +02:00
Matthias Schiffer
e916579340
image: allow specifying additional packages for device-specific rootfs
Add a new option to each device in multi-profile mode, allowing to provide
a list of packages to add or remove. In case of added packages, the user
must take care that these are selected to be built.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2016-09-24 18:25:25 +02:00
Matthias Schiffer
6177b649ca
scripts/package-metadata.pl: fix generation of dependencies on virtual packages
Dependencies on purely virtual packages (satisfied by PROVIDES) that were
not using "selects" ("+" flag) would be prepended with the prefix
"PACKAGE_" twice, breaking the first alternative.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2016-09-23 23:17:09 +02:00
Felix Fietkau
a84a74f618 scripts/ubinize-image.sh: add support for adding custom partitions
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-08-31 13:05:19 +02:00
Jo-Philipp Wich
3a3424981c scripts: ipkg-build: do not require git or svn
Move the "which svn" and "which git" calls next to the timestamp commands
using those tools to not prematurely fail on systems where svn or git are
not present.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2016-08-15 13:33:32 +02:00
Matteo Croce
1645abffea kernel: add plan 9 fs package
9pfs is used by kvm to share files between host and guest,
add proper config option to enable it.

Signed-off-by: Matteo Croce <matteo.croce@canonical.com>
2016-08-11 10:45:33 +02:00
Felix Fietkau
9bfa6971ae scripts/config: properly handle select on symbols with unmet direct dependencies
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-08-03 20:03:36 +02:00
Felix Fietkau
2d7e602381 scripts/config: sync with latest linux upstream
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-08-03 20:03:35 +02:00
Hauke Mehrtens
34bffe5806 scripts: fix remote-gdb with CONFIG_BUILD_SUFFIX
Fix the scripts/remote-gdb script when CONFIG_BUILD_SUFFIX is set.
CONFIG_BUILD_SUFFIX extends the name of the folder build_dir/target*

Signed-off-by: Hauke Mehrtens <hauke@hauke-m.de>
2016-08-01 22:02:49 +02:00
Jo-Philipp Wich
db49dd894e build: rename sysupgrade-nand to sysupgrade-tar
Now that the "sysupgrade-nand" step is used by non-NAND targets as well,
rename it to "sysupgrade-tar" to make it more generic.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2016-07-29 16:53:03 +02:00
Felix Fietkau
f4ce133ccf build: add option to enable all profiles
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-07-29 10:18:26 +02:00
Felix Fietkau
76341cfc5f build: add support for per-device rootfs based on device profile packges
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-07-29 10:18:26 +02:00
Jonas Gorski
efdd3bf5fb scripts/getver.sh: fix older git versions from printing stuff to stdout
Older git versions seem output the original argument to stdout if there
is no upstream, presumably because they try to do things with it
internally. This can be prevented by passing --verify to it, which
should be safe on newer git versions.

Signed-off-by: Jonas Gorski <jonas.gorski@gmail.com>
2016-07-16 20:50:56 +02:00
Felix Fietkau
7ed215437c scripts/getver.sh: stop relying on the reboot tag
It may not be present when cloning a staging tree or another fork

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-07-14 13:37:58 +02:00
Jonas Gorski
6f86d2e2ab scripts/getver.sh: fix one more wc -l call
The revision to hash conversion was missed when fixing up the script.

Signed-off-by: Jonas Gorski <jonas.gorski@gmail.com>
2016-07-13 16:56:41 +02:00
Jonas Gorski
4eb5aad667 scripts/getver.sh: try to get branch/upstream automatically
Instead of assuming master is the current branch and origin the right
upstream, try to get both dynamically. If the current branch is not
tracking any upstream, use the origin of the master branch.

Signed-off-by: Jonas Gorski <jonas.gorski@gmail.com>
2016-07-13 16:55:32 +02:00
Felix Fietkau
c19381dfca scripts/getver.sh: fix revision number on BSD/MacOS
Strip leading whitespace from wc -l output

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-07-09 20:02:21 +02:00
Jonas Gorski
f1765277ba scripts/getver.sh: avoid use of git rev-list --count
This is not a valid option in older git version, used in e.g. RHEL6.

Reported-by: Steven Haigh <netwiz@crc.id.au>
Signed-off-by: Jonas Gorski <jonas.gorski@gmail.com>
2016-07-09 15:08:46 +02:00
Jonas Gorski
15b88df87f scripts/getver.sh: improve revision output
Change the revision output to r<upstream-revision>+<local commits> so
it is easier to get the base revision (and see if there are local
commits).

Example:
$ ./scripts/getver.sh
r794+3
$

Signed-off-by: Jonas Gorski <jonas.gorski@gmail.com>
2016-07-09 13:41:28 +02:00
Jonas Gorski
1001b5d77c scripts/getver.sh: allow conversion between git hash and revision
Add code allowing easy conversion between git commit ids and revisions.

Example:

$ ./scripts/getver.sh
r792
$ ./scripts/getver.sh r123
b7fc892eb5
$ ./scrpts/getver.sh b7fc892eb5
r123

Signed-off-by: Jonas Gorski <jonas.gorski@gmail.com>
2016-07-09 13:41:28 +02:00
Jonas Gorski
575be9d182 scripts/getver.sh: simplify revision calculation
Use git rev-list --count to get the revision number.

Signed-off-by: Jonas Gorski <jonas.gorski@gmail.com>
2016-07-09 13:41:28 +02:00
Felix Fietkau
d8dbd33eba scripts/package-metadata.pl: fix kmod pakage dependencies within the SDK
Instead of ignoring all metadata for package/kernel/linux, process it
and only suppress emitting config data to tmp/.config-package.in

This ensures that packages that select kmod-* packages can inherit their
depdendencies.

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-07-07 12:47:04 +02:00
Felix Fietkau
b80dfe4cde scripts/diffconfig.sh: fix handing of CONFIG_TARGET_MULTI_PROFILE
Fixes FS#43

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-07-07 11:23:41 +02:00