The referenced commit replaced 'make' with '${make}' everywhere. This is
wrong for at least the utilities that we may build as companion tools
(make, libtool): this will always invoke the version detected by configure
by supplying the absolute path. In other words, the wrappers in
.build/tools/bin are not fallbacks - they are either temporary (in case
a respective companion tool is built) or permanent redirectors.
This is the reason why the PATH= has .build/*/buildtools/bin at higher
precedence than .build/tools/bin; the latter has the versions detected by
configure and the former has the versions built as companion tools.
Revert the rest of the gang (grep/sed/...) for consistency. After all,
we may decide to supply some of them as well (awk, for instance).
Signed-off-by: Alexey Neyman <stilor@att.net>
Rather than echo-ing the new value, set the value into the variable with
the name passed as an argument (similar to CT_SanitizeVarDir). This
allows to use CT_DoLog in these functions.
Signed-off-by: Alexey Neyman <stilor@att.net>
This step was only used in uClibc. However, with upcoming multilib, the
config management will have to be done for each variant differently,
anyway.
uClibc was the only user of libc_check_config step, as well as
CT_CONFIG_DIR directory. Retire these.
Two other clean-ups in uClibc.sh:
- KERNEL_HEADERS check seems to be bogus, this config option is not
present even in 0.9.30 - which is not supported already.
- SHARED_LIB_LOADER_PREFIX was renamed to MULTILIB_DIR in 0.9.31,
according to ChangeLog - and MULTILIB_DIR is passed from command line
instead.
Signed-off-by: Alexey Neyman <stilor@att.net>
Install startfiles for libc variants into the most specific combination
(suffixed sysroot, if applicable + suffixed multi-os dir, if
applicable). Install headers once in every suffixed sysroot (although it
seems that GCC picks up headers from top-level sysroot, GCC manual
claims that sysroot suffix affects headers search path).
In uClibc, this requires a better sanitization of the directory: it
creates symlinks from {sysroot}/usr/lib/{multi_os_dir} to
{sysroot}/lib/{multi_os_dir} and to do so, it counts the number of path
components in the libdir. This breaks if one of such components is `..'
- symlinks contain an extra `../..' then. Since such sanitization had to
be implemented anyway, use it in other places to print more sensible
directory names.
Also, fix the description of configure --host/--target per musl's
configure help message (and its actual code).
Signed-off-by: Alexey Neyman <stilor@att.net>
On some arches (e.g. MIPS) the options like -mabi do not work if
specified more than once (see the comment in 100-gcc.sh). Therefore,
we need to determine which of the options produced by <arch>.sh can
be passed to multilib builds and which must be removed (i.e., which
options vary among the multilibs).
This presents a chicken-and-egg problem. GCC developers, in their
infinite wisdom, do not allow arbitrary multilib specification to be
supplied to GCC's configure. Instead, the target (and sometimes some
extra options) determine the set of multilibs - which may include
different CPUs, different ABIs, different endianness, different FPUs,
different floating-point ABIs, ... That is, we don't know which parts
vary until we build GCC and ask it.
So, the solution implemented here is:
- For multilib builds, start with empty CT_ARCH_TARGET_CFLAGS/LDFLAGS.
- For multilib builds, require core pass 1. Pass 1 does not build any
target binaries, so at that point, our target options have not been
used yet.
- Provide an API to modify the environment variables for the steps that
follow the current one.
- As a part of multilib-related housekeeping, determine the variable
part of multilibs and filter out these options; pass the rest into
CT_TARGET_CFLAGS/LDFLAGS.
This still does not handle extra dependencies between GCC options (like
-ma implying -mcpu=X -mtune=Y, etc.) but I feel that would complicate
matters too much. Let's leave this until there's a compelling case for
it.
Also, query GCC's sysroot suffix for targets that use it (SuperH,
for example) - the default multilib may not work if the command line
specifies the default option explicitly (%sysroot_suffix_spec is not
aware of multilib defaults).
Signed-off-by: Alexey Neyman <stilor@att.net>
The previous patch added the function 'CT_DoMultilibTarget()' to
scripts/build/arch/*.sh.
This patch calls the common function to (currently) get just the target
tuple for the current multilib target.
This patch was originally by: Cody P Schafer
Changed by Alexey Neyman: first, try `gcc -print-multiarch`. If it is
supported, use whatever it reports. Otherwise, fall back to our
guesswork. Move "i486" quirk into glibc.sh, as it is specific to glibc
(e.g. uclibc will need i386, which is what GCC reports).
Signed-off-by: Bryan Hundven <bryanhundven@gmail.com>
Signed-off-by: Ray Donnelly <mingw.android@gmail.com>
Signed-off-by: Alexey Neyman <stilor@att.net>
I should have just used ln -sf when I rewrote the custom locations
change. BSD based systems don't have 'cp -s', so switch to using 'ln
-sf'.
Signed-off-by: Bryan Hundven <bryanhundven@gmail.com>
Hardlinking the custom source directory does not work across separate
mount points. Chnage this to a softlink instead.
This closes#336
Reported-by: Jasmin Jessich <jasmin@anw.at>
Signed-off-by: Bryan Hundven <bryanhundven@gmail.com>
The previous version of CT_GetCustom was a bit... funky.
It didn't handle custom versions to location very well.
This new version is exactly as it appears:
CT_GetCustom <name> <version> <location>
The name is the beginning of the archive (file or directory).
The version is the second half of the archive.
The location is where it can be found. This should be made an absolute
path, but this version is expecting the path in kconfig to be absolute.
A file should extract to a directory: <name>-<version>
A directory will be copied to: <name>-<version>
This keeps our expectations of what we should get.
Signed-off-by: Bryan Hundven <bryanhundven@gmail.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
If we are using a custom location, and that custom location is a
directory that does not have an associated tarball, then we shouldn't
warn about not finding a tarball in CT_TARBALLS_DIR if
CT_SRC_DIR/.<basename>.extracted is found.
If the extracted file is not found, then we can warn that the tarball
was not found then error out that the tarball is missing.
Signed-off-by: Bryan Hundven <bryanhundven@gmail.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
We check for apps:
* make
* sed
* grep
* awk
* libtool/libtoolize
* install
* patch
* and more
...during configure. Our scripts should be consistent about using the
variables that define where the found tool was found.
Of course, we do hard-link these tools in buildtools, but that should be
a backup for the components we are building. Our scripts should always
use the tools we find.
Signed-off-by: Bryan Hundven <bryanhundven@gmail.com>
This commit adds 4 new functions to aid in the process of managing a
kconfig .config file:
* CT_KconfigSetOption <option> <value> <file>
* CT_KconfigEnableOption <option> <file>
* CT_KconfigDisableOption <option> <file>
* CT_KconfigDeleteOption <option> <file>
(akin to how buildroot manages the uClibc.config)
These functions are global so that we can manage any component that also
uses kconfig, or to be able to use it internally on Crosstool-NG's
kconfig files.
Last but not least, be consistent and update sed to be ${sed}!
Signed-off-by: Bryan Hundven <bryanhundven@gmail.com>
If the build is aborted before the working directory is created,
there's also an induced error message about inability to create
a 'backtrace' file. But in that case, there is no subshells executing
yet.
Signed-off-by: Alexey Neyman <stilor@att.net>
avr-libc doesn't have write permissions in these by default in the 1.8.1
tar release, this caused an error during build with
CT_OVERIDE_CONFIG_GUESS_SUB enabled.
chmod u+w them before overriding to avoid an this error.
Signed-off-by: Erico Nunes <nunes.erico@gmail.com>
Add support for applying arch-specific patches found in
"patches/${pkgname}/${version}/${CT_ARCH}".
This is needed for applying a popular binutils patch specific for the
AVR architecture but which isn't isolated for AVR in binutils' code.
In this case, applying it for every architecture would end up bloating
binutils' "size" options with AVR specifics.
This feels like a bit of a hack but it is easy enough to support with
current crosstool-ng infrastructure, seems like worth it for this case.
Signed-off-by: Erico Nunes <nunes.erico@gmail.com>
This commit adds support for the avr-libc C library.
According to the project page at http://www.nongnu.org/avr-libc , the
avr-libc package provides a subset of the standard C library for Atmel
AVR 8-bit RISC microcontrollers. In addition, the library provides the
basic startup code needed by most applications.
Support for this library in crosstool-ng is only enabled for the AVR
8-bit target.
The avr-libc manual and most distributions build the AVR 8-bit gcc
toolchain with the "avr" (non-canonical) target.
Some experimentation also led to the conclusion that other (canonical)
targets are not very well supported, so we force the "avr" target for
crosstool-ng as well.
The manual also recommends building avr-libc after the final gcc build.
To accomplish this with crosstool-ng, a new do_libc_post_cc step is
added, in which currently only avr-libc performs its build, and is a
no-op for the other libc options.
Signed-off-by: Erico Nunes <nunes.erico@gmail.com>
This functionality was provided so that crosstool-ng could have a
further set of patches considered experimental and unsupported.
Now that musl-libc support is making it's way upstream in gcc, I'm
removing this support and the experimental musl patches.
In later commits, backports from gcc upstream will be added to the
supported patch sets to support musl-libc.
Signed-off-by: Bryan Hundven <bryanhundven@gmail.com>
when specifying a custom kernel provided as a tar ball, the tar ball gets symlinked. the -e test will fail.
Signed-off-by: Dirk Husemann <dirk@d2h.net>
Pass cset as ref=somename to use this feature. CT_GetGit echos
the cset sha1 on exit since the caller will need to know that
information as it forms part of the downloaded tarball name.
Signed-off-by: Ray Donnelly <mingw.android@gmail.com>
As posted on http://www.eglibc.org/
====================
EGLIBC is no longer developed and such goals are now being addressed
directly in GLIBC.
====================
I'm not interested in maintaining build support for unsupported
software.
Older branches of crosstool-ng continue to have eglibc support.
If you find issues with older branches, I'm always open to pull
requests.
Removing eglibc also frees up glibc cleanup and build optimization.
Signed-off-by: Bryan Hundven <bryanhundven@gmail.com>
This patch adds initial support for musl-libc.
Musl-libc versions currently supported:
* 1.0.3 (Stable)
* 1.1.3 (Previous Mainline)
* 1.1.4 (Mainline)
Futher improvements are needed.
* gcc-4.9.x has issues (Might be fixed in musl-1.1.4).
* Multilib support is needed.
* Checks to make sure paths are correct.
* Move to 2-step gcc build. 3-step build is not necessary.
Signed-off-by: Bryan Hundven <bryanhundven@gmail.com>
[yann.morin.1998@free.fr: removed the gcc musl patch, to be added later;
removed dead code do_get_arch()]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
This change adds support for experimental patches to be introduced to
crosstool-ng. The patches enabled by this option are to be located here:
patches/experimental/<package>/<version>/XXXX-NAME.patch
Where, XXXX is the patch number to be applied in order, like:
0001-some_patch_one.patch
0002-some_patch_two.patch
9999-some_patch_to_be_applied_last.patch
In the first patch series, all patches in the EXPERIMENTAL_PATCHES
option will be applied all at once, or none at all.
In a later [RFC] patch, I plan on adding finer tuned patch
enable/disable options based on the name of the patch and where it is
located in the patches/experimental sub-tree. So the name of the patch
should use underscores between words in the patch name.
Signed-off-by: Bryan Hundven <bryanhundven@gmail.com>
[yann.morin.1998@free.fr: slightly reword prompt]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
These variables behave the same for bitness as their counterparts do
for endianness: they are defined to the appropriate bitness.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Cody P Schafer <dev@codyps.com>
We currently define target_endian_el and target_endian_eb to be the
tuple extension depending on endianness, defined to be respectively
'el' or 'eb' according to the endianness.
Some architecture do not use 'el' or 'eb', but use 'le' or 'be'.
Provide that as well, as two new variables: target_endian_le and
target_endian_be.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Cody P Schafer <dev@codyps.com>
Allow '-1' to be specified as CONNECTION_TIMEOUT to disable the use
of the connection timeout for wget.
Signed-off-by: Cody P Schafer <dev@codyps.com>
Message-Id: <cb33f8c2cbaf802d4f04.1399687632@localhost>
Patchwork-Id: 347582
'zcat' on MacOS-X is broken (it is not gzip's zcat, but compres' zcat).
Use 'gzip -dc' for portability, as suggested by Anthony.
Reported-by: Fernando Ortiz <fortiz2k@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Anthony Foiani <anthony.foiani@gmail.com>
This patch fixes the download of the avr32 headers in crosstool-ng by
fetching them directly from Atmel's web site instead of the now-broken URL
given by the original author of the avr32-header-fetching modification,
who fetched them from a copy on his own, now-defunct server.
It also adds the necessary logic to extract from a zip file, as that is
how the headers are packaged.
To configure it for avr32 after launching ct-ng menuconfig in an empty
directory:
Paths and misc options ->
Shell to use as CONFIG_SHELL = sh
Target options ->
Target Architecture = avr32
Toolchain options ->
Tuple's alias = avr32
Binary utilities ->
binutils version = 2.18a
C compiler
gcc version = 4.2.2
C-library
newlib version = 1.17.0
Enable IOs on long long = yes
Enable IOs on floats and doubles = yes
Disable the syscalls supplied with newlib = yes
CONFIG_SHELL is necessary to get round the "fragment: command not
found" bug when binutils-2.18 is configured using bash.
Prepared against crosstool-ng mercurial trunk on 31 March 2012.
Signed-off-by: Martin Guy <martinwguy@gmail.com>
[yann.morin.1998@free.fr: update bundles sample accordingly]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Message-Id: <CAL4-wQrg_NQ7jm-NCADqeyQr9twyhtx42OUGNThP6gWeqZc=kw@mail.gmail.com>
Patchwork-Id: 232612
For some architectures, it is legit to have an alternate value in the
'architecture' part of the tuple. For example:
armv5te-*
armv7a8-*
Besides, some packages expect the tuple to reflect the arch variant
(eg. openMPI) to detect the variant's capabilities (eg. atomic
primitives).
This patch adds an option for the user to specify a suffix to be added
to the arch-part of the tuple.
Signed-off-by: Willy Tarreau <w@1wt.eu>
Message-ID: <20130120225822.GS6838@1wt.eu>
Patch-Id: 213994
[yann.morin.1998@free.fr: make it a suffix, not an override]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Properly catch resuming the build when continuing past the
failed command.
The 'case ;;&' construct is a bash4ism. Get rid of it.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
CT_DEBUG_INTERACTIVE is disabled when stdin, stdout or
stderr are redirected, but the check is only done at
the start of the build and doesn't catch when individual
build commands use redirection. When stdin is redirected
it will cause the debug shell to exit immediately, causing
and endless loop. Thus, save the stdin/our/err file handles
and restore them before invoking the debug shell.
Signed-off-by: Johannes Stezenbach <js@sig21.net>
Message-Id: <20121030102225.GA8303@sig21.net>
Patchwork-Id: 195409
This makes the patch name show up on the command line
logged by CT_DoExecLog so it's easier to see
what is going on. The -i for patch is specified
by Posix and supported by GNU patch and busybox patch.
Signed-off-by: Johannes Stezenbach <js@sig21.net>
[yann.morin.1998@free.fr: remove now-useless debug message]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Message-Id: <20121030103620.GB8303@sig21.net>
Patchwork-Id: 195418
Since we added the debug-shell feature, CT_DoExecLog no longer
returns the error code of the command, but always return 0.
This breaks the download mechanism, which relies on CT_DoExecLog
to fail _on_purpose_ to detect that the ressource was not found
at the specified URL.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
So we get caught by the trap-handler and
have a chance to run the debug-shell.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Johannes Stezenbach <js@sig21.net>
Add an option that, when a command fails:
- starts an interactive shell with the failed command's environment
- attempts re-execution of the failed command, continues, or aborts
at user's whim.
Before starting the debug-shell, the backtrace is printed.
When exiting for an abort, the standard error message is printed.
Based on an idea and a patch from: Johannes Stezenbach <js@sig21.net>
http://sourceware.org/ml/crossgcc/2012-09/msg00144.html
Signed-off-by: Johannes Stezenbach <js@sig21.net>
[yann.morin.1998@free.fr: integrate in the fault handler]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Acked-by: Johannes Stezenbach <js@sig21.net>
Patchwork-Id: 191571
Patchwork-Id: 191668
Avoid error when commands in scripts/crosstool-NG.sh fail
before CT_BUILD_DIR is set.
So we need to remove the backtrace marker of a potential previous
build. Previously, it was implicitly removed because we did remove
the directory it was in, which is no longer the case.
Signed-off-by: Johannes Stezenbach <js@sig21.net>
[yann.morin.1998@free.fr: remove backtrace marker on start of build]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Message-Id: <20121015094615.GA18673@sig21.net>
Patchwork-Id: 191498
Add a generic custom location infrastructure (inspired by the one in
kernel/linux) to allow the user to use custom tarballs or directories
for any component.
Signed-off-by: "David Holsgrove" <david.holsgrove@xilinx.com>
[yann.morin.1998@free.fr: move config option, improve help text, fix API doc]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Message-Id: <131c163c69f9cc81d2be.1349931191@localhost.localdomain>
PatchWork-Id: 190784
Message-Id: <0bbaba9190a76ba97f72.1349931192@localhost.localdomain>
PatchWork-Id: 190785
Currently, if downloads are forbidden, the mirror is still tried for.
Change this way:
- if downlaods forbidden, do not try neither upstream locations nor mirror
- add option to only use the mirror, and avoid upstream locations
Signed-off-by: Austin Morton <austinpmorton@gmail.com>
[yann.morin.1998@free.fr: broaden the if USE_MIRRORto enclode mirror location]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
It's easier to have as much as possible stuff in the same place to
ease backup/restore, and make things easier to follow.
Move the host companion libraries install dir as a sub-dir of the
build-tools install dir (but not directly in it, it would break
for canadian or cross-native).
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
There really is no good reason to install the core compilers in their
own places, one for each pass. We can install them with the other
build tools.
Also, this implies that:
- there are fewer directories to save/restore
- there are fewer symlinks to create for binutils
- the PATH is shorter
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
A lot of places are currently doing:
mkdir -p foo/bar
cd foo/bar
Or even:
mkdir -p foo/bar
pushd foo/bar
[...]
popd
Provide both wrapper to ease doing this.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
In the process of converting to autoconf, the kconfig option
were not properly translated.
Fix that.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
Dumping the backtrace has been broken since changeset #652e56d6d35a:
scripts: execute each steps in a subshell
We can spawn sub-sub-shells in some cases.
The way the fault handler works is to dump the backtrace, but to avoid
printing it once for every sub-shell (which could get quite confusing),
it simply exits when it detects that it is being run in a sub-shell,
leaving to the top-level shell the work to dump the backtrace.
Because each step is executed in its own sub-shell, the variable arrays
that contain the step name, the source file and line number, are lost
when exiting the per-step sub-shell.
Hence, the backtrace is currently limited to printing only the top-level
main procedure of the shell.
Fix this thus:
- when dumping the bckatraces for the steps & the functions, remember
it was dumped, and only dump it if it was not already dumped
- at the top-level shell, print the hints
Also, rename the top-level step label.
Reported-by: Benoît Thébaudeau <benoit.thebaudeau@advansee.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
Some architectures support a mixed hard/soft floating point, where
the compiler emits hardware floating point instructions, but passes
the operands in core (aka integer) registers.
For example, ARM supports this mode (to come in the next changeset).
Add support for softfp cross compilers to the GCC and GLIBC
configuration. Needed for Ubuntu and other distros that are softfp.
Signed-off-by: Michael Hope <michael.hope@linaro.org>
[yann.morin.1998@anciens.enib.fr: split the original patch]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
When hardfloat is selected, we need to pass that selection down to
./configure and in the CFLAGS.
Signed-off-by: Michael Hope <michael.hope@linaro.org>
[yann.morin.1998@anciens.enib.fr: split the original patch]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
With the upcoming softfp support, the case..esac test would become
a bit convoluted if it were to test three different booleans.
Introduce a new blind string config option that defaults to the
selected floating point type used.
Signed-off-by: Michael Hope <michael.hope@linaro.org>
[yann.morin.1998@anciens.enib.fr: split the original patch]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
Change CT_ExtractGit so that it clones the repository, instead of just
symlinking it. After cloning, any given ref is checked out, or if no
ref is given, the HEAD of the repository is checked out.
This makes CT_Extract behave similar for git repositories as it does
for tarballs, so that it for example can be used for passing glibc-ports
as a git repository.
Signed-off-by: "Esben Haabendal" <esben.haabendal@prevas.dk>
[yann.morin.1998@anciens.enib.fr: fix incomplete var rename]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
Virtually all FTP server available on-line support passive FTP.
At least, this is the case for the servers crosstool-NG needs to
connect to.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
Currently, we use either wget or curl, whichever is installed.
In case both are installed, both are used. This means that it
takes a while trying all extensions.
Remove use of wget, and use only curl.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
Some packages are available as LZMA tarballs. LZMA is a relatively recent
compression algorithm; it's slightly better than bzip2, but offers much
faster decompression. LZMA is now deprecated in favor of XZ, but some
packages switched to LZMA when XZ was not yet available, or still in its
infancy. Latest XZ (which totaly obsoletes LZMA) offers a backward LZMA-
compatible utility, so we can check for 'lzma' nonetheless.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
./configure does check for the presence of gz and bzip2, so we can
safely use them in the build scripts.
On the other hand, more recent formats (eg. XZ) are not yet widely
available, and we do not want, and can't, force the user to install
them as a pre-requisite.
So, build up a list of allowed tarball formats based on the available
decompressors. For no, this is a static list, but the upcoming XZ
support will conditionnaly add to this list.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
When downloading via svn/cvs/... an attempt to retrieve from the
mirror is made. If the mirror does not have the required tarball,
an error message is printed. This is misleading, as the download
may later succeed via svn/cvs/...
Remove the messages about failed downloads altogether.
At the same time, use "if ... then ... fi" instead of "... && ..."
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
This is needed later, when we'll conditionnally use both the
upstream and the mirror URLs.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
Currently, the mirror can be used either:
- as a fallback in case upstream is unavailable (default behavior)
- as the preferred source for downloads
But the most common use-case seems to provide a truely-LAN mirror
to speed up downloads in big corpos', and/or provide a 'trusted'
source for the tarballs.
So, make the following changes;
- if a mirror is specified, always try that before trying upstream
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
The cvs download helper looks for the local tarballs dir to see if it
can find a pre-downloaded tarball, and if it does not find it, does
the actual fetch to upstream via cvs.
In the process, it does not even try to get a tarball from the local
mirror, which can be useful if the mirror has been pre-populated
manually (or with a previously downloaded tree).
Fake a tarball get with the standard tarball-download helper, but
without specifying any upstream URL, which makes the helper directly
try the LAN mirror.
Of course, if no mirror is specified, no URL wil be available, and
the standard cvs retrieval will kick in.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
The svn download helper looks for the local tarballs dir to see if it
can find a pre-downloaded tarball, and if it does not find it, does
the actual fetch to upstream via svn.
In the process, it does not even try to get a tarball from the local
mirror, which can be useful if the mirror has been pre-populated
manually (or with a previously downloaded tree).
Fake a tarball get with the standard tarball-download helper, but
without specifying any upstream URL, which makes the helper directly
try the LAN mirror.
Of course, if no mirror is specified, no URL wil be available, and
the standard svn retrieval will kick in.
Reported-by: ANDY KENNEDY <ANDY.KENNEDY@adtran.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
When retrieving tarballs from upstream, if no URL was given, do not
fail; simmply ignore that fact.
This will be used later when the SVN helper will call the standard
helper to try the LAN mirror before trying svn.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
Some archives like those of the 2011.07 revisions of Linaro GCC contain a folder
name different from the archive basename, which leads to errors afterwards, e.g.
when patching. E.g.:
gcc-linaro-4.5-2011.07.tar.bz2 extracts to gcc-linaro-4.5-2011.07-0/
This patch changes CT_Extract() to force the extraction of all archives to a
folder named like the archive basename. E.g.:
gcc-linaro-4.5-2011.07.tar.bz2 now extracts to gcc-linaro-4.5-2011.07/
Signed-off-by: "Benoît THÉBAUDEAU" <benoit.thebaudeau@advansee.com>
In case of eglibc, some add-ons that were previously external are
now internal (bundled with the main sources).
So we do not want to fail if an add-on can't be downloaded; we
want to post-pone the check until we can extract the main archive.
So:
- try to retrieve the add-on
- if it fails, print a warning instead of calling CT_Abort
- return 1
So, components that want to catch the error and want to handle it can,
while components that do not will gracefuly fail thanks to our catching
every errors.
Bonus: it works without changing any existing retrieval procedure! :-)
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
For glibc/eglibc, if the add-on can not be extracted, we want to catch
the error to print a meaningful error message.
So:
- try to extract the tarball
- if it fails, print a waring instead of calling CT_Abort
- return 1
So, components that want to catch the error and want to handle it can,
while components that do not will gracefuly fail thanks to our catching
every errors.
Bonus: it works without changing any existing extract procedure! :-)
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
In case of glibc/eglibc, some add-ons that were previously external are
now internal (bundled with the main sources).
So we do not want to fail if an add-on tarball can't be downloaded; we
want to post-pone the check until we can extract the main archive.
So:
- try to download the tarball
- if it fails, print a warning instead of calling CT_Abort
- return 1
So, components that want to catch the error and want to handle it can,
while components that do not will gracefuly fail thanks to our catching
every errors.
Bonus: it works without changing any existing retrieval procedure! :-)
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
Reformat the error messages:
- strip ${CT_LIB_DIR} from scripts path names
- strip ${CT_TOP_DIR} from build.log path and docs path
- overall shorter lines
- point to the known-issues file
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
In case date(1) does not support nanosecond resolution, it does
not interpret '%N', and leave it as-is. So we have to remove it.
Note that some versions replaces '%N' with 'N', so we have to
take this into account as well.
Reported-by: Kyle Grieb <grieb.kyle@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
Using CT-NG with progress bar disabled, still prints CR ('\r') characters
on the output. When you capture the output to a file as part of an
automated build, it shows extra empty lines.
For example:
------------------------------------------------------------
[INFO ] Performing some trivial sanity checks
[INFO ] Build started 20110404.113619
[INFO ] Building environment variables
[EXTRA] Preparing working directories
[EXTRA] Installing user-supplied crosstool-NG configuration
------------------------------------------------------------
Signed-off-by: Javier Viguera <javier.viguera@digi.com>
Managing the shared version of the companion libraries
has become cumbersome.
Also, it will one day be possible to use the companion
libraries from the host distribution, and then we will
be able to easily use either shared or static libs.
As a side note, while working on the canadian-rework
series, it has become quite more complex to properly
handle shared companion libraries, as they need to be
built both for the build and gost systems. That's not
easy to handle. At all.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
In a lot of places, we need to call some commands with specific
variable settings, a-la:
var1=val1 var2=val2 /foo/bar/buz opt1 opt2
Unfortunately, we currently can not log the variable settings.
Enhance CT_DoExecLog with a crude heuristic that works pretty well
and that can also log setting variables.
Reported-by: ANDY KENNEDY <ANDY.KENNEDY@adtran.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
Users tend to look for the build log in the current working directory,
rather than in the toolchain's installation dir. While bundling the build
log in the toolchain installation dir is nice for distribution and review,
it can be easier to have the build log readily available in the working
directory, as it is quicker to get to it.
So, the build log stays in the working directory until the toolchain is
completely and successfully built, and then a (compressed) copy is made.
Reported-by: Trevor Woerner <twoerner@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
Doing a chmod on the whole source dir after every packages
are extracted can take a hell of a lot of time.
The offending packages are far from legion, and they now
have their own chmod u+w to cleanup their own mess...
Reported-by: ANDY KENNEDY <ANDY.KENNEDY@adtran.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
Consider the buildtools install directory as a prefix directory,
that is, install buildtools in prefix/bin/, not in prefix/.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
There is absolutely *no* reason for the buildtools (wrappers to gcc, g++,
as, ld... for the local machine) to be in the toolchain directory. Moreover,
they are removed after the build completes.
Move them out of the toolchain directory, and into the build directory (but
yet the part specific to the current toolchain). This means we no longer
need to explicitly remove them either, BTW, but we need to save/restore them
for the restart feature.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
Computed paths may contain double slashes.
This is not an issue but it is just ugly to look at.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
Download to an intermediate temp file, and rename it to its final
name only of download succeeds.
This catches both a failed download, and also the case where the user
interrupts the download. Thus, the a partial download gets discarded,
and we no longer try to extract a partial tarball, which we would
previously have done.
Suggested by Thomas PETAZZONI.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
It can happen, in some circumpstances, than one can succeed where
the other would fail. Those cases involves convoluted enterprise
networks with proxies playing tricks.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
The save/restore state output is voluminous; using this flag allows us
to quickly see or ignore when something is just being saved.
[Yann E. MORIN: this is a blind log level, and is used only to search
in the build-log afterward.]
Signed-off-by: Anthony Foiani <anthony.foiani@gmail.com>
I ran into some minor difficulties looking through the build log for a
particular file: I wasn't interested in seeing it unpacked, but only
when it is built or installed. Adding these two levels allows me to
differentiate between those cases.
[Yann E. MORIN: Those are blind log levels, and are used only to search
in the build-log afterward.]
Signed-off-by: Anthony Foiani <anthony.foiani@gmail.com>
To decide whether we need to backup the companion libraries,
do not rely on the !shared case. In the future other cases
may require not to save the companion libraries (eg. if using
the ones provided by the host distro).
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
It happens from time to time that the server mis-behaves, and breaks the
connection right in the middle of nowhere, for no good reason, leaving us
with a partial file, on which the extract pass would choke.
Remove partial downloads, to fail early.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
Even when // downloads are not enabled, aria2 can
fail on some servers (eg. uclibc.org).
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
Replace the over-engineered and buggy test in CT_SanitizePath
with a straight forward string pattern match, and also
handle empty PATH elements which are qeuivalent to ".".
Thanks-To: Arnaud Lacombe <lacombar@gmail.com>
Signed-off-by: Johannes Stezenbach <js@sig21.net>
Add CT_SanitizePath function which removes entries referring to ., /tmp
and non-existing directories from $PATH, and call it early in the
build script.
If . is in PATH, gcc-4.4.4 build breaks:
[ALL ] checking what assembler to use...
/tmp/build/targets/arm-unknown-linux-uclibcgnueabi/build/gcc-core-static/arm-unknown-linux-uclibcgnueabi/bin/as
...
[ALL ] config.status: creating as
i.e. "as" is supposed to be the arm-unknown-linux-uclibcgnueabi cross assembler,
but config.status creates a local "as" script which is calling the
host assembler.
Signed-off-by: Johannes Stezenbach <js@sig21.net>
[Yann E. MORIN: style fixes + explanations]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
Call to get the directory mode depending on $CT_SYS_OS
yann.morin.1998@anciens.enib.fr:
CT_SYS_OS has changed on Linuxsystem, it only gets the kernel name "Linux",
and not the system name, 'GNU/'.
Saving and restoring the steps requires saving/restoring multiple
directories. Depending on the configuration, some may not exist.
Add a wrapper that checks before creating/extracting the tarballs.
Not all target tuples consist of an VENDOR, KERNEL and SYSTEM part, build up the
tuple in such a way to no extra or trailing dashes are added to CT_TARGET
Signed-off-by: Bart vdr Meulen <bartvdrmeulen@gmail.com>
On some systems (eg. *BSD and Darwin), date does not support nanoseconds
(%N) precision. Instead of printing '%N' in this case, it just prints 'N'.
Fix the sed expression to handle this case.
Add a git wrapper to retrieve components from their git tree.
Add a git wrapper to create a working copy (in our tarballs dir).
Recognise git trees when searching for local copies.
By default curl doesn't folow redirects. This breaks sourceforge downloads.
Add the -L option to curl to fix this.
Curl also downloads the html as a file even when it gets a 404. This breaks
http downloads when using the failback system. Add the -f option to curl to fix
this.
Signed-off-by: Richard Strand <richard.strand@icomera.com>
In case the remote file does not exist (and probably for some
other reasons as well), aria2 nonetheless creates an empty file
(or not empty for some other reasons).
The solution is to delete the file whenever aria2 fails.
aria2 is a powerfull downloader that is capable of chunking and
parallel retrieval.
Due to li;itations in crosstool-NG retrieval facilities, it's not possible
to take fully advantage of aria2. It might happen that, in the future,
those limitations get lifted away, so we can take use features such as
parallel downloading from more than one server at the same time. For now,
it should still speed up downloads thanks to parallel downloading of chunks.
The newlib "team" rolls new releases about once a year (december).
This is quite a long time between releases, in case code was fixed.
So, allow user to use a CVS snapshot to benefit early from fixes
and enhancements to newlib.
Some projects' module (eg. newlib) are checked-out into a sudirectory
rather than into their own directory. Handle this case in the CT_GetCVS
function.
They have nothing to do in here, just let the user
configure his/her system appropriately.
-------- diffstat follows --------
/trunk/scripts/build/libc/eglibc.sh | 1 0 1 0 -
/trunk/scripts/functions | 100 0 100 0 -----------------------------
/trunk/config/global/download.in | 148 0 148 0 -------------------------------------------
3 files changed, 249 deletions(-)
- introduce the config dir, where components can store their config files
- move the munged uClibc config file to the config dir
- now, the state dir really is an indication that a build can be restarted
Thanks to Groleo Marius <groleo@gmail.com> for spotting the inconsistency
of the state dir usage, and suggesting this change.
/trunk/scripts/build/libc/uClibc.sh | 6 3 3 0 +++---
/trunk/scripts/crosstool-NG.sh.in | 9 7 2 0 +++++++--
/trunk/scripts/functions | 15 12 3 0 ++++++++++++---
3 files changed, 22 insertions(+), 8 deletions(-)
