Doing a chmod on the whole source dir after every packages
are extracted can take a hell of a lot of time.
The offending packages are far from legion, and they now
have their own chmod u+w to cleanup their own mess...
Reported-by: ANDY KENNEDY <ANDY.KENNEDY@adtran.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
Consider the buildtools install directory as a prefix directory,
that is, install buildtools in prefix/bin/, not in prefix/.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
There is absolutely *no* reason for the buildtools (wrappers to gcc, g++,
as, ld... for the local machine) to be in the toolchain directory. Moreover,
they are removed after the build completes.
Move them out of the toolchain directory, and into the build directory (but
yet the part specific to the current toolchain). This means we no longer
need to explicitly remove them either, BTW, but we need to save/restore them
for the restart feature.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
Computed paths may contain double slashes.
This is not an issue but it is just ugly to look at.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
Download to an intermediate temp file, and rename it to its final
name only of download succeeds.
This catches both a failed download, and also the case where the user
interrupts the download. Thus, the a partial download gets discarded,
and we no longer try to extract a partial tarball, which we would
previously have done.
Suggested by Thomas PETAZZONI.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
It can happen, in some circumpstances, than one can succeed where
the other would fail. Those cases involves convoluted enterprise
networks with proxies playing tricks.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
The save/restore state output is voluminous; using this flag allows us
to quickly see or ignore when something is just being saved.
[Yann E. MORIN: this is a blind log level, and is used only to search
in the build-log afterward.]
Signed-off-by: Anthony Foiani <anthony.foiani@gmail.com>
I ran into some minor difficulties looking through the build log for a
particular file: I wasn't interested in seeing it unpacked, but only
when it is built or installed. Adding these two levels allows me to
differentiate between those cases.
[Yann E. MORIN: Those are blind log levels, and are used only to search
in the build-log afterward.]
Signed-off-by: Anthony Foiani <anthony.foiani@gmail.com>
To decide whether we need to backup the companion libraries,
do not rely on the !shared case. In the future other cases
may require not to save the companion libraries (eg. if using
the ones provided by the host distro).
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
It happens from time to time that the server mis-behaves, and breaks the
connection right in the middle of nowhere, for no good reason, leaving us
with a partial file, on which the extract pass would choke.
Remove partial downloads, to fail early.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
Even when // downloads are not enabled, aria2 can
fail on some servers (eg. uclibc.org).
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
Replace the over-engineered and buggy test in CT_SanitizePath
with a straight forward string pattern match, and also
handle empty PATH elements which are qeuivalent to ".".
Thanks-To: Arnaud Lacombe <lacombar@gmail.com>
Signed-off-by: Johannes Stezenbach <js@sig21.net>
Add CT_SanitizePath function which removes entries referring to ., /tmp
and non-existing directories from $PATH, and call it early in the
build script.
If . is in PATH, gcc-4.4.4 build breaks:
[ALL ] checking what assembler to use...
/tmp/build/targets/arm-unknown-linux-uclibcgnueabi/build/gcc-core-static/arm-unknown-linux-uclibcgnueabi/bin/as
...
[ALL ] config.status: creating as
i.e. "as" is supposed to be the arm-unknown-linux-uclibcgnueabi cross assembler,
but config.status creates a local "as" script which is calling the
host assembler.
Signed-off-by: Johannes Stezenbach <js@sig21.net>
[Yann E. MORIN: style fixes + explanations]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
Call to get the directory mode depending on $CT_SYS_OS
yann.morin.1998@anciens.enib.fr:
CT_SYS_OS has changed on Linuxsystem, it only gets the kernel name "Linux",
and not the system name, 'GNU/'.
Saving and restoring the steps requires saving/restoring multiple
directories. Depending on the configuration, some may not exist.
Add a wrapper that checks before creating/extracting the tarballs.
Not all target tuples consist of an VENDOR, KERNEL and SYSTEM part, build up the
tuple in such a way to no extra or trailing dashes are added to CT_TARGET
Signed-off-by: Bart vdr Meulen <bartvdrmeulen@gmail.com>
On some systems (eg. *BSD and Darwin), date does not support nanoseconds
(%N) precision. Instead of printing '%N' in this case, it just prints 'N'.
Fix the sed expression to handle this case.
Add a git wrapper to retrieve components from their git tree.
Add a git wrapper to create a working copy (in our tarballs dir).
Recognise git trees when searching for local copies.
By default curl doesn't folow redirects. This breaks sourceforge downloads.
Add the -L option to curl to fix this.
Curl also downloads the html as a file even when it gets a 404. This breaks
http downloads when using the failback system. Add the -f option to curl to fix
this.
Signed-off-by: Richard Strand <richard.strand@icomera.com>
In case the remote file does not exist (and probably for some
other reasons as well), aria2 nonetheless creates an empty file
(or not empty for some other reasons).
The solution is to delete the file whenever aria2 fails.
aria2 is a powerfull downloader that is capable of chunking and
parallel retrieval.
Due to li;itations in crosstool-NG retrieval facilities, it's not possible
to take fully advantage of aria2. It might happen that, in the future,
those limitations get lifted away, so we can take use features such as
parallel downloading from more than one server at the same time. For now,
it should still speed up downloads thanks to parallel downloading of chunks.
The newlib "team" rolls new releases about once a year (december).
This is quite a long time between releases, in case code was fixed.
So, allow user to use a CVS snapshot to benefit early from fixes
and enhancements to newlib.
Some projects' module (eg. newlib) are checked-out into a sudirectory
rather than into their own directory. Handle this case in the CT_GetCVS
function.
They have nothing to do in here, just let the user
configure his/her system appropriately.
-------- diffstat follows --------
/trunk/scripts/build/libc/eglibc.sh | 1 0 1 0 -
/trunk/scripts/functions | 100 0 100 0 -----------------------------
/trunk/config/global/download.in | 148 0 148 0 -------------------------------------------
3 files changed, 249 deletions(-)
- introduce the config dir, where components can store their config files
- move the munged uClibc config file to the config dir
- now, the state dir really is an indication that a build can be restarted
Thanks to Groleo Marius <groleo@gmail.com> for spotting the inconsistency
of the state dir usage, and suggesting this change.
/trunk/scripts/build/libc/uClibc.sh | 6 3 3 0 +++---
/trunk/scripts/crosstool-NG.sh.in | 9 7 2 0 +++++++--
/trunk/scripts/functions | 15 12 3 0 ++++++++++++---
3 files changed, 22 insertions(+), 8 deletions(-)
"chmod u+w" the full src tree: because of nochdir and cvs snapshots, we can't reliably know were we are...
/trunk/scripts/functions | 11 3 8 0 +++--------
1 file changed, 3 insertions(+), 8 deletions(-)
... I added a step after
"debug" called "finish", and moved the code in crosstool.sh
after the loop that processes the steps from crosstool.sh
into a do_finish function in functions. Thus, it is now
possible to restart after the "debug" step to re-do the
final few things (clean and compress).
/trunk/scripts/crosstool-NG.sh | 38 0 38 0 --------------------------------------
/trunk/scripts/functions | 42 42 0 0 ++++++++++++++++++++++++++++++++++++++++++
/trunk/steps.mk | 3 2 1 0 ++-
3 files changed, 44 insertions(+), 39 deletions(-)
- renaming the dircetory in CT_ExtratAndPatch is wrong:
- patches against the C library addons may be build against the short *or* long name... :-(
- symlink is more robust, even if less nice
- renaming the directory _after_ CT_ExtractAndPatch is too late:
- if patches are against the short name, and we renamed too the long name, patches don't apply
- so we'll never reach the point where we rename
/trunk/scripts/build/libc/glibc.sh | 1 0 1 0 -
/trunk/scripts/build/libc/eglibc.sh | 1 0 1 0 -
/trunk/scripts/functions | 2 1 1 0 +-
3 files changed, 1 insertion(+), 3 deletions(-)
CT_LIBC_FILE:
- that one was not easy, as it had sneaked into CT_ExtractAndPatch
- which in turn made CT_ExtractAndPatch have references to C library addons
- which in turn relieved the C library _extract functions from doing their own job
- which in turn imposed some nasty tricks in CT_ExtractAndPatch
- which in turn made life easier for the DUMA _get and _extract functions
- which unveiled some bizare behavior for pushd and popd:
- if using smthg ike: 'pushd foo |bar':
- the directory is *neither* changed
- *nor* is it pushed onto the stack
- which made popd fail
CT_MakeAbsolutePath:
- used only to make CT_LOCAL_TARBALLS_DIR canonical
- which is ((almost) useless:
- hopefully, the user entered a full path already
- if it's not the case, too bad...
/trunk/scripts/build/debug/200-duma.sh | 5 1 4 0 +--
/trunk/scripts/build/libc/glibc.sh | 61 32 29 0 +++++++++++++++++---------------
/trunk/scripts/build/libc/uClibc.sh | 16 10 6 0 +++++---
/trunk/scripts/build/libc/eglibc.sh | 48 26 22 0 ++++++++++++++-----------
/trunk/scripts/crosstool.sh | 8 0 8 0 ----
/trunk/scripts/functions | 77 15 62 0 ++++++++--------------------------------
6 files changed, 84 insertions(+), 131 deletions(-)
- retrieve from local storage (CT_GetLocal)
- save to local storage (CT_SaveLocal)
- retrieve from CVS (CT_GetCVS)
- make CT_GetFile and CT_GetCVS use CT_GetLocal and CT_SaveLocal
/trunk/scripts/functions | 126 91 35 0 +++++++++++++++++++++++++++++++++++++++++++++-----------------
1 file changed, 91 insertions(+), 35 deletions(-)
- vendor and alias must not contain spaces
- vendor must not contain dashes '-'
- sed_expr must not generate an alias with a space in it
/trunk/scripts/functions | 17 16 1 0 ++++++++++++++++-
/trunk/config/toolchain.in | 1 1 0 0 +
2 files changed, 17 insertions(+), 1 deletion(-)
We need GNU Awk? Then check for, and use 'gawk', not plain 'awk'.
Be a little mre verbose if a tool was not found.
/trunk/configure | 7 4 3 0 ++++---
/trunk/scripts/build/kernel/linux.sh | 2 1 1 0 +-
/trunk/scripts/functions | 16 8 8 0 ++++++++--------
/trunk/scripts/saveSample.sh | 4 2 2 0 ++--
4 files changed, 15 insertions(+), 14 deletions(-)
Be a little less verbose when extracting (and patching) files.
/trunk/scripts/functions | 7 2 5 0 ++-----
1 file changed, 2 insertions(+), 5 deletions(-)
Use a little bit more of CT_DoExecLog.
/trunk/scripts/functions | 35 18 17 0 ++++++++++++++++++-----------------
1 file changed, 18 insertions(+), 17 deletions(-)
Simplify CT_DoExecLog: it does not support affectations prior to the command, anyway.
/trunk/scripts/functions | 5 1 4 0 +----
1 file changed, 1 insertion(+), 4 deletions(-)
Print the time at which at step was finished (along with the time it took to complete).
/trunk/scripts/functions | 7 5 2 0 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
It is similar to CT_DoLog, but instead of printing its arguments, it uses them as a command, and logs the output of that command.
/trunk/scripts/functions | 8 8 0 0 ++++++++
1 file changed, 8 insertions(+)
It seems to be helping gcc somewhat into telling the correct endianness to ld that sticks with little endian even when the target is big (eg armeb-unknown-linux-uclibcgnueabi).
There's still work to do, especially finish the gcc part that is not in this commit.
/trunk/scripts/functions | 9 7 2 0 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)
network, most probably due to proxies. Have downloaders (wget and curl)
timeout on too slow connections (they don't by default).
scripts/functions | 17 12 5 0 ++++++++++++-----
1 file changed, 12 insertions(+), 5 deletions(-)
Rationale:
Most of the time, soft-float problems are caused by this sucker of gcc:
it has support for soft float for all of the targets I've tried so far,
but does not activate this code until you dwelve into half a dozen of
files to make it accept to build and link the support code...
So, yes: gcc has soft-float support. And again, yes: gcc is a sucker.
If you select to debug ct-ng, then you have two new options:
- DEBUG_CT_PAUSE_STEPS : pause between every steps,
- DEBUG_CT_SAVE_STEPS : save state between every steps.
To restart a saved state, just set the RESTART make variable when calling make:
- make RESTART=<step_name>
- pipe size in Linux is only 8*512=4096 bytes
- pipe size is not setable
- when the feeding process spits out data faster than the eating
process can read it, then the feeding process stalls after 4KiB
of data sent to the pipe
- for us, the progress bar would spawn a sub-shell every line,
and the sub-shell would in turn spawn a 'date' command.
Which was sloooww as hell, and would cause some kind of a
starvation: the pipe was full most of the time, and the
feeding process was stalled all this time.
Now, we use internal variables and a little hack based onan offset
to determine the elapsed time. Much faster this way, but still
CPU-intensive.
Add a uClibc-0.9.29 patch directory with one patch (from me!).
Update the armeb-unknown-linux-uclibc sample to uClibc-0.9.29.
Some eyecandy in the gdb build process.
- add a framework to easily add new ones
- add gdb as a first debug facility
- add patches for gdb
After the kernel checked its installed headers, clean up the mess of .checked.* files.
Reorder scripts/crosstool.sh:
- dump the configuration early
- renice early
- get info about build system early, when setting up the environment
- when in cross or native, the host tools are those of the build system, and only in this case
- elapsed time calculations moved to scripts/functions
Remove handling of the color: it's gone once and for all.
Update tools/addToolVersion.sh:
- handle debug facilities
- commonalise some code
- remove dead tools (cygwin, tcc)
Point to my address for bug reports.
- the tarball directory is considered as a local copy, and tarballs are copied to a working area,
- the sources and build directories (CT_SRC_DIR and CT_BUILD_DIR) are now computed, and no longer an option,
- the build dir has been renamed from 'build' to 'targets'.
That should ease preparing a tarball of the resulting target.
- use ports addon even when installing headers,
- use optimisation (-O) when installing headers, to avoid unnecessary warnings (thanks Robert P. J. DAY for pointing this out!),
- lowest kernel version to use is only X.Y.Z, not X.Y.Z.T,
- a bit of preparations for NPTL (RSN I hope),
- fix fixing the linker scripts (changing the backup file is kind of useless and stupid);
Shut uClibc finish step: there really is nothing to do;
Add a patch for glibc-2.3.6 weak aliases handling on some archs (ARM and ALPHA at least);
Did not catch the make errors: fixed the pattern matching in scripts/functions;
Introduce a new log level, ALL:
- send components' build messages there,
- DEBUG log level is destined only for crosstool-NG debug messages,
- migrate sub-actions to use appropriate log levels;
Update the armeb-unknown-linux-gnu sample:
- it builds!
- uses gcc-4.0.4 and glibc-2.3.6,
- updated to latest config options set.
- reorder most of the environment setup,
- geting, extracting and patching are now components' sub-actions,
- save the current config as a sample to be used as a pre-configured target.
You might just say: 'Yeah! crosstool-NG's got its own repo!".
Unfortunately, that's because the previous repo got damaged beyond repair and I had no backup.
That means I'm putting backups in place in the afternoon.
That also means we've lost history... :-(
