Commit Graph

55 Commits

Author SHA1 Message Date
Baptiste Jonglez
b30ba14e2a scripts/download.pl: fail loudly if provided hash is unsupported
Currently, if the provided hash is unsupported (length different from 32
or 64 bytes), we happily download the requested file without any kind of
checksum verification.

This is quite dangerous and may provide a false sense of security, because
a single typo in the hash (e.g. one character deleted by mistake) may skip
checksum verification entirely.

Instead, fail immediately if we don't support the provided hash.
In particular, if an external package repository decides to change the
hash algorithm one day, we will now fail loudly instead of skipping
checksum verification without complaints.

Note: if some users of scripts/download.pl knowingly provide an empty hash
because they don't need checksum verification, this change will break
them.  This does not seem to be the case currently, but if this feature is
ever needed, an option should be added to download.pl instead of relying
on the hash being empty.

Fixes: eaa4eba10a ("scripts/download.pl: add SHA-256 support")

Signed-off-by: Baptiste Jonglez <git@bitsofnetworks.org>
2017-09-11 17:13:02 +02:00
Zoltan Gyarmati
0a761fe8ef scripts/dowload.pl: use glob to expand target dir
If CONFIG_DOWNLOAD_FOLDER is set to for example "~/dl", the download
script fails to create the .hash and .dl files with the following
errors:
 Cannot create file ~/dl/dropbear-2017.75.tar.bz2.dl: No such file or directory
 sh: 1: cannot create ~/dl/dropbear-2017.75.tar.bz2.hash: Directory nonexistent

If the tarball already exists in the ~/dl dir, it's properly found and
used, so this issue only affects the download.pl script.
 This patch calls glob() on the target dir parameter, which will expand `~`.

Signed-off-by: Zoltan Gyarmati <mr.zoltan.gyarmati@gmail.com>
2017-08-30 18:12:36 +02:00
Daniel Engberg
99c4295126 scripts/download.pl: Adjust URLs
Internet2 isn't considered a trusted issuer meaning that https links to
rit.edu will fail.

The host mirror.csclub.uwaterloo.ca has a trusted SSL cert and peering
is good so it can replace rit.edu without performance issues.

Signed-off-by: Daniel Engberg <daniel.engberg.lists@pyret.net>
[Jo-Philipp Wich: rewrapped commit message]
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2017-07-21 08:09:45 +02:00
Felix Fietkau
cbe0a7ecc0 scripts/download.pl: print the command used to download files
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-05-25 19:01:07 +02:00
Felix Fietkau
2fd1503378 scripts/download.pl: fix error message on hash mismatch
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-04-04 12:33:37 +02:00
Felix Fietkau
74c09e714e download.pl: fix detecting download errors with curl
Avoid treating error pages as successfully downloaded content

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-31 11:24:19 +01:00
Felix Fietkau
84bd74057f build: use mkhash to replace various quirky md5sum/openssl calls
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-05 11:09:12 +01:00
Brian J. Murrell
fc6b6f4583 download.pl: use curl in preference to wget
Because wget doesn't know how to do Negotiate authentication with a proxy
and curl does, use curl if it's present. The user is expected to have a
~/.curlrc that sets the options necessary for any proxy authentication.

A ~/.curlrc is completely optional however and curl will work in exactly
the same manner as wget without one.

Signed-off-by: Brian J. Murrell <brian@interlinx.bc.ca>
[Jo-Philipp Wich: Rework code to detect curl usability by checking --version,
                  Use vararg style open() to bypass the shell when downloading,
                  Use Text::ParseWords to decompose env vars into arguments]
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2017-01-03 14:26:41 +01:00
Felix Fietkau
651bc94df4 download.pl: check for existing file before the first download attempt
In the build system, flock will prevent multiple concurrent downloads
for the same file. However, if one download request for the same file is
waiting for another one to finish, it will result in downloading the
same file twice consecutively.

Prevent this issue by exiting immediately if the file has already been
downloaded

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-12-16 15:16:47 +01:00
Daniel Engberg
8372a7f922 download.pl: Rework URLs
Provide HTTPS URL when possible, try to keep 8 mirrors per entry and spread
over several locations of the world. Since most active contributors are in
US/CA and/or EU prioritize mirrors that are within those regions if possible.

Signed-off-by: Daniel Engberg <daniel.engberg.lists@pyret.net>
2016-05-28 09:14:55 +02:00
Petr Štetiar
e0b241bb48 scripts/download.pl: Use CDN for kernel downloads
More info at https://www.kernel.org/introducing-fastly-cdn.html

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2016-05-23 13:39:50 +02:00
Felix Fietkau
111285e742 download.pl: use http://sources.lede-project.org as download mirror
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-05-13 16:32:35 +02:00
Álvaro Fernández Rojas
694f060550 download: add @GITHUB download facility
Define a new alias (@GITHUB) for downloading raw files from github repos.

Signed-off-by: Álvaro Fernández Rojas <noltari@gmail.com>
2016-04-03 22:02:46 +02:00
John Crispin
5c408ede6c download: add @GITHUB download facility
Define a new alias (@GITHUB) for downloading raw github repository files

Signed-off-by: Álvaro Fernández Rojas <noltari@gmail.com>

SVN-Revision: 49138
2016-04-09 10:25:34 +00:00
Felix Fietkau
9a94538272 scripts/download.pl: fix sha256 hash command (#21931)
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 48809
2016-02-27 16:20:06 +00:00
Felix Fietkau
883b5b8191 build: add @APACHE download facility
The Apache Software Foundation offers diverse download mirros.

For packaging Apache software a new alias @APACHE is defined.

Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>

SVN-Revision: 48270
2016-01-17 10:47:32 +00:00
Felix Fietkau
526f470c5f download.pl: fix openssl sha256 digest output
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 48259
2016-01-16 10:24:15 +00:00
Felix Fietkau
eaa4eba10a scripts/download.pl: add SHA-256 support
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 48253
2016-01-16 00:19:41 +00:00
Felix Fietkau
4eaeee7151 scripts/download.pl: cleanup GNOME source location definitions
Cleanup the @GNOME source download location definitions:
* remove dead and stale mirrors
* adjust to changes at directory structure
* add one new working mirror

Signed-off-by: Hannu Nyman <hannu.nyman@iki.fi>

SVN-Revision: 47825
2015-12-10 12:40:08 +00:00
Felix Fietkau
9b68fe961b build: add support for choosing a different url filename part than the output file
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 47591
2015-11-22 19:06:33 +00:00
Hauke Mehrtens
accc5e3693 script: downlaod: change mirror for kernel.org
kernel.org now suggests a different mirror address. this one also
support IPv6 connections and was faster for me.

Signed-off-by: Hauke Mehrtens <hauke@hauke-m.de>

SVN-Revision: 46875
2015-09-11 17:59:12 +00:00
Felix Fietkau
3d640eccc1 scripts/download.pl: add support for adding mirrors through an environment variable
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 43654
2014-12-12 12:35:23 +00:00
John Crispin
b0a1c7dc0d include/download.mk: Add download mirrors for tools from GNU Savannah (bug #15184)
I defined a new download method @SAVANNAH in include/download.mk and scripts/download.pl,
and converted quilt and qemu to use that method.

Signed-off-by: Hannu Nyman <hannu.nyman@iki.fi>

SVN-Revision: 42840
2014-10-08 08:01:39 +00:00
Felix Fietkau
27fe688c5f scripts/download.pl: prefer the GNU mirror redirect over the primary site (#14603)
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 39045
2013-12-13 16:43:07 +00:00
Mirko Vogt
53d4d18f07 remove broken mirrors for linux kernel
ftp.de.kernel.org / ftp.fr.kernel.org don't resolv anymore,
ftp.all.kernel.org is DNS round robin anyway

Signed-off-by: Mirko Vogt <mirko@openwrt.org>

SVN-Revision: 37323
2013-07-14 21:26:35 +00:00
Felix Fietkau
40b3db713d scripts/download.pl: use HTTP instead of FTP for the default GNU site
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 37213
2013-07-09 12:52:12 +00:00
Luka Perkov
819472b2d0 scripts: update GNU mirrors
SVN-Revision: 35838
2013-02-28 23:49:36 +00:00
Jo-Philipp Wich
12728ea66b download.pl: - cleanup cache handling code - use alternative m// and s/// delimiters to get rid of tedious slash escaping - use multi-argument form of system() where possible to avoid quoting & escaping issues - avoid a temporary variable
SVN-Revision: 31677
2012-05-11 18:17:15 +00:00
John Crispin
0c4bcae7a1 allow arbitary folder layout when using localmirrors
Make the download script follow symlinks and search subfolders when looking for
a file in a local download mirror.

Signed-off-by: Tathagata Das <tathagata@alumnux.com>

SVN-Revision: 31240
2012-04-10 14:11:45 +00:00
Felix Fietkau
548155ab7d Revert "scripts/download.pl: fix sourceforge url"
To use the new url, the project name would need to be appended multiple times,
let's hope the old redirect will continue to work in the future

SVN-Revision: 30730
2012-02-26 09:29:53 +00:00
Felix Fietkau
4dc4ebe0c8 scripts/download.pl: fix sourceforge url
SVN-Revision: 30726
2012-02-25 20:44:02 +00:00
Jo-Philipp Wich
0c3a383974 download.pl: remove ftp.geo.kernel.org mirror, it does not resolve
SVN-Revision: 29947
2012-01-29 19:28:12 +00:00
John Crispin
4d8b9fafcb ignore empty lines in scripts/localmirrors
SVN-Revision: 27416
2011-07-03 19:33:24 +00:00
Jo-Philipp Wich
0649dfdfb6 download.pl: cope with potential two-digit linux versions
SVN-Revision: 27376
2011-07-03 15:00:24 +00:00
Jo-Philipp Wich
a9c7feb0f7 download.pl: adjust kernel download urls for longterm and testing releases
SVN-Revision: 25589
2011-02-19 15:41:00 +00:00
Jo-Philipp Wich
65fad8645d download.pl: pass --no-check-certificate to wget
SVN-Revision: 24686
2010-12-18 20:54:53 +00:00
Travis Kemen
cf7a37f119 add geodns addresses for the kernel, closes #6574
SVN-Revision: 19621
2010-02-13 01:07:44 +00:00
Ralph Hempel
77df307183 add multiple download mirrors
SVN-Revision: 14573
2009-02-20 10:16:47 +00:00
Felix Fietkau
625b692c1f use #!/usr/bin/env perl instead of #!/usr/bin/perl in openwrt scripts (fixes #2998)
SVN-Revision: 12674
2008-09-23 18:29:44 +00:00
Lars-Peter Clausen
2552e9362b Add @GNOME download source.
SVN-Revision: 12469
2008-09-01 21:46:17 +00:00
Matteo Croce
f5e550f438 use the european kernel.org mirror too
SVN-Revision: 11448
2008-06-12 14:48:32 +00:00
Ralph Hempel
c15ca4fb80 adding the possibility of local file cache specify file://path_to_file_cache within localmirrors file to add new local file cache
SVN-Revision: 10617
2008-03-18 21:07:22 +00:00
Andy Boyett
197fafaff6 Increased to five download attempts in download.pl
SVN-Revision: 10486
2008-02-18 21:05:24 +00:00
Felix Fietkau
9bfa0d1d08 show md5sums on mismatch in download.pl
SVN-Revision: 9367
2007-10-20 15:30:16 +00:00
Felix Fietkau
dbeb0fad24 Refactor downloading code into download.mk Support multiple file downloads Support svn downloads
SVN-Revision: 9057
2007-09-29 00:05:48 +00:00
Nicolas Thill
1b226e4de1 add a '@KERNEL' download facility, tweak script
SVN-Revision: 8495
2007-08-26 18:21:24 +00:00
Felix Fietkau
35967a174c add a menuconfig option for specifying a local download mirror
SVN-Revision: 6877
2007-04-06 23:15:39 +00:00
Felix Fietkau
863183ed6e fix download.pl (#1257)
SVN-Revision: 6200
2007-01-25 13:58:52 +00:00
Tim Yardley
7ed63d8d91 add functionality to provide local mirrors for development purposes. * any URLs in scripts/localmirrors will be prepended to the mirror list * this facilitates local builds and recompilations * will also be useful for the buildbot and auto-build of images
SVN-Revision: 6192
2007-01-24 22:38:02 +00:00
Felix Fietkau
8404334073 update sourceforge downloading
SVN-Revision: 5721
2006-12-08 12:44:26 +00:00