Commit Graph

199 Commits

Author SHA1 Message Date
Florian Eckert
42d78fedfb base-files: change logging for upgrade on stage2
Remove vn call in favour of v call. This commit serves as preparation
for removing the vn function call.

Signed-off-by: Florian Eckert <fe@dev.tdt.de>
2021-06-20 15:15:32 -10:00
Florian Eckert
bb2b7156a4 base-files: add syslog logging for v function
The logging output should not only be displayed in the calling shell
session but also in the syslog. A sysupgrade and a configuration
import, export can thus be traced in the syslog.

Signed-off-by: Florian Eckert <fe@dev.tdt.de>
2021-06-20 15:15:32 -10:00
Chuck Fan
d374e2cdb4 base-files: upgrade: use zcat command provided by busybox
Calling `switch_to_ramfs()` will not copy the gzip executable
(/bin/gzip) to ramfs, but `/bin/zcat` will call `/bin/gzip` when
package gzip is installed, instead of the busybox-supplied zcat.
This will cause `zcat` to fail to find `gzip`, then cause the
sysupgrade to fail. Adding the `busybox` prefix here will solve
the problem.

Signed-off-by: Chuck Fan <fanck0605@qq.com>
2021-06-20 14:19:54 -10:00
Sergey Ponomarev
bffee5ea19 busybox: disable bzip2
bzip2 adds about 8kb of size. For tiny builds it's often disabled.
It's not directly used by stock OpenWrt programs.
Kernel images compressed with bzip2 are also not fully supported.

Signed-off-by: Sergey Ponomarev <stokito@gmail.com>
[fix \ indention]
Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-06-20 14:17:49 -10:00
Michael Pratt
b118aba8cb base-files: upgrade: use procd to kill managed daemons
These processes are managed by procd and set to start again when killed
via the procd instance parameter "respawn" being set during init.

Example:
procd_set_param respawn 3600 1 0

When they are killed manually during sysupgrade,
they are started again in 5 seconds or less, depending on
how the "respawn" parameter is set.

Use procd through ubus to disable the instances that respawn them,
however, allow dnsmasq, netifd, and logd to restart for remote logging.

Properly closing all these processes increases free memory by about 3 MB,
which should help low memory devices upgrade without crashing.

For very low memory devices (set to 32 MB for now)
also kill dnsmasq, netifd, and logd for an additional 3 MB of free memory.

Also, bump sleep values to allow at least 10 seconds
for network interfaces and daemons
to come up after they are killed and restarted
before caches are dropped.

Signed-off-by: Michael Pratt <mcpratt@pm.me>
2021-06-12 13:59:27 -10:00
Florian Eckert
e0c6506158 base-files: change logging for upgrade on fwtool
Remove vn call in favour of v call. This commit serves as preparation
for removing the v function call.

Signed-off-by: Florian Eckert <fe@dev.tdt.de>
[alter slightly to prevent double space after colon]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2021-05-17 01:00:14 +02:00
Daniel Golle
a2b9ec0231
base-files: upgrade: take down loop and LVM before upgrade
Users of devices with large block storage may choose to have an LVM
partition on the same device which is used for booting OpenWrt.
The presents a problem during sysupgrade as the root device is then
still busy and changing partitions will not work as desired,
leading to data corruption in case the newly flashed image is larger
than the currently installed one.
Having loop devices setup causes similar havoc.
Make sure all volume groups are offline and all loop devices have been
released before sysupgrade.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2021-05-05 13:18:43 +01:00
Daniel Golle
bb107ad9c1
base-files: functions: introduce new helper functions
Introduce cmdline_get_var() to /lib/function.sh and make use of it in
export_rootdev() in /lib/upgrade/common.sh, making the code more
simple and removing one level of indentation.
Introduce get_partition_by_name() to /lib/upgrade/common.sh which is
useful on non-EFI GPT platforms like mt7622.
Remove some dead-code while at it.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2021-03-31 16:53:39 +01:00
Daniel Golle
287bd78e6f
base-files: remove unneeded '$' signs in nand.sh
When using Shell arithmetric evaluation via $((..)) the variables in
the expression do not need to be prefixed by the '$' sign.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2021-02-24 11:42:02 +00:00
Daniel Golle
5c10f26c28 sysupgrade-nand: allow limiting rootfs_data by setting env variable
Check if firmware environment variable 'rootfs_data_max' exists and is
set to a numerical value greater than 0. If so, limit rootfs_data
volume to that size instead of using the maximum available size.

This is useful on devices with lots of flash where users may want to
have eg. a volume for persistent logs and statistics or for external
applications/containers. Persistence on rootfs overlay is limited by
the size of memory available during the sysugprade process as that
data needs to be copied to RAM while the volume is being recreated
during sysupgrade. Hence it is unsuitable for keeping larger amounts
of data accross upgrade which makes additional volume(s) for
application data desirable.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2021-02-24 01:35:20 +00:00
Daniel Golle
e6aac8d98f image: add support for building FIT image with filesystem
Allow for single (external-data) FIT image to hold kernel, dtb and
squashfs. In that way, the bootloader verifies the system integrity
including the rootfs, because what's the point of checking that the
hash of the kernel is correct if it won't boot in case of squashfs
being corrupted? Better allow bootloader to check everything needed
to make it at least up to failsafe mode. As a positive side effect
this change also makes the sysupgrade process on nand potentially
much easier as it is now.
In short: mkimage has a parameter '-E' which allows generating FIT
images with 'external' data rather than embedding the data into the
device-tree blob itself. In this way, the FIT structure itself remains
small and can be parsed easily (rather than having to page around
megabytes of image content). This patch makes use of that and adds
support for adding sub-images of type 'filesystem' which are used to
store the squashfs. Now U-Boot can verify the whole OS and the new
partition parsers added in the Linux kernel can detect the filesystem
sub-images, create partitions for them, and select the active rootfs
volume based on the configuration in FIT (passing configuration via
device tree could be implemented easily at a later stage).

This new FIT partition parser works for NOR flash (on top of mtdblock),
NAND flash (on top of ubiblock) as well as classic block devices
(ie. eMMC, SDcard, SATA, NVME, ...).
It could even be used to mount such FIT images via `losetup -P` on a
user PC if this patch gets included in Linux upstream one day ;)

Signed-off-by: John Crispin <john@phrozen.org>
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2021-02-24 01:35:20 +00:00
Adrian Schmutzler
331892f85f treewide: drop shebang from non-executable lib files
This drops the shebang from another bunch of files in various /lib
folders, as these are sourced and the shebang is useless.

Fix execute bit in one case, too.

This should cover almost all trivial cases now, i.e. where /lib is
actually used for library files.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2021-01-29 14:29:41 +01:00
Adrian Schmutzler
81655e1450 base-files: read all 3 bytes in get_magic_vfat() at once
While the speed improvement might be negligible, there is still no
reason to read individual bytes.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2021-01-07 19:51:50 +01:00
Kagurazaka Kotori
927b9df938 x86/efi: add FAT32 esp mounting support
Adds a new function get_magic_fat32() in base-files to read FAT32 magic.
Now FAT32 EFI system partition can be handled in the same way as FAT12/FAT16.

Signed-off-by: Kagurazaka Kotori <kagurazakakotori@gmail.com>
[replace '-o' with '] || [' to satisfy shellsheck]
Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-01-03 13:48:50 -10:00
Hannu Nyman
3d12b47985 base-files: flush kernel memory cache during sysupgrade
Flush kernel memory caches during sysupgrade in order
to mitigate the impact from memory consumption spikes
in low-RAM devices.

This may help to prevent sysupgrade causing a reboot
before the actual flashing starts.

Signed-off-by: Hannu Nyman <hannu.nyman@iki.fi>
2020-12-22 18:59:10 +01:00
Yousong Zhou
00fb51f97e base-files: upgrade: stage2: use v for log lines
Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
2020-11-11 21:29:55 +08:00
Yousong Zhou
128bb4e8d3 base-files: upgrade: fwtool.sh: rewording logs
The intent is to make it sound more like info level message, not some
error like "404 not found".  x86 target at the moment makes image with
only signature but no metadata (ref commit f8141216 "x86: append
metadata to combined images").

Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
Reviewed-By: Philip Prindeville <philipp@redfish-solutions.com>
2020-11-11 21:29:55 +08:00
Yousong Zhou
856b288d27 base-files: upgrade: fwtool.sh: use v for log lines
This will have at least the following effects

 - Log lines will have common prefix
 - They will be output to stderr instead of stdout

Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
2020-11-11 21:29:55 +08:00
Yousong Zhou
50b870ee3c base-files: upgrade: add get_image_dd()
This is mainly to handle stderr message "Broken pipe", "F+P records
in/out" by common pattern "xcat | dd .."

Ref: https://bugs.openwrt.org/index.php?do=details&task_id=3140
Reported-by: Philip Prindeville <philipp@redfish-solutions.com>
Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
Reviewed-By: Philip Prindeville <philipp@redfish-solutions.com>
2020-11-11 21:29:55 +08:00
Yousong Zhou
bd21e4a936 base-files: upgrade: use stdin redirection to replace cat command
Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
2020-11-11 21:29:55 +08:00
Yousong Zhou
7863c33cea base-files: upgrade: add vn and variants
To be used with in the following pattern

  vn "Remaining: "
  for p in $xx; do
    _vn "$p"
  done
  _v

Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
2020-11-11 21:29:55 +08:00
Yousong Zhou
826bb13742 base-files: upgrade: log with date prefix
And log to stderr

Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
2020-11-11 21:29:55 +08:00
Felix Fietkau
4a0688ed71 base-files: remove block2mtd checks from sysupgrade
This hasn't been used in a long time

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2020-10-17 16:43:17 +02:00
Paul Spooren
1fdf6b745c treewide: replace which with command -v
Fix shellcheck SC2230
> which is non-standard. Use builtin 'command -v' instead.

Using `command -v` is POSIX compliant while `which` is not.  Also to
mention, `command -v` is a shell builtin whereas `which` is a separate
busybox applet.

Once applied to everything concerning OpenWrt we can disable the busybox
feature `which` and save 3.8kB.

Acked-by: Stijn Tintel <stijn@linux-ipv6.be>
Signed-off-by: Paul Spooren <mail@aparcar.org>
[also replace cases in zram-swap]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2020-08-12 11:08:05 +02:00
Adrian Schmutzler
02d6ac1060 base-files: fwtool: make compat_version backward compatible
So far, the compatibility mechanism only works if both device and
image are already updated to the new routines. This patch extends
the sysupgrade metadata and fwtool_check_image() to account for
"older" images as well:

The basic mechanism for older devices to check for image compatibility
is the supported_devices entry. This can be exploited by putting
a custom message into this variable of the metadata, so older FW
will produce a mismatch and print the message as it thinks it's the
list of supported devices. So, we have two cases:

device 1.0, image 1.0:
  The metadata will just contain supported_devices as before.

device 1.0, image 1.1:
  The metadata will contain:

  "new_supported_devices":["device_string1", "device_string2", ...],
  "supported_devices":["Image version 1.1 incompatible to device: ..."]

  If the device is "legacy", i.e. does not have the updated fwtool.sh,
  it will just fail with image check and print the content of
  supported_devices. If DEVICE_COMPAT_MESSAGE is set, this will be
  printed on old devices as well through the same mechanism. Otherwise
  a generic "Please check documentation ..." is appended.

  Upgrade can still be performed with -F like when
  SUPPORTED_DEVICES has been removed to prevent bricking.

  If the device has updated fwtool.sh (but is 1.0), it will just use
  the new_supported_devices instead, and work as intended (flashing
  with -n will work, flashing without will print the appropriate
  warning).

This mechanism should provide a fair tradeoff between simplicity
and functionality.

Since we touched a lot of fields in metadata, this also bumps
metadata_version to 1.1.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2020-07-31 11:40:15 +02:00
Adrian Schmutzler
ad3e1f9db4 base-files: fwtool: implement compatibility check for images
We regularly encounter the situation that devices are subject to
changes that will make them incompatible to previous versions.
Removing SUPPORTED_DEVICES will not really be helpful in most of these
cases, as this only helps after a rename.

To solve this situation, this patchset introduces a compatibility
version for devices. In this patch, the actual checks are implemented
into fwtool_check_image():

If an incompatible change is introduced, one can increase either
the minor version (1.0->1.1) or the major version (1.0->2.0).

Minor version increment:
This will still allow sysupgrade, but require to reset config
(-n or SAVE_CONFIG=0). If sysupgrade is called without -n, a
corresponding message will be printed. If sysupgrade is called
with -n, it will just pass, with supported devices being checked
as usual. (Which will allow us to add back SUPPORTED_DEVICES for
many cases.)

Major version increment:
This is meant for potential (rare) cases where sysupgrade is
not possible at all, because it would break the device.
In this case, a warning will be printed, and -n won't help.

If image check fails because of one of the versions parts not
matching, the content of DEVICE_COMPAT_MESSAGE is printed in
addition to the generic message (if set).

For both cases, upgrade can still be forced with -F as usual.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2020-07-31 11:40:15 +02:00
Adrian Schmutzler
e49fac097c base-files: replace backticks in lib/upgrade/nand.sh
This replaces deprecated backticks by more versatile $(...) syntax.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2020-07-15 18:33:56 +02:00
Álvaro Fernández Rojas
8339f8d95e base-files: switch_to_ramfs: add nand-utils
Signed-off-by: Álvaro Fernández Rojas <noltari@gmail.com>
2020-05-18 18:24:06 +02:00
Javier Marcet
02656caa7b base-files: upgrade: fix indent
Use same indent as for the rest of the file.

Signed-off-by: Javier Marcet <javier@marcet.info>
[add commit description]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2020-05-08 20:15:43 +02:00
Alexander Couzens
b77fd0d30b base-files: ensure VERBOSE is set
If not set, it shows the following error
sh: out of range

Signed-off-by: Alexander Couzens <lynxis@fe80.eu>
2020-04-21 00:01:49 +02:00
Kevin Darbyshire-Bryant
c14c6902f5 treewide: convert sed -r to posix -E
Signed-off-by: Kevin Darbyshire-Bryant <ldir@darbyshire-bryant.me.uk>
2020-04-04 11:28:15 +01:00
李国
a6b7c3e672 x86: generate EFI platform bootable images
Add EFI platform bootable images for x86 platforms. These images can
also boot from legacy BIOS platform.

EFI System Partition need to be fat12/fat16/fat32 (not need to load
filesystem drivers), so the first partition of EFI images are not ext4
filesystem any more.

GPT partition table has an alternate partition table, we did not
generate it. This may cause problems when use these images as qemu disk
(kernel can not find rootfs), we pad enough sectors will be ok.

Signed-off-by: 李国 <uxgood.org@gmail.com>
[part_magic_* refactoring, removed genisoimage checks]
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-03-31 16:20:47 +02:00
Jeffery To
12020f8a95 base-files: Add /etc/shinit for non-login shell init
Because /etc/profile (and ~/.profile) are read by login shells only,
aliases and functions defined there are not available to non-login
shells, e.g. when using screen or tmux.

If the ENV environment variable exists (exported by /etc/profile or
~/.profile) and references an existing file, then all interactive shells
(login or non-login) will read that file as well.

This sets the ENV environment variable in /etc/profile, pointing to
/etc/shinit.

This also adds /etc/shinit, which:

* Contains alias and function definitions originally in /etc/profile

* Sources /etc/mkshrc if the user is using mksh (also originally in
  /etc/profile), as /etc/mkshrc is meant for all interactive shells

* Sources ~/.mkshrc if the user is using mksh, to compensate for the
  fact that mksh will not read ~/.mkshrc if ENV is set

* Sources ~/.shinit if the user is not using mksh

This also removes the shebang from /etc/profile, as the file is sourced,
not executed.

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
2020-03-01 21:35:59 +01:00
Russell Senior
b20b997c68 base-files: add /usr/share/libubox/jshn.sh to sysupgrade stage2
Discovered recent changes had broken sysupgrade for ar71xx mikrotik
rb-493g, traced the problem to missing /usr/share/libubox/jshn.sh after
switching to tmpfs.

Signed-off-by: Russell Senior <russell@personaltelco.net>
2019-11-13 18:55:00 +01:00
Klaus Kudielka
3a4f587c46 base-files: upgrade: add case to export_bootdevice
The factory uboot of the Turris Omnia boots with "root=b301", and we
instruct new users to sysupgrade from there (e.g. method 1, step 7).
Currently, this will fail with "Unable to determine upgrade device".
Add a new case to export_bootdevice, which parses the hex argument.

Fixes commit 2e5a0b81 ("mvebu: sysupgrade: sdcard: keep user added ...")

Signed-off-by: Klaus Kudielka <klaus.kudielka@gmail.com>
2019-10-09 21:00:02 +02:00
Rafał Miłecki
a858db3136 treewide: sysupgrade: use $UPGRADE_BACKUP to check for backup
Now that $UPGRADE_BACKUP is set conditionally there is no need to check
the $UPGRADE_OPT_SAVE_CONFIG anymore. All conditions can be simplified.

Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
2019-09-11 09:05:35 +02:00
Rafał Miłecki
641f6b6c26 treewide: use new procd sysupgrade $UPGRADE_BACKUP variable
It's a variable set by procd that should replace hardcoded
/tmp/sysupgrade.tgz.

This change requires the most recent procd with the commit 0f3c136
("sysupgrade: set UPGRADE_BACKUP env variable").

Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
2019-09-05 23:33:19 +02:00
Rafał Miłecki
bf39047872 treewide: don't hardcode "sysupgrade.tgz" file name
1) Add BACKUP_FILE and use it when copying an archive to be restored
   after sysupgrade (on the next preinit).
2) Use CONF_TAR for copying backup prepared by the /sbin/sysupgrade

Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
2019-09-05 14:33:19 +02:00
Rafał Miłecki
b6f4cd57e1 treewide: sysupgrade: pass "save_partitions" option to the "sysupgrade" method
This explicitly lets stage2 know if partitions should be preserved. No
more "touch /tmp/sysupgrade.always.overwrite.bootdisk.partmap" hack.

Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
2019-08-22 13:47:51 +02:00
Rafał Miłecki
b534ba9611 base-files: pass "save_config" option to the "sysupgrade" method
This explicitly lets stage2 know if config should be preserved.

Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
2019-08-22 13:47:47 +02:00
Rafał Miłecki
790692dde2 base-files: drop support for the platform_nand_pre_upgrade()
No target uses it anymore. All code from that callback was moved into
the platform_do_upgrade().

Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
2019-07-22 14:27:37 +02:00
Rafał Miłecki
3f4c785a6b base-files: don't set ARGV and ARGC
Those are not used by any image check function anymore.

Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
2019-07-17 08:10:40 +02:00
Rafał Miłecki
f7edd94a65 base-files: move stage2 upgrade to separated file
do_upgrade_stage2() isn't really any common code. It isn't used anywhere
except for /sbin/sysupgrade that passes it to the stage2.

Moving its code to separated file also simplifies COMMAND variable.

Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
2019-07-11 17:05:20 +02:00
Rafał Miłecki
ea4e1dac71 base-files: drop support for NAND upgrade in platform_pre_upgrade()
With bcm53xx switched to the new procedure there is no more need for
keeping that backward compatibility code.

Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
2019-07-08 07:38:51 +02:00
Rafał Miłecki
f58ca6ee57 base-files: drop unused jffs2_copy_config()
Its last usage was dropped back in 2013 in the commit b95bdc8ab5
("kernel/base-files: clean up old code related to refreshing mtd
partitions, it is no longer used anywhere").

Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
2019-07-08 07:38:06 +02:00
Klaus Kudielka
ad62247800 base-files: improve lib/upgrade/common.sh
Recently, upgrade device autodetection has been added to the mvebu target.
This exposes some shortcomings of the generic export_bootdevice function,
e.g. on the Turris Omnia: export_bootdevice silently reports the root
partition to be the boot device. This makes the sysupgrade process fail at
several places.

Fix this by clearly distinguishing between /proc/cmdline arguments which
specify the boot disk, and those which specify the root partition. Only in
the latter case, strip off the partition, and do it consistently.
root=PARTUUID=<pseudo PARTUUID for MBR> (any partition) and root=/dev/*
(any partition) are accepted.

The root of the problem is that the *existing* export_bootdevice in
/lib/upgrade/common.sh behaves differently, if the kernel is booted with
root=/dev/..., or if it is booted with root=PARTUUID=...

In the former case, it reports back major/minor of the root partition,
in the latter case it reports back major/minor of the complete boot disk.

Targets, which boot with root=/dev/... *and* use export_bootdevice /
export_partdevice, have added workarounds to this behaviour, by specifying
*negative* increments to the export_partdevice function.

Consequently, those targets have to be adapted to use positive increments,
otherwise they are broken by the change to export_bootdevice.

Fixes: 4e8345ff68 ("mvebu: base-files: autodetect upgrade device")
Signed-off-by: Klaus Kudielka <klaus.kudielka@gmail.com>
Tested-by: Tomasz Maciej Nowak <tomek_n@o2.pl>
2019-05-11 16:37:11 +02:00
Felix Fietkau
b044b52ab9 base-files: fix ucert verification
ucert needs to check the firmware part with metadata, but without the signature.
Use the new fwtool mode to extract that without altering the firmware image inside
the check

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2019-02-12 16:42:03 +01:00
Felix Fietkau
61e01f248e base-files: do not strip fwtool signature data during check
Same reason as in commit 9808bd2799 -
sysupgrade --test must not alter the image in any way

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2019-02-09 14:34:24 +01:00
David Bauer
28cd2caa35 base-files: sysupgrade: support additional mtd options
Add support for passing additional parameters to mtd called during
sysupgrade. It will be required to toggle the "recovery moe" flag
supported by recent tp-link boards.

Signed-off-by: David Bauer <mail@david-bauer.net>
[split code from board support patch; add commit message]
Signed-off-by: Mathias Kresin <dev@kresin.me>
2019-01-26 21:46:32 +01:00
INAGAKI Hiroshi
7e9b93fb1b base-files: allow non-standard rootfs volume name in UBI in sysupgrade
This commit allows to use non-standard UBI volume name as the rootfs
volume in sysupgrade.

ex.:
  The U-Boot on Buffalo WXR-2533DHP checks existence and checksum of
  "ubi_rootfs" volume when booting, so this name is required.

OpenWrt currently provides several patches:

490-ubi-auto-attach-mtd-device-named-ubi-or-data-on-boot.patch
491-ubi-auto-create-ubiblock-device-for-rootfs.patch
492-try-auto-mounting-ubi0-rootfs-in-init-do_mounts.c.patch

to facilitate ubi rootfs automount. However the upstream kernel
also supports the means of booting from a fully custom ubi
partition name and ubi volume name via bootargs/kernel's cmdline
parameters:

ubi.mtd=mtd_partition_name
ubi.block=rootfs_volume_name
root=/dev/ubiblock$X_$Y

For more information and examples visit the wiki over at linux-mtd:
<http://www.linux-mtd.infradead.org/faq/ubifs.html>
<http://www.linux-mtd.infradead.org/doc/ubi.html>

Signed-off-by: INAGAKI Hiroshi <musashino.open@gmail.com>
Signed-off-by: Christian Lamparter <chunkeey@gmail.com> [reworded commit]
2019-01-01 16:38:20 +01:00
Michael Heimpold
d6ac8ca76c base-files: fix several bashisms
For equality test a simple = is sufficient, the == is
usually disregarded as bashism.

Signed-off-by: Michael Heimpold <mhei@heimpold.de>
2018-12-05 22:16:52 +01:00
Jo-Philipp Wich
9808bd2799 Revert "base-files: fwtool: Fix wrong checksum on combined-image with metadata"
This reverts commit 41770add03.

The fwtool_check_image() procedure is used by `sysupgrade --test` which must
not alter the image under test in any way.

Currently, when the LuCI ui or any other sysupgrade wrapper first invokes
sysupgrade --test to verify the compatibility of the image and then calculates
the sha256sum over it, the resulting checksum will differ from the original
image since the test invocation will implicitely strip the metadata trailer.

To properly fix the underlying issue, the combined image checksumming code
must be modified to skip the metadata trailer.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2018-12-02 16:05:44 +01:00
Petr Štetiar
41770add03 base-files: fwtool: Fix wrong checksum on combined-image with metadata
If I create following image:

define Device/engenius-m36
  IMAGE/sysupgrade.bin := combined-image | append-metadata
endef

Sysupgrade then errors out:

  Invalid image. Contents do not match checksum (image:cd285595eaf297370404ae0e2815ec1a calculated:2cf9a2286fb6b01af3ea189128017d44)
  Image check 'platform_check_image' failed.

By removing the metadata from the image I get combined-image checksum
working again and sysupgrade works.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2018-11-26 12:05:45 +01:00
Mathias Kresin
56b8ac1e86 treewide: consolidate upgrade state set
Set the (sys)upgrade state when sourcing the stage2 script instead of
setting the state for each target individual.

This change fixes the, due to a missing state set, not working upgrade
led on ath79 and apm821xx.

Signed-off-by: Mathias Kresin <dev@kresin.me>
2018-08-16 21:20:57 +02:00
Daniel Golle
8174853c78 base-files: introduce sysupgrade signature chain verification
Verify ucert signature chains in sysupgrade images in case ucert is
installed and $CHECK_IMAGE_SIGNARURE = 1.
Also make sure ucert host binary is present and generate a self-signed
ucert in case $TOPDIR/key-build.ucert is missing.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2018-08-08 02:22:54 +02:00
John Crispin
5e1b4c57de base-files: drop fwtool_pre_upgrade
this feature has never worked, the fw image name was not passed and the -t
parameter was missing in the tool invocation. drop the feature.

Signed-off-by: John Crispin <john@phrozen.org>
2018-07-30 17:42:39 +02:00
Rafał Miłecki
5b2e20807d base-files: exit if mtd write command fails during sysupgrade
It avoids confusing situations like:
> Could not get image magic
> Image check failed.
> Upgrade completed
> Rebooting system...

Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
2018-06-18 06:37:28 +02:00
Matthias Schiffer
3b6f865b01
base-files: sysupgrade: fix handing get_image unpack commands
On bcm53xx and brcm47xx, commands are passed to default_do_upgrade that
expect the image to be passed on stdin, rather than as an argument.

Fixes: 30f61a34b4 ("base-files: always use staged sysupgrade")
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-06-11 19:25:40 +02:00
Daniel Golle
267873ac9b
base-files: don't evaluate block-device uevent
Current code and also before commit da52dd0c83 was vulnerable to shell
injection using volume lables in the GPT partition table of block
devices. Given that partition names can be freely defined in GPT tables
we really shouldn't evaluate a string which is potentially crafted with
evil intentions. Hence rather use `export -n` to absorb the uevent's
variables into the environment.

Fixes commit da52dd0c83 (base-files: quote values when evaluating uevent)
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
[mschiffer@universe-factory.net: suggested export -n usage]
2018-02-13 00:01:44 +01:00
Daniel Golle
da52dd0c83 base-files: quote values when evaluating uevent
When sourcing /sys/class/block/*/uevent values have to be quoted as
they may contain spaces (e.g. in PARTNAME).
Fix this by pre-processing with sed before sourcing.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2018-02-11 16:42:01 +01:00
Mathias Kresin
93bd46b719 procd: nand: remove nand_board_name platform override
It isn't uses anymore by any target.

Signed-off-by: Mathias Kresin <dev@kresin.me>
2017-12-08 20:57:12 +01:00
Mathias Kresin
c90a8cb755 procd: nand: dont rely on boardname in nand_upgrade_tar
Kernel and rootfs in a subdirectory matching the userspace boardname,
was intended to use a single sysupgrade-tar archive for multiple boards
with different kernel/rootfs images. This feature was never used.

Use the first found directory in the tar archive instead of relying on
a directory named according to the userspace boardname.

It allows to change the boardname without adding another compatibility
layer - using the nand_board_name() function - for (sub)targets using
the metadata based image validation in favour to
nand_do_platform_check().

Signed-off-by: Mathias Kresin <dev@kresin.me>
2017-12-08 20:57:12 +01:00
Christian Lamparter
4e3f6dae04 base-files: upgrade: make get_partitions() endian agnostic
This patch fixes two issues with the current get_partitions()
function.

First: "Invalid partition table on $disk" will pop up on
legitimate images on big endian system.

This is because the little-endian representation of "55 AA" is
assumed in the context of little-endian architectures. On these
comparing it to the 16-bit word 0xAA55 does work as intented.
Whereas on big-endian systems, this would have to be 0x55AA.

This patch fixes the issue by replacing the integer conversion
and value match check with just a string comparision.

Second: The extraction of the type, start LBA and LBA num from
the partition table has the same endianness issue. This has been
fixed by using the new hex_le32_to_cpu() function. This function
will translate the stored little-endian data to the correct
byte-order if necessary.

Signed-off-by: Christian Lamparter <chunkeey@gmail.com>
2017-12-08 19:54:16 +01:00
Mat Trudel
2dca7177e7 sysupgrade: don't kill our own parent
Add the parent of the sysupgrade script to the list of pids not getting
killed

Signed-off-by: Mat Trudel <mat@geeky.net>
Signed-off-by: John Crispin <john@phrozen.org>
2017-11-15 21:11:23 +01:00
Matthias Schiffer
1ab227d688
base-files: upgrade: don't loop forever trying to kill processes
When processes don't die on SIGKILL (usually because of kernel bugs), it's
better to give up instead of looping forever.

upgraded will trigger a reboot in this case (and if this fails, a hardware
watchdog will eventually time out and reset the system, if present).

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2017-07-20 21:41:33 +02:00
Matthias Schiffer
438dcbfe74
base-files: automatically handle paths and symlinks for RAMFS_COPY_BIN
Depending on busybox applet selection, paths of basic utiilties may differ,
and may not work as symlinks to busybox. Simply using whatever binary is
found in PATH and detecting symlinks automatically is more robust and
easier to maintain.

The list of binaries is also slightly cleaned up and duplicates are
removed.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2017-07-11 17:26:32 +02:00
Matthias Schiffer
73f675415c
bcm53xx: upgrade: fix RAMFS_COPY_*
Fixes: 30f61a34b4 "base-files: always use staged sysupgrade"
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2017-07-11 17:19:23 +02:00
Matthias Schiffer
7a29e44f90
base-files: upgrade: correctly handle nand_do_upgrade argument passed from preupgrade
Fixes: 30f61a34b4 "base-files: always use staged sysupgrade"
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2017-07-11 17:19:23 +02:00
Matthias Schiffer
5654a03768
mvebu: fix sysupgrade
mvebu was modifying RAMFS_COPY_BIN and RAMFS_COPY_DATA from a
sysupgrade_pre_upgrade hook. As the ramfs is created from stage2, this
did not have an effect anymore after the staged sysupgrade changes.

As it doesn't really hurt to copy fw_printenv and fw_setenv
unconditionally, simply add them in /lib/upgrade/platform.sh, so stage2
will see them.

Config copying is moved to a function called by platform_copy_config, where
it belongs.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
Fixes: FS#821
Fixes: 30f61a34b4 "base-files: always use staged sysupgrade"
2017-06-01 20:41:19 +02:00
Christian Lamparter
7783f31359 base-files: nand: use CI_KERNPART whenever the kernel volume is needed
The sender domain has a DMARC Reject/Quarantine policy which disallows
sending mailing list messages using the original "From" header.

To mitigate this problem, the original message has been wrapped
automatically by the mailing list software.
This patch is in continuation of: commit 93aa860405
"procd: nand: make it possible to configure kernel and ubi partition"

The $CI_KERNPART variable should be used in place
of the fixed "kernel" partition name. This allows
targets to specifiy alternate names for the kernel
partition.

Cc: Chris Blake <chrisrblake93@gmail.com>
Signed-off-by: Christian Lamparter <chunkeey@googlemail.com>
2017-05-31 09:55:01 +02:00
Matthias Schiffer
b2adb9a7b5
base-files: sysupgrade cleanup
Some functions only used by stage2 are moved there from common.sh.

One piece that could still use more cleanup is platform_pre_upgrade: many
targets reference files from there are aren't available in the ramfs, so
we need to evaluate it before the switch; conversely, flash writes happen
in that function on some targets. Targets that do the latter should be
fixed eventually to use platform_do_upgrade for that purpose.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2017-05-29 23:50:33 +02:00
Matthias Schiffer
30f61a34b4
base-files: always use staged sysupgrade
Support for the -d and -p options is dropped; it may be added again at some
point by adding these flags to the ubus sysupgrade call.

A downside of this is that we get a lot less information about the progress
of the upgrade: as soon as the actual upgrade starts, all shell sessions
are killed to allow unmounting the root filesystem.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2017-05-29 23:50:32 +02:00
Matthias Schiffer
393817df5d
procd: remove procd-nand package
We always want to support staged upgrades now, so it's better to include
upgraded into the main package. /lib/upgrade/nand.sh is moved to
base-files.

The procd-nand-firstboot package is removed for now, it may return later
as a separate package.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2017-05-29 23:50:32 +02:00
Felix Fietkau
7304510392 base-files: save /bin/mknod for sysupgrade
It is used on NAND devices in case hotplug is too slow

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-06 15:34:14 +01:00
Yousong Zhou
6f61d8511e base-files: export x86 platform upgrade functions to common.sh
Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
2017-01-05 11:09:14 +01:00
Felix Fietkau
81b5e8e5d2 base-files: add a hint in sysupgrade that shows what to do when the image metadata check fails
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-12-04 11:41:49 +01:00
Felix Fietkau
cc853810a4 base-files: validate metadata of sysupgrade images
Use fwtool to extract it, only require metadata to be present if the
platform sysupgrade script sets REQUIRE_IMAGE_METADATA=1

Image metadata is in JSON format and contains a list of supported
devices, along with version information that could be displayed by a UI
later before the actual upgrade happens.

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-11-19 11:24:09 +01:00
André Valentin
783875f18b package/basefiles: add mkfs.ext4 and losetup binaries to ramfs list
mkfs.ext4 und losetup are needed for sysupgrade support on mmc devices
with automatic rootfs split (loopback device usage).

Signed-off-by: André Valentin <avalentin@marcant.net>
2016-10-26 12:37:47 +02:00
Mathias Kresin
3004298e62 sysupgrade: unmount filesystems before reboot
sysupgrade immediately reboots after flashing an image and doesn't
allow to unmount filesystems. At least in case the image used for
sysupgrade is stored on a FAT formatted usb flash drive, the following
warning is printed during the next mount of the flash drive:

FAT-fs (sda1): Volume was not properly unmounted. Some data may be
corrupt. Please run fsck.

Although a data corruption during read operations is unlikely, there is
no need to scare the users.

Signed-off-by: Mathias Kresin <dev@kresin.me>
2016-08-04 18:19:46 +02:00
Conn O'Griofa
e1406cd31a base-files: sysupgrade: fix pseudobridge upgrades
Treat 'relayd' as an essential service to avoid connection interruptions during sysupgrade on devices configured as a pseudobridge.

Signed-off-by: Conn O'Griofa <connogriofa@gmail.com>
2016-07-24 06:54:40 +02:00
Jo-Philipp Wich
1012701014 x86: generalize partition discovery for sysupgrade
Generalize the partition discovery in sysupgrade in order to fix sysupgrade
and config backup/recovery on MMC block devices which use a different naming
scheme compared to mtdblock or sd* devices.

The change also adds the find applet to the ramdisk utilities so that upgrade
code can rely on it.

The commit is based on the initial submission by Russell Senior at
http://patchwork.ozlabs.org/patch/625440/ .

Signed-off-by: Russell Senior <russell@personaltelco.net>
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2016-05-24 13:30:58 +02:00
Jo-Philipp Wich
4aeeead303 x86: preserve partition table on sysupgrade
With this patch sysupgrade will write directly to the partitions
instead of to the main disk.  The UUID is copied from the image
to the MBR as well.  This prevents the mbr from being completely
overwritten and losing the partition table.  The -p option has
been added to maintain the original behavior and overwite the
entire disk with the new image.  Tests have been added to ensure
that the image partitions match up with the active partitions.

Signed-off-by: Rob Mosher <nyt-openwrt@countercultured.net>

SVN-Revision: 48682
2016-02-09 12:33:17 +00:00
Felix Fietkau
b7ab6c66cc base-files: fix sysupgrade 'wget' handling for uclient-fetch
change 48451 tried to add support for uclient-fetch by moving
/usr/bin/wget to /bin/wget, but this change kept the symbolic
link to /bin/busybox as install_bin creates links to param 1

the desired fix is to link to uclient-fetch to wget:
  install_bin /bin/uclient-fetch /bin/wget

Signed-off-by: John Clark <inindev@gmail.com>

SVN-Revision: 48483
2016-01-25 12:35:21 +00:00
Felix Fietkau
f57419d26e base-files: fix sysupgrade 'wget' handling
with r48379 and r48386 the path of wget changed.
respect that and adjust the dirname.

this fixes #21680

Signed-off-by: Bastian Bittorf <bittorf@bluebottle.com>

SVN-Revision: 48451
2016-01-22 09:41:58 +00:00
John Crispin
4ef319fad2 base-files: sysupgrade: quote source image name
get_image() eval a filename without quoting it, resulting in errors with
filenames containing characters that need to be escaped.
    $ sysupgrade -T -f ./cfg\(12\).tar.gz img.bin; echo $?
    Image check 'platform_check_image' failed.
    Invalid config file. Please use only .tar.gz files
    1
    $ mv cfg\(12\).tar.gz cfg_12.tar.gz
    $ mv img\(1\).bin img.bin
    sysupgrade -T -f ./cfg_12.tar.gz img.bin; echo $?
    0

Enclose the content of $from in double quotes.

Signed-off-by: Adrien Schildknecht <adrien+dev@schischi.me>

SVN-Revision: 46919
2015-09-14 20:09:06 +00:00
John Crispin
f57076956e base-files: default_do_upgrade: accept & forward image extraction command
get_image allows passing 2nd argument that is used in a pipe for
extracting firmware from a non-native format. By accepting such command
in default_do_upgrade we allow platforms to use this helper for vendor
specific images.

Signed-off-by: Rafał Miłecki <zajec5@gmail.com>

SVN-Revision: 45900
2015-06-05 14:12:05 +00:00
Luka Perkov
0a4bcd5be6 base-files: remove ubi tool
ubi tool was removed from fstools in r41128.

Signed-off-by: Luka Perkov <luka@openwrt.org>

SVN-Revision: 45813
2015-05-29 08:14:38 +00:00
Jonas Gorski
02c6798a0d base-files: supress errors from ldd
ldd might be called for shell scrips during sysupgrade, causing it to
complain that they are not a dynamic executables.

This is a harmless error, so supress it to avoid confusing about them
being serious ones.

Signed-off-by: Jonas Gorski <jogo@openwrt.org>

SVN-Revision: 43315
2014-11-19 12:17:37 +00:00
Jonas Gorski
2e95b1fda7 base-files: properly fix copying ld*.so with eglibc
r36107 ("base-files: sysupgrade fail with eglibc") tried to fix
sysupgrade by changing the ld-soname to what is expected, but only
fixed MIPS while breaking ARM.

The underlying issue is that the ld.so name varies widely across
different architectures for eglibc:

eglibc-2.19-r25243$ grep -r "ld-soname :=" . | awk '{ print $3 }' | sort -u
ld64.so.1
ld64.so.2
ld-linux-aarch64_be.so.1
ld-linux-aarch64.so.1
ld-linux-armhf.so.3
ld-linux-mipsn8.so.1
ld-linux.so.2
ld-linux.so.3
ld-linux-x32.so.2
ld-linux-x86-64.so.2
ld.so.1

Instead of adding each different soname to check for and copy it,
replace the awk script with a sed script to extract it properly and
drop the hardcoded so-name.

Signed-off-by: Jonas Gorski <jogo@openwrt.org>

SVN-Revision: 43295
2014-11-18 15:19:46 +00:00
Felix Fietkau
7a5002ec9d base-files: add /etc/shadow to list of essential files to keep in sysupgrade (bug 18206)
Based on bug #18206 sysupgrade can lead to loss of password information in
certain situations. Most likely all users who will upgrade from versions
r43017-43040, will lose their current passwords. :-(
https://dev.openwrt.org/ticket/18206

Currently /etc/shadow is defined as a conffile in base-files:
https://dev.openwrt.org/browser/trunk/package/base-files/Makefile#L37

But it is not defined in the default list of essential files to keep in
sysupgrade:
https://dev.openwrt.org/browser/trunk/package/base-files/files/lib/upgrade/keep.d/base-files-essential

If exporting conffiles info fails, /etc/shadow can get lost.

Shadow passwords are now the default, so saying that preserving /etc/passwd
is essential while /etc/shadow is not, makes no sense.

The attached patch adds /etc/shadow to the list of essential files.

Signed-off-by: Hannu Nyman <hannu.nyman@iki.fi>

SVN-Revision: 43061
2014-10-25 17:22:46 +00:00
Felix Fietkau
0ef474a118 base-files: fix ldd definition if /usr/bin/ldd exists
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 42976
2014-10-19 21:46:08 +00:00
John Crispin
be26421f45 sysupgrade: Enable killing of all processes under upgraded
If the sysupgrade scripts is called under upgraded, it will not kill all
other processes as it should to avoid interference by locked filesystem.
This patch checks the parent and if it is upgraded, it kills all.

Signed-off-by: André Valentin <avalentin@marcant.net>

SVN-Revision: 41563
2014-07-10 19:16:27 +00:00
John Crispin
66ae4803e2 base-files: fix ramfs / nand upgrade
/lib/{imx6,lantiq,...}.sh are needed in the ramfs for sysupgrade to work.

Signed-off-by: John Crispin <blogic@openwrt.org>

SVN-Revision: 41440
2014-07-01 10:26:14 +00:00
John Crispin
09273f5c0a base-files: regenerate the /lib64 symlink when switching to a ramdisk
Signed-off-by: John Crispin <blogic@openwrt.org>

SVN-Revision: 41284
2014-06-20 13:41:08 +00:00
John Crispin
4b1c48bc93 base-files: add upgraded to the list of daemons not killed
Signed-off-by: John Crispin <blogic@openwrt.org>

SVN-Revision: 41268
2014-06-19 14:12:55 +00:00
John Crispin
bedd6f44a5 base-files: properly remove stderr from zcat/bzcat
Signed-off-by: John Crispin <blogic@openwrt.org>

SVN-Revision: 41135
2014-06-11 13:00:25 +00:00
John Crispin
e73538ba56 base-files: include more ubi* tools in sysupgrade ramdisk
Also add some more busybox symlinks which are useful for sysupgrade
on UBI-enabled devices.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>

SVN-Revision: 41125
2014-06-11 12:59:33 +00:00
John Crispin
0de0310a9c base-files: Allow installation of symlinks using RAMFS_COPY_BIN env var.
http://patchwork.openwrt.org/patch/5086/

Signed-off-by: karl.vogel@gmail.com

SVN-Revision: 40943
2014-06-02 12:45:13 +00:00
John Crispin
665d39f698 base-files: the ubi tol should be installed when jumping to ram_fs
Signed-off-by: John Crispin <blogic@openwrt.org>

SVN-Revision: 40420
2014-04-07 15:35:20 +00:00
John Crispin
1d5ac95850 base-files: make use of new fstools
Signed-off-by: John Crispin <blogic@openwrt.org>

SVN-Revision: 40416
2014-04-07 15:10:15 +00:00