1. Return error if any step of generating tar file fails
2. Use pipefail to avoid calling "gzip" if tar failed
Fixes: e36cc53092 ("base-files: sysupgrade: use tar helper to include installed_packages.txt")
Reported-by: Luiz Angelo Daros de Luca <luizluca@gmail.com>
Cc: Luiz Angelo Daros de Luca <luizluca@gmail.com>
Cc: Jo-Philipp Wich <jo@mein.io>
Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
Some platforms like Raspberry Pi require patching some backup files like
cmdline.txt in order to set the correct root PARTUUID.
Signed-off-by: Álvaro Fernández Rojas <noltari@gmail.com>
Due to previous refactoring in sysupgrade, writing backup archives to
stdout became impossible since the hardcoded gzip output redirection
did not account for the `-` special case filename.
Fix this issue by substituting `-` with `/proc/self/fd/1` in the tar
archive output path variable.
Also remove a redundant `rm -f` of the target file path that occurs
before the file could've possibly been written.
Fixes: #14773
Fixes: https://github.com/openwrt/luci/issues/6961
Fixes: e36cc53092 ("base-files: sysupgrade: use tar helper to include installed_packages.txt")
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Disabled services should be kept disabled after sysupgrade. This can be
easily handled using a proper uci-defaults script.
Extend sysupgrade to check for disabled services, generate uci-defaults
script disabling them and include it in backup.
Cc: Christian Marangi <ansuelsmth@gmail.com>
Cc: Jo-Philipp Wich <jo@mein.io>
Cc: Jonas Gorski <jonas.gorski@gmail.com>
Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
Acked-by: Jo-Philipp Wich <jo@mein.io>
Replace mount + overlay with manually built tar archive that gets
prepended to the actual config files backup. This allows more
flexibility with including extra backup files. They can be included at
any paths and don't require writing to flash or mounting an overlay
which has its own limitations (mount points).
Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
Acked-by: Jo-Philipp Wich <jo@mein.io>
Usage of word "add" was somehow misleading in those functions:
1. They don't really add (as in: append) anything. Result files are
created from scratch.
2. It wasn't clear what adding files means. It could be understood as
adding actual files somewhere (to existing archive?).
Also the word "add" was also a bit ambiguous.
Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
Rename function to more accurate and self-explanatory name:
1. Use "archive" in name as this functions creates tar archive
2. Avoid "conffiles" as this function may archive more than that
Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
This reverts commit 4fa9aaf0be.
That seemed like a good idea allowing us to include any runtime
generated file in archive. Unfortuantely it broke backups with files
from mounted directories.
When mounting overlay with / as lowerdir its mounts don't propagete in
the mountpoint. That resulted in empty directories:
/tmp/overlay.XXXXXX/backup/tmp/
/tmp/overlay.XXXXXX/backup/var/
/tmp/overlay.XXXXXX/backup/dev/
/tmp/overlay.XXXXXX/backup/proc/
etc.
As some platforms / users try to backup files like /var/dhcp.leases or
/boot/cmdline.txt it means we can't use that solution.
Link: http://lists.openwrt.org/pipermail/openwrt-devel/2024-February/042320.html
Link: https://lore.kernel.org/linux-fsdevel/67bb0571-a6e0-44ea-9ab6-91c267d0642f@gmail.com/T/#u
Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
This reverts commit bf304d10e9.
That uci-defaults script worked great but generating it required
mounting root dir as overlay lowerdir that needs to be reverted.
Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
Disabled services should be kept disabled after sysupgrade. This can be
easily handled using a proper uci-defaults script.
Extend sysupgrade to check for disabled services, generate uci-defaults
script disabling them and include it in backup.
Cc: Christian Marangi <ansuelsmth@gmail.com>
Cc: Jo-Philipp Wich <jo@mein.io>
Cc: Jonas Gorski <jonas.gorski@gmail.com>
Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
Acked-by: Jo-Philipp Wich <jo@mein.io>
Setting overlay while creating backup allows including extra files in
archive without actually writing them to flash. Right now this feature
is limited to /etc/backup/ directory and is used only for including
installed_packages.txt.
Extend this solution to make it more generic:
1. Always mount overlay while creating backup
2. Overlay whole / to don't limit it to /etc/backup/
This allows including any additional files in backups and adding more
sysupgrade features.
Cc: Luiz Angelo Daros de Luca <luizluca@gmail.com>
Cc: Christian Marangi <ansuelsmth@gmail.com>
Cc: Jo-Philipp Wich <jo@mein.io>
Cc: Jonas Gorski <jonas.gorski@gmail.com>
Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
When tar was failing, it was exiting immediately. Some files and the
tmpfs mount (-k) would remain breaking the next backup attempt.
Also remove redundant $? from exit builtin call as exit already returns
the last command exit code when called.
Signed-off-by: Luiz Angelo Daros de Luca <luizluca@gmail.com>
tar stderr was probably discarded only to remove this message:
tar: removing leading '/' from member names
However, together with that, any other error would also be discarded.
It is easier to fix that allowing the error message to be printed.
In sysupgrade, the backup file list only uses absolute paths. That way,
the solution is to remove the leading '/' from all files (sed) and chdir
to / (option -C /)
Signed-off-by: Luiz Angelo Daros de Luca <luizluca@gmail.com>
Downstream projects might re-generate device-specific configuration
based on OpenWrt's defaults on each upgrade, thus being unaffected by
forward- as well as backwards-breaking configuration.
Add a new sysupgrade parameter, which allows sysupgrades between minor
compat-versions. Upgrades will still fail upon mismatching major compat
versions.
Signed-off-by: David Bauer <mail@david-bauer.net>
The find command to retrieve files from /etc/sysupgrade.conf and
/lib/upgrade/keep.d/* is used twice in almost the same way.
Move it into a function to consolidate, enhance readability and make
future adjustments easier.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
Acked-by: Paul Spooren <mail@aparcar.org>
Reviewed-by: Philip Prindeville <philipp@redfish-solutions.com>
Commit "initramfs: switch to tmpfs to fix ujail" switched initramfs to
now use tmpfs, it causes $(rootfs_type) to now return tmpfs when
running initramfs image instead of being empty.
This broke initramfs detection which prevents config files from
being saved as it does not work from initramfs.
So, lets test for $(rootfs_type) returning "tmpfs" instead.
Fixes: 7fd3c68 ("initramfs: switch to tmpfs to fix ujail)
Signed-off-by: Robert Marko <robimarko@gmail.com>
With package "coreutil-sha256sum" installed "sysupgrade" fails to perform 'sha256sum -s' and instead returns 'invalid option -- 's''.
This is caused due to:
different syntax for a sha256sum status check ('sha256sum --status' with "coreutil-sha256sum")
'/usr/bin/sha256sum' being symlinked to '/usr/bin/gnu-sha256sum' (after installation of "coreutil-sha256sum")
"coreutil-sha256sum" package from the packages feed replaces the Busybox sha256sum
This patch restores for 'sysupgrade' the busybox call to its sha256sum applet.
Signed-off-by: Huangbin Zhan <zhanhb88@gmail.com>
With this change the well known jshn library will be used, to build the
json arguments for the ubus sysupgrade method. This is also used in all
other shell program that uses JSON. This commit unifies that.
Signed-off-by: Florian Eckert <fe@dev.tdt.de>
Now that $UPGRADE_BACKUP is set conditionally there is no need to check
the $UPGRADE_OPT_SAVE_CONFIG anymore. All conditions can be simplified.
Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
This explicitly tells procd what backup file should be used during
sysupgrade (if any). It's much more generic this way compared to the
magic /tmp/sysupgrade.tgz file that had to be created before a call.
Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
So far firmware validation result was binary limited: it was either
successful or not. That meant various limitations, e.g.:
1) Lack of proper feedback on validation problems
2) No way of marking firmware as totally broken (impossible to install)
This change introduces JSON for storing detailed validation info. It
provides a list of performed validation tests and their results. It
allows marking firmware as non-forceable (broken image that can't be
even forced to install).
Example:
{
"tests": {
"fwtool_signature": true,
"fwtool_device_match": true
},
"valid": true,
"forceable": true
}
Implementation is based on *internal* check_image bash script that:
1) Uses existing validation functions
2) Provides helpers for setting extra validation info
This allows e.g. platform_check_image() to call notify_check_broken()
when needed & prevent user from bricking a device.
Right now the new JSON info is used by /sbin/sysupgrade only. It still
doesn't make use of "forceable" as that is planned for later
development.
Further plans for this feature are:
1) Expose firmware validation using some new ubus method
2) Move validation step from /sbin/sysupgrade into "sysupgrade" ubus
method so:
a) It's possible to safely sysupgrade using ubus only
b) /sbin/sysupgrade can be more like just a CLI
Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
This explicitly lets stage2 know if partitions should be preserved. No
more "touch /tmp/sysupgrade.always.overwrite.bootdisk.partmap" hack.
Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
do_upgrade_stage2() isn't really any common code. It isn't used anywhere
except for /sbin/sysupgrade that passes it to the stage2.
Moving its code to separated file also simplifies COMMAND variable.
Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
Add support for passing additional parameters to mtd called during
sysupgrade. It will be required to toggle the "recovery moe" flag
supported by recent tp-link boards.
Signed-off-by: David Bauer <mail@david-bauer.net>
[split code from board support patch; add commit message]
Signed-off-by: Mathias Kresin <dev@kresin.me>
When '-k' is used, sysupgrade inserts into backup a new file
/etc/backup/installed_packages.txt which contains pkgname and origin (rom,
overlay, unknown) without touching rootfs.
It's mainly used to reinstall all extra packages:
# opkg update
# grep "\toverlay" /etc/backup/installed_packages.txt | cut -f1 | xargs -r opkg install
# rm /etc/backup/installed_packages.txt
Signed-off-by: Luiz Angelo Daros de Luca <luizluca@gmail.com>
Add sysupgrade '-o' option in order to include all overlay files in
backup, except for those that are from packages but including files
listed in conffiles, sysupgrade.conf or /lib/upgrade/keep.d.
With '-u' option, it will skip files equals to /rom and conffiles that
were not changed.
Signed-off-by: Luiz Angelo Daros de Luca <luizluca@gmail.com>
With '-u', for a file /aaa/bbb/ccc enlisted for backup,
it will only get into backup if /rom/aaa/bbb/ccc does not
exist or /aaa/bbb/ccc is different from /rom/aaa/bbb/ccc.
It also works with '-c', but only effective for files touched
but not modified.
Signed-off-by: Luiz Angelo Daros de Luca <luizluca@gmail.com>
Renamed add_uci_conffiles to add_conffiles as it includes
any conffiles listed, not only UCI ones.
Make do_save_conffiles arg mandatory
Allow other options after -l (like -c)
Do not use stdout for error messages (fixes backup to stdout)
Signed-off-by: Luiz Angelo Daros de Luca <luizluca@gmail.com>
Verify ucert signature chains in sysupgrade images in case ucert is
installed and $CHECK_IMAGE_SIGNARURE = 1.
Also make sure ucert host binary is present and generate a self-signed
ucert in case $TOPDIR/key-build.ucert is missing.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Sysupgrade shouldn't proceed, if the backup of the configuration
fails because tar (or gzip) exit with a non-zero code.
Signed-off-by: Andreas Ziegler <dev@andreas-ziegler.de>
this feature has never worked, the fw image name was not passed and the -t
parameter was missing in the tool invocation. drop the feature.
Signed-off-by: John Crispin <john@phrozen.org>
Many packages use the opkg conffiles field to list configuration files that
are to be retained on upgrades. Make this work on systems without opkg.
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
Commit 30f61a34b4 claimed to drop -d & -p
options. In reality only -d was dropped. Update command help text to
reflect that -d is no longer a supported option.
Fixes FS#1187
Signed-off-by: Kevin Darbyshire-Bryant <ldir@darbyshire-bryant.me.uk>
Since /overlay/upper appeared, -b ignored -c silently (cause it was
still checking for /overlay/etc). Now, if /overlay/upper is absent,
sysupgrade -c will fail and exit verbosely.
Fix -l to consider -c (it never did).
Clean up to always use /overlay/upper/xxx instead of still checking
for /overlay/xxx.
Signed-off-by: Luiz Angelo Daros de Luca <luizluca@gmail.com>
Depending on busybox applet selection, paths of basic utiilties may differ,
and may not work as symlinks to busybox. Simply using whatever binary is
found in PATH and detecting symlinks automatically is more robust and
easier to maintain.
The list of binaries is also slightly cleaned up and duplicates are
removed.
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
Some functions only used by stage2 are moved there from common.sh.
One piece that could still use more cleanup is platform_pre_upgrade: many
targets reference files from there are aren't available in the ramfs, so
we need to evaluate it before the switch; conversely, flash writes happen
in that function on some targets. Targets that do the latter should be
fixed eventually to use platform_do_upgrade for that purpose.
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
Support for the -d and -p options is dropped; it may be added again at some
point by adding these flags to the ubus sysupgrade call.
A downside of this is that we get a lot less information about the progress
of the upgrade: as soon as the actual upgrade starts, all shell sessions
are killed to allow unmounting the root filesystem.
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
This reverts commit e96a9a9af8.
The change breaks sysupgrade through LuCI and two-stage sysupgrade on
NAND targets. There is also a mismatch of file paths in lock and unlock
operations.
This commit was apparently neither properly tested, nor reviewed, so
drop it for now.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
