Leon M. Busch-George 98d325aaf8 ipq40xx: wpj428: panic on squashfs error to work around boot limbo
Apparently, a few ipq40xx devices have sporadic problems when reading the
flash over SPI. When that happens, the result of the faulty SPI read is
cached and it isn't re-attempted. Depending on when it happens, the router
either panics and reboots or is left in a partially broken state (an
application wont start).
The data on the flash is alright.

This wasn't the case with Openwrt with Linux < 5.x but I wasn't able to
work out which software change was responsible.

Github user karlpip created a patch for testing that disabled the cache
entirely and added logs. Typically, only one or two SPI operations fail at
a time:

  [689200.631152] spi-nor spi0.0: SPI transfer failed: -110
  [689200.631280] spi_master spi0: failed to transfer one message from queue
  [689200.635369] jffs2: Write of 68 bytes at 0x00ffccf4 failed. returned -110, retlen 0
  [689200.642014] jffs2: Not marking the space at 0x00ffccf4 as dirty because the flash driver returned retlen zero

Because reads aren't re-attempted, squashfs can't recover:

  [3171844.279235] SQUASHFS error: Failed to read block 0x2bb912: -5
  [3171844.279284] SQUASHFS error: Unable to read fragment cache entry [2bb912]
  [3171844.283980] SQUASHFS error: Unable to read page, block 2bb912, size 14e6c
  [3171844.291650] SQUASHFS error: Unable to read fragment cache entry [2bb912]
  [3171844.297831] SQUASHFS error: Unable to read page, block 2bb912, size 14e6c

I assume there to be some kind of underlying electrical problem because,
in my experience, this happens a lot more when PoE is used.

NoTengoBattery has made an in-depth investigation:
https://forum.openwrt.org/t/patch-squashfs-data-probably-corrupt/70480

.. and created a patch that evicts the page cache and retries reading:
https://github.com/NoTengoBattery/openwrt/blob/linksys-ea6350v3-mastertrack/target/linux/ipq40xx/patches-5.4/9996-fs_squashfs_improve_squashfs_error_resistance.patch

The patch also works well with the WPJ428 but NoTengoBattery didn't try to
upstream it ("This is not the solution that should be used").

In 2020, I tried and failed to create a working patch that prevents faulty pages to
be cached in the first place. Because I needed a solution, I backported
  "squashfs: add option to panic on errors " (10dde05b89980ef)
which has since become available in Openwrt.

The 'error=panic' option has been tested on a fleet of multiple hundred
WPJ428s over multiple years. Without this patch, devices regularly went
into 'limbo' on reboot or update and required a manual reboot.
Devices with this patch don't. I was initially concerned that the kernel
panic would leave devices with a real corrupted data but I haven't seen a
case of actual corruption since (outside of people turning off the power
during upgrades).

The WPJ428 is the only device I tested this patch on - others might also
benefit.

Reviewed-by: Robert Marko <robimarko@gmail.com>
Signed-off-by: Leon M. Busch-George <leon@georgemail.eu>
2023-09-24 18:55:35 +02:00
2023-09-24 12:45:34 +02:00
2023-09-19 20:22:59 +02:00
2023-09-24 12:49:50 +02:00
2021-02-05 14:54:47 +01:00
2023-06-05 08:31:47 +02:00
2023-05-22 13:23:35 +02:00

OpenWrt logo

OpenWrt Project is a Linux operating system targeting embedded devices. Instead of trying to create a single, static firmware, OpenWrt provides a fully writable filesystem with package management. This frees you from the application selection and configuration provided by the vendor and allows you to customize the device through the use of packages to suit any application. For developers, OpenWrt is the framework to build an application without having to build a complete firmware around it; for users this means the ability for full customization, to use the device in ways never envisioned.

Sunshine!

Download

Built firmware images are available for many architectures and come with a package selection to be used as WiFi home router. To quickly find a factory image usable to migrate from a vendor stock firmware to OpenWrt, try the Firmware Selector.

If your device is supported, please follow the Info link to see install instructions or consult the support resources listed below.

An advanced user may require additional or specific package. (Toolchain, SDK, ...) For everything else than simple firmware download, try the wiki download page:

Development

To build your own firmware you need a GNU/Linux, BSD or MacOSX system (case sensitive filesystem required). Cygwin is unsupported because of the lack of a case sensitive file system.

Requirements

You need the following tools to compile OpenWrt, the package names vary between distributions. A complete list with distribution specific packages is found in the Build System Setup documentation.

binutils bzip2 diff find flex gawk gcc-6+ getopt grep install libc-dev libz-dev
make4.1+ perl python3.7+ rsync subversion unzip which

Quickstart

  1. Run ./scripts/feeds update -a to obtain all the latest package definitions defined in feeds.conf / feeds.conf.default

  2. Run ./scripts/feeds install -a to install symlinks for all obtained packages into package/feeds/

  3. Run make menuconfig to select your preferred configuration for the toolchain, target system & firmware packages.

  4. Run make to build your firmware. This will download all sources, build the cross-compile toolchain and then cross-compile the GNU/Linux kernel & all chosen applications for your target system.

The main repository uses multiple sub-repositories to manage packages of different categories. All packages are installed via the OpenWrt package manager called opkg. If you're looking to develop the web interface or port packages to OpenWrt, please find the fitting repository below.

  • LuCI Web Interface: Modern and modular interface to control the device via a web browser.

  • OpenWrt Packages: Community repository of ported packages.

  • OpenWrt Routing: Packages specifically focused on (mesh) routing.

  • OpenWrt Video: Packages specifically focused on display servers and clients (Xorg and Wayland).

Support Information

For a list of supported devices see the OpenWrt Hardware Database

Documentation

Support Community

  • Forum: For usage, projects, discussions and hardware advise.
  • Support Chat: Channel #openwrt on oftc.net.

Developer Community

License

OpenWrt is licensed under GPL-2.0

Description
This repository is a mirror of https://git.openwrt.org/openwrt/openwrt.git It is for reference only and is not active for check-ins. We will continue to accept Pull Requests here. They will be merged via staging trees then into openwrt.git.
Readme 906 MiB
Languages
C 61.7%
Makefile 18.7%
Shell 6.7%
Roff 6.6%
Perl 2.4%
Other 3.7%