openwrt/tools
Michał Kępień a60721f2ed mikrotik: switch to Yafut for building MikroTik NOR images
The Yafut tool now has limited capabilities for working on filesystem
images stored in regular files.  This enables preparing Yaffs2 images
for devices with NOR flash using upstream Yaffs2 filesystem code instead
of the custom kernel2minor tool.

Since minimizing the size of the resulting filesystem image size is
important and upstream Yaffs2 code requires two allocator reserve blocks
to be available when writing a file to the filesystem, a trick is
employed while preparing an OpenWRT image: the blank filesystem image
that Yafut operates on initially contains two extra erase blocks that
are chopped off after the kernel file is written.  This is safe to do
because Yaffs2 has a true log structure and therefore only ever writes
sequentially (and the size of the kernel file is known beforehand).
While the two extra erase blocks are necessary for writes, Yaffs2 code
seems to be perfectly capable of reading back files from a "truncated"
filesystem that does not contain these extra erase blocks.

In terms of image size, this new approach is only marginally worse than
the current kernel2minor-based one: specifically, upstream Yaffs2 code
needs to write three object headers (each of which takes up an entire
data chunk) when the kernel file is written to the filesystem:

  - an object header for the kernel file when it is created,

  - an object header for the root directory when the kernel file is
    created,

  - an updated object header for the kernel file when the latter is
    fully written (so that its new size can be recorded).

kernel2minor only writes two of these headers, which is the absolute
minimum required for reading the file back.  This means that the
Yafut-based approach causes firmware images to be at most one erase
block (64 kB) larger than those created using kernel2minor, but only in
the very unfortunate scenario where the size of the kernel file is
really close to a multiple of the erase block size.

The rest of the calculations performed when the empty filesystem image
is first prepared stems from the Yaffs2 layout used by MikroTik NOR
devices: each 65,536-byte erase block contains 63 chunks, each of which
consists of 1024 bytes of data followed by 16-byte Yaffs tags without
ECC data; each such group of 63 chunks is then followed by 16 bytes of
padding, which translates to "-C 1040 -B 64k -E" in the Yafut
invocation.  Yaffs2 checkpoints and summaries are disabled (using
Yafut's -P and -S switches, respectively) as they are merely performance
optimizations that require extra storage space.  The -L and -M switches
are used to force little-endian or big-endian byte order (respectively)
in the resulting filesystem image, no matter what byte order the build
host uses.  The tr invocation is used to ensure that the filesystem
image is initialized with 0xFF bytes (which are an indicator of unused
space for Yaffs2 code).

Signed-off-by: Michał Kępień <openwrt@kempniu.pl>
Link: https://github.com/openwrt/openwrt/pull/13453
Signed-off-by: Robert Marko <robimarko@gmail.com>
2024-06-05 17:03:24 +02:00
..
7z tools: assign PKG_CPE_ID 2023-09-19 20:21:33 +02:00
autoconf tools: prefer gz or bz2 tarballs 2024-04-06 11:24:18 +02:00
autoconf-archive tools/autoconf-archive: bump to 2023.02.20 2023-03-20 23:43:28 +01:00
automake tools: prefer gz or bz2 tarballs 2024-04-06 11:24:18 +02:00
b43-tools treewide: update PKG_MIRROR_HASH to zst 2024-04-06 11:24:18 +02:00
bash tools/bash: update to 5.2.21 2023-12-08 16:28:40 +01:00
bc tools/bc: update to 1.07.1 2022-10-22 22:50:37 +02:00
bison tools/bison: hardcode path to m4 with STAGING_DIR_HOST 2024-04-25 21:33:51 +02:00
bzip2 tools/bzip2: switch to cmake 2024-04-03 18:56:58 +02:00
cbootimage treewide: update PKG_MIRROR_HASH to zst 2024-04-06 11:24:18 +02:00
cbootimage-configs treewide: update PKG_MIRROR_HASH to zst 2024-04-06 11:24:18 +02:00
ccache tools: prefer gz or bz2 tarballs 2024-04-06 11:24:18 +02:00
cmake tools/cmake: disable xcode generator 2024-03-29 15:40:25 +01:00
coreutils tools/coreutils: update to 9.5 2024-04-25 21:33:51 +02:00
cpio tools/cpio: update to 2.15 2024-01-25 16:06:49 +01:00
dosfstools dosfstools: switch to AC_CHECK_LIB 2023-01-16 18:29:04 +01:00
dwarves tools/dwarves: add myself as PKG_MAINTAINER 2024-04-15 12:43:37 +02:00
e2fsprogs tools: prefer gz or bz2 tarballs 2024-04-06 11:24:18 +02:00
elftosb tools: refresh all patches 2024-05-23 11:49:03 +02:00
elfutils tools/elfutils: pass -O2 in HOST_CXXFLAGS 2024-05-31 16:01:43 +02:00
expat tools/expat: fix PKG_CPE_ID 2024-04-27 23:37:57 +02:00
fakeroot Revert "tools/fakeroot: update to 1.30.1" 2023-02-12 01:02:22 +01:00
findutils tools/findutils: define list of installed programs 2023-06-05 08:31:54 +02:00
firmware-utils treewide: update PKG_MIRROR_HASH to zst 2024-04-06 11:24:18 +02:00
flex tools/flex: fix PKG_CPE_ID 2024-04-27 12:08:11 +02:00
flock tools: add Host/Uninstall where possible 2022-10-20 00:33:22 +02:00
genext2fs tools/genext2fs: update to 1.5.0 2022-10-22 22:53:24 +02:00
gengetopt gengetopt: backport patch fixing support for c++17 2024-05-01 15:45:25 +02:00
gmp tools: prefer gz or bz2 tarballs 2024-04-06 11:24:18 +02:00
gnulib tools/gnulib: unmangle fts header on macOS 2024-05-31 16:01:43 +02:00
include tools: add more linux include files to fix x86 build on macos 2024-05-07 12:50:27 +02:00
isl tools: prefer gz or bz2 tarballs 2024-04-06 11:24:18 +02:00
kernel2minor treewide: update PKG_MIRROR_HASH to zst 2024-04-06 11:24:18 +02:00
libdeflate tools: libdeflate: backport fix for AVX-VNNI 2024-04-15 13:52:37 +02:00
liblzo tools: refresh all patches 2024-05-23 11:49:03 +02:00
libressl tools/libressl: update to version 3.7.3 2023-09-22 14:11:22 +02:00
libtool tools: prefer gz or bz2 tarballs 2024-04-06 11:24:18 +02:00
llvm-bpf tools: assign PKG_CPE_ID 2023-09-19 20:21:33 +02:00
lz4 tools/lz4: remove patches 2024-05-23 11:49:02 +02:00
lzma tools: refresh all patches 2024-05-23 11:49:03 +02:00
lzma-old treewide: unify OpenWrt hosted source via @OPENWRT 2021-02-05 12:00:24 -10:00
lzop tools: refresh all patches 2024-05-23 11:49:03 +02:00
m4 tools: prefer gz or bz2 tarballs 2024-04-06 11:24:18 +02:00
make-ext4fs treewide: update PKG_MIRROR_HASH to zst 2024-04-06 11:24:18 +02:00
meson tools/meson: update to 1.3.2 2024-03-29 23:42:22 +01:00
missing-macros tools/missing-macros: install files from HOST_BUILD_DIR instead of src 2024-05-22 11:53:52 +02:00
mkimage tools/mkimage: update to v2024.04 2024-05-29 10:43:50 +02:00
mklibs tools/mklibs: update to 0.1.45 2022-03-01 00:08:08 +01:00
mold tools: mold: update to 2.31.0 2024-05-09 10:45:29 +02:00
mpc tools/mpc: use STAGING_DIR_HOST instead of hardcoding default 2023-01-09 21:33:19 +01:00
mpfr tools: prefer gz or bz2 tarballs 2024-04-06 11:24:18 +02:00
mtd-utils tools/mtd-utils: update to 2.1.6 2023-10-14 08:03:06 +02:00
mtools tools/mtools: update to 4.0.43 2023-04-08 15:51:32 +02:00
ninja tools/ninja: update to 1.11.1 2023-01-24 23:13:25 +01:00
padjffs2 tools/padjffs2: use Host/Prepare/Default instead of raw commands 2024-05-22 11:56:45 +02:00
patch tools: refresh all patches 2024-05-23 11:49:03 +02:00
patch-image rb532: drop target 2020-09-02 16:29:22 +02:00
patchelf tools/patchelf: update to 0.18.0 2023-04-29 17:37:45 +02:00
pkgconf tools: pkgconf: provide prefix for host and hostpkg stagind directory 2024-05-05 17:22:26 +02:00
quilt tools/quilt: update to 0.67 2022-02-25 14:12:39 +01:00
sdimage treewide: update PKG_MIRROR_HASH to zst 2024-04-06 11:24:18 +02:00
sed tools: prefer gz or bz2 tarballs 2024-04-06 11:24:18 +02:00
sparse tools: prefer gz or bz2 tarballs 2024-04-06 11:24:18 +02:00
squashfs3-lzma tools: squashfs3-lzma: generate reproducible output 2023-05-12 02:27:17 +02:00
squashfs4 treewide: update PKG_MIRROR_HASH to zst 2024-04-06 11:24:18 +02:00
sstrip tools/sstrip: update to 3.2 2021-09-20 15:21:17 +02:00
tar tools: prefer gz or bz2 tarballs 2024-04-06 11:24:18 +02:00
util-linux tools: prefer gz or bz2 tarballs 2024-04-06 11:24:18 +02:00
xz Revert "tools/xz: update to 5.6.1" (CVE-2024-3094) 2024-03-29 16:59:56 +00:00
yafut tools: add Yafut to enable creating Yaffs filesystem images 2024-06-05 17:03:24 +02:00
zip tools: assign PKG_CPE_ID 2023-09-19 20:21:33 +02:00
zlib tools/zlib: fix PKG_CPE_ID 2024-04-27 12:07:37 +02:00
zstd tools/zstd: enable position independent code compilation 2024-04-24 12:16:48 +02:00
Makefile mikrotik: switch to Yafut for building MikroTik NOR images 2024-06-05 17:03:24 +02:00