ultix perf/ops workbench: complete 2026-08-31 optimization pass for VM 5111

Everything for the ultix-streaming (VM 5111, pfv-tsys5) performance pass:
full report + host audit results, staged/gated configs, guest prep + host
one-shot + post-reboot-fix + netcheck lifecycle scripts, grow-root manual
runbook, rolling tracking HUD, questions v1, and the gateway boot-race
hardening units. Applied and verified live 2026-08-31; open work is
tracked in Redmine project 55 as #601-#607. [#602]

💘 Generated with Crush

Assisted-by: Crush:glm-5.2
This commit is contained in:
2026-08-31 16:46:13 -05:00
commit cba2a64d67
43 changed files with 3137 additions and 0 deletions
+38
View File
@@ -0,0 +1,38 @@
# NEXT — perf-opt closeout COMPLETE (2026-08-31 16:4x)
## Final state — nothing owed
- All guest tuning live and verified (see 16:04-16:15 pass below).
- Host flags live: ssd=1/discard (ROTA=0), iothread, onboot=1, startup
order=10, cpuunits 9000/50, ide2 gone.
- Net multiqueue: queues=4/2 now IN live qm config (staged 16:40 by crush
via ssh; activates at next VM start = Friday). Guest oneshot unit
ukrrs-net-multiqueue auto-runs ethtool -L on that boot.
- Gateway boot-race + live-restore endpoint loss: permanently fixed by
ukrrs-gateway-ensure unit (enabled, both lanes).
- Root cause of the 15:59 no-op: the script copy executed then predates the
queues= fix (silent same-value rewrite); current config verified correct.
## Verified live after the 16:01 bounce (all read-only from the guest)
- Disks: ROTA=0, discard 4K/1G on sda+sdb+sdc → ssd=1/discard=on ACTIVE ✅
- Kernel: bbr+fq, 16M socket buffers, dirty_bytes 1G/256M, min_free 384M,
aio 1M, inotify 512/1M, port range 10240-65535, slow_start_after_idle=0,
tcp_tw_reuse=1, THP=madvise, DefaultLimitNOFILE 65536 ✅
- Mounts /,/data1,/data2 = noatime; swap sda5 prio -2 unused ✅
- Units: all ukrrs slices + day/night + psi + builder-prune + thp units
installed and enabled; PSI textfile writing (16:06) ✅
- Docker: daemon.json fully live (live-restore, log caps 20m×3, 172.16/12
pools, metrics :9323); 21 containers up ✅
- Desktop: sddm autologin worked (Relogin=true), session alive on seat0/tty2,
greeter burn GONE ✅
- Gateway prod was DOWN at boot (bind race + live-restore endpoint loss) →
recovered via compose --force-recreate; /status = mode normal, serving ✅
## Nothing owed. Optional sanity check any time (crush can ssh now):
~/optimize/4-host-netcheck.sh
## Noted, no action taken
- mopac-demo/mcli fake containers: restart=no, exited at the first bounce,
left down (test stubs; your call).
- Prometheus/harness daemons not deployed here yet (that is OPT-9's wiring).
- Friday unchanged: CPU/RAM swap (plan NFS outage for -02), 2nd USB3 card,
q35 + 20 vCPU/128G reshape, GPUs. Enable the ensure unit BEFORE Friday.