The login banner on subopi/pfvsvrpi systems was bleeding into the sudo
check output, making SUDOOK results look garbled and broken. Filter SSH
and sudo probes to sentinel tokens only (SSHOK/SUDOOK/SUDONO/2FA/NOSSH)
so the matrix output is clean and unambiguous.
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
Previous sessions used `qm guest exec` to back-door SSH keys into ~30 VMs,
bypassing sshd's audit trail in an ITAR/CMMC environment. Wire the ban deep
so it cannot recur:
- tests/remote.sh: remove the vm-guest mode + qm-guest-exec path entirely
- scripts/check-rules.sh: rule #11 fails on any `qm guest exec` / `vm-guest`
pattern in code (scans .sh/.bash/.py; docs may describe the ban freely)
- AGENTS.md: codify "Access-channel policy: SSH only" as non-negotiable;
add "Questions" rule banning harness question tools (use questions-v1.md)
- tests/vm-validation.sh: drop guest-agent key re-injection; SSH-only
- proxmox/perf/scripts/perf-matrix.sh + deploy-tuned-guests.sh: convert
guest-agent execution to SSH (vmroot) now that VMs have key + sudo
- bootstrap-all.sh: re-target the 8 remaining locked-out systems with
correct users/methods; print a console one-liner for publickey-only Pis
Guest-agent remains installable/checkable for Proxmox state visibility —
never as an execution or key-delivery path.
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
Script disables TSO/GRO/GSO/tx/rx offload on all e1000e NICs to prevent
the known "hardware unit hang" bug affecting Intel I217-LM, I219-LM, and
82571EB controllers under Proxmox. Applies live + installs systemd
service for persistence.
Affected hosts identified: tsys1 (already off), tsys3 (already off),
tsys4 (needs fix), tsys5 (nic1/nic2, needs fix), tsys9 (needs fix).
tsys6/7 use Broadcom (bnx2/tg3), not affected.
Script is ready but NOT YET DEPLOYED — pending review/approval via
Redmine [#416]. Initial attempt to deploy did not execute properly.
Based on archive/provisioning/Dell/Server/fixeth.sh.
Refs: https://forum.proxmox.com/threads/e1000-driver-hang.58284/💘 Generated with Crush
Assisted-by: Crush:glm-5.2
perf-matrix.sh provides on-demand any-to-any performance testing:
- datanet: host-to-host bandwidth matrix over VLAN 1000
- guests: guest-to-guest (k8s/ultix VMs over datanet)
- storage: NFS read/write I/O per host
iperf3 installed on all 7 hosts (systemd service iperf3-server, enabled
but not auto-started) and all production VMs with guest-agent.
Also: tsys3 datanet persistence fix (USB NIC late-enumerate systemd
service), 4 missing reverse DNS zones created, netbird/ultix-mini/
ultix-sidecar/ultix-highside DNS records added (DNS now fully
consistent across all Tailscale nodes). [#413]
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
Perf testing results (#413):
- Host-to-host datanet: all 7 hosts at ~900-940 Mbps (1Gbps line rate)
- Guest-to-guest: all k8s nodes at ~920-1040 Mbps over VLAN 1000
- Storage I/O: NFS writes 27-65 MB/s (spinning disk), reads cached
Fixed tsys3 datanet: physical NIC enx8cae4ccda774 was DOWN and not
enslaved to the datanet bridge (config was correct but runtime state
was wrong). Enslaved manually, verified connectivity.
Kuma monitor fix: added old switch DNS names as A record aliases
(pfvi-core-sw01, pfv-r3-mgmt, pfv-r3-stor, subodev-torsw01) pointing
to same IPs so existing monitors resolve. Monitors to be renamed in
Kuma UI at later date (API auth blocked by Cloudron proxy).
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
mrcharles