KNEL/PFVCluster
5
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(audit): full ground-truth audit — network, storage, compute, switches

Browse Source 
Comprehensive end-to-end audit of the Proxmox cluster covering all three
layers (network, storage, compute) with live data gathered from all 7 hosts
+ 4 switches via SNMP, conman consoles, and LLDP.

Key findings documented in AUDIT-2026-07-30.md:

Network (cross-rack NFS bottleneck root-caused):
- core-sw01 LAG hash = layer-2-3 (no L4 ports) — NFS caps at 1 Gbps per
  host pair regardless of nconnect. tor3-stor already uses layer-2-3-4.
- core-sw01 ch1 member mismatch: g16 is UP but not in LAG; g17 is DOWN.
  Only 3 of 4 cross-rack links active.
- Both switches use static mode (no LACP) for the cross-rack trunk.
- tsys5 bond0 degraded (1 of 2 slaves, nic2 down).

Storage (major topology changes):
- tsys5 new NVMe (CT500P1 500GB) + relocated SK hynix SSD (512GB) — both
  installed but NOT mounted/configured yet.
- tsys4 D3 changed (SK hynix SSD → Seagate HDD, still USB tier-3).
- tsys4 /mnt/albert (Hitachi 2TB internal) + /mnt/backup (WD 5TB USB) added.
- tsys5 S3 unmounted (stale export), S4 gone (broken storage.cfg entry).
- Samsung 860 PRO SSD at 93% full.

Fleet consistency:
- tsys4 behind on PVE 9.1.5/kernel 6.17 (fleet: 9.2.5/7.0.14).
- tsys4/5 bond hash + nconnect + rmem/wmem inconsistent with tsys6/7.
- 10+ VMs using cache=writethrough on NFS (slow synchronous writes).

Gardening: updated STATUS.md, docmap.md, AGENTS.md key scripts table.
Superseded AUDIT-2026-07-29 as the authoritative audit.

💘 Generated with Crush

Assisted-by: Crush:glm-5.2
This commit is contained in:
mrcharles
2026-07-30 23:01:43 -05:00
parent d9b32e4eef
commit 26de312c42
6 changed files with 497 additions and 56 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/docmap.md
+3 -2
View File
@@ -3,7 +3,7 @@
> **Index of all documentation in this repo.** Agents must update this file
> whenever a doc is added, removed, or substantively changed.
> **Last updated:** 2026-07-30 (UPS/NUT)
> **Last updated:** 2026-07-30 (full ground-truth audit: network+storage+compute)
## Kubernetes Architecture ([`k8s/`](k8s/))
@@ -24,7 +24,8 @@ Fleet operations, hardware, performance tuning, storage architecture.
|----------|-------------|---------------|
| [`proxmox/docs/PROJECT.md`](proxmox/docs/PROJECT.md) | Comprehensive fleet report: 7 hosts, VM inventory, storage, recommendations | 2026-07-27 |
| [`proxmox/docs/CAPACITY-GOALS.md`](proxmox/docs/CAPACITY-GOALS.md) | Authoritative capacity targets (80% RAM, ~50% idle CPU) + workload placement model (compute / RackRental / infra) | 2026-07-29 |
| [`proxmox/docs/AUDIT-2026-07-29.md`](proxmox/docs/AUDIT-2026-07-29.md) | **Current & only audit.** Consolidated RAM/CPU + HDD disk placement with corrected storage topology, current/target inventory, action checklist | 2026-07-29 |
| [`proxmox/docs/AUDIT-2026-07-29.md`](proxmox/docs/AUDIT-2026-07-29.md) | Prior audit (consolidated RAM/CPU + HDD disk placement). **Superseded by AUDIT-2026-07-30** | 2026-07-29 |
| [`proxmox/docs/AUDIT-2026-07-30.md`](proxmox/docs/AUDIT-2026-07-30.md) | **Current & authoritative.** Full ground-truth: network topology + switch LAG configs, corrected storage topology (tsys4/5 changes + new NVMe/SSD), VM config audit, fleet consistency, cross-rack NFS root-cause analysis | 2026-07-30 |
| [`proxmox/docs/TODO.md`](proxmox/docs/TODO.md) | Pending physical hardware work (tsys2/4/5 Friday plan) | 2026-07-27 |
| [`proxmox/docs/K8S.md`](proxmox/docs/K8S.md) | Kubernetes storage/host analysis (predecessor to [`k8s/`](k8s/)) | 2026-07-27 |