diff --git a/perf/deploy-tuning.sh b/perf/deploy-tuning.sh
index b1f82cb..3f21ed4 100755
--- a/perf/deploy-tuning.sh
+++ b/perf/deploy-tuning.sh
@@ -4,7 +4,7 @@
# Default mode is dry-run. Pass --apply to commit. Pass --no-nfs to skip NFS section.
set -uo pipefail
-SCRIPT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)/scripts/apply-tunings.sh"
+SCRIPT="$(cd "$(dirname "${BASH_SOURCE[0]}")/scripts" && pwd)/apply-tunings.sh"
SSH_OPTS=(-o BatchMode=yes -o ConnectTimeout=8 -o ServerAliveInterval=10 -o StrictHostKeyChecking=accept-new)
MODE=""
EXTRA_FLAGS=""
diff --git a/proxmox/docs/HOST-REFERENCE.md b/proxmox/docs/HOST-REFERENCE.md
index d2c525b..8ef23a2 100644
--- a/proxmox/docs/HOST-REFERENCE.md
+++ b/proxmox/docs/HOST-REFERENCE.md
@@ -13,17 +13,17 @@
```mermaid
graph TB
- subgraph "RACK 5 — Core / Data / Infra"
- T1["tsys1 — Infra
i7-4770 · 32GB · 9 VMs
57% RAM"]
- T3["tsys3 — Compute
Xeon E3-1535M v5 · 32GB · 1 VM
10% RAM"]
- T4["tsys4 — Storage+PBS
Xeon E3-1246 v3 · 16GB · 1 VM
15% RAM"]
- T5["tsys5 — Storage+Sandbox
Xeon E5620 · 96GB · 4 VMs
15% RAM"]
- T9["tsys9 — Infra+Compute
i5-10500 · 24GB · 6 VMs
69% RAM"]
+ subgraph "RACK 5 -- Core / Data / Infra"
+ T1["tsys1 -- Infra
i7-4770 | 32GB | 9 VMs
57% RAM"]
+ T3["tsys3 -- Compute
Xeon E3-1535M v5 | 32GB | 1 VM
10% RAM"]
+ T4["tsys4 -- Storage+PBS
Xeon E3-1246 v3 | 16GB | 1 VM
15% RAM"]
+ T5["tsys5 -- Storage+Sandbox
Xeon E5620 | 96GB | 4 VMs
15% RAM"]
+ T9["tsys9 -- Infra+Compute
i5-10500 | 24GB | 6 VMs
69% RAM"]
end
- subgraph "RACK 3 — Compute / RackRental"
- T6["tsys6 — RackRental
2× Xeon E5530 · 126GB · 2 VMs
12% RAM"]
- T7["tsys7 — RackRental
2× Xeon E5-2630 v2 · 189GB · 5 VMs
7% RAM"]
+ subgraph "RACK 3 -- Compute / RackRental"
+ T6["tsys6 -- RackRental
2x Xeon E5530 | 126GB | 2 VMs
12% RAM"]
+ T7["tsys7 -- RackRental
2x Xeon E5-2630 v2 | 189GB | 5 VMs
7% RAM"]
end
```
@@ -33,15 +33,15 @@ graph TB
| tsys3 | Xeon E3-1535M v5 Skylake (2015) | 8 | 32 GB | 9.2.5 | 7.0.14 | Compute | 1 | 10% |
| tsys4 | Xeon E3-1246 v3 Haswell (2013) | 8 | 16 GB | **9.1.5** | **6.17.9** | Storage + PBS | 1 | 15% |
| tsys5 | Xeon E5620 Westmere (2010) | 8 | 96 GB | 9.2.5 | 7.0.14 | Storage + Sandbox | 4 | 15% |
-| tsys6 | 2× Xeon E5530 Nehalem (2009) | 16 | 126 GB | 9.2.5 | 7.0.14 | RackRental | 2 | 12% |
-| tsys7 | 2× Xeon E5-2630 v2 Ivy Bridge (2013) | 24 | 189 GB | 9.2.5 | 7.0.14 | RackRental | 5 | 7% |
+| tsys6 | 2x Xeon E5530 Nehalem (2009) | 16 | 126 GB | 9.2.5 | 7.0.14 | RackRental | 2 | 12% |
+| tsys7 | 2x Xeon E5-2630 v2 Ivy Bridge (2013) | 24 | 189 GB | 9.2.5 | 7.0.14 | RackRental | 5 | 7% |
| tsys9 | i5-10500 Comet Lake (2020) | 12 | 24 GB | 9.2.5 | 7.0.14 | Infra + Compute | 6 | 69% |
**Retired:** tsys2 (Win10, pending rebuild), tsys8 (permanently retired)
---
-## pfv-tsys1 — Infrastructure host
+## pfv-tsys1 -- Infrastructure host
| Attribute | Value |
|-----------|-------|
@@ -60,7 +60,7 @@ graph TB
---
-## pfv-tsys3 — Compute host
+## pfv-tsys3 -- Compute host
| Attribute | Value |
|-----------|-------|
@@ -70,14 +70,14 @@ graph TB
| **mgmt IP** | 192.168.2.5 (vmbr0, DHCP) |
| **Storage IP** | 10.100.100.3 (datanet via enx USB dongle, no bond) |
| **NFS mounts** | D2, D5, S2, S4, T5-SSD (all nconnect=4) |
-| **Local storage** | local-lvm (Samsung PM961 NVMe, 349 GB, 0% used) — **best existing local storage** |
+| **Local storage** | local-lvm (Samsung PM961 NVMe, 349 GB, 0% used) -- **best existing local storage** |
| **Tuning** | governor=performance, swappiness=10, bbr, rmem/wmem=128MB, tuned=virtual-host |
| **Bond** | None (single USB NIC for storage) |
| **Running VMs** | 313 |
---
-## pfv-tsys4 — Storage server + PBS
+## pfv-tsys4 -- Storage server + PBS
| Attribute | Value |
|-----------|-------|
@@ -99,7 +99,7 @@ graph TB
---
-## pfv-tsys5 — Storage server + sandboxes
+## pfv-tsys5 -- Storage server + sandboxes
| Attribute | Value |
|-----------|-------|
@@ -107,7 +107,7 @@ graph TB
| **Hardware** | Dell T7500, Xeon E5620 Westmere (2010) **single CPU** (CPU2 socket empty), 8 threads, 96 GB RAM |
| **PVE** | 9.2.5 / kernel 7.0.14-6-pve |
| **mgmt IP** | 192.168.3.179 (vmbr0, DHCP) |
-| **Storage IP** | 10.100.100.5 (storagenet via bond0, **1 of 2 slaves active** — nic2 DOWN) |
+| **Storage IP** | 10.100.100.5 (storagenet via bond0, **1 of 2 slaves active** -- nic2 DOWN) |
| **NFS mounts** | D2, D3, D5 from tsys4 (**missing nconnect**) |
| **NFS exports** | S1 (IronWolf 1TB), S2 (IronWolf 1TB), S3 (stale), SSD (Samsung 860 PRO) |
| **Local storage** | local-lvm (Hitachi 2TB), local-nonprod (=S1), SSD dir (Samsung 860 PRO 256GB) |
@@ -119,36 +119,36 @@ graph TB
---
-## pfv-tsys6 — RackRental host
+## pfv-tsys6 -- RackRental host
| Attribute | Value |
|-----------|-------|
| **Role** | RackRental: k8s wnode-tsys6 (64 GB RAM) + AWX |
-| **Hardware** | 2× Xeon E5530 Nehalem (2009), 16 threads, 126 GB RAM |
+| **Hardware** | 2x Xeon E5530 Nehalem (2009), 16 threads, 126 GB RAM |
| **PVE** | 9.2.5 / kernel 7.0.14-6-pve |
| **mgmt IP** | 192.168.3.169 (vmbr0, DHCP) |
| **Tailscale** | 100.73.35.111 |
| **Storage IP** | 10.100.100.6 (storagenet via bond0, **2 of 2 slaves active**, 2G LACP) |
| **NFS mounts** | D1, D2, D3, D5, S1, S2, S3, S4, T5-SSD (9 mounts, all nconnect=4) |
-| **Local storage** | local-lvm (WD My Passport USB 2.0 HDD, 1.7 TB, **30 MB/s — do not use for VMs**) |
+| **Local storage** | local-lvm (WD My Passport USB 2.0 HDD, 1.7 TB, **30 MB/s -- do not use for VMs**) |
| **Tuning** | governor=performance, swappiness=10, bbr, rmem/wmem=128MB, tuned=virtual-host |
| **Bond** | bond0 (2 of 2 slaves), **hash=layer3+4** (correct) |
| **Running VMs** | 100 (wnode-tsys6, 64GB), 600 (tsys-awx) |
-| **Notes** | Best storage bond in fleet (2G LACP). Local storage is USB 2.0 — unusable for scratch. |
+| **Notes** | Best storage bond in fleet (2G LACP). Local storage is USB 2.0 -- unusable for scratch. |
---
-## pfv-tsys7 — RackRental host
+## pfv-tsys7 -- RackRental host
| Attribute | Value |
|-----------|-------|
| **Role** | RackRental: k8s wnode-tsys7 + hfnoc-uisp + rr-middleware + TCTC + k8s cnode2 |
-| **Hardware** | 2× Xeon E5-2630 v2 Ivy Bridge (2013), 24 threads, 189 GB RAM |
+| **Hardware** | 2x Xeon E5-2630 v2 Ivy Bridge (2013), 24 threads, 189 GB RAM |
| **PVE** | 9.2.5 / kernel 7.0.14-6-pve |
| **mgmt IP** | 192.168.3.55 (vmbr0, DHCP) |
| **Storage IP** | 10.100.100.7 (datanet via bond0, **2 of 2 slaves active**, 2G LACP) |
| **NFS mounts** | D1, D2, D3, D5, S1, S2, S3, S4, T5-SSD (9 mounts, all nconnect=4) |
-| **Local storage** | local-lvm (WD portable USB 2.0 HDD, 1.7 TB, **30 MB/s — do not use for VMs**) |
+| **Local storage** | local-lvm (WD portable USB 2.0 HDD, 1.7 TB, **30 MB/s -- do not use for VMs**) |
| **Tuning** | governor=performance, swappiness=10, bbr, rmem/wmem=128MB, tuned=virtual-host |
| **Bond** | bond0 (2 of 2 slaves), **hash=layer3+4** (correct) |
| **Running VMs** | 701, 702, 703, 704, 705 |
@@ -156,7 +156,7 @@ graph TB
---
-## pfv-tsys9 — Infra + Compute
+## pfv-tsys9 -- Infra + Compute
| Attribute | Value |
|-----------|-------|
@@ -166,8 +166,8 @@ graph TB
| **mgmt IP** | 192.168.3.58 (vmbr0, DHCP) |
| **Storage IP** | 10.100.100.9 (datanet via enx9c69d36a5b6c USB dongle, no bond) |
| **NFS mounts** | D2, D5, S2, S3, S4, T5-SSD (all nconnect=4) |
-| **Local storage** | local-lvm (PNY CS900 SSD, 136 GB, 0% used) — **excellent for scratch** |
+| **Local storage** | local-lvm (PNY CS900 SSD, 136 GB, 0% used) -- **excellent for scratch** |
| **Tuning** | governor=performance, swappiness=10, bbr, rmem/wmem=128MB, tuned=virtual-host |
| **Bond** | None (single USB NIC for storage) |
| **Running VMs** | 901, 902, 903, 904, 905, 906 |
-| **Notes** | Newest CPU (best IPC in fleet). **69% RAM — approaching 80% goal.** Target for compute growth (after siem migrates off). |
+| **Notes** | Newest CPU (best IPC in fleet). **69% RAM -- approaching 80% goal.** Target for compute growth (after siem migrates off). |
diff --git a/proxmox/docs/NETWORK-TOPOLOGY.md b/proxmox/docs/NETWORK-TOPOLOGY.md
index 29fedd3..01ee9c0 100644
--- a/proxmox/docs/NETWORK-TOPOLOGY.md
+++ b/proxmox/docs/NETWORK-TOPOLOGY.md
@@ -2,7 +2,7 @@
> **Complete physical + logical network reference for the PFVCluster.**
> Data gathered 2026-07-30 via SNMP, LLDP, conman console captures, and direct
-> host probes. All data is live ground truth — not documentation inherited.
+> host probes. All data is live ground truth -- not documentation inherited.
>
> Companion docs: [`STORAGE-ARCHITECTURE.md`](STORAGE-ARCHITECTURE.md),
> [`HOST-REFERENCE.md`](HOST-REFERENCE.md), [`VM-INVENTORY.md`](VM-INVENTORY.md).
@@ -14,31 +14,31 @@
```mermaid
graph TB
- subgraph "RACK 5 — Core / Data / Infra"
- CORE["pfv-core-sw01
PowerConnect 5448
192.168.0.12
FW 2.0.0.46
(mgmt + storage VLANs)"]
+ subgraph "RACK 5 - Core / Data / Infra"
+ CORE["pfv-core-sw01
PowerConnect 5448
192.168.0.12
(mgmt + storage VLANs)"]
- TSYS1["pfv-tsys1
i7-4770 Haswell
Infra host"]
- TSYS3["pfv-tsys3
Xeon E3-1535M v5
Compute"]
- TSYS4["pfv-tsys4
Xeon E3-1246 v3
NFS Storage + PBS"]
- TSYS5["pfv-tsys5
Xeon E5620 Westmere
NFS Storage + Sandboxes"]
- TSYS9["pfv-tsys9
i5-10500 Comet Lake
Infra + Compute"]
+ TSYS1["pfv-tsys1
i7-4770 Haswell
Infra host"]
+ TSYS3["pfv-tsys3
Xeon E3-1535M v5
Compute"]
+ TSYS4["pfv-tsys4
Xeon E3-1246 v3
NFS Storage + PBS"]
+ TSYS5["pfv-tsys5
Xeon E5620 Westmere
NFS Storage + Sandboxes"]
+ TSYS9["pfv-tsys9
i5-10500 Comet Lake
Infra + Compute"]
- TSYS4_CONSOLE["conman + ser2net
7 switch consoles
(Tailscale 100.70.77.93:7890)"]
- TSYS1_PDU["Cyclades PM10i PDU
powerman on tsys1
(port 10101)"]
- TSYS1_UPS["APC Smart-UPS C 1500
NUT on tsys1
(port 3493)"]
+ TSYS4_CONSOLE["conman + ser2net
7 switch consoles
Tailscale 100.70.77.93:7890"]
+ TSYS1_PDU["Cyclades PM10i PDU
powerman on tsys1"]
+ TSYS1_UPS["APC Smart-UPS C 1500
NUT on tsys1"]
end
- subgraph "RACK 3 — Compute / RackRental"
- TOR3STOR["pfv-tor3-stor
PowerConnect 5324 (Neyland 24T)
192.168.0.9
FW 2.0.1.4
(storage VLAN)"]
- TOR3MGMT["pfv-tor3-mgmt
PowerConnect 5324 (Neyland 24T)
192.168.0.7
(mgmt VLAN)"]
+ subgraph "RACK 3 - Compute / RackRental"
+ TOR3STOR["pfv-tor3-stor
PowerConnect 5324
192.168.0.9
storage VLAN"]
+ TOR3MGMT["pfv-tor3-mgmt
PowerConnect 5324
192.168.0.7
mgmt VLAN"]
- TSYS6["pfv-tsys6
2× Xeon E5530 Nehalem
RackRental"]
- TSYS7["pfv-tsys7
2× Xeon E5-2630 v2 Ivy Bridge
RackRental"]
+ TSYS6["pfv-tsys6
2x Xeon E5530 Nehalem
RackRental"]
+ TSYS7["pfv-tsys7
2x Xeon E5-2630 v2
RackRental"]
end
- %% Cross-rack trunk
- CORE <--. "ch1 LAG: 3×1G active
(g13,g14,g15)
hash=layer-2-3
mode=on (static)" .--> TOR3STOR
- TOR3STOR -. "ch1: 4×1G
(g20,g21,g22,g23)
hash=layer-2-3-4
mode=on (static)" .-> CORE
+ %% Cross-rack trunk (two one-way dotted arrows with labels)
+ CORE -. "ch1: 3x1G active (g13-15)
hash=layer-2-3
mode=on" .-> TOR3STOR
+ TOR3STOR -. "ch1: 4x1G (g20-23)
hash=layer-2-3-4
mode=on" .-> CORE
%% Rack 5 connections to core
CORE --> TSYS1
@@ -56,7 +56,7 @@ graph TB
### How to read this diagram
-The dashed line between core-sw01 and tor3-stor is the **cross-rack trunk** —
+The dashed line between core-sw01 and tor3-stor is the **cross-rack trunk** --
the most critical link in the cluster. It carries all NFS storage traffic
between rack-3 compute hosts (tsys6/7) and rack-5 storage hosts (tsys4/5).
The trunk is a 4-port static LAG, but only 3 links are active due to a cable
@@ -70,7 +70,7 @@ mismatch (see §3 below).
|------|--------|---------|----------------|
| **1** (default) | 192.168.0.0/22 | Management network | All switches, all hosts (vmbr0 DHCP) |
| **1000** | 10.100.100.0/24 | Storage network (NFS) | core-sw01 + tor3-stor only (trunked) |
-| — (Tailscale) | 100.x.y.z/32 (CGNAT) | Remote access / overlay | All hosts + select VMs |
+| -- (Tailscale) | 100.x.y.z/32 (CGNAT) | Remote access / overlay | All hosts + select VMs |
**Management IPs (VLAN 1, via DHCP reservations):**
@@ -109,7 +109,7 @@ mismatch (see §3 below).
---
-## 3. Cross-rack trunk detail (ch1) — the critical link
+## 3. Cross-rack trunk detail (ch1) -- the critical link
This trunk carries ALL NFS storage traffic between rack 3 (tsys6/7) and
rack 5 (tsys4/5). It is the single most important network path in the cluster.
@@ -117,18 +117,18 @@ rack 5 (tsys4/5). It is the single most important network path in the cluster.
```mermaid
graph LR
subgraph "core-sw01 (rack5)"
- G13["g13
1G UP"]
- G14["g14
1G UP"]
- G15["g15
1G UP"]
- G16["g16
1G UP
NOT in LAG!"]
- G17["g17
DOWN
in config but
no cable"]
+ G13["g13
1G UP"]
+ G14["g14
1G UP"]
+ G15["g15
1G UP"]
+ G16["g16
1G UP
NOT in LAG!"]
+ G17["g17
DOWN
in config but
no cable"]
end
subgraph "tor3-stor (rack3)"
- G20["g20
1G UP"]
- G21["g21
1G UP"]
- G22["g22
1G UP"]
- G23["g23
1G UP"]
+ G20["g20
1G UP"]
+ G21["g21
1G UP"]
+ G22["g22
1G UP"]
+ G23["g23
1G UP"]
end
G13 --- G20
@@ -142,10 +142,10 @@ graph LR
| Property | core-sw01 (rack5) | tor3-stor (rack3) | Match? |
|----------|-------------------|--------------------|--------|
-| Physical ports configured | g(13-15,17) | g(20-23) | **No — asymmetric** |
-| Physical ports UP | g13, g14, g15, g16 | g20, g21, g22, g23 | — |
-| Active in LAG | g13, g14, g15 (3) | g20, g21, g22, g23 (4) | **No — 3 vs 4** |
-| Load-balance hash | layer-2-3 (MAC+IP) | layer-2-3-4 (MAC+IP+Port) | **No — asymmetric** |
+| Physical ports configured | g(13-15,17) | g(20-23) | **No -- asymmetric** |
+| Physical ports UP | g13, g14, g15, g16 | g20, g21, g22, g23 | -- |
+| Active in LAG | g13, g14, g15 (3) | g20, g21, g22, g23 (4) | **No -- 3 vs 4** |
+| Load-balance hash | layer-2-3 (MAC+IP) | layer-2-3-4 (MAC+IP+Port) | **No -- asymmetric** |
| Channel mode | on (static) | on (static) | Yes (both should be LACP) |
| VLAN | access vlan 1000 | access vlan 1000 | Yes |
@@ -162,19 +162,19 @@ graph LR
one link** regardless of how many TCP connections (nconnect) are used.
3. **Static mode (`on`).** Neither switch runs LACP. There is no link-failure
- detection — a dead cable stays in the LAG until manually discovered.
+ detection -- a dead cable stays in the LAG until manually discovered.
---
## 4. Complete switch port map
-### pfv-core-sw01 (PowerConnect 5448, 48 ports) — rack 5
+### pfv-core-sw01 (PowerConnect 5448, 48 ports) -- rack 5
| Port | Speed | Status | VLAN | Description / Connected device |
|------|-------|--------|------|-------------------------------|
| g1 | 1000 | UP | 1 | pfvsvrpi (Raspberry Pi) |
| g4 | 100 | UP | 1 | pfv-tsys4 (mgmt, secondary) |
-| g5 | — | UP | 1 | pfv-tsys1 (mgmt) |
+| g5 | -- | UP | 1 | pfv-tsys1 (mgmt) |
| g6 | 100 | UP | 1 | pfvsvrpi (secondary) |
| g8 | 1000 | UP | 1 | pfv-tsys4-mgmt |
| g9 | 1000 | UP | 1 | AP-Wallmount (UAP-AC-LR) **⚠ 2902 inErrors** |
@@ -184,72 +184,72 @@ graph LR
| **g13** | **1000** | **UP** | **1000 (ch1)** | **cross-rack trunk → tor3-stor g20** |
| **g14** | **1000** | **UP** | **1000 (ch1)** | **cross-rack trunk → tor3-stor g21** |
| **g15** | **1000** | **UP** | **1000 (ch1)** | **cross-rack trunk → tor3-stor g22** |
-| **g16** | **1000** | **UP** | **1000** | **cross-rack trunk cable (NOT in LAG — should be g17)** |
-| **g17** | — | **DOWN** | **1000 (ch1)** | **in LAG config but no cable** |
+| **g16** | **1000** | **UP** | **1000** | **cross-rack trunk cable (NOT in LAG -- should be g17)** |
+| **g17** | -- | **DOWN** | **1000 (ch1)** | **in LAG config but no cable** |
| g19 | 1000 | UP | 1 | pfv-tsys5-mgmt |
| g21 | 1000 | UP | 1 | (unknown) |
| g23 | 100 | UP | 1 | pfv-tsys9 (mgmt) |
| g26 | 1000 | UP | 1 | AP-Tabletop **⚠ 73 inErrors** |
| g27 | 1000 | UP | 1000 | pfv-tsys3-stor (datanet) |
| g31 | 1000 | UP | 1000 | pfv-tsys4-stor (datanet, USB) |
-| g32 | — | — | — | pfv-tsys4-storage (configured, DOWN) |
-| g33 | — | — | — | pfv-tsys5-stor (ch2, DOWN/Not Present) |
+| g32 | -- | -- | -- | pfv-tsys4-storage (configured, DOWN) |
+| g33 | -- | -- | -- | pfv-tsys5-stor (ch2, DOWN/Not Present) |
| g34 | 1000 | UP | 1000 (ch3) | pfv-tsys5-stor (bond0, 1 link active) |
| g38 | 1000 | UP | 1000 | pfv-tsys9-stor (datanet) **⚠ 590 inErrors** |
| g40 | 1000 | UP | 1000 | pfv-tsys1-stor (datanet) |
| g41 | 1000 | UP | 1 | (unknown) |
| g43 | 1000 | UP | 1 | pfv-tsys3 (mgmt) |
| g44 | 1000 | UP | 1 | (unknown) |
-| g45-g48 | — | DOWN | — | (Combo ports, unused) |
+| g45-g48 | -- | DOWN | -- | (Combo ports, unused) |
**Port-channels on core-sw01:**
| Channel | Speed | Status | Members | VLAN | Notes |
|---------|-------|--------|---------|------|-------|
| ch1 | 1000 | UP | g13-15 (active), g17 (inactive) | 1000 | Cross-rack trunk to tor3-stor |
-| ch2 | — | DOWN | g32-33 (non-candidate) | — | Unused |
+| ch2 | -- | DOWN | g32-33 (non-candidate) | -- | Unused |
| ch3 | 1000 | UP | g34 (active) | 1000 | tsys5 storage bond (degraded, 1 of 2 links) |
**LAG hash:** `port-channel load-balance layer-2-3`
-### pfv-tor3-stor (PowerConnect 5324 / Neyland 24T, 24 ports) — rack 3
+### pfv-tor3-stor (PowerConnect 5324 / Neyland 24T, 24 ports) -- rack 3
| Port | Speed | Status | VLAN | Description / Connected device |
|------|-------|--------|------|-------------------------------|
-| g1 | 1000 | UP | 1 | (unknown — uplink to tor3-mgmt or router) |
+| g1 | 1000 | UP | 1 | (unknown -- uplink to tor3-mgmt or router) |
| g2-g7 | 1000 | UP | 1 | (unused but UP) |
| **g8** | **1000** | **UP** | **1000 (ch2)** | **tsys6 stor nic1** |
| **g9** | **1000** | **UP** | **1000 (ch2)** | **tsys6 stor nic2** |
| **g10** | **1000** | **UP** | **1000 (ch3)** | **tsys7 stor nic1** |
| **g11** | **1000** | **UP** | **1000 (ch3)** | **tsys7 stor nic2** |
-| **g13** | **1000** | **UP** | **—** | **tsys8-stor (STALE — tsys8 retired)** |
-| **g14** | **1000** | **UP** | **—** | **tsys8-stor (STALE — tsys8 retired)** |
+| **g13** | **1000** | **UP** | **--** | **tsys8-stor (STALE -- tsys8 retired)** |
+| **g14** | **1000** | **UP** | **--** | **tsys8-stor (STALE -- tsys8 retired)** |
| g15-g19 | 1000 | UP | 1000 | (unused storage ports) |
| **g20** | **1000** | **UP** | **1000 (ch1)** | **cross-rack trunk → core-sw01 g13** |
| **g21** | **1000** | **UP** | **1000 (ch1)** | **cross-rack trunk → core-sw01 g14** |
| **g22** | **1000** | **UP** | **1000 (ch1)** | **cross-rack trunk → core-sw01 g15** |
| **g23** | **1000** | **UP** | **1000 (ch1)** | **cross-rack trunk → core-sw01 g16** |
-| g24 | — | DOWN | 1 | (spanning-tree disabled, portfast) |
+| g24 | -- | DOWN | 1 | (spanning-tree disabled, portfast) |
**Port-channels on tor3-stor:**
| Channel | Speed | Status | Members | VLAN | Notes |
|---------|-------|--------|---------|------|-------|
-| ch1 | 1000 | UP | g20-23 (`mode on`) | 1000 | Cross-rack trunk to core-sw01 (4×1G) |
-| ch2 | 1000 | UP | g8-9 (`mode auto`) | 1000 | tsys6 storage bond (2×1G LACP) |
-| ch3 | 1000 | UP | g10-11 (`mode auto`) | 1000 | tsys7 storage bond (2×1G LACP) |
-| ch4 | 1000 | UP | g13-14 (`mode on`) | 1000 | **tsys8-stor (STALE — retired host)** |
+| ch1 | 1000 | UP | g20-23 (`mode on`) | 1000 | Cross-rack trunk to core-sw01 (4x1G) |
+| ch2 | 1000 | UP | g8-9 (`mode auto`) | 1000 | tsys6 storage bond (2x1G LACP) |
+| ch3 | 1000 | UP | g10-11 (`mode auto`) | 1000 | tsys7 storage bond (2x1G LACP) |
+| ch4 | 1000 | UP | g13-14 (`mode on`) | 1000 | **tsys8-stor (STALE -- retired host)** |
**LAG hash:** `port-channel load-balance layer-2-3-4`
-### pfv-tor3-mgmt (PowerConnect 5324 / Neyland 24T, 24 ports) — rack 3
+### pfv-tor3-mgmt (PowerConnect 5324 / Neyland 24T, 24 ports) -- rack 3
| Port | Speed | Status | Connected device |
|------|-------|--------|-----------------|
-| g3 | — | UP | tsys7 (mgmt nic0) |
-| g7 | — | UP | tsys6 (mgmt nic0) |
+| g3 | -- | UP | tsys7 (mgmt nic0) |
+| g7 | -- | UP | tsys6 (mgmt nic0) |
-(Full port map not captured — this switch carries only VLAN 1 management
+(Full port map not captured -- this switch carries only VLAN 1 management
traffic. Low priority for storage performance.)
---
@@ -258,24 +258,24 @@ traffic. Low priority for storage performance.)
| Host | mgmt NIC | Switch port | Storage NIC | Switch port | Storage bond | Bond hash | Storage link speed |
|------|----------|-------------|-------------|-------------|-------------|-----------|-------------------|
-| tsys1 | nic0 | core g5 | nic1 (onboard) | core g40 | **No bond** (single NIC) | — | 1G |
-| tsys3 | nic0 | core g43 | enx8cae4ccda774 (USB) | core g27 | **No bond** (USB dongle) | — | 1G |
+| tsys1 | nic0 | core g5 | nic1 (onboard) | core g40 | **No bond** (single NIC) | -- | 1G |
+| tsys3 | nic0 | core g43 | enx8cae4ccda774 (USB) | core g27 | **No bond** (USB dongle) | -- | 1G |
| tsys4 | nic0 | core g8 | enx8cae4ccda926 (USB) | core g31 | bond0 (1 slave, USB) | layer2+3 | 1G |
| tsys5 | nic0 | core g19 | nic1 + nic2 | core ch3 (g34) | bond0 (1 of 2 slaves) | layer2+3 | **1G (degraded)** |
| tsys6 | nic0 | tor3-mgmt g7 | nic1 + nic2 | tor3-stor g8/g9 (ch2) | bond0 (2 of 2 slaves) | **layer3+4** | **2G LACP** |
| tsys7 | nic0 | tor3-mgmt g3 | nic1 + nic2 | tor3-stor g10/g11 (ch3) | bond0 (2 of 2 slaves) | **layer3+4** | **2G LACP** |
-| tsys9 | nic0 | core g23 | enx9c69d36a5b6c (USB) | core g38 | **No bond** (USB dongle) | — | 1G |
+| tsys9 | nic0 | core g23 | enx9c69d36a5b6c (USB) | core g38 | **No bond** (USB dongle) | -- | 1G |
### Key observations
- **tsys6/7 are the only hosts with working 2G LACP bonds** (layer3+4 hash, 2
active slaves). All NFS traffic from rack 3 uses these bonds.
-- **tsys1/3/4/9 use USB dongles** for storage network — single 1G, no redundancy.
-- **tsys5 bond0 is degraded** — nic2 is DOWN (no cable connected). Only nic1 is
+- **tsys1/3/4/9 use USB dongles** for storage network -- single 1G, no redundancy.
+- **tsys5 bond0 is degraded** -- nic2 is DOWN (no cable connected). Only nic1 is
active via core-sw01 ch3 (g34). This is the LACP problem the operator has been
troubleshooting.
- **Bond hash inconsistency:** tsys6/7 use layer3+4 (optimal for nconnect NFS),
- tsys4/5 use layer2+3 (suboptimal — same src/dst IP pair always hashes to one
+ tsys4/5 use layer2+3 (suboptimal -- same src/dst IP pair always hashes to one
link even with nconnect's multiple TCP connections).
---
@@ -285,17 +285,17 @@ traffic. Low priority for storage performance.)
```mermaid
graph TB
subgraph "Storage Servers (rack 5)"
- T4D2["tsys4 D2
WDC Red 3TB
Tier 2"]
- T4D5["tsys4 D5
Hitachi 2TB
Tier 2"]
- T4D1["tsys4 D1
WD 1TB USB
Tier 3"]
- T4D3["tsys4 D3
Seagate 500GB USB
Tier 3"]
- T5S1["tsys5 S1
IronWolf 1TB
Tier 2"]
- T5S2["tsys5 S2
IronWolf 1TB
Tier 2"]
- T5SSD["tsys5 SSD
Samsung 860 PRO
Tier 1 (93% full)"]
+ T4D2["tsys4 D2
WDC Red 3TB
Tier 2"]
+ T4D5["tsys4 D5
Hitachi 2TB
Tier 2"]
+ T4D1["tsys4 D1
WD 1TB USB
Tier 3"]
+ T4D3["tsys4 D3
Seagate 500GB USB
Tier 3"]
+ T5S1["tsys5 S1
IronWolf 1TB
Tier 2"]
+ T5S2["tsys5 S2
IronWolf 1TB
Tier 2"]
+ T5SSD["tsys5 SSD
Samsung 860 PRO
Tier 1 (93% full)"]
end
subgraph "VLAN 1000 storage network"
- NFS["NFS v4.2
nconnect=4
rsize/wsize=1MB"]
+ NFS["NFS v4.2
nconnect=4
rsize/wsize=1MB"]
end
subgraph "NFS Clients"
@@ -327,15 +327,15 @@ graph TB
| Server | Export | tsys1 | tsys3 | tsys4 | tsys5 | tsys6 | tsys7 | tsys9 |
|--------|--------|-------|-------|-------|-------|-------|-------|-------|
-| tsys4 | D1 | — | — | — | — | ✓ | ✓ | — |
-| tsys4 | **D2** | ✓ | ✓ | — | ✓ | ✓ | ✓ | ✓ |
-| tsys4 | D3 | — | — | — | ✓ | ✓ | ✓ | — |
-| tsys4 | D5 | — | ✓ | — | ✓ | ✓ | ✓ | ✓ |
-| tsys5 | S1 | ✓ | — | — | — | ✓ | ✓ | — |
-| tsys5 | S2 | ✓ | ✓ | — | — | ✓ | ✓ | — |
-| tsys5 | S3 | ✓ | — | — | — | ✓ | ✓ | ✓ |
-| tsys5 | S4 | ✓ | ✓ | — | — | ✓ | ✓ | ✓ |
-| tsys5 | T5-SSD | — | ✓ | — | — | ✓ | ✓ | ✓ |
+| tsys4 | D1 | -- | -- | -- | -- | ✓ | ✓ | -- |
+| tsys4 | **D2** | ✓ | ✓ | -- | ✓ | ✓ | ✓ | ✓ |
+| tsys4 | D3 | -- | -- | -- | ✓ | ✓ | ✓ | -- |
+| tsys4 | D5 | -- | ✓ | -- | ✓ | ✓ | ✓ | ✓ |
+| tsys5 | S1 | ✓ | -- | -- | -- | ✓ | ✓ | -- |
+| tsys5 | S2 | ✓ | ✓ | -- | -- | ✓ | ✓ | -- |
+| tsys5 | S3 | ✓ | -- | -- | -- | ✓ | ✓ | ✓ |
+| tsys5 | S4 | ✓ | ✓ | -- | -- | ✓ | ✓ | ✓ |
+| tsys5 | T5-SSD | -- | ✓ | -- | -- | ✓ | ✓ | ✓ |
(✓ = NFS client mount active. tsys4 = NFS server only, no client mounts.
tsys5 = NFS server + 3 client mounts from tsys4, **missing nconnect**.)
@@ -350,7 +350,7 @@ tsys5 = NFS server + 3 client mounts from tsys4, **missing nconnect**.)
| **Console (conman)** | 7 serial consoles via ser2net + conman on pfv-tsys4. Conman server at `pfv-tsys4:7890` (Tailscale). Escape: `&.` |
| **Console tool** | `perf/scripts/conman-console.py` (PTY-based driver, portable) |
| **SNMP audit tool** | `perf/scripts/snmp-switch-audit.py` (config-driven) |
-| **Direct telnet** | `telnet pfv-tsys4 200X` — ONLY when conmand stopped (conflicts). Use conman instead. |
+| **Direct telnet** | `telnet pfv-tsys4 200X` -- ONLY when conmand stopped (conflicts). Use conman instead. |
### Console port assignments
diff --git a/proxmox/docs/STORAGE-ARCHITECTURE.md b/proxmox/docs/STORAGE-ARCHITECTURE.md
index 7f80e5e..f284099 100644
--- a/proxmox/docs/STORAGE-ARCHITECTURE.md
+++ b/proxmox/docs/STORAGE-ARCHITECTURE.md
@@ -28,16 +28,16 @@ is acceptable.
## 2. Physical disk inventory
-### tsys4 — NFS storage server + PBS (7 disks)
+### tsys4 -- NFS storage server + PBS (7 disks)
| Device | Model | Size | RPM | Interface | Mount | NFS ID | Tier | SMART | Used |
|--------|-------|------|-----|-----------|-------|--------|------|-------|------|
| sda | Hitachi HDS723020BLA642 | 2.0 TB | 7200 | SATA | /mnt/albert | (not exported) | **Tier 2** | PASSED, 0 reallocated | 0% |
| sdb | HITACHI HUA723020ALA640 | 2.0 TB | 7200 | SATA | /mnt/tsys4/D5 | **D5** | Tier 2 | PASSED, 0 reallocated | 13% (220 GB) |
| sdc | WDC WD30EFRX (Red) | 3.0 TB | 5400 | SATA | /mnt/tsys4/D2 | **D2** | Tier 2 | PASSED, 0 reallocated | 7% (170 GB) |
-| sdd | WDC WD10EZRX | 1.0 TB | — | SATA | (pve boot/swap/lvm) | — | Tier 2 | PASSED, 1 reallocated (sdf has 1) | OS disk |
+| sdd | WDC WD10EZRX | 1.0 TB | -- | SATA | (pve boot/swap/lvm) | -- | Tier 2 | PASSED, 1 reallocated (sdf has 1) | OS disk |
| sde | ST500LM021 (Seagate) | 500 GB | 7200 | **USB** | /mnt/tsys4/D3 | **D3** | **Tier 3** | PASSED | 0% |
-| sdf | WDC WD10EARS | 1.0 TB | — | **USB** | /mnt/tsys4/D1 | **D1** | **Tier 3** | PASSED, 1 reallocated sector | 0% |
+| sdf | WDC WD10EARS | 1.0 TB | -- | **USB** | /mnt/tsys4/D1 | **D1** | **Tier 3** | PASSED, 1 reallocated sector | 0% |
| sdg | WDC WD50NDZW (easystore) | 5.0 TB | 5400 | **USB** | /mnt/backup | (not exported) | **Tier 3** | PASSED | 0% |
**Changes from previous audit:**
@@ -46,25 +46,25 @@ is acceptable.
- **/mnt/backup added:** WD 5TB USB easystore. Bulk backup only.
- **D7 removed:** Previously held DellOpenManageEnterprise. Now gone.
-### tsys5 — NFS storage server + sandboxes (7 disks)
+### tsys5 -- NFS storage server + sandboxes (7 disks)
| Device | Model | Size | RPM | Interface | Mount | NFS/Local ID | Tier | SMART | Used |
|--------|-------|------|-----|-----------|-------|-------------|------|-------|------|
-| sda | ST1000VN002 (IronWolf) | 1.0 TB | 5900 | SATA | **(not mounted)** | was S3 | Tier 2 | PASSED | — |
-| sdb | SK hynix SC300B | 512 GB | SSD | SATA | **(not mounted)** | — | **Tier 1** | PASSED | **NEW, not configured** |
-| sdc | HITACHI HUS724020ALA640 | 2.0 TB | 7200 | SAS | (pve boot/swap/lvm) | — | Tier 2 | PASSED | OS disk |
+| sda | ST1000VN002 (IronWolf) | 1.0 TB | 5900 | SATA | **(not mounted)** | was S3 | Tier 2 | PASSED | -- |
+| sdb | SK hynix SC300B | 512 GB | SSD | SATA | **(not mounted)** | -- | **Tier 1** | PASSED | **NEW, not configured** |
+| sdc | HITACHI HUS724020ALA640 | 2.0 TB | 7200 | SAS | (pve boot/swap/lvm) | -- | Tier 2 | PASSED | OS disk |
| sdd | Samsung SSD 860 PRO | 256 GB | SSD | SAS | /mnt/pfv-tsys5/ssd | `ssd` / `SSD` dir (local) | **Tier 1** | PASSED | **93% full!** |
| sde | ST1000VN002 (IronWolf) | 1.0 TB | 5900 | SAS | /mnt/pfv-tsys5/S1 | **S1** + `local-nonprod` | Tier 2 | PASSED | 11% (96 GB) |
| sdf | ST1000VN002 (IronWolf) | 1.0 TB | 5900 | SAS | /mnt/pfv-tsys5/S2 | **S2** | Tier 2 | PASSED | 4% (32 GB) |
-| **nvme0n1** | **CT500P1SSD8 (Crucial)** | **500 GB** | **NVMe** | **PCIe** | **(not mounted)** | — | **Tier 1** | PASSED, 0 integrity errors, 34K hrs | **NEW, not configured** |
+| **nvme0n1** | **CT500P1SSD8 (Crucial)** | **500 GB** | **NVMe** | **PCIe** | **(not mounted)** | -- | **Tier 1** | PASSED, 0 integrity errors, 34K hrs | **NEW, not configured** |
**Changes from previous audit:**
-- **NVMe added:** Crucial CT500P1 500GB. True NVMe — highest-performance local storage in the fleet. Not mounted, not in fstab, not in storage.cfg.
+- **NVMe added:** Crucial CT500P1 500GB. True NVMe -- highest-performance local storage in the fleet. Not mounted, not in fstab, not in storage.cfg.
- **SK hynix SSD relocated:** Moved from tsys4 USB enclosure to tsys5 internal SATA. Not mounted or configured.
- **S3 unmounted:** sda (was S3) is present but fstab entry is commented out. Stale export remains in /etc/exports.
- **S4 gone:** Old Toshiba 458GB no longer present. fstab entry commented. storage.cfg has broken `dir: S4` entry.
- **S1/S2 drives changed:** Both are now ST1000VN002 IronWolf 1TB (5900 rpm).
-- **SSD nearly full:** Samsung 860 PRO at 93% (206/234 GB) — needs attention.
+- **SSD nearly full:** Samsung 860 PRO at 93% (206/234 GB) -- needs attention.
- **Boot disk changed:** HITACHI HUS724020ALA640 2TB (was Hitachi 2TB in old audit, now confirmed model).
---
@@ -73,21 +73,21 @@ is acceptable.
```mermaid
graph LR
- subgraph "tsys4 — NFS Server"
- D2["D2: /mnt/tsys4/D2
WDC Red 3TB
Tier 2 · 7% used"]
- D5["D5: /mnt/tsys4/D5
Hitachi 2TB
Tier 2 · 13% used"]
- D1["D1: /mnt/tsys4/D1
WD 1TB USB
Tier 3 · 0% used"]
- D3["D3: /mnt/tsys4/D3
Seagate 500GB USB
Tier 3 · 0% used"]
+ subgraph "tsys4 -- NFS Server"
+ D2["D2: /mnt/tsys4/D2
WDC Red 3TB
Tier 2 | 7% used"]
+ D5["D5: /mnt/tsys4/D5
Hitachi 2TB
Tier 2 | 13% used"]
+ D1["D1: /mnt/tsys4/D1
WD 1TB USB
Tier 3 | 0% used"]
+ D3["D3: /mnt/tsys4/D3
Seagate 500GB USB
Tier 3 | 0% used"]
end
- subgraph "tsys5 — NFS Server"
- S1["S1: /mnt/pfv-tsys5/S1
IronWolf 1TB
Tier 2 · 11% used"]
- S2["S2: /mnt/pfv-tsys5/S2
IronWolf 1TB
Tier 2 · 4% used"]
- S3stale["S3: (stale export)
disk not mounted"]
+ subgraph "tsys5 -- NFS Server"
+ S1["S1: /mnt/pfv-tsys5/S1
IronWolf 1TB
Tier 2 | 11% used"]
+ S2["S2: /mnt/pfv-tsys5/S2
IronWolf 1TB
Tier 2 | 4% used"]
+ S3stale["S3: (stale export)
disk not mounted"]
end
- subgraph "All hosts — NFS Clients"
- MNT["/mnt/pve/D1 D2 D3 D5
/mnt/pve/S1 S2 S3 S4
/mnt/pve/T5-SSD"]
+ subgraph "All hosts -- NFS Clients"
+ MNT["/mnt/pve/D1 D2 D3 D5
/mnt/pve/S1 S2 S3 S4
/mnt/pve/T5-SSD"]
end
D2 -->|NFS v4.2| MNT
@@ -111,7 +111,7 @@ graph LR
| tsys5 | /mnt/pfv-tsys5/S2 | S2 | NFSv4.2 | nconnect=4,noatime,rsize=1048576,wsize=1048576,hard |
| tsys5 | /mnt/pfv-tsys5/S3 | S3 | NFSv4.2 | **stale** (disk not mounted on server) |
| tsys5 | /mnt/pfv-tsys5/ssd | T5-SSD | NFSv4.2 | nconnect=4,noatime,rsize=1048576,wsize=1048576,hard |
-| tsys5 | /mnt/pfv-tsys5/S4 | S4 | — | **broken** (disk gone, storage.cfg entry stale) |
+| tsys5 | /mnt/pfv-tsys5/S4 | S4 | -- | **broken** (disk gone, storage.cfg entry stale) |
> **Note:** tsys5 is the only host where NFS client mounts (D2/D3/D5 from
> tsys4) are **missing nconnect=4**. All other client hosts have nconnect on
@@ -122,27 +122,27 @@ graph LR
## 4. Per-spindle VM placement (running VMs only)
> Shows every running VM's primary disk and the physical spindle it lives on.
-> This is the IOPS distribution map — the key to storage balancing.
+> This is the IOPS distribution map -- the key to storage balancing.
```mermaid
graph TB
subgraph "tsys4 spindles"
- D2["D2 — WDC Red 3TB
★★★ 13 VMs — WORST HOTSPOT"]
- D5["D5 — Hitachi 2TB
4 VMs"]
- D1["D1 — WD 1TB USB
0 VMs (empty)"]
- D3["D3 — Seagate 500GB USB
0 VMs (empty)"]
- ALB["/mnt/albert — Hitachi 2TB
0 VMs (not in Proxmox)"]
+ D2["D2 -- WDC Red 3TB
*** 13 VMs -- WORST HOTSPOT"]
+ D5["D5 -- Hitachi 2TB
4 VMs"]
+ D1["D1 -- WD 1TB USB
0 VMs (empty)"]
+ D3["D3 -- Seagate 500GB USB
0 VMs (empty)"]
+ ALB["/mnt/albert -- Hitachi 2TB
0 VMs (not in Proxmox)"]
end
subgraph "tsys5 spindles"
- S1["S1 — IronWolf 1TB
1 NFS VM + 12 local stopped"]
- S2["S2 — IronWolf 1TB
5 VMs"]
- SSD["SSD — Samsung 860 PRO
1 VM (93% full!)"]
- NVME["NVMe — Crucial CT500P1
NOT CONFIGURED"]
- HYNIX["SK hynix SSD 512GB
NOT CONFIGURED"]
+ S1["S1 -- IronWolf 1TB
1 NFS VM + 12 local stopped"]
+ S2["S2 -- IronWolf 1TB
5 VMs"]
+ SSD["SSD -- Samsung 860 PRO
1 VM (93% full!)"]
+ NVME["NVMe -- Crucial CT500P1
NOT CONFIGURED"]
+ HYNIX["SK hynix SSD 512GB
NOT CONFIGURED"]
end
- D2 --- D2vms["100 101 104 105 108
114 117 509 600 702
704 901 904"]
+ D2 --- D2vms["100 101 104 105 108
114 117 509 600 702
704 901 904"]
D5 --- D5vms["106 313 600(d0) 903"]
S1 --- S1vms["702 hfnoc-uisp"]
S2 --- S2vms["103 703 705 902 905"]
@@ -158,11 +158,11 @@ graph TB
| S1 (IronWolf 1TB) | tsys5 | S1 + local-nonprod | Tier 2 | 1 (NFS) + 0 (local, 12 stopped) | 702 | Low; 12 stopped sandboxes on local-nonprod |
| S2 (IronWolf 1TB) | tsys5 | S2 | Tier 2 | 5 | 103,703,705,902,905 | Moderate |
| ssd (Samsung 860 PRO) | tsys5 | SSD dir | Tier 1 | 1 | 5111 | **93% full** |
-| D1 (WD 1TB USB) | tsys4 | D1 | Tier 3 | 0 | — | Empty |
-| D3 (Seagate 500GB USB) | tsys4 | D3 | Tier 3 | 0 | — | Empty |
-| /mnt/albert (Hitachi 2TB) | tsys4 | (none) | Tier 2 | 0 | — | Not in Proxmox |
-| NVMe (CT500P1 500GB) | tsys5 | (none) | Tier 1 | 0 | — | **Not configured** |
-| SK hynix SSD (512GB) | tsys5 | (none) | Tier 1 | 0 | — | **Not configured** |
+| D1 (WD 1TB USB) | tsys4 | D1 | Tier 3 | 0 | -- | Empty |
+| D3 (Seagate 500GB USB) | tsys4 | D3 | Tier 3 | 0 | -- | Empty |
+| /mnt/albert (Hitachi 2TB) | tsys4 | (none) | Tier 2 | 0 | -- | Not in Proxmox |
+| NVMe (CT500P1 500GB) | tsys5 | (none) | Tier 1 | 0 | -- | **Not configured** |
+| SK hynix SSD (512GB) | tsys5 | (none) | Tier 1 | 0 | -- | **Not configured** |
### Local-only VM disks (not on NFS)
@@ -188,8 +188,8 @@ graph TB
| S1 (IronWolf 1TB) | 916 GB | 96 GB | 774 GB | 11% |
| S2 (IronWolf 1TB) | 916 GB | 32 GB | 838 GB | 4% |
| SSD (Samsung 860 PRO) | 234 GB | 206 GB | **16 GB** | **93%** |
-| NVMe (CT500P1) | 466 GB | — | — | Not configured |
-| SK hynix SSD | 477 GB | — | — | Not configured |
+| NVMe (CT500P1) | 466 GB | -- | -- | Not configured |
+| SK hynix SSD | 477 GB | -- | -- | Not configured |
**Total Tier 2 available (NFS production):** ~5.5 TB unused across D2/D5/S1/S2
**Total Tier 3 available (USB):** ~6 TB unused across D1/D3/backup
diff --git a/proxmox/docs/VM-INVENTORY.md b/proxmox/docs/VM-INVENTORY.md
index 6a6c664..3f72551 100644
--- a/proxmox/docs/VM-INVENTORY.md
+++ b/proxmox/docs/VM-INVENTORY.md
@@ -13,53 +13,53 @@
```mermaid
graph TB
- subgraph tsys1["tsys1 — Infra (57% RAM)"]
- VM100["100 pfv-bms
2c/4G D2"]
- VM101["101 tsys-ca
2c/2G D2"]
- VM103["103 netinfra-01
2c/4G S2"]
- VM104["104 librenms
2c/2G D2"]
- VM105["105 proxmox-dc
2c/2G D2"]
- VM106["106 cnode3
4c/4G D5"]
- VM108["108 ucs-01
2c/8G D2"]
- VM114["114 kali-tsys
2c/2G D2"]
- VM117["117 workbench
2c/4G D2"]
+ subgraph tsys1["tsys1 -- Infra (57% RAM)"]
+ VM100["100 pfv-bms
2c/4G D2"]
+ VM101["101 tsys-ca
2c/2G D2"]
+ VM103["103 netinfra-01
2c/4G S2"]
+ VM104["104 librenms
2c/2G D2"]
+ VM105["105 proxmox-dc
2c/2G D2"]
+ VM106["106 cnode3
4c/4G D5"]
+ VM108["108 ucs-01
2c/8G D2"]
+ VM114["114 kali-tsys
2c/2G D2"]
+ VM117["117 workbench
2c/4G D2"]
end
- subgraph tsys3["tsys3 — Compute (10% RAM)"]
- VM313["313 wnode-tsys3
8c/28G D5"]
+ subgraph tsys3["tsys3 -- Compute (10% RAM)"]
+ VM313["313 wnode-tsys3
8c/28G D5"]
end
- subgraph tsys4["tsys4 — Storage (15% RAM)"]
- VM400["400 PBS
2c/2G local-lvm"]
+ subgraph tsys4["tsys4 -- Storage (15% RAM)"]
+ VM400["400 PBS
2c/2G local-lvm"]
end
- subgraph tsys5["tsys5 — Storage+Sandbox (15% RAM)"]
- VM509["509 wnode-tsys5
2c/32G D2"]
- VM515["515 hfnoc-uisp-preprod
2c/2G local-nonprod"]
- VM5111["5111 ultix-streaming
4c/9G SSD"]
- VM5112["5112 ultix-offstage
4c/6G local-lvm"]
+ subgraph tsys5["tsys5 -- Storage+Sandbox (15% RAM)"]
+ VM509["509 wnode-tsys5
2c/32G D2"]
+ VM515["515 hfnoc-uisp-preprod
2c/2G local-nonprod"]
+ VM5111["5111 ultix-streaming
4c/9G SSD"]
+ VM5112["5112 ultix-offstage
4c/6G local-lvm"]
end
- subgraph tsys6["tsys6 — RackRental (12% RAM)"]
- VM100b["100 wnode-tsys6
2c/64G S4"]
- VM600["600 tsys-awx
2c/12G D5"]
+ subgraph tsys6["tsys6 -- RackRental (12% RAM)"]
+ VM100b["100 wnode-tsys6
2c/64G S4"]
+ VM600["600 tsys-awx
2c/12G D5"]
end
- subgraph tsys7["tsys7 — RackRental (7% RAM)"]
- VM701["701 wnode-tsys7
4c/32G S3"]
- VM702["702 hfnoc-uisp
2c/8G S1"]
- VM703["703 rr-middleware
2c/2G S2"]
- VM704["704 TCTC
4c/6G D2"]
- VM705["705 cnode2
4c/4G S2"]
+ subgraph tsys7["tsys7 -- RackRental (7% RAM)"]
+ VM701["701 wnode-tsys7
4c/32G S3"]
+ VM702["702 hfnoc-uisp
2c/8G S1"]
+ VM703["703 rr-middleware
2c/2G S2"]
+ VM704["704 TCTC
4c/6G D2"]
+ VM705["705 cnode2
4c/4G S2"]
end
- subgraph tsys9["tsys9 — Infra+Compute (69% RAM)"]
- VM901["901 tsys-siem
2c/8G D2"]
- VM902["902 ucs-02
2c/8G S2"]
- VM903["903 kali-rd
2c/2G D5"]
- VM904["904 netinfra-02
2c/4G D2"]
- VM905["905 wnode-tsys9
4c/4G S2"]
- VM906["906 cnode1
2c/4G S3"]
+ subgraph tsys9["tsys9 -- Infra+Compute (69% RAM)"]
+ VM901["901 tsys-siem
2c/8G D2"]
+ VM902["902 ucs-02
2c/8G S2"]
+ VM903["903 kali-rd
2c/2G D5"]
+ VM904["904 netinfra-02
2c/4G D2"]
+ VM905["905 wnode-tsys9
4c/4G S2"]
+ VM906["906 cnode1
2c/4G S3"]
end
```
@@ -71,14 +71,14 @@ graph TB
| VMID | Name | Host | vCPU | CPU type | RAM | Disk store | Spindle | Disk type | Cache | iothread | qga | Notes |
|------|------|------|------|----------|-----|-----------|---------|-----------|-------|----------|-----|-------|
-| 100 | pfv-bms (HomeAssistant) | tsys1 | 2 | host | 4G | D2 (raw) | D2 | raw | — | — | ✓ | discard=on, ssd=1. HA for power/temp monitoring. |
-| 101 | tsys-ca | tsys1 | 2 | v2-AES | 2G | D2 (virtio) | D2 | qcow2 | — | ✓ | — | no nested-virt flag. Certificate Authority. |
-| 103 | pfv-netinfra-01 | tsys1 | 2 | v2-AES | 4G | S2 | S2 | qcow2 | — | ✓ | — | DNS primary (Technitium + Pi-hole). |
-| 104 | tsys-librenms | tsys1 | 2 | host | 2G | D2 | D2 | qcow2 | — | ✓ | ✓ | SNMP monitoring for switches. |
-| 105 | tsys-proxmox-datacenter | tsys1 | 2 | host | 2G | D2 | D2 | qcow2 | — | ✓ | ✓ | PDM (Proxmox Datacenter Manager). |
+| 100 | pfv-bms (HomeAssistant) | tsys1 | 2 | host | 4G | D2 (raw) | D2 | raw | -- | -- | ✓ | discard=on, ssd=1. HA for power/temp monitoring. |
+| 101 | tsys-ca | tsys1 | 2 | v2-AES | 2G | D2 (virtio) | D2 | qcow2 | -- | ✓ | -- | no nested-virt flag. Certificate Authority. |
+| 103 | pfv-netinfra-01 | tsys1 | 2 | v2-AES | 4G | S2 | S2 | qcow2 | -- | ✓ | -- | DNS primary (Technitium + Pi-hole). |
+| 104 | tsys-librenms | tsys1 | 2 | host | 2G | D2 | D2 | qcow2 | -- | ✓ | ✓ | SNMP monitoring for switches. |
+| 105 | tsys-proxmox-datacenter | tsys1 | 2 | host | 2G | D2 | D2 | qcow2 | -- | ✓ | ✓ | PDM (Proxmox Datacenter Manager). |
| 108 | tsys-ucs-01 | tsys1 | 2 | host | 8G | D2 | D2 | qcow2 | **writethrough** | ✓ | ✓ | Univention Corporate Server half 1. |
-| 902 | tsys-ucs-02 | tsys9 | 2 | host | 8G | S2 | S2 | qcow2 | — | ✓ | ✓ | UCS half 2 (HA pair with 108). |
-| 904 | pfv-netinfra-02 | tsys9 | 2 | v2-AES | 4G | D2 | D2 | qcow2 | — | ✓ | — | DNS secondary. HA pair with 103. |
+| 902 | tsys-ucs-02 | tsys9 | 2 | host | 8G | S2 | S2 | qcow2 | -- | ✓ | ✓ | UCS half 2 (HA pair with 108). |
+| 904 | pfv-netinfra-02 | tsys9 | 2 | v2-AES | 4G | D2 | D2 | qcow2 | -- | ✓ | -- | DNS secondary. HA pair with 103. |
### k8s control plane (cnode1/2/3)
@@ -86,17 +86,17 @@ graph TB
|------|------|------|------|----------|-----|-----------|---------|-----------|-------|----------|-----|-------|
| 106 | pfv-k8s-cnode3 | tsys1 | 4 | host | 4G | D5 | D5 | qcow2 | **writethrough** | ✓ | ✓ | etcd member. |
| 705 | pfv-k8s-cnode2 | tsys7 | 4 | host | 4G | S2 | S2 | qcow2 | **writethrough** | ✓ | ✓ | etcd member. |
-| 906 | pfv-k8s-cnode1 | tsys9 | 2 | host | 4G | S3 | S3 | qcow2 | — | ✓ | — | etcd member. |
+| 906 | pfv-k8s-cnode1 | tsys9 | 2 | host | 4G | S3 | S3 | qcow2 | -- | ✓ | -- | etcd member. |
### k8s workers (wnodes)
| VMID | Name | Host | vCPU | CPU type | RAM | Disk store | Spindle | Disk type | Cache | iothread | qga | Notes |
|------|------|------|------|----------|-----|-----------|---------|-----------|-------|----------|-----|-------|
| 313 | pfv-k8s-wnode-tsys3 | tsys3 | 8 | host | 28G | D5 | D5 | qcow2 | **writethrough** | ✓ | ✓ | Compute worker. |
-| 509 | pfv-k8s-wnode-tsys5 | tsys5 | 2 | host | 32G | D2 (virtio) | D2 | qcow2 | — | ✓ | — | Undersized vCPU (2). |
-| 100 | pfv-k8s-wnode-tsys6 | tsys6 | 2 | host | 64G | S4 | S4 | qcow2 | — | ✓ | — | Undersized vCPU (2), high RAM. |
-| 701 | pfv-k8s-wnode-tsys7 | tsys7 | 4 | host | 32G | S3 (virtio) | S3 | qcow2 | — | ✓ | — | RackRental worker. |
-| 905 | pfv-k8s-wnode-tsys9 | tsys9 | 4 | v2-AES | 4G | S2 | S2 | qcow2 | — | ✓ | ✓ | Target: grow to 6c/8G. |
+| 509 | pfv-k8s-wnode-tsys5 | tsys5 | 2 | host | 32G | D2 (virtio) | D2 | qcow2 | -- | ✓ | -- | Undersized vCPU (2). |
+| 100 | pfv-k8s-wnode-tsys6 | tsys6 | 2 | host | 64G | S4 | S4 | qcow2 | -- | ✓ | -- | Undersized vCPU (2), high RAM. |
+| 701 | pfv-k8s-wnode-tsys7 | tsys7 | 4 | host | 32G | S3 (virtio) | S3 | qcow2 | -- | ✓ | -- | RackRental worker. |
+| 905 | pfv-k8s-wnode-tsys9 | tsys9 | 4 | v2-AES | 4G | S2 | S2 | qcow2 | -- | ✓ | ✓ | Target: grow to 6c/8G. |
### Application VMs
@@ -104,14 +104,14 @@ graph TB
|------|------|------|------|----------|-----|-----------|---------|-----------|-------|----------|-----|-------|
| 114 | kali-tsys | tsys1 | 2 | host | 2G | D2 | D2 | qcow2 | **writethrough** | ✓ | ✓ | Kali Linux pentest. |
| 117 | tsys-secure-workbench | tsys1 | 2 | host | 4G | D2 | D2 | qcow2 | **writethrough** | ✓ | ✓ | Secure workbench. |
-| 400 | pfv-proxmox-backup-server | tsys4 | 2 | — | 2G | local-lvm | (local) | raw | — | — | — | PBS — all VM backups. |
-| 515 | hfnoc-uisp-preprod | tsys5 | 2 | host | 2G | local-nonprod | S1 | qcow2 | — | ✓ | — | Preprod. Locked to tsys5. |
-| 5111 | ultix-streaming | tsys5 | 4 | — | 9G | SSD dir | SSD | qcow2 | — | — | — | 288GB on Samsung 860 PRO (**93% full**). |
-| 5112 | ultix-offstage | tsys5 | 4 | — | 6G | local-lvm | (local) | raw | — | — | — | 288GB. Locked to tsys5. |
+| 400 | pfv-proxmox-backup-server | tsys4 | 2 | -- | 2G | local-lvm | (local) | raw | -- | -- | -- | PBS -- all VM backups. |
+| 515 | hfnoc-uisp-preprod | tsys5 | 2 | host | 2G | local-nonprod | S1 | qcow2 | -- | ✓ | -- | Preprod. Locked to tsys5. |
+| 5111 | ultix-streaming | tsys5 | 4 | -- | 9G | SSD dir | SSD | qcow2 | -- | -- | -- | 288GB on Samsung 860 PRO (**93% full**). |
+| 5112 | ultix-offstage | tsys5 | 4 | -- | 6G | local-lvm | (local) | raw | -- | -- | -- | 288GB. Locked to tsys5. |
| 600 | tsys-awx | tsys6 | 2 | host | 12G | D5 | D5 | qcow2 | **writethrough** | ✓ | ✓ | Ansible AWX. |
-| 702 | hfnoc-uisp | tsys7 | 2 | host | 8G | S1 | S1 | qcow2 | — | ✓ | ✓ | 100GB disk. UISP network monitoring. |
+| 702 | hfnoc-uisp | tsys7 | 2 | host | 8G | S1 | S1 | qcow2 | -- | ✓ | ✓ | 100GB disk. UISP network monitoring. |
| 703 | rr-middleware | tsys7 | 2 | host | 2G | S2 | S2 | qcow2 | **writethrough** | ✓ | ✓ | RackRental middleware. |
-| 704 | TCTC | tsys7 | 4 | host | 6G | D2 (virtio) | D2 | qcow2 | — | ✓ | — | |
+| 704 | TCTC | tsys7 | 4 | host | 6G | D2 (virtio) | D2 | qcow2 | -- | ✓ | -- | |
| 901 | tsys-siem | tsys9 | 2 | host | 8G | D2 | D2 | qcow2 | **writethrough** | ✓ | ✓ | 132GB disk. SIEM. |
| 903 | kali-rd | tsys9 | 2 | host | 2G | D5 | D5 | qcow2 | **writethrough** | ✓ | ✓ | Kali R&D. |
@@ -122,7 +122,7 @@ graph TB
### 3.1 Cache mode audit
`cache=writethrough` forces synchronous writes to the NFS server on every write
-operation. For NFS-backed qcow2, this is unnecessarily slow — `writeback` or
+operation. For NFS-backed qcow2, this is unnecessarily slow -- `writeback` or
`none` is recommended (qcow2 already provides journaling integrity).
| Cache mode | Count | VMs |
@@ -153,7 +153,7 @@ CPU generations. K8s cnodes/wnodes should ideally match within each tier.
| Controller | Count | Notes |
|-----------|-------|-------|
| **scsi0** (virtio-scsi) | 22 | Optimal for Proxmox |
-| **virtio0** | 4 | 101, 509, 701, 704 — also good |
+| **virtio0** | 4 | 101, 509, 701, 704 -- also good |
| raw (no virtio) | 2 | 100 (pfv-bms), 500 (stopped) |
All running VMs use virtio-based disk controllers. No IDE/SATA legacy disks.
@@ -167,7 +167,7 @@ No legacy e1000/rtl8139 adapters.
| Status | Count | VMs without qga |
|--------|-------|-----------------|
-| **enabled** | 20 | — |
+| **enabled** | 20 | -- |
| **not set** | 8 | 101, 103, 509, 100(tsys6), 701, 704, 904, 906 |
**Fix:** Enable `qm set --agent 1` on the 8 VMs missing it. No VM
@@ -218,7 +218,7 @@ is immediate).
| VMID | Name | Host | RAM | Disk | Notes |
|------|------|------|-----|------|-------|
| 102 | pfv-k8s-wnode-tsys1 | tsys1 | 4G | (no disk) | Stopped, no boot disk |
-| 500 | DellOpenManageEnterprise | tsys5 | 20G | **D7 (gone!)** | **Would fail — disk export removed** |
+| 500 | DellOpenManageEnterprise | tsys5 | 20G | **D7 (gone!)** | **Would fail -- disk export removed** |
| 5500 | RestoreTemplate | tsys5 | 2G | D2 | Template |
| 1000 | KNELTemplate | tsys1 | 2G | D2 | Base template |
| 3000 | KNELTemplate | tsys3 | 2G | D2 | Base template copy |