KNEL/PFVCluster
5
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: Mermaid diagram syntax + deploy-tuning.sh path + fleet quick fixes

Browse Source 
Mermaid diagrams fixed for Gitea rendering:
- Replaced all <br/> → <br> (86 tags across 4 docs)
- Replaced Unicode chars with ASCII (em dash, multiply, middle dot, star)
- Fixed invalid bidirectional dotted arrow in NETWORK-TOPOLOGY.md
  (<--. "..." .--> is not valid Mermaid; replaced with two -.-> arrows)
- Simplified node labels for cleaner rendering

Host fixes applied (live, not in repo — documented here for audit trail):
- SSH keys standardized to exactly 2 approved keys on all 7 hosts
  (removed host-generated RSA keys, added missing VP TechOps key to tsys9)
- Packages installed: iperf3 (tsys9), sysstat (tsys5), nvme-cli (tsys4/5),
  net-tools (tsys1/6/7)
- tsys4 tuning: rmem/wmem 16MB→128MB, netdev_max_backlog 5000→250000,
  tcp_max_syn_backlog 1024→2048, tuned profile→network-throughput
- tsys5: nconnect options added to storage.cfg (3 NFS stanzas patched;
  will apply on next Proxmox NFS remount/reboot)
- noatime applied to root fs on tsys4 + tsys5 (was only tsys5)
- tsys1 lldpd: installed but systemd/dbus issue prevents enabling
  ("Transport endpoint is not connected") — needs investigation/reboot

Bug fix: deploy-tuning.sh computed wrong script path (scripts/ vs
perf/scripts/) — fixed to use dirname/scripts/apply-tunings.sh

💘 Generated with Crush

Assisted-by: Crush:glm-5.2
This commit is contained in:
mrcharles
2026-07-31 00:12:04 -05:00
parent 290245349f
commit 9c2882ea00
5 changed files with 207 additions and 207 deletions
Download Patch File Download Diff File Expand all files Collapse all files
perf/deploy-tuning.sh
+1 -1
View File
@@ -4,7 +4,7 @@
# Default mode is dry-run. Pass --apply to commit. Pass --no-nfs to skip NFS section. # Default mode is dry-run. Pass --apply to commit. Pass --no-nfs to skip NFS section.
set -uo pipefail set -uo pipefail
SCRIPT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)/scripts/apply-tunings.sh" SCRIPT="$(cd "$(dirname "${BASH_SOURCE[0]}")/scripts" && pwd)/apply-tunings.sh"
SSH_OPTS=(-o BatchMode=yes -o ConnectTimeout=8 -o ServerAliveInterval=10 -o StrictHostKeyChecking=accept-new) SSH_OPTS=(-o BatchMode=yes -o ConnectTimeout=8 -o ServerAliveInterval=10 -o StrictHostKeyChecking=accept-new)
MODE="" MODE=""
EXTRA_FLAGS="" EXTRA_FLAGS=""
proxmox/docs/HOST-REFERENCE.md
+27 -27
View File
@@ -13,17 +13,17 @@
```mermaid ```mermaid
graph TB graph TB
subgraph "RACK 5 — Core / Data / Infra" subgraph "RACK 5 -- Core / Data / Infra"
T1["tsys1 — Infra<br/>i7-4770 · 32GB · 9 VMs<br/>57% RAM"] T1["tsys1 -- Infra<br>i7-4770 | 32GB | 9 VMs<br>57% RAM"]
T3["tsys3 — Compute<br/>Xeon E3-1535M v5 · 32GB · 1 VM<br/>10% RAM"] T3["tsys3 -- Compute<br>Xeon E3-1535M v5 | 32GB | 1 VM<br>10% RAM"]
T4["tsys4 — Storage+PBS<br/>Xeon E3-1246 v3 · 16GB · 1 VM<br/>15% RAM"] T4["tsys4 -- Storage+PBS<br>Xeon E3-1246 v3 | 16GB | 1 VM<br>15% RAM"]
T5["tsys5 — Storage+Sandbox<br/>Xeon E5620 · 96GB · 4 VMs<br/>15% RAM"] T5["tsys5 -- Storage+Sandbox<br>Xeon E5620 | 96GB | 4 VMs<br>15% RAM"]
T9["tsys9 — Infra+Compute<br/>i5-10500 · 24GB · 6 VMs<br/>69% RAM"] T9["tsys9 -- Infra+Compute<br>i5-10500 | 24GB | 6 VMs<br>69% RAM"]
end end
subgraph "RACK 3 — Compute / RackRental" subgraph "RACK 3 -- Compute / RackRental"
T6["tsys6 — RackRental<br/>2× Xeon E5530 · 126GB · 2 VMs<br/>12% RAM"] T6["tsys6 -- RackRental<br>2x Xeon E5530 | 126GB | 2 VMs<br>12% RAM"]
T7["tsys7 — RackRental<br/>2× Xeon E5-2630 v2 · 189GB · 5 VMs<br/>7% RAM"] T7["tsys7 -- RackRental<br>2x Xeon E5-2630 v2 | 189GB | 5 VMs<br>7% RAM"]
end end
``` ```
@@ -33,15 +33,15 @@ graph TB
| tsys3 | Xeon E3-1535M v5 Skylake (2015) | 8 | 32 GB | 9.2.5 | 7.0.14 | Compute | 1 | 10% | | tsys3 | Xeon E3-1535M v5 Skylake (2015) | 8 | 32 GB | 9.2.5 | 7.0.14 | Compute | 1 | 10% |
| tsys4 | Xeon E3-1246 v3 Haswell (2013) | 8 | 16 GB | **9.1.5** | **6.17.9** | Storage + PBS | 1 | 15% | | tsys4 | Xeon E3-1246 v3 Haswell (2013) | 8 | 16 GB | **9.1.5** | **6.17.9** | Storage + PBS | 1 | 15% |
| tsys5 | Xeon E5620 Westmere (2010) | 8 | 96 GB | 9.2.5 | 7.0.14 | Storage + Sandbox | 4 | 15% | | tsys5 | Xeon E5620 Westmere (2010) | 8 | 96 GB | 9.2.5 | 7.0.14 | Storage + Sandbox | 4 | 15% |
| tsys6 | 2× Xeon E5530 Nehalem (2009) | 16 | 126 GB | 9.2.5 | 7.0.14 | RackRental | 2 | 12% | | tsys6 | 2x Xeon E5530 Nehalem (2009) | 16 | 126 GB | 9.2.5 | 7.0.14 | RackRental | 2 | 12% |
| tsys7 | 2× Xeon E5-2630 v2 Ivy Bridge (2013) | 24 | 189 GB | 9.2.5 | 7.0.14 | RackRental | 5 | 7% | | tsys7 | 2x Xeon E5-2630 v2 Ivy Bridge (2013) | 24 | 189 GB | 9.2.5 | 7.0.14 | RackRental | 5 | 7% |
| tsys9 | i5-10500 Comet Lake (2020) | 12 | 24 GB | 9.2.5 | 7.0.14 | Infra + Compute | 6 | 69% | | tsys9 | i5-10500 Comet Lake (2020) | 12 | 24 GB | 9.2.5 | 7.0.14 | Infra + Compute | 6 | 69% |
**Retired:** tsys2 (Win10, pending rebuild), tsys8 (permanently retired) **Retired:** tsys2 (Win10, pending rebuild), tsys8 (permanently retired)
--- ---
## pfv-tsys1 — Infrastructure host ## pfv-tsys1 -- Infrastructure host
| Attribute | Value | | Attribute | Value |
|-----------|-------| |-----------|-------|
@@ -60,7 +60,7 @@ graph TB
--- ---
## pfv-tsys3 — Compute host ## pfv-tsys3 -- Compute host
| Attribute | Value | | Attribute | Value |
|-----------|-------| |-----------|-------|
@@ -70,14 +70,14 @@ graph TB
| **mgmt IP** | 192.168.2.5 (vmbr0, DHCP) | | **mgmt IP** | 192.168.2.5 (vmbr0, DHCP) |
| **Storage IP** | 10.100.100.3 (datanet via enx USB dongle, no bond) | | **Storage IP** | 10.100.100.3 (datanet via enx USB dongle, no bond) |
| **NFS mounts** | D2, D5, S2, S4, T5-SSD (all nconnect=4) | | **NFS mounts** | D2, D5, S2, S4, T5-SSD (all nconnect=4) |
| **Local storage** | local-lvm (Samsung PM961 NVMe, 349 GB, 0% used) — **best existing local storage** | | **Local storage** | local-lvm (Samsung PM961 NVMe, 349 GB, 0% used) -- **best existing local storage** |
| **Tuning** | governor=performance, swappiness=10, bbr, rmem/wmem=128MB, tuned=virtual-host | | **Tuning** | governor=performance, swappiness=10, bbr, rmem/wmem=128MB, tuned=virtual-host |
| **Bond** | None (single USB NIC for storage) | | **Bond** | None (single USB NIC for storage) |
| **Running VMs** | 313 | | **Running VMs** | 313 |
--- ---
## pfv-tsys4 — Storage server + PBS ## pfv-tsys4 -- Storage server + PBS
| Attribute | Value | | Attribute | Value |
|-----------|-------| |-----------|-------|
@@ -99,7 +99,7 @@ graph TB
--- ---
## pfv-tsys5 — Storage server + sandboxes ## pfv-tsys5 -- Storage server + sandboxes
| Attribute | Value | | Attribute | Value |
|-----------|-------| |-----------|-------|
@@ -107,7 +107,7 @@ graph TB
| **Hardware** | Dell T7500, Xeon E5620 Westmere (2010) **single CPU** (CPU2 socket empty), 8 threads, 96 GB RAM | | **Hardware** | Dell T7500, Xeon E5620 Westmere (2010) **single CPU** (CPU2 socket empty), 8 threads, 96 GB RAM |
| **PVE** | 9.2.5 / kernel 7.0.14-6-pve | | **PVE** | 9.2.5 / kernel 7.0.14-6-pve |
| **mgmt IP** | 192.168.3.179 (vmbr0, DHCP) | | **mgmt IP** | 192.168.3.179 (vmbr0, DHCP) |
| **Storage IP** | 10.100.100.5 (storagenet via bond0, **1 of 2 slaves active** — nic2 DOWN) | | **Storage IP** | 10.100.100.5 (storagenet via bond0, **1 of 2 slaves active** -- nic2 DOWN) |
| **NFS mounts** | D2, D3, D5 from tsys4 (**missing nconnect**) | | **NFS mounts** | D2, D3, D5 from tsys4 (**missing nconnect**) |
| **NFS exports** | S1 (IronWolf 1TB), S2 (IronWolf 1TB), S3 (stale), SSD (Samsung 860 PRO) | | **NFS exports** | S1 (IronWolf 1TB), S2 (IronWolf 1TB), S3 (stale), SSD (Samsung 860 PRO) |
| **Local storage** | local-lvm (Hitachi 2TB), local-nonprod (=S1), SSD dir (Samsung 860 PRO 256GB) | | **Local storage** | local-lvm (Hitachi 2TB), local-nonprod (=S1), SSD dir (Samsung 860 PRO 256GB) |
@@ -119,36 +119,36 @@ graph TB
--- ---
## pfv-tsys6 — RackRental host ## pfv-tsys6 -- RackRental host
| Attribute | Value | | Attribute | Value |
|-----------|-------| |-----------|-------|
| **Role** | RackRental: k8s wnode-tsys6 (64 GB RAM) + AWX | | **Role** | RackRental: k8s wnode-tsys6 (64 GB RAM) + AWX |
| **Hardware** | 2× Xeon E5530 Nehalem (2009), 16 threads, 126 GB RAM | | **Hardware** | 2x Xeon E5530 Nehalem (2009), 16 threads, 126 GB RAM |
| **PVE** | 9.2.5 / kernel 7.0.14-6-pve | | **PVE** | 9.2.5 / kernel 7.0.14-6-pve |
| **mgmt IP** | 192.168.3.169 (vmbr0, DHCP) | | **mgmt IP** | 192.168.3.169 (vmbr0, DHCP) |
| **Tailscale** | 100.73.35.111 | | **Tailscale** | 100.73.35.111 |
| **Storage IP** | 10.100.100.6 (storagenet via bond0, **2 of 2 slaves active**, 2G LACP) | | **Storage IP** | 10.100.100.6 (storagenet via bond0, **2 of 2 slaves active**, 2G LACP) |
| **NFS mounts** | D1, D2, D3, D5, S1, S2, S3, S4, T5-SSD (9 mounts, all nconnect=4) | | **NFS mounts** | D1, D2, D3, D5, S1, S2, S3, S4, T5-SSD (9 mounts, all nconnect=4) |
| **Local storage** | local-lvm (WD My Passport USB 2.0 HDD, 1.7 TB, **30 MB/s — do not use for VMs**) | | **Local storage** | local-lvm (WD My Passport USB 2.0 HDD, 1.7 TB, **30 MB/s -- do not use for VMs**) |
| **Tuning** | governor=performance, swappiness=10, bbr, rmem/wmem=128MB, tuned=virtual-host | | **Tuning** | governor=performance, swappiness=10, bbr, rmem/wmem=128MB, tuned=virtual-host |
| **Bond** | bond0 (2 of 2 slaves), **hash=layer3+4** (correct) | | **Bond** | bond0 (2 of 2 slaves), **hash=layer3+4** (correct) |
| **Running VMs** | 100 (wnode-tsys6, 64GB), 600 (tsys-awx) | | **Running VMs** | 100 (wnode-tsys6, 64GB), 600 (tsys-awx) |
| **Notes** | Best storage bond in fleet (2G LACP). Local storage is USB 2.0 — unusable for scratch. | | **Notes** | Best storage bond in fleet (2G LACP). Local storage is USB 2.0 -- unusable for scratch. |
--- ---
## pfv-tsys7 — RackRental host ## pfv-tsys7 -- RackRental host
| Attribute | Value | | Attribute | Value |
|-----------|-------| |-----------|-------|
| **Role** | RackRental: k8s wnode-tsys7 + hfnoc-uisp + rr-middleware + TCTC + k8s cnode2 | | **Role** | RackRental: k8s wnode-tsys7 + hfnoc-uisp + rr-middleware + TCTC + k8s cnode2 |
| **Hardware** | 2× Xeon E5-2630 v2 Ivy Bridge (2013), 24 threads, 189 GB RAM | | **Hardware** | 2x Xeon E5-2630 v2 Ivy Bridge (2013), 24 threads, 189 GB RAM |
| **PVE** | 9.2.5 / kernel 7.0.14-6-pve | | **PVE** | 9.2.5 / kernel 7.0.14-6-pve |
| **mgmt IP** | 192.168.3.55 (vmbr0, DHCP) | | **mgmt IP** | 192.168.3.55 (vmbr0, DHCP) |
| **Storage IP** | 10.100.100.7 (datanet via bond0, **2 of 2 slaves active**, 2G LACP) | | **Storage IP** | 10.100.100.7 (datanet via bond0, **2 of 2 slaves active**, 2G LACP) |
| **NFS mounts** | D1, D2, D3, D5, S1, S2, S3, S4, T5-SSD (9 mounts, all nconnect=4) | | **NFS mounts** | D1, D2, D3, D5, S1, S2, S3, S4, T5-SSD (9 mounts, all nconnect=4) |
| **Local storage** | local-lvm (WD portable USB 2.0 HDD, 1.7 TB, **30 MB/s — do not use for VMs**) | | **Local storage** | local-lvm (WD portable USB 2.0 HDD, 1.7 TB, **30 MB/s -- do not use for VMs**) |
| **Tuning** | governor=performance, swappiness=10, bbr, rmem/wmem=128MB, tuned=virtual-host | | **Tuning** | governor=performance, swappiness=10, bbr, rmem/wmem=128MB, tuned=virtual-host |
| **Bond** | bond0 (2 of 2 slaves), **hash=layer3+4** (correct) | | **Bond** | bond0 (2 of 2 slaves), **hash=layer3+4** (correct) |
| **Running VMs** | 701, 702, 703, 704, 705 | | **Running VMs** | 701, 702, 703, 704, 705 |
@@ -156,7 +156,7 @@ graph TB
--- ---
## pfv-tsys9 — Infra + Compute ## pfv-tsys9 -- Infra + Compute
| Attribute | Value | | Attribute | Value |
|-----------|-------| |-----------|-------|
@@ -166,8 +166,8 @@ graph TB
| **mgmt IP** | 192.168.3.58 (vmbr0, DHCP) | | **mgmt IP** | 192.168.3.58 (vmbr0, DHCP) |
| **Storage IP** | 10.100.100.9 (datanet via enx9c69d36a5b6c USB dongle, no bond) | | **Storage IP** | 10.100.100.9 (datanet via enx9c69d36a5b6c USB dongle, no bond) |
| **NFS mounts** | D2, D5, S2, S3, S4, T5-SSD (all nconnect=4) | | **NFS mounts** | D2, D5, S2, S3, S4, T5-SSD (all nconnect=4) |
| **Local storage** | local-lvm (PNY CS900 SSD, 136 GB, 0% used) — **excellent for scratch** | | **Local storage** | local-lvm (PNY CS900 SSD, 136 GB, 0% used) -- **excellent for scratch** |
| **Tuning** | governor=performance, swappiness=10, bbr, rmem/wmem=128MB, tuned=virtual-host | | **Tuning** | governor=performance, swappiness=10, bbr, rmem/wmem=128MB, tuned=virtual-host |
| **Bond** | None (single USB NIC for storage) | | **Bond** | None (single USB NIC for storage) |
| **Running VMs** | 901, 902, 903, 904, 905, 906 | | **Running VMs** | 901, 902, 903, 904, 905, 906 |
| **Notes** | Newest CPU (best IPC in fleet). **69% RAM — approaching 80% goal.** Target for compute growth (after siem migrates off). | | **Notes** | Newest CPU (best IPC in fleet). **69% RAM -- approaching 80% goal.** Target for compute growth (after siem migrates off). |
proxmox/docs/NETWORK-TOPOLOGY.md
+81 -81
View File
@@ -2,7 +2,7 @@
> **Complete physical + logical network reference for the PFVCluster.** > **Complete physical + logical network reference for the PFVCluster.**
> Data gathered 2026-07-30 via SNMP, LLDP, conman console captures, and direct > Data gathered 2026-07-30 via SNMP, LLDP, conman console captures, and direct
> host probes. All data is live ground truth — not documentation inherited. > host probes. All data is live ground truth -- not documentation inherited.
> >
> Companion docs: [`STORAGE-ARCHITECTURE.md`](STORAGE-ARCHITECTURE.md), > Companion docs: [`STORAGE-ARCHITECTURE.md`](STORAGE-ARCHITECTURE.md),
> [`HOST-REFERENCE.md`](HOST-REFERENCE.md), [`VM-INVENTORY.md`](VM-INVENTORY.md). > [`HOST-REFERENCE.md`](HOST-REFERENCE.md), [`VM-INVENTORY.md`](VM-INVENTORY.md).
@@ -14,31 +14,31 @@
```mermaid ```mermaid
graph TB graph TB
subgraph "RACK 5 — Core / Data / Infra" subgraph "RACK 5 - Core / Data / Infra"
CORE["pfv-core-sw01<br/>PowerConnect 5448<br/>192.168.0.12<br/>FW 2.0.0.46<br/>(mgmt + storage VLANs)"] CORE["pfv-core-sw01<br>PowerConnect 5448<br>192.168.0.12<br>(mgmt + storage VLANs)"]
TSYS1["pfv-tsys1<br/>i7-4770 Haswell<br/>Infra host"] TSYS1["pfv-tsys1<br>i7-4770 Haswell<br>Infra host"]
TSYS3["pfv-tsys3<br/>Xeon E3-1535M v5<br/>Compute"] TSYS3["pfv-tsys3<br>Xeon E3-1535M v5<br>Compute"]
TSYS4["pfv-tsys4<br/>Xeon E3-1246 v3<br/>NFS Storage + PBS"] TSYS4["pfv-tsys4<br>Xeon E3-1246 v3<br>NFS Storage + PBS"]
TSYS5["pfv-tsys5<br/>Xeon E5620 Westmere<br/>NFS Storage + Sandboxes"] TSYS5["pfv-tsys5<br>Xeon E5620 Westmere<br>NFS Storage + Sandboxes"]
TSYS9["pfv-tsys9<br/>i5-10500 Comet Lake<br/>Infra + Compute"] TSYS9["pfv-tsys9<br>i5-10500 Comet Lake<br>Infra + Compute"]
TSYS4_CONSOLE["conman + ser2net<br/>7 switch consoles<br/>(Tailscale 100.70.77.93:7890)"] TSYS4_CONSOLE["conman + ser2net<br>7 switch consoles<br>Tailscale 100.70.77.93:7890"]
TSYS1_PDU["Cyclades PM10i PDU<br/>powerman on tsys1<br/>(port 10101)"] TSYS1_PDU["Cyclades PM10i PDU<br>powerman on tsys1"]
TSYS1_UPS["APC Smart-UPS C 1500<br/>NUT on tsys1<br/>(port 3493)"] TSYS1_UPS["APC Smart-UPS C 1500<br>NUT on tsys1"]
end end
subgraph "RACK 3 — Compute / RackRental" subgraph "RACK 3 - Compute / RackRental"
TOR3STOR["pfv-tor3-stor<br/>PowerConnect 5324 (Neyland 24T)<br/>192.168.0.9<br/>FW 2.0.1.4<br/>(storage VLAN)"] TOR3STOR["pfv-tor3-stor<br>PowerConnect 5324<br>192.168.0.9<br>storage VLAN"]
TOR3MGMT["pfv-tor3-mgmt<br/>PowerConnect 5324 (Neyland 24T)<br/>192.168.0.7<br/>(mgmt VLAN)"] TOR3MGMT["pfv-tor3-mgmt<br>PowerConnect 5324<br>192.168.0.7<br>mgmt VLAN"]
TSYS6["pfv-tsys6<br/>2× Xeon E5530 Nehalem<br/>RackRental"] TSYS6["pfv-tsys6<br>2x Xeon E5530 Nehalem<br>RackRental"]
TSYS7["pfv-tsys7<br/>2× Xeon E5-2630 v2 Ivy Bridge<br/>RackRental"] TSYS7["pfv-tsys7<br>2x Xeon E5-2630 v2<br>RackRental"]
end end
%% Cross-rack trunk %% Cross-rack trunk (two one-way dotted arrows with labels)
CORE <--. "ch1 LAG: 3×1G active<br/>(g13,g14,g15)<br/>hash=layer-2-3<br/>mode=on (static)" .--> TOR3STOR CORE -. "ch1: 3x1G active (g13-15)<br>hash=layer-2-3<br>mode=on" .-> TOR3STOR
TOR3STOR -. "ch1: 4×1G<br/>(g20,g21,g22,g23)<br/>hash=layer-2-3-4<br/>mode=on (static)" .-> CORE TOR3STOR -. "ch1: 4x1G (g20-23)<br>hash=layer-2-3-4<br>mode=on" .-> CORE
%% Rack 5 connections to core %% Rack 5 connections to core
CORE --> TSYS1 CORE --> TSYS1
@@ -56,7 +56,7 @@ graph TB
### How to read this diagram ### How to read this diagram
The dashed line between core-sw01 and tor3-stor is the **cross-rack trunk** — The dashed line between core-sw01 and tor3-stor is the **cross-rack trunk** --
the most critical link in the cluster. It carries all NFS storage traffic the most critical link in the cluster. It carries all NFS storage traffic
between rack-3 compute hosts (tsys6/7) and rack-5 storage hosts (tsys4/5). between rack-3 compute hosts (tsys6/7) and rack-5 storage hosts (tsys4/5).
The trunk is a 4-port static LAG, but only 3 links are active due to a cable The trunk is a 4-port static LAG, but only 3 links are active due to a cable
@@ -70,7 +70,7 @@ mismatch (see §3 below).
|------|--------|---------|----------------| |------|--------|---------|----------------|
| **1** (default) | 192.168.0.0/22 | Management network | All switches, all hosts (vmbr0 DHCP) | | **1** (default) | 192.168.0.0/22 | Management network | All switches, all hosts (vmbr0 DHCP) |
| **1000** | 10.100.100.0/24 | Storage network (NFS) | core-sw01 + tor3-stor only (trunked) | | **1000** | 10.100.100.0/24 | Storage network (NFS) | core-sw01 + tor3-stor only (trunked) |
| — (Tailscale) | 100.x.y.z/32 (CGNAT) | Remote access / overlay | All hosts + select VMs | | -- (Tailscale) | 100.x.y.z/32 (CGNAT) | Remote access / overlay | All hosts + select VMs |
**Management IPs (VLAN 1, via DHCP reservations):** **Management IPs (VLAN 1, via DHCP reservations):**
@@ -109,7 +109,7 @@ mismatch (see §3 below).
--- ---
## 3. Cross-rack trunk detail (ch1) — the critical link ## 3. Cross-rack trunk detail (ch1) -- the critical link
This trunk carries ALL NFS storage traffic between rack 3 (tsys6/7) and This trunk carries ALL NFS storage traffic between rack 3 (tsys6/7) and
rack 5 (tsys4/5). It is the single most important network path in the cluster. rack 5 (tsys4/5). It is the single most important network path in the cluster.
@@ -117,18 +117,18 @@ rack 5 (tsys4/5). It is the single most important network path in the cluster.
```mermaid ```mermaid
graph LR graph LR
subgraph "core-sw01 (rack5)" subgraph "core-sw01 (rack5)"
G13["g13<br/>1G UP"] G13["g13<br>1G UP"]
G14["g14<br/>1G UP"] G14["g14<br>1G UP"]
G15["g15<br/>1G UP"] G15["g15<br>1G UP"]
G16["g16<br/>1G UP<br/>NOT in LAG!"] G16["g16<br>1G UP<br>NOT in LAG!"]
G17["g17<br/>DOWN<br/>in config but<br/>no cable"] G17["g17<br>DOWN<br>in config but<br>no cable"]
end end
subgraph "tor3-stor (rack3)" subgraph "tor3-stor (rack3)"
G20["g20<br/>1G UP"] G20["g20<br>1G UP"]
G21["g21<br/>1G UP"] G21["g21<br>1G UP"]
G22["g22<br/>1G UP"] G22["g22<br>1G UP"]
G23["g23<br/>1G UP"] G23["g23<br>1G UP"]
end end
G13 --- G20 G13 --- G20
@@ -142,10 +142,10 @@ graph LR
| Property | core-sw01 (rack5) | tor3-stor (rack3) | Match? | | Property | core-sw01 (rack5) | tor3-stor (rack3) | Match? |
|----------|-------------------|--------------------|--------| |----------|-------------------|--------------------|--------|
| Physical ports configured | g(13-15,17) | g(20-23) | **No — asymmetric** | | Physical ports configured | g(13-15,17) | g(20-23) | **No -- asymmetric** |
| Physical ports UP | g13, g14, g15, g16 | g20, g21, g22, g23 | — | | Physical ports UP | g13, g14, g15, g16 | g20, g21, g22, g23 | -- |
| Active in LAG | g13, g14, g15 (3) | g20, g21, g22, g23 (4) | **No — 3 vs 4** | | Active in LAG | g13, g14, g15 (3) | g20, g21, g22, g23 (4) | **No -- 3 vs 4** |
| Load-balance hash | layer-2-3 (MAC+IP) | layer-2-3-4 (MAC+IP+Port) | **No — asymmetric** | | Load-balance hash | layer-2-3 (MAC+IP) | layer-2-3-4 (MAC+IP+Port) | **No -- asymmetric** |
| Channel mode | on (static) | on (static) | Yes (both should be LACP) | | Channel mode | on (static) | on (static) | Yes (both should be LACP) |
| VLAN | access vlan 1000 | access vlan 1000 | Yes | | VLAN | access vlan 1000 | access vlan 1000 | Yes |
@@ -162,19 +162,19 @@ graph LR
one link** regardless of how many TCP connections (nconnect) are used. one link** regardless of how many TCP connections (nconnect) are used.
3. **Static mode (`on`).** Neither switch runs LACP. There is no link-failure 3. **Static mode (`on`).** Neither switch runs LACP. There is no link-failure
detection — a dead cable stays in the LAG until manually discovered. detection -- a dead cable stays in the LAG until manually discovered.
--- ---
## 4. Complete switch port map ## 4. Complete switch port map
### pfv-core-sw01 (PowerConnect 5448, 48 ports) — rack 5 ### pfv-core-sw01 (PowerConnect 5448, 48 ports) -- rack 5
| Port | Speed | Status | VLAN | Description / Connected device | | Port | Speed | Status | VLAN | Description / Connected device |
|------|-------|--------|------|-------------------------------| |------|-------|--------|------|-------------------------------|
| g1 | 1000 | UP | 1 | pfvsvrpi (Raspberry Pi) | | g1 | 1000 | UP | 1 | pfvsvrpi (Raspberry Pi) |
| g4 | 100 | UP | 1 | pfv-tsys4 (mgmt, secondary) | | g4 | 100 | UP | 1 | pfv-tsys4 (mgmt, secondary) |
| g5 | — | UP | 1 | pfv-tsys1 (mgmt) | | g5 | -- | UP | 1 | pfv-tsys1 (mgmt) |
| g6 | 100 | UP | 1 | pfvsvrpi (secondary) | | g6 | 100 | UP | 1 | pfvsvrpi (secondary) |
| g8 | 1000 | UP | 1 | pfv-tsys4-mgmt | | g8 | 1000 | UP | 1 | pfv-tsys4-mgmt |
| g9 | 1000 | UP | 1 | AP-Wallmount (UAP-AC-LR) **⚠ 2902 inErrors** | | g9 | 1000 | UP | 1 | AP-Wallmount (UAP-AC-LR) **⚠ 2902 inErrors** |
@@ -184,72 +184,72 @@ graph LR
| **g13** | **1000** | **UP** | **1000 (ch1)** | **cross-rack trunk → tor3-stor g20** | | **g13** | **1000** | **UP** | **1000 (ch1)** | **cross-rack trunk → tor3-stor g20** |
| **g14** | **1000** | **UP** | **1000 (ch1)** | **cross-rack trunk → tor3-stor g21** | | **g14** | **1000** | **UP** | **1000 (ch1)** | **cross-rack trunk → tor3-stor g21** |
| **g15** | **1000** | **UP** | **1000 (ch1)** | **cross-rack trunk → tor3-stor g22** | | **g15** | **1000** | **UP** | **1000 (ch1)** | **cross-rack trunk → tor3-stor g22** |
| **g16** | **1000** | **UP** | **1000** | **cross-rack trunk cable (NOT in LAG — should be g17)** | | **g16** | **1000** | **UP** | **1000** | **cross-rack trunk cable (NOT in LAG -- should be g17)** |
| **g17** | — | **DOWN** | **1000 (ch1)** | **in LAG config but no cable** | | **g17** | -- | **DOWN** | **1000 (ch1)** | **in LAG config but no cable** |
| g19 | 1000 | UP | 1 | pfv-tsys5-mgmt | | g19 | 1000 | UP | 1 | pfv-tsys5-mgmt |
| g21 | 1000 | UP | 1 | (unknown) | | g21 | 1000 | UP | 1 | (unknown) |
| g23 | 100 | UP | 1 | pfv-tsys9 (mgmt) | | g23 | 100 | UP | 1 | pfv-tsys9 (mgmt) |
| g26 | 1000 | UP | 1 | AP-Tabletop **⚠ 73 inErrors** | | g26 | 1000 | UP | 1 | AP-Tabletop **⚠ 73 inErrors** |
| g27 | 1000 | UP | 1000 | pfv-tsys3-stor (datanet) | | g27 | 1000 | UP | 1000 | pfv-tsys3-stor (datanet) |
| g31 | 1000 | UP | 1000 | pfv-tsys4-stor (datanet, USB) | | g31 | 1000 | UP | 1000 | pfv-tsys4-stor (datanet, USB) |
| g32 | — | — | — | pfv-tsys4-storage (configured, DOWN) | | g32 | -- | -- | -- | pfv-tsys4-storage (configured, DOWN) |
| g33 | — | — | — | pfv-tsys5-stor (ch2, DOWN/Not Present) | | g33 | -- | -- | -- | pfv-tsys5-stor (ch2, DOWN/Not Present) |
| g34 | 1000 | UP | 1000 (ch3) | pfv-tsys5-stor (bond0, 1 link active) | | g34 | 1000 | UP | 1000 (ch3) | pfv-tsys5-stor (bond0, 1 link active) |
| g38 | 1000 | UP | 1000 | pfv-tsys9-stor (datanet) **⚠ 590 inErrors** | | g38 | 1000 | UP | 1000 | pfv-tsys9-stor (datanet) **⚠ 590 inErrors** |
| g40 | 1000 | UP | 1000 | pfv-tsys1-stor (datanet) | | g40 | 1000 | UP | 1000 | pfv-tsys1-stor (datanet) |
| g41 | 1000 | UP | 1 | (unknown) | | g41 | 1000 | UP | 1 | (unknown) |
| g43 | 1000 | UP | 1 | pfv-tsys3 (mgmt) | | g43 | 1000 | UP | 1 | pfv-tsys3 (mgmt) |
| g44 | 1000 | UP | 1 | (unknown) | | g44 | 1000 | UP | 1 | (unknown) |
| g45-g48 | — | DOWN | — | (Combo ports, unused) | | g45-g48 | -- | DOWN | -- | (Combo ports, unused) |
**Port-channels on core-sw01:** **Port-channels on core-sw01:**
| Channel | Speed | Status | Members | VLAN | Notes | | Channel | Speed | Status | Members | VLAN | Notes |
|---------|-------|--------|---------|------|-------| |---------|-------|--------|---------|------|-------|
| ch1 | 1000 | UP | g13-15 (active), g17 (inactive) | 1000 | Cross-rack trunk to tor3-stor | | ch1 | 1000 | UP | g13-15 (active), g17 (inactive) | 1000 | Cross-rack trunk to tor3-stor |
| ch2 | — | DOWN | g32-33 (non-candidate) | — | Unused | | ch2 | -- | DOWN | g32-33 (non-candidate) | -- | Unused |
| ch3 | 1000 | UP | g34 (active) | 1000 | tsys5 storage bond (degraded, 1 of 2 links) | | ch3 | 1000 | UP | g34 (active) | 1000 | tsys5 storage bond (degraded, 1 of 2 links) |
**LAG hash:** `port-channel load-balance layer-2-3` **LAG hash:** `port-channel load-balance layer-2-3`
### pfv-tor3-stor (PowerConnect 5324 / Neyland 24T, 24 ports) — rack 3 ### pfv-tor3-stor (PowerConnect 5324 / Neyland 24T, 24 ports) -- rack 3
| Port | Speed | Status | VLAN | Description / Connected device | | Port | Speed | Status | VLAN | Description / Connected device |
|------|-------|--------|------|-------------------------------| |------|-------|--------|------|-------------------------------|
| g1 | 1000 | UP | 1 | (unknown — uplink to tor3-mgmt or router) | | g1 | 1000 | UP | 1 | (unknown -- uplink to tor3-mgmt or router) |
| g2-g7 | 1000 | UP | 1 | (unused but UP) | | g2-g7 | 1000 | UP | 1 | (unused but UP) |
| **g8** | **1000** | **UP** | **1000 (ch2)** | **tsys6 stor nic1** | | **g8** | **1000** | **UP** | **1000 (ch2)** | **tsys6 stor nic1** |
| **g9** | **1000** | **UP** | **1000 (ch2)** | **tsys6 stor nic2** | | **g9** | **1000** | **UP** | **1000 (ch2)** | **tsys6 stor nic2** |
| **g10** | **1000** | **UP** | **1000 (ch3)** | **tsys7 stor nic1** | | **g10** | **1000** | **UP** | **1000 (ch3)** | **tsys7 stor nic1** |
| **g11** | **1000** | **UP** | **1000 (ch3)** | **tsys7 stor nic2** | | **g11** | **1000** | **UP** | **1000 (ch3)** | **tsys7 stor nic2** |
| **g13** | **1000** | **UP** | **—** | **tsys8-stor (STALE — tsys8 retired)** | | **g13** | **1000** | **UP** | **--** | **tsys8-stor (STALE -- tsys8 retired)** |
| **g14** | **1000** | **UP** | **—** | **tsys8-stor (STALE — tsys8 retired)** | | **g14** | **1000** | **UP** | **--** | **tsys8-stor (STALE -- tsys8 retired)** |
| g15-g19 | 1000 | UP | 1000 | (unused storage ports) | | g15-g19 | 1000 | UP | 1000 | (unused storage ports) |
| **g20** | **1000** | **UP** | **1000 (ch1)** | **cross-rack trunk → core-sw01 g13** | | **g20** | **1000** | **UP** | **1000 (ch1)** | **cross-rack trunk → core-sw01 g13** |
| **g21** | **1000** | **UP** | **1000 (ch1)** | **cross-rack trunk → core-sw01 g14** | | **g21** | **1000** | **UP** | **1000 (ch1)** | **cross-rack trunk → core-sw01 g14** |
| **g22** | **1000** | **UP** | **1000 (ch1)** | **cross-rack trunk → core-sw01 g15** | | **g22** | **1000** | **UP** | **1000 (ch1)** | **cross-rack trunk → core-sw01 g15** |
| **g23** | **1000** | **UP** | **1000 (ch1)** | **cross-rack trunk → core-sw01 g16** | | **g23** | **1000** | **UP** | **1000 (ch1)** | **cross-rack trunk → core-sw01 g16** |
| g24 | — | DOWN | 1 | (spanning-tree disabled, portfast) | | g24 | -- | DOWN | 1 | (spanning-tree disabled, portfast) |
**Port-channels on tor3-stor:** **Port-channels on tor3-stor:**
| Channel | Speed | Status | Members | VLAN | Notes | | Channel | Speed | Status | Members | VLAN | Notes |
|---------|-------|--------|---------|------|-------| |---------|-------|--------|---------|------|-------|
| ch1 | 1000 | UP | g20-23 (`mode on`) | 1000 | Cross-rack trunk to core-sw01 (4×1G) | | ch1 | 1000 | UP | g20-23 (`mode on`) | 1000 | Cross-rack trunk to core-sw01 (4x1G) |
| ch2 | 1000 | UP | g8-9 (`mode auto`) | 1000 | tsys6 storage bond (2×1G LACP) | | ch2 | 1000 | UP | g8-9 (`mode auto`) | 1000 | tsys6 storage bond (2x1G LACP) |
| ch3 | 1000 | UP | g10-11 (`mode auto`) | 1000 | tsys7 storage bond (2×1G LACP) | | ch3 | 1000 | UP | g10-11 (`mode auto`) | 1000 | tsys7 storage bond (2x1G LACP) |
| ch4 | 1000 | UP | g13-14 (`mode on`) | 1000 | **tsys8-stor (STALE — retired host)** | | ch4 | 1000 | UP | g13-14 (`mode on`) | 1000 | **tsys8-stor (STALE -- retired host)** |
**LAG hash:** `port-channel load-balance layer-2-3-4` **LAG hash:** `port-channel load-balance layer-2-3-4`
### pfv-tor3-mgmt (PowerConnect 5324 / Neyland 24T, 24 ports) — rack 3 ### pfv-tor3-mgmt (PowerConnect 5324 / Neyland 24T, 24 ports) -- rack 3
| Port | Speed | Status | Connected device | | Port | Speed | Status | Connected device |
|------|-------|--------|-----------------| |------|-------|--------|-----------------|
| g3 | — | UP | tsys7 (mgmt nic0) | | g3 | -- | UP | tsys7 (mgmt nic0) |
| g7 | — | UP | tsys6 (mgmt nic0) | | g7 | -- | UP | tsys6 (mgmt nic0) |
(Full port map not captured — this switch carries only VLAN 1 management (Full port map not captured -- this switch carries only VLAN 1 management
traffic. Low priority for storage performance.) traffic. Low priority for storage performance.)
--- ---
@@ -258,24 +258,24 @@ traffic. Low priority for storage performance.)
| Host | mgmt NIC | Switch port | Storage NIC | Switch port | Storage bond | Bond hash | Storage link speed | | Host | mgmt NIC | Switch port | Storage NIC | Switch port | Storage bond | Bond hash | Storage link speed |
|------|----------|-------------|-------------|-------------|-------------|-----------|-------------------| |------|----------|-------------|-------------|-------------|-------------|-----------|-------------------|
| tsys1 | nic0 | core g5 | nic1 (onboard) | core g40 | **No bond** (single NIC) | — | 1G | | tsys1 | nic0 | core g5 | nic1 (onboard) | core g40 | **No bond** (single NIC) | -- | 1G |
| tsys3 | nic0 | core g43 | enx8cae4ccda774 (USB) | core g27 | **No bond** (USB dongle) | — | 1G | | tsys3 | nic0 | core g43 | enx8cae4ccda774 (USB) | core g27 | **No bond** (USB dongle) | -- | 1G |
| tsys4 | nic0 | core g8 | enx8cae4ccda926 (USB) | core g31 | bond0 (1 slave, USB) | layer2+3 | 1G | | tsys4 | nic0 | core g8 | enx8cae4ccda926 (USB) | core g31 | bond0 (1 slave, USB) | layer2+3 | 1G |
| tsys5 | nic0 | core g19 | nic1 + nic2 | core ch3 (g34) | bond0 (1 of 2 slaves) | layer2+3 | **1G (degraded)** | | tsys5 | nic0 | core g19 | nic1 + nic2 | core ch3 (g34) | bond0 (1 of 2 slaves) | layer2+3 | **1G (degraded)** |
| tsys6 | nic0 | tor3-mgmt g7 | nic1 + nic2 | tor3-stor g8/g9 (ch2) | bond0 (2 of 2 slaves) | **layer3+4** | **2G LACP** | | tsys6 | nic0 | tor3-mgmt g7 | nic1 + nic2 | tor3-stor g8/g9 (ch2) | bond0 (2 of 2 slaves) | **layer3+4** | **2G LACP** |
| tsys7 | nic0 | tor3-mgmt g3 | nic1 + nic2 | tor3-stor g10/g11 (ch3) | bond0 (2 of 2 slaves) | **layer3+4** | **2G LACP** | | tsys7 | nic0 | tor3-mgmt g3 | nic1 + nic2 | tor3-stor g10/g11 (ch3) | bond0 (2 of 2 slaves) | **layer3+4** | **2G LACP** |
| tsys9 | nic0 | core g23 | enx9c69d36a5b6c (USB) | core g38 | **No bond** (USB dongle) | — | 1G | | tsys9 | nic0 | core g23 | enx9c69d36a5b6c (USB) | core g38 | **No bond** (USB dongle) | -- | 1G |
### Key observations ### Key observations
- **tsys6/7 are the only hosts with working 2G LACP bonds** (layer3+4 hash, 2 - **tsys6/7 are the only hosts with working 2G LACP bonds** (layer3+4 hash, 2
active slaves). All NFS traffic from rack 3 uses these bonds. active slaves). All NFS traffic from rack 3 uses these bonds.
- **tsys1/3/4/9 use USB dongles** for storage network — single 1G, no redundancy. - **tsys1/3/4/9 use USB dongles** for storage network -- single 1G, no redundancy.
- **tsys5 bond0 is degraded** — nic2 is DOWN (no cable connected). Only nic1 is - **tsys5 bond0 is degraded** -- nic2 is DOWN (no cable connected). Only nic1 is
active via core-sw01 ch3 (g34). This is the LACP problem the operator has been active via core-sw01 ch3 (g34). This is the LACP problem the operator has been
troubleshooting. troubleshooting.
- **Bond hash inconsistency:** tsys6/7 use layer3+4 (optimal for nconnect NFS), - **Bond hash inconsistency:** tsys6/7 use layer3+4 (optimal for nconnect NFS),
tsys4/5 use layer2+3 (suboptimal — same src/dst IP pair always hashes to one tsys4/5 use layer2+3 (suboptimal -- same src/dst IP pair always hashes to one
link even with nconnect's multiple TCP connections). link even with nconnect's multiple TCP connections).
--- ---
@@ -285,17 +285,17 @@ traffic. Low priority for storage performance.)
```mermaid ```mermaid
graph TB graph TB
subgraph "Storage Servers (rack 5)" subgraph "Storage Servers (rack 5)"
T4D2["tsys4 D2<br/>WDC Red 3TB<br/>Tier 2"] T4D2["tsys4 D2<br>WDC Red 3TB<br>Tier 2"]
T4D5["tsys4 D5<br/>Hitachi 2TB<br/>Tier 2"] T4D5["tsys4 D5<br>Hitachi 2TB<br>Tier 2"]
T4D1["tsys4 D1<br/>WD 1TB USB<br/>Tier 3"] T4D1["tsys4 D1<br>WD 1TB USB<br>Tier 3"]
T4D3["tsys4 D3<br/>Seagate 500GB USB<br/>Tier 3"] T4D3["tsys4 D3<br>Seagate 500GB USB<br>Tier 3"]
T5S1["tsys5 S1<br/>IronWolf 1TB<br/>Tier 2"] T5S1["tsys5 S1<br>IronWolf 1TB<br>Tier 2"]
T5S2["tsys5 S2<br/>IronWolf 1TB<br/>Tier 2"] T5S2["tsys5 S2<br>IronWolf 1TB<br>Tier 2"]
T5SSD["tsys5 SSD<br/>Samsung 860 PRO<br/>Tier 1 (93% full)"] T5SSD["tsys5 SSD<br>Samsung 860 PRO<br>Tier 1 (93% full)"]
end end
subgraph "VLAN 1000 storage network" subgraph "VLAN 1000 storage network"
NFS["NFS v4.2<br/>nconnect=4<br/>rsize/wsize=1MB"] NFS["NFS v4.2<br>nconnect=4<br>rsize/wsize=1MB"]
end end
subgraph "NFS Clients" subgraph "NFS Clients"
@@ -327,15 +327,15 @@ graph TB
| Server | Export | tsys1 | tsys3 | tsys4 | tsys5 | tsys6 | tsys7 | tsys9 | | Server | Export | tsys1 | tsys3 | tsys4 | tsys5 | tsys6 | tsys7 | tsys9 |
|--------|--------|-------|-------|-------|-------|-------|-------|-------| |--------|--------|-------|-------|-------|-------|-------|-------|-------|
| tsys4 | D1 | — | — | — | — | ✓ | ✓ | — | | tsys4 | D1 | -- | -- | -- | -- | ✓ | ✓ | -- |
| tsys4 | **D2** | ✓ | ✓ | — | ✓ | ✓ | ✓ | ✓ | | tsys4 | **D2** | ✓ | ✓ | -- | ✓ | ✓ | ✓ | ✓ |
| tsys4 | D3 | — | — | — | ✓ | ✓ | ✓ | — | | tsys4 | D3 | -- | -- | -- | ✓ | ✓ | ✓ | -- |
| tsys4 | D5 | — | ✓ | — | ✓ | ✓ | ✓ | ✓ | | tsys4 | D5 | -- | ✓ | -- | ✓ | ✓ | ✓ | ✓ |
| tsys5 | S1 | ✓ | — | — | — | ✓ | ✓ | — | | tsys5 | S1 | ✓ | -- | -- | -- | ✓ | ✓ | -- |
| tsys5 | S2 | ✓ | ✓ | — | — | ✓ | ✓ | — | | tsys5 | S2 | ✓ | ✓ | -- | -- | ✓ | ✓ | -- |
| tsys5 | S3 | ✓ | — | — | — | ✓ | ✓ | ✓ | | tsys5 | S3 | ✓ | -- | -- | -- | ✓ | ✓ | ✓ |
| tsys5 | S4 | ✓ | ✓ | — | — | ✓ | ✓ | ✓ | | tsys5 | S4 | ✓ | ✓ | -- | -- | ✓ | ✓ | ✓ |
| tsys5 | T5-SSD | — | ✓ | — | — | ✓ | ✓ | ✓ | | tsys5 | T5-SSD | -- | ✓ | -- | -- | ✓ | ✓ | ✓ |
(✓ = NFS client mount active. tsys4 = NFS server only, no client mounts. (✓ = NFS client mount active. tsys4 = NFS server only, no client mounts.
tsys5 = NFS server + 3 client mounts from tsys4, **missing nconnect**.) tsys5 = NFS server + 3 client mounts from tsys4, **missing nconnect**.)
@@ -350,7 +350,7 @@ tsys5 = NFS server + 3 client mounts from tsys4, **missing nconnect**.)
| **Console (conman)** | 7 serial consoles via ser2net + conman on pfv-tsys4. Conman server at `pfv-tsys4:7890` (Tailscale). Escape: `&.` | | **Console (conman)** | 7 serial consoles via ser2net + conman on pfv-tsys4. Conman server at `pfv-tsys4:7890` (Tailscale). Escape: `&.` |
| **Console tool** | `perf/scripts/conman-console.py` (PTY-based driver, portable) | | **Console tool** | `perf/scripts/conman-console.py` (PTY-based driver, portable) |
| **SNMP audit tool** | `perf/scripts/snmp-switch-audit.py` (config-driven) | | **SNMP audit tool** | `perf/scripts/snmp-switch-audit.py` (config-driven) |
| **Direct telnet** | `telnet pfv-tsys4 200X` — ONLY when conmand stopped (conflicts). Use conman instead. | | **Direct telnet** | `telnet pfv-tsys4 200X` -- ONLY when conmand stopped (conflicts). Use conman instead. |
### Console port assignments ### Console port assignments
proxmox/docs/STORAGE-ARCHITECTURE.md
+41 -41
View File
@@ -28,16 +28,16 @@ is acceptable.
## 2. Physical disk inventory ## 2. Physical disk inventory
### tsys4 — NFS storage server + PBS (7 disks) ### tsys4 -- NFS storage server + PBS (7 disks)
| Device | Model | Size | RPM | Interface | Mount | NFS ID | Tier | SMART | Used | | Device | Model | Size | RPM | Interface | Mount | NFS ID | Tier | SMART | Used |
|--------|-------|------|-----|-----------|-------|--------|------|-------|------| |--------|-------|------|-----|-----------|-------|--------|------|-------|------|
| sda | Hitachi HDS723020BLA642 | 2.0 TB | 7200 | SATA | /mnt/albert | (not exported) | **Tier 2** | PASSED, 0 reallocated | 0% | | sda | Hitachi HDS723020BLA642 | 2.0 TB | 7200 | SATA | /mnt/albert | (not exported) | **Tier 2** | PASSED, 0 reallocated | 0% |
| sdb | HITACHI HUA723020ALA640 | 2.0 TB | 7200 | SATA | /mnt/tsys4/D5 | **D5** | Tier 2 | PASSED, 0 reallocated | 13% (220 GB) | | sdb | HITACHI HUA723020ALA640 | 2.0 TB | 7200 | SATA | /mnt/tsys4/D5 | **D5** | Tier 2 | PASSED, 0 reallocated | 13% (220 GB) |
| sdc | WDC WD30EFRX (Red) | 3.0 TB | 5400 | SATA | /mnt/tsys4/D2 | **D2** | Tier 2 | PASSED, 0 reallocated | 7% (170 GB) | | sdc | WDC WD30EFRX (Red) | 3.0 TB | 5400 | SATA | /mnt/tsys4/D2 | **D2** | Tier 2 | PASSED, 0 reallocated | 7% (170 GB) |
| sdd | WDC WD10EZRX | 1.0 TB | — | SATA | (pve boot/swap/lvm) | — | Tier 2 | PASSED, 1 reallocated (sdf has 1) | OS disk | | sdd | WDC WD10EZRX | 1.0 TB | -- | SATA | (pve boot/swap/lvm) | -- | Tier 2 | PASSED, 1 reallocated (sdf has 1) | OS disk |
| sde | ST500LM021 (Seagate) | 500 GB | 7200 | **USB** | /mnt/tsys4/D3 | **D3** | **Tier 3** | PASSED | 0% | | sde | ST500LM021 (Seagate) | 500 GB | 7200 | **USB** | /mnt/tsys4/D3 | **D3** | **Tier 3** | PASSED | 0% |
| sdf | WDC WD10EARS | 1.0 TB | — | **USB** | /mnt/tsys4/D1 | **D1** | **Tier 3** | PASSED, 1 reallocated sector | 0% | | sdf | WDC WD10EARS | 1.0 TB | -- | **USB** | /mnt/tsys4/D1 | **D1** | **Tier 3** | PASSED, 1 reallocated sector | 0% |
| sdg | WDC WD50NDZW (easystore) | 5.0 TB | 5400 | **USB** | /mnt/backup | (not exported) | **Tier 3** | PASSED | 0% | | sdg | WDC WD50NDZW (easystore) | 5.0 TB | 5400 | **USB** | /mnt/backup | (not exported) | **Tier 3** | PASSED | 0% |
**Changes from previous audit:** **Changes from previous audit:**
@@ -46,25 +46,25 @@ is acceptable.
- **/mnt/backup added:** WD 5TB USB easystore. Bulk backup only. - **/mnt/backup added:** WD 5TB USB easystore. Bulk backup only.
- **D7 removed:** Previously held DellOpenManageEnterprise. Now gone. - **D7 removed:** Previously held DellOpenManageEnterprise. Now gone.
### tsys5 — NFS storage server + sandboxes (7 disks) ### tsys5 -- NFS storage server + sandboxes (7 disks)
| Device | Model | Size | RPM | Interface | Mount | NFS/Local ID | Tier | SMART | Used | | Device | Model | Size | RPM | Interface | Mount | NFS/Local ID | Tier | SMART | Used |
|--------|-------|------|-----|-----------|-------|-------------|------|-------|------| |--------|-------|------|-----|-----------|-------|-------------|------|-------|------|
| sda | ST1000VN002 (IronWolf) | 1.0 TB | 5900 | SATA | **(not mounted)** | was S3 | Tier 2 | PASSED | — | | sda | ST1000VN002 (IronWolf) | 1.0 TB | 5900 | SATA | **(not mounted)** | was S3 | Tier 2 | PASSED | -- |
| sdb | SK hynix SC300B | 512 GB | SSD | SATA | **(not mounted)** | — | **Tier 1** | PASSED | **NEW, not configured** | | sdb | SK hynix SC300B | 512 GB | SSD | SATA | **(not mounted)** | -- | **Tier 1** | PASSED | **NEW, not configured** |
| sdc | HITACHI HUS724020ALA640 | 2.0 TB | 7200 | SAS | (pve boot/swap/lvm) | — | Tier 2 | PASSED | OS disk | | sdc | HITACHI HUS724020ALA640 | 2.0 TB | 7200 | SAS | (pve boot/swap/lvm) | -- | Tier 2 | PASSED | OS disk |
| sdd | Samsung SSD 860 PRO | 256 GB | SSD | SAS | /mnt/pfv-tsys5/ssd | `ssd` / `SSD` dir (local) | **Tier 1** | PASSED | **93% full!** | | sdd | Samsung SSD 860 PRO | 256 GB | SSD | SAS | /mnt/pfv-tsys5/ssd | `ssd` / `SSD` dir (local) | **Tier 1** | PASSED | **93% full!** |
| sde | ST1000VN002 (IronWolf) | 1.0 TB | 5900 | SAS | /mnt/pfv-tsys5/S1 | **S1** + `local-nonprod` | Tier 2 | PASSED | 11% (96 GB) | | sde | ST1000VN002 (IronWolf) | 1.0 TB | 5900 | SAS | /mnt/pfv-tsys5/S1 | **S1** + `local-nonprod` | Tier 2 | PASSED | 11% (96 GB) |
| sdf | ST1000VN002 (IronWolf) | 1.0 TB | 5900 | SAS | /mnt/pfv-tsys5/S2 | **S2** | Tier 2 | PASSED | 4% (32 GB) | | sdf | ST1000VN002 (IronWolf) | 1.0 TB | 5900 | SAS | /mnt/pfv-tsys5/S2 | **S2** | Tier 2 | PASSED | 4% (32 GB) |
| **nvme0n1** | **CT500P1SSD8 (Crucial)** | **500 GB** | **NVMe** | **PCIe** | **(not mounted)** | — | **Tier 1** | PASSED, 0 integrity errors, 34K hrs | **NEW, not configured** | | **nvme0n1** | **CT500P1SSD8 (Crucial)** | **500 GB** | **NVMe** | **PCIe** | **(not mounted)** | -- | **Tier 1** | PASSED, 0 integrity errors, 34K hrs | **NEW, not configured** |
**Changes from previous audit:** **Changes from previous audit:**
- **NVMe added:** Crucial CT500P1 500GB. True NVMe — highest-performance local storage in the fleet. Not mounted, not in fstab, not in storage.cfg. - **NVMe added:** Crucial CT500P1 500GB. True NVMe -- highest-performance local storage in the fleet. Not mounted, not in fstab, not in storage.cfg.
- **SK hynix SSD relocated:** Moved from tsys4 USB enclosure to tsys5 internal SATA. Not mounted or configured. - **SK hynix SSD relocated:** Moved from tsys4 USB enclosure to tsys5 internal SATA. Not mounted or configured.
- **S3 unmounted:** sda (was S3) is present but fstab entry is commented out. Stale export remains in /etc/exports. - **S3 unmounted:** sda (was S3) is present but fstab entry is commented out. Stale export remains in /etc/exports.
- **S4 gone:** Old Toshiba 458GB no longer present. fstab entry commented. storage.cfg has broken `dir: S4` entry. - **S4 gone:** Old Toshiba 458GB no longer present. fstab entry commented. storage.cfg has broken `dir: S4` entry.
- **S1/S2 drives changed:** Both are now ST1000VN002 IronWolf 1TB (5900 rpm). - **S1/S2 drives changed:** Both are now ST1000VN002 IronWolf 1TB (5900 rpm).
- **SSD nearly full:** Samsung 860 PRO at 93% (206/234 GB) — needs attention. - **SSD nearly full:** Samsung 860 PRO at 93% (206/234 GB) -- needs attention.
- **Boot disk changed:** HITACHI HUS724020ALA640 2TB (was Hitachi 2TB in old audit, now confirmed model). - **Boot disk changed:** HITACHI HUS724020ALA640 2TB (was Hitachi 2TB in old audit, now confirmed model).
--- ---
@@ -73,21 +73,21 @@ is acceptable.
```mermaid ```mermaid
graph LR graph LR
subgraph "tsys4 — NFS Server" subgraph "tsys4 -- NFS Server"
D2["D2: /mnt/tsys4/D2<br/>WDC Red 3TB<br/>Tier 2 · 7% used"] D2["D2: /mnt/tsys4/D2<br>WDC Red 3TB<br>Tier 2 | 7% used"]
D5["D5: /mnt/tsys4/D5<br/>Hitachi 2TB<br/>Tier 2 · 13% used"] D5["D5: /mnt/tsys4/D5<br>Hitachi 2TB<br>Tier 2 | 13% used"]
D1["D1: /mnt/tsys4/D1<br/>WD 1TB USB<br/>Tier 3 · 0% used"] D1["D1: /mnt/tsys4/D1<br>WD 1TB USB<br>Tier 3 | 0% used"]
D3["D3: /mnt/tsys4/D3<br/>Seagate 500GB USB<br/>Tier 3 · 0% used"] D3["D3: /mnt/tsys4/D3<br>Seagate 500GB USB<br>Tier 3 | 0% used"]
end end
subgraph "tsys5 — NFS Server" subgraph "tsys5 -- NFS Server"
S1["S1: /mnt/pfv-tsys5/S1<br/>IronWolf 1TB<br/>Tier 2 · 11% used"] S1["S1: /mnt/pfv-tsys5/S1<br>IronWolf 1TB<br>Tier 2 | 11% used"]
S2["S2: /mnt/pfv-tsys5/S2<br/>IronWolf 1TB<br/>Tier 2 · 4% used"] S2["S2: /mnt/pfv-tsys5/S2<br>IronWolf 1TB<br>Tier 2 | 4% used"]
S3stale["S3: (stale export)<br/>disk not mounted"] S3stale["S3: (stale export)<br>disk not mounted"]
end end
subgraph "All hosts — NFS Clients" subgraph "All hosts -- NFS Clients"
MNT["/mnt/pve/D1 D2 D3 D5<br/>/mnt/pve/S1 S2 S3 S4<br/>/mnt/pve/T5-SSD"] MNT["/mnt/pve/D1 D2 D3 D5<br>/mnt/pve/S1 S2 S3 S4<br>/mnt/pve/T5-SSD"]
end end
D2 -->|NFS v4.2| MNT D2 -->|NFS v4.2| MNT
@@ -111,7 +111,7 @@ graph LR
| tsys5 | /mnt/pfv-tsys5/S2 | S2 | NFSv4.2 | nconnect=4,noatime,rsize=1048576,wsize=1048576,hard | | tsys5 | /mnt/pfv-tsys5/S2 | S2 | NFSv4.2 | nconnect=4,noatime,rsize=1048576,wsize=1048576,hard |
| tsys5 | /mnt/pfv-tsys5/S3 | S3 | NFSv4.2 | **stale** (disk not mounted on server) | | tsys5 | /mnt/pfv-tsys5/S3 | S3 | NFSv4.2 | **stale** (disk not mounted on server) |
| tsys5 | /mnt/pfv-tsys5/ssd | T5-SSD | NFSv4.2 | nconnect=4,noatime,rsize=1048576,wsize=1048576,hard | | tsys5 | /mnt/pfv-tsys5/ssd | T5-SSD | NFSv4.2 | nconnect=4,noatime,rsize=1048576,wsize=1048576,hard |
| tsys5 | /mnt/pfv-tsys5/S4 | S4 | — | **broken** (disk gone, storage.cfg entry stale) | | tsys5 | /mnt/pfv-tsys5/S4 | S4 | -- | **broken** (disk gone, storage.cfg entry stale) |
> **Note:** tsys5 is the only host where NFS client mounts (D2/D3/D5 from > **Note:** tsys5 is the only host where NFS client mounts (D2/D3/D5 from
> tsys4) are **missing nconnect=4**. All other client hosts have nconnect on > tsys4) are **missing nconnect=4**. All other client hosts have nconnect on
@@ -122,27 +122,27 @@ graph LR
## 4. Per-spindle VM placement (running VMs only) ## 4. Per-spindle VM placement (running VMs only)
> Shows every running VM's primary disk and the physical spindle it lives on. > Shows every running VM's primary disk and the physical spindle it lives on.
> This is the IOPS distribution map — the key to storage balancing. > This is the IOPS distribution map -- the key to storage balancing.
```mermaid ```mermaid
graph TB graph TB
subgraph "tsys4 spindles" subgraph "tsys4 spindles"
D2["D2 — WDC Red 3TB<br/>★★★ 13 VMs — WORST HOTSPOT"] D2["D2 -- WDC Red 3TB<br>*** 13 VMs -- WORST HOTSPOT"]
D5["D5 — Hitachi 2TB<br/>4 VMs"] D5["D5 -- Hitachi 2TB<br>4 VMs"]
D1["D1 — WD 1TB USB<br/>0 VMs (empty)"] D1["D1 -- WD 1TB USB<br>0 VMs (empty)"]
D3["D3 — Seagate 500GB USB<br/>0 VMs (empty)"] D3["D3 -- Seagate 500GB USB<br>0 VMs (empty)"]
ALB["/mnt/albert — Hitachi 2TB<br/>0 VMs (not in Proxmox)"] ALB["/mnt/albert -- Hitachi 2TB<br>0 VMs (not in Proxmox)"]
end end
subgraph "tsys5 spindles" subgraph "tsys5 spindles"
S1["S1 — IronWolf 1TB<br/>1 NFS VM + 12 local stopped"] S1["S1 -- IronWolf 1TB<br>1 NFS VM + 12 local stopped"]
S2["S2 — IronWolf 1TB<br/>5 VMs"] S2["S2 -- IronWolf 1TB<br>5 VMs"]
SSD["SSD — Samsung 860 PRO<br/>1 VM (93% full!)"] SSD["SSD -- Samsung 860 PRO<br>1 VM (93% full!)"]
NVME["NVMe — Crucial CT500P1<br/>NOT CONFIGURED"] NVME["NVMe -- Crucial CT500P1<br>NOT CONFIGURED"]
HYNIX["SK hynix SSD 512GB<br/>NOT CONFIGURED"] HYNIX["SK hynix SSD 512GB<br>NOT CONFIGURED"]
end end
D2 --- D2vms["100 101 104 105 108<br/>114 117 509 600 702<br/>704 901 904"] D2 --- D2vms["100 101 104 105 108<br>114 117 509 600 702<br>704 901 904"]
D5 --- D5vms["106 313 600(d0) 903"] D5 --- D5vms["106 313 600(d0) 903"]
S1 --- S1vms["702 hfnoc-uisp"] S1 --- S1vms["702 hfnoc-uisp"]
S2 --- S2vms["103 703 705 902 905"] S2 --- S2vms["103 703 705 902 905"]
@@ -158,11 +158,11 @@ graph TB
| S1 (IronWolf 1TB) | tsys5 | S1 + local-nonprod | Tier 2 | 1 (NFS) + 0 (local, 12 stopped) | 702 | Low; 12 stopped sandboxes on local-nonprod | | S1 (IronWolf 1TB) | tsys5 | S1 + local-nonprod | Tier 2 | 1 (NFS) + 0 (local, 12 stopped) | 702 | Low; 12 stopped sandboxes on local-nonprod |
| S2 (IronWolf 1TB) | tsys5 | S2 | Tier 2 | 5 | 103,703,705,902,905 | Moderate | | S2 (IronWolf 1TB) | tsys5 | S2 | Tier 2 | 5 | 103,703,705,902,905 | Moderate |
| ssd (Samsung 860 PRO) | tsys5 | SSD dir | Tier 1 | 1 | 5111 | **93% full** | | ssd (Samsung 860 PRO) | tsys5 | SSD dir | Tier 1 | 1 | 5111 | **93% full** |
| D1 (WD 1TB USB) | tsys4 | D1 | Tier 3 | 0 | — | Empty | | D1 (WD 1TB USB) | tsys4 | D1 | Tier 3 | 0 | -- | Empty |
| D3 (Seagate 500GB USB) | tsys4 | D3 | Tier 3 | 0 | — | Empty | | D3 (Seagate 500GB USB) | tsys4 | D3 | Tier 3 | 0 | -- | Empty |
| /mnt/albert (Hitachi 2TB) | tsys4 | (none) | Tier 2 | 0 | — | Not in Proxmox | | /mnt/albert (Hitachi 2TB) | tsys4 | (none) | Tier 2 | 0 | -- | Not in Proxmox |
| NVMe (CT500P1 500GB) | tsys5 | (none) | Tier 1 | 0 | — | **Not configured** | | NVMe (CT500P1 500GB) | tsys5 | (none) | Tier 1 | 0 | -- | **Not configured** |
| SK hynix SSD (512GB) | tsys5 | (none) | Tier 1 | 0 | — | **Not configured** | | SK hynix SSD (512GB) | tsys5 | (none) | Tier 1 | 0 | -- | **Not configured** |
### Local-only VM disks (not on NFS) ### Local-only VM disks (not on NFS)
@@ -188,8 +188,8 @@ graph TB
| S1 (IronWolf 1TB) | 916 GB | 96 GB | 774 GB | 11% | | S1 (IronWolf 1TB) | 916 GB | 96 GB | 774 GB | 11% |
| S2 (IronWolf 1TB) | 916 GB | 32 GB | 838 GB | 4% | | S2 (IronWolf 1TB) | 916 GB | 32 GB | 838 GB | 4% |
| SSD (Samsung 860 PRO) | 234 GB | 206 GB | **16 GB** | **93%** | | SSD (Samsung 860 PRO) | 234 GB | 206 GB | **16 GB** | **93%** |
| NVMe (CT500P1) | 466 GB | — | — | Not configured | | NVMe (CT500P1) | 466 GB | -- | -- | Not configured |
| SK hynix SSD | 477 GB | — | — | Not configured | | SK hynix SSD | 477 GB | -- | -- | Not configured |
**Total Tier 2 available (NFS production):** ~5.5 TB unused across D2/D5/S1/S2 **Total Tier 2 available (NFS production):** ~5.5 TB unused across D2/D5/S1/S2
**Total Tier 3 available (USB):** ~6 TB unused across D1/D3/backup **Total Tier 3 available (USB):** ~6 TB unused across D1/D3/backup
proxmox/docs/VM-INVENTORY.md
+57 -57
View File
@@ -13,53 +13,53 @@
```mermaid ```mermaid
graph TB graph TB
subgraph tsys1["tsys1 — Infra (57% RAM)"] subgraph tsys1["tsys1 -- Infra (57% RAM)"]
VM100["100 pfv-bms<br/>2c/4G D2"] VM100["100 pfv-bms<br>2c/4G D2"]
VM101["101 tsys-ca<br/>2c/2G D2"] VM101["101 tsys-ca<br>2c/2G D2"]
VM103["103 netinfra-01<br/>2c/4G S2"] VM103["103 netinfra-01<br>2c/4G S2"]
VM104["104 librenms<br/>2c/2G D2"] VM104["104 librenms<br>2c/2G D2"]
VM105["105 proxmox-dc<br/>2c/2G D2"] VM105["105 proxmox-dc<br>2c/2G D2"]
VM106["106 cnode3<br/>4c/4G D5"] VM106["106 cnode3<br>4c/4G D5"]
VM108["108 ucs-01<br/>2c/8G D2"] VM108["108 ucs-01<br>2c/8G D2"]
VM114["114 kali-tsys<br/>2c/2G D2"] VM114["114 kali-tsys<br>2c/2G D2"]
VM117["117 workbench<br/>2c/4G D2"] VM117["117 workbench<br>2c/4G D2"]
end end
subgraph tsys3["tsys3 — Compute (10% RAM)"] subgraph tsys3["tsys3 -- Compute (10% RAM)"]
VM313["313 wnode-tsys3<br/>8c/28G D5"] VM313["313 wnode-tsys3<br>8c/28G D5"]
end end
subgraph tsys4["tsys4 — Storage (15% RAM)"] subgraph tsys4["tsys4 -- Storage (15% RAM)"]
VM400["400 PBS<br/>2c/2G local-lvm"] VM400["400 PBS<br>2c/2G local-lvm"]
end end
subgraph tsys5["tsys5 — Storage+Sandbox (15% RAM)"] subgraph tsys5["tsys5 -- Storage+Sandbox (15% RAM)"]
VM509["509 wnode-tsys5<br/>2c/32G D2"] VM509["509 wnode-tsys5<br>2c/32G D2"]
VM515["515 hfnoc-uisp-preprod<br/>2c/2G local-nonprod"] VM515["515 hfnoc-uisp-preprod<br>2c/2G local-nonprod"]
VM5111["5111 ultix-streaming<br/>4c/9G SSD"] VM5111["5111 ultix-streaming<br>4c/9G SSD"]
VM5112["5112 ultix-offstage<br/>4c/6G local-lvm"] VM5112["5112 ultix-offstage<br>4c/6G local-lvm"]
end end
subgraph tsys6["tsys6 — RackRental (12% RAM)"] subgraph tsys6["tsys6 -- RackRental (12% RAM)"]
VM100b["100 wnode-tsys6<br/>2c/64G S4"] VM100b["100 wnode-tsys6<br>2c/64G S4"]
VM600["600 tsys-awx<br/>2c/12G D5"] VM600["600 tsys-awx<br>2c/12G D5"]
end end
subgraph tsys7["tsys7 — RackRental (7% RAM)"] subgraph tsys7["tsys7 -- RackRental (7% RAM)"]
VM701["701 wnode-tsys7<br/>4c/32G S3"] VM701["701 wnode-tsys7<br>4c/32G S3"]
VM702["702 hfnoc-uisp<br/>2c/8G S1"] VM702["702 hfnoc-uisp<br>2c/8G S1"]
VM703["703 rr-middleware<br/>2c/2G S2"] VM703["703 rr-middleware<br>2c/2G S2"]
VM704["704 TCTC<br/>4c/6G D2"] VM704["704 TCTC<br>4c/6G D2"]
VM705["705 cnode2<br/>4c/4G S2"] VM705["705 cnode2<br>4c/4G S2"]
end end
subgraph tsys9["tsys9 — Infra+Compute (69% RAM)"] subgraph tsys9["tsys9 -- Infra+Compute (69% RAM)"]
VM901["901 tsys-siem<br/>2c/8G D2"] VM901["901 tsys-siem<br>2c/8G D2"]
VM902["902 ucs-02<br/>2c/8G S2"] VM902["902 ucs-02<br>2c/8G S2"]
VM903["903 kali-rd<br/>2c/2G D5"] VM903["903 kali-rd<br>2c/2G D5"]
VM904["904 netinfra-02<br/>2c/4G D2"] VM904["904 netinfra-02<br>2c/4G D2"]
VM905["905 wnode-tsys9<br/>4c/4G S2"] VM905["905 wnode-tsys9<br>4c/4G S2"]
VM906["906 cnode1<br/>2c/4G S3"] VM906["906 cnode1<br>2c/4G S3"]
end end
``` ```
@@ -71,14 +71,14 @@ graph TB
| VMID | Name | Host | vCPU | CPU type | RAM | Disk store | Spindle | Disk type | Cache | iothread | qga | Notes | | VMID | Name | Host | vCPU | CPU type | RAM | Disk store | Spindle | Disk type | Cache | iothread | qga | Notes |
|------|------|------|------|----------|-----|-----------|---------|-----------|-------|----------|-----|-------| |------|------|------|------|----------|-----|-----------|---------|-----------|-------|----------|-----|-------|
| 100 | pfv-bms (HomeAssistant) | tsys1 | 2 | host | 4G | D2 (raw) | D2 | raw | — | — | ✓ | discard=on, ssd=1. HA for power/temp monitoring. | | 100 | pfv-bms (HomeAssistant) | tsys1 | 2 | host | 4G | D2 (raw) | D2 | raw | -- | -- | ✓ | discard=on, ssd=1. HA for power/temp monitoring. |
| 101 | tsys-ca | tsys1 | 2 | v2-AES | 2G | D2 (virtio) | D2 | qcow2 | — | ✓ | — | no nested-virt flag. Certificate Authority. | | 101 | tsys-ca | tsys1 | 2 | v2-AES | 2G | D2 (virtio) | D2 | qcow2 | -- | ✓ | -- | no nested-virt flag. Certificate Authority. |
| 103 | pfv-netinfra-01 | tsys1 | 2 | v2-AES | 4G | S2 | S2 | qcow2 | — | ✓ | — | DNS primary (Technitium + Pi-hole). | | 103 | pfv-netinfra-01 | tsys1 | 2 | v2-AES | 4G | S2 | S2 | qcow2 | -- | ✓ | -- | DNS primary (Technitium + Pi-hole). |
| 104 | tsys-librenms | tsys1 | 2 | host | 2G | D2 | D2 | qcow2 | — | ✓ | ✓ | SNMP monitoring for switches. | | 104 | tsys-librenms | tsys1 | 2 | host | 2G | D2 | D2 | qcow2 | -- | ✓ | ✓ | SNMP monitoring for switches. |
| 105 | tsys-proxmox-datacenter | tsys1 | 2 | host | 2G | D2 | D2 | qcow2 | — | ✓ | ✓ | PDM (Proxmox Datacenter Manager). | | 105 | tsys-proxmox-datacenter | tsys1 | 2 | host | 2G | D2 | D2 | qcow2 | -- | ✓ | ✓ | PDM (Proxmox Datacenter Manager). |
| 108 | tsys-ucs-01 | tsys1 | 2 | host | 8G | D2 | D2 | qcow2 | **writethrough** | ✓ | ✓ | Univention Corporate Server half 1. | | 108 | tsys-ucs-01 | tsys1 | 2 | host | 8G | D2 | D2 | qcow2 | **writethrough** | ✓ | ✓ | Univention Corporate Server half 1. |
| 902 | tsys-ucs-02 | tsys9 | 2 | host | 8G | S2 | S2 | qcow2 | — | ✓ | ✓ | UCS half 2 (HA pair with 108). | | 902 | tsys-ucs-02 | tsys9 | 2 | host | 8G | S2 | S2 | qcow2 | -- | ✓ | ✓ | UCS half 2 (HA pair with 108). |
| 904 | pfv-netinfra-02 | tsys9 | 2 | v2-AES | 4G | D2 | D2 | qcow2 | — | ✓ | — | DNS secondary. HA pair with 103. | | 904 | pfv-netinfra-02 | tsys9 | 2 | v2-AES | 4G | D2 | D2 | qcow2 | -- | ✓ | -- | DNS secondary. HA pair with 103. |
### k8s control plane (cnode1/2/3) ### k8s control plane (cnode1/2/3)
@@ -86,17 +86,17 @@ graph TB
|------|------|------|------|----------|-----|-----------|---------|-----------|-------|----------|-----|-------| |------|------|------|------|----------|-----|-----------|---------|-----------|-------|----------|-----|-------|
| 106 | pfv-k8s-cnode3 | tsys1 | 4 | host | 4G | D5 | D5 | qcow2 | **writethrough** | ✓ | ✓ | etcd member. | | 106 | pfv-k8s-cnode3 | tsys1 | 4 | host | 4G | D5 | D5 | qcow2 | **writethrough** | ✓ | ✓ | etcd member. |
| 705 | pfv-k8s-cnode2 | tsys7 | 4 | host | 4G | S2 | S2 | qcow2 | **writethrough** | ✓ | ✓ | etcd member. | | 705 | pfv-k8s-cnode2 | tsys7 | 4 | host | 4G | S2 | S2 | qcow2 | **writethrough** | ✓ | ✓ | etcd member. |
| 906 | pfv-k8s-cnode1 | tsys9 | 2 | host | 4G | S3 | S3 | qcow2 | — | ✓ | — | etcd member. | | 906 | pfv-k8s-cnode1 | tsys9 | 2 | host | 4G | S3 | S3 | qcow2 | -- | ✓ | -- | etcd member. |
### k8s workers (wnodes) ### k8s workers (wnodes)
| VMID | Name | Host | vCPU | CPU type | RAM | Disk store | Spindle | Disk type | Cache | iothread | qga | Notes | | VMID | Name | Host | vCPU | CPU type | RAM | Disk store | Spindle | Disk type | Cache | iothread | qga | Notes |
|------|------|------|------|----------|-----|-----------|---------|-----------|-------|----------|-----|-------| |------|------|------|------|----------|-----|-----------|---------|-----------|-------|----------|-----|-------|
| 313 | pfv-k8s-wnode-tsys3 | tsys3 | 8 | host | 28G | D5 | D5 | qcow2 | **writethrough** | ✓ | ✓ | Compute worker. | | 313 | pfv-k8s-wnode-tsys3 | tsys3 | 8 | host | 28G | D5 | D5 | qcow2 | **writethrough** | ✓ | ✓ | Compute worker. |
| 509 | pfv-k8s-wnode-tsys5 | tsys5 | 2 | host | 32G | D2 (virtio) | D2 | qcow2 | — | ✓ | — | Undersized vCPU (2). | | 509 | pfv-k8s-wnode-tsys5 | tsys5 | 2 | host | 32G | D2 (virtio) | D2 | qcow2 | -- | ✓ | -- | Undersized vCPU (2). |
| 100 | pfv-k8s-wnode-tsys6 | tsys6 | 2 | host | 64G | S4 | S4 | qcow2 | — | ✓ | — | Undersized vCPU (2), high RAM. | | 100 | pfv-k8s-wnode-tsys6 | tsys6 | 2 | host | 64G | S4 | S4 | qcow2 | -- | ✓ | -- | Undersized vCPU (2), high RAM. |
| 701 | pfv-k8s-wnode-tsys7 | tsys7 | 4 | host | 32G | S3 (virtio) | S3 | qcow2 | — | ✓ | — | RackRental worker. | | 701 | pfv-k8s-wnode-tsys7 | tsys7 | 4 | host | 32G | S3 (virtio) | S3 | qcow2 | -- | ✓ | -- | RackRental worker. |
| 905 | pfv-k8s-wnode-tsys9 | tsys9 | 4 | v2-AES | 4G | S2 | S2 | qcow2 | — | ✓ | ✓ | Target: grow to 6c/8G. | | 905 | pfv-k8s-wnode-tsys9 | tsys9 | 4 | v2-AES | 4G | S2 | S2 | qcow2 | -- | ✓ | ✓ | Target: grow to 6c/8G. |
### Application VMs ### Application VMs
@@ -104,14 +104,14 @@ graph TB
|------|------|------|------|----------|-----|-----------|---------|-----------|-------|----------|-----|-------| |------|------|------|------|----------|-----|-----------|---------|-----------|-------|----------|-----|-------|
| 114 | kali-tsys | tsys1 | 2 | host | 2G | D2 | D2 | qcow2 | **writethrough** | ✓ | ✓ | Kali Linux pentest. | | 114 | kali-tsys | tsys1 | 2 | host | 2G | D2 | D2 | qcow2 | **writethrough** | ✓ | ✓ | Kali Linux pentest. |
| 117 | tsys-secure-workbench | tsys1 | 2 | host | 4G | D2 | D2 | qcow2 | **writethrough** | ✓ | ✓ | Secure workbench. | | 117 | tsys-secure-workbench | tsys1 | 2 | host | 4G | D2 | D2 | qcow2 | **writethrough** | ✓ | ✓ | Secure workbench. |
| 400 | pfv-proxmox-backup-server | tsys4 | 2 | — | 2G | local-lvm | (local) | raw | — | — | — | PBS — all VM backups. | | 400 | pfv-proxmox-backup-server | tsys4 | 2 | -- | 2G | local-lvm | (local) | raw | -- | -- | -- | PBS -- all VM backups. |
| 515 | hfnoc-uisp-preprod | tsys5 | 2 | host | 2G | local-nonprod | S1 | qcow2 | — | ✓ | — | Preprod. Locked to tsys5. | | 515 | hfnoc-uisp-preprod | tsys5 | 2 | host | 2G | local-nonprod | S1 | qcow2 | -- | ✓ | -- | Preprod. Locked to tsys5. |
| 5111 | ultix-streaming | tsys5 | 4 | — | 9G | SSD dir | SSD | qcow2 | — | — | — | 288GB on Samsung 860 PRO (**93% full**). | | 5111 | ultix-streaming | tsys5 | 4 | -- | 9G | SSD dir | SSD | qcow2 | -- | -- | -- | 288GB on Samsung 860 PRO (**93% full**). |
| 5112 | ultix-offstage | tsys5 | 4 | — | 6G | local-lvm | (local) | raw | — | — | — | 288GB. Locked to tsys5. | | 5112 | ultix-offstage | tsys5 | 4 | -- | 6G | local-lvm | (local) | raw | -- | -- | -- | 288GB. Locked to tsys5. |
| 600 | tsys-awx | tsys6 | 2 | host | 12G | D5 | D5 | qcow2 | **writethrough** | ✓ | ✓ | Ansible AWX. | | 600 | tsys-awx | tsys6 | 2 | host | 12G | D5 | D5 | qcow2 | **writethrough** | ✓ | ✓ | Ansible AWX. |
| 702 | hfnoc-uisp | tsys7 | 2 | host | 8G | S1 | S1 | qcow2 | — | ✓ | ✓ | 100GB disk. UISP network monitoring. | | 702 | hfnoc-uisp | tsys7 | 2 | host | 8G | S1 | S1 | qcow2 | -- | ✓ | ✓ | 100GB disk. UISP network monitoring. |
| 703 | rr-middleware | tsys7 | 2 | host | 2G | S2 | S2 | qcow2 | **writethrough** | ✓ | ✓ | RackRental middleware. | | 703 | rr-middleware | tsys7 | 2 | host | 2G | S2 | S2 | qcow2 | **writethrough** | ✓ | ✓ | RackRental middleware. |
| 704 | TCTC | tsys7 | 4 | host | 6G | D2 (virtio) | D2 | qcow2 | — | ✓ | — | | | 704 | TCTC | tsys7 | 4 | host | 6G | D2 (virtio) | D2 | qcow2 | -- | ✓ | -- | |
| 901 | tsys-siem | tsys9 | 2 | host | 8G | D2 | D2 | qcow2 | **writethrough** | ✓ | ✓ | 132GB disk. SIEM. | | 901 | tsys-siem | tsys9 | 2 | host | 8G | D2 | D2 | qcow2 | **writethrough** | ✓ | ✓ | 132GB disk. SIEM. |
| 903 | kali-rd | tsys9 | 2 | host | 2G | D5 | D5 | qcow2 | **writethrough** | ✓ | ✓ | Kali R&D. | | 903 | kali-rd | tsys9 | 2 | host | 2G | D5 | D5 | qcow2 | **writethrough** | ✓ | ✓ | Kali R&D. |
@@ -122,7 +122,7 @@ graph TB
### 3.1 Cache mode audit ### 3.1 Cache mode audit
`cache=writethrough` forces synchronous writes to the NFS server on every write `cache=writethrough` forces synchronous writes to the NFS server on every write
operation. For NFS-backed qcow2, this is unnecessarily slow — `writeback` or operation. For NFS-backed qcow2, this is unnecessarily slow -- `writeback` or
`none` is recommended (qcow2 already provides journaling integrity). `none` is recommended (qcow2 already provides journaling integrity).
| Cache mode | Count | VMs | | Cache mode | Count | VMs |
@@ -153,7 +153,7 @@ CPU generations. K8s cnodes/wnodes should ideally match within each tier.
| Controller | Count | Notes | | Controller | Count | Notes |
|-----------|-------|-------| |-----------|-------|-------|
| **scsi0** (virtio-scsi) | 22 | Optimal for Proxmox | | **scsi0** (virtio-scsi) | 22 | Optimal for Proxmox |
| **virtio0** | 4 | 101, 509, 701, 704 — also good | | **virtio0** | 4 | 101, 509, 701, 704 -- also good |
| raw (no virtio) | 2 | 100 (pfv-bms), 500 (stopped) | | raw (no virtio) | 2 | 100 (pfv-bms), 500 (stopped) |
All running VMs use virtio-based disk controllers. No IDE/SATA legacy disks. All running VMs use virtio-based disk controllers. No IDE/SATA legacy disks.
@@ -167,7 +167,7 @@ No legacy e1000/rtl8139 adapters.
| Status | Count | VMs without qga | | Status | Count | VMs without qga |
|--------|-------|-----------------| |--------|-------|-----------------|
| **enabled** | 20 | — | | **enabled** | 20 | -- |
| **not set** | 8 | 101, 103, 509, 100(tsys6), 701, 704, 904, 906 | | **not set** | 8 | 101, 103, 509, 100(tsys6), 701, 704, 904, 906 |
**Fix:** Enable `qm set <vmid> --agent 1` on the 8 VMs missing it. No VM **Fix:** Enable `qm set <vmid> --agent 1` on the 8 VMs missing it. No VM
@@ -218,7 +218,7 @@ is immediate).
| VMID | Name | Host | RAM | Disk | Notes | | VMID | Name | Host | RAM | Disk | Notes |
|------|------|------|-----|------|-------| |------|------|------|-----|------|-------|
| 102 | pfv-k8s-wnode-tsys1 | tsys1 | 4G | (no disk) | Stopped, no boot disk | | 102 | pfv-k8s-wnode-tsys1 | tsys1 | 4G | (no disk) | Stopped, no boot disk |
| 500 | DellOpenManageEnterprise | tsys5 | 20G | **D7 (gone!)** | **Would fail — disk export removed** | | 500 | DellOpenManageEnterprise | tsys5 | 20G | **D7 (gone!)** | **Would fail -- disk export removed** |
| 5500 | RestoreTemplate | tsys5 | 2G | D2 | Template | | 5500 | RestoreTemplate | tsys5 | 2G | D2 | Template |
| 1000 | KNELTemplate | tsys1 | 2G | D2 | Base template | | 1000 | KNELTemplate | tsys1 | 2G | D2 | Base template |
| 3000 | KNELTemplate | tsys3 | 2G | D2 | Base template copy | | 3000 | KNELTemplate | tsys3 | 2G | D2 | Base template copy |