fix: Mermaid diagram syntax + deploy-tuning.sh path + fleet quick fixes

Mermaid diagrams fixed for Gitea rendering:
- Replaced all <br/> → <br> (86 tags across 4 docs)
- Replaced Unicode chars with ASCII (em dash, multiply, middle dot, star)
- Fixed invalid bidirectional dotted arrow in NETWORK-TOPOLOGY.md
  (<--. "..." .--> is not valid Mermaid; replaced with two -.-> arrows)
- Simplified node labels for cleaner rendering

Host fixes applied (live, not in repo — documented here for audit trail):
- SSH keys standardized to exactly 2 approved keys on all 7 hosts
  (removed host-generated RSA keys, added missing VP TechOps key to tsys9)
- Packages installed: iperf3 (tsys9), sysstat (tsys5), nvme-cli (tsys4/5),
  net-tools (tsys1/6/7)
- tsys4 tuning: rmem/wmem 16MB→128MB, netdev_max_backlog 5000→250000,
  tcp_max_syn_backlog 1024→2048, tuned profile→network-throughput
- tsys5: nconnect options added to storage.cfg (3 NFS stanzas patched;
  will apply on next Proxmox NFS remount/reboot)
- noatime applied to root fs on tsys4 + tsys5 (was only tsys5)
- tsys1 lldpd: installed but systemd/dbus issue prevents enabling
  ("Transport endpoint is not connected") — needs investigation/reboot

Bug fix: deploy-tuning.sh computed wrong script path (scripts/ vs
perf/scripts/) — fixed to use dirname/scripts/apply-tunings.sh

💘 Generated with Crush

Assisted-by: Crush:glm-5.2
This commit is contained in:
2026-07-31 00:12:04 -05:00
parent 290245349f
commit 9c2882ea00
5 changed files with 207 additions and 207 deletions
+57 -57
View File
@@ -13,53 +13,53 @@
```mermaid
graph TB
subgraph tsys1["tsys1 Infra (57% RAM)"]
VM100["100 pfv-bms<br/>2c/4G D2"]
VM101["101 tsys-ca<br/>2c/2G D2"]
VM103["103 netinfra-01<br/>2c/4G S2"]
VM104["104 librenms<br/>2c/2G D2"]
VM105["105 proxmox-dc<br/>2c/2G D2"]
VM106["106 cnode3<br/>4c/4G D5"]
VM108["108 ucs-01<br/>2c/8G D2"]
VM114["114 kali-tsys<br/>2c/2G D2"]
VM117["117 workbench<br/>2c/4G D2"]
subgraph tsys1["tsys1 -- Infra (57% RAM)"]
VM100["100 pfv-bms<br>2c/4G D2"]
VM101["101 tsys-ca<br>2c/2G D2"]
VM103["103 netinfra-01<br>2c/4G S2"]
VM104["104 librenms<br>2c/2G D2"]
VM105["105 proxmox-dc<br>2c/2G D2"]
VM106["106 cnode3<br>4c/4G D5"]
VM108["108 ucs-01<br>2c/8G D2"]
VM114["114 kali-tsys<br>2c/2G D2"]
VM117["117 workbench<br>2c/4G D2"]
end
subgraph tsys3["tsys3 Compute (10% RAM)"]
VM313["313 wnode-tsys3<br/>8c/28G D5"]
subgraph tsys3["tsys3 -- Compute (10% RAM)"]
VM313["313 wnode-tsys3<br>8c/28G D5"]
end
subgraph tsys4["tsys4 Storage (15% RAM)"]
VM400["400 PBS<br/>2c/2G local-lvm"]
subgraph tsys4["tsys4 -- Storage (15% RAM)"]
VM400["400 PBS<br>2c/2G local-lvm"]
end
subgraph tsys5["tsys5 Storage+Sandbox (15% RAM)"]
VM509["509 wnode-tsys5<br/>2c/32G D2"]
VM515["515 hfnoc-uisp-preprod<br/>2c/2G local-nonprod"]
VM5111["5111 ultix-streaming<br/>4c/9G SSD"]
VM5112["5112 ultix-offstage<br/>4c/6G local-lvm"]
subgraph tsys5["tsys5 -- Storage+Sandbox (15% RAM)"]
VM509["509 wnode-tsys5<br>2c/32G D2"]
VM515["515 hfnoc-uisp-preprod<br>2c/2G local-nonprod"]
VM5111["5111 ultix-streaming<br>4c/9G SSD"]
VM5112["5112 ultix-offstage<br>4c/6G local-lvm"]
end
subgraph tsys6["tsys6 RackRental (12% RAM)"]
VM100b["100 wnode-tsys6<br/>2c/64G S4"]
VM600["600 tsys-awx<br/>2c/12G D5"]
subgraph tsys6["tsys6 -- RackRental (12% RAM)"]
VM100b["100 wnode-tsys6<br>2c/64G S4"]
VM600["600 tsys-awx<br>2c/12G D5"]
end
subgraph tsys7["tsys7 RackRental (7% RAM)"]
VM701["701 wnode-tsys7<br/>4c/32G S3"]
VM702["702 hfnoc-uisp<br/>2c/8G S1"]
VM703["703 rr-middleware<br/>2c/2G S2"]
VM704["704 TCTC<br/>4c/6G D2"]
VM705["705 cnode2<br/>4c/4G S2"]
subgraph tsys7["tsys7 -- RackRental (7% RAM)"]
VM701["701 wnode-tsys7<br>4c/32G S3"]
VM702["702 hfnoc-uisp<br>2c/8G S1"]
VM703["703 rr-middleware<br>2c/2G S2"]
VM704["704 TCTC<br>4c/6G D2"]
VM705["705 cnode2<br>4c/4G S2"]
end
subgraph tsys9["tsys9 Infra+Compute (69% RAM)"]
VM901["901 tsys-siem<br/>2c/8G D2"]
VM902["902 ucs-02<br/>2c/8G S2"]
VM903["903 kali-rd<br/>2c/2G D5"]
VM904["904 netinfra-02<br/>2c/4G D2"]
VM905["905 wnode-tsys9<br/>4c/4G S2"]
VM906["906 cnode1<br/>2c/4G S3"]
subgraph tsys9["tsys9 -- Infra+Compute (69% RAM)"]
VM901["901 tsys-siem<br>2c/8G D2"]
VM902["902 ucs-02<br>2c/8G S2"]
VM903["903 kali-rd<br>2c/2G D5"]
VM904["904 netinfra-02<br>2c/4G D2"]
VM905["905 wnode-tsys9<br>4c/4G S2"]
VM906["906 cnode1<br>2c/4G S3"]
end
```
@@ -71,14 +71,14 @@ graph TB
| VMID | Name | Host | vCPU | CPU type | RAM | Disk store | Spindle | Disk type | Cache | iothread | qga | Notes |
|------|------|------|------|----------|-----|-----------|---------|-----------|-------|----------|-----|-------|
| 100 | pfv-bms (HomeAssistant) | tsys1 | 2 | host | 4G | D2 (raw) | D2 | raw | | | ✓ | discard=on, ssd=1. HA for power/temp monitoring. |
| 101 | tsys-ca | tsys1 | 2 | v2-AES | 2G | D2 (virtio) | D2 | qcow2 | | ✓ | | no nested-virt flag. Certificate Authority. |
| 103 | pfv-netinfra-01 | tsys1 | 2 | v2-AES | 4G | S2 | S2 | qcow2 | | ✓ | | DNS primary (Technitium + Pi-hole). |
| 104 | tsys-librenms | tsys1 | 2 | host | 2G | D2 | D2 | qcow2 | | ✓ | ✓ | SNMP monitoring for switches. |
| 105 | tsys-proxmox-datacenter | tsys1 | 2 | host | 2G | D2 | D2 | qcow2 | | ✓ | ✓ | PDM (Proxmox Datacenter Manager). |
| 100 | pfv-bms (HomeAssistant) | tsys1 | 2 | host | 4G | D2 (raw) | D2 | raw | -- | -- | ✓ | discard=on, ssd=1. HA for power/temp monitoring. |
| 101 | tsys-ca | tsys1 | 2 | v2-AES | 2G | D2 (virtio) | D2 | qcow2 | -- | ✓ | -- | no nested-virt flag. Certificate Authority. |
| 103 | pfv-netinfra-01 | tsys1 | 2 | v2-AES | 4G | S2 | S2 | qcow2 | -- | ✓ | -- | DNS primary (Technitium + Pi-hole). |
| 104 | tsys-librenms | tsys1 | 2 | host | 2G | D2 | D2 | qcow2 | -- | ✓ | ✓ | SNMP monitoring for switches. |
| 105 | tsys-proxmox-datacenter | tsys1 | 2 | host | 2G | D2 | D2 | qcow2 | -- | ✓ | ✓ | PDM (Proxmox Datacenter Manager). |
| 108 | tsys-ucs-01 | tsys1 | 2 | host | 8G | D2 | D2 | qcow2 | **writethrough** | ✓ | ✓ | Univention Corporate Server half 1. |
| 902 | tsys-ucs-02 | tsys9 | 2 | host | 8G | S2 | S2 | qcow2 | | ✓ | ✓ | UCS half 2 (HA pair with 108). |
| 904 | pfv-netinfra-02 | tsys9 | 2 | v2-AES | 4G | D2 | D2 | qcow2 | | ✓ | | DNS secondary. HA pair with 103. |
| 902 | tsys-ucs-02 | tsys9 | 2 | host | 8G | S2 | S2 | qcow2 | -- | ✓ | ✓ | UCS half 2 (HA pair with 108). |
| 904 | pfv-netinfra-02 | tsys9 | 2 | v2-AES | 4G | D2 | D2 | qcow2 | -- | ✓ | -- | DNS secondary. HA pair with 103. |
### k8s control plane (cnode1/2/3)
@@ -86,17 +86,17 @@ graph TB
|------|------|------|------|----------|-----|-----------|---------|-----------|-------|----------|-----|-------|
| 106 | pfv-k8s-cnode3 | tsys1 | 4 | host | 4G | D5 | D5 | qcow2 | **writethrough** | ✓ | ✓ | etcd member. |
| 705 | pfv-k8s-cnode2 | tsys7 | 4 | host | 4G | S2 | S2 | qcow2 | **writethrough** | ✓ | ✓ | etcd member. |
| 906 | pfv-k8s-cnode1 | tsys9 | 2 | host | 4G | S3 | S3 | qcow2 | | ✓ | | etcd member. |
| 906 | pfv-k8s-cnode1 | tsys9 | 2 | host | 4G | S3 | S3 | qcow2 | -- | ✓ | -- | etcd member. |
### k8s workers (wnodes)
| VMID | Name | Host | vCPU | CPU type | RAM | Disk store | Spindle | Disk type | Cache | iothread | qga | Notes |
|------|------|------|------|----------|-----|-----------|---------|-----------|-------|----------|-----|-------|
| 313 | pfv-k8s-wnode-tsys3 | tsys3 | 8 | host | 28G | D5 | D5 | qcow2 | **writethrough** | ✓ | ✓ | Compute worker. |
| 509 | pfv-k8s-wnode-tsys5 | tsys5 | 2 | host | 32G | D2 (virtio) | D2 | qcow2 | | ✓ | | Undersized vCPU (2). |
| 100 | pfv-k8s-wnode-tsys6 | tsys6 | 2 | host | 64G | S4 | S4 | qcow2 | | ✓ | | Undersized vCPU (2), high RAM. |
| 701 | pfv-k8s-wnode-tsys7 | tsys7 | 4 | host | 32G | S3 (virtio) | S3 | qcow2 | | ✓ | | RackRental worker. |
| 905 | pfv-k8s-wnode-tsys9 | tsys9 | 4 | v2-AES | 4G | S2 | S2 | qcow2 | | ✓ | ✓ | Target: grow to 6c/8G. |
| 509 | pfv-k8s-wnode-tsys5 | tsys5 | 2 | host | 32G | D2 (virtio) | D2 | qcow2 | -- | ✓ | -- | Undersized vCPU (2). |
| 100 | pfv-k8s-wnode-tsys6 | tsys6 | 2 | host | 64G | S4 | S4 | qcow2 | -- | ✓ | -- | Undersized vCPU (2), high RAM. |
| 701 | pfv-k8s-wnode-tsys7 | tsys7 | 4 | host | 32G | S3 (virtio) | S3 | qcow2 | -- | ✓ | -- | RackRental worker. |
| 905 | pfv-k8s-wnode-tsys9 | tsys9 | 4 | v2-AES | 4G | S2 | S2 | qcow2 | -- | ✓ | ✓ | Target: grow to 6c/8G. |
### Application VMs
@@ -104,14 +104,14 @@ graph TB
|------|------|------|------|----------|-----|-----------|---------|-----------|-------|----------|-----|-------|
| 114 | kali-tsys | tsys1 | 2 | host | 2G | D2 | D2 | qcow2 | **writethrough** | ✓ | ✓ | Kali Linux pentest. |
| 117 | tsys-secure-workbench | tsys1 | 2 | host | 4G | D2 | D2 | qcow2 | **writethrough** | ✓ | ✓ | Secure workbench. |
| 400 | pfv-proxmox-backup-server | tsys4 | 2 | | 2G | local-lvm | (local) | raw | | | | PBS all VM backups. |
| 515 | hfnoc-uisp-preprod | tsys5 | 2 | host | 2G | local-nonprod | S1 | qcow2 | | ✓ | | Preprod. Locked to tsys5. |
| 5111 | ultix-streaming | tsys5 | 4 | | 9G | SSD dir | SSD | qcow2 | | | | 288GB on Samsung 860 PRO (**93% full**). |
| 5112 | ultix-offstage | tsys5 | 4 | | 6G | local-lvm | (local) | raw | | | | 288GB. Locked to tsys5. |
| 400 | pfv-proxmox-backup-server | tsys4 | 2 | -- | 2G | local-lvm | (local) | raw | -- | -- | -- | PBS -- all VM backups. |
| 515 | hfnoc-uisp-preprod | tsys5 | 2 | host | 2G | local-nonprod | S1 | qcow2 | -- | ✓ | -- | Preprod. Locked to tsys5. |
| 5111 | ultix-streaming | tsys5 | 4 | -- | 9G | SSD dir | SSD | qcow2 | -- | -- | -- | 288GB on Samsung 860 PRO (**93% full**). |
| 5112 | ultix-offstage | tsys5 | 4 | -- | 6G | local-lvm | (local) | raw | -- | -- | -- | 288GB. Locked to tsys5. |
| 600 | tsys-awx | tsys6 | 2 | host | 12G | D5 | D5 | qcow2 | **writethrough** | ✓ | ✓ | Ansible AWX. |
| 702 | hfnoc-uisp | tsys7 | 2 | host | 8G | S1 | S1 | qcow2 | | ✓ | ✓ | 100GB disk. UISP network monitoring. |
| 702 | hfnoc-uisp | tsys7 | 2 | host | 8G | S1 | S1 | qcow2 | -- | ✓ | ✓ | 100GB disk. UISP network monitoring. |
| 703 | rr-middleware | tsys7 | 2 | host | 2G | S2 | S2 | qcow2 | **writethrough** | ✓ | ✓ | RackRental middleware. |
| 704 | TCTC | tsys7 | 4 | host | 6G | D2 (virtio) | D2 | qcow2 | | ✓ | | |
| 704 | TCTC | tsys7 | 4 | host | 6G | D2 (virtio) | D2 | qcow2 | -- | ✓ | -- | |
| 901 | tsys-siem | tsys9 | 2 | host | 8G | D2 | D2 | qcow2 | **writethrough** | ✓ | ✓ | 132GB disk. SIEM. |
| 903 | kali-rd | tsys9 | 2 | host | 2G | D5 | D5 | qcow2 | **writethrough** | ✓ | ✓ | Kali R&D. |
@@ -122,7 +122,7 @@ graph TB
### 3.1 Cache mode audit
`cache=writethrough` forces synchronous writes to the NFS server on every write
operation. For NFS-backed qcow2, this is unnecessarily slow `writeback` or
operation. For NFS-backed qcow2, this is unnecessarily slow -- `writeback` or
`none` is recommended (qcow2 already provides journaling integrity).
| Cache mode | Count | VMs |
@@ -153,7 +153,7 @@ CPU generations. K8s cnodes/wnodes should ideally match within each tier.
| Controller | Count | Notes |
|-----------|-------|-------|
| **scsi0** (virtio-scsi) | 22 | Optimal for Proxmox |
| **virtio0** | 4 | 101, 509, 701, 704 also good |
| **virtio0** | 4 | 101, 509, 701, 704 -- also good |
| raw (no virtio) | 2 | 100 (pfv-bms), 500 (stopped) |
All running VMs use virtio-based disk controllers. No IDE/SATA legacy disks.
@@ -167,7 +167,7 @@ No legacy e1000/rtl8139 adapters.
| Status | Count | VMs without qga |
|--------|-------|-----------------|
| **enabled** | 20 | |
| **enabled** | 20 | -- |
| **not set** | 8 | 101, 103, 509, 100(tsys6), 701, 704, 904, 906 |
**Fix:** Enable `qm set <vmid> --agent 1` on the 8 VMs missing it. No VM
@@ -218,7 +218,7 @@ is immediate).
| VMID | Name | Host | RAM | Disk | Notes |
|------|------|------|-----|------|-------|
| 102 | pfv-k8s-wnode-tsys1 | tsys1 | 4G | (no disk) | Stopped, no boot disk |
| 500 | DellOpenManageEnterprise | tsys5 | 20G | **D7 (gone!)** | **Would fail disk export removed** |
| 500 | DellOpenManageEnterprise | tsys5 | 20G | **D7 (gone!)** | **Would fail -- disk export removed** |
| 5500 | RestoreTemplate | tsys5 | 2G | D2 | Template |
| 1000 | KNELTemplate | tsys1 | 2G | D2 | Base template |
| 3000 | KNELTemplate | tsys3 | 2G | D2 | Base template copy |