KNEL/PFVCluster
5
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: lock in storage philosophy and PDM migration capability

Browse Source 
Storage philosophy (user directive):
- NVMe/SSD: k8s worker scratch + ultix-streaming (dev workstation
  running "cluster of 1" pre-prod jobs before full k8s deployment)
- Spinning rust: all infrastructure VMs (UCS, netinfra, LibreNMS, SIEM)

Clarified that hosts are standalone but managed via Proxmox Datacenter
Manager (PDM), which supports VM migration between nodes through the
UI -- eliminating the need for manual disk copies in the migration plan.

Updated all migration steps to reference PDM storage migrate instead
of manual cp commands.

💘 Generated with Crush

Assisted-by: Crush:glm-5.2
This commit is contained in:
mrcharles
2026-07-27 21:46:48 -05:00
parent 6814374bac
commit 53954d8494
8 changed files with 2028 additions and 31 deletions
Download Patch File Download Diff File Expand all files Collapse all files
PROJECT.md
+14 -11
View File
@@ -187,7 +187,7 @@ NFS-over-HDD and unsuitable for wnode boot disks.
|------|-------|------------|----------|
| **NVMe** | tsys3 (Samsung PM961), tsys5 (Friday addition), **tsys2 (Samsung 960 PRO 512GB)** | 2000-3500 MB/s | HPC scratch, ETL staging, container runtime, wnode boot, etcd |
| **Local SSD** | tsys9 (PNY CS900, 136 GB), **tsys2 (Samsung 850 EVO 1TB)** | 500 MB/s | wnode boot disk, etcd |
| **NFS-SSD** | tsys5 D3 (SK hynix, **SAS post-Friday**), tsys5 T5-SSD (Samsung) | 200-400 MB/s over NFS | UCS/LDAP, latency-sensitive VMs |
| **NFS-SSD** | tsys5 D3 (SK hynix, **SAS post-Friday**), tsys5 T5-SSD (Samsung) | 200-400 MB/s over NFS | **k8s worker scratch only** (etcd, container cache, ephemeral volumes) |
| **NFS-HDD** | tsys4 D2/D5, tsys5 S1-S4 | 80-120 MB/s over NFS | Bulk data, large disks, non-critical VMs, **wnode boot on tsys6/7** |
| **Local SATA HDD** | tsys1 local-lvm | 100-150 MB/s | Host OS only |
| **USB 2.0 HDD** | tsys6/7 local-lvm (WD My Passport) | **~30-35 MB/s** | **NOTHING -- slower than NFS, do not use for VMs** |
@@ -679,19 +679,22 @@ This maps to two k8s StorageClasses:
- `local-storage`: bound to the wnode's boot/local disk (fast, ephemeral)
- `nfs-bulk`: bound to NFS exports (slow, persistent, large capacity)
### 10.4 HIGH: Use D3 SSD and T5-SSD for latency-sensitive VMs
### 10.4 HIGH: Dedicate D3 SSD exclusively to k8s scratch
Currently wasted:
- **D3 (tsys4 SK hynix SSD, 445 GB free, 0% used)** -- nobody using it
- **T5-SSD (tsys5 Samsung 860 PRO, 140 GB free)** -- only ultix-streaming
**Storage philosophy (user directive): NVMe/SSD is for k8s worker scratch
space and ultix-streaming (developer workstation running "cluster of 1"
pre-production jobs). Spinning rust hosts all other infrastructure VMs**
(UCS, netinfra, LibreNMS, SIEM, etc.).
Recommended uses for D3 (after moving ucs-01 there, 32 GB):
- Remaining D3 space (413 GB): sectestbed-k8s-cnode, sectestbed-k8s-wnode
(move from local-nonprod HDD to NFS-SSD for faster test runs)
SSD allocation after Friday:
- **D3 (tsys5 SAS, 445 GB free, 0% used)** -- dedicated to k8s scratch via
the `nfs-ssd` StorageClass (etcd, container cache, ephemeral volumes).
- **T5-SSD (tsys5 SAS, 140 GB free)** -- ultix-streaming (VM 5111) stays here
(developer workstation, runs single-node test jobs before k8s). Remaining
140 GB available for k8s.
Recommended uses for T5-SSD (after moving nothing new, ultix-streaming uses 83 GB):
- Remaining T5-SSD space (140 GB): tsys-siem (VM 901, 132 GB disk currently
on D2 HDD) would benefit from SSD for SIEM database writes.
**Deep-dive on exact k8s scratch allocation is deferred to the next session**
(K8S.md) once we know the k8s distribution, job scheduler, and workload mix.
### 10.5 MEDIUM: Restart wnode-tsys6