mrcharles
a1beb6cd3e
Fleet-wide changes (live, documented for audit/AWX):
Persistent tuning via systemd oneshot (all 7 hosts):
- New pfv-nfs-tuning.service runs AFTER tuned.service to override
16MB TCP buffer caps with 128MB for high-BDP NFS
- New 99-pfv-nfs.conf sysctl fragment (128MB rmem/wmem, 250K backlog)
- Deployed to: tsys1, tsys3, tsys4, tsys5, tsys6, tsys7, tsys9
- Survives reboot — no more tuned clobbering our sysctl values
tsys5 fixes:
- VM 509 stopped, NFS D2/D3/D5 remounted with nconnect=4, VM 509
restarted (nconnect now live on all tsys5 NFS client mounts)
- SK hynix SSD (sdb1) persisted in fstab at /mnt/ssd2 (UUID-based)
- ssd2 added to Proxmox storage.cfg as dir storage
- Stale S3 export removed from /etc/exports (disk not mounted)
- Broken S4 dir entry removed from storage.cfg
- Duplicate noatime,noatime in fstab fixed
- version=4.2 bug in storage.cfg options fixed (invalid mount option)
tsys4 fixes:
- SSH keys re-standardized to 2 approved (upgrade regenerated host key)
- Tuning persistence working (128MB buffers confirmed live + persistent)
apply-tunings.sh fix:
- Removed version=4.2 from NFS options string (not a valid mount.nfs
option — causes mount failures; Proxmox auto-negotiates NFSv4.2)
Config files saved to provisioning/ConfigFiles/ for AWX/Ansible consumption:
- 99-pfv-nfs.conf
- pfv-nfs-tuning.service
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
PFVCluster
Unified infrastructure repo for the Known Element Enterprises Proxmox R&D cluster.
→ Current Status · → Documentation Index · → Agent Guidelines
Directory Structure
| Directory | Description |
|---|---|
provisioning/ |
Server provisioning (SetupNewSystem.sh, security hardening, 2FA, NTP/DNS config, SNMP, Dell OMSA) |
tests/ |
Test suite + VM validation harness |
dns-cluster-setup/ |
Technitium DNS cluster replication scripts |
k8s/ |
k3s cluster setup scripts (3-node HA control plane over Tailscale) + co-located docs |
powerman/ |
Cyclades PM10i PDU management via powerman on pfv-tsys1 |
console/ |
Serial console management (ser2net + conman) for network switches on pfv-tsys4 |
ups/ |
UPS management (NUT) for APC Smart-UPS C 1500 on pfv-tsys1 — upsd on Tailscale |
perf/ |
Proxmox performance tuning, fleet audit, iperf, switch diagnostics |
proxmox/ |
Proxmox fleet docs: hardware audit, capacity analysis, k8s host planning |
netinfra/ |
pfv-netinfra-01/02 DNS/NTP/DHCP setup + audit scripts |
switches/ |
Switch configuration captures |
awx/ |
Ansible AWX deployment on tsys-awx (k3s + AWX Operator 2.19.1, v24.6.1) |
docs/ |
Server-build docs, documentation index (docmap), and archive |
vendor/ |
Vendored KNELShellFramework |
Quick Start
Provision a new server
sudo bash provisioning/SetupNewSystem.sh
Validate on the sandbox VM
VM_ID=6000 ./tests/vm-validation.sh all
Deploy DNS cluster
cd dns-cluster-setup/ && ./setup.sh all
Deploy perf tunings
cd perf/ && ./deploy-tuning.sh
Architecture
- Proxmox hosts: 7 standalone PVE installs managed via PDM
- DNS: Technitium (authoritative) + Pi-hole (recursive) on pfv-netinfra-01/02
- NTP: pfv-netinfra-01/02 (redundant, LAN IPs, stratum 2/3)
- Production: Cloudron VPS in Reston VA (this cluster is R&D only)
- Backups: Proxmox Backup Server (PBS)