fix(tuning): persistent NFS tuning fleet-wide + apply-tunings.sh bug fix

Fleet-wide changes (live, documented for audit/AWX):

Persistent tuning via systemd oneshot (all 7 hosts):
- New pfv-nfs-tuning.service runs AFTER tuned.service to override
  16MB TCP buffer caps with 128MB for high-BDP NFS
- New 99-pfv-nfs.conf sysctl fragment (128MB rmem/wmem, 250K backlog)
- Deployed to: tsys1, tsys3, tsys4, tsys5, tsys6, tsys7, tsys9
- Survives reboot — no more tuned clobbering our sysctl values

tsys5 fixes:
- VM 509 stopped, NFS D2/D3/D5 remounted with nconnect=4, VM 509
  restarted (nconnect now live on all tsys5 NFS client mounts)
- SK hynix SSD (sdb1) persisted in fstab at /mnt/ssd2 (UUID-based)
- ssd2 added to Proxmox storage.cfg as dir storage
- Stale S3 export removed from /etc/exports (disk not mounted)
- Broken S4 dir entry removed from storage.cfg
- Duplicate noatime,noatime in fstab fixed
- version=4.2 bug in storage.cfg options fixed (invalid mount option)

tsys4 fixes:
- SSH keys re-standardized to 2 approved (upgrade regenerated host key)
- Tuning persistence working (128MB buffers confirmed live + persistent)

apply-tunings.sh fix:
- Removed version=4.2 from NFS options string (not a valid mount.nfs
  option — causes mount failures; Proxmox auto-negotiates NFSv4.2)

Config files saved to provisioning/ConfigFiles/ for AWX/Ansible consumption:
- 99-pfv-nfs.conf
- pfv-nfs-tuning.service

💘 Generated with Crush

Assisted-by: Crush:glm-5.2

perf/scripts/apply-tunings.sh

@@ -364,7 +364,7 @@ if [ -r /etc/pve/storage.cfg ]; then
         if grep -A1 '^nfs:' /etc/pve/storage.cfg | grep -q 'options.*nconnect=4'; then
             echo "    some stanzas already have nconnect=4 — check manually"
         else
-            echo "    proposed change: add 'options nconnect=4,noatime,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,version=4.2' to each nfs stanza"
+            echo "    proposed change: add 'options nconnect=4,noatime,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2' to each nfs stanza"
             if [ "$ACTION" = "apply" ]; then
                 # Apply via perl for safety (in-place edit with backup)
                 if cp -a /etc/pve/storage.cfg "$BACKUP_DIR/etc__pve__storage.cfg"; then
@@ -381,7 +381,7 @@ def fix(m):
     if 'options' in block:
         return block
     lines = block.rstrip('\n').split('\n')
-    lines.insert(1, '\toptions nconnect=4,noatime,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,version=4.2')
+    lines.insert(1, '\toptions nconnect=4,noatime,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2')
     modified += 1
     return '\n'.join(lines) + '\n'
 new_text = pattern.sub(fix, text)

provisioning/ConfigFiles/99-pfv-nfs.conf

@@ -0,0 +1,23 @@
+# PFV NFS tuning sysctl overrides
+#
+# Applied AFTER tuned via pfv-nfs-tuning.service (systemd oneshot).
+# These override tuned's network-throughput/virtual-host 16MB TCP buffer
+# caps with 128MB for high-BDP NFS over 1-4 GbE LACP links.
+#
+# Install on ALL Proxmox hosts:
+#   cp 99-pfv-nfs.conf /etc/sysctl.d/99-pfv-nfs.conf
+#   cp pfv-nfs-tuning.service /etc/systemd/system/pfv-nfs-tuning.service
+#   systemctl daemon-reload && systemctl enable --now pfv-nfs-tuning.service
+#
+# Created: 2026-07-31
+# Deployed: tsys1, tsys3, tsys4, tsys5, tsys6, tsys7, tsys9
+
+net.core.rmem_max = 134217728
+net.core.wmem_max = 134217728
+net.core.rmem_default = 26214400
+net.core.wmem_default = 26214400
+net.core.netdev_max_backlog = 250000
+net.core.somaxconn = 65535
+net.ipv4.tcp_rmem = 4096 87380 134217728
+net.ipv4.tcp_wmem = 4096 65536 134217728
+net.ipv4.tcp_max_syn_backlog = 4096

provisioning/ConfigFiles/pfv-nfs-tuning.service

@@ -0,0 +1,28 @@
+# PFV NFS tuning service
+#
+# Systemd oneshot that runs AFTER tuned.service to apply TCP buffer
+# overrides. The tuned daemon's profiles (network-throughput for storage
+# hosts, virtual-host for compute hosts) set 16MB TCP buffer caps which
+# are too small for high-BDP NFS over LACP links. This service force-
+# applies 128MB buffers after tuned has finished its configuration.
+#
+# Install:
+#   cp pfv-nfs-tuning.service /etc/systemd/system/pfv-nfs-tuning.service
+#   systemctl daemon-reload
+#   systemctl enable --now pfv-nfs-tuning.service
+#
+# Created: 2026-07-31
+# Deployed: all 7 Proxmox hosts (tsys1/3/4/5/6/7/9)
+
+[Unit]
+Description=PFV NFS tuning (override tuned TCP buffer caps)
+After=tuned.service
+Requires=tuned.service
+
+[Service]
+Type=oneshot
+ExecStart=/sbin/sysctl -p /etc/sysctl.d/99-pfv-nfs.conf
+RemainAfterExit=yes
+
+[Install]
+WantedBy=multi-user.target