diff --git a/TRACKING.md b/TRACKING.md index 3b22328..7c7becf 100644 --- a/TRACKING.md +++ b/TRACKING.md @@ -13,7 +13,8 @@ One table, updated in place at each checkpoint. States: done ✅ / doing 🔄 / | OPT-7 | guest | ⏳ | #603: root growth MANUAL ONLY, GROW-ROOT-RUNBOOK.md, user-executed | | OPT-8 | repos | ✅ | PMO backpressure design note in ukrrs/docs/harness; code = ticketed | | OPT-9 | integ | ⏳ | #604: node_exporter svc + gateway pressure scrape (Q10) | -| OPT-10 | integ | ⏳ | #605: k8s join (Q11, blocked by #601) + proxmox-ctl token (Q12); GPUs #606 | +| OPT-10 | integ | ⏳ | #605: k8s join (Q11, blocked by #601) + proxmox-ctl token (Q12) | +| #606 | gpu | 🔄 | Quadro 4000 passthrough STAGED on 5111 (vfio ids + q35 + hostpci0, staged/gpu-passthrough.md); activates at the human's pre-Friday roll; nouveau-only on kernel 6.12 (no CUDA/NVENC — compute cards come with #601) | | #610 | fleet | ✅ | Open Terminal 9-account fleet (tailscale-only :30000-30008) + OpenWebUI UAT PASS | | #611 | rdp | 🔄 | Jump/iPad + 5160x2160 monitor: EGFX frame-budget fix live (12-fix-rdp-bigres.sh); freerdp UAT all-clean; human UAT pending | | #612 | dotfiles | 🔄 | kali-addendum: Kali twoline prompt + completions (kali-comforts.sh) on top of omz, p10k = fallback (KALI_PROMPT=0); awaiting UAT | diff --git a/staged/gpu-passthrough.md b/staged/gpu-passthrough.md new file mode 100644 index 0000000..53eb3fd --- /dev/null +++ b/staged/gpu-passthrough.md @@ -0,0 +1,58 @@ +# GPU passthrough — STAGED (Quadro 4000 -> VM 5111) — #606 + +Staged 2026-09-01 by crush via host ssh. Activates at the NEXT VM 5111 +start (the human's pre-Friday roll). Nothing below touches the running +VM or other guests. + +## What is staged + +**On pfv-tsys5 (host):** + +- `/etc/modprobe.d/vfio-quadro4000.conf` + - `options vfio-pci ids=10de:06dd,10de:0be5` (GPU + HDMI audio) + - `softdep nouveau/nvidia/snd_hda_intel pre: vfio-pci` + - initramfs rebuilt for all kernels. + - Result at next host boot: vfio-pci claims 03:00.0 + 03:00.1 + (IOMMU group 20, verified isolated). +- Runtime note: nouveau was unbound from 03:00.0 during staging + (driverless until next boot — host console on that card is dark + already; ssh/PVE web unaffected). + +**On VM 5111 (pending config, live at next start):** + + hostpci0: 0000:03:00,pcie=1 + machine: q35 (was default pc/i440fx; virtio disks unaffected) + +## What to expect after the roll + +- Guest: `lspci -nnk -d 10de:06dd` → NVIDIA GF100GL, kernel 6.12 binds + **nouveau** (Fermi-class). There is NO proprietary NVIDIA driver for + Fermi on kernel 6.x — no CUDA, no NVENC. Usable for KMS display / + capture (Sunshine KMS backend) and basic GL. Real GPU compute waits + for the #601 hardware (new cards get their own staging pass). +- Host console: permanently headless once vfio owns the card (this box + is ssh/PVE-web managed; console access = the pfv-tsys5 console infra). +- GPU-as-primary variant (drops std VGA): change hostpci0 to + `0000:03:00,pcie=1,x-vga=1` — only if the guest should boot WITH the + Quadro as its only display. + +## Verify after the roll + + # host (ssh root@100.87.135.12) + lspci -nnk -s 03:00 | grep "driver in use" # -> vfio-pci (both fns) + # guest + lspci -nnk -d 10de:06dd # -> nouveau + dmesg | grep -i vfio # assignment trace + +## Revert (if the roll goes sideways) + + # host: stop claiming the card + rm /etc/modprobe.d/vfio-quadro4000.conf && update-initramfs -u -k all + # VM 5111: back to pre-staging machine + no passthrough + qm set 5111 --delete hostpci0 --delete machine + # then roll VM 5111 again; host reclaims the card at its next reboot + +## Roll procedure + +Same as any 5111 roll: stop crush sessions, `qm shutdown 5111`, wait, +`qm start 5111` (onboot order=10 also covers a host reboot).