GPU passthrough staged: vfio claim + q35 + hostpci0 on VM 5111 (#606)

vfio-pci owns 10de:06dd/10de:0be5 at next boot (IOMMU group 20 clean);
VM 5111 gains machine q35 + hostpci0 pcie=1, live at the human's
pre-Friday roll. Revert + verification steps in staged/gpu-passthrough.md.

Details: https://projects.knownelement.com/issues/606
This commit is contained in:
2026-09-01 13:42:11 -05:00
parent 46fe0cb6a5
commit 534c9340b8
2 changed files with 60 additions and 1 deletions
+2 -1
View File
@@ -13,7 +13,8 @@ One table, updated in place at each checkpoint. States: done ✅ / doing 🔄 /
| OPT-7 | guest | ⏳ | #603: root growth MANUAL ONLY, GROW-ROOT-RUNBOOK.md, user-executed | | OPT-7 | guest | ⏳ | #603: root growth MANUAL ONLY, GROW-ROOT-RUNBOOK.md, user-executed |
| OPT-8 | repos | ✅ | PMO backpressure design note in ukrrs/docs/harness; code = ticketed | | OPT-8 | repos | ✅ | PMO backpressure design note in ukrrs/docs/harness; code = ticketed |
| OPT-9 | integ | ⏳ | #604: node_exporter svc + gateway pressure scrape (Q10) | | OPT-9 | integ | ⏳ | #604: node_exporter svc + gateway pressure scrape (Q10) |
| OPT-10 | integ | ⏳ | #605: k8s join (Q11, blocked by #601) + proxmox-ctl token (Q12); GPUs #606 | | OPT-10 | integ | ⏳ | #605: k8s join (Q11, blocked by #601) + proxmox-ctl token (Q12) |
| #606 | gpu | 🔄 | Quadro 4000 passthrough STAGED on 5111 (vfio ids + q35 + hostpci0, staged/gpu-passthrough.md); activates at the human's pre-Friday roll; nouveau-only on kernel 6.12 (no CUDA/NVENC — compute cards come with #601) |
| #610 | fleet | ✅ | Open Terminal 9-account fleet (tailscale-only :30000-30008) + OpenWebUI UAT PASS | | #610 | fleet | ✅ | Open Terminal 9-account fleet (tailscale-only :30000-30008) + OpenWebUI UAT PASS |
| #611 | rdp | 🔄 | Jump/iPad + 5160x2160 monitor: EGFX frame-budget fix live (12-fix-rdp-bigres.sh); freerdp UAT all-clean; human UAT pending | | #611 | rdp | 🔄 | Jump/iPad + 5160x2160 monitor: EGFX frame-budget fix live (12-fix-rdp-bigres.sh); freerdp UAT all-clean; human UAT pending |
| #612 | dotfiles | 🔄 | kali-addendum: Kali twoline prompt + completions (kali-comforts.sh) on top of omz, p10k = fallback (KALI_PROMPT=0); awaiting UAT | | #612 | dotfiles | 🔄 | kali-addendum: Kali twoline prompt + completions (kali-comforts.sh) on top of omz, p10k = fallback (KALI_PROMPT=0); awaiting UAT |
+58
View File
@@ -0,0 +1,58 @@
# GPU passthrough — STAGED (Quadro 4000 -> VM 5111) — #606
Staged 2026-09-01 by crush via host ssh. Activates at the NEXT VM 5111
start (the human's pre-Friday roll). Nothing below touches the running
VM or other guests.
## What is staged
**On pfv-tsys5 (host):**
- `/etc/modprobe.d/vfio-quadro4000.conf`
- `options vfio-pci ids=10de:06dd,10de:0be5` (GPU + HDMI audio)
- `softdep nouveau/nvidia/snd_hda_intel pre: vfio-pci`
- initramfs rebuilt for all kernels.
- Result at next host boot: vfio-pci claims 03:00.0 + 03:00.1
(IOMMU group 20, verified isolated).
- Runtime note: nouveau was unbound from 03:00.0 during staging
(driverless until next boot — host console on that card is dark
already; ssh/PVE web unaffected).
**On VM 5111 (pending config, live at next start):**
hostpci0: 0000:03:00,pcie=1
machine: q35 (was default pc/i440fx; virtio disks unaffected)
## What to expect after the roll
- Guest: `lspci -nnk -d 10de:06dd` → NVIDIA GF100GL, kernel 6.12 binds
**nouveau** (Fermi-class). There is NO proprietary NVIDIA driver for
Fermi on kernel 6.x — no CUDA, no NVENC. Usable for KMS display /
capture (Sunshine KMS backend) and basic GL. Real GPU compute waits
for the #601 hardware (new cards get their own staging pass).
- Host console: permanently headless once vfio owns the card (this box
is ssh/PVE-web managed; console access = the pfv-tsys5 console infra).
- GPU-as-primary variant (drops std VGA): change hostpci0 to
`0000:03:00,pcie=1,x-vga=1` — only if the guest should boot WITH the
Quadro as its only display.
## Verify after the roll
# host (ssh root@100.87.135.12)
lspci -nnk -s 03:00 | grep "driver in use" # -> vfio-pci (both fns)
# guest
lspci -nnk -d 10de:06dd # -> nouveau
dmesg | grep -i vfio # assignment trace
## Revert (if the roll goes sideways)
# host: stop claiming the card
rm /etc/modprobe.d/vfio-quadro4000.conf && update-initramfs -u -k all
# VM 5111: back to pre-staging machine + no passthrough
qm set 5111 --delete hostpci0 --delete machine
# then roll VM 5111 again; host reclaims the card at its next reboot
## Roll procedure
Same as any 5111 roll: stop crush sessions, `qm shutdown 5111`, wait,
`qm start 5111` (onboot order=10 also covers a host reboot).