ultix/REGULAR-VMS.md

REGULAR-VMS.md — full (non-micro) VMs on the workstation (#615)

Proven 2026-09-01: regular libvirt/qemu VMs run on this workstation (nested KVM). Test record at the bottom. microvm covers the fast disposable lane; this is the standard lane — virt-manager over RDP works for GUI management too.

The three workstation-specific rules (learned the hard way)

1. Disks must live under /var/lib/libvirt/images/reachableceo/ (home dirs are 0700 by house rule — libvirt-qemu cannot traverse $HOME). Dir is 770 reachableceo:libvirt-qemu; make new images 660 with group libvirt-qemu: chgrp libvirt-qemu FILE && chmod 660 FILE
2. Direct-kernel-boot kernels/initrds must be under /boot (libvirt's AppArmor virt-aa-helper whitelist). Staged copies: /boot/microvm-vmlinuz, /boot/microvm-initrd (from the microvm base; refresh if the base image changes). Out-of-tree paths fail with error: skipped restricted file / cannot load AppArmor profile.
3. Serial visibility needs console=ttyS0 — the Debian cloud image grub does not set it. Use direct kernel boot (below) or expect no serial output. (Quirk: even with a serial-file chardev, output was empty in the test; domifaddr --source lease + ssh is the reliable health check.)

Working recipe (import a cloud image)

# seed: same vfat cidata trick as microvm (user-data + meta-data)
truncate -s 4M seed.img && mkfs.vfat -n CIDATA seed.img
mcopy -i seed.img user-data meta-data ::

virt-install --connect qemu:///system \
  --name NAME --memory 1024 --vcpus 1 --import --os-variant debian12 \
  --disk path=/var/lib/libvirt/images/reachableceo/NAME.qcow2,format=qcow2,bus=virtio \
  --disk path=/var/lib/libvirt/images/reachableceo/seed.img,format=raw,bus=virtio,readonly=on \
  --network network=default,model=virtio \
  --graphics none --noautoconsole

virsh -c qemu:///system domifaddr NAME --source lease   # wait for IP
ssh debian@<ip>                                          # key from seed

Defaults in play: machine pc-q35, cpu host-passthrough, KVM accel, default NAT network (192.168.122.0/24, now autostarted). For console debugging add to the domain XML <os>: <kernel>/boot/microvm-vmlinuz</kernel><initrd>/boot/microvm-initrd</initrd> <cmdline>console=ttyS0 root=/dev/vda1 rw</cmdline>.

Test record (2026-09-01, #615)

• regvm-test: Debian 13 cloud image import, 1 vCPU/1G, q35 + host-passthrough, direct-kernel-boot cmdline.
• Result: DHCP lease (192.168.122.229, hostname regvm) within ~60s of start, ssh OK (key from NoCloud seed), systemctl is-system-running = running, 6 virtio-PCI devices, root auto-grew to 9.7G.
• Nested virt: guest CPU shows E5620 with vmx (VMs-in-VMs possible).
• Teardown verified: destroy + undefine --remove-all-storage clean.
• Compare: microvm lane boots to ssh in ~28s first boot / ~5-8s warm; the full libvirt lane is slower but gives q35/PCI, virt-manager, snapshots, standard tooling.