mrcharles
0b54273fd5
Full VMs run on the workstation (q35 + host-passthrough + KVM): Debian cloud guest reached DHCP + ssh + running systemd. REGULAR-VMS.md holds the recipe and the three workstation gotchas (disk perms under images/, /boot-only kernels for virt-aa-helper, console=ttyS0 for serial). Details: https://projects.knownelement.com/issues/615
61 lines
3.0 KiB
Markdown
61 lines
3.0 KiB
Markdown
# REGULAR-VMS.md — full (non-micro) VMs on the workstation (#615)
|
|
|
|
**Proven 2026-09-01**: regular libvirt/qemu VMs run on this workstation
|
|
(nested KVM). Test record at the bottom. `microvm` covers the fast
|
|
disposable lane; this is the standard lane — virt-manager over RDP works
|
|
for GUI management too.
|
|
|
|
## The three workstation-specific rules (learned the hard way)
|
|
|
|
1. **Disks must live under `/var/lib/libvirt/images/reachableceo/`**
|
|
(home dirs are 0700 by house rule — `libvirt-qemu` cannot traverse
|
|
`$HOME`). Dir is `770 reachableceo:libvirt-qemu`; make new images
|
|
`660` with group `libvirt-qemu`:
|
|
chgrp libvirt-qemu FILE && chmod 660 FILE
|
|
2. **Direct-kernel-boot kernels/initrds must be under `/boot`**
|
|
(libvirt's AppArmor `virt-aa-helper` whitelist). Staged copies:
|
|
`/boot/microvm-vmlinuz`, `/boot/microvm-initrd` (from the microvm
|
|
base; refresh if the base image changes). Out-of-tree paths fail
|
|
with `error: skipped restricted file` / `cannot load AppArmor
|
|
profile`.
|
|
3. **Serial visibility needs `console=ttyS0`** — the Debian cloud image
|
|
grub does not set it. Use direct kernel boot (below) or expect no
|
|
serial output. (Quirk: even with a serial-file chardev, output was
|
|
empty in the test; `domifaddr --source lease` + ssh is the reliable
|
|
health check.)
|
|
|
|
## Working recipe (import a cloud image)
|
|
|
|
# seed: same vfat cidata trick as microvm (user-data + meta-data)
|
|
truncate -s 4M seed.img && mkfs.vfat -n CIDATA seed.img
|
|
mcopy -i seed.img user-data meta-data ::
|
|
|
|
virt-install --connect qemu:///system \
|
|
--name NAME --memory 1024 --vcpus 1 --import --os-variant debian12 \
|
|
--disk path=/var/lib/libvirt/images/reachableceo/NAME.qcow2,format=qcow2,bus=virtio \
|
|
--disk path=/var/lib/libvirt/images/reachableceo/seed.img,format=raw,bus=virtio,readonly=on \
|
|
--network network=default,model=virtio \
|
|
--graphics none --noautoconsole
|
|
|
|
virsh -c qemu:///system domifaddr NAME --source lease # wait for IP
|
|
ssh debian@<ip> # key from seed
|
|
|
|
Defaults in play: machine `pc-q35`, cpu `host-passthrough`, KVM accel,
|
|
`default` NAT network (192.168.122.0/24, now `autostart`ed).
|
|
For console debugging add to the domain XML `<os>`:
|
|
`<kernel>/boot/microvm-vmlinuz</kernel><initrd>/boot/microvm-initrd</initrd>
|
|
<cmdline>console=ttyS0 root=/dev/vda1 rw</cmdline>`.
|
|
|
|
## Test record (2026-09-01, #615)
|
|
|
|
- `regvm-test`: Debian 13 cloud image import, 1 vCPU/1G, q35 +
|
|
host-passthrough, direct-kernel-boot cmdline.
|
|
- Result: DHCP lease (`192.168.122.229`, hostname `regvm`) within ~60s
|
|
of start, ssh OK (key from NoCloud seed), `systemctl is-system-running`
|
|
= `running`, 6 virtio-PCI devices, root auto-grew to 9.7G.
|
|
- Nested virt: guest CPU shows E5620 with `vmx` (VMs-in-VMs possible).
|
|
- Teardown verified: destroy + undefine --remove-all-storage clean.
|
|
- Compare: microvm lane boots to ssh in ~28s first boot / ~5-8s warm;
|
|
the full libvirt lane is slower but gives q35/PCI, virt-manager,
|
|
snapshots, standard tooling.
|