go-freebsd-vmm

Why

Inspect and control virtual machines on FreeBSD from Go at the level the kernel exposes them — CPU state, registers, statistics, guest memory, suspend and resume — without running bhyve, bhyvectl or bhyveload. It is a systems binding, not a virtualization runtime: it says nothing about how a VM ought to be configured.

It’s meant to be the bottom of a stack. A higher-level runtime — go-freebsd-bhyve — starts a VM its own way, then opens it here for runtime inspection and control. The dependency only ever points down.

How

• One cgo package. import "C" appears only in internal/vmmapi, and no C type crosses out of it. The public API is ordinary Go: *VM, *VCPU, Register, CPUSet.
• A thin C bridge, not a second libvmmapi. It exists to capture errno right after each call, to keep cgo call shapes regular, and to collapse loops that would otherwise cost one cgo transition per iteration.
• Bulk calls, because transitions are expensive. Reading 14 registers one by one takes 7.3 µs and 28 allocations; one Registers call takes 1.5 µs and 6 — 4.8× faster for the same work. The statistics gather and the guest address-space walk live in C for the same reason.
• Typed VM exits. VCPU.Run enters the guest and returns the exit that brought it back, decoded into typed values; RunLoop does it repeatedly on a pinned host thread. An exit reason a future FreeBSD adds still decodes with its numeric value, so an unknown exit is something to handle, not something that breaks.
• Spurious exits are expected. The kernel leaves the guest whenever the calling thread has a signal pending, and the Go runtime signals its threads routinely — so ExitBogus is common here, and ExitCode.Spurious() is there for it. Run can’t be cancelled, which is why it takes no context.Context.
• Explicit ownership, no finalizers. A VM owns one libvmmapi context and the vCPU handles opened from it. Close keeps the VM in the kernel, Destroy removes it, and whichever runs first wins. Methods are safe to call concurrently; work on independent vCPUs doesn’t serialize.
• Errors keep their errno. errors.Is(err, syscall.ENOENT) works; misuse of the API wraps ErrClosed, ErrInvalidArgument or ErrUnsupported. Nothing panics on a kernel error.
• Ask, don’t compare versions. One build serves FreeBSD 14, 15 and CURRENT by calling only the libvmmapi 2.0 symbol set; where releases differ, vmm.Supports(feature) answers. (VMMAPI_VERSION is an octal literal — 0200 is 128 — so the version is decoded from its digits, not its value.)

Guest input is untrusted

Every field of a VM exit is guest-controlled, including instruction bytes and lengths; the decoder clamps the one length it uses as a slice bound and hands the rest over as reported. MemoryRegion.Bytes aliases guest physical memory with no synchronization against running vCPUs. Strings bound for C are checked for embedded NUL, VM names may not contain a path separator, and integer conversions across cgo are range-checked.

Testing

Three levels. Pure-Go tests run anywhere. Integration tests drive a real kernel, need root and vmm(4), and clean up every VM they create. Privileged tests enter the guest: a five-byte real-mode payload exercises VM_RUN and exit decoding without a guest operating system, and a second payload spins forever so a blocked Run returning on suspend can be observed.

All three are green on FreeBSD 14.3 amd64 — 114 pure-Go, 43 integration and 20 privileged tests.

Status

The v1.0 surface for amd64 is complete: VM and vCPU lifecycle, CPU topology, registers, guest and device memory, capabilities, statistics, guest execution with typed exits, interrupt injection, PCI passthrough, the RTC, address translation and the low-level snapshot calls. Everything except snapshots has run against a real kernel.

Known gaps: arm64 and riscv64 (the tree is ready — architecture-specific code is behind build constraints); snapshots, which need a kernel built with BHYVE_SNAPSHOT; passthrough on real hardware with a device bound to ppt(4); and CI for the kernel-level tests.

vm, err := vmm.Open("vm01")
if err != nil {
	return err
}
defer vm.Close()

cpu, err := vm.VCPU(0)
if err != nil {
	return err
}
defer cpu.Close()

rip, err := cpu.Register(vmm.RegRIP)
if err != nil {
	return err
}
fmt.Printf("RIP: %#x\n", rip)

$ git clone https://github.com/zombocoder/go-freebsd-vmm