Skip to content

Instantly share code, notes, and snippets.

@bjackman
Created February 20, 2025 17:11
Show Gist options
  • Save bjackman/5485b13c8bdf1a47e28787243ebb8179 to your computer and use it in GitHub Desktop.
Save bjackman/5485b13c8bdf1a47e28787243ebb8179 to your computer and use it in GitHub Desktop.

QEMU Wrappers

QEMU is a pain to use, lots of people have built wrappers for it that can be used for kernel testing.

  • At Google we wrote an internal one called go/vm-cli. It is slightly coupled with Debian, it generates an image via Debootstrap.

  • At Isovalent they wrote https://github.com/cilium/little-vm-helper which is similar but has a JSON config language and also knows how to build kernels and manage Kconfigs.

  • Andy Lutomirski wrote https://github.com/amluto/virtme/ (Google link: go/virtme) which works with your host rootfs. It does this by having custom fancy initramfs business in the boot path. Unfortunately the maintainer is not responding to pull requests (I also emailed him directly) and it doesn't work with current QEMU

  • virtme-ng is a fork of that that seems to be maintained. It also takes ownership of configuring the kernel (and can build it for you).

    I had some issues that I haven't debugged with using a gLinux host rootfs directly. However, if I built one with mkosi using this config I was able to boot into the vng-built kernel with:

    unshare -r vng --verbose --root ~/src/limmat-kernel/mkosi-rootfs/image/ --user root

  • Vagrant can do this. In theory I believe it can be used for arbitrarily complex combinations of test devices. It's horrible to set up and use unless you already know Ruby and live in Ruby dependency hell.

  • [[Virter]] aims to have a usage model much like Docker (also somewhat like Vagrant) where you have relatively declarative config files. It's basically a wrapper around libvirt.

  • https://github.com/quickemu-project/quickemu? Never tried it. Has a logo. Found it in the comments of a Reddit post about Virter lol. Doesn't really seem oriented towards OS development, has a very janky-looking setup procedure.

  • vmtest is particularly designed for running tests in a VM. The BPF CI uses this. It can boot using the host rootfs or a specified disk image. This doesn't seem to offer much for interactive testing but it is quite good for programmatic testing. One nice thing it does is conveniently succeed and shut down if your tests pass, and fail if the guest doesn't boot or whatever. (virtme-ng seems to be able to do that too in "script mode" where you pass a command to run after the -- - I haven't tried this).

  • mkt is... another one, that also containerises the dependencies and can also build images and kernels.

  • mkosi is mostly for building OS images but it also knows how to manage VMs and by default it wraps QEMU.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment