[Libguestfs] Improving supermin appliance startup time (lkvm/qboot)

Hi, I remembered reading about Intel Clear Containers [1], and Hyper/Qboot [2] that support startup times measured in the hundreds of miliseconds range. On an AMD FX(tm)-8350 'guestfish -a /dev/null run' takes ~4s when run the 2nd/3rd time: real 0m4.152s user 0m2.120s sys 0m0.564s Are there any plans on applying these improvements to the supermin appliance? I did some quick tests on what would be possible, see below: If I try to use virtio 9P instead it becomes a little faster (instead of root fs on block device): real 0m3.299s user 0m2.084s sys 0m1.828s To do this we need to load just the virtio and 9p modules: let kmods = [ "virtio*.ko*"; "9p*.ko*"; ] Modify init.c to mount the 9p root: https://gist.github.com/anonymous/cb32986fd9b85c315ae4 And wrap qemu to create the 9p device (the mount is just there for a quick test, in reality supermin itself should probably leave the unpacker dir around): https://gist.github.com/anonymous/5cdbc18974f88d6ea9e0 Next step: kvmtool [3]. Avoids some delays introduced by legacy BIOS boot, and device probing, however lkvm currently lacks support for additional virtio-serial channels, so - although the supermin appliance would boot - guestfsd wouldn't be able to communicate with the host: https://gist.github.com/anonymous/e6b0a12e1da811f24e5b I tried QBoot [4], which requires that you first strip down your kernel+initrd to fit into 8128k, and uses an unmodified Qemu otherwise. But then I haven't been able to get it to actually launch guestfsd, it hangs somewhere earlier: https://gist.github.com/anonymous/52fc2890a0884230e4f8 Maybe I removed too many modules here? Another possibility would be to run guestfsd with rkt+lkvm stage1, or hyper.sh; but I don't see a way to forward a unix socket or a virtio-serial channel through them, would guestfsd be able to work over a TCP socket or the main virtio console? (its probably preferable to avoid having any networking in the guestfsd appliance) [1] https://lwn.net/Articles/644675/ [2] https://hyper.sh/faq.html [3] https://git.kernel.org/cgit/linux/kernel/git/will/kvmtool.git/tree/README [4] https://github.com/bonzini/qboot Best regards, -- Edwin Török | Co-founder and Lead Developer Skylable open-source object storage: reliable, fast, secure http://www.skylable.com