[Libguestfs] [nbdkit] Serving "temporary disks" over NBD

(This email is largely me just thinking aloud ... So ideas welcome!) For the RISC-V builders (http://fedora.riscv.rocks) I'm using an NBD disk to host the heavily used /var/lib/mock build directory. This is because the build machines have only SD-cards as local disk and SD is both slow and unreliable. The builders (NBD clients) are set up with an /etc/rc.d/rc.local file which runs these commands on boot: modprobe nbd nbd-client -b 512 nbd.server.example.com 10809 /dev/nbd0 mount /dev/nbd0 /var/lib/mock The /var/lib/mock directory is essentially temporary and doesn't need to persist across boots or even mounts. Ideally every time the client connects over NBD we'd get a fresh ext4 filesystem. Also note as there are multiple clients they must all see a freshly created, independent view of the disk. At the moment I'm using nbdkit as the server, but: (1) There is a separate server and port for each builder (client). "10809" in the command above is actually a variable port number depending on the client. (2) The filesystem is actually persistent because I'm using regular nbdkit-file-plugin backed by a block device. (3) The filesystems are very large (100G), much larger than the available memory on the server. So they have to be backed by the server's filesystem rather than being RAM disks. (This is a round-about way of saying that nbdkit-memory-plugin cannot be used). Funnily enough nbd-server despite being very inflexible in many ways actually makes this particular use-case easy because of the nbd-server -c option: https://github.com/NetworkBlockDevice/nbd/blob/cb20c16354cccf4698fde74c42... Of course I could very easily write a custom plugin just for this purpose, but could we do something in upstream nbdkit to make it easier? Currently the cow filter (unlike nbd-server -c) serves identical content to all clients. Adding a new option to make it serve a new COW to each client is possible but not an easy change to make. Or a new COW filter which works more like nbd-server -c? An upstream custom plugin which implements this as a special case? Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org