Re: [Libguestfs] Plans for changing libguestfs appliance building

Well you can all read the thread here for the technical details: http://lists.gnu.org/archive/html/qemu-devel/2010-08/threads.html#00133 but the bottom line for anyone who wasn't in that discussion or on IRC last week is that we have to change the way that the appliance is built in libguestfs so that we don't depend (as much) on the qemu -initrd option. The bad thing is this is loads of make-work at a particularly inconvenient time. The good thing is that it should make libguestfs 'boot' in under 5 seconds, even if your hardware is old and slow. It will also reduce the memory and I/O requirements for using libguestfs. Because this issue is adversely affecting people using kernel 2.6.35 and/or the latest qemu right now (and will affect many people when Fedora 14 is released which has both), this is my top priority to fix in the coming week. This email documents how I will change appliance building. The patches will follow some time later in the week when I've written and tested them. The new boot method will be this sequence: (a) (Same as now) A suitable kernel to use is located on the host, or from $libdir/guestfs in the non-supermin case. (b) A tiny initrd is built (on the fly for supermin). This will contain just these files: any CD-ROM, IDE, virtio kmods required to read the virtual CD device ext2.ko if required /sbin/modprobe a tiny init script, written in C (c) An ext2 image is created for the root filesystem. I will say a lot more about how this is created below. (d) qemu is invoked with something along these lines: qemu -kernel kernel -initrd initrd -drive file=isofile,media=cdrom where: kernel = kernel found in step (a) initrd = tiny initrd created in step (b) isofile = ext2 filesystem image created in step (c) (e) The boot process proceeds by starting the kernel, reading the /init script from the tiny initrd which mounts the filesystem from the CD-ROM device and pivot_root(2)s into it, running another /init script from that filesystem. At this point, boot continues as it does in the current libguestfs.

Since this involves attaching an extra device to the appliance, we also need to change the daemon to ignore this extra device, adding somewhat to the complexity of several operations. We believe this should improve the speed of boot greatly. Obviously there is the saving because we are no longer using the (now broken) qemu -initrd support with a large initrd, but that's purely because of a qemu regression. We also save because we don't need to unpack the initrd inside the appliance, and because the device is loaded on demand. Also there are some unrelated changes that I intend to make which will improve boot speeds. We have every reason to believe that 5 seconds will be achievable, even on relatively old hardware. Step (c) above involves creating an ext2 filesystem for root. I chose ext2 because it is considerably less complex than ISO9660, and is the native Linux filesystem so it supports all Linux features (long filenames, extended attributes and so on). Creating an ext2 filesystem is more complex than the initrd that we currently create using some hand-coded cpio-like code. Of course we can't use libguestfs to help us! We plan to write some C code to assemble the ext2 filesystem from scratch (or starting from a mke2fs blank filesystem). To further improve boot times, we intend to cache this so it will only be created the first time it is used, and only when it needs to be updated. Caching this means that most times that you use libguestfs or other tools, no appliance creation will be required at all, and only the bits of the appliance that you use will be loaded. If you have any further questions about this, please follow up on this mailing list. Rich.