[Libguestfs] Virt-v2v split performance

[Some random mumblings that I want to put on record ...] The new "modular virt-v2v" (which I'll probably release as "virt-v2v 2.0" even though it's not really a rewrite) is in the state where it can convert guests: Alpha-quality code: https://github.com/rwmjones/virt-v2v/commits/2021-virt-v2v-split Original plan: https://listman.redhat.com/archives/libguestfs/2020-November/msg00022.html One thing of note is that it's slower in some places and maybe will be faster in others. Because I'm using NBD disk pipelines for everything, when we want to read or write to a local disk image we go through nbdkit (instead of just accessing the local file as virt-v2v would do). However the local file case is not very common in real world usage (except for testing). On the flip side, nbdcopy is faster than qemu-img convert when copying between NBD source and target. In fact when copying from file to file through 2 x nbdkits, nbdcopy is about the same speed as qemu-img convert copying between local files. Some actual timings. As a baseline here's original virt-v2v doing a file to file conversion: $ virt-v2v -i disk /var/tmp/rhel-8.4.img -o disk -os /var/tmp [ 0.0] Opening the source -i disk /var/tmp/rhel-8.4.img [ 0.0] Creating an overlay to protect the source from being modified [ 0.1] Opening the overlay [ 4.8] Inspecting the overlay [ 9.9] Checking for sufficient free disk space in the guest [ 9.9] Estimating space required on target for each disk [ 9.9] Converting Red Hat Enterprise Linux 8.4 (Ootpa) to run on KVM virt-v2v: warning: /files/boot/grub2/device.map/hd0 references unknown device "vda". You may have to fix this entry manually after conversion. virt-v2v: This guest has virtio drivers installed. [ 48.8] Mapping filesystem data to avoid copying unused and blank areas [ 50.7] Closing the overlay [ 51.2] Assigning disks to buses [ 51.2] Checking if the guest needs BIOS or UEFI to boot [ 51.2] Initializing the target -o local -os /var/tmp [ 51.2] Copying disk 1/1 to /var/tmp/rhel-8.4-sda (raw) (100.00/100%) [ 58.3] Creating output metadata [ 58.3] Finishing off Now virt-v2v 2.0 doing just the conversion step: + ./convert/helper-v2v-convert /tmp/v2v [ 0.0] Opening the source [ 4.8] Inspecting the source [ 11.6] Checking for sufficient free disk space in the guest [ 11.6] Converting Red Hat Enterprise Linux 8.4 (Ootpa) to run on KVM helper-v2v-convert: warning: /files/boot/grub2/device.map/hd0 references unknown device "vda". You may have to fix this entry manually after conversion. helper-v2v-convert: This guest has virtio drivers installed. [ 50.7] Mapping filesystem data to avoid copying unused and blank areas [ 53.9] Closing the overlay [ 54.5] Assigning disks to buses [ 54.5] Checking if the guest needs BIOS or UEFI to boot [ 54.5] Creating output metadata [ 54.5] Finishing off You can see that it's about 2-3 seconds slower overall. You have to add to this: * About 2 seconds taken to start the input and output nbdkit instances. That's because we start the processes and then wait for them to write PID files, in a loop which sleeps for 1 second each iteration. We could be more intelligent here to reduce the sleep time, and we could also start the two sides in parallel. * nbdcopy takes 7.1 seconds, which is precisely the same time that qemu-img convert took (which is quite remarkable considering it's doing more work). We also hope to make nbdcopy much faster. So you can see what filters and options I'm using, the two nbdkit commands are: 3008833 pts/3 S+ 0:00 nbdkit --exit-with-parent --foreground --newstyle --pidfile /run/user/1000/v2vnbdkit.Fzy4p3/nbdkit1.pid --unix /tmp/v2v/in0 --threads 16 -D nbdkit.backend.datapath=0 --filter cow file file=/var/tmp/rhel-8.4.img fadvise=sequential cache=none 3008863 pts/3 S+ 0:00 nbdkit --exit-with-parent --foreground --newstyle --pidfile /run/user/1000/v2vnbdkit.pXctgg/nbdkit1.pid --unix /tmp/v2v/out0 --threads 16 -D nbdkit.backend.datapath=0 file file=/var/tmp/rhel-8.4-sda fadvise=sequential cache=none I'm not yet using nbdkit-cache-filter. Old virt-v2v uses copy-on-read caching. I'm also not yet using readahead, because nbdkit-readahead-filter is broken and needs a rewrite. Old virt-v2v uses readahead and adjusts it dynamically. The main aim of the virt-v2v split is not necessarily to make single instances faster (although we hope because of optimizations in nbdcopy to do that), but to make the overall system more flexible and take a holistic overview of what's happening on the system. The shell script I'm using to automate the new v2v is attached (because I haven't yet converted the command line parsing code). Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top