Re: [Libguestfs] [PATCH nbdkit INCOMPLETE] readahead: Rewrite this filter so it prefetches using .cache
On Tue, Apr 19, 2022 at 04:31:44PM +0200, Laszlo Ersek wrote:
On 04/19/22 14:47, Richard W.M. Jones wrote:
> The previous readahead filter did not work well and we have stopped
> using it in virt-v2v. However the concept is a good one if we can
> make it work. This commit completely rethinks the filter.
I started responding to the cover letter, "what is the motivation for
this? furthermore, can we add this as a new filter instead, so
comparisons (back-and-forth) can be made easily"?
TBH the motivation is not very clear, but here are several current
reasons:
(1) Virt-v2v does some random access conversion up front, but then
spends most of its time sequentially copying, using originally
“qemu-img convert” and now nbdcopy. It was thought that
nbdkit-readahead-filter could help in some way, but the current
implementation does not help, and in fact had some pretty negative
effects.
If we believe that nbdkit can help (which is not entirely clear), then
fixing nbdkit-readahead-filter is worthwhile.
For virt-v2v it's often the case that the source is remote, over a
relatively slow network. On the one hand, prefetching would be good
because it gets the data across the network and into the local cache.
On the other hand, it's likely we can saturate most networks, so
prefetching will make no difference or be net negative.
(2) Libvirt XML exposes the "readahead" setting of the qemu curl driver.
https://libvirt.org/formatdomain.html#hard-drives-floppy-disks-cdroms
https://gitlab.com/qemu/qemu/-/blob/master/block/curl.c#L897
In order to convert libvirt to use nbdkit-curl-plugin it's therefore
felt that we ought to have an equivalent feature in nbdkit, although
another way to fix this would simply be to ignore this setting.
(3) We have a readahead filter which definitely doesn't work, so we
ought to either fix it or retire it.
(4) A feature of nbdkit is that you can write simple plugins (eg. no
threads, no fancy features), and then layer filters on top to add some
of those features (caching, logging, readahead), so that would be a
reason to have a readahead filter.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines. Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v