Re: [Libguestfs] [PATCH nbdkit 0/6] curl: Use a curl handle pool
On Sun, Feb 05, 2023 at 05:09:21PM +0000, Richard W.M. Jones wrote:
On Sun, Feb 05, 2023 at 04:35:41PM +0000, Richard W.M. Jones wrote:
> I'm still adding instrumentation to see if the theory above is right,
> plus I have no idea how to fix this.
Turns out I didn't need to add instrumentation. Simply forcing
nbdcopy to use at most 1 request per connection (-R 1) recovers all
the performance.
$ time ./nbdkit -r -U - curl file:/var/tmp/big --run 'nbdcopy --no-extents -R 1 -p
"$uri" null:'
I still have no good idea how to solve this. Somehow I had to adjust
the libcurl handle pool so that it isn't first-come first-served, but
prefers to spread available handles across connections.
Or I could keep one pool of libcurl handles per NBD connection, at the
risk of exploding the number of libcurl handles we are opening, which
means in practice we'd end up making a lot of connections to the web
server. Rather defeats the idea of using the libcurl handle pool to
control how many connections we make to the web server.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines. Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v