On Wed, Aug 05, 2020 at 03:39:15PM +0300, Nir Soffer wrote:
On Wed, Aug 5, 2020 at 3:28 PM Richard W.M. Jones <rjones@redhat.com> wrote:
Nir, BTW what are you using for performance testing?
virt-v2v with local image, or imageio client with local image.
As far as I can tell it's not possible to make qemu-img convert use multi-conn when connecting to the source (which is going to be a problem if we want to use this stuff in virt-v2v).
But do we need multiple connections? qemu can send multiple requests on one connection.
As implemented now there is only one VDDK handle per connection and VDDK doesn't allow multiple requests at the same time on one handle, so with a single NBD connection everything will be serialized. Now if we were to implement a thread pool inside nbdkit-vddk-plugin we could get around that restriction (with a lot of complexity). But the results I posted a moment ago show that we wouldn't get anything like a linear speed up. It hardly seems worth it to me.
Did you try to copy an image from nbdkit file plugin to another nbdkit file plugin using qemu-img convert?
nbdkit file pluing -> qemu-img convert -W nbd:///?socket=src.sock nbd:///?socket=dst.sock-> nbdkit file plugin
I did not try it but I will be surprised if we don't get all 8 threads busy in both sides
Likely, but that's because the file plugin uses PARALLEL as thread model, so even on a single NBD connection it can keep all threads inside nbdkit busy.
The reason we use multiple connection in imageio is that we don't support async I/O in http client, http server, and nbd client, and it is much easier to open new connection with the entire stuck compared to rewriting the http server and nbd client.
It also much hard to provide easy to use interface for users supporting async I/O.
Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW