Re: [Libguestfs] [PATCH virt-v2v] v2v: -o rhv-upload: Enable multi-conn
On Mon, Aug 2, 2021 at 10:41 AM Richard W.M. Jones <rjones(a)redhat.com> wrote:
Hi Nir,
The experimental patch attached (for discussion) would enable
multi-conn. For current virt-v2v this would have no effect since
"qemu-img convert" does not make multi-conn NBD connections.
However for modular virt-v2v[1] which uses nbdcopy this would allow
nbdcopy to make multiple NBD connections to nbdkit running the plugin.
(By default 4 connections).
If I understand the rhv-upload-plugin code correctly, there is a pool
per NBD connection at the moment, so this would create 4 pools and 4*4
backend connections to imageio. This is possibly not desirable, but I
guess we can turn the pool into a global object. (Note each NBD
connection has the same (destination_url, host, options) tuple).
Creating 4*4 connections will not work since on imageio side we have qem-nbd
supporting up to 8 connections. What will happen is that http request
will block once
there are 8 connected client on imageio side. The http client will
connect only when
some other clients have disconnected, which may never happen, so this
will likely
end with a deadlock.
If we have multi_conn we don't need the pool, we can have a configuration value
to control the number of connections, and use single connection (that
looks like a pool)
in this case.
I'm not about how multi_con works in the python plugin - do we get one
open() call
or 4 open() calls?
If we get 4 open() calls, this cannot work since we mix control flow
and data flow in the
rhv plugin. Each open call will try to create a new disk and every
connection will upload
to a different disk.
multi_con can work if we have:
- single open() call
- many write/zero/flush calls
- single close() call
If we have:
- one open() for nbd connection
- many write/zero calls per connection
- one close() for nbd connection
We need to separate the control flow stuff (e.g. create disk) from the
plugin, and do it in
another step of the process.
Naturally I've not actually tested any of this. There are some
advantages from the nbdcopy side of things, especially because it is
able to query extents on the input side in parallel which seems to be
advantageous for inputs with slow extent querying (like VDDK). There
may be further advantages on the output side because it would allow us
to write data to the RHV upload plugin over four sockets, allowing
better use of multiple cores.
It may work better when using HTTPS, the TLS part is implemented in C
and can run in parallel.
Nir