Re: [Libguestfs] More parallelism in VDDK driver (was: Re: CFME-5.11.7.3 Perf. Tests)
On Wed, Aug 05, 2020 at 04:49:04PM +0300, Nir Soffer wrote:
I see, can change the python plugin to support multiple connections
to imageio
using SERIALIZE_REQUESTS?
The GiL should not limit us since the GIL is released when you write to
imageio socket, and this is likely where the plugin spends most of the time.
It's an interesting question and one I'd not really considered at all.
Does the Python GIL actively mutex different threads if they call into
Python code at the same time? If it's truly a lock, then it should,
in which case it should be safe to change the Python plugin to
PARALLEL ...
I'll try it out and get back to you.
I'm not sure what triggers using multiple connections in qemu-img
and
how it decide how many connections should be used, but we report
the number of writers in OPTIONS:
http://ovirt.github.io/ovirt-imageio/images.html#max_writers
There is a hard limit in vdsm, because it runs qemu-nbd with
--shared=8, so you should
not use more than 8 connections, they will just block on qemu-nbd forever.
It's different for qemu NBD client and server. Eric told me on IRC a
few minutes ago that qemu NBD client does not "yet" support
multi-conn. However it is implemented by qemu-nbd (the server) for
readonly connections.
Note that multi-conn and --shared are (a bit) different. Multi-conn
is a flag which the server can send to clients to indicate not only
that multiple connections are possible, but also that they are come
with certain guarantees:
bit 8, NBD_FLAG_CAN_MULTI_CONN: Indicates that the server operates
entirely without cache, or that the cache it uses is shared among
all connections to the given device. In particular, if this flag is
present, then the effects of NBD_CMD_FLUSH and NBD_CMD_FLAG_FUA MUST
be visible across all connections when the server sends its reply to
that command to the client. In the absence of this flag, clients
SHOULD NOT multiplex their commands over more than one connection to
the export.
“--shared” limits the number of connections that the server will
accept at the same time (like the nbdkit limit filter). But it would
still be possible for an NBD server to accept multiple connections
from a single client but not be multi-conn safe.
Also NBD lets you multiplex commands on a single connection (which
does not require multi-conn or --shared).
BTW I found that multi-conn is a big win with the Linux kernel NBD
client.
We use 4 connections by default, giving about 100% speed up compared
with one connection. 2 connections give about 80% speed up. If the
number of connections is related to the number of coroutines, you
can use -m 4 to use 4 coroutines.
Using -W will improve performance. In this mode every coroutine will
do the I/O when it is ready, instead of waiting for other coroutines
and submit the I/O in the right order.
I think Eric might have a better idea about what -m and -W really do
for qemu NBD client. Maybe improve multiplexing? They don't enable
multi-conn :-(
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine. Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/