NOTE: This is not complete, I'm just posting it for early review. In
particular I need to test and benchmark this a lot more to make sure
it really is a performance improvement.
At the moment nbdkit-vddk-plugin suffers because it works
synchronously. We found in testing that copying multiple VMs in
parallel is much faster than copying a single VM, the obvious reason
being that each single nbdkit spends a lot of time waiting on round
trips to the VMware server.
Since VDDK 6.0 (but only implemented in VDDK 6.7+) the library has
offered a form of asynchronous read and write operation. We can use
that to implement the parallel thread model, hopefully allowing
multiple outstanding read/write commands to overlap in flight.
One thing of note is that multi-conn is probably not possible without
a lot more work. The reason is that VDDK locks the backing disk when
you open a handle, so multiple connections (ie. handles) cannot be
opened. I am also not clear if VDDK/VMware offers the correct data
integrity guarantees to allow us to implement multi-conn, and since
the whole lot of closed source there's no easy way to find out.
Rich.