Re: [Libguestfs] [PATCH nbdkit 0/5 ] vddk: Implement parallel thread model

NOTE: This is not complete, I'm just posting it for early review. In particular I need to test and benchmark this a lot more to make sure it really is a performance improvement. At the moment nbdkit-vddk-plugin suffers because it works synchronously. We found in testing that copying multiple VMs in parallel is much faster than copying a single VM, the obvious reason being that each single nbdkit spends a lot of time waiting on round trips to the VMware server. Since VDDK 6.0 (but only implemented in VDDK 6.7+) the library has offered a form of asynchronous read and write operation. We can use that to implement the parallel thread model, hopefully allowing multiple outstanding read/write commands to overlap in flight. One thing of note is that multi-conn is probably not possible without a lot more work. The reason is that VDDK locks the backing disk when you open a handle, so multiple connections (ie. handles) cannot be opened. I am also not clear if VDDK/VMware offers the correct data integrity guarantees to allow us to implement multi-conn, and since the whole lot of closed source there's no easy way to find out. Rich.