On 3/7/19 9:19 AM, Richard W.M. Jones wrote:
Hi Eric:
Here's the problem: We have a VMware VDDK file or snapshot. The
virtual size of the snapshot is generally much larger than what's
allocated. Copying the snapshot using:
nbdkit vddk file=foo.vmdk --run 'qemu-img convert $nbd out'
is therefore much slower than it could be. (In the real world version
of this we're actually copying the source over the network, but the
command line for that is too complicated to show in a simple example).
VDDK has an API for querying allocated blocks, see
‘VixDiskLib_QueryAllocatedBlocks’ in
https://vdc-download.vmware.com/vmwb-repository/dcr-public/8f96698a-0e7b-...
It's awkward to use but Martin (CC'd) has played with it and it does
seem to work.
The question is if we add Structured Replies + Block Status support to
nbdkit & implement it for the VDDK plugin, will ‘qemu-img convert’
above be able to use that to optimize the copy? My brief reading of
the qemu sources suggests that something has been implemented here so
it likely ought to work?
Yes, my gut feel is that qemu-img IS able to take advantage of
NBD_CMD_BLOCK_STATUS to avoid reading sparse portions of the source file
during qemu-img convert, so it IS worth figuring out how to get
structured replies and block status wired into nbdkit. It's on my list
of fun projects to attempt (but trailing behind other stuff like libvirt
incremental snapshots at the moment).
I guess the other question is if there are any limitations around
alignment that we should be aware of. It seems as if VDDK uses a
larger cluster size for sparseness than its sector size (as does
qcow2).
In general, block status will naturally report holes aligned to
granularity, but it really shouldn't matter to correctness (reporting
holes is merely an optimization, after all).
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization:
qemu.org |
libvirt.org