On Thu, Jan 21, 2021 at 1:57 PM Nir Soffer <nirsof@gmail.com> wrote: ... How I tested: I could not use the example python plugin with the added delay, since os.preadv() is not available in python 3.6 (added in 3.7). I could not use the delay filter since it supports only integer delay. With fedora 32 image with 231 extents, qemu-img calls block_status 960 times during import and this is too slow to test. So I used this hack: $ git diff diff --git a/plugins/file/file.c b/plugins/file/file.c index 1651079a..99a95af0 100644 --- a/plugins/file/file.c +++ b/plugins/file/file.c @@ -785,6 +785,8 @@ do_extents (void *handle, uint32_t count, uint64_t offset, const bool req_one = flags & NBDKIT_FLAG_REQ_ONE; uint64_t end = offset + count; + usleep(300000); + do { off_t pos; Started nbdkit with fedora 32 image: $ ./nbdkit -f -v ./plugins/file/.libs/nbdkit-file-plugin.so file=/var/tmp/fedora-32.raw Add overlay to make it work with virt-v2v -i disk: $ qemu-img create -f qcow2 -b nbd://localhost -F raw /var/tmp/fedora-32.qcow2 With this, getting all extents takes more than 60 seconds, enough to get the connection closed: $ time qemu-img map --output json nbd://localhost >/dev/null real 1m9.416s user 0m0.007s sys 0m0.017s I imported the qcow2 image on ovirt host, with --oo rhv_direct=true/false. Looking at imageio log, we see that the initial connection is closed, and the second connection is made once qemu finish to get extetents for the entire image. I also tried to import using imageio client, using this pipeline: imageio nbd client -> qemu-nbd -> overlay.qcow2 -> nbdkit On imageio side, we always try to get extents for 2g, and we don't use NBD_CMD_FLAG_REQ_ONE. But on nbdkit we see that all extents calls use req_one=1. Looks like the issue is this code in qemu nbd driver: 1676 static int coroutine_fn nbd_client_co_block_status( 1677 BlockDriverState *bs, bool want_zero, int64_t offset, int64_t bytes, 1678 int64_t *pnum, int64_t *map, BlockDriverState **file) 1679 { ... 1685 NBDRequest request = { 1686 .type = NBD_CMD_BLOCK_STATUS, 1687 .from = offset, 1688 .len = MIN(QEMU_ALIGN_DOWN(INT_MAX, bs->bl.request_alignment), 1689 MIN(bytes, s->info.size - offset)), 1690 .flags = NBD_CMD_FLAG_REQ_ONE, 1691 }; Also qemu-img convert is getting extents twice: 1. One call per extents before starting the copy: nbdkit: file.0: debug: file: extents count=196608 offset=851968 req_one=1 nbdkit: file.1: debug: file: extents count=196608 offset=1376256 req_one=1 nbdkit: file.3: debug: file: extents count=393216 offset=1638400 req_one=1 nbdkit: file.4: debug: file: extents count=458752 offset=2686976 req_one=1 nbdkit: file.5: debug: file: extents count=983040 offset=3211264 req_one=1 ... nbdkit: file.14: debug: file: extents count=524288 offset=6441402368 req_one=1 2. One call per extent before each pread(), during copy: nbdkit: file.0: debug: file: extents count=196608 offset=851968 req_one=1 nbdkit: file.1: debug: file: extents count=196608 offset=1376256 req_one=1 nbdkit: file.3: debug: file: extents count=393216 offset=1638400 req_one=1 nbdkit: file.4: debug: file: extents count=458752 offset=2686976 req_one=1 nbdkit: file.5: debug: file: pread count=458752 offset=2686976 nbdkit: file.5: debug: file: extents count=983040 offset=3211264 req_one=1 nbdkit: file.6: debug: file: pread count=983040 offset=3211264 nbdkit: file.7: debug: file: extents count=2031616 offset=4259840 req_one=1 nbdkit: file.8: debug: file: pread count=2031616 offset=4259840 nbdkit: file.9: debug: file: extents count=11599872 offset=6356992 req_one=1 nbdkit: file.10: debug: file: pread count=2097152 offset=6356992 nbdkit: file.11: debug: file: pread count=2097152 offset=8454144 ... nbdkit: file.14: debug: file: extents count=393216 offset=5591269376 req_one=1 nbdkit: file.6: debug: file: pread count=393216 offset=5591269376 nbdkit: file.6: debug: file: extents count=313851904 offset=5591728128 req_one=1 nbdkit: file.4: debug: file: pread count=1048576 offset=5591728128 nbdkit: file.4: debug: file: extents count=312803328 offset=5592776704 req_one=1 nbdkit: file.8: debug: file: extents count=535232512 offset=5905580032 req_one=1 nbdkit: file.3: debug: file: extents count=524288 offset=6441402368 req_one=1 nbdkit: file.13: debug: client sent NBD_CMD_DISC, closing connection Total 960 calls per one import. A tiny delay in getting extents can slow down the entire import. Nir