Re: [Libguestfs] [PATCH 6/6] v2v: nbdkit: Match qemu-img number of parallel coroutines

On Sat, Jan 23, 2021 at 12:45 AM Nir Soffer <nirsof(a)gmail.com&gt; wrote: Here is an example nbdkit output: hw_id = 'd8635601-ea8b-4c5c-a624-41bd72b862d6' datacenter = ost host.id = '9de3098d-89e8-444c-95ed-0fc2ea74e170' disk.id = '6263dab4-e299-43c4-8f10-9a227b9bed01' transfer.id = 'fc27844a-00bc-4035-afd6-83c9a456cae4' using https connection using 4 connections using unix socket '\x00/org/ovirt/imageio' using unix socket '\x00/org/ovirt/imageio' using unix socket '\x00/org/ovirt/imageio' using unix socket '\x00/org/ovirt/imageio' imageio features: flush=True zero=True unix_socket='\x00/org/ovirt/imageio'max_readers=8 max_writers=8 nbdkit: python[1]: debug: python: open returned handle 0x7f17840016d0 nbdkit: python[1]: debug: python: prepare readonly=0 nbdkit: python[1]: debug: python: get_size nbdkit: python[1]: debug: python: can_write nbdkit: python[1]: debug: python: can_zero nbdkit: python[1]: debug: python: can_fast_zero nbdkit: python[1]: debug: python: can_trim nbdkit: python[1]: debug: python: can_fua nbdkit: python[1]: debug: python: can_flush nbdkit: python[1]: debug: python: is_rotational nbdkit: python[1]: debug: python: can_multi_conn nbdkit: python[1]: debug: python: can_cache nbdkit: python[1]: debug: python: can_extents nbdkit: python[1]: debug: newstyle negotiation: flags: export 0xcd nbdkit: python[1]: debug: newstyle negotiation: NBD_OPT_GO: ignoring NBD_INFO_* request 3 (NBD_INFO_BLOCK_SIZE) nbdkit: python[1]: debug: handshake complete, processing requests with 8 threads nbdkit: debug: starting worker thread python.1 nbdkit: debug: starting worker thread python.4 nbdkit: debug: starting worker thread python.6 nbdkit: debug: starting worker thread python.5 nbdkit: debug: starting worker thread python.0 nbdkit: debug: starting worker thread python.3 nbdkit: debug: starting worker thread python.7 nbdkit: debug: starting worker thread python.2 (100.00/100%) nbdkit: python.1: debug: client sent NBD_CMD_DISC, closing connection nbdkit: python.1: debug: exiting worker thread python.1 nbdkit: python.4: debug: exiting worker thread python.4 nbdkit: python.5: debug: exiting worker thread python.5 nbdkit: python.2: debug: exiting worker thread python.2 nbdkit: python.3: debug: exiting worker thread python.3 nbdkit: python.0: debug: exiting worker thread python.0 nbdkit: python.7: debug: exiting worker thread python.7 nbdkit: python.6: debug: exiting worker thread python.6 nbdkit: virtual copying rate: 6518.0 M bits/sec python[1]: debug: python: finalize nbdkit: python[1]: debug: python: close finalizing transfer fc27844a-00bc-4035-afd6-83c9a456cae4 transfer fc27844a-00bc-4035-afd6-83c9a456cae4 finalized in 2.031 seconds Example imageio log: 2021-01-23 00:44:38,239 INFO (Thread-427) [http] OPEN connection=427 client=local 2021-01-23 00:44:38,240 INFO (Thread-427) [tickets] [local] ADD ticket={'dirty': False, 'ops': ['write'], 'size': 6442450944, 'sparse': True, 'transfer_id': 'fc27844a-00bc-4035-afd6-83c9a456cae4', 'uuid': 'ecbcd8a1-eeca-4c1d-97d6-b907a0c3999b', 'timeout': 300, 'url': 'nbd:unix:/run/vdsm/nbd/ecbcd8a1-eeca-4c1d-97d6-b907a0c3999b.sock'} 2021-01-23 00:44:38,240 INFO (Thread-427) [http] CLOSE connection=427 client=local [connection 1 ops, 0.000776 s] [dispatch 1 ops, 0.000259 s] 2021-01-23 00:44:38,375 INFO (Thread-428) [http] OPEN connection=428 client=local 2021-01-23 00:44:38,376 INFO (Thread-428) [http] CLOSE connection=428 client=local [connection 1 ops, 0.000708 s] [dispatch 1 ops, 0.000187 s] 2021-01-23 00:44:39,276 INFO (Thread-429) [http] OPEN connection=429 client=::ffff:192.168.122.23 2021-01-23 00:44:39,276 INFO (Thread-429) [images] [::ffff:192.168.122.23] OPTIONS ticket=ecbcd8a1-eeca-4c1d-97d6-b907a0c3999b 2021-01-23 00:44:39,277 INFO (Thread-429) [backends.nbd] Open backend address='/run/vdsm/nbd/ecbcd8a1-eeca-4c1d-97d6-b907a0c3999b.sock' export_name='' sparse=True max_connections=8 2021-01-23 00:44:39,277 INFO (Thread-429) [backends.nbd] Close backend address='/run/vdsm/nbd/ecbcd8a1-eeca-4c1d-97d6-b907a0c3999b.sock' 2021-01-23 00:44:39,278 INFO (Thread-429) [http] CLOSE connection=429 client=::ffff:192.168.122.23 [connection 1 ops, 0.001338 s] [dispatch 1 ops, 0.000824 s] 2021-01-23 00:44:39,287 INFO (Thread-430) [http] OPEN connection=430 client=local 2021-01-23 00:44:39,288 INFO (Thread-431) [http] OPEN connection=431 client=local 2021-01-23 00:44:39,289 INFO (Thread-432) [http] OPEN connection=432 client=local 2021-01-23 00:44:39,289 INFO (Thread-433) [http] OPEN connection=433 client=local 2021-01-23 00:44:39,290 INFO (Thread-430) [backends.nbd] Open backend address='/run/vdsm/nbd/ecbcd8a1-eeca-4c1d-97d6-b907a0c3999b.sock' export_name='' sparse=True max_connections=8 2021-01-23 00:44:39,290 INFO (Thread-431) [backends.nbd] Open backend address='/run/vdsm/nbd/ecbcd8a1-eeca-4c1d-97d6-b907a0c3999b.sock' export_name='' sparse=True max_connections=8 2021-01-23 00:44:39,291 INFO (Thread-432) [backends.nbd] Open backend address='/run/vdsm/nbd/ecbcd8a1-eeca-4c1d-97d6-b907a0c3999b.sock' export_name='' sparse=True max_connections=8 2021-01-23 00:44:39,294 INFO (Thread-433) [backends.nbd] Open backend address='/run/vdsm/nbd/ecbcd8a1-eeca-4c1d-97d6-b907a0c3999b.sock' export_name='' sparse=True max_connections=8 2021-01-23 00:44:40,399 INFO (Thread-434) [http] OPEN connection=434 client=local 2021-01-23 00:44:40,401 INFO (Thread-434) [http] CLOSE connection=434 client=local [connection 1 ops, 0.001947 s] [dispatch 1 ops, 0.000121 s] 2021-01-23 00:44:43,154 INFO (Thread-432) [images] [local] FLUSH ticket=ecbcd8a1-eeca-4c1d-97d6-b907a0c3999b 2021-01-23 00:44:43,155 INFO (Thread-430) [images] [local] FLUSH ticket=ecbcd8a1-eeca-4c1d-97d6-b907a0c3999b 2021-01-23 00:44:43,156 INFO (Thread-433) [images] [local] FLUSH ticket=ecbcd8a1-eeca-4c1d-97d6-b907a0c3999b 2021-01-23 00:44:43,157 INFO (Thread-431) [images] [local] FLUSH ticket=ecbcd8a1-eeca-4c1d-97d6-b907a0c3999b 2021-01-23 00:44:43,158 INFO (Thread-432) [images] [local] FLUSH ticket=ecbcd8a1-eeca-4c1d-97d6-b907a0c3999b 2021-01-23 00:44:43,159 INFO (Thread-430) [images] [local] FLUSH ticket=ecbcd8a1-eeca-4c1d-97d6-b907a0c3999b 2021-01-23 00:44:43,160 INFO (Thread-433) [images] [local] FLUSH ticket=ecbcd8a1-eeca-4c1d-97d6-b907a0c3999b 2021-01-23 00:44:43,161 INFO (Thread-431) [images] [local] FLUSH ticket=ecbcd8a1-eeca-4c1d-97d6-b907a0c3999b 2021-01-23 00:44:43,164 INFO (Thread-431) [backends.nbd] Close backend address='/run/vdsm/nbd/ecbcd8a1-eeca-4c1d-97d6-b907a0c3999b.sock' 2021-01-23 00:44:43,164 INFO (Thread-431) [http] CLOSE connection=431 client=local [connection 1 ops, 3.875194 s] [dispatch 554 ops, 2.995257 s] [zero 62 ops, 0.139778 s, 793.44 MiB, 5.54 GiB/s] [zero.zero 62 ops, 0.138456 s, 793.44 MiB, 5.60 GiB/s] [write 490 ops, 2.655315 s, 328.25 MiB, 123.62 MiB/s] [write.read 490 ops, 0.570355 s, 328.25 MiB, 575.52 MiB/s] [write.write 490 ops, 2.057818 s, 328.25 MiB, 159.51 MiB/s] [flush 2 ops, 0.000371 s] 2021-01-23 00:44:43,164 INFO (Thread-433) [backends.nbd] Close backend address='/run/vdsm/nbd/ecbcd8a1-eeca-4c1d-97d6-b907a0c3999b.sock' 2021-01-23 00:44:43,165 INFO (Thread-433) [http] CLOSE connection=433 client=local [connection 1 ops, 3.874932 s] [dispatch 554 ops, 3.111739 s] [zero 67 ops, 0.137024 s, 1.20 GiB, 8.75 GiB/s] [zero.zero 67 ops, 0.135575 s, 1.20 GiB, 8.84 GiB/s] [write 485 ops, 2.770474 s, 337.38 MiB, 121.78 MiB/s] [write.read 485 ops, 0.537087 s, 337.38 MiB, 628.16 MiB/s] [write.write 485 ops, 2.209982 s, 337.38 MiB, 152.66 MiB/s] [flush 2 ops, 0.000434 s] 2021-01-23 00:44:43,165 INFO (Thread-430) [backends.nbd] Close backend address='/run/vdsm/nbd/ecbcd8a1-eeca-4c1d-97d6-b907a0c3999b.sock' 2021-01-23 00:44:43,166 INFO (Thread-430) [http] CLOSE connection=430 client=local [connection 1 ops, 3.877590 s] [dispatch 542 ops, 3.105687 s] [zero 78 ops, 0.171958 s, 1.35 GiB, 7.86 GiB/s] [zero.zero 78 ops, 0.170382 s, 1.35 GiB, 7.93 GiB/s] [write 462 ops, 2.754178 s, 338.25 MiB, 122.81 MiB/s] [write.read 462 ops, 0.565677 s, 338.25 MiB, 597.96 MiB/s] [write.write 462 ops, 2.161230 s, 338.25 MiB, 156.51 MiB/s] [flush 2 ops, 0.000213 s] 2021-01-23 00:44:43,166 INFO (Thread-432) [backends.nbd] Close backend address='/run/vdsm/nbd/ecbcd8a1-eeca-4c1d-97d6-b907a0c3999b.sock' 2021-01-23 00:44:43,167 INFO (Thread-432) [http] CLOSE connection=432 client=local [connection 1 ops, 3.876394 s] [dispatch 541 ops, 3.029115 s] [zero 70 ops, 0.139170 s, 1.36 GiB, 9.80 GiB/s] [zero.zero 70 ops, 0.137861 s, 1.36 GiB, 9.90 GiB/s] [write 469 ops, 2.709666 s, 338.56 MiB, 124.95 MiB/s] [write.read 469 ops, 0.539536 s, 338.56 MiB, 627.51 MiB/s] [write.write 469 ops, 2.146016 s, 338.56 MiB, 157.76 MiB/s] [flush 2 ops, 0.000305 s] 2021-01-23 00:44:44,440 INFO (Thread-435) [http] OPEN connection=435 client=local 2021-01-23 00:44:44,440 INFO (Thread-435) [tickets] [local] REMOVE ticket=ecbcd8a1-eeca-4c1d-97d6-b907a0c3999b 2021-01-23 00:44:44,441 INFO (Thread-435) [http] CLOSE connection=435 client=local [connection 1 ops, 0.000628 s] [dispatch 1 ops, 0.000202 s]