Re: [Libguestfs] nbdkit random seek performance

hi richard---arthur and I are working with nbdkit v1.12.3 on qemu/kvm. we found that our linux (ubuntu 16.04 32-bit) boot time from a local .img file went from about 10 seconds to about 3 minutes when using the nbdkit file plugin instead of directly connecting qemu to the file. on further inspection with bonnie++, we think the problem may be related to poor random seek performance:

DIRECT FROM QEMU TO .IMG FILE: ------------------------------------------------------------------------------------------------------------------------------ Version 1.97 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP TEST 4G 109468 11 58405 16 180613 33 3204 147 Latency 7125ms 5812ms 131ms 101ms ------------------------------------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------------------------------------ NBDKIT: ------------------------------------------------------------------------------------------------------------------------------ 1.97,1.97,TEST,1,1564697809,4G,,,,109468,11,58405,16,,,180613,33,3204,147,,,,,,,,,,,,,,,,,,,7125ms,5812ms,,131ms,101ms,,,,,, Version 1.97 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP TEST 4G 105761 11 58992 14 178472 30 39.3 6 Latency 8166ms 3683ms 187ms 1197ms ------------------------------------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------------------------------------ NBDKIT (-fv): ------------------------------------------------------------------------------------------------------------------------------ 1.97,1.97,TEST,1,1564698256,4G,,,,105761,11,58992,14,,,178472,30,39.3,6,,,,,,,,,,,,,,,,,,,8166ms,3683ms,,187ms,1197ms,,,,,, Version 1.97 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP TEST 4G 104941 10 56501 14 160557 28 39.9 6 Latency 15318ms 5200ms 101ms 1152ms 1.97,1.97,TEST,1,1564695086,4G,,,,104941,10,56501,14,,,160557,28,39.9,6,,,,,,,,,,,,,,,,,,,15318ms,5200ms,,101ms,1152ms,,,,,, ------------------------------------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------------------------------------ is this an error on our part? do you have any recommendations on what we could investigate next? regards, /iaw -- Ivo Welch (ivo.welch(a)ucla.edu) http://www.ivo-welch.info/