Hi Rich, Thanks for the explanation. I measured the time to read a block for the first time through nbdkit-xz-filter, at different block sizes. Setup: Fedora-Cloud-Base-AmazonEC2-45-1.4 raw (sha256 from the official CHECKSUM), recompressed with xz -6 at each block size. The 24M archive is byte-identical to Fedora's own .raw.xz: 24 MiB is simply xz's default block size in multi-threaded mode (3 x the 8 MiB dictionary of -6), so that is what Fedora ships, not 16M. nbdkit 1.24.1 (Ubuntu 22.04), liblzma 5.2.5, xz-max-depth default, libnbd client, 4 KiB reads at random offsets inside blocks that contain data; 1,000 reads per point, median of 5 runs, every reply checked byte-for-byte against the raw image. block size vs 24M first read p50 / p99 cached read nbdkit peak RSS 1M +3.1% 27 / 34 ms 0.024 ms 16 MiB 2M +2.2% 52 / 69 ms 0.023 ms 26 MiB 4M +1.5% 110 / 137 ms 0.024 ms 48 MiB 8M +0.8% 144 / 272 ms 0.025 ms 116 MiB 16M +0.4% 224 / 545 ms 0.026 ms 157 MiB 24M 0 276 / 804 ms 0.028 ms 253 MiB 64M -0.3% 597 / 2128 ms 0.029 ms 589 MiB (plain nbdkit file on the raw image: 0.014 ms per 4 KiB read) The first-read cost grows with block size, but not linearly, and even 1M costs 27 ms. This is an old nbdkit version, so please treat the absolute numbers with that caveat. So there is no single best size, but going from 24M to 1-4M cuts first-read latency by 2.5-10x for +1.5-3.1% size and much less memory. Whether that matters depends on how often a VM touches a new block. Raw data, scripts and the full table (including uniform reads over the whole image, where zero blocks make reads cheaper): https://www.google.com/url?q=https://github.com/yasha1971-coder/hw-apex-bench/blob/3c9a68329ba591603f20a7405af0de853ee09c67/evidence/hwab-nbd-1-20261121/README.md&source=gmail&ust=1795376551358000&sa=E Yakiv On Sun, Nov 15, 2026 10:23 AM, Yasha Friend <yasha1971@gmail.com> wrote:
Hi Richard,
I maintain hw-apex-bench (Apache-2.0), a small benchmark that measures the cost of reading one region of a compressed file — latency and decoded bytes — at different block sizes, for xz, zstd seekable and BGZF. Disclosure: I build it with heavy help from AI tools; every number is reproducible from the repository.
The nbdkit-xz-filter man page suggests many small blocks (16 MiB in its example) for random access. Would a table of region-read latency and ratio versus xz block size, measured on a public VM disk image, be useful to you or to users choosing --block-size? If so, which image and read sizes would you consider representative?
That is the whole question — "not useful" is a fine answer too.
Thanks,
Yakiv Shavidze