Re: [Libguestfs] [nbdkit PATCH] RFC: noextents: Add extentmode=data config parameter

On Tue, Feb 08, 2022 at 12:39:02PM +0100, Laszlo Ersek wrote: Hmm, we aren't always consistent, but I agree that it can be pruned without loss. The code in server/ guarantees that we cannot call into a filter or plugin with an extents request that would read out of bounds; ie. on input, offset+count will never exceed what next->get_size() would tell us anyways. Conversely, nbdkit_add_extent() already permits us to pass in redundant information prior to offset (as long as we make forward progress by eventually using nbdkit_add_extent for at least one byte at offset before returning), as well as to provide more information than needed (the upper layer can set a clamp, such as when FLAG_REQ_ONE is in use by the client, or at the 32-bit boundary condition, where our additions beyond that clamp are merely ignored). So we could just as easily write return nbdkit_add_extent (ret_extents, 0, INT64_MAX, 0); which will be trimmed to size as needed before going over the wire back to the client. But now that I've written that, I can see that there is indeed a useful benchmarking distinction between using offset/count vs. 0/INT_MAX - the former behaves as if the client had always passed in FLAG_REQ_ONE (we can never progress faster than the client wants us to go), while the latter gives as much information to the client as possible (subject to 32-bit limits, until my 64-bit block status work is revived). So instead of just one new mode named 'data', maybe we want a larger family of possible modes: - mask (default; existing behavior of masking extent support so client can't query them at all); client must assume disk is data - data-match (reply that the extent is data, but match the size of the client request; behaves as if the client unconditionally uses FLAG_REQ_ONE); client sees disk as data - data-full (reply that the entire remainder of the extent was data, although the client's use of FLAG_REQ_ONE clamps it back to the query size); client sees disk as data - req-one (pass the query on through to the plugin, but then purposefully trim it back before replying to the client so that it behaves as if the client unconditionally uses FLAG_REQ_ONE); client can see holes, but cannot read ahead - plugin (pass the query on through to the plugin, with no alterations to the plugin's result - normalizes the impact of having the filter in the stack when comparing to other modes); client can see holes, and can benefit from reading ahead We can also bike-shed on better names for those modes (maybe 'disabled' instead of 'mask'; or 'passthrough' instead of 'plugin', for example). In fact, as written above, it means the 'noextents' filter could serve the role of controlling three orthogonal aspects: are extents advertised, can extents return additional information beyond the client's requested length (when the client didn't use REQ_ONE), and can extents report hole/zero information. Is there ever a reason that we would want to always call into the plugin, but then mask some (or all) of the returned extent bits to only allow certain bits to return to the client? For example, a mode that lets the plugin report the 'zero' bit but not the 'hole' bit? Being able to time how long the plugin takes to answer the query (even if we then throw away the plugin's answer to claim that we have only data) may be an interesting benchmark, as well, when comparing to the speed at which we simulate the entire disk as data without reading into the plugin. The nbdkit core takes care of that. A filter or plugin can report on nonexistent data, but it will be clamped back so that the client never sees more than the advertised disk length. Already done implicitly by the core. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org