Re: [Libguestfs] [PATCH nbdkit INCOMPLETE 0/6] Introduce filters to nbdkit.
On 01/14/2018 06:11 AM, Richard W.M. Jones wrote:
This patch isn't complete (patch 6/6 isn't finished) so
it's just for
discussion, although it does compile and run.
This introduces to nbdkit a concept of "filters" which can be placed
in front of plugins to modify their behaviour. Some examples where
you might use filters:
* Serve a subset of the data, such as (offset, range) or a
single partition from a disk image.
* Inject delays or errors for testing clients.
* Implement "copy-on-write" (a feature found in other NBD servers).
Filters are implemented by allowing them to intercept methods before a
plugin gets them. For example to implement a read delay the filter
would register for a .pread hook which is implemented like this:
static int
delay_pread (void *handle, void *buf, uint32_t count, uint64_t offset,
int (*next) (void *data,
void *buf, uint32_t count, uint64_t offset),
void *data)
Interesting idea. I'm also wondering if we should start exposing flags
from the client to the end user; in particular, NBD_CMD_FLAG_FUA is a
useful optimization that avoids the overhead of a full-device flush if
it is supported through the entire callstack - but that means we need
both a new .pread callback variant that accepts a flag, as well as a
filter signature that can pass the flag unchanged, as well as a way for
a plugin to inform the main engine which flags it is willing to accept
(at least with the NBD plugin, the ability to accept the FUA flag is
dependent on the remote server, and it's nicer to have the main engine
continue to emulate the needed flushes rather than having to duplicate
the code in the NBD plugin for remote servers that don't accept the flag).
For the filters I want to write this works fine, but with two
caveats:
(1) If new datapath methods are introduced then filters won't get to
see them by default. For example, if we modify offsets when calling
.pread and .pwrite, and then later the .zero method is added to
plugins, then existing filters won't modify the parameters of .zero
correctly (resulting in wrong data being zeroed).
Indeed, and my concern that we want new callbacks for handling flags in
existing plugins is another instance of this.
(2) You cannot do anything more complex in one of these functions than
calling the single underlying plugin method, possibly modifying the
arguments. For example it's hard to see how a "qcow2 decoder" filter
could be written since it would need to have full access to the plugin
methods, not just to the single method.
Yeah, it's definitely conceivable that a filter will want to be able to
access the underlying .read as part of implementing an intelligent .write.
Unfortunately solving (1) & (2) makes the whole thing a lot more
complicated.
In a lot of ways, it sounds like the development of filters in nbdkit
will resemble the BDS filters in qemu by the time we get through with
everything.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
Attachments:
- signature.asc (application/pgp-signature — 619 bytes)