1:38 p.m.
On 01/19/2018 11:12 AM, Richard W.M. Jones wrote:
>> +
>> +=head2 C<.pread>
>> +
>> + int (*pread) (struct nbdkit_next_ops *next_ops, void *nxdata,
>> + void *handle, void *buf, uint32_t count, uint64_t offset);
>
> Wrong signature, missing the uint32_t flags that the backend interface
> has, and that I'm adding for plugins API_VERSION 2 for FUA support.
Right, but this patch is against the version 1 API. We will need to
add flags at some point.
So, do we ever want filters to be usable via version 1 instead of
version 2 backend API, or should we be aiming that all filters are
automatically version 2 from the point where we release this? I'm fine
if we have some churn in .git by taking your patches with version 1
signatures then my FUA patches on top that converts to version 2; I'm
more worried that if we allow filters with version 1 semantics that we
may run into some interesting difficulties (not the least of which is
that a version 2 plugin that can support efficient FUA is inherently
penalized if it is used with a filter talking only version 1).
>> +Filters may have to use pthread primitives like mutexes to
achieve
>> +this.
>
> Would it ever make sense to allow a filter to intentionally request a
> lower concurrency level than the underlying plugin? At connection time,
> you'd compute the threading level as the lowest level requested by any
> element of the chain; it might make it easier to write plugins that
> aren't fully parallel (such filters may penalize the speed that can
> otherwise be achieved by using the plugin in isolation - but again, it
> may be useful for testing). That would imply adding a
> backend.thread_model() callback, which needs to be exposed to filters
> but not to plugins (plugins continue to use the #define at compile
> time); if the filter does not define the .thread_model callback, it
> defaults to fully-parallel; but if it does define the callback, it can
> result in something less concurrent.
Yes, this is a good idea actually.
Are you planning on writing that, or do you want me to take a shot at it?
>> +=item B<--filter> FILTER
>> +
>> +Add a filter before the plugin. This option may be given one or more
>> +times to stack filters in front of the plugin. They are processed in
>> +the order they appear on the command line. See L</FILTERS> and
>> +L<nbdkit-filter(3)>.
>> +
>
> Does it ever make sense to provide the same filter more than once on the
> command line? Or are we stating that a given filter can only be used
> once in the chain?
I couldn't think of a case where we'd need the same filter multiple
times. It could be made to work but we'd have to ban global
variables and modify the load() function.
I can (sort of) see it if we support composition; but again, if
composition is done right, you still have just a single use of each
plugin per nbdkit process. Let's say I want to access [0-100M) and
[200M-300M) of a given file via composition. Here's a possible way to
write it:
nbdkit --filter=compose compose='--filter=offset offset=0 range=100M
file file=FILE' --filter=offset offset=200M range=100M file file=FILE
Is there a way to rewrite it so I don't have to nest quotes when doing
more than one composition? If there is, then --filter=offset appears
twice, as does 'file file=FILE'. But either way, under the hood, we'd
implement it as two separate nbdkit processes if composition is a filter:
nbdkit --> compose --> offset 0/100 --> file
\-> nbd ==> nbdkit --> offset 200/100 --> file
where -> is the backend chain, and ==> is an NBD socket.
Or maybe composition is a plugin, in which case it might look more like:
nbdkit compose sub='--filter=offset offset=0 range=100M file file=FILE'
sub='--filter=offset offset=200M range=100M file file=FILE'
(Can config even accept the same key= with different values more than
once on the command line?)
nbdkit --> compose ==> nbdkit --> offset 0/100 --> file
\=> nbdkit --> offset 200/100 --> file
Certainly some tradeoffs to revisit depending on whether composition
fits better as a filter or a plugin
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org