On 01/19/2018 11:12 AM, Richard W.M. Jones wrote:
+ +=head2 C<.pread> + + int (*pread) (struct nbdkit_next_ops *next_ops, void *nxdata, + void *handle, void *buf, uint32_t count, uint64_t offset);
Wrong signature, missing the uint32_t flags that the backend interface has, and that I'm adding for plugins API_VERSION 2 for FUA support.
Right, but this patch is against the version 1 API. We will need to add flags at some point.
So, do we ever want filters to be usable via version 1 instead of version 2 backend API, or should we be aiming that all filters are automatically version 2 from the point where we release this? I'm fine if we have some churn in .git by taking your patches with version 1 signatures then my FUA patches on top that converts to version 2; I'm more worried that if we allow filters with version 1 semantics that we may run into some interesting difficulties (not the least of which is that a version 2 plugin that can support efficient FUA is inherently penalized if it is used with a filter talking only version 1).
+Filters may have to use pthread primitives like mutexes to achieve +this.
Would it ever make sense to allow a filter to intentionally request a lower concurrency level than the underlying plugin? At connection time, you'd compute the threading level as the lowest level requested by any element of the chain; it might make it easier to write plugins that aren't fully parallel (such filters may penalize the speed that can otherwise be achieved by using the plugin in isolation - but again, it may be useful for testing). That would imply adding a backend.thread_model() callback, which needs to be exposed to filters but not to plugins (plugins continue to use the #define at compile time); if the filter does not define the .thread_model callback, it defaults to fully-parallel; but if it does define the callback, it can result in something less concurrent.
Yes, this is a good idea actually.
Are you planning on writing that, or do you want me to take a shot at it?
+=item B<--filter> FILTER + +Add a filter before the plugin. This option may be given one or more +times to stack filters in front of the plugin. They are processed in +the order they appear on the command line. See L</FILTERS> and +L<nbdkit-filter(3)>. +
Does it ever make sense to provide the same filter more than once on the command line? Or are we stating that a given filter can only be used once in the chain?
I couldn't think of a case where we'd need the same filter multiple times. It could be made to work but we'd have to ban global variables and modify the load() function.
I can (sort of) see it if we support composition; but again, if composition is done right, you still have just a single use of each plugin per nbdkit process. Let's say I want to access [0-100M) and [200M-300M) of a given file via composition. Here's a possible way to write it: nbdkit --filter=compose compose='--filter=offset offset=0 range=100M file file=FILE' --filter=offset offset=200M range=100M file file=FILE Is there a way to rewrite it so I don't have to nest quotes when doing more than one composition? If there is, then --filter=offset appears twice, as does 'file file=FILE'. But either way, under the hood, we'd implement it as two separate nbdkit processes if composition is a filter: nbdkit --> compose --> offset 0/100 --> file \-> nbd ==> nbdkit --> offset 200/100 --> file where -> is the backend chain, and ==> is an NBD socket. Or maybe composition is a plugin, in which case it might look more like: nbdkit compose sub='--filter=offset offset=0 range=100M file file=FILE' sub='--filter=offset offset=200M range=100M file file=FILE' (Can config even accept the same key= with different values more than once on the command line?) nbdkit --> compose ==> nbdkit --> offset 0/100 --> file \=> nbdkit --> offset 200/100 --> file Certainly some tradeoffs to revisit depending on whether composition fits better as a filter or a plugin -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org