Re: [Libguestfs] nbdkit background threads
On 2/10/20 1:39 PM, Richard W.M. Jones wrote:
https://github.com/libguestfs/nbdkit/blob/ecef5b16359fb5af7e7abf4fd2fb3ad...
Already existing filters (readahead, cache) could be improved if
filters could open a background work thread or threads which connect
independently to the plugin. A proposed new filter (scan) cannot
really be written at all without this capability.
First of all the reason this can't be done today is because filters
are called with a next_ops structure which is only valid transiently
during the filter callback. It cannot safely be saved or passed to
another thread.
(https://github.com/libguestfs/nbdkit/blob/master/docs/nbdkit-filter.pod#n...)
I was independently thinking of fixing this: we want next_ops to be have
a life equal to the connection itself, rather than stack-allocated. I'm
hoping to post a patch for that shortly, as part of my experimentation
with implementing ext2 as a filter instead of a plugin, where ext2 has
the limitation that when writing a custom io_manager, you only get ONE
spot where you can pass in an opaque pointer: our .prepare will have to
pass a long-lived nxdata to ext2fs_open().
But that's only a partial solution towards the rest of your goal of
having threading available.
The seemingly obvious implementation - which is what I tried today -
would be to let filters create background threads in .config_complete.
We would provide a filter API something like:
int nbdkit_open_connection (struct nbdkit_next_ops **next_ops,
void **nxdata,
/* needs a close function */);
It would return a next_ops and nxdata that the filter could then use
to make data calls from the background thread into the underlying
layers.
To ensure safe unloading of filters and plugins we would also need a
new filter callback (which I called .bg_kill) which must close all
background connections opened by the filter before returning.
Seems reasonable.
I believe from my rough implementation that this is feasible. However
I also thought about another way we might do this: We might open a
loopback socket (eg. socketpair) which is passed to the filter and
which the filter connects to using libnbd. nbdkit internally would
treat this as if it was a regular external connection. Of course this
would require libnbd as a dependency, or disable filters / features if
not available.
There's also the question of how we would do compostion. For example,
right now, the 'split' plugin basically acts as a compose operation over
the 'file' plugin (you can concatenate more than one regular file), but
more generically, we'd want some way to compose an arbitrary number of
plugins and/or filters, where having a socketpair to each such composed
child lets us handle more than just files.
The 'nbd' plugin already has a dependency on libnbd, but allowing
socketpair background threading would make this dependency extend to
much more than that one plugin.
Any thoughts on this?
Rich.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org