On 2/10/20 1:39 PM, Richard W.M. Jones wrote:
https://github.com/libguestfs/nbdkit/blob/ecef5b16359fb5af7e7abf4fd2fb3ad543...
Already existing filters (readahead, cache) could be improved if filters could open a background work thread or threads which connect independently to the plugin. A proposed new filter (scan) cannot really be written at all without this capability.
First of all the reason this can't be done today is because filters are called with a next_ops structure which is only valid transiently during the filter callback. It cannot safely be saved or passed to another thread. (https://github.com/libguestfs/nbdkit/blob/master/docs/nbdkit-filter.pod#next...)
I was independently thinking of fixing this: we want next_ops to be have a life equal to the connection itself, rather than stack-allocated. I'm hoping to post a patch for that shortly, as part of my experimentation with implementing ext2 as a filter instead of a plugin, where ext2 has the limitation that when writing a custom io_manager, you only get ONE spot where you can pass in an opaque pointer: our .prepare will have to pass a long-lived nxdata to ext2fs_open(). But that's only a partial solution towards the rest of your goal of having threading available.
The seemingly obvious implementation - which is what I tried today - would be to let filters create background threads in .config_complete. We would provide a filter API something like:
int nbdkit_open_connection (struct nbdkit_next_ops **next_ops, void **nxdata, /* needs a close function */);
It would return a next_ops and nxdata that the filter could then use to make data calls from the background thread into the underlying layers.
To ensure safe unloading of filters and plugins we would also need a new filter callback (which I called .bg_kill) which must close all background connections opened by the filter before returning.
Seems reasonable.
I believe from my rough implementation that this is feasible. However I also thought about another way we might do this: We might open a loopback socket (eg. socketpair) which is passed to the filter and which the filter connects to using libnbd. nbdkit internally would treat this as if it was a regular external connection. Of course this would require libnbd as a dependency, or disable filters / features if not available.
There's also the question of how we would do compostion. For example, right now, the 'split' plugin basically acts as a compose operation over the 'file' plugin (you can concatenate more than one regular file), but more generically, we'd want some way to compose an arbitrary number of plugins and/or filters, where having a socketpair to each such composed child lets us handle more than just files. The 'nbd' plugin already has a dependency on libnbd, but allowing socketpair background threading would make this dependency extend to much more than that one plugin.
Any thoughts on this?
Rich.
-- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org