Re: [Libguestfs] nbdkit background threads

On Mon, Feb 10, 2020 at 01:52:25PM -0600, Eric Blake wrote: > On 2/10/20 1:39 PM, Richard W.M. Jones wrote: >> https://github.com/libguestfs/nbdkit/blob/ecef5b16359fb5af7e7abf4fd2fb3ad... >> >> Already existing filters (readahead, cache) could be improved if >> filters could open a background work thread or threads which connect >> independently to the plugin. A proposed new filter (scan) cannot >> really be written at all without this capability. >> >> First of all the reason this can't be done today is because filters >> are called with a next_ops structure which is only valid transiently >> during the filter callback. It cannot safely be saved or passed to >> another thread. >> (https://github.com/libguestfs/nbdkit/blob/master/docs/nbdkit-filter.pod#n...) > > I was independently thinking of fixing this: we want next_ops to be > have a life equal to the connection itself, rather than > stack-allocated. I'm hoping to post a patch for that shortly, as > part of my experimentation with implementing ext2 as a filter > instead of a plugin, where ext2 has the limitation that when writing > a custom io_manager, you only get ONE spot where you can pass in an > opaque pointer: our .prepare will have to pass a long-lived nxdata > to ext2fs_open(). Yes I can see this could be tricky, especially in the presence of multiple threads serving a single connection. The only alternative way I can think to do it would be to use a pointer in thread-local storage which would would update with next_ops / nxdata each time. Do you have a preliminary patch for this yet, or do you want me to look at it today?