[Libguestfs] [nbdkit PATCH v3 12/16] filters: New API for plugin context management

As promised in an earlier patch, get rid of the temporary nbdkit_backend_reopen, and replace it with the finer-grained combination of nbdkit_context_get_backend, nbdkit_context_next_open, nbdkit_context_next_close, and nbdkit_context_set_next. Update the retry filter to use the new API. Change the (temporary) preprocessor guard from NBDKIT_RETRY_FILTER to NBDKIT_TYPESAFE (will be dropped later once more filters are converted). Drop the now-unused backend_reopen() function. Expose .prepare and .finalize callbacks into struct nbdkit_next_ops for use during manual context manipulation. With this in place, it is now possible for a filter to open more than one context into the backend from a single connection (shown here in the retry filter), and conversely to share a single backend context across multiple connections (an upcoming patch will demonstrate this in the ext2 filter, although TODO mentions a few more steps we need to get there). --- docs/nbdkit-filter.pod | 207 ++++++++++++++++++++++++++++------------ include/nbdkit-filter.h | 27 ++++-- server/internal.h | 3 - server/backend.c | 30 +----- server/filters.c | 43 +++++++-- server/nbdkit.syms | 11 ++- filters/retry/retry.c | 42 ++++++-- TODO | 36 +++---- 8 files changed, 260 insertions(+), 139 deletions(-) diff --git a/docs/nbdkit-filter.pod b/docs/nbdkit-filter.pod index 0b120efe..d13ac1d7 100644 --- a/docs/nbdkit-filter.pod +++ b/docs/nbdkit-filter.pod @@ -132,25 +132,33 @@ C<nbdkit_next_after_fork>, C<nbdkit_next_preconnect>, C<nbdkit_next_list_exports>, C<nbdkit_next_default_export>, C<nbdkit_next_open>) and a structure called C<struct nbdkit_next_ops>. These abstract the next plugin or filter in the chain. There is also -an opaque pointer C<context> or C<nxdata> which must be passed along -when calling these functions. The value of C<nxdata> passed to -C<.prepare> has a stable lifetime that lasts to the corresponding -C<.finalize>, with all intermediate functions (such as C<.pread>) -receiving the same value for convenience. Functions where C<nxdata> -is not reused are C<.config>, C<.config_complete>, C<.get_ready>, and -C<.after_fork>, which are called during initialization outside any -connections, and C<.preconnect>, C<.list_exports>, C<.default_export>, -and C<.open> which are called based on client connections but prior to -the stable lifetime of C<.prepare>. The value of C<context> passed to -C<.open> has a lifetime that lasts until the matching C<.close> for -use by C<nbdkit_backend_reopen>. +an opaque pointer C<backend>, C<context> or C<nxdata> which must be +passed along when calling these functions. The value of C<backend> is +stable between C<.after_fork>, C<.preconnect>, C<.list_exports>, and +C<.default_export>, and can also be obtained by using +C<nbdkit_context_get_backend> on the C<context> parameter to C<.open>. + +Meanwhile, if the filter does not use C<nbdkit_context_set_next>, the +value of C<nxdata> passed to C<.prepare> has a stable lifetime that +lasts to the corresponding C<.finalize>, with all intermediate +functions (such as C<.pread>) receiving the same value for +convenience. Functions where C<nxdata> is not reused are C<.config>, +C<.config_complete>, C<.get_ready>, and C<.after_fork>, which are +called during initialization outside any connections, and +C<.preconnect>, C<.list_exports>, C<.default_export>, and C<.open> +which are called based on client connections but prior to the stable +lifetime of C<.prepare>. The value of C<context> passed to C<.open> +has a lifetime that lasts until the matching C<.close> for use by +C<nbdkit_context_get_backend>. =head2 Next config, open and close The filter’s C<.config>, C<.config_complete>, C<.get_ready>, -C<.after_fork> and C<.open> methods may only call the next C<.config>, -C<.config_complete>, C<.get_ready>, C<.after_fork> and C<.open> method -in the chain (optionally for C<.config>). +C<.after_fork>, C<.preconnect>, C<.list_exports>, C<.default_export> +and C<.open> methods may only call the next C<.config>, +C<.config_complete>, C<.get_ready>, C<.after_fork>, C<.preconnect>, +C<.list_exports>, C<.default_export> and C<.open> method in the chain +(optionally for C<.config> and C<.open>). The filter’s C<.close> method is called when an old connection closed, and this has no C<next> parameter because it cannot be @@ -158,15 +166,98 @@ short-circuited. =head2 C<next_ops> -The filter’s other methods like C<.prepare>, C<.get_size>, C<.pread> -etc ― always called in the context of a connection ― are passed a -pointer to C<struct nbdkit_next_ops> which contains a comparable set -of accessors to plugin methods that can be called during a connection. -The C<next_ops> parameter is stable between C<.prepare> and -C<.finalize>; intermediate functions (such as C<.pread>) receive the -same value for convenience. - -It is possible for a filter to issue (for example) extra +The filter generally needs to call into the underlying plugin, which +is done via a pointer to C<struct nbdkit_next_ops>. The most common +behavior is to create a next context per connection by calling the +C<next_open> parameter during C<.open>, at which point the next +context will be automatically provided to the filter’s other methods +like C<.prepare>, C<.get_size>, C<.pread> etc. The C<next_ops> struct +contains a comparable set of accessors to plugin methods that can be +called during a connection. When using automatic registration, the +C<next_ops> parameter is stable between C<.prepare> and C<.finalize>, +and nbdkit automatically prepares, finalizes, and closes the next +context at the right point in the filter connection lifecycle. + +Alternatively, the filter can manage plugin contexts manually, whether +to multiplex multiple client connections through a single context into +the plugin, or to open multiple plugin contexts to perform retries or +otherwise service a single client connection more efficiently. In +this mode of operation, the filter uses C<nbdkit_next_context_open> to +open a plugin context using the C<backend> parameter passed to +C<.after_fork>, C<.preconnect>, C<.list_exports>, C<.default_export>, +or obtained from using C<nbdkit_context_get_backend> on the C<context> +parameter to C<.open>. The resulting next context has a lifecycle +under manual control, where the filter must use C<next-E<gt>prepare +(next)> before using any other function pointers within the next +context, and must reclaim the memory using C<next-E<gt>finalize +(next)> and C<nbdkit_next_context_close> when done. A filter using +manual lifecycle management may use C<nbdkit_context_set_next> to +associate the next context into the current connection, which lets +nbdkit then pass that context as the C<next_ops> parameter to future +connection-related functions like C<.pread> and take over lifecycle +responsibility. + +=head3 C<nbdkit_context_get_backend> + +=head3 C<nbdkit_next_context_open> + +=head3 C<nbdkit_next_context_close> + +=head3 C<nbdkit_context_set_next> + + nbdkit_backend *nbdkit_context_get_backend (nbdkit_context *context); + +Obtains the backend pointer from the C<context> parameter to C<.open>, +matching the backend pointer available to C<.after_fork>, +C<.preconnect>, C<.list_exports>, and C<.default_export>. This +backend pointer has a stable lifetime from the time of C<.after_fork> +until C<.unload>. + + nbdkit_next *nbdkit_next_context_open (nbdkit_backend *backend, + int readonly, const char *exportname); + +This function attempts to open a new context into the plugin in +relation to the filter's current C<backend>. The C<readonly> and +C<exportname> parameters behave the same as documented in C<.open>. +The resulting context will be under the filter's manual lifecycle +control unless the filter associates it into the connection with +C<nbdkit_context_set_next>. The filter should be careful to not +violate any threading model restrictions of the plugin if it opens +more than one context. + + void nbdkit_next_context_close (nbdkit_next *next); + +This function closes a context into the plugin. If the context has +previously been prepared, it should first be finalized before using +this function. This function does not need to be called for a plugin +context that has been associated with the filter connection via +C<nbdkit_context_set_next> prior to the C<.close> callback. + + nbdkit_next *nbdkit_context_set_next (nbdkit_context *context, + nbdkit_next *next); + +This function associates a plugin context with the filter's current +connection context, given by the C<context> parameter to C<.open>. +Once associated, this plugin context will be given as the C<next_ops> +parameter to all other connection-specific callbacks. If associated +during C<.open>, nbdkit will take care of preparing the context prior +to C<.prepare>; if still associated before C<.finalize>, nbdkit will +take care of finalizing the context, and also for closing it. A +filter may also pass C<NULL> for C<next>, to remove any association; +if no plugin context is associated with the connection, then filter +callbacks such as C<.pread> will receive C<NULL> for their C<next_ops> +parameter. + +This function returns the previous context that had been associated +with the connection prior to switching the association to C<next>; +this result will be C<NULL> if there was no previous association. The +filter assumes manual responsibility for any remaining lifecycle +functions that must be called on the returned context. + +=head2 Using C<next_ops> + +Regardless of whether the plugin context is managed automatically or +manually, it is possible for a filter to issue (for example) extra C<next_ops-E<gt>pread> calls in response to a single C<.pwrite> call. Note that the semantics of the functions in C<struct nbdkit_next_ops> @@ -177,6 +268,9 @@ C<nbdkit_set_error> or setting C<errno>), whereas C<next_ops-E<gt>pread> exposes this via an explicit parameter, allowing a filter to learn or modify this error if desired. +Use of C<next_ops-E<gt>prepare> and C<next_ops-E<gt>finalize> is only +needed when manually managing the plugin context lifetime. + =head2 Other considerations You can modify parameters when you call the C<next> function. However @@ -189,7 +283,8 @@ for avoiding a memory leak while still obeying lifecycle constraints. Note that if your filter registers a callback but in that callback it doesn't call the C<next> function then the corresponding method in the plugin will never be called. In particular, your C<.open> method, if -you have one, B<must> call the C<.next> method. +you have one, B<must> call the C<next> method if you want the +underlying plugin to be available to all further C<next_ops> use. =head1 CALLBACKS @@ -468,31 +563,21 @@ C<nbdkit_is_tls>. The filter should generally call C<next> as its first step, to allocate from the plugin outwards, so that C<.close> running from the outer filter to the plugin will be in reverse. Skipping a call to -C<next> is acceptable if the filter will not access C<next_ops> -during any of the remaining callbacks reached on the same connection. +C<next> is acceptable if the filter will not access C<next_ops> during +any of the remaining callbacks reached on the same connection. The +C<next> function is provided for convenience; the same functionality +can be obtained manually (other than error checking) by using the +following: + + nbdkit_context_set_next (context, nbdkit_next_context_open + (nbdkit_context_get_backend (context), readonly, exportname)); The value of C<context> in this call has a lifetime that lasts until -the counterpart C<.close>, and it is this value (and not the distinct -C<nxdata> of C<.pread> and friends) that must be passed as the first -parameter to C<nbdkit_backend_reopen> by a filter attempting to retry -a connection into the backend. - -=head3 Reopening the backend - -Filters have access to a function for reopening the backend: - - int (nbdkit_context *context, int readonly, const char *exportname, - void **nxdata); - -This function is used by L<nbdkit-retry-filter(1)> to close and reopen -the underlying plugin, with possible changes to the C<readonly> and -C<exportname> parameters in relation to the original opening. It -should be used with caution because it is difficult to use safely. -The C<context> parameter to this function should be the C<context> -parameter originally passed in to C<.open>; while the C<nxdata> -pointer should be the address of C<nxdata> from any function with a -C<next_ops> parameter (such as C<.pread>) that wants to call into the -plugin after the reopen. +the counterpart C<.close>, and it is this value that may be passed to +C<nbdkit_context_get_backend> to obtain the C<backend> parameter used +to open a plugin context with C<nbdkit_next_context_open>, as well as +the C<context> parameter used to associate a plugin context into the +current connection with C<nbdkit_context_set_next>. =head2 C<.close> @@ -524,7 +609,8 @@ the plugin's C<.get_size> and C<.pread> methods via C<next_ops>). Or if you need to cleanly update superblock data in the image on close you can do it in your C<.finalize> method (calling the plugin's C<.pwrite> method). Doing these things in the filter's C<.open> or -C<.close> method is not possible. +C<.close> method is not possible without using manual context +lifecycle management. For C<.prepare>, the value of C<readonly> is the same as was passed to C<.open>, declaring how this filter will be used. @@ -541,17 +627,18 @@ overrides query functions or makes I/O calls into the plugin before handshaking is complete, where the filter needs to make those prerequisite calls manually during C<.prepare>. -There is no C<next_ops-E<gt>prepare> or C<next_ops-E<gt>finalize>. -Unlike other filter methods, prepare and finalize are not chained -through the C<next_ops> structure. Instead the core nbdkit server -calls the prepare and finalize methods of all filters. Prepare -methods are called starting with the filter closest to the plugin and -proceeding outwards (matching the order of C<.open> if all filters -call C<next> before doing anything locally), and only when an outer -filter did not skip the C<next> call during C<.open>. Finalize -methods are called in the reverse order of prepare methods, with the -outermost filter first (and matching the order of C<.close>), and only -if the prepare method succeeded. +While there are C<next_ops-E<gt>prepare> and C<next_ops-E<gt>finalize> +functions, these are different from other filter methods, in that any +plugin context associated with the current connection (via the C<next> +parameter to C<.open>, or via C<nbdkit_context_set_next>, is prepared +and finalized automatically by nbdkit, so they are only used during +manual lifecycle management. Prepare methods are called starting with +the filter closest to the plugin and proceeding outwards (matching the +order of C<.open> if all filters call C<next> before doing anything +locally), and only when an outer filter did not skip the C<next> call +during C<.open>. Finalize methods are called in the reverse order of +prepare methods, with the outermost filter first (and matching the +order of C<.close>), and only if the prepare method succeeded. If there is an error, both callbacks should call C<nbdkit_error> with an error message and return C<-1>. An error in C<.prepare> is diff --git a/include/nbdkit-filter.h b/include/nbdkit-filter.h index 3f3f61cd..a8819c9b 100644 --- a/include/nbdkit-filter.h +++ b/include/nbdkit-filter.h @@ -56,7 +56,7 @@ extern "C" { typedef struct backend nbdkit_backend; typedef struct context nbdkit_context; typedef struct context nbdkit_next; -#elif defined NBDKIT_RETRY_FILTER /* Hack to expose reopen to retry filter */ +#elif defined NBDKIT_TYPESAFE /* Temporary define while converting filters */ typedef struct nbdkit_backend nbdkit_backend; typedef struct nbdkit_context nbdkit_context; typedef struct nbdkit_next_ops nbdkit_next; @@ -81,6 +81,12 @@ typedef int nbdkit_next_open (nbdkit_context *context, int readonly, const char *exportname); struct nbdkit_next_ops { + /* These callbacks are only needed when managing the backend manually + * rather than via nbdkit_next_open. + */ + int (*prepare) (nbdkit_next *nxdata); + int (*finalize) (nbdkit_next *nxdata); + /* These callbacks are the same as normal plugin operations. */ int64_t (*get_size) (nbdkit_next *nxdata); const char * (*export_description) (nbdkit_next *nxdata); @@ -120,7 +126,7 @@ struct nbdkit_extent { uint32_t type; }; -NBDKIT_EXTERN_DECL (struct nbdkit_extents *,nbdkit_extents_new, +NBDKIT_EXTERN_DECL (struct nbdkit_extents *, nbdkit_extents_new, (uint64_t start, uint64_t end)); NBDKIT_EXTERN_DECL (void, nbdkit_extents_free, (struct nbdkit_extents *)); NBDKIT_EXTERN_DECL (size_t, nbdkit_extents_count, @@ -153,14 +159,15 @@ NBDKIT_EXTERN_DECL (size_t, nbdkit_exports_count, NBDKIT_EXTERN_DECL (const struct nbdkit_export, nbdkit_get_export, (const struct nbdkit_exports *, size_t)); -#ifdef NBDKIT_RETRY_FILTER -/* Performs close + open on the underlying chain. - * Used by the retry filter. - */ -NBDKIT_EXTERN_DECL (int, nbdkit_backend_reopen, - (nbdkit_context *context, int readonly, - const char *exportname, nbdkit_next **nxdata)); -#endif +/* Manual management of backend access. */ +NBDKIT_EXTERN_DECL (nbdkit_backend *, nbdkit_context_get_backend, + (nbdkit_context *context)); +NBDKIT_EXTERN_DECL (nbdkit_next *, nbdkit_next_context_open, + (nbdkit_backend *backend, int readonly, + const char *exportname)); +NBDKIT_EXTERN_DECL (void, nbdkit_next_context_close, (nbdkit_next *next)); +NBDKIT_EXTERN_DECL (nbdkit_next *, nbdkit_context_set_next, + (nbdkit_context *context, nbdkit_next *next)); /* Filter struct. */ struct nbdkit_filter { diff --git a/server/internal.h b/server/internal.h index b944e56d..849edbf8 100644 --- a/server/internal.h +++ b/server/internal.h @@ -430,9 +430,6 @@ extern bool backend_valid_range (struct context *c, uint64_t offset, uint32_t count) __attribute__((__nonnull__ (1))); -extern int backend_reopen (struct context *c, - int readonly, const char *exportname) - __attribute__((__nonnull__ (1, 3))); extern const char *backend_export_description (struct context *c) __attribute__((__nonnull__ (1))); extern int64_t backend_get_size (struct context *c) diff --git a/server/backend.c b/server/backend.c index 1c78556f..69625597 100644 --- a/server/backend.c +++ b/server/backend.c @@ -210,6 +210,8 @@ backend_default_export (struct backend *b, int readonly) } static struct nbdkit_next_ops next_ops = { + .prepare = backend_prepare, + .finalize = backend_finalize, .export_description = backend_export_description, .get_size = backend_get_size, .can_write = backend_can_write, @@ -363,34 +365,6 @@ backend_valid_range (struct context *c, uint64_t offset, uint32_t count) offset + count <= c->exportsize; } -/* Core functionality of nbdkit_backend_reopen for retry filter */ - -int -backend_reopen (struct context *c, int readonly, const char *exportname) -{ - struct backend *b = c->b; - - controlpath_debug ("%s: reopen readonly=%d exportname=\"%s\"", - b->next->name, readonly, exportname); - - if (c->c_next) { - if (backend_finalize (c->c_next) == -1) - return -1; - backend_close (c->c_next); - c->c_next = NULL; - } - c->c_next = backend_open (b->next, readonly, exportname); - if (c->c_next == NULL) - return -1; - if (backend_prepare (c->c_next) == -1) { - backend_finalize (c->c_next); - backend_close (c->c_next); - c->c_next = NULL; - return -1; - } - return 0; -} - /* Wrappers for all callbacks in a filter's struct nbdkit_next_ops. */ const char * diff --git a/server/filters.c b/server/filters.c index 228e2610..d40c928a 100644 --- a/server/filters.c +++ b/server/filters.c @@ -263,12 +263,14 @@ filter_default_export (struct backend *b, int readonly, int is_tls) static int next_open (struct context *c, int readonly, const char *exportname) { - struct backend *b = c->b; - struct context *c_next = backend_open (b->next, readonly, exportname); + struct backend *b = nbdkit_context_get_backend (c); + struct context *c_next = nbdkit_next_context_open (b, readonly, exportname); + struct context *old; if (c_next == NULL) return -1; - c->c_next = c_next; + old = nbdkit_context_set_next (c, c_next); + assert (old == NULL); return 0; } @@ -694,12 +696,35 @@ filter_register (struct backend *next, size_t index, const char *filename, return (struct backend *) f; } -int -nbdkit_backend_reopen (struct context *c, int readonly, - const char *exportname, struct context **nxdata) +struct backend * +nbdkit_context_get_backend (struct context *c) { - int r = backend_reopen (c, readonly, exportname); + assert (c); + return c->b; +} + +struct context * +nbdkit_next_context_open (struct backend *b, + int readonly, const char *exportname) +{ + assert (b); + return backend_open (b->next, readonly, exportname); +} + +void +nbdkit_next_context_close (struct context *c) +{ + if (c) + backend_close (c); +} + +struct context * +nbdkit_context_set_next (struct context *c, struct context *next) +{ + struct context *old; - *nxdata = c->c_next; - return r; + assert (c); + old = c->c_next; + c->c_next = next; + return old; } diff --git a/server/nbdkit.syms b/server/nbdkit.syms index 50411d27..143c4ebf 100644 --- a/server/nbdkit.syms +++ b/server/nbdkit.syms @@ -41,33 +41,36 @@ nbdkit_absolute_path; nbdkit_add_export; nbdkit_add_extent; - nbdkit_backend_reopen; + nbdkit_context_get_backend; + nbdkit_context_set_next; nbdkit_debug; nbdkit_error; nbdkit_export_name; nbdkit_exports_count; nbdkit_exports_free; nbdkit_exports_new; - nbdkit_get_export; nbdkit_extents_aligned; nbdkit_extents_count; nbdkit_extents_free; nbdkit_extents_full; nbdkit_extents_new; + nbdkit_get_export; nbdkit_get_extent; nbdkit_is_tls; nbdkit_nanosleep; + nbdkit_next_context_close; + nbdkit_next_context_open; nbdkit_parse_bool; - nbdkit_parse_int8_t; nbdkit_parse_int16_t; nbdkit_parse_int32_t; nbdkit_parse_int64_t; + nbdkit_parse_int8_t; nbdkit_parse_int; nbdkit_parse_size; - nbdkit_parse_uint8_t; nbdkit_parse_uint16_t; nbdkit_parse_uint32_t; nbdkit_parse_uint64_t; + nbdkit_parse_uint8_t; nbdkit_parse_unsigned; nbdkit_peer_gid; nbdkit_peer_name; diff --git a/filters/retry/retry.c b/filters/retry/retry.c index cfaea7d4..22527c7d 100644 --- a/filters/retry/retry.c +++ b/filters/retry/retry.c @@ -40,7 +40,7 @@ #include <string.h> #include <sys/time.h> -#define NBDKIT_RETRY_FILTER /* Hack to expose reopen */ +#define NBDKIT_TYPESAFE /* Hack to expose context APIs */ #include <nbdkit-filter.h> #include "cleanup.h" @@ -109,7 +109,7 @@ retry_config (nbdkit_next_config *next, nbdkit_backend *nxdata, struct retry_handle { int readonly; /* Save original readonly setting. */ char *exportname; /* Client exportname. */ - nbdkit_context *backend; /* Backend learned during .open. */ + nbdkit_context *context; /* Context learned during .open. */ unsigned reopens; bool open; }; @@ -131,7 +131,7 @@ retry_open (nbdkit_next_open *next, nbdkit_context *nxdata, h->readonly = readonly; h->exportname = strdup (exportname); - h->backend = nxdata; + h->context = nxdata; if (h->exportname == NULL) { nbdkit_error ("strdup: %m"); free (h); @@ -178,6 +178,8 @@ static bool do_retry (struct retry_handle *h, struct retry_data *data, nbdkit_next **next, const char *method, int *err) { + nbdkit_next *new_next, *old_next; + /* If it's the first retry, initialize the other fields in *data. */ if (data->retry == 0) data->delay = initial_delay; @@ -207,17 +209,39 @@ do_retry (struct retry_handle *h, struct retry_data *data, if (exponential_backoff) data->delay *= 2; - /* Reopen the connection. */ + /* Close the old connection. */ h->reopens++; - if (nbdkit_backend_reopen (h->backend, h->readonly || force_readonly, - h->exportname, next) == -1) { - /* If the reopen fails we treat it the same way as a command - * failing. + h->open = false; + if (*next != NULL) { + /* Failure to finalize a connection indicates permanent data loss, + * which we treat the same as the original command failing. */ - h->open = false; + if ((*next)->finalize (*next) == -1) { + *err = ESHUTDOWN; + goto again; + } + nbdkit_next_context_close (*next); + old_next = nbdkit_context_set_next (h->context, NULL); + assert (old_next == *next); + *next = NULL; + } + /* Open a new connection. */ + new_next = nbdkit_next_context_open (nbdkit_context_get_backend (h->context), + h->readonly || force_readonly, + h->exportname); + if (new_next == NULL) { + *err = ESHUTDOWN; + goto again; + } + if (new_next->prepare (new_next) == -1) { + new_next->finalize (new_next); + nbdkit_next_context_close (new_next); *err = ESHUTDOWN; goto again; } + old_next = nbdkit_context_set_next (h->context, new_next); + assert (old_next == NULL); + *next = new_next; h->open = true; /* Retry the data command. */ diff --git a/TODO b/TODO index c96a8b22..2bde2478 100644 --- a/TODO +++ b/TODO @@ -73,22 +73,13 @@ General ideas for improvements * Background thread for filters. Some filters (readahead, cache and proposed scan filter - see below) could be more effective if they - were able to defer work to a background thread. However it's not as - simple as just creating a background thread, because you only have - access to the connection / next_ops from the context of a filter - callback. (Also you can't "save" next_ops because it becomes - invalid outside the callback). The background thread would need to - have its own connection to the plugin, which would be independent of - any client connection, and this requires some care because it breaks - an existing assumption. - -* Add scan filter. This would be placed on top of cache filters and - would scan (read) the whole disk in the background, ensuring it is - copied into the cache. Useful if you have a slow plugin, limited - size device, and lots of local disk space, especially if you know - that the NBD clients will eventually read all of the device. RWMJ - wrote an implementation of this but it doesn't work well without a - background thread. + were able to defer work to a background thread. We finally have + nbdkit_next_context_open and friends for allowing a background + thread to have access into the plugin, but still need to worry about + thread-safety (how much must the filter do vs. nbdkit, to avoid + calling into the plugin too many times at once) and cleanup + (spawning the thread during .after_fork is viable, but cleaning it + up during .unload is too late). * "nbdkit.so": nbdkit as a loadable shared library. The aim of nbdkit is to make it reusable from other programs (see nbdkit-captive(1)). @@ -196,6 +187,19 @@ Python: Suggestions for filters ----------------------- +* Add scan filter. This would be placed on top of cache filters and + would scan (read) the whole disk in the background, ensuring it is + copied into the cache. Useful if you have a slow plugin, limited + size device, and lots of local disk space, especially if you know + that the NBD clients will eventually read all of the device. RWMJ + wrote an implementation of this but it doesn't work well without a + background thread. + +* Add shared filter. Take advantage of filter context APIs to open a + single context into the backend shared among multiple client + connections. This may even allow a filter to offer a more parallel + threading model than the underlying plugin. + * LUKS encrypt/decrypt filter, bonus points if compatible with qemu LUKS-encrypted disk images -- 2.30.1