[libnbd] More thoughts on callbacks and more
More thoughts on callbacks, etc. following on from: https://www.redhat.com/archives/libguestfs/2019-July/thread.html#00184 Closure lifetimes ----------------- Closures could have a lifetime if we had a little bit of support from the C library. We would generate (from C only): nbd_set_free_<fn>_<closure> (nbd, free_closure); which calls free_closure (user_data) as soon as the closure will no longer be called by the library. This function would be used to decrement the refcount from Python or remove the global root from OCaml. Note this is a family of functions, eg: nbd_set_free_set_debug_callback_debug_fn corresponding to the debug_fn arg of nbd_set_debug_callback. Luckily they can all be generated along with the internal machinery to call them. Buffer lifetimes ---------------- Similar to the above, persistent buffers (BytesPersist*) can have lifetimes. Remove nbd_add_close_callback ----------------------------- The above changes (actually, just the closure change) lets us remove nbd_add_close_callback. Fixing callback / cookie problem -------------------------------- I think an easier way to fix this would be to simply detect the problematic situation and queue up the callback to be call the next time the library is entered. Detecting the situation is fairly easy - we know the current cookie number, so can check it before we call the completion function. And calling the deferred callback(s) is easy from the generated code. The only questions in my mind are: (1) If this could cause a deadlock because the caller is waiting for the completion signal before reentering the library. (2) If there's some implicit guarantee that we should call the callback as easy as possible after the command finishes. I think (1) is not possible in ordinary code since callers must always enter the main loop after calling one of the nbd_aio_* functions. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW
On Sat, Jul 20, 2019 at 07:38:45AM +0100, Richard W.M. Jones wrote:
More thoughts on callbacks, etc. following on from: https://www.redhat.com/archives/libguestfs/2019-July/thread.html#00184
Closure lifetimes -----------------
Closures could have a lifetime if we had a little bit of support from the C library. We would generate (from C only):
nbd_set_free_<fn>_<closure> (nbd, free_closure);
which calls free_closure (user_data) as soon as the closure will no longer be called by the library. This function would be used to decrement the refcount from Python or remove the global root from OCaml.
Note this is a family of functions, eg:
nbd_set_free_set_debug_callback_debug_fn
corresponding to the debug_fn arg of nbd_set_debug_callback. Luckily they can all be generated along with the internal machinery to call them.
As written above this doesn't quite work. However it could work to pass an optional free function with the closure. In other words it would look like: struct nbd_closure { .cl = my_debug_fn, .user_data = foo, .free = my_free } cl; nbd_set_debug_callback (nbd, &cl); cl->free (cl->user_data) is called if cl->free != NULL when the closure is no longer used by the library. This is a bit of a change to the API however. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/
On Mon, Jul 22, 2019 at 10:08:25AM +0100, Richard W.M. Jones wrote:
On Sat, Jul 20, 2019 at 07:38:45AM +0100, Richard W.M. Jones wrote:
More thoughts on callbacks, etc. following on from: https://www.redhat.com/archives/libguestfs/2019-July/thread.html#00184
Closure lifetimes -----------------
Here's a possibly better idea which still changes the API a bit but not as invasively. We overload the callback so that it can either be a callback function or a "free function". They are distinguished by an extra flag argument passed to the callback: extern int nbd_set_debug_callback ( struct nbd_handle *h, int (*debug_fn) (int valid_flag, // <-- note void *user_data, const char *context, const char *msg), void *user_data); The extra ‘valid_flag’ contains one or both of LIBNBD_CALLBACK_VALID and LIBNBD_CALLBACK_FREE. Callback code would look like: int my_debug_fn (int valid_flag, void *user_data, const char *context, const char *msg) { if (valid_flag & LIBNBD_CALLBACK_VALID) { // This is the normal callback as before. printf ("debug msg = %s\n", msg); } if (valid_flag & LIBNBD_CALLBACK_FREE) { // In this case only user_data is valid. The other parameters // may be unspecified (unless VALID was also set). free (user_data); } } Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-builder quickly builds VMs from scratch http://libguestfs.org/virt-builder.1.html
On 7/22/19 6:50 AM, Richard W.M. Jones wrote:
On Mon, Jul 22, 2019 at 10:08:25AM +0100, Richard W.M. Jones wrote:
On Sat, Jul 20, 2019 at 07:38:45AM +0100, Richard W.M. Jones wrote:
More thoughts on callbacks, etc. following on from: https://www.redhat.com/archives/libguestfs/2019-July/thread.html#00184
Closure lifetimes -----------------
Here's a possibly better idea which still changes the API a bit but not as invasively.
We overload the callback so that it can either be a callback function or a "free function". They are distinguished by an extra flag argument passed to the callback:
extern int nbd_set_debug_callback ( struct nbd_handle *h, int (*debug_fn) (int valid_flag, // <-- note void *user_data, const char *context, const char *msg), void *user_data);
Would the 'valid_flag' be presented to non-C bindings, or is it only needed for C code? At any rate, the idea makes sense as a lighter-weight way for libnbd to always inform the callback about the last invocation. For nbd_aio_pread_structured and nbd_aio_block_status, the chunk/extent callback can either match server state (if the server replies replies with NBD_REPLY_FLAG_DONE on an OFFSET_DATA/BLOCK_STATUS reply, we could set VALID|FREE; if the server defers NBD_REPLY_FLAG_DONE to an NBD_REPLY_TYPE_NONE packet then the two flags will definitely not be set at the same time), or we could always defer the FREE flag until the overall command is ready to retire; for nbd_aio_FOO_callback, it may indeed be easier to set VALID|FREE on the single use of the callback at the time it is ready to retire. But this idea does make it possible for libnbd to inform the callback about its last expected use.
The extra ‘valid_flag’ contains one or both of LIBNBD_CALLBACK_VALID and LIBNBD_CALLBACK_FREE. Callback code would look like:
int my_debug_fn (int valid_flag, void *user_data, const char *context, const char *msg) { if (valid_flag & LIBNBD_CALLBACK_VALID) { // This is the normal callback as before. printf ("debug msg = %s\n", msg); }
if (valid_flag & LIBNBD_CALLBACK_FREE) { // In this case only user_data is valid. The other parameters // may be unspecified (unless VALID was also set). free (user_data); } }
I also like the idea of letting the callback handle both flags in a single call where sensible, for fewer indirect function calls. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
On Mon, Jul 22, 2019 at 08:13:12AM -0500, Eric Blake wrote:
On 7/22/19 6:50 AM, Richard W.M. Jones wrote:
On Mon, Jul 22, 2019 at 10:08:25AM +0100, Richard W.M. Jones wrote:
On Sat, Jul 20, 2019 at 07:38:45AM +0100, Richard W.M. Jones wrote:
More thoughts on callbacks, etc. following on from: https://www.redhat.com/archives/libguestfs/2019-July/thread.html#00184
Closure lifetimes -----------------
Here's a possibly better idea which still changes the API a bit but not as invasively.
We overload the callback so that it can either be a callback function or a "free function". They are distinguished by an extra flag argument passed to the callback:
extern int nbd_set_debug_callback ( struct nbd_handle *h, int (*debug_fn) (int valid_flag, // <-- note void *user_data, const char *context, const char *msg), void *user_data);
Would the 'valid_flag' be presented to non-C bindings, or is it only needed for C code?
C only. It's not needed for languages which have garbage collection. For non-C languages without GC, well I guess we can decide on a case-by-case basis.
At any rate, the idea makes sense as a lighter-weight way for libnbd to always inform the callback about the last invocation.
I've almost got a working implementation. Will post it for review as soon as I can. [..] Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top
This has an annoying subtlety around the fact that we can pass a single user_data and multiple closures in one function call. The LIBNBD_CALLBACK_FREE function would be called several times with the same user_data in this case, which means the callback must do some kind of reference counting before the user_data can be freed. I propose that we split up Closure so it describes a single closure, although that does mean that separate user_data must be passed with each function pointer (which may not be a bad thing). Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW
On 7/22/19 9:25 AM, Richard W.M. Jones wrote:
This has an annoying subtlety around the fact that we can pass a single user_data and multiple closures in one function call.
The LIBNBD_CALLBACK_FREE function would be called several times with the same user_data in this case, which means the callback must do some kind of reference counting before the user_data can be freed.
I propose that we split up Closure so it describes a single closure, although that does mean that separate user_data must be passed with each function pointer (which may not be a bad thing).
Indeed, and it is a bit more API churn (nbd_aio_pread_structured_allback and nbd_aio_block_status_callback would now take two void* user_data parameters, which means they must have different parameter names: maybe extent_user_data vs. callback_user_data), but it makes sense. But it also simplifies 'Closure' in the generator, as you no longer have an array of callback functions that share a common data. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
participants (2)
-
Eric Blake -
Richard W.M. Jones