10:03 p.m.
On 6/17/19 7:07 PM, Eric Blake wrote:
I've mentioned this topic before (in fact, the idea of adding
NBD_CMD_FLAG_DF was first mentioned at [1]), but finally finished
enough of an implementation to feel confident in posting it.
I'd still like to add something under examples/ that uses the new API
to implement strict checking of a server's structured replies read
implementation (ensure that a server never sends data after an error
at the same offset, that it does not send overlapping data, and that
it does not report success unless all of the range is covered).
For wrapping the synchronous API, this seems simple enough:
int
wrapped_nbd_pread(struct nbd_handle *h, void *buf, size_t count,
uint64_t offset, uint32_t flags)
{
int r;
struct data *data = prep (buf, count, offset, flags);
r = nbd_pread_callback (h, buf, count, offset, data, check, flags);
return cleanup (data, r);
}
where prep() mallocs an initial structure to record the entire range of
[buf, buf+count) as not yet seen (probably by tracking a sorted linked
list of consecutive ranges initially with a single list element), each
call to the check() callback updates the list (as long as the callback
covers only bytes that were previously unseen, the list is adjusted to
remove those bytes, possible by adding or coalescing elements of the
linked list if replies come out of order; if the callback includes any
range already seen, then it returns -1), then cleanup() checks that the
linked list was properly emptied as a final chance to convert success
into an error if the server was buggy before cleaning up resources.
But for wrapping asynchronous API, I'm wishing that we had a way to
associate an arbitrary callback function to be called at the conclusion
of any given command. The existing nbd_aio_command_completed() does not
provide an easy hook for that. We could either have a single one-shot
registration that gets called for all completions (or all completions of
a given command type) by storing the callback in struct nbd_handle, as in:
nbd_set_read_completion_hook (struct nbd_handle *h, void *opaque, int
(*hook) (void *opaque, int64_t handle, int status));
Register a completion hook callback to be called on every command,
right before the command will be available in
nbd_aio_command_completed(). The hook is passed the command's handle
obtained from nbd_aio_pread or similar, as well the status of the
command so far (0 if successful, or a positive errno value); if the
callback returns -1, the command will fail even if status had been 0.
Or we can make the hook a per-handle choice, by adding counterpart aio
APIs and storing the callback in struct command_in_flight:
nbd_aio_pread_with_notify (struct nbd_handle *h, void *buf, size_t
count, uint64_t offset, void *opaque, int (*notify) (void *opaque,
int64_t handle, int status), uint32_t flags)
where notify() is called for that specific command just before it
completes. There's a small question of what to do with APIs that
already take a callback (think nbd_aio_pread_callback - do we allow the
user two separate opaque arguments, one for the per-chunk callback and
one for the final cleanup callback, or do we require the user to
tolerate the same callback in both calls?)
If we really wanted, we could even copy POSIX aio and take a struct
sigevent* parameter with each aio command (the existing nbd_aio_pread
would be shorthand for calling a new nbd_aio_pread_event where struct
sigevent is initialized to SIGEV_NONE). But that starts to feel a bit
complex (SIGEV_SIGNAL may be better if we just tell the user to call
pthread_signal from their own function callback).
Thoughts?
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org