[Libguestfs] [libnbd PATCH v4 10/25] api: Add [aio_]nbd_block_status_64

Overcome the inherent 32-bit limitation of our existing nbd_block_status command by adding a 64-bit variant. The command sent to the server does not change, but the user's callback is now handed 64-bit information regardless of whether the server replies with 32- or 64-bit extents. Unit tests will be added in the next patch. We can also get rid of the temporary hack added to appease the compiler in the previous patch. Signed-off-by: Eric Blake <eblake(a)redhat.com&gt; --- v4: major rework, split unit tests into subsequent patch --- docs/libnbd.pod | 18 ++-- sh/nbdsh.pod | 2 +- generator/API.ml | 146 +++++++++++++++++++++++++++++---- generator/states-reply-chunk.c | 2 - generator/OCaml.ml | 1 - generator/Python.ml | 1 - lib/rw.c | 68 ++++++++++++--- fuzzing/libnbd-fuzz-wrapper.c | 20 ++++- 8 files changed, 218 insertions(+), 40 deletions(-) diff --git a/docs/libnbd.pod b/docs/libnbd.pod index 997da0df..2a26bbda 100644 --- a/docs/libnbd.pod +++ b/docs/libnbd.pod @@ -668,14 +668,14 @@ In order to utilize block status, the client must call L<nbd_add_meta_context(3)> prior to connecting, for each meta context in which it is interested, then check L<nbd_can_meta_context(3)> after connection to see which contexts the server actually supports. If a -context is supported, the client can then use L<nbd_block_status(3)> -with a callback function that will receive an array of 32-bit integer -pairs describing consecutive extents within a context. In each pair, -the first integer is the length of the extent, the second is a bitmask -description of that extent (for the "base:allocation" context, the -bitmask may include C<LIBNBD_STATE_HOLE> for unallocated portions of -the file, and/or C<LIBNBD_STATE_ZERO> for portions of the file known -to read as zero). +context is supported, the client can then use +L<nbd_block_status_64(3)> with a callback function that will receive +an array of structs describing consecutive extents within a context. +Each struct gives the length of the extent, then a bitmask description +of that extent (for the "base:allocation" context, the bitmask may +include C<LIBNBD_STATE_HOLE> for unallocated portions of the file, +and/or C<LIBNBD_STATE_ZERO> for portions of the file known to read as +zero). There is a full example of requesting meta context and using block status available at @@ -933,7 +933,7 @@ will tie up resources until L<nbd_close(3)> is eventually reached). =head2 Callbacks with C<int *error> parameter Some of the high-level commands (L<nbd_pread_structured(3)>, -L<nbd_block_status(3)>) involve the use of a callback function invoked +L<nbd_block_status_64(3)>) involve the use of a callback function invoked by the state machine at appropriate points in the server's reply before the overall command is complete. These callback functions, along with all of the completion callbacks, include a parameter diff --git a/sh/nbdsh.pod b/sh/nbdsh.pod index d50b738a..d222c0ff 100644 --- a/sh/nbdsh.pod +++ b/sh/nbdsh.pod @@ -59,7 +59,7 @@ Display brief command line help and exit. =item B<--base-allocation> Request the use of the "base:allocation" meta context, which is the -most common context used with L<nbd_block_status(3)>. This is +most common context used with L<nbd_block_status_64(3)>. This is equivalent to calling S<C<h.set_meta_context(nbd.CONTEXT_BASE_ALLOCATION)>> in the shell prior to connecting, and works even when combined with C<--uri> (while diff --git a/generator/API.ml b/generator/API.ml index 858d86df..6843223f 100644 --- a/generator/API.ml +++ b/generator/API.ml @@ -1565,7 +1565,7 @@ "add_meta_context", { During connection libnbd can negotiate zero or more metadata contexts with the server. Metadata contexts are features (such as C<\"base:allocation\">) which describe information returned -by the L<nbd_block_status(3)> command (for C<\"base:allocation\"> +by the L<nbd_block_status_64(3)> command (for C<\"base:allocation\"> this is whether blocks of data are allocated, zero or sparse). This call adds one metadata context to the list to be negotiated. @@ -1592,7 +1592,7 @@ "add_meta_context", { Other metadata contexts are server-specific, but include C<\"qemu:dirty-bitmap:...\"> and C<\"qemu:allocation-depth\"> for qemu-nbd (see qemu-nbd I<-B> and I<-A> options)."; - see_also = [Link "block_status"; Link "can_meta_context"; + see_also = [Link "block_status_64"; Link "can_meta_context"; Link "get_nr_meta_contexts"; Link "get_meta_context"; Link "clear_meta_contexts"]; }; @@ -1605,14 +1605,14 @@ "get_nr_meta_contexts", { During connection libnbd can negotiate zero or more metadata contexts with the server. Metadata contexts are features (such as C<\"base:allocation\">) which describe information returned -by the L<nbd_block_status(3)> command (for C<\"base:allocation\"> +by the L<nbd_block_status_64(3)> command (for C<\"base:allocation\"> this is whether blocks of data are allocated, zero or sparse). This command returns how many meta contexts have been added to the list to request from the server via L<nbd_add_meta_context(3)>. The server is not obligated to honor all of the requests; to see what it actually supports, see L<nbd_can_meta_context(3)>."; - see_also = [Link "block_status"; Link "can_meta_context"; + see_also = [Link "block_status_64"; Link "can_meta_context"; Link "add_meta_context"; Link "get_meta_context"; Link "clear_meta_contexts"]; }; @@ -1625,13 +1625,13 @@ "get_meta_context", { During connection libnbd can negotiate zero or more metadata contexts with the server. Metadata contexts are features (such as C<\"base:allocation\">) which describe information returned -by the L<nbd_block_status(3)> command (for C<\"base:allocation\"> +by the L<nbd_block_status_64(3)> command (for C<\"base:allocation\"> this is whether blocks of data are allocated, zero or sparse). This command returns the i'th meta context request, as added by L<nbd_add_meta_context(3)>, and bounded by L<nbd_get_nr_meta_contexts(3)>."; - see_also = [Link "block_status"; Link "can_meta_context"; + see_also = [Link "block_status_64"; Link "can_meta_context"; Link "add_meta_context"; Link "get_nr_meta_contexts"; Link "clear_meta_contexts"]; }; @@ -1645,7 +1645,7 @@ "clear_meta_contexts", { During connection libnbd can negotiate zero or more metadata contexts with the server. Metadata contexts are features (such as C<\"base:allocation\">) which describe information returned -by the L<nbd_block_status(3)> command (for C<\"base:allocation\"> +by the L<nbd_block_status_64(3)> command (for C<\"base:allocation\"> this is whether blocks of data are allocated, zero or sparse). This command resets the list of meta contexts to request back to @@ -1654,7 +1654,7 @@ "clear_meta_contexts", { negotiation mode is selected (see L<nbd_set_opt_mode(3)>), for altering the list of attempted contexts between subsequent export queries."; - see_also = [Link "block_status"; Link "can_meta_context"; + see_also = [Link "block_status_64"; Link "can_meta_context"; Link "add_meta_context"; Link "get_nr_meta_contexts"; Link "get_meta_context"; Link "set_opt_mode"]; }; @@ -2281,7 +2281,7 @@ "can_meta_context", { ^ non_blocking_test_call_description; see_also = [SectionLink "Flag calls"; Link "opt_info"; Link "add_meta_context"; - Link "block_status"; Link "aio_block_status"; + Link "block_status_64"; Link "aio_block_status_64"; Link "set_request_meta_context"; Link "opt_set_meta_context"]; }; @@ -2712,7 +2712,7 @@ "block_status", { optargs = [ OFlags ("flags", cmd_flags, Some ["REQ_ONE"]) ]; ret = RErr; permitted_states = [ Connected ]; - shortdesc = "send block status command to the NBD server"; + shortdesc = "send block status command, with 32-bit callback"; longdesc = "\ Issue the block status command to the NBD server. If supported by the server, this causes metadata context @@ -2727,7 +2727,15 @@ "block_status", { The NBD protocol does not yet have a way for a client to learn if the server will enforce an even smaller maximum block status size, although a future extension may add a constraint visible in -L<nbd_get_block_size(3)>. +L<nbd_get_block_size(3)>. Furthermore, this function is inherently +limited to 32-bit values. If the server replies with a larger +extent, the length of that extent will be truncated to just +below 32 bits and any further extents from the server will be +ignored. If the server replies with a status value larger than +32 bits (only possible when extended headers are in use), the +callback function will be passed an C<EOVERFLOW> error. To get +the full extent information from a server that supports 64-bit +extents, you must use L<nbd_block_status_64(3)>. Depending on which metadata contexts were enabled before connecting (see L<nbd_add_meta_context(3)>) and which are @@ -2747,7 +2755,7 @@ "block_status", { array is an array of pairs of integers with the first entry in each pair being the length (in bytes) of the block and the second entry being a status/flags field which is specific to the metadata context. -(The number of pairs passed to the function is C<nr_entries/2>.) The +The number of pairs passed to the function is C<nr_entries/2>. The NBD protocol document in the section about C<NBD_REPLY_TYPE_BLOCK_STATUS> describes the meaning of this array; for contexts known to libnbd, B<E<lt>libnbd.hE<gt>> contains constants @@ -2771,10 +2779,79 @@ "block_status", { does not exceed C<count> bytes; however, libnbd does not validate that the server obeyed the flag." ^ strict_call_description; - see_also = [Link "add_meta_context"; Link "can_meta_context"; + see_also = [Link "block_status_64"; + Link "add_meta_context"; Link "can_meta_context"; Link "aio_block_status"; Link "set_strict_mode"]; }; + "block_status_64", { + default_call with + args = [ UInt64 "count"; UInt64 "offset"; Closure extent64_closure ]; + optargs = [ OFlags ("flags", cmd_flags, Some ["REQ_ONE"]) ]; + ret = RErr; + permitted_states = [ Connected ]; + shortdesc = "send block status command, with 64-bit callback"; + longdesc = "\ +Issue the block status command to the NBD server. If +supported by the server, this causes metadata context +information about blocks beginning from the specified +offset to be returned. The C<count> parameter is a hint: the +server may choose to return less status, or the final block +may extend beyond the requested range. If multiple contexts +are supported, the number of blocks and cumulative length +of those blocks need not be identical between contexts. + +Note that not all servers can support a C<count> of 4GiB or larger. +The NBD protocol does not yet have a way for a client to learn if +the server will enforce an even smaller maximum block status size, +although a future extension may add a constraint visible in +L<nbd_get_block_size(3)>. + +Depending on which metadata contexts were enabled before +connecting (see L<nbd_add_meta_context(3)>) and which are +supported by the server (see L<nbd_can_meta_context(3)>) this call +returns information about extents by calling back to the +C<extent64> function. The callback cannot call C<nbd_*> APIs on the +same handle since it holds the handle lock and will +cause a deadlock. If the callback returns C<-1>, and no earlier +error has been detected, then the overall block status command +will fail with any non-zero value stored into the callback's +C<error> parameter (with a default of C<EPROTO>); but any further +contexts will still invoke the callback. + +The C<extent64> function is called once per type of metadata available, +with the C<user_data> passed to this function. The C<metacontext> +parameter is a string such as C<\"base:allocation\">. The C<entries> +array is an array of B<nbd_extent> structs, containing length (in bytes) +of the block and a status/flags field which is specific to the metadata +context. The number of array entries passed to the function is +C<nr_entries>. The NBD protocol document in the section about +C<NBD_REPLY_TYPE_BLOCK_STATUS> describes the meaning of this array; +for contexts known to libnbd, B<E<lt>libnbd.hE<gt>> contains constants +beginning with C<LIBNBD_STATE_> that may help decipher the values. +On entry to the callback, the C<error> parameter contains the errno +value of any previously detected error. + +It is possible for the extent function to be called +more times than you expect (if the server is buggy), +so always check the C<metacontext> field to ensure you +are receiving the data you expect. It is also possible +that the extent function is not called at all, even for +metadata contexts that you requested. This indicates +either that the server doesn't support the context +or for some other reason cannot return the data. + +The C<flags> parameter may be C<0> for no flags, or may contain +C<LIBNBD_CMD_FLAG_REQ_ONE> meaning that the server should +return only one extent per metadata context where that extent +does not exceed C<count> bytes; however, libnbd does not +validate that the server obeyed the flag." +^ strict_call_description; + see_also = [Link "block_status"; + Link "add_meta_context"; Link "can_meta_context"; + Link "aio_block_status_64"; Link "set_strict_mode"]; + }; + "poll", { default_call with args = [ Int "timeout" ]; ret = RInt; @@ -3369,7 +3446,7 @@ "aio_block_status", { OFlags ("flags", cmd_flags, Some ["REQ_ONE"]) ]; ret = RCookie; permitted_states = [ Connected ]; - shortdesc = "send block status command to the NBD server"; + shortdesc = "send block status command, with 32-bit callback"; longdesc = "\ Send the block status command to the NBD server. @@ -3377,13 +3454,48 @@ "aio_block_status", { Or supply the optional C<completion_callback> which will be invoked as described in L<libnbd(3)/Completion callbacks>. -Other parameters behave as documented in L<nbd_block_status(3)>." +Other parameters behave as documented in L<nbd_block_status(3)>. + +This function is inherently limited to 32-bit values. If the +server replies with a larger extent, the length of that extent +will be truncated to just below 32 bits and any further extents +from the server will be ignored. If the server replies with a +status value larger than 32 bits (only possible when extended +headers are in use), the callback function will be passed an +C<EOVERFLOW> error. To get the full extent information from a +server that supports 64-bit extents, you must use +L<nbd_aio_block_status_64(3)>. +" ^ strict_call_description; see_also = [SectionLink "Issuing asynchronous commands"; + Link "aio_block_status_64"; Link "can_meta_context"; Link "block_status"; Link "set_strict_mode"]; }; + "aio_block_status_64", { + default_call with + args = [ UInt64 "count"; UInt64 "offset"; Closure extent64_closure ]; + optargs = [ OClosure completion_closure; + OFlags ("flags", cmd_flags, Some ["REQ_ONE"]) ]; + ret = RCookie; + permitted_states = [ Connected ]; + shortdesc = "send block status command, with 64-bit callback"; + longdesc = "\ +Send the block status command to the NBD server. + +To check if the command completed, call L<nbd_aio_command_completed(3)>. +Or supply the optional C<completion_callback> which will be invoked +as described in L<libnbd(3)/Completion callbacks>. + +Other parameters behave as documented in L<nbd_block_status_64(3)>." +^ strict_call_description; + see_also = [SectionLink "Issuing asynchronous commands"; + Link "aio_block_status"; + Link "can_meta_context"; Link "block_status_64"; + Link "set_strict_mode"]; + }; + "aio_get_fd", { default_call with args = []; ret = RFd; @@ -3909,6 +4021,10 @@ let first_version = "set_socket_activation_name", (1, 16); "get_socket_activation_name", (1, 16); + (* Added in 1.17.x development cycle, will be stable and supported in 1.18. *) + "block_status_64", (1, 18); + "aio_block_status_64", (1, 18); + (* These calls are proposed for a future version of libnbd, but * have not been added to any released version so far. "get_tls_certificates", (1, ??); diff --git a/generator/states-reply-chunk.c b/generator/states-reply-chunk.c index 796262d2..2cebe456 100644 --- a/generator/states-reply-chunk.c +++ b/generator/states-reply-chunk.c @@ -564,8 +564,6 @@ REPLY.CHUNK_REPLY.RECV_BS_ENTRIES: * error, and to simplify alignment, we truncate to 4G-64M); but * do not diagnose issues with the server's length alignments, * flag values, nor compliance with the REQ_ONE command flag. - * - * FIXME: still need to add nbd_block_status_64 API */ for (i = 0, stop = false; i < h->bs_count && !stop; ++i) { if (type == NBD_REPLY_TYPE_BLOCK_STATUS) { diff --git a/generator/OCaml.ml b/generator/OCaml.ml index ba4c8403..621a4348 100644 --- a/generator/OCaml.ml +++ b/generator/OCaml.ml @@ -592,7 +592,6 @@ let pr "}\n"; pr "\n"; pr "static int\n"; - pr "__attribute__ ((unused)) /* XXX temporary hack */\n"; pr "%s_wrapper " cbname; C.print_cbarg_list ~wrap:true cbargs; pr "\n"; diff --git a/generator/Python.ml b/generator/Python.ml index 199d0265..761f4511 100644 --- a/generator/Python.ml +++ b/generator/Python.ml @@ -158,7 +158,6 @@ let let print_python_closure_wrapper { cbname; cbargs } = pr "/* Wrapper for %s callback. */\n" cbname; pr "static int\n"; - pr "__attribute__ ((unused)) /* XXX temporary hack */\n"; pr "%s_wrapper " cbname; C.print_cbarg_list ~wrap:true cbargs; pr "\n"; diff --git a/lib/rw.c b/lib/rw.c index 15b309ee..8e8c024c 100644 --- a/lib/rw.c +++ b/lib/rw.c @@ -150,7 +150,7 @@ nbd_unlocked_zero (struct nbd_handle *h, return wait_for_command (h, cookie); } -/* Issue a block status command and wait for the reply. */ +/* Issue a block status command and wait for the reply, 32-bit callback. */ int nbd_unlocked_block_status (struct nbd_handle *h, uint64_t count, uint64_t offset, @@ -168,6 +168,25 @@ nbd_unlocked_block_status (struct nbd_handle *h, return wait_for_command (h, cookie); } +/* Issue a block status command and wait for the reply, 64-bit callback. */ +int +nbd_unlocked_block_status_64 (struct nbd_handle *h, + uint64_t count, uint64_t offset, + nbd_extent64_callback *extent64, + uint32_t flags) +{ + int64_t cookie; + nbd_completion_callback c = NBD_NULL_COMPLETION; + + cookie = nbd_unlocked_aio_block_status_64 (h, count, offset, extent64, &c, + flags); + if (cookie == -1) + return -1; + + assert (CALLBACK_IS_NULL (*extent64)); + return wait_for_command (h, cookie); +} + /* count_err represents the errno to return if bounds check fail */ int64_t nbd_internal_command_common (struct nbd_handle *h, @@ -484,16 +503,10 @@ nbd_unlocked_aio_zero (struct nbd_handle *h, count, ENOSPC, NULL, &cb); } -int64_t -nbd_unlocked_aio_block_status (struct nbd_handle *h, - uint64_t count, uint64_t offset, - nbd_extent_callback *extent, - nbd_completion_callback *completion, - uint32_t flags) +static int +check_aio_block_status (struct nbd_handle *h, uint64_t count, uint64_t offset, + uint32_t flags) { - struct command_cb cb = { .fn.extent32 = *extent, .wide = false, - .completion = *completion }; - if (h->strict & LIBNBD_STRICT_COMMANDS) { if (!h->structured_replies) { set_error (ENOTSUP, "server does not support structured replies"); @@ -508,8 +521,43 @@ nbd_unlocked_aio_block_status (struct nbd_handle *h, } } + return 0; +} + +int64_t +nbd_unlocked_aio_block_status (struct nbd_handle *h, + uint64_t count, uint64_t offset, + nbd_extent_callback *extent, + nbd_completion_callback *completion, + uint32_t flags) +{ + struct command_cb cb = { .fn.extent32 = *extent, .wide = false, + .completion = *completion }; + + if (check_aio_block_status (h, count, offset, flags) == -1) + return -1; + SET_CALLBACK_TO_NULL (*extent); SET_CALLBACK_TO_NULL (*completion); return nbd_internal_command_common (h, flags, NBD_CMD_BLOCK_STATUS, offset, count, EINVAL, NULL, &cb); } + +int64_t +nbd_unlocked_aio_block_status_64 (struct nbd_handle *h, + uint64_t count, uint64_t offset, + nbd_extent64_callback *extent64, + nbd_completion_callback *completion, + uint32_t flags) +{ + struct command_cb cb = { .fn.extent64 = *extent64, .wide = true, + .completion = *completion }; + + if (check_aio_block_status (h, count, offset, flags) == -1) + return -1; + + SET_CALLBACK_TO_NULL (*extent64); + SET_CALLBACK_TO_NULL (*completion); + return nbd_internal_command_common (h, flags, NBD_CMD_BLOCK_STATUS, offset, + count, EINVAL, NULL, &cb); +} diff --git a/fuzzing/libnbd-fuzz-wrapper.c b/fuzzing/libnbd-fuzz-wrapper.c index e111866f..cbd55380 100644 --- a/fuzzing/libnbd-fuzz-wrapper.c +++ b/fuzzing/libnbd-fuzz-wrapper.c @@ -165,6 +165,17 @@ extent_callback (void *user_data, return 0; } +/* Block status (extents64) callback, does nothing. */ +static int +extent64_callback (void *user_data, + const char *metacontext, + uint64_t offset, nbd_extent *entries, + size_t nr_entries, int *error) +{ + //fprintf (stderr, "extent64 called, nr_entries = %zu\n", nr_entries); + return 0; +} + /* This is the client (parent process) running libnbd. */ static char buf[512]; static char prbuf[65536]; @@ -213,7 +224,7 @@ client (int sock) }, 0); - /* Test block status. */ + /* Test both sizes of block status. */ nbd_block_status (nbd, length, 0, (nbd_extent_callback) { .callback = extent_callback, @@ -221,6 +232,13 @@ client (int sock) .free = NULL }, 0); + nbd_block_status_64 (nbd, length, 0, + (nbd_extent64_callback) { + .callback = extent64_callback, + .user_data = NULL, + .free = NULL + }, + 0); nbd_shutdown (nbd, 0); } -- 2.41.0