[libnbd PATCH] docs: Mention explicit and inherent limits on lengths
Prior to implementing an NBD extension that will allow some commands to accept a 64-bit count, it is worth documenting places where there are 32-bit inherent limits, as well as where libnbd itself enforces a smaller arbitrary limit, in spite of the API having a size_t or uint64_t parameter. --- generator/API.ml | 45 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 44 insertions(+), 1 deletion(-) diff --git a/generator/API.ml b/generator/API.ml index 602f51e0..cf2e7543 100644 --- a/generator/API.ml +++ b/generator/API.ml @@ -1,6 +1,6 @@ (* hey emacs, this is OCaml code: -*- tuareg -*- *) (* nbd client library in userspace: the API - * Copyright (C) 2013-2020 Red Hat Inc. + * Copyright (C) 2013-2021 Red Hat Inc. * * This library is free software; you can redistribute it and/or * modify it under the terms of the GNU Lesser General Public @@ -1822,6 +1822,12 @@ error. See also L<nbd_pread_structured(3)>, if finer visibility is required into the server's replies, or if you want to use C<LIBNBD_CMD_FLAG_DF>. +Note that libnbd currently enforces a maximum read buffer of 64MiB, +even if the server would permit a larger buffer in a single transaction; +attempts to exceed this will result in an C<ERANGE> error. The server +may enforce a smaller limit, which can be learned with +L<nbd_get_block_size(3)>. + The C<flags> parameter must be C<0> for now (it exists for future NBD protocol extensions)." ^ strict_call_description; @@ -1898,6 +1904,12 @@ the server obeyed the requirement that a read call must not have overlapping chunks and must not succeed without enough chunks to cover the entire request. +Note that libnbd currently enforces a maximum read buffer of 64MiB, +even if the server would permit a larger buffer in a single transaction; +attempts to exceed this will result in an C<ERANGE> error. The server +may enforce a smaller limit, which can be learned with +L<nbd_get_block_size(3)>. + The C<flags> parameter may be C<0> for no flags, or may contain C<LIBNBD_CMD_FLAG_DF> meaning that the server should not reply with more than one fragment (if that is supported - some servers cannot do @@ -1924,6 +1936,12 @@ using this call. The call returns when the command has been acknowledged by the server, or there is an error. Note this will generally return an error if L<nbd_is_read_only(3)> is true. +Note that libnbd currently enforces a maximum write buffer of 64MiB, +even if the server would permit a larger buffer in a single transaction; +attempts to exceed this will result in an C<ERANGE> error. The server +may enforce a smaller limit, which can be learned with +L<nbd_get_block_size(3)>. + The C<flags> parameter may be C<0> for no flags, or may contain C<LIBNBD_CMD_FLAG_FUA> meaning that the server should not return until the data has been committed to permanent storage @@ -2007,6 +2025,12 @@ The call returns when the command has been acknowledged by the server, or there is an error. Note this will generally return an error if L<nbd_can_trim(3)> is false or L<nbd_is_read_only(3)> is true. +Note that not all servers can support a C<count> of 4GiB or larger. +The NBD protocol does not yet have a way for a client to learn if +the server will enforce an even smaller maximum trim size, although +a future extension may add a constraint visible in +L<nbd_get_block_size(3)>. + The C<flags> parameter may be C<0> for no flags, or may contain C<LIBNBD_CMD_FLAG_FUA> meaning that the server should not return until the data has been committed to permanent storage @@ -2032,6 +2056,12 @@ L<nbd_pread(3)> call. The server can also silently ignore this command. Note this will generally return an error if L<nbd_can_cache(3)> is false. +Note that not all servers can support a C<count> of 4GiB or larger. +The NBD protocol does not yet have a way for a client to learn if +the server will enforce an even smaller maximum cache size, although +a future extension may add a constraint visible in +L<nbd_get_block_size(3)>. + The C<flags> parameter must be C<0> for now (it exists for future NBD protocol extensions)." ^ strict_call_description; @@ -2055,6 +2085,13 @@ The call returns when the command has been acknowledged by the server, or there is an error. Note this will generally return an error if L<nbd_can_zero(3)> is false or L<nbd_is_read_only(3)> is true. +Note that not all servers can support a C<count> of 4GiB or larger. +The NBD protocol does not yet have a way for a client to learn if +the server will enforce an even smaller maximum zero size, although +a future extension may add a constraint visible in +L<nbd_get_block_size(3)>. Also, some servers may permit a larger +zero request only when the C<LIBNBD_CMD_FLAG_FAST_ZERO> is in use. + The C<flags> parameter may be C<0> for no flags, or may contain C<LIBNBD_CMD_FLAG_FUA> meaning that the server should not return until the data has been committed to permanent storage @@ -2088,6 +2125,12 @@ may extend beyond the requested range. If multiple contexts are supported, the number of blocks and cumulative length of those blocks need not be identical between contexts. +Note that not all servers can support a C<count> of 4GiB or larger. +The NBD protocol does not yet have a way for a client to learn if +the server will enforce an even smaller maximum block status size, +although a future extension may add a constraint visible in +L<nbd_get_block_size(3)>. + Depending on which metadata contexts were enabled before connecting (see L<nbd_add_meta_context(3)>) and which are supported by the server (see L<nbd_can_meta_context(3)>) this call -- 2.33.1
On 12/02/21 22:52, Eric Blake wrote:
Prior to implementing an NBD extension that will allow some commands to accept a 64-bit count, it is worth documenting places where there are 32-bit inherent limits, as well as where libnbd itself enforces a smaller arbitrary limit, in spite of the API having a size_t or uint64_t parameter.
--- generator/API.ml | 45 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 44 insertions(+), 1 deletion(-)
diff --git a/generator/API.ml b/generator/API.ml index 602f51e0..cf2e7543 100644 --- a/generator/API.ml +++ b/generator/API.ml @@ -1,6 +1,6 @@ (* hey emacs, this is OCaml code: -*- tuareg -*- *) (* nbd client library in userspace: the API - * Copyright (C) 2013-2020 Red Hat Inc. + * Copyright (C) 2013-2021 Red Hat Inc. * * This library is free software; you can redistribute it and/or * modify it under the terms of the GNU Lesser General Public @@ -1822,6 +1822,12 @@ error. See also L<nbd_pread_structured(3)>, if finer visibility is required into the server's replies, or if you want to use C<LIBNBD_CMD_FLAG_DF>.
+Note that libnbd currently enforces a maximum read buffer of 64MiB, +even if the server would permit a larger buffer in a single transaction; +attempts to exceed this will result in an C<ERANGE> error. The server +may enforce a smaller limit, which can be learned with +L<nbd_get_block_size(3)>. + The C<flags> parameter must be C<0> for now (it exists for future NBD protocol extensions)." ^ strict_call_description;
This hunk is for pread, OK.
@@ -1898,6 +1904,12 @@ the server obeyed the requirement that a read call must not have overlapping chunks and must not succeed without enough chunks to cover the entire request.
+Note that libnbd currently enforces a maximum read buffer of 64MiB, +even if the server would permit a larger buffer in a single transaction; +attempts to exceed this will result in an C<ERANGE> error. The server +may enforce a smaller limit, which can be learned with +L<nbd_get_block_size(3)>. + The C<flags> parameter may be C<0> for no flags, or may contain C<LIBNBD_CMD_FLAG_DF> meaning that the server should not reply with more than one fragment (if that is supported - some servers cannot do
This one for pread_structured.
@@ -1924,6 +1936,12 @@ using this call. The call returns when the command has been acknowledged by the server, or there is an error. Note this will generally return an error if L<nbd_is_read_only(3)> is true.
+Note that libnbd currently enforces a maximum write buffer of 64MiB, +even if the server would permit a larger buffer in a single transaction; +attempts to exceed this will result in an C<ERANGE> error. The server +may enforce a smaller limit, which can be learned with +L<nbd_get_block_size(3)>. + The C<flags> parameter may be C<0> for no flags, or may contain C<LIBNBD_CMD_FLAG_FUA> meaning that the server should not return until the data has been committed to permanent storage
This one for pwrite.
@@ -2007,6 +2025,12 @@ The call returns when the command has been acknowledged by the server, or there is an error. Note this will generally return an error if L<nbd_can_trim(3)> is false or L<nbd_is_read_only(3)> is true.
+Note that not all servers can support a C<count> of 4GiB or larger. +The NBD protocol does not yet have a way for a client to learn if +the server will enforce an even smaller maximum trim size, although +a future extension may add a constraint visible in +L<nbd_get_block_size(3)>. + The C<flags> parameter may be C<0> for no flags, or may contain C<LIBNBD_CMD_FLAG_FUA> meaning that the server should not return until the data has been committed to permanent storage
This one for trim.
@@ -2032,6 +2056,12 @@ L<nbd_pread(3)> call. The server can also silently ignore this command. Note this will generally return an error if L<nbd_can_cache(3)> is false.
+Note that not all servers can support a C<count> of 4GiB or larger. +The NBD protocol does not yet have a way for a client to learn if +the server will enforce an even smaller maximum cache size, although +a future extension may add a constraint visible in +L<nbd_get_block_size(3)>. + The C<flags> parameter must be C<0> for now (it exists for future NBD protocol extensions)." ^ strict_call_description;
This for "cache".
@@ -2055,6 +2085,13 @@ The call returns when the command has been acknowledged by the server, or there is an error. Note this will generally return an error if L<nbd_can_zero(3)> is false or L<nbd_is_read_only(3)> is true.
+Note that not all servers can support a C<count> of 4GiB or larger. +The NBD protocol does not yet have a way for a client to learn if +the server will enforce an even smaller maximum zero size, although +a future extension may add a constraint visible in +L<nbd_get_block_size(3)>. Also, some servers may permit a larger +zero request only when the C<LIBNBD_CMD_FLAG_FAST_ZERO> is in use. + The C<flags> parameter may be C<0> for no flags, or may contain C<LIBNBD_CMD_FLAG_FUA> meaning that the server should not return until the data has been committed to permanent storage
For "zero".
@@ -2088,6 +2125,12 @@ may extend beyond the requested range. If multiple contexts are supported, the number of blocks and cumulative length of those blocks need not be identical between contexts.
+Note that not all servers can support a C<count> of 4GiB or larger. +The NBD protocol does not yet have a way for a client to learn if +the server will enforce an even smaller maximum block status size, +although a future extension may add a constraint visible in +L<nbd_get_block_size(3)>. + Depending on which metadata contexts were enabled before connecting (see L<nbd_add_meta_context(3)>) and which are supported by the server (see L<nbd_can_meta_context(3)>) this call
For "block_status". Request for the future: - please let us document the affected functions (function descriptions) in the commit message, or else - please let us figure out a pattern for git-diff that lets us see the affected functions directly in the diff (in the @@ hunk headers). For example, we could put the following in the .gitattributes file, in the libnbd project root (or else in each local .git/info/attributes): API.ml diff=api and then git config diff.api.xfuncname '^ "[^"]*", \{$' With that in place, your patch is formatted (e.g. by git-show) with the following hunk headers: @@ -1822,6 +1822,12 @@ "pread", { @@ -1898,6 +1904,12 @@ "pread_structured", { @@ -1924,6 +1936,12 @@ "pwrite", { @@ -2007,6 +2025,12 @@ "trim", { @@ -2032,6 +2056,12 @@ "cache", { @@ -2055,6 +2085,13 @@ "zero", { @@ -2088,6 +2125,12 @@ "block_status", { Acked-by: Laszlo Ersek <lersek@redhat.com> Thanks, Laszlo
On Fri, Dec 03, 2021 at 01:50:36PM +0100, Laszlo Ersek wrote:
@@ -1822,6 +1822,12 @@ error. See also L<nbd_pread_structured(3)>, if finer visibility is required into the server's replies, or if you want to use C<LIBNBD_CMD_FLAG_DF>.
+Note that libnbd currently enforces a maximum read buffer of 64MiB, +even if the server would permit a larger buffer in a single transaction; +attempts to exceed this will result in an C<ERANGE> error. The server +may enforce a smaller limit, which can be learned with +L<nbd_get_block_size(3)>. + The C<flags> parameter must be C<0> for now (it exists for future NBD protocol extensions)." ^ strict_call_description;
This hunk is for pread, OK. ...
Request for the future:
- please let us document the affected functions (function descriptions) in the commit message, or else
Always a good idea, regardless of your next suggestion (since your next one is not an out-of-the-box solution).
- please let us figure out a pattern for git-diff that lets us see the affected functions directly in the diff (in the @@ hunk headers).
For example, we could put the following in the .gitattributes file, in the libnbd project root (or else in each local .git/info/attributes):
API.ml diff=api
and then
git config diff.api.xfuncname '^ "[^"]*", \{$'
Cool. Even better: '^(let .*=|"[^"]*", \{$)', which also gives context for top-level OCaml let statements. I'll propose that as another patch shortly. The problem is we have no where to automate this into a new developer's workflow (we don't have a bootstrap script they are required to run), so the best we can do is document and hope they read it. Then again, there are currently very few active libnbd developers, all of whom will read this message and make the right tweaks...
With that in place, your patch is formatted (e.g. by git-show) with the following hunk headers:
@@ -1822,6 +1822,12 @@ "pread", { @@ -1898,6 +1904,12 @@ "pread_structured", { @@ -1924,6 +1936,12 @@ "pwrite", { @@ -2007,6 +2025,12 @@ "trim", { @@ -2032,6 +2056,12 @@ "cache", { @@ -2055,6 +2085,13 @@ "zero", { @@ -2088,6 +2125,12 @@ "block_status", {
Acked-by: Laszlo Ersek <lersek@redhat.com>
Thanks. Now in as commit 430ed75eb0 -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
participants (2)
-
Eric Blake -
Laszlo Ersek