On Wed, Feb 21, 2024 at 12:19:47PM -0600, Eric Blake wrote:
I noticed during integration testing that nbd-server blindly reports
a
size of 0 for all NBD_OPT_INFO requests, unless I pass a size argument
on the command line to nbd-server. At first, I thought it was a side
effect of me trying to use nbd-server on a block device (an LVM
partition), as it is a common bug to rely on stat().st_size which only
works for regular files (a block device has to use lseek(SEEK_END));
but then I noticed it happening when using nbd-server to serve regular
files as well.
I then turned to the source code, where I see that client->exportsize
is set in just these places:
commit_client()
- exportsize = OFFT_MAX, then try setupexport()
setupexport()
- default to client->server->expected_size (if one was provided),
further validating that actual size is large enough when actual size
can be computed
- if neither treefile or F_WAIT is set, compute actual size by opening
one or more files and using size_autodetect() (which does the right
thing for block devices, so my earlier thought about over-reliance
on stat() was wrong)
but these functions are only reached for NBD_OPT_EXPORT_NAME and
NBD_OPT_GO, not NBD_OPT_INFO. The upshot is that for NBD_OPT_GO,
there are some scenarios (treefile, F_WAIT) where nbd-server
advertises a size of 9223372036854775807 (0x7fffffff_ffffffff) meaning
unknown, but a size of 0 there is only possible if the file was
successfully opened and really is zero bytes in length. Conversely,
NBD_OPT_INFO is always advertising a size of 0, which means most of
the time, the size changes between NBD_OPT_INFO and NBD_OPT_GO.
For comparison, both nbdkit and qemu as an NBD server always advertise
the same size for both NBD_OPT_INFO and NBD_OPT_GO (but it should also
be noted that neither of these has the complexity of multifile like
nbd-server).
Should we tweak the NBD standard to recommend that a server advertise
a size of OFFT_MAX for NBD_OPT_INFO if it is prohibitive to determine
an actual size, rather than 0?
No no no, this is a cop out! Just calculate the correct size.
If nbdkit & qemu-nbd can do it, then so can nbd-server.
Furthermore, is it worth adding code
to make NBD_OPT_INFO try harder to provide a sensible value when
possible (if expected_size was set, or if we are not multifile or
F_WAIT, then a stat() is enough to get the size for serving a regular
file; and if stat() says we have a block device, we can still try the
open/lseek/close)?
Also, is it worth trying to specify that since no known NBD servers
allow exports with sizes larger than OFFT_MAX, a server MUST NOT
report a size equal or larger than 9223372036854775808
(0x80000000_00000000)?
Yes, nbdkit supports up to and including 7fffffff_ffffffff and qemu up
to and including 7fffffff_c0000000 (although that's a bug in qemu IMHO).
The Linux kernel itself doesn't support anything larger than
7fffffff_ffffffff.
There are definitely a few places that I could
simplify in libnbd if we have a protocol guarantee that a valid export
size will never have the most significant bit set, and therefore we
don't have to worry about whether size is represented as a signed or
unsigned value (which also implies that libnbd's function
nbd_get_size() returning ssize_t instead of size_t is acceptable).
But if we do that, then maybe 0x80000000_00000000 or even
0xffffffff_ffffffff would serve as a better recommended sentinel than
0x7fffffff_ffffffff for representing an indeterminate size.
Yeah I don't know where 7fffffff_ffffffff meaning "unknown" comes
from. nbdkit supports real, known sizes as large as this, and there's
no such concept as "unknown size" in nbdkit.
Rich.
--
Richard Jones, Virtualization Group, Red Hat
http://people.redhat.com/~rjones
Read my programming and virtualization blog:
http://rwmj.wordpress.com
nbdkit - Flexible, fast NBD server with plugins
https://gitlab.com/nbdkit/nbdkit