what size should NBD_OPT_INFO report?
by Eric Blake
I noticed during integration testing that nbd-server blindly reports a
size of 0 for all NBD_OPT_INFO requests, unless I pass a size argument
on the command line to nbd-server. At first, I thought it was a side
effect of me trying to use nbd-server on a block device (an LVM
partition), as it is a common bug to rely on stat().st_size which only
works for regular files (a block device has to use lseek(SEEK_END));
but then I noticed it happening when using nbd-server to serve regular
files as well.
I then turned to the source code, where I see that client->exportsize
is set in just these places:
commit_client()
- exportsize = OFFT_MAX, then try setupexport()
setupexport()
- default to client->server->expected_size (if one was provided),
further validating that actual size is large enough when actual size
can be computed
- if neither treefile or F_WAIT is set, compute actual size by opening
one or more files and using size_autodetect() (which does the right
thing for block devices, so my earlier thought about over-reliance
on stat() was wrong)
but these functions are only reached for NBD_OPT_EXPORT_NAME and
NBD_OPT_GO, not NBD_OPT_INFO. The upshot is that for NBD_OPT_GO,
there are some scenarios (treefile, F_WAIT) where nbd-server
advertises a size of 9223372036854775807 (0x7fffffff_ffffffff) meaning
unknown, but a size of 0 there is only possible if the file was
successfully opened and really is zero bytes in length. Conversely,
NBD_OPT_INFO is always advertising a size of 0, which means most of
the time, the size changes between NBD_OPT_INFO and NBD_OPT_GO.
For comparison, both nbdkit and qemu as an NBD server always advertise
the same size for both NBD_OPT_INFO and NBD_OPT_GO (but it should also
be noted that neither of these has the complexity of multifile like
nbd-server).
Should we tweak the NBD standard to recommend that a server advertise
a size of OFFT_MAX for NBD_OPT_INFO if it is prohibitive to determine
an actual size, rather than 0? Furthermore, is it worth adding code
to make NBD_OPT_INFO try harder to provide a sensible value when
possible (if expected_size was set, or if we are not multifile or
F_WAIT, then a stat() is enough to get the size for serving a regular
file; and if stat() says we have a block device, we can still try the
open/lseek/close)?
Also, is it worth trying to specify that since no known NBD servers
allow exports with sizes larger than OFFT_MAX, a server MUST NOT
report a size equal or larger than 9223372036854775808
(0x80000000_00000000)? There are definitely a few places that I could
simplify in libnbd if we have a protocol guarantee that a valid export
size will never have the most significant bit set, and therefore we
don't have to worry about whether size is represented as a signed or
unsigned value (which also implies that libnbd's function
nbd_get_size() returning ssize_t instead of size_t is acceptable).
But if we do that, then maybe 0x80000000_00000000 or even
0xffffffff_ffffffff would serve as a better recommended sentinel than
0x7fffffff_ffffffff for representing an indeterminate size.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc.
Virtualization: qemu.org | libguestfs.org
3 months, 3 weeks
[libguestfs BUG] ISO images can't be inspected properly
by Andrey Drobyshev
I discovered that when a VM has an ISO attached via CD-ROM, libguestfs
fails to read its partition table and therefore fails to perform basic
operations on a VM, e.g. listing a directory or editing a file:
# virsh domblklist rocky9-vm
Target Source
---------------------------------------------------------------------------
sda /vz/vmprivate/e827caad-b9e4-4858-b3dc-53ae6b8a5145/harddisk.hdd
sdb /vzt/images/Rocky-9.3-x86_64-minimal.iso
# virt-ls -d rocky9-vm /
libguestfs: error: inspect_os: sgdisk: Invalid partition data!
# virt-edit -d rocky9-vm /etc/fstab
libguestfs: error: inspect_os: sgdisk: Invalid partition data!
Even virt-inspector can't give us proper output on an ISO9660 image:
# virt-inspector /vzt/images/Rocky-9.3-x86_64-minimal.iso
libguestfs: error: inspect_os: sgdisk: Invalid partition data!
virt-inspector: no operating system could be detected inside this disk
image.
...
First I found a 3 years old topic where you suggested that sgdisk itself
might be too old:
https://listman.redhat.com/archives/libguestfs/2021-March/025939.html.
However this doesn't seem to be it in this case:
I've compiled the latest version from
https://sourceforge.net/projects/gptfdisk -- no luck as well. Seems
that sgdisk isn't capable of recognizing ISO9660 images.
Then I noticed 2 things:
1. libguestfs appliance is being created (with backend=libvirt) with ISO
attached as device='disk' (not 'cdrom'):
> <disk type='file' device='disk'>
> <driver name='qemu' type='qcow2' cache='unsafe'/>
> <source file='/tmp/libguestfsD7IHvF/overlay2.qcow2' index='2'/>
> <backingStore type='file' index='5'>
> <format type='raw'/>
> <source file='/vzt/images/Rocky-9.3-x86_64-minimal.iso'/>
> <backingStore/>
> </backingStore>
> <target dev='sdb' bus='scsi'/>
> <alias name='scsi0-0-1-0'/>
> <address type='drive' controller='0' bus='0' target='1' unit='0'/>
> </disk>
That forces this disk to be supplied with "scsi-hd" driver in use, which
in turn leads to the device being visible inside the guest as a regular
SCSI hdd /dev/sdb. Shouldn't we, for instance, make sure that for such
images device='cdrom' and "scsi-cd" driver are being used? In this case
the guest will see it as /dev/sr0, and I suspect srX devices are being
skipped by guestfs_inspect_os.
2. When trying to perform the same operation via guestfish, everything
appears to be working:
# guestfish -a /vzt/images/Rocky-9.3-x86_64-minimal.iso -m /dev/sda ls /
>/dev/null && echo $?
0
File edit works as well. From debug output I see that sgdisk isn't
being invoked at all. That means there's a code path for the same
operations where sgdisk can be omitted. Maybe we should follow that
same code path when invoking other tools like virt-ls or virt-edit?
Overall, what would be an appropriate fix in your opinion? We should at
least make sure that working with domains (with the "-d" option) works
no matter what.
Andrey
5 months, 4 weeks
Patch for LoongArch
by liuxiang＠loongson.cn
diff -ruN libguestfs-1.52.0-bk/generator/actions_core.ml libguestfs-1.52.0/generator/actions_core.ml
--- libguestfs-1.52.0-bk/generator/actions_core.ml 2023-11-16 10:48:23.470064099 +0000
+++ libguestfs-1.52.0/generator/actions_core.ml 2024-02-18 07:41:58.633573217 +0000
@@ -9377,6 +9377,8 @@
InitISOFS, Always, TestResultString (
[["file_architecture"; "/bin-riscv64-dynamic"]], "riscv64"), [];
InitISOFS, Always, TestResultString (
+ [["file_architecture"; "/bin-loongarch64-dynamic"]], "loongarch64"), [];
+ InitISOFS, Always, TestResultString (
[["file_architecture"; "/bin-s390x-dynamic"]], "s390x"), [];
InitISOFS, Always, TestResultString (
[["file_architecture"; "/bin-sparc-dynamic"]], "sparc"), [];
@@ -9399,6 +9401,8 @@
InitISOFS, Always, TestResultString (
[["file_architecture"; "/lib-riscv64.so"]], "riscv64"), [];
InitISOFS, Always, TestResultString (
+ [["file_architecture"; "/lib-loongarch64.so"]], "loongarch64"), [];
+ InitISOFS, Always, TestResultString (
[["file_architecture"; "/lib-s390x.so"]], "s390x"), [];
InitISOFS, Always, TestResultString (
[["file_architecture"; "/lib-sparc.so"]], "sparc"), [];
@@ -9457,6 +9461,10 @@
64 bit Power PC (little endian).
+=item \"loongarch64\"
+
+64 bit LoongArch64 (little endian).
+
=item \"riscv32\"
=item \"riscv64\"
diff -ruN libguestfs-1.52.0-bk/lib/appliance-cpu.c libguestfs-1.52.0/lib/appliance-cpu.c
--- libguestfs-1.52.0-bk/lib/appliance-cpu.c 2023-11-16 10:48:23.479064013 +0000
+++ libguestfs-1.52.0/lib/appliance-cpu.c 2024-02-18 07:45:40.668965101 +0000
@@ -87,6 +87,9 @@
#elif defined(__riscv)
/* qemu-system-riscv64 (7.0) doesn't yet support -cpu max */
return NULL;
+#elif defined(__loongarch64)
+ /* qemu-system-loongarch64 doesn't yet support -cpu max */
+ return NULL;
#else
/* On most architectures we can use "max" to get the best possible CPU.
* For recent qemu this should work even on TCG.
二进制文件 libguestfs-1.52.0-bk/test-data/binaries/bin-loongarch64-dynamic 和 libguestfs-1.52.0/test-data/binaries/bin-loongarch64-dynamic 不同
二进制文件 libguestfs-1.52.0-bk/test-data/binaries/lib-loongarch64.so 和 libguestfs-1.52.0/test-data/binaries/lib-loongarch64.so 不同
diff -ruN libguestfs-1.52.0-bk/test-data/binaries/Makefile.am libguestfs-1.52.0/test-data/binaries/Makefile.am
--- libguestfs-1.52.0-bk/test-data/binaries/Makefile.am 2023-11-16 10:48:23.680062088 +0000
+++ libguestfs-1.52.0/test-data/binaries/Makefile.am 2024-02-18 07:52:12.736828137 +0000
@@ -27,6 +27,7 @@
bin-ppc64-dynamic \
bin-ppc64le-dynamic \
bin-riscv64-dynamic \
+ bin-loongarch64-dynamic \
bin-s390x-dynamic \
bin-sparc-dynamic \
bin-win32.exe \
@@ -40,6 +41,7 @@
lib-ppc64le.so \
lib-mipsel.so \
lib-riscv64.so \
+ lib-loongarch64.so \
lib-s390x.so \
lib-sparc.so \
lib-win32.dll \
diff -ruN libguestfs-1.52.0-bk/test-data/Makefile.am libguestfs-1.52.0/test-data/Makefile.am
--- libguestfs-1.52.0-bk/test-data/Makefile.am 2023-11-16 10:48:23.680062088 +0000
+++ libguestfs-1.52.0/test-data/Makefile.am 2024-02-18 07:51:07.614179690 +0000
@@ -38,6 +38,7 @@
$(srcdir)/binaries/bin-ppc64-dynamic \
$(srcdir)/binaries/bin-ppc64le-dynamic \
$(srcdir)/binaries/bin-riscv64-dynamic \
+ $(srcdir)/binaries/bin-loongarch64-dynamic \
$(srcdir)/binaries/bin-s390x-dynamic \
$(srcdir)/binaries/bin-sparc-dynamic \
$(srcdir)/binaries/bin-win32.exe \
@@ -51,6 +52,7 @@
$(srcdir)/binaries/lib-ppc64.so \
$(srcdir)/binaries/lib-ppc64le.so \
$(srcdir)/binaries/lib-riscv64.so \
+ $(srcdir)/binaries/lib-loongarch64.so \
$(srcdir)/binaries/lib-s390x.so \
$(srcdir)/binaries/lib-sparc.so \
$(srcdir)/binaries/lib-win32.dll \
6 months, 1 week
[nbdkit PATCH] main: Add --no-structured-replies alias for --no-sr
by Eric Blake
While abbreviations are easier to type, spelling out an option makes a
complex command line easier to understand.
For symmetry, the recent addition of --no-meta-contexts now documents
that it has an alias --no-mc.
Signed-off-by: Eric Blake <eblake(a)redhat.com>
---
docs/nbdkit-protocol.pod | 8 ++++++--
docs/nbdkit.pod | 2 ++
docs/synopsis.txt | 3 ++-
server/options.h | 1 +
tests/test-synopsis.sh | 1 +
5 files changed, 12 insertions(+), 3 deletions(-)
diff --git a/docs/nbdkit-protocol.pod b/docs/nbdkit-protocol.pod
index 620ea16e..570ef23b 100644
--- a/docs/nbdkit-protocol.pod
+++ b/docs/nbdkit-protocol.pod
@@ -4,8 +4,8 @@ nbdkit-protocol - which parts of the NBD protocol nbdkit supports
=head1 SYNOPSIS
- nbdkit [-n|--newstyle] [--mask-handshake MASK] [--no-meta-contexts]
- [--no-sr] [-o|--oldstyle] [...]
+ nbdkit [-n|--newstyle] [--mask-handshake MASK] [--no-mc|--no-meta-contexts]
+ [--no-sr|--no-structured-replies] [-o|--oldstyle] [...]
=head1 DESCRIPTION
@@ -38,6 +38,8 @@ parts of the NBD protocol, with successively larger impacts:
=over 4
+=item B<--no-mc>
+
=item B<--no-meta-contexts>
Force the newstyle protocol to treat all requests for meta context
@@ -51,6 +53,8 @@ emulating nbd-server 3.25.
=item B<--no-sr>
+=item B<--no-structured-replies>
+
Force the newstyle protocol to decline any client request for
structured replies; this is stronger than I<--no-meta-contexts> in
that it also disables the opportunity for sparse reads. This is
diff --git a/docs/nbdkit.pod b/docs/nbdkit.pod
index 4391fc7b..8e82b337 100644
--- a/docs/nbdkit.pod
+++ b/docs/nbdkit.pod
@@ -334,6 +334,8 @@ L<nbdkit-protocol(1)>.
=item B<--no-sr>
+=item B<--no-structured-replies>
+
Do not advertise structured replies. A client must request structured
replies to take advantage of block status and potential sparse reads;
however, as structured reads are not a mandatory part of the newstyle
diff --git a/docs/synopsis.txt b/docs/synopsis.txt
index 4fbca826..7fb107e8 100644
--- a/docs/synopsis.txt
+++ b/docs/synopsis.txt
@@ -4,7 +4,8 @@ nbdkit [-4|--ipv4-only] [-6|--ipv6-only]
[--filter=FILTER ...] [-f|--foreground]
[-g|--group GROUP] [-i|--ipaddr IPADDR]
[--log=stderr|syslog|null] [--mask-handshake=MASK]
- [-n|--newstyle] [--no-meta-contexts] [--no-sr] [-o|--oldstyle]
+ [-n|--newstyle] [--no-mc|--no-meta-contexts]
+ [--no-sr|--no-structured-replies] [-o|--oldstyle]
[-P|--pidfile PIDFILE] [-p|--port PORT]
[-r|--readonly] [--run 'COMMAND ARGS ...']
[--selinux-label=LABEL] [-s|--single] [--swap]
diff --git a/server/options.h b/server/options.h
index 544f8335..7d0730ba 100644
--- a/server/options.h
+++ b/server/options.h
@@ -86,6 +86,7 @@ static const struct option long_options[] = {
{ "no-mc", no_argument, NULL, NO_MC_OPTION },
{ "no-meta-contexts", no_argument, NULL, NO_MC_OPTION },
{ "no-sr", no_argument, NULL, NO_SR_OPTION },
+ { "no-structured-replies", no_argument, NULL, NO_SR_OPTION },
{ "old-style", no_argument, NULL, 'o' },
{ "oldstyle", no_argument, NULL, 'o' },
{ "pid-file", required_argument, NULL, 'P' },
diff --git a/tests/test-synopsis.sh b/tests/test-synopsis.sh
index 59886847..f8d5bcc4 100755
--- a/tests/test-synopsis.sh
+++ b/tests/test-synopsis.sh
@@ -57,6 +57,7 @@ for i in $(nbdkit --long-options | $nocr); do
--ip-addr) ;; # alias of -i, --ipaddr
--new-style) ;; # alias of -n, --newstyle
--no-mc) ;; # alias of --no-meta-contexts
+ --no-sr) ;; # alias of --no-structured-replies
--old-style) ;; # alias of -o, --oldstyle
--pid-file) ;; # alias of -P, --pidfile
--read-only) ;; # alias of -r, --readonly
--
2.43.2
6 months, 1 week
Re: [EXTERNAL] - Re: SLES 15 disk mount is failing
by Charles Arnold
On 2/27/24 4:05 AM, Teja Konapalli wrote:
> Thanks Richard,
>
> @carnold(a)suse.com Could you please help with the release download url of SLES15 SP6.
>
> Regards,
> Teja K
Teja,
openSUSE Leap 15.6 may be downloaded from this page.
https://get.opensuse.org/leap/15.6/#download
For SLES 15-SP6 you will need to be part of the Beta program to download
it. See,
https://www.suse.com/betaprogram/sle-beta/
Both of these products are still in the beta phase of development. The
official release
will be later this year. openSUSE Leap is derived from SLES so it is a
good option to
test with. Downloading and using it doesn't require any special
registration.
- Charles
>
> -----Original Message-----
> From: Richard W.M. Jones <rjones(a)redhat.com>
> Sent: Monday, February 26, 2024 8:55 PM
> To: Teja Konapalli <tkonapalli(a)opentext.com>; carnold(a)suse.com
> Cc: Eric Blake <eblake(a)redhat.com>; Surendra CH <sch2(a)opentext.com>; guestfs(a)lists.libguestfs.org; Raja Ram Sharma <rsharma6(a)opentext.com>
> Subject: Re: [EXTERNAL] - Re: SLES 15 disk mount is failing
>
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. If you feel that the email is suspicious, please report it using PhishAlarm.
>
>
> For context the error was:
>
>> guestfsd: error while loading shared libraries: libjansson.so.4:
>> cannot open
> shared object file: No such file or directory
>
> On Mon, Feb 26, 2024 at 03:12:15PM +0000, Teja Konapalli wrote:
>> rpm -q libguestfs
>> libguestfs-1.48.4-150500.1.13.x86_64
> This is quite an old version of libguestfs in OpenSUSE (from 2022-10-20).
> Can you try a newer version?
>
> I looked at the build log for libguestfs 1.52 in OpenSUSE 15.6 and the dependencies are being calculated correctly now:
>
> https://urldefense.com/v3/__https://build.opensuse.org/package/show/Virtu... -> https://urldefense.com/v3/__https://build.opensuse.org/public/build/Virtu...
>
> [ 284s] guestfsd.deps:
> [ 284s] glibc
> [ 284s] libacl1
> [ 284s] libaugeas0
> [ 284s] libcap2
> [ 284s] libhivex0
> [ 284s] libjansson4
> [ 284s] libpcre2-8-0
> [ 284s] libselinux1
> [ 284s] libtirpc3
> [ 284s] libzstd1
> [ 284s] rpm
>
> This almost exactly matches the dependencies calculated on Fedora, and as you can see it includes libjansson4.
>
> Rich.
>
> --
> Richard Jones, Virtualization Group, Red Hat https://urldefense.com/v3/__http://people.redhat.com/*rjones__;fg!!Obbck6...
> Read my programming and virtualization blog: https://urldefense.com/v3/__http://rwmj.wordpress.com__;!!Obbck6kTJA!fuyN...
> virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc.
> https://urldefense.com/v3/__http://people.redhat.com/*rjones/virt-top__;f...
>
6 months, 1 week
Re: [EXTERNAL] - Re: SLES 15 disk mount is failing
by Richard W.M. Jones
For context the error was:
> guestfsd: error while loading shared libraries: libjansson.so.4: cannot open
shared object file: No such file or directory
On Mon, Feb 26, 2024 at 03:12:15PM +0000, Teja Konapalli wrote:
> rpm -q libguestfs
> libguestfs-1.48.4-150500.1.13.x86_64
This is quite an old version of libguestfs in OpenSUSE (from 2022-10-20).
Can you try a newer version?
I looked at the build log for libguestfs 1.52 in OpenSUSE 15.6 and the
dependencies are being calculated correctly now:
https://build.opensuse.org/package/show/Virtualization/libguestfs ->
https://build.opensuse.org/public/build/Virtualization/15.6/x86_64/libgue...
[ 284s] guestfsd.deps:
[ 284s] glibc
[ 284s] libacl1
[ 284s] libaugeas0
[ 284s] libcap2
[ 284s] libhivex0
[ 284s] libjansson4
[ 284s] libpcre2-8-0
[ 284s] libselinux1
[ 284s] libtirpc3
[ 284s] libzstd1
[ 284s] rpm
This almost exactly matches the dependencies calculated on Fedora, and
as you can see it includes libjansson4.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
6 months, 1 week
Re: [EXTERNAL] - Re: SLES 15 disk mount is failing
by Richard W.M. Jones
On Mon, Feb 26, 2024 at 12:18:44PM +0000, Surendra CH wrote:
> Hi,
>
> PFB.
>
> PATH=/sbin:/usr/sbin:/usr/local/sbin:/root/bin:/usr/local/bin:/usr/bin:/bin:/opt/cmcluster/bin
> XDG_RUNTIME_DIR=/run/user/0
> SELinux: guestfs_get_append: (null)
> guestfs_get_autosync: 1
> guestfs_get_backend: direct
> guestfs_get_backend_settings: []
> guestfs_get_cachedir: /var/tmp
> guestfs_get_hv: /usr/bin/qemu-system-x86_64
> guestfs_get_memsize: 1280
> guestfs_get_network: 0
> guestfs_get_path: /usr/lib64/guestfs
> guestfs_get_pgroup: 0
> guestfs_get_program: libguestfs-test-tool
> guestfs_get_recovery_proc: 1
> guestfs_get_smp: 1
> guestfs_get_sockdir: /tmp
> guestfs_get_tmpdir: /tmp
> guestfs_get_trace: 0
> guestfs_get_verbose: 1
> host_cpu: x86_64
> Launching appliance, timeout set to 600 seconds.
That's really the whole output?
Rich.
> Regards,
> Surendra
>
> -----Original Message-----
> From: Richard W.M. Jones <rjones(a)redhat.com>
> Sent: Monday, February 26, 2024 5:14 PM
> To: Teja Konapalli <tkonapalli(a)opentext.com>
> Cc: Eric Blake <eblake(a)redhat.com>; Surendra CH <sch2(a)opentext.com>; guestfs(a)lists.libguestfs.org; Raja Ram Sharma <rsharma6(a)opentext.com>
> Subject: [EXTERNAL] - Re: SLES 15 disk mount is failing
>
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. If you feel that the email is suspicious, please report it using PhishAlarm.
>
>
> On Mon, Feb 26, 2024 at 11:41:47AM +0000, Teja Konapalli wrote:
> > [ 2024-02-27 03:52:37 ] errno: ( 0 ) : cannot find any suitable
> > libguestfs supermin, fixed or old-style appliance on LIBGUESTFS_PATH
> > (search path: /usr/
> > lib64/guestfs)
>
> It's a SUSE bug, I suppose. What is the complete output of libguestfs-test-tool?
>
> Rich.
>
> --
> Richard Jones, Virtualization Group, Red Hat https://urldefense.com/v3/__http://people.redhat.com/*rjones__;fg!!Obbck6...
> Read my programming and virtualization blog: https://urldefense.com/v3/__http://rwmj.wordpress.com__;!!Obbck6kTJA!Zmwb...
> nbdkit - Flexible, fast NBD server with plugins https://urldefense.com/v3/__https://gitlab.com/nbdkit/nbdkit__;!!Obbck6kT...
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
6 months, 2 weeks
