September 2022 - Libguestfs - libguestfs List Archives

[libnbd PATCH 0/4] API: stats tracking
by Eric Blake 09 Sep '22

09 Sep '22

With this, I'll be able to clean up test 3/12 in the pending opt_set_meta_context() series to test whether filtering was done client- or server-side. Eric Blake (4): internal: Track traffic stats generator: Add RUInt64 for 64-bit counters without error api: Add new nbd_stats_*() statistics counters tests: Add language binding tests for stats reception docs/libnbd.pod | 20 ++ lib/internal.h | 6 + generator/API.ml | 83 +++++++- generator/API.mli | 3 +- generator/C.ml | 13 +- generator/GoLang.ml | 9 +- generator/OCaml.ml | 5 +- generator/Python.ml | 2 +- generator/states-issue-command.c | 3 +- generator/states-magic.c | 7 +- generator/states-newstyle-opt-export-name.c | 3 +- generator/states-newstyle-opt-go.c | 1 + generator/states-newstyle-opt-list.c | 3 +- generator/states-newstyle-opt-meta-context.c | 1 + generator/states-newstyle-opt-starttls.c | 3 +- .../states-newstyle-opt-structured-reply.c | 1 + generator/states-newstyle.c | 1 + generator/states-reply.c | 6 +- generator/states.c | 2 + lib/handle.c | 28 +++ python/t/620-stats.py | 77 ++++++++ ocaml/tests/Makefile.am | 1 + ocaml/tests/test_620_stats.ml | 77 ++++++++ examples/strict-structured-reads.c | 13 +- golang/Makefile.am | 3 +- golang/libnbd_620_stats.go | 181 ++++++++++++++++++ 26 files changed, 527 insertions(+), 25 deletions(-) create mode 100644 python/t/620-stats.py create mode 100644 ocaml/tests/test_620_stats.ml create mode 100644 golang/libnbd_620_stats.go -- 2.37.2

3 22

[libnbd PATCH v2 00/12] Improve NBD_OPT_ control
by Eric Blake 09 Sep '22

09 Sep '22

v1 was here (under the subject Smarter nbd_opt_info) https://listman.redhat.com/archives/libguestfs/2022-August/029641.html Since then, I've done a lot. The original patch 1/2 is now split across 7/12 and 9/12; while original patch 2/2 is expanded across several patches to test more scenarios. Several new prerequisite patches were added to address smaller issues one at a time and with better commit messages. Where possible, I have kept tests separate from the patch introducing a fix to allow temporary reordering at that point in the series to prove the test catches the issue. It was not possible on tests for new APIs, nor on patch 2/12 where the only test I could come up with involves hacking a one-off nbdkit that does not behave like a normal server; it was also hard to justify on 3/12 where reordering the patches results in a test skip rather than failure, where distinguishing the difference requires reading logs to see whether an API failed client-side or server-side. Maybe we want to add an API to track how many packets are sent over the wire (client->server, and server->client), to make it easier to actually catch when things are filtered client-side, or how often a server uses multiple reply chunks to answer a single client request. I've also fixed a bug in nbd_can_meta_context that was discovered while responding to v1 reviews, and completed part of the work at adding new APIs hinted at earlier (but still to add: APIs for nbd_opt_structured_replies and nbd_opt_starttls). Eric Blake (12): internal: Use vector instead of linked list for meta_contexts api: Fix nbd_can_meta_context if NBD_OPT_SET_META_CONTEXT fails api: Allow nbd_opt_list_meta_context without SR api: Add nbd_set_request_meta_context() tests: Language port of nbd_set_request_meta_context() tests info: Explicitly skip NBD_OPT_SET_META_CONTEXT in --list mode api: Make nbd_opt_list_meta_context stateless tests: Add coverage for stateless nbd_opt_list_meta_context api: Reset state on changed nbd_set_export_name() tests: Add coverage for nbd_set_export_name fix api: Add nbd_[aio_]opt_set_meta_context tests: Language port of nbd_opt_set_meta_context() tests lib/internal.h | 19 +- generator/API.ml | 174 ++++++++++++- generator/states-newstyle-opt-go.c | 1 + generator/states-newstyle-opt-meta-context.c | 77 +++--- generator/states-newstyle.c | 1 + generator/states-reply-structured.c | 11 +- lib/flags.c | 27 +- lib/handle.c | 26 ++ lib/opt.c | 53 +++- lib/rw.c | 2 +- python/t/110-defaults.py | 1 + python/t/120-set-non-defaults.py | 2 + python/t/230-opt-info.py | 36 ++- python/t/240-opt-list-meta.py | 29 ++- python/t/250-opt-set-meta.py | 126 ++++++++++ ocaml/tests/Makefile.am | 5 +- ocaml/tests/test_110_defaults.ml | 2 + ocaml/tests/test_120_set_non_defaults.ml | 3 + ocaml/tests/test_230_opt_info.ml | 43 +++- ocaml/tests/test_240_opt_list_meta.ml | 34 ++- ocaml/tests/test_250_opt_set_meta.ml | 150 +++++++++++ tests/Makefile.am | 9 + tests/opt-info.c | 91 ++++++- tests/opt-list-meta.c | 104 +++++++- tests/opt-set-meta | 210 ++++++++++++++++ tests/opt-set-meta.c | 236 ++++++++++++++++++ .gitignore | 1 + golang/Makefile.am | 3 +- golang/libnbd_110_defaults_test.go | 8 + golang/libnbd_120_set_non_defaults_test.go | 12 + golang/libnbd_230_opt_info_test.go | 111 ++++++++- golang/libnbd_240_opt_list_meta_test.go | 106 ++++++-- golang/libnbd_250_opt_set_meta_test.go | 248 +++++++++++++++++++ info/list.c | 3 +- info/show.c | 3 +- 35 files changed, 1813 insertions(+), 154 deletions(-) create mode 100644 python/t/250-opt-set-meta.py create mode 100644 ocaml/tests/test_250_opt_set_meta.ml create mode 100755 tests/opt-set-meta create mode 100644 tests/opt-set-meta.c create mode 100644 golang/libnbd_250_opt_set_meta_test.go -- 2.37.2

3 47

libnbd | Failed pipeline for master | 92e7a396
by GitLab 08 Sep '22

08 Sep '22

Pipeline #635106659 has failed! Project: libnbd ( https://gitlab.com/nbdkit/libnbd ) Branch: master ( https://gitlab.com/nbdkit/libnbd/-/commits/master ) Commit: 92e7a396 ( https://gitlab.com/nbdkit/libnbd/-/commit/92e7a39679e7742067a0140caaac9bd25… ) Commit Message: generator: Simplify generated trace of return v... Commit Author: Eric Blake ( https://gitlab.com/ebblake ) Pipeline #635106659 ( https://gitlab.com/nbdkit/libnbd/-/pipelines/635106659 ) triggered by Eric Blake ( https://gitlab.com/ebblake ) had 2 failed jobs. Job #3000936302 ( https://gitlab.com/nbdkit/libnbd/-/jobs/3000936302/raw ) Stage: builds Name: x86_64-opensuse-leap-153 Job #3000936323 ( https://gitlab.com/nbdkit/libnbd/-/jobs/3000936323/raw ) Stage: builds Name: x86_64-freebsd-12 -- You're receiving this email because of your account on gitlab.com.

1 0

[p2v PATCH 0/6] restrict vCPU topology to (a) fully populated physical, or (b) 1 * N * 1
by Laszlo Ersek 08 Sep '22

08 Sep '22

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1590721 The first patch is a small improvement ("bugfix") for an obscure corner case that is triggerable on the current code base (@ 0687cea6a86e). It is not overly important in practice in itself, but it's a pre-requisite for patch#4. The other patches restrict the CPU topology options to { fully populated physical, 1 * N * 1 } -- currently it's too easy to create an inconsistent topology. These options are available through the GUI and the kernel command line; "fully populated physical" is made the default as well. Laszlo Laszlo Ersek (6): gui: check VCPU & memory limits upon showing the conversion dialog restrict vCPU topology to (a) fully populated physical, or (b) 1 * N * 1 gui: set row count from a running variable when populating tables gui: offer copying the vCPU topology from the fully populated physical one copy fully populated vCPU topology by default Makefile.am: set vCPU topology to 1*2*2 in the "p2v in a VM" tests Makefile.am | 2 + cpuid.c | 39 ++++--- generate-p2v-config.pl | 45 ++++--- gui-gtk3-compat.h | 8 +- gui.c | 123 ++++++++++++++------ main.c | 8 +- p2v.h | 6 + physical-xml.c | 54 ++++----- test-virt-p2v-cmdline.sh | 5 +- 9 files changed, 182 insertions(+), 108 deletions(-)

3 20

libnbd | Failed pipeline for master | f79a1ac1
by GitLab 06 Sep '22

06 Sep '22

Pipeline #632124104 has failed! Project: libnbd ( https://gitlab.com/nbdkit/libnbd ) Branch: master ( https://gitlab.com/nbdkit/libnbd/-/commits/master ) Commit: f79a1ac1 ( https://gitlab.com/nbdkit/libnbd/-/commit/f79a1ac1e48e16d394a9e5b2b67e25526… ) Commit Message: python: Fix stats return type In the python bi... Commit Author: Eric Blake ( https://gitlab.com/ebblake ) Pipeline #632124104 ( https://gitlab.com/nbdkit/libnbd/-/pipelines/632124104 ) triggered by Eric Blake ( https://gitlab.com/ebblake ) had 2 failed jobs. Job #2984290350 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2984290350/raw ) Stage: builds Name: x86_64-opensuse-leap-153 Job #2984290366 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2984290366/raw ) Stage: builds Name: x86_64-freebsd-12 -- You're receiving this email because of your account on gitlab.com.

1 0

[libnbd PATCH 0/2] Trace even may_set_error=false functions
by Eric Blake 05 Sep '22

05 Sep '22

In my series to add nbd_stats_bytes_sent(), I was trying to see what values were being printed, only to learn we weren't printing any. This series fixes that. Eric Blake (2): debug: Change trace of RStaticString returns debug: Trace even may_set_error=false functions lib/internal.h | 16 +++++++---- generator/C.ml | 73 +++++++++++++++++++++++++------------------------- lib/debug.c | 9 ++++--- 3 files changed, 53 insertions(+), 45 deletions(-) -- 2.37.2

2 5

[libnbd PATCH] copy: Write progress to stderr, not /dev/tty
by Eric Blake 05 Sep '22

05 Sep '22

dd writes progress messages to stderr, with no complaints. Having nbdcopy open /dev/tty instead of reusing stderr to write messages litters the output of a terminal during an interactive 'make check', which is rather distracting, even though that test also ran a second copy using a different fd for inspecting the output. Fixes: 1df1b05a ("New nbdcopy tool", v1.3.8) --- copy/copy-progress-bar.sh | 4 ++-- copy/progress.c | 12 ++---------- 2 files changed, 4 insertions(+), 12 deletions(-) diff --git a/copy/copy-progress-bar.sh b/copy/copy-progress-bar.sh index 00380d17..4a0b9a3b 100755 --- a/copy/copy-progress-bar.sh +++ b/copy/copy-progress-bar.sh @@ -1,6 +1,6 @@ #!/usr/bin/env bash # nbd client library in userspace -# Copyright (C) 2020 Red Hat Inc. +# Copyright (C) 2020-2022 Red Hat Inc. # # This library is free software; you can redistribute it and/or # modify it under the terms of the GNU Lesser General Public @@ -32,7 +32,7 @@ cleanup_fn rm -f $file $file2 $file3 dd if=/dev/urandom of=$file bs=512 count=1 # Check that a regular progress bar works. -# This writes to /dev/tty :-) +# This writes to stderr $VG nbdcopy --progress $file $file2 # Check that a machine-readable progress bar works. diff --git a/copy/progress.c b/copy/progress.c index a8b1a470..c1a873c9 100644 --- a/copy/progress.c +++ b/copy/progress.c @@ -1,5 +1,5 @@ /* NBD client library in userspace. - * Copyright (C) 2020 Red Hat Inc. + * Copyright (C) 2020-2022 Red Hat Inc. * * This library is free software; you can redistribute it and/or * modify it under the terms of the GNU Lesser General Public @@ -37,8 +37,6 @@ static void do_progress_bar (off_t pos, int64_t size) { - static int tty = -1; - /* Note the spinner is covered with the cursor which usually makes * it appear inverse video. */ @@ -50,12 +48,6 @@ do_progress_bar (off_t pos, int64_t size) char msg[80]; size_t n, i; - if (tty == -1) { - tty = open ("/dev/tty", O_WRONLY); - if (tty == -1) - return; - } - if (frac < 0) frac = 0; else if (frac > 1) frac = 1; if (frac == 1) { @@ -75,7 +67,7 @@ do_progress_bar (off_t pos, int64_t size) #pragma GCC diagnostic push #pragma GCC diagnostic ignored "-Wunused-result" - write (tty, msg, strlen (msg)); + write (fileno (stderr), msg, strlen (msg)); #pragma GCC diagnostic pop } -- 2.37.2

3 4

IRC question about error while loading shared libraries: libgcrypt.so.20
by Richard W.M. Jones 04 Sep '22

04 Sep '22

Ran libguestfs-test-tool and got lots of output, the pertinent line being: 15:55 < Scamdotnet> guestfsd: error while loading shared libraries: libgcrypt.so.20: cannot open shared object file: No such file or directory (https://pastebin.com/9PgqEPuq) 16:17 < rwmjones> Scamdotnet: ok I see what's happening, libgcrypt has changed SONAME since libguestfs was built 16:17 < rwmjones> the only long term solution is to rebuild the package 16:17 < rwmjones> in the short term you could try this: 16:17 < rwmjones> echo '/usr/lib64/libgcrypt.so*' > /usr/lib64/supermin.d/guestfs/zz-libgcrypt 16:18 < rwmjones> (maybe replace /usr/lib64 with /usr/lib depending on distro) 16:18 < rwmjones> this is suse? 16:18 < rwmjones> anyway get the distro to rebuild the libguestfs package to fix the long term problem 16:19 < rwmjones> https://libguestfs.org/supermin.1.html#SUPERMIN-APPLIANCES explains more Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com nbdkit - Flexible, fast NBD server with plugins https://gitlab.com/nbdkit/nbdkit

1 0

libnbd | Failed pipeline for master | f18ad368
by GitLab 03 Sep '22

03 Sep '22

Pipeline #630565927 has failed! Project: libnbd ( https://gitlab.com/nbdkit/libnbd ) Branch: master ( https://gitlab.com/nbdkit/libnbd/-/commits/master ) Commit: f18ad368 ( https://gitlab.com/nbdkit/libnbd/-/commit/f18ad3680bfcb90ad30d909bc4b5b3c2d… ) Commit Message: golang: Rename test to fix EXTRA_DIST The exis... Commit Author: Richard W_M_ Jones ( https://gitlab.com/rwmjones ) Pipeline #630565927 ( https://gitlab.com/nbdkit/libnbd/-/pipelines/630565927 ) triggered by Richard W_M_ Jones ( https://gitlab.com/rwmjones ) had 2 failed jobs. Job #2976484741 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2976484741/raw ) Stage: builds Name: x86_64-opensuse-leap-153 Job #2976484747 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2976484747/raw ) Stage: builds Name: x86_64-freebsd-12 -- You're receiving this email because of your account on gitlab.com.

1 0

libnbd | Failed pipeline for master | 0c0082c6
by GitLab 03 Sep '22

03 Sep '22

Pipeline #630556602 has failed! Project: libnbd ( https://gitlab.com/nbdkit/libnbd ) Branch: master ( https://gitlab.com/nbdkit/libnbd/-/commits/master ) Commit: 0c0082c6 ( https://gitlab.com/nbdkit/libnbd/-/commit/0c0082c67bf5af91f5deb80688d729a1e… ) Commit Message: tests: Add language binding tests for stats col... Commit Author: Eric Blake ( https://gitlab.com/ebblake ) Pipeline #630556602 ( https://gitlab.com/nbdkit/libnbd/-/pipelines/630556602 ) triggered by Eric Blake ( https://gitlab.com/ebblake ) had 19 failed jobs. Job #2976447159 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2976447159/raw ) Stage: builds Name: x86_64-alpine-edge Job #2976447161 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2976447161/raw ) Stage: builds Name: x86_64-debian-10 Job #2976447174 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2976447174/raw ) Stage: builds Name: x86_64-freebsd-12 Job #2976447169 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2976447169/raw ) Stage: builds Name: x86_64-opensuse-tumbleweed Job #2976447170 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2976447170/raw ) Stage: builds Name: x86_64-ubuntu-1804 Job #2976447154 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2976447154/raw ) Stage: builds Name: x86_64-alpine-314 Job #2976447172 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2976447172/raw ) Stage: builds Name: i686-debian-10 Job #2976447168 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2976447168/raw ) Stage: builds Name: x86_64-opensuse-leap-153 Job #2976447173 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2976447173/raw ) Stage: builds Name: i686-debian-11 Job #2976447165 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2976447165/raw ) Stage: builds Name: x86_64-fedora-36 Job #2976447163 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2976447163/raw ) Stage: builds Name: x86_64-debian-sid Job #2976447171 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2976447171/raw ) Stage: builds Name: x86_64-ubuntu-2004 Job #2976447156 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2976447156/raw ) Stage: builds Name: x86_64-almalinux-8 Job #2976447164 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2976447164/raw ) Stage: builds Name: x86_64-fedora-35 Job #2976447158 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2976447158/raw ) Stage: builds Name: x86_64-almalinux-8-clang Job #2976447166 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2976447166/raw ) Stage: builds Name: x86_64-fedora-rawhide Job #2976447162 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2976447162/raw ) Stage: builds Name: x86_64-debian-11 Job #2976447160 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2976447160/raw ) Stage: builds Name: x86_64-centos-stream-8 Job #2976447167 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2976447167/raw ) Stage: builds Name: x86_64-fedora-rawhide-clang -- You're receiving this email because of your account on gitlab.com.

1 0