October 2020 - Libguestfs - libguestfs List Archives

[PATCH nbdkit] tests/test-exportname.sh: Fix test.
by Richard W.M. Jones 05 Oct '20

05 Oct '20

When “nbdkit ... --run 'nbdinfo ...' > out” is expected to fail, we cannot be sure of the content of the "out" file. In particular nbdinfo produces its output incrementally, so in JSON mode there's usually a stray "{" in the output, and in non-JSON mode it usually prints the "protocol: " line. Thus it's not correct to test for the output file being non-empty. Remove these lines which caused the test to fail. However I replaced them with "cat" for diagnostic purposes. Fixes: commit 7623b2cc45078cca88fdd2d96c70c7f82a0db49d --- tests/test-exportname.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/test-exportname.sh b/tests/test-exportname.sh index 2ec45023..54e04f86 100755 --- a/tests/test-exportname.sh +++ b/tests/test-exportname.sh @@ -172,12 +172,12 @@ test "$(jq -c "$query" exportname.out)" = '[["c","cc",3]]' # Test strict mode nbdkit -U - --filter=exportname sh exportname.sh exportname-strict=true \ --run 'nbdinfo --no-content --json "$uri"' > exportname.out && fail=1 -test ! -s exportname.out +cat exportname.out nbdkit -U - --filter=exportname sh exportname.sh exportname-strict=true \ exportname=a exportname=b exportname=c \ --run 'nbdinfo --no-content --json "$uri"' > exportname.out && fail=1 -test ! -s exportname.out +cat exportname.out nbdkit -U - --filter=exportname sh exportname.sh exportname-strict=true \ exportname=a exportname=b exportname= default-export=a\ -- 2.27.0

2 1

nbdkit test failure with new libnbd
by Richard W.M. Jones 05 Oct '20

05 Oct '20

-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v

1 0

[PATCH nbdkit v2 0/3] ip: Add filtering by process ID, user ID and group ID.
by Richard W.M. Jones 05 Oct '20

05 Oct '20

This is just a simple update to: https://www.redhat.com/archives/libguestfs/2020-October/msg00015.html rebased on top of current nbdkit master because I pushed a few simple refactorings. Rich.

3 13

[PATCH nbdkit v3 0/4] ip: Add filtering by process ID, user ID and group ID.
by Richard W.M. Jones 05 Oct '20

05 Oct '20

v2 was here: https://www.redhat.com/archives/libguestfs/2020-October/msg00019.html v3: * defence -> defense * Use int64_t instead of int. This compiles on Windows. * Add GC wrappers to OCaml bindings. * New FreeBSD patch. * Removed "pid:" example from the ip filter manual, and added a warning beside the pid documentation. Rich.

1 4

[libnbd PATCH v2 0/2] opt_list_meta_context
by Eric Blake 03 Oct '20

03 Oct '20

In v2: ack'ed preliminary patches have been pushed, and I've added a lot of testsuite coverage as well as putting the new API to use in nbdinfo. Eric Blake (2): api: Add nbd_opt_list_meta_context info: List available meta-contexts lib/internal.h | 1 + generator/API.ml | 84 ++++++++- generator/states-newstyle-opt-meta-context.c | 83 +++++++-- generator/states-newstyle.c | 3 + lib/opt.c | 73 ++++++++ python/t/240-opt-list-meta.py | 79 +++++++++ ocaml/tests/Makefile.am | 3 + ocaml/tests/test_240_opt_list_meta.ml | 80 +++++++++ tests/Makefile.am | 10 ++ tests/opt-list-meta.c | 167 ++++++++++++++++++ .gitignore | 1 + TODO | 11 ++ golang/Makefile.am | 1 + .../libnbd/libnbd_240_opt_list_meta_test.go | 146 +++++++++++++++ info/info-json.sh | 2 + info/info-list-json.sh | 4 + info/info-list.sh | 2 + info/info-text.sh | 2 + info/nbdinfo.c | 89 +++++++++- 19 files changed, 815 insertions(+), 26 deletions(-) create mode 100644 python/t/240-opt-list-meta.py create mode 100644 ocaml/tests/test_240_opt_list_meta.ml create mode 100644 tests/opt-list-meta.c create mode 100644 golang/src/libguestfs.org/libnbd/libnbd_240_opt_list_meta_test.go -- 2.28.0

2 4

[PATCH nbdkit 0/2] ip: Add filtering by process ID, user ID and group ID.
by Richard W.M. Jones 03 Oct '20

03 Oct '20

These two commits add new APIs and enhance nbdkit-ip-filter to allow filtering of Unix domain sockets by the client's PID, UID or GID. eg: nbdkit -U sock --filter=ip ... allow=uid:`id -u` deny=all Rich.

1 2

virt-builder error: Failed to connect to archive.libguestfs.org port 80: Network is unreachable
by Nir Soffer 02 Oct '20

02 Oct '20

Seems that archive.libguestfs.org has issues: $ virt-builder --list curl: (7) Failed to connect to archive.libguestfs.org port 80: Network is unreachable virt-builder: error: external command ‘'curl' -q --config '/tmp/virt-builder.zfIvaP/guestfscurl5afb25.conf'’ exited with error 7 If reporting bugs, run virt-builder with debugging enabled and include the complete output: virt-builder -v -x [...] This breaks my tests, trying to create images with virt-builder. https://travis-ci.org/github/nirs/ovirt-imageio/jobs/732057837 Nir

2 1

[libnbd PATCH 0/3] opt_list_meta_context
by Eric Blake 01 Oct '20

01 Oct '20

I'm posting this now, as I'm at the end of a workday and I got things working for manual experimentation. Still to do: - write interop tests for qemu-nbd and nbdkit (including my proposed patch addition of qemu-nbd -A to show qemu:allocation-depth) - figure out if we can make 'nbdinfo --map' use the new API to automatically select all contexts advertised by the server Eric Blake (3): api: Add get_nr_meta_contexts, clear_meta_contexts generator: Rename OPT_SET_META_CONTEXT states api: Add nbd_opt_list_meta_context lib/internal.h | 1 + generator/API.ml | 162 +++++++++++++++++- generator/Makefile.am | 4 +- generator/state_machine.ml | 6 +- ...t.c => states-newstyle-opt-meta-context.c} | 106 ++++++++---- .../states-newstyle-opt-structured-reply.c | 4 +- generator/states-newstyle.c | 5 +- lib/handle.c | 32 ++++ lib/opt.c | 73 ++++++++ tests/meta-base-allocation.c | 36 +++- TODO | 11 ++ 11 files changed, 394 insertions(+), 46 deletions(-) rename generator/{states-newstyle-opt-set-meta-context.c => states-newstyle-opt-meta-context.c} (66%) -- 2.28.0

2 13

[PATCH nbdinfo v2] info: Add a --map option for displaying allocation metadata.
by Richard W.M. Jones 01 Oct '20

01 Oct '20

Fixes bugs with JSON output in the previous patch, and adds some tests. Rich.

2 3

[nbdkit PATCH] server: Adjust limit on max NBD_OPT_* from client
by Eric Blake 01 Oct '20

01 Oct '20

Up to nbdkit 1.22, we never advertised large export lists, so a client had no real reason to give more than 32 NBD_OPT_ commands before finally selecting an export or quitting, and we were justified in dropping indecisive chatty clients as being a waste of server resources. But now that we support .list_exports, it is reasonable for a client (such as 'qemu-nbd --list' or 'nbdinfo --list') to want to call NBD_OPT_INFO and several NBD_OPT_LIST_META_CONTEXT commands for every export returned in NBD_OPT_LIST. We still want to avoid clients that can tie us up in eternal handshaking, so let's reject a second call to NBD_OPT_LIST; but once the first call is made, the client now has a chance to get info on everything listed. Signed-off-by: Eric Blake <eblake(a)redhat.com> --- server/protocol-handshake-newstyle.c | 49 ++++++++++++++++++++-------- 1 file changed, 35 insertions(+), 14 deletions(-) diff --git a/server/protocol-handshake-newstyle.c b/server/protocol-handshake-newstyle.c index 9c86639b..dedb7f9d 100644 --- a/server/protocol-handshake-newstyle.c +++ b/server/protocol-handshake-newstyle.c @@ -44,5 +44,8 @@ #include "nbd-protocol.h" #include "protostrings.h" -/* Maximum number of client options we allow before giving up. */ +/* Initial bound of client options we allow before giving up. + * However, a client that issues NBD_OPT_LIST is permitted to follow + * up with another round of options per export listed. + */ #define MAX_NR_OPTIONS 32 /* Receive newstyle options. */ @@ -78,12 +81,13 @@ send_newstyle_option_reply (uint32_t option, uint32_t reply) /* Reply to NBD_OPT_LIST with the plugin's list of export names. */ static int -send_newstyle_option_reply_exportnames (uint32_t option) +send_newstyle_option_reply_exportnames (uint32_t option, size_t *nr_options) { GET_CONN; struct nbd_fixed_new_option_reply fixed_new_option_reply; - size_t i; + size_t i, list_len; CLEANUP_EXPORTS_FREE struct nbdkit_exports *exps = NULL; + int r; exps = nbdkit_exports_new (); if (exps == NULL) @@ -91,7 +95,8 @@ send_newstyle_option_reply_exportnames (uint32_t option) if (backend_list_exports (top, read_only, exps) == -1) return send_newstyle_option_reply (option, NBD_REP_ERR_PLATFORM); - for (i = 0; i < nbdkit_exports_count (exps); i++) { + list_len = nbdkit_exports_count (exps); + for (i = 0; i < list_len; i++) { const struct nbdkit_export export = nbdkit_get_export (exps, i); size_t name_len = strlen (export.name); size_t desc_len = export.description ? strlen (export.description) : 0; @@ -127,7 +132,11 @@ send_newstyle_option_reply_exportnames (uint32_t option) } } - return send_newstyle_option_reply (option, NBD_REP_ACK); + r = send_newstyle_option_reply (option, NBD_REP_ACK); + /* Allow additional per-export NBD_OPT_INFO and friends. */ + if (r == 0) + *nr_options += MAX_NR_OPTIONS * list_len; + return r; } static int @@ -326,6 +335,7 @@ negotiate_handshake_newstyle_options (void) GET_CONN; struct nbd_new_option new_option; size_t nr_options; + bool list_seen = false; uint64_t version; uint32_t option; uint32_t optlen; @@ -334,7 +344,7 @@ negotiate_handshake_newstyle_options (void) uint64_t exportsize; struct backend *b; - for (nr_options = 0; nr_options < MAX_NR_OPTIONS; ++nr_options) { + for (nr_options = MAX_NR_OPTIONS; nr_options > 0; --nr_options) { CLEANUP_FREE char *data = NULL; if (conn_recv_full (&new_option, sizeof new_option, @@ -431,11 +441,22 @@ negotiate_handshake_newstyle_options (void) continue; } - /* Send back the exportname list. */ - debug ("newstyle negotiation: %s: advertising exports", - name_of_nbd_opt (option)); - if (send_newstyle_option_reply_exportnames (option) == -1) - return -1; + if (list_seen) { + debug ("newstyle negotiation: %s: export list already advertised", + name_of_nbd_opt (option)); + if (send_newstyle_option_reply (option, NBD_REP_ERR_INVALID) + == -1) + return -1; + continue; + } + else { + /* Send back the exportname list. */ + debug ("newstyle negotiation: %s: advertising exports", + name_of_nbd_opt (option)); + if (send_newstyle_option_reply_exportnames (option, &nr_options) == -1) + return -1; + list_seen = true; + } break; case NBD_OPT_STARTTLS: @@ -826,8 +847,8 @@ negotiate_handshake_newstyle_options (void) break; } - if (nr_options >= MAX_NR_OPTIONS) { - nbdkit_error ("client exceeded maximum number of options (%d)", - MAX_NR_OPTIONS); + if (nr_options == 0) { + nbdkit_error ("client spent too much time negotiating without selecting " + "an export"); return -1; } -- 2.28.0

2 1