[nbdkit PATCH 0/4] Allow plugins to disconnect a single client
by Eric Blake
nbdkit_shutdown() is a harsh hammer (it kills the entire server);
sometimes in testing, it is more useful to disconnect just one client
while keeping the server responsive to a reconnect request. Start
this by implementing a new API and putting it to use in the
blocksize-policy filter. I also have a pending libnbd series that
plans to make use of this feature addition in its testsuite for
proving a bug-fix for libnbd not obeying qemu's 32M block_maximum
sizing constraints.
Still to come as followup to this: Rich and I discussed on IRC how to
extend the sh/eval plugins to allow rapid-prototype testing of
requesting client disconnects. The idea we came up with was defining
exit status 4 (nbdkit_shutdown), 5 (nbdkit_disconnect(true)), and 6
(nbdkit_disconnect(false)) as new recognized return values from the
shell scriptlets. I'd probably also change the documentation to
reserve return status 7-15 as currently undefined (right now, only 7
is undefined and 8-15 are currently documented as behaving the same as
exit status 1). The chance of this affecting a real client of the sh
or eval plugin is minimal.
Eric Blake (4):
server: Switch connection status to enum
server: Give client EOF when we are done writing
api: Add nbdkit_disconnect(int)
blocksize-policy: Add blocksize-write-disconnect=N parameter
docs/nbdkit-plugin.pod | 17 ++-
.../nbdkit-blocksize-policy-filter.pod | 21 +++
include/nbdkit-common.h | 3 +-
tests/Makefile.am | 40 +++++-
server/internal.h | 19 ++-
server/connections.c | 57 ++++----
server/crypto.c | 22 +--
server/nbdkit.syms | 3 +-
server/protocol.c | 107 ++++++++-------
server/public.c | 15 ++-
server/test-public.c | 14 +-
plugins/ocaml/NBDKit.mli | 5 +-
plugins/ocaml/NBDKit.ml | 1 +
plugins/ocaml/bindings.c | 12 +-
plugins/python/modfunctions.c | 14 ++
plugins/rust/src/lib.rs | 6 +
filters/blocksize-policy/policy.c | 27 +++-
tests/test-blocksize-write-disconnect.sh | 107 +++++++++++++++
tests/test-disconnect-tls.sh | 126 ++++++++++++++++++
tests/test-disconnect.sh | 100 ++++++++++++++
tests/test-disconnect-plugin.c | 95 +++++++++++++
21 files changed, 709 insertions(+), 102 deletions(-)
create mode 100755 tests/test-blocksize-write-disconnect.sh
create mode 100755 tests/test-disconnect-tls.sh
create mode 100755 tests/test-disconnect.sh
create mode 100644 tests/test-disconnect-plugin.c
--
2.37.3
2 years
[libnbd PATCH 0/3] Improve nbdsh -u handling
by Eric Blake
I was trying to add a TLS test to nbdkit, and got annoyed that I
couldn't just write --run 'nbdsh -u "$uri" -c ...'. This fixes the
libnbd side of the problem (a URI with ?tls-psk-file=... is rejected
unless we tell libnbd that parsing local files is okay); but I will
also be working on patches to nbdkit ($uri is more useful if it
includes 'alice@' and '?tls-psk-file=...' in the first place).
Eric Blake (3):
nbdsh: Refactor handling of -u and friends
nbdsh: Improve --help and initial banner contents.
nbdsh: Allow -u interleaved with -c
python/nbdsh.py | 142 +++++++++++++++++++++++++++------------------
sh/test-context.sh | 26 ++++-----
sh/test-error.sh | 37 +++++++-----
3 files changed, 119 insertions(+), 86 deletions(-)
--
2.37.3
2 years
distutils use in hivex libdnet libnbd (and more) [was: Re: Help needed triaging build failures without distutils]
by Richard W.M. Jones
On Tue, Oct 18, 2022 at 11:27:47AM +0200, Miro Hrončok wrote:
> Hey Pythonistas.
>
> The Python standard library distutils module will be removed from Python 3.12+
>
> https://peps.python.org/pep-0632/
Annoying, but OK ...
> As preparatory work, we build all python packages in a Copr repository
> with Python 3.11 sans distutils:
>
> https://copr.fedorainfracloud.org/coprs/g/python/python-without-distutils/
>
> I've rebuilt all the failed builds again and also in a control-group copr:
>
> https://copr.fedorainfracloud.org/coprs/g/python/python-with-distutils/
>
> 250 packages that failed to build without distuils but succeeded
> with distutils need to be examined and categorized into various
> different groups:
>
> - package uses distutils only if sys.version_info < (3, 12)
> -- this is OK but still fails here
> - package uses distutils unconditionally and the package needs to be fixed
> - package uses another package that uses distutils unconditionally
> and the dependency needs to be fixed
>
> I suspect most of the packages will fail to build with Python 3.12
> (planned for Fedora 39, change proposal TBD). The python3-setutpools
> package provides a distutils module [^1], so sometimes "simply"
> adding BuildRequires: python3-setuptools might workaround the
> problem.
...
> rjones hivex libdnet libnbd
I think your testing methodology might have been wrong because we use
distutils in other packages that I maintain, notably:
https://github.com/libguestfs/libguestfs/blob/master/m4/guestfs-python.m4
I'm not sure what could have happened here. The RPM would have failed
to build if the Python bindings had been ./configure-d out. The link
shows libguestfs as "not build yet / Disabled":
https://copr.fedorainfracloud.org/coprs/g/python/python-without-distutils...
but then there's a build which succeeded:
https://copr.fedorainfracloud.org/coprs/g/python/python-without-distutils...
and it has Python bindings. (The logs of the build don't seem to be
available.)
Anyway I'll see if I can fix this upstream as requested.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
2 years
[nbdkit PATCH 00/10] Various CI improvements
by Eric Blake
This thread is more for reference, since I've already pushed these to
see if it helps turn more of the CI green - although at the moment
debian-sid is not playing nice.
I'm still not sure how to add a VPATH build to the CI tools; what's
more, while I am now getting a native VPATH 'make' to work on Fedora,
'make check' still hits lots of errors (probably where we need more
$builddir references).
Eric Blake (10):
ci: Update with lcitool
build: Fix VPATH build of protostrings.c
mingw: Avoid broken gnutls/socket.h header
build: Fix VPATH build when nbdkit not installed
golang: Attempt VPATH builds
build: Only attempt to build vddk on Linux
rust: Skip building in VPATH
mingw: Provide nbdkit_debug() in Windows replacements
perl: Auto-disable on mingw
ci: Fix typo in last patch
configure.ac | 27 +++++--
common/allocators/Makefile.am | 3 +-
common/bitmap/Makefile.am | 3 +-
common/protocol/Makefile.am | 4 +-
common/regions/Makefile.am | 3 +-
common/replacements/Makefile.am | 7 +-
common/utils/Makefile.am | 3 +-
filters/blocksize-policy/Makefile.am | 5 +-
filters/blocksize/Makefile.am | 3 +-
filters/cache/Makefile.am | 1 +
filters/cacheextents/Makefile.am | 3 +-
filters/checkwrite/Makefile.am | 3 +-
filters/cow/Makefile.am | 3 +-
filters/ddrescue/Makefile.am | 3 +-
filters/delay/Makefile.am | 3 +-
filters/error/Makefile.am | 3 +-
filters/exitlast/Makefile.am | 7 +-
filters/exitwhen/Makefile.am | 5 +-
filters/exportname/Makefile.am | 3 +-
filters/ext2/Makefile.am | 5 +-
filters/extentlist/Makefile.am | 3 +-
filters/fua/Makefile.am | 3 +-
filters/gzip/Makefile.am | 3 +-
filters/ip/Makefile.am | 3 +-
filters/limit/Makefile.am | 5 +-
filters/log/Makefile.am | 5 +-
filters/luks/Makefile.am | 1 +
filters/multi-conn/Makefile.am | 3 +-
filters/nocache/Makefile.am | 3 +-
filters/noextents/Makefile.am | 3 +-
filters/nofilter/Makefile.am | 3 +-
filters/noparallel/Makefile.am | 3 +-
filters/nozero/Makefile.am | 3 +-
filters/offset/Makefile.am | 5 +-
filters/partition/Makefile.am | 3 +-
filters/pause/Makefile.am | 3 +-
filters/protect/Makefile.am | 3 +-
filters/rate/Makefile.am | 3 +-
filters/readahead/Makefile.am | 3 +-
filters/retry-request/Makefile.am | 3 +-
filters/retry/Makefile.am | 3 +-
filters/scan/Makefile.am | 3 +-
filters/stats/Makefile.am | 3 +-
filters/swab/Makefile.am | 3 +-
filters/tar/Makefile.am | 3 +-
filters/tls-fallback/Makefile.am | 3 +-
filters/truncate/Makefile.am | 3 +-
filters/xz/Makefile.am | 5 +-
plugins/cc/Makefile.am | 5 +-
plugins/cdi/Makefile.am | 5 +-
plugins/curl/Makefile.am | 3 +-
plugins/data/Makefile.am | 3 +-
plugins/eval/Makefile.am | 5 +-
plugins/example1/Makefile.am | 3 +-
plugins/example2/Makefile.am | 3 +-
plugins/example3/Makefile.am | 3 +-
plugins/file/Makefile.am | 3 +-
plugins/floppy/Makefile.am | 3 +-
plugins/full/Makefile.am | 3 +-
plugins/golang/Makefile.am | 18 ++---
plugins/guestfs/Makefile.am | 3 +-
plugins/info/Makefile.am | 3 +-
plugins/iso/Makefile.am | 5 +-
plugins/libvirt/Makefile.am | 3 +-
plugins/linuxdisk/Makefile.am | 5 +-
plugins/lua/Makefile.am | 3 +-
plugins/memory/Makefile.am | 5 +-
plugins/nbd/Makefile.am | 3 +-
plugins/null/Makefile.am | 3 +-
plugins/ocaml/Makefile.am | 3 +-
plugins/ondemand/Makefile.am | 5 +-
plugins/partitioning/Makefile.am | 3 +-
plugins/pattern/Makefile.am | 3 +-
plugins/perl/Makefile.am | 5 +-
plugins/python/Makefile.am | 5 +-
plugins/random/Makefile.am | 5 +-
plugins/ruby/Makefile.am | 3 +-
plugins/sh/Makefile.am | 3 +-
plugins/sparse-random/Makefile.am | 5 +-
plugins/split/Makefile.am | 5 +-
plugins/ssh/Makefile.am | 3 +-
plugins/tcl/Makefile.am | 3 +-
plugins/tmpdisk/Makefile.am | 5 +-
plugins/torrent/Makefile.am | 5 +-
plugins/vddk/Makefile.am | 3 +-
plugins/zero/Makefile.am | 6 +-
server/Makefile.am | 4 +-
tests/Makefile.am | 37 ++++++++--
server/crypto.c | 12 +++-
common/replacements/fsync.c | 3 +-
common/replacements/pread.c | 3 +-
common/replacements/pwrite.c | 3 +-
ci/build.sh | 11 +--
ci/buildenv/fedora-35-cross-mingw32.sh | 2 +-
ci/buildenv/fedora-35-cross-mingw64.sh | 2 +-
ci/buildenv/opensuse-leap-154.sh | 71 ++++++++++++++++++
.../fedora-35-cross-mingw32.Dockerfile | 2 +-
.../fedora-35-cross-mingw64.Dockerfile | 2 +-
ci/containers/opensuse-leap-154.Dockerfile | 72 +++++++++++++++++++
ci/gitlab.yml | 1 +
ci/gitlab/build-templates.yml | 5 ++
ci/gitlab/builds.yml | 26 +++++--
ci/gitlab/container-templates.yml | 1 +
ci/gitlab/containers.yml | 7 ++
ci/manifest.yml | 8 ++-
105 files changed, 472 insertions(+), 147 deletions(-)
create mode 100644 ci/buildenv/opensuse-leap-154.sh
create mode 100644 ci/containers/opensuse-leap-154.Dockerfile
--
2.37.3
2 years
Why cant't use the pie link option when compiling libgustfs
by guiHua
Hello, When I compile libgustfs, it will throw exception: undefined reference to 'caml_local_roots' when adding pie link option.
Why is that? The command is like this: ./configure CFLAGS="-O2 -fPIC" LDFLAGS="-pie -Wl".
I look forward to hearing from you.
2 years
[libnbd PATCH 0/2] Get back to green CI
by Eric Blake
With some help from Dan, I've managed to get back to a passing
pipeline on gitlab! I'm pushing this now.
Eric Blake (2):
ci: Add opensuse-leap 15.4, skip TLS on 15.3
ci: Add coverage of --without-libxml2
ci/build.sh | 22 ++++++--
ci/buildenv/opensuse-leap-154.sh | 61 +++++++++++++++++++++
ci/containers/opensuse-leap-154.Dockerfile | 62 ++++++++++++++++++++++
ci/gitlab.yml | 1 +
ci/gitlab/build-templates.yml | 5 ++
ci/gitlab/builds.yml | 40 ++++++++++++++
ci/gitlab/container-templates.yml | 1 +
ci/gitlab/containers.yml | 7 +++
ci/manifest.yml | 15 +++++-
dump/dump-data.sh | 1 +
dump/dump-pattern.sh | 1 +
11 files changed, 210 insertions(+), 6 deletions(-)
create mode 100644 ci/buildenv/opensuse-leap-154.sh
create mode 100644 ci/containers/opensuse-leap-154.Dockerfile
--
2.37.3
2 years