[PATCH libnbd 0/4] Various nbdcopy cleanups
by Nir Soffer
Few minor refactorings, hopefully makeing the code more clear and less error
prone.
Nir Soffer (4):
copy: nbd-ops.c: Less convoluted error handling
copy: file-ops.c: Remove unneeded check
copy: file-ops.c: Remove unneeded arguments
copy: file-ops.c: Rename clean up label
copy/file-ops.c | 41 +++++++++++++++++++++--------------------
copy/nbd-ops.c | 27 ++++++++++++++++-----------
2 files changed, 37 insertions(+), 31 deletions(-)
--
2.26.2
3 years, 2 months
[libnbd PATCH] opt_go: Tolerate unplanned server death
by Eric Blake
While debugging some experimental nbdkit code that was triggering an
assertion failure in nbdkit, I noticed a secondary failure of nbdsh
also dying from an assertion:
libnbd: debug: nbdsh: nbd_opt_go: transition: NEWSTYLE.OPT_GO.SEND -> DEAD
libnbd: debug: nbdsh: nbd_opt_go: option queued, ignoring state machine failure
nbdsh: opt.c:86: nbd_unlocked_opt_go: Assertion `nbd_internal_is_state_negotiating (get_next_state (h))' failed.
Although my trigger was from non-production nbdkit code, libnbd should
never die from an assertion failure merely because a server
disappeared at the wrong moment during an incomplete reply to
NBD_OPT_GO or NBD_OPT_INFO.
Fixes: bbf1c51392 (api: Give aio_opt_go a completion callback)
---
lib/opt.c | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/lib/opt.c b/lib/opt.c
index 2317b72..e5802f4 100644
--- a/lib/opt.c
+++ b/lib/opt.c
@@ -1,5 +1,5 @@
/* NBD client library in userspace
- * Copyright (C) 2020 Red Hat Inc.
+ * Copyright (C) 2020-2021 Red Hat Inc.
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
@@ -83,7 +83,8 @@ nbd_unlocked_opt_go (struct nbd_handle *h)
r = wait_for_option (h);
if (r == 0 && err) {
- assert (nbd_internal_is_state_negotiating (get_next_state (h)));
+ assert (nbd_internal_is_state_negotiating (get_next_state (h)) ||
+ nbd_internal_is_state_dead (get_next_state (h)));
set_error (err, "server replied with error to opt_go request");
return -1;
}
@@ -105,7 +106,8 @@ nbd_unlocked_opt_info (struct nbd_handle *h)
r = wait_for_option (h);
if (r == 0 && err) {
- assert (nbd_internal_is_state_negotiating (get_next_state (h)));
+ assert (nbd_internal_is_state_negotiating (get_next_state (h)) ||
+ nbd_internal_is_state_dead (get_next_state (h)));
set_error (err, "server replied with error to opt_info request");
return -1;
}
--
2.30.1
3 years, 2 months
[PATCH libnbd] copy: Test --request-size option
by Nir Soffer
Add simple test similar to copy-sparse-no-extents testing that
--request-size is respected.
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
copy/Makefile.am | 2 +
copy/copy-sparse-request-size.sh | 92 ++++++++++++++++++++++++++++++++
2 files changed, 94 insertions(+)
create mode 100755 copy/copy-sparse-request-size.sh
diff --git a/copy/Makefile.am b/copy/Makefile.am
index c8b28ac..d85ceff 100644
--- a/copy/Makefile.am
+++ b/copy/Makefile.am
@@ -36,6 +36,7 @@ EXTRA_DIST = \
copy-sparse.sh \
copy-sparse-allocated.sh \
copy-sparse-no-extents.sh \
+ copy-sparse-request-size.sh \
copy-sparse-to-stream.sh \
copy-stdin-to-nbd.sh \
copy-stdin-to-null.sh \
@@ -122,6 +123,7 @@ TESTS += \
copy-sparse.sh \
copy-sparse-allocated.sh \
copy-sparse-no-extents.sh \
+ copy-sparse-request-size.sh \
copy-sparse-to-stream.sh \
$(ROOT_TESTS) \
test-long-options.sh \
diff --git a/copy/copy-sparse-request-size.sh b/copy/copy-sparse-request-size.sh
new file mode 100755
index 0000000..b539f5e
--- /dev/null
+++ b/copy/copy-sparse-request-size.sh
@@ -0,0 +1,92 @@
+#!/usr/bin/env bash
+# nbd client library in userspace
+# Copyright (C) 2021 Red Hat Inc.
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+
+# Adapted from copy-sparse-no-extents.sh
+#
+# This test depends on the nbdkit default sparse block size (32K).
+
+. ../tests/functions.sh
+
+set -e
+set -x
+
+# Skip this test under valgrind, it takes too long.
+if [ "x$LIBNBD_VALGRIND" = "x1" ]; then
+ echo "$0: test skipped under valgrind"
+ exit 77
+fi
+
+requires nbdkit --version
+requires nbdkit --exit-with-parent --version
+requires nbdkit data --version
+requires nbdkit eval --version
+
+out=copy-sparse-request-size.out
+cleanup_fn rm -f $out
+
+$VG nbdcopy --no-extents -S 0 --request-size=1048576 -- \
+ [ nbdkit --exit-with-parent data data='
+ 1
+ @33554431 1
+ ' ] \
+ [ nbdkit --exit-with-parent eval \
+ get_size=' echo 33554432 ' \
+ pwrite=" echo \$@ >> $out " \
+ trim=" echo \$@ >> $out " \
+ zero=" echo \$@ >> $out " ]
+
+sort -n -o $out $out
+
+echo Output:
+cat $out
+
+if [ "$(cat $out)" != "pwrite 1048576 0
+pwrite 1048576 1048576
+pwrite 1048576 10485760
+pwrite 1048576 11534336
+pwrite 1048576 12582912
+pwrite 1048576 13631488
+pwrite 1048576 14680064
+pwrite 1048576 15728640
+pwrite 1048576 16777216
+pwrite 1048576 17825792
+pwrite 1048576 18874368
+pwrite 1048576 19922944
+pwrite 1048576 2097152
+pwrite 1048576 20971520
+pwrite 1048576 22020096
+pwrite 1048576 23068672
+pwrite 1048576 24117248
+pwrite 1048576 25165824
+pwrite 1048576 26214400
+pwrite 1048576 27262976
+pwrite 1048576 28311552
+pwrite 1048576 29360128
+pwrite 1048576 30408704
+pwrite 1048576 3145728
+pwrite 1048576 31457280
+pwrite 1048576 32505856
+pwrite 1048576 4194304
+pwrite 1048576 5242880
+pwrite 1048576 6291456
+pwrite 1048576 7340032
+pwrite 1048576 8388608
+pwrite 1048576 9437184" ]; then
+ echo "$0: output does not match expected"
+ exit 1
+fi
--
2.26.2
3 years, 2 months
[nbdkit PATCH v2 0/6] Add multi-conn filter
by Eric Blake
In v2:
- lots more patches to allow cross-connection plugin calls
- added patch for exportname filtering
- working testsuite, including bug fixes in v1 that it uncovered
- more cross-references in man pages
Eric Blake (6):
backend: Split out new backend_handle_* functions
Revert "filters: Remove most next_* wrappers"
Revert "server: filters: Remove struct b_h."
filters: Track next handle alongside next backend
multi-conn: New filter
multi-conn: Add knob to limit consistency emulation by export name
docs/nbdkit-filter.pod | 4 +-
filters/cache/nbdkit-cache-filter.pod | 5 +-
filters/fua/nbdkit-fua-filter.pod | 7 +
.../multi-conn/nbdkit-multi-conn-filter.pod | 201 ++++++++
filters/nocache/nbdkit-nocache-filter.pod | 1 +
filters/noextents/nbdkit-noextents-filter.pod | 1 +
.../noparallel/nbdkit-noparallel-filter.pod | 1 +
filters/nozero/nbdkit-nozero-filter.pod | 1 +
include/nbdkit-filter.h | 145 +++---
configure.ac | 4 +-
filters/multi-conn/Makefile.am | 68 +++
tests/Makefile.am | 13 +-
server/internal.h | 60 ++-
server/backend.c | 147 +++++-
server/extents.c | 7 +-
server/filters.c | 424 +++++++++++++---
filters/checkwrite/checkwrite.c | 2 +-
filters/exitlast/exitlast.c | 2 +-
filters/exitwhen/exitwhen.c | 2 +-
filters/gzip/gzip.c | 2 +-
filters/limit/limit.c | 6 +-
filters/log/log.c | 2 +-
filters/multi-conn/multi-conn.c | 470 ++++++++++++++++++
filters/tar/tar.c | 2 +-
tests/test-multi-conn-name.sh | 88 ++++
tests/test-multi-conn-plugin.sh | 141 ++++++
tests/test-multi-conn.sh | 293 +++++++++++
TODO | 7 -
28 files changed, 1921 insertions(+), 185 deletions(-)
create mode 100644 filters/multi-conn/nbdkit-multi-conn-filter.pod
create mode 100644 filters/multi-conn/Makefile.am
create mode 100644 filters/multi-conn/multi-conn.c
create mode 100755 tests/test-multi-conn-name.sh
create mode 100755 tests/test-multi-conn-plugin.sh
create mode 100755 tests/test-multi-conn.sh
--
2.30.1
3 years, 2 months
[PATCH libnbd 0/3] Playing with request size and libev
by Nir Soffer
The first 2 patches add --request-size option, to allow tuning nbdcopy for
particular environment. Testing in our scale lab shows significant improvement
using smaller request size.
The last patch adds example for integrating libnbd with libev event loop,
similar to the glib main loop example. It is not related but it was useful to
test it with --request-size.
Nir Soffer (3):
copy: Allocate required size instead of maximum
copy: Add --request-size option
examples: Add example for integrating with libev
.gitignore | 1 +
configure.ac | 19 +++
copy/main.c | 18 +++
copy/multi-thread-copying.c | 12 +-
copy/nbdcopy.h | 2 +
copy/nbdcopy.pod | 7 +-
copy/synch-copying.c | 17 +-
examples/Makefile.am | 22 +++
examples/copy-libev.c | 304 ++++++++++++++++++++++++++++++++++++
9 files changed, 391 insertions(+), 11 deletions(-)
create mode 100644 examples/copy-libev.c
--
2.26.2
3 years, 2 months