3:20 p.m.
Similar to the existing fua, nocache, nozero and noextents filters,
add a filter to make it easy to override the plugin's choice of
parallel processing. Several possible uses:
- facilitate timing tests of whether a plugin's parallel support
and/or client's use of multi-conn increases throughput
- enable a quick workaround of any plugin that was mistakenly compiled
with a claim of parallel support but with insufficient locking
- serve a parallel plugin to a client that batches up requests but is
not prepared to handle out-of-order replies (however, this does not
necessarily prevent deadlock if the client batches a large write
immediately behind a large read before reading the server's response
to the read)
Update test-eflags (showing the effect on the multi-conn flag) and
test-parallel-file (showing the effect on preventing out-of-order
replies).
In implementing this, I wonder if we should have a thread model
inbetween serialize-requests and parallel. Right now, if I use
--filter=delay rdelay=2 wdelay=3, coupled with a client that starts a
write followed by a read, the behaviors are:
Parallel (filter not used), latency 3, out-of-order replies:
0 1 2 3 4 5
+---+---+---+---+---+
=>write request
=>read request
<=read reply
<=write reply
Serialize-requests (--filter=noparallel), latency 5, in-order replies:
0 1 2 3 4 5
+---+---+---+---+---+
=>write request
<=write reply
============>read request
<=read reply
But merely guaranteeing in-order completion falls in between:
New mode, latency 3, in-order replies:
0 1 2 3 4 5
+---+---+---+---+---+
=>write request
<=write reply
=>read request
<=====read reply
Since nbdkit is already farming out requests to different threads, it
could keep a list of which thread was fired off in which order, then
use pthread_cond_broadcast and friends to block a thread from sending
the reply to the client until all other threads with earlier
sequencing have completed first. But adding such a mode is tricky -
our existing modes are API in nbdkit-common.h, so we can't renumber
NBDKIT_THREAD_MODEL_PARALLEL, yet adding a new mode 5 that falls in
between 3 and 4 changes all our existing code that has so far relied
on descending values being stricter. At the same time, such a mode HAS
to be done in nbdkit proper - by the time a filter's callbacks are
reached within separate threads, scheduling races means the filter has
no reliable way of knowing which of the two threads represents the
earlier client request.
Anyways, if nbdkit adds such a threading model, noparallel would be
the obvious filter to allow easy use of that model with any
pre-existing plugin :)
Signed-off-by: Eric Blake <eblake(a)redhat.com>
---
filters/fua/nbdkit-fua-filter.pod | 1 +
filters/nocache/nbdkit-nocache-filter.pod | 1 +
filters/noextents/nbdkit-noextents-filter.pod | 1 +
.../noparallel/nbdkit-noparallel-filter.pod | 66 ++++++++++++++
filters/nozero/nbdkit-nozero-filter.pod | 1 +
configure.ac | 2 +
filters/noparallel/noparallel.c | 87 +++++++++++++++++++
filters/noparallel/Makefile.am | 61 +++++++++++++
tests/test-eflags.sh | 16 ++++
tests/test-parallel-file.sh | 13 ++-
10 files changed, 248 insertions(+), 1 deletion(-)
create mode 100644 filters/noparallel/nbdkit-noparallel-filter.pod
create mode 100644 filters/noparallel/noparallel.c
create mode 100644 filters/noparallel/Makefile.am
diff --git a/filters/fua/nbdkit-fua-filter.pod b/filters/fua/nbdkit-fua-filter.pod
index b76917b..a5595da 100644
--- a/filters/fua/nbdkit-fua-filter.pod
+++ b/filters/fua/nbdkit-fua-filter.pod
@@ -69,6 +69,7 @@ L<nbdkit-blocksize-filter(1)>,
L<nbdkit-log-filter(1)>,
L<nbdkit-nocache-filter(1)>,
L<nbdkit-noextents-filter(1)>,
+L<nbdkit-noparallel-filter(1)>,
L<nbdkit-nozero-filter(1)>.
=head1 AUTHORS
diff --git a/filters/nocache/nbdkit-nocache-filter.pod
b/filters/nocache/nbdkit-nocache-filter.pod
index 0f43433..3802a7e 100644
--- a/filters/nocache/nbdkit-nocache-filter.pod
+++ b/filters/nocache/nbdkit-nocache-filter.pod
@@ -58,6 +58,7 @@ L<nbdkit-filter(3)>,
L<nbdkit-cache-filter(1)>,
L<nbdkit-fua-filter(1)>,
L<nbdkit-noextents-filter(1)>,
+L<nbdkit-noparallel-filter(1)>,
L<nbdkit-nozero-filter(1)>.
=head1 AUTHORS
diff --git a/filters/noextents/nbdkit-noextents-filter.pod
b/filters/noextents/nbdkit-noextents-filter.pod
index 24519a0..4722392 100644
--- a/filters/noextents/nbdkit-noextents-filter.pod
+++ b/filters/noextents/nbdkit-noextents-filter.pod
@@ -29,6 +29,7 @@ L<nbdkit(1)>,
L<nbdkit-filter(3)>,
L<nbdkit-fua-filter(1)>,
L<nbdkit-nocache-filter(1)>,
+L<nbdkit-noparallel-filter(1)>,
L<nbdkit-nozero-filter(1)>,
L<nbdkit-file-plugin(1)>.
diff --git a/filters/noparallel/nbdkit-noparallel-filter.pod
b/filters/noparallel/nbdkit-noparallel-filter.pod
new file mode 100644
index 0000000..284f6eb
--- /dev/null
+++ b/filters/noparallel/nbdkit-noparallel-filter.pod
@@ -0,0 +1,66 @@
+=head1 NAME
+
+nbdkit-noparallel-filter - nbdkit noparallel filter
+
+=head1 SYNOPSIS
+
+ nbdkit --filter=noparallel plugin [serialize=MODE] [plugin-args...]
+
+=head1 DESCRIPTION
+
+C<nbdkit-noparallel-filter> is a filter that intentionally disables
+parallelism in handling requests from clients. It is mainly useful for
+evaluating timing differences between various levels of
+parallelism. It can also be used as a way to work around any bugs in a
+plugin's claimed level of parallel support, without recompiling the
+plugin, or to ease efforts when connecting with a client that can
+batch up several requests but is not prepared to handle out-of-order
+replies.
+
+=head1 PARAMETERS
+
+=over 4
+
+=item B<serialize=requests|all-requests|connections>
+
+Optional, controls how much serialization the filter will
+enforce. Mode B<requests> (default) prevents a single client from
+having more than one in-flight request, but does not prevent parallel
+requests from a second connection (if the plugin supports that). Mode
+B<all-requests> is stricter, enforcing that at most one request
+(regardless of connection) will be active, but does not prevent
+parallel connections (if the plugin supports that). Mode
+B<connections> is strictest, where there can be at most one client at
+a time, and the server no longer advertise C<NBD_FLAG_MULTI_CONN> to
+clients.
+
+=back
+
+=head1 EXAMPLES
+
+Serve the file F<disk.img>, but disallow out-of-order transaction
+completion to a given client:
+
+ nbdkit --filter=noparallel file disk.img
+
+Serve the file F<disk.img>, but allowing only one client at a time:
+
+ nbdkit --filter=noparallel file serialize=connections disk.img
+
+=head1 SEE ALSO
+
+L<nbdkit(1)>,
+L<nbdkit-file-plugin(1)>,
+L<nbdkit-filter(3)>,
+L<nbdkit-fua-filter(1)>,
+L<nbdkit-nocache-filter(1)>,
+L<nbdkit-noextents-filter(1)>,
+L<nbdkit-nozero-filter(1)>.
+
+=head1 AUTHORS
+
+Eric Blake
+
+=head1 COPYRIGHT
+
+Copyright (C) 2018-2019 Red Hat Inc.
diff --git a/filters/nozero/nbdkit-nozero-filter.pod
b/filters/nozero/nbdkit-nozero-filter.pod
index d94dd8d..144b823 100644
--- a/filters/nozero/nbdkit-nozero-filter.pod
+++ b/filters/nozero/nbdkit-nozero-filter.pod
@@ -54,6 +54,7 @@ L<nbdkit-file-plugin(1)>,
L<nbdkit-filter(3)>,
L<nbdkit-fua-filter(1)>,
L<nbdkit-nocache-filter(1)>,
+L<nbdkit-noparallel-filter(1)>,
L<nbdkit-noextents-filter(1)>.
=head1 AUTHORS
diff --git a/configure.ac b/configure.ac
index 2163930..270e981 100644
--- a/configure.ac
+++ b/configure.ac
@@ -833,6 +833,7 @@ filters="\
log \
nocache \
noextents \
+ noparallel \
nozero \
offset \
partition \
@@ -909,6 +910,7 @@ AC_CONFIG_FILES([Makefile
filters/log/Makefile
filters/nocache/Makefile
filters/noextents/Makefile
+ filters/noparallel/Makefile
filters/nozero/Makefile
filters/offset/Makefile
filters/partition/Makefile
diff --git a/filters/noparallel/noparallel.c b/filters/noparallel/noparallel.c
new file mode 100644
index 0000000..7034b01
--- /dev/null
+++ b/filters/noparallel/noparallel.c
@@ -0,0 +1,87 @@
+/* nbdkit
+ * Copyright (C) 2019 Red Hat Inc.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are
+ * met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ *
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * * Neither the name of Red Hat nor the names of its contributors may be
+ * used to endorse or promote products derived from this software without
+ * specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+ * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
+ * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+ * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+ * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#include <config.h>
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <string.h>
+#include <stdbool.h>
+#include <assert.h>
+
+#include <nbdkit-filter.h>
+
+#define THREAD_MODEL NBDKIT_THREAD_MODEL_PARALLEL
+
+static int thread_model = NBDKIT_THREAD_MODEL_SERIALIZE_REQUESTS;
+
+static int
+noparallel_config (nbdkit_next_config *next, void *nxdata,
+ const char *key, const char *value)
+{
+ if (strcmp (key, "serialize") == 0 ||
+ strcmp (key, "serialise") == 0) {
+ if (strcmp (value, "connections") == 0)
+ thread_model = NBDKIT_THREAD_MODEL_SERIALIZE_CONNECTIONS;
+ else if (strcmp (value, "all_requests") == 0 ||
+ strcmp (value, "all-requests") == 0)
+ thread_model = NBDKIT_THREAD_MODEL_SERIALIZE_ALL_REQUESTS;
+ else if (strcmp (value, "requests") != 0) {
+ nbdkit_error ("unknown noparallel serialize mode '%s'", value);
+ return -1;
+ }
+ return 0;
+ }
+ return next (nxdata, key, value);
+}
+
+#define noparallel_config_help \
+ "serialize=<MODE> 'requests' (default), 'all-requests',
or 'connections'.\n" \
+
+/* Apply runtime reduction to thread model. */
+static int
+noparallel_thread_model (void)
+{
+ return thread_model;
+}
+
+static struct nbdkit_filter filter = {
+ .name = "noparallel",
+ .longname = "nbdkit noparallel filter",
+ .version = PACKAGE_VERSION,
+ .config = noparallel_config,
+ .config_help = noparallel_config_help,
+ .thread_model = noparallel_thread_model,
+};
+
+NBDKIT_REGISTER_FILTER(filter)
diff --git a/filters/noparallel/Makefile.am b/filters/noparallel/Makefile.am
new file mode 100644
index 0000000..9c34ee2
--- /dev/null
+++ b/filters/noparallel/Makefile.am
@@ -0,0 +1,61 @@
+# nbdkit
+# Copyright (C) 2019 Red Hat Inc.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are
+# met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+#
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+#
+# * Neither the name of Red Hat nor the names of its contributors may be
+# used to endorse or promote products derived from this software without
+# specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND
+# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
+# USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+# SUCH DAMAGE.
+
+include $(top_srcdir)/common-rules.mk
+
+EXTRA_DIST = nbdkit-noparallel-filter.pod
+
+filter_LTLIBRARIES = nbdkit-noparallel-filter.la
+
+nbdkit_noparallel_filter_la_SOURCES = \
+ noparallel.c \
+ $(top_srcdir)/include/nbdkit-filter.h
+
+nbdkit_noparallel_filter_la_CPPFLAGS = \
+ -I$(top_srcdir)/include \
+ -I$(top_srcdir)/common/include
+nbdkit_noparallel_filter_la_CFLAGS = \
+ $(WARNINGS_CFLAGS)
+nbdkit_noparallel_filter_la_LDFLAGS = \
+ -module -avoid-version -shared \
+ -Wl,--version-script=$(top_srcdir)/filters/filters.syms
+
+if HAVE_POD
+
+man_MANS = nbdkit-noparallel-filter.1
+CLEANFILES += $(man_MANS)
+
+nbdkit-noparallel-filter.1: nbdkit-noparallel-filter.pod
+ $(PODWRAPPER) --section=1 --man $@ \
+ --html $(top_builddir)/html/$@.html \
+ $<
+
+endif HAVE_POD
diff --git a/tests/test-eflags.sh b/tests/test-eflags.sh
index 14a0099..8158f75 100755
--- a/tests/test-eflags.sh
+++ b/tests/test-eflags.sh
@@ -277,6 +277,22 @@ EOF
# -r
# can_multi_conn=true
+late_args="serialize=connections" do_nbdkit -r --filter=noparallel
<<'EOF'
+case "$1" in
+ get_size) echo 1M ;;
+ can_multi_conn) exit 0 ;;
+ *) exit 2 ;;
+esac
+EOF
+
+[ $eflags -eq $(( HAS_FLAGS|READ_ONLY|SEND_DF )) ] ||
+ fail "expected HAS_FLAGS|READ_ONLY|SEND_DF"
+
+#----------------------------------------------------------------------
+# -r
+# --filter=noparallel serialize=connections
+# can_multi_conn=true
+
do_nbdkit -r <<'EOF'
case "$1" in
get_size) echo 1M ;;
diff --git a/tests/test-parallel-file.sh b/tests/test-parallel-file.sh
index 3012384..8335dc9 100755
--- a/tests/test-parallel-file.sh
+++ b/tests/test-parallel-file.sh
@@ -1,6 +1,6 @@
#!/usr/bin/env bash
# nbdkit
-# Copyright (C) 2017-2018 Red Hat Inc.
+# Copyright (C) 2017-2019 Red Hat Inc.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
@@ -72,4 +72,15 @@ wrote 512/512 bytes at offset 512"; then
exit 1
fi
+# With --filter=noparallel, the write should complete first because it was issued first
+nbdkit -v -U - --filter=noparallel --filter=delay file test-parallel-file.data \
+ wdelay=2 rdelay=1 --run 'qemu-io -f raw -c "aio_write -P 2 512 512" \
+ -c "aio_read -P 1 0 512" -c aio_flush $nbd' |
+ tee test-parallel-file.out
+if test "$(grep '512/512' test-parallel-file.out)" != \
+"wrote 512/512 bytes at offset 512
+read 512/512 bytes at offset 0"; then
+ exit 1
+fi
+
exit 0
--
2.20.1