[Libguestfs] [nbdkit PATCH v2 24/24] nocache: Implement new filter

Similar to the existing fua, nozero and noextents filters, add a filter to make it easy to override the basic caching functionality, in part to facilitate timing tests of whether a plugin's cache implementation is worthwhile. A worthwhile test to add to the testsuite would connect the log filter both before and after the nocache filter, to prove how caching requests are altered. However, until we use libnbd in the nbdkit testsuite, there isn't really a clean way to trigger NBD_CMD_CACHE calls to test this (qemu-nbd can't do it). But I can at least exploit test-eflags to use 'qemu-nbd --list' to demonstrate that the advertised flag changes, with a little tweak to allow the test to run filters with arguments. Signed-off-by: Eric Blake <eblake(a)redhat.com&gt; --- filters/fua/nbdkit-fua-filter.pod | 5 +- filters/nocache/nbdkit-nocache-filter.pod | 69 +++++++++++ filters/noextents/nbdkit-noextents-filter.pod | 2 + filters/nozero/nbdkit-nozero-filter.pod | 2 + configure.ac | 2 + filters/nocache/nocache.c | 113 ++++++++++++++++++ filters/nocache/Makefile.am | 61 ++++++++++ tests/test-eflags.sh | 21 +++- 8 files changed, 272 insertions(+), 3 deletions(-) create mode 100644 filters/nocache/nbdkit-nocache-filter.pod create mode 100644 filters/nocache/nocache.c create mode 100644 filters/nocache/Makefile.am diff --git a/filters/fua/nbdkit-fua-filter.pod b/filters/fua/nbdkit-fua-filter.pod index bd08a8c..b76917b 100644 --- a/filters/fua/nbdkit-fua-filter.pod +++ b/filters/fua/nbdkit-fua-filter.pod @@ -66,7 +66,10 @@ L<nbdkit(1)>, L<nbdkit-file-plugin(1)>, L<nbdkit-filter(3)>, L<nbdkit-blocksize-filter(1)>, -L<nbdkit-log-filter(1)>. +L<nbdkit-log-filter(1)>, +L<nbdkit-nocache-filter(1)>, +L<nbdkit-noextents-filter(1)>, +L<nbdkit-nozero-filter(1)>. =head1 AUTHORS diff --git a/filters/nocache/nbdkit-nocache-filter.pod b/filters/nocache/nbdkit-nocache-filter.pod new file mode 100644 index 0000000..0f43433 --- /dev/null +++ b/filters/nocache/nbdkit-nocache-filter.pod @@ -0,0 +1,69 @@ +=head1 NAME + +nbdkit-nocache-filter - nbdkit nocache filter + +=head1 SYNOPSIS + + nbdkit --filter=nocache plugin [cachemode=MODE] [plugin-args...] + +=head1 DESCRIPTION + +C<nbdkit-nocache-filter> is a filter that intentionally disables +efficient handling of advisory client cache requests across the NBD +protocol. It is mainly useful for evaluating timing differences to +determine the impact of caching requests. + +Note that the effects of this filter (in crippling handling of client +cache requests) is somewhat orthogonal from that of the +L<nbdkit-cache-filter(1)> (adding local caching of client read/write +requests); the two filters can be run together to experiment with +timings. + +=head1 PARAMETERS + +=over 4 + +=item B<cachemode=none|emulate|nop> + +Optional, controls which mode the filter will use. Mode B<none> +(default) means that cache support is not advertised to the +client. Mode B<emulate> means that cache support is emulated by the +filter using the plugin's C<pread> callback, regardless of whether the +plugin itself implemented the C<cache> callback. Mode B<nop> means +that cache requests are always accepted and immediately ignored, +rather than having any actual impact. + +=back + +=head1 EXAMPLES + +Serve the file F<disk.img>, but prevent C<NBD_CMD_CACHE> requests +altogether, to get a baseline timing of behavior when the client is +unable to make cache requests: + + nbdkit --filter=nocache file disk.img + +Serve the file F<disk.img>, but with cache requests silently ignored, +rather than being forwarded on to the file plugin (which attempts to +use L<posix_fadvise(3)>), to compare against the timings without the +filter and determine whether the file plugin caching was worthwhile: + + nbdkit --filter=nocache file disk.img cachemode=nop + +=head1 SEE ALSO + +L<nbdkit(1)>, +L<nbdkit-file-plugin(1)>, +L<nbdkit-filter(3)>, +L<nbdkit-cache-filter(1)>, +L<nbdkit-fua-filter(1)>, +L<nbdkit-noextents-filter(1)>, +L<nbdkit-nozero-filter(1)>. + +=head1 AUTHORS + +Eric Blake + +=head1 COPYRIGHT + +Copyright (C) 2019 Red Hat Inc. diff --git a/filters/noextents/nbdkit-noextents-filter.pod b/filters/noextents/nbdkit-noextents-filter.pod index 46f6bdb..24519a0 100644 --- a/filters/noextents/nbdkit-noextents-filter.pod +++ b/filters/noextents/nbdkit-noextents-filter.pod @@ -27,6 +27,8 @@ plugin in the normal way. L<nbdkit(1)>, L<nbdkit-filter(3)>, +L<nbdkit-fua-filter(1)>, +L<nbdkit-nocache-filter(1)>, L<nbdkit-nozero-filter(1)>, L<nbdkit-file-plugin(1)>. diff --git a/filters/nozero/nbdkit-nozero-filter.pod b/filters/nozero/nbdkit-nozero-filter.pod index 8e694bb..d94dd8d 100644 --- a/filters/nozero/nbdkit-nozero-filter.pod +++ b/filters/nozero/nbdkit-nozero-filter.pod @@ -52,6 +52,8 @@ the data to be written explicitly rather than punching any holes: L<nbdkit(1)>, L<nbdkit-file-plugin(1)>, L<nbdkit-filter(3)>, +L<nbdkit-fua-filter(1)>, +L<nbdkit-nocache-filter(1)>, L<nbdkit-noextents-filter(1)>. =head1 AUTHORS diff --git a/configure.ac b/configure.ac index 06124c5..2163930 100644 --- a/configure.ac +++ b/configure.ac @@ -831,6 +831,7 @@ filters="\ error \ fua \ log \ + nocache \ noextents \ nozero \ offset \ @@ -906,6 +907,7 @@ AC_CONFIG_FILES([Makefile filters/error/Makefile filters/fua/Makefile filters/log/Makefile + filters/nocache/Makefile filters/noextents/Makefile filters/nozero/Makefile filters/offset/Makefile diff --git a/filters/nocache/nocache.c b/filters/nocache/nocache.c new file mode 100644 index 0000000..abb042e --- /dev/null +++ b/filters/nocache/nocache.c @@ -0,0 +1,113 @@ +/* nbdkit + * Copyright (C) 2018-2019 Red Hat Inc. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * * Neither the name of Red Hat nor the names of its contributors may be + * used to endorse or promote products derived from this software without + * specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, + * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A + * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR + * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF + * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND + * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, + * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT + * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include <config.h> + +#include <stdio.h> +#include <stdlib.h> +#include <stdint.h> +#include <string.h> +#include <stdbool.h> +#include <assert.h> + +#include <nbdkit-filter.h> + +#include "minmax.h" + +#define THREAD_MODEL NBDKIT_THREAD_MODEL_PARALLEL + +static enum CacheMode { + NONE, + EMULATE, + NOP, +} cachemode; + +static int +nocache_config (nbdkit_next_config *next, void *nxdata, + const char *key, const char *value) +{ + if (strcmp (key, "cachemode") == 0) { + if (strcmp (value, "emulate") == 0) + cachemode = EMULATE; + else if (strcmp (value, "nop") == 0 || + strcmp (value, "no-op") == 0) + cachemode = NOP; + else if (strcmp (value, "none") != 0) { + nbdkit_error ("unknown cachemode '%s'", value); + return -1; + } + return 0; + } + return next (nxdata, key, value); +} + +#define nocache_config_help \ + "cachemode=<MODE> Either 'none' (default), 'emulate', or 'nop'.\n" \ + +/* Advertise desired FLAG_SEND_CACHE mode. */ +static int +nocache_can_cache (struct nbdkit_next_ops *next_ops, void *nxdata, + void *handle) +{ + switch (cachemode) { + case NONE: + return NBDKIT_CACHE_NONE; + case EMULATE: + return NBDKIT_CACHE_EMULATE; + case NOP: + return NBDKIT_CACHE_NATIVE; + } + abort (); +} + +static int +nocache_cache (struct nbdkit_next_ops *next_ops, void *nxdata, + void *handle, uint32_t count, uint64_t offs, uint32_t flags, + int *err) +{ + assert (cachemode == NOP); + assert (!flags); + + return 0; +} + +static struct nbdkit_filter filter = { + .name = "nocache", + .longname = "nbdkit nocache filter", + .version = PACKAGE_VERSION, + .config = nocache_config, + .config_help = nocache_config_help, + .can_cache = nocache_can_cache, + .cache = nocache_cache, +}; + +NBDKIT_REGISTER_FILTER(filter) diff --git a/filters/nocache/Makefile.am b/filters/nocache/Makefile.am new file mode 100644 index 0000000..aa83937 --- /dev/null +++ b/filters/nocache/Makefile.am @@ -0,0 +1,61 @@ +# nbdkit +# Copyright (C) 2018-2019 Red Hat Inc. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are +# met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# * Neither the name of Red Hat nor the names of its contributors may be +# used to endorse or promote products derived from this software without +# specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, +# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A +# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR +# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF +# USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND +# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT +# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# SUCH DAMAGE. + +include $(top_srcdir)/common-rules.mk + +EXTRA_DIST = nbdkit-nocache-filter.pod + +filter_LTLIBRARIES = nbdkit-nocache-filter.la + +nbdkit_nocache_filter_la_SOURCES = \ + nocache.c \ + $(top_srcdir)/include/nbdkit-filter.h + +nbdkit_nocache_filter_la_CPPFLAGS = \ + -I$(top_srcdir)/include \ + -I$(top_srcdir)/common/include +nbdkit_nocache_filter_la_CFLAGS = \ + $(WARNINGS_CFLAGS) +nbdkit_nocache_filter_la_LDFLAGS = \ + -module -avoid-version -shared \ + -Wl,--version-script=$(top_srcdir)/filters/filters.syms + +if HAVE_POD + +man_MANS = nbdkit-nocache-filter.1 +CLEANFILES += $(man_MANS) + +nbdkit-nocache-filter.1: nbdkit-nocache-filter.pod + $(PODWRAPPER) --section=1 --man $@ \ + --html $(top_builddir)/html/$@.html \ + $< + +endif HAVE_POD diff --git a/tests/test-eflags.sh b/tests/test-eflags.sh index eaaaae0..14a0099 100755 --- a/tests/test-eflags.sh +++ b/tests/test-eflags.sh @@ -51,6 +51,7 @@ if ! qemu-nbd --help | grep -sq -- --list; then fi files="eflags.out" +late_args= rm -f $files cleanup_fn rm -f $files @@ -70,7 +71,7 @@ SEND_CACHE=$(( 1 << 10 )) do_nbdkit () { - nbdkit -v -U - "$@" sh - --run 'qemu-nbd --list -k $unixsocket' | + nbdkit -v -U - "$@" sh - $late_args --run 'qemu-nbd --list -k $unixsocket' | grep -E "flags: 0x" | grep -Eoi '0x[a-f0-9]+' > eflags.out echo -n eflags=; cat eflags.out @@ -289,7 +290,7 @@ EOF #---------------------------------------------------------------------- # -r -# can_cache=true +# can_cache=native do_nbdkit -r <<'EOF' case "$1" in @@ -301,3 +302,19 @@ EOF [ $eflags -eq $(( HAS_FLAGS|READ_ONLY|SEND_DF|SEND_CACHE )) ] || fail "expected HAS_FLAGS|READ_ONLY|SEND_DF|SEND_CACHE" + +#---------------------------------------------------------------------- +# -r +# --filter=nocache cachemode=none +# can_cache=native + +late_args="cachemode=none" do_nbdkit -r --filter=nocache <<'EOF' +case "$1" in + get_size) echo 1M ;; + can_cache) echo "emulate" ;; + *) exit 2 ;; +esac +EOF + +[ $eflags -eq $(( HAS_FLAGS|READ_ONLY|SEND_DF )) ] || + fail "expected HAS_FLAGS|READ_ONLY|SEND_DF" -- 2.20.1