--- filters/delay/nbdkit-delay-filter.pod | 4 +- filters/rate/nbdkit-rate-filter.pod | 84 +++++++++ configure.ac | 2 + filters/rate/bucket.h | 62 +++++++ filters/rate/bucket.c | 173 +++++++++++++++++++ filters/rate/rate.c | 235 ++++++++++++++++++++++++++ TODO | 9 + filters/rate/Makefile.am | 64 +++++++ tests/Makefile.am | 6 +- tests/test-rate.sh | 60 +++++++ 10 files changed, 697 insertions(+), 2 deletions(-) diff --git a/filters/delay/nbdkit-delay-filter.pod b/filters/delay/nbdkit-delay-filter.pod index 7009a8c..c2eb172 100644 --- a/filters/delay/nbdkit-delay-filter.pod +++ b/filters/delay/nbdkit-delay-filter.pod @@ -17,6 +17,7 @@ nbdkit-delay-filter - nbdkit delay filter C<nbdkit-delay-filter> is a filter that delays read and write requests by some seconds or milliseconds. This is used to simulate a slow or remote server, or to test certain kinds of race conditions in Linux. +To limit server bandwidth use L<nbdkit-rate-filter(1)> instead. =head1 EXAMPLES @@ -74,7 +75,8 @@ milliseconds. =head1 SEE ALSO L<nbdkit(1)>, -L<nbdkit-filter(3)>. +L<nbdkit-filter(3)>, +L<nbdkit-rate-filter(1)>. =head1 AUTHORS diff --git a/filters/rate/nbdkit-rate-filter.pod b/filters/rate/nbdkit-rate-filter.pod new file mode 100644 index 0000000..795c7b8 --- /dev/null +++ b/filters/rate/nbdkit-rate-filter.pod @@ -0,0 +1,84 @@ +=head1 NAME + +nbdkit-rate-filter - limit bandwidth by connection or server + +=head1 SYNOPSIS + + nbdkit --filter=rate PLUGIN [PLUGIN-ARGS...] + [rate=BITSPERSEC] + [connection-rate=BITSPERSEC] + +=head1 DESCRIPTION + +C<nbdkit-rate-filter> is a filter that limits the bandwidth that can +be used by the server. Limits can be applied per connection and/or +for the server as a whole. + +=head1 EXAMPLES + +=over 4 + +=item nbdkit --filter=rate memory size=64M rate=1M + +Create a 64M RAM disk and limit server bandwidth as a whole to a +maximum of S<1 Mbps> (megabit per second). + +=item nbdkit --filter=rate memory size=64M connection-rate=50K + +Limit each connection to S<50 Kbps> (kilobits per second). However as +there is no limit to the number of simultaneous connections this does +not limit overall server bandwidth. + +=item nbdkit --filter=rate memory size=64M connection-rate=50K rate=1M + +Limit each connection to S<50 Kbps>. Additionally the total bandwidth +across all connections to the server is limited to S<1 Mbps>. + +=back + +=head1 PARAMETERS + +=over 4 + +=item B<connection-rate=>BITSPERSEC + +Limit each connection to C<BITSPERSEC>. + +=item B<rate=>BITSPERSEC + +Limit total bandwidth across all connections to C<BITSPERSEC>. + +=back + +You can specify C<rate> and C<connection-rate> on their own or +together. If you specify neither then the filter is turned off. + +C<BITSPERSEC> can be specified as a simple number, or you can use a +number followed by C<K>, C<M> etc to mean kilobits, megabits and so +on. + +=head1 NOTES + +The rate filter approximates the bandwidth used by the NBD protocol on +the wire. Some operations such as zeroing and trimming are +effectively free (because only a tiny NBD message is sent over the +network) and so do not count against the bandwidth limit. NBD and TCP +protocol overhead is not included, so you may find that other tools +such as L<tc(8)> and L<iptables(8)> give more accurate results. + +There are separate bandwidth limits for read and write (ie. upload and +download to the server). + +=head1 SEE ALSO + +L<nbdkit(1)>, +L<nbdkit-delay-filter(1)>. +L<nbdkit-filter(3)>. + +=head1 AUTHORS + +Richard W.M. Jones + +=head1 COPYRIGHT + +Copyright (C) 2019 Red Hat Inc. diff --git a/configure.ac b/configure.ac index 9e7e5ca..467d48f 100644 --- a/configure.ac +++ b/configure.ac @@ -819,6 +819,7 @@ filters="\ nozero \ offset \ partition \ + rate \ truncate \ xz \ " @@ -889,6 +890,7 @@ AC_CONFIG_FILES([Makefile filters/nozero/Makefile filters/offset/Makefile filters/partition/Makefile + filters/rate/Makefile filters/truncate/Makefile filters/xz/Makefile fuzzing/Makefile diff --git a/filters/rate/bucket.h b/filters/rate/bucket.h new file mode 100644 index 0000000..0e8c8f7 --- /dev/null +++ b/filters/rate/bucket.h @@ -0,0 +1,62 @@ +/* nbdkit + * Copyright (C) 2018-2019 Red Hat Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * * Neither the name of Red Hat nor the names of its contributors may be + * used to endorse or promote products derived from this software without + * specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, + * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A + * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR + * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF + * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND + * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, + * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT + * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#ifndef NBDKIT_BUCKET_H +#define NBDKIT_BUCKET_H + +#include <stdint.h> +#include <time.h> +#include <sys/time.h> + +/* A token bucket. */ +struct bucket { + uint64_t rate; /* Fill rate. 0 = no limit set. */ + uint64_t capacity; /* Maximum capacity of the bucket. */ + uint64_t level; /* How full is the bucket now? */ + struct timeval tv; /* Last time we updated the level. */ +}; + +/* Initialize the bucket structure. Capacity is expressed in + * rate-equivalent seconds. + */ +extern void init_bucket (struct bucket *bucket, + uint64_t rate, double capacity); + +/* Take up to N tokens from the bucket. If we couldn't take N tokens, + * then *N is updated with the number of remaining tokens that we are + * waiting for. Returns the length of time we should sleep (which may + * be 0). + */ +extern struct timespec bucket_run (struct bucket *bucket, uint64_t *n); + +#endif /* NBDKIT_BUCKET_H */ diff --git a/filters/rate/bucket.c b/filters/rate/bucket.c new file mode 100644 index 0000000..ccd657e --- /dev/null +++ b/filters/rate/bucket.c @@ -0,0 +1,173 @@ +/* nbdkit + * Copyright (C) 2018-2019 Red Hat Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * * Neither the name of Red Hat nor the names of its contributors may be + * used to endorse or promote products derived from this software without + * specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, + * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A + * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR + * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF + * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND + * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, + * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT + * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +/* This filter is implemented using a Token Bucket + * (https://en.wikipedia.org/wiki/Token_bucket). There are two + * buckets per connection (one each for reading and writing) and two + * global buckets (also for reading and writing). + * + * We add tokens at the desired rate (the per-connection rate for the + * connection buckets, and the global rate for the global buckets). + * Note that we don't actually keep the buckets updated in real time + * because as a filter we are called asynchronously. Instead for each + * bucket we store the last time we were called and add the + * appropriate number of tokens when we are called next. + * + * The bucket capacity controls the burstiness allowed. This is + * hard-coded at the moment but could be configurable. All buckets + * start off full. + * + * When a packet is to be read or written, if there are sufficient + * tokens in the bucket then the packet may be immediately passed + * through to the underlying plugin. The number of bits used is + * deducted from the appropriate per-connection and global bucket. + * + * If there are insufficient tokens then the packet must be delayed. + * This is done by inserting a sleep which has an estimated length + * that is long enough based on the rate at which enough tokens will + * replenish the bucket to allow the packet to be sent next time. + */ + +#include <config.h> + +#include <stdio.h> +#include <stdlib.h> +#include <stdint.h> +#include <inttypes.h> +#include <string.h> +#include <time.h> +#include <sys/time.h> + +#include <nbdkit-filter.h> + +#include "minmax.h" + +#include "bucket.h" + +int rate_debug_bucket; /* -D rate.bucket=1 */ + +void +init_bucket (struct bucket *bucket, uint64_t rate, double capacity) +{ + bucket->rate = rate; + + /* Capacity is expressed in seconds, but we want to know the + * capacity in tokens, so multiply by the rate to get this. + */ + bucket->capacity = rate * capacity; + + /* Buckets start off full. */ + bucket->level = capacity; + + gettimeofday (&bucket->tv, NULL); +} + +/* Return the number of microseconds in y - x. */ +static int64_t +tvdiff (const struct timeval *x, const struct timeval *y) +{ + int64_t usec; + + usec = (y->tv_sec - x->tv_sec) * 1000000; + usec += y->tv_usec - x->tv_usec; + return usec; +} + +struct timespec +bucket_run (struct bucket *bucket, uint64_t *n) +{ + struct timespec ts; + struct timeval now; + int64_t usec; + uint64_t add, nsec; + + /* rate == 0 is a special case meaning that there is no limit being + * enforced. + */ + if (bucket->rate == 0) { + *n = 0; + ts.tv_sec = ts.tv_nsec = 0; + return ts; + } + + gettimeofday (&now, NULL); + + /* Can we add tokens to the bucket? Work out how much time has + * elapsed since we last did this. + */ + usec = tvdiff (&bucket->tv, &now); + if (usec < 0) /* Maybe happens if system time not monotonic? */ + usec = 0; + + add = bucket->rate * usec / 1000000; + add = MIN (add, bucket->capacity - bucket->level); + if (rate_debug_bucket) + nbdkit_debug ("bucket %p: adding %" PRIu64 " tokens, new level %" PRIu64, + bucket, add, bucket->level + add); + bucket->level += add; + bucket->tv = now; + + /* Can we deduct N tokens from the bucket? If yes then we're good, + * we can return with TS = 0 which means the caller won't sleep. + */ + if (bucket->level >= *n) { + if (rate_debug_bucket) + nbdkit_debug ("bucket %p: deducting %" PRIu64 " tokens, no sleep", + bucket, *n); + bucket->level -= *n; + *n = 0; + ts.tv_sec = ts.tv_nsec = 0; + return ts; + } + + if (rate_debug_bucket) + nbdkit_debug ("bucket %p: deducting %" PRIu64 " tokens, empty, " + "need another %" PRIu64 " tokens", + bucket, bucket->level, *n - bucket->level); + + *n -= bucket->level; + bucket->level = 0; + + /* Now we need to estimate how long it will take to add N tokens to + * the bucket, which is how long the caller must sleep for. + */ + nsec = 1000000000 * (*n) / bucket->rate; + ts.tv_sec = nsec / 1000000000; + ts.tv_nsec = nsec % 1000000000; + + if (rate_debug_bucket) + nbdkit_debug ("bucket %p: sleeping for %.1f seconds", bucket, + nsec / 1000000000.); + + return ts; +} diff --git a/filters/rate/rate.c b/filters/rate/rate.c new file mode 100644 index 0000000..6827e16 --- /dev/null +++ b/filters/rate/rate.c @@ -0,0 +1,235 @@ +/* nbdkit + * Copyright (C) 2018-2019 Red Hat Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * * Neither the name of Red Hat nor the names of its contributors may be + * used to endorse or promote products derived from this software without + * specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, + * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A + * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR + * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF + * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND + * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, + * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT + * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +/* For a note on the implementation of this filter, see bucket.c. */ + +#include <config.h> + +#include <stdio.h> +#include <stdlib.h> +#include <stdint.h> +#include <string.h> +#include <time.h> +#include <sys/time.h> + +#include <pthread.h> + +#include <nbdkit-filter.h> + +#include "bucket.h" + +#define THREAD_MODEL NBDKIT_THREAD_MODEL_PARALLEL + +/* Per-connection and global limit, both in bits per second, with zero + * meaning not set / not enforced. + */ +static uint64_t connection_rate = 0; +static uint64_t rate = 0; + +/* Bucket capacity controls the burst rate. It is expressed as the + * length of time in "rate-equivalent seconds" that the client can + * burst for after a period of inactivity. This could be adjustable + * in future. + */ +#define BUCKET_CAPACITY 2.0 + +/* Global read and write buckets. */ +static struct bucket read_bucket; +static pthread_mutex_t read_bucket_lock = PTHREAD_MUTEX_INITIALIZER; +static struct bucket write_bucket; +static pthread_mutex_t write_bucket_lock = PTHREAD_MUTEX_INITIALIZER; + +/* Per-connection handle. */ +struct rate_handle { + /* Per-connection read and write buckets. */ + struct bucket read_bucket; + pthread_mutex_t read_bucket_lock; + struct bucket write_bucket; + pthread_mutex_t write_bucket_lock; +}; + +/* Called for each key=value passed on the command line. */ +static int +rate_config (nbdkit_next_config *next, void *nxdata, + const char *key, const char *value) +{ + if (strcmp (key, "rate") == 0) { + if (rate > 0) { + nbdkit_error ("rate set twice on the command line"); + return -1; + } + rate = nbdkit_parse_size (value); + if (rate == -1) + return -1; + if (rate == 0) { + nbdkit_error ("rate cannot be set to 0"); + return -1; + } + return 0; + } + else if (strcmp (key, "connection-rate") == 0) { + if (connection_rate > 0) { + nbdkit_error ("connection-rate set twice on the command line"); + return -1; + } + connection_rate = nbdkit_parse_size (value); + if (connection_rate == -1) + return -1; + if (connection_rate == 0) { + nbdkit_error ("connection-rate cannot be set to 0"); + return -1; + } + return 0; + } + else + return next (nxdata, key, value); +} + +static int +rate_config_complete (nbdkit_next_config_complete *next, void *nxdata) +{ + /* Initialize the global buckets. */ + init_bucket (&read_bucket, rate, BUCKET_CAPACITY); + init_bucket (&write_bucket, rate, BUCKET_CAPACITY); + + return next (nxdata); +} + +#define rate_config_help \ + "rate=BITSPERSEC Limit total bandwidth.\n" \ + "connection-rate=BITSPERSEC Limit per-connection bandwidth." + +/* Create the per-connection handle. */ +static void * +rate_open (nbdkit_next_open *next, void *nxdata, int readonly) +{ + struct rate_handle *h; + + if (next (nxdata, readonly) == -1) + return NULL; + + h = malloc (sizeof *h); + if (h == NULL) { + nbdkit_error ("malloc: %m"); + return NULL; + } + + init_bucket (&h->read_bucket, connection_rate, BUCKET_CAPACITY); + init_bucket (&h->write_bucket, connection_rate, BUCKET_CAPACITY); + pthread_mutex_init (&h->read_bucket_lock, NULL); + pthread_mutex_init (&h->write_bucket_lock, NULL); + + return h; +} + +/* Free up the per-connection handle. */ +static void +rate_close (void *handle) +{ + struct rate_handle *h = handle; + + pthread_mutex_destroy (&h->read_bucket_lock); + pthread_mutex_destroy (&h->write_bucket_lock); + free (h); +} + +static inline void +maybe_sleep (struct bucket *bucket, pthread_mutex_t *lock, uint32_t count) +{ + struct timespec ts; + uint64_t bits; + + /* Count is in bytes, but we rate limit using bits. We could + * multiply this by 10 to include start/stop but let's not + * second-guess the transport layers underneath. + */ + bits = count * UINT64_C(8); + + /* Run the token bucket algorithm. It will set ts to the sleep time + * required, if any, and update bits with the remaining number of + * bits we have to wait for. + */ + again: + pthread_mutex_lock (lock); + ts = bucket_run (bucket, &bits); + pthread_mutex_unlock (lock); + + if (ts.tv_sec > 0 || ts.tv_nsec > 0) { + nanosleep (&ts, NULL); + goto again; + } +} + +/* Read data. */ +static int +rate_pread (struct nbdkit_next_ops *next_ops, void *nxdata, + void *handle, void *buf, uint32_t count, uint64_t offset, + uint32_t flags, int *err) +{ + struct rate_handle *h = handle; + + maybe_sleep (&read_bucket, &read_bucket_lock, count); + maybe_sleep (&h->read_bucket, &h->read_bucket_lock, count); + + return next_ops->pread (nxdata, buf, count, offset, flags, err); +} + +/* Write data. */ +static int +rate_pwrite (struct nbdkit_next_ops *next_ops, void *nxdata, + void *handle, + const void *buf, uint32_t count, uint64_t offset, uint32_t flags, + int *err) +{ + struct rate_handle *h = handle; + + maybe_sleep (&write_bucket, &write_bucket_lock, count); + maybe_sleep (&h->write_bucket, &h->write_bucket_lock, count); + + return next_ops->pwrite (nxdata, buf, count, offset, flags, err); +} + +static struct nbdkit_filter filter = { + .name = "rate", + .longname = "nbdkit rate filter", + .version = PACKAGE_VERSION, + .config = rate_config, + .config_complete = rate_config_complete, + .config_help = rate_config_help, + .open = rate_open, + .close = rate_close, + .pread = rate_pread, + .pwrite = rate_pwrite, +}; + +NBDKIT_REGISTER_FILTER(filter) diff --git a/TODO b/TODO index 59590a1..b589127 100644 --- a/TODO +++ b/TODO @@ -129,6 +129,15 @@ Suggestions for filters * nbdkit-cache-filter should handle ENOSPC errors automatically by reclaiming blocks from the cache +nbdkit-rate-filter: + +* allow other kinds of traffic shaping such as VBR + +* limit traffic per client (ie. per IP address) + +* split large requests to avoid long, lumpy sleeps when request size + is much larger than rate limit + Filters for security -------------------- diff --git a/filters/rate/Makefile.am b/filters/rate/Makefile.am new file mode 100644 index 0000000..c39aa01 --- /dev/null +++ b/filters/rate/Makefile.am @@ -0,0 +1,64 @@ +# nbdkit +# Copyright (C) 2018-2019 Red Hat Inc. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are +# met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# * Neither the name of Red Hat nor the names of its contributors may be +# used to endorse or promote products derived from this software without +# specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, +# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A +# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR +# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF +# USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND +# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT +# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# SUCH DAMAGE. + +include $(top_srcdir)/common-rules.mk + +EXTRA_DIST = nbdkit-rate-filter.pod + +filter_LTLIBRARIES = nbdkit-rate-filter.la + +nbdkit_rate_filter_la_SOURCES = \ + bucket.c \ + bucket.h \ + rate.c \ + $(top_srcdir)/include/nbdkit-filter.h + +nbdkit_rate_filter_la_CPPFLAGS = \ + -I$(top_srcdir)/include \ + -I$(top_srcdir)/common/include +nbdkit_rate_filter_la_CFLAGS = \ + $(WARNINGS_CFLAGS) +nbdkit_rate_filter_la_LDFLAGS = \ + -module -avoid-version -shared \ + -Wl,--version-script=$(top_srcdir)/filters/filters.syms + +if HAVE_POD + +man_MANS = nbdkit-rate-filter.1 +CLEANFILES += $(man_MANS) + +nbdkit-rate-filter.1: nbdkit-rate-filter.pod + $(PODWRAPPER) --section=1 --man $@ \ + --html $(top_builddir)/html/$@.html \ + $< + +endif HAVE_POD diff --git a/tests/Makefile.am b/tests/Makefile.am index 3992d9b..d1e6f0e 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -1,5 +1,5 @@ # nbdkit -# Copyright (C) 2013-2018 Red Hat Inc. +# Copyright (C) 2013-2019 Red Hat Inc. # All rights reserved. # # Redistribution and use in source and binary forms, with or without @@ -97,6 +97,7 @@ EXTRA_DIST = \ test-python-exception.sh \ test.pl \ test.py \ + test-rate.sh \ test.rb \ test.tcl \ test-shebang-perl.sh \ @@ -810,6 +811,9 @@ if HAVE_GUESTFISH TESTS += test-partition2.sh endif HAVE_GUESTFISH +# rate filter test. +TESTS += test-rate.sh + # truncate filter tests. TESTS += \ test-truncate1.sh \ diff --git a/tests/test-rate.sh b/tests/test-rate.sh new file mode 100755 index 0000000..010ef19 --- /dev/null +++ b/tests/test-rate.sh @@ -0,0 +1,60 @@ +#!/usr/bin/env bash +# nbdkit +# Copyright (C) 2018-2019 Red Hat Inc. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are +# met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# * Neither the name of Red Hat nor the names of its contributors may be +# used to endorse or promote products derived from this software without +# specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, +# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A +# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR +# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF +# USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND +# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT +# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# SUCH DAMAGE. + +source ./functions.sh +set -e +set -x + +requires qemu-img --version + +files="rate.img rate.time rate.err" +rm -f $files +cleanup_fn rm -f $files + +# This should take no less than 20 seconds to run: +# (8 * 25 * 1024 * 1024) / (10 * 1024 * 1024) = 20 + +# We are using the bash time builtin, so setting TIMEFORMAT will +# control the output format of the time builtin. For strange use of +# { ; } here, see: https://stackoverflow.com/a/13356654 +set +x +{ TIMEFORMAT="%0R" ; time nbdkit --filter=rate memory size=25M rate=10M --run 'qemu-img convert -p $nbd rate.img' 2>rate.err ; } 2>rate.time +set -x + +cat rate.err ||: + +seconds="$( cat rate.time )" +if [ "$seconds" -lt 20 ]; then + echo "$0: rate filter failed: command took $seconds seconds, expected > 20" + exit 1 +fi -- 2.20.1

On Fri, Mar 08, 2019 at 06:38:10AM -0600, Eric Blake wrote: Probably it doesn't. It just does the simple/stupid thing of checking the global bucket first and the per-connection bucket second. Presumably in the overloaded case the global bucket would be heavily contended, meaning that it would depend on how threads are scheduled, but I would expect to see some threads starving. Yes, I've added a commit to mention this: https://github.com/libguestfs/nbdkit/commit/b165ff3ef6c20f62108bafa2834cd... Yes, but that involves the user taking deliberate action. It should also be possible to split up requests transparently in the rate filter. Note that the filter has unique information here because it knows how long it will sleep for given the request block size and rate, which for the user of nbdkit involves a complex calculation or tuning. However by mentioning the blocksize filter in the man page (previous point) that does give the user the option to do this. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/