Re: [Libguestfs] [PATCH nbdkit] Add new filter for rate-limiting connections.
On 3/5/19 4:38 AM, Richard W.M. Jones wrote:
---
filters/delay/nbdkit-delay-filter.pod | 4 +-
filters/rate/nbdkit-rate-filter.pod | 84 +++++++++
configure.ac | 2 +
filters/rate/bucket.h | 62 +++++++
filters/rate/bucket.c | 173 +++++++++++++++++++
filters/rate/rate.c | 235 ++++++++++++++++++++++++++
TODO | 9 +
filters/rate/Makefile.am | 64 +++++++
tests/Makefile.am | 6 +-
tests/test-rate.sh | 60 +++++++
10 files changed, 697 insertions(+), 2 deletions(-)
I see I was too slow in reviewing this before you pushed (that's okay,
overall it looks reasonable).
+
+=item nbdkit --filter=rate memory size=64M connection-rate=50K rate=1M
+
+Limit each connection to S<50 Kbps>. Additionally the total bandwidth
+across all connections to the server is limited to S<1 Mbps>.
If there are more than 40 clients, does this setup fairly service all of
them at a lower per-client rate?
+=head1 NOTES
+
+The rate filter approximates the bandwidth used by the NBD protocol on
+the wire. Some operations such as zeroing and trimming are
+effectively free (because only a tiny NBD message is sent over the
+network) and so do not count against the bandwidth limit. NBD and TCP
+protocol overhead is not included, so you may find that other tools
+such as L<tc(8)> and L<iptables(8)> give more accurate results.
+
+There are separate bandwidth limits for read and write (ie. upload and
+download to the server).
+
Is it worth mentioning that the blocksize filter can be used to ensure
smoother spreading of the bandwidth? (If a large client request is
broken into 128k max packets, that hits the network in a smoother
pattern than a single request).
+++ b/TODO
@@ -129,6 +129,15 @@ Suggestions for filters
* nbdkit-cache-filter should handle ENOSPC errors automatically by
reclaiming blocks from the cache
+nbdkit-rate-filter:
+
+* allow other kinds of traffic shaping such as VBR
+
+* limit traffic per client (ie. per IP address)
+
+* split large requests to avoid long, lumpy sleeps when request size
+ is much larger than rate limit
Can't you pair the blocksize filter in front of this one to accomplish
that already?
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
Attachments:
- signature.asc (application/pgp-signature — 488 bytes)