4:22 p.m.
Hi Rich and list,
Thanks for your reply and the suggestions. I kept doing some more tests, this time between
two cloud servers in two parts of the world and was able to get the throughput up quite a
bit.
I am not sure this was the exact culprit, but with my original tests, so as to avoid
exposing the NBD tcp port to the entire world, I was doing it via SSH. However, when I
removed ssh from the picture (temporarily, just as a test), then it seemed to allow much
better throughput.
I wonder if using wireguard would be a way to avoid publicly exposing the nbd socket, but
still retain most of the performance. Either that, or maybe just doing TLS like you
suggested would be the way to go.
Sounds like in general I probably need to start using/testing with nbdkit, rather than
nbd-server/client. I will look into that next. I was testing this with nixos on the
server, and there is not yet a nixos module or package for nbdkit, so I was lazy and used
nbd-server.
Anyway, the fact that it worked at all over a connection with 175ms of latency was really
neat to see! From what I could tell, LUKs did not even cause much of a slowdown either.
On Friday, June 14th, 2024 at 10:12 AM, Richard W.M. Jones <rjones(a)redhat.com>
wrote:
> On Mon, Jun 10, 2024 at 08:38:02PM +0000, netpleb wrote:
>
> > Hi Richard et. al,
> >
> > I was attempting to ask this question via a comment on this post (https://
> > rwmj.wordpress.com/2022/05/14/nbdkit-now-supports-luks-encryption/) on your
> > wordpress site, but it was giving me some trouble, so I will ask it here:
>
>
> Sorry, at devconf.cz this week, so things are a little chaotic/busy ...
>
> > I hope this is not too naive of a question, but is nbdkit (or even
> > just nbd in general) a good fit for mounting and using a LUKs
> > encrypted block device over the internet?
>
>
> It makes it possible.
>
> Separate from the LUKS issue, whether it's a good idea to expose an
> NBD socket to the whole world is interesting. I would certainly be
> looking at TLS client certificates to try to control who can connect,
> and being careful to sandbox the machine running nbdkit.
>
> https://libguestfs.org/nbdkit-tls.1.html
>
> > Specifically, the client would encrypt the block device using luks
> > and thereby keep the luks secret key local like you did in your blog
> > post.
> >
> > I have done some rudimentary tests with nbd-client but have been
> > unable to achieve speeds faster than approximately 1.4
> > MB/sec. However, doing the same thing on my local network gave me
> > closer to 50 MB/sec. The latter seems usable, but the former seems
> > not, and I would like to better understand if/how nbd can be used
> > over the internet.
>
>
> I'm going to guess that round trips and/or the client not using
> multi-conn are killing your performance. Multi-conn is really
> essential when trying to get good performance when you have large
> latencies. nbd-client -connections parameter should let you control
> it.
>
> Another potential issue would be that the client is making lots of
> tiny requests and not coallescing. There are various filters
> including nbdkit-log-filter and nbdkit-stats-filter which let you view
> this.
>
> You could also play with nbdcopy, since it allows you to control all
> the parameters, like multi-conn, requests in flight, request size,
> etc.
>
> Rich.
>
> > Thank you in advance for your reply.
>
>
> --
> Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
> Read my programming and virtualization blog: http://rwmj.wordpress.com
> virt-builder quickly builds VMs from scratch
> http://libguestfs.org/virt-builder.1.html