Re: [Libguestfs] [PATCH nbdkit] New plugin: blkio [incomplete]
On Wed, Jan 04, 2023 at 06:14:34PM +0000, Richard W.M. Jones wrote:
> (3) It seems like some drivers require pre-allocated memory regions,
> and since some do that means we might as well implement this. It
> also seems like some drivers require file-backed pre-allocated
> memory regions, and so we might as well implement that too.
>
> However what is not clear: does memfd_create(2) always allocate
> sufficiently aligned memory regions, such that we never need to bother
> reading the mem-region-alignment property?
>
> I notice that the example:
> https://gitlab.com/libblkio/libblkio/-/blob/main/examples/blkio-copy.c
> just passes on this and calls blkio_alloc_mem_region(). Is that the
> safest and easiest thing to do which will always work?
So this seems to be the reverse of what I thought.
In nbdkit plugins ideally we'd like to avoid using bounce buffers if
possible. Also NBD requests can be up to (IIRC) 32M, although we can
hint to the client to limit them to some smaller number.
Since you're currently serializing requests, you may be able to
manually break up big requests into smaller requests that don't exceed
max-transfer, and then wait for the right number of completions with
blkioq_do_io().
If I call blkio_alloc_mem_region() unconditionally then it seems as
if
I always need to use this as a bounce buffer. Plus, is 32M too large
for a memory region allocated this way? The example allocates 128K.
There is no general limit to the size of memory regions. 32M should
always be fine. In particular, using different parts of a single big
memory region for many concurrent requests is a reasonable use case.
It seems like for drivers which _don't_ require pre-allocated
memory
regions then we should avoid calling blkio_alloc_mem_region which
would avoid bounce buffers.
Some guidance about this would be appreciated.
A possible approach is to make nbdkit always allocate page-aligned,
file-backed buffers, and attempt to blkio_map_mem_region() them at
init time in the libblkio plugin. Note that mapping memory regions
even for drivers that don't strictly need them can still improve
performance.
If this fails and the driver requires mapped memory regions, then one
could fall back to bounce buffering with buffers allocated by
blkio_alloc_mem_region(). But this probably won't happen in practice,
so just failing in this case is probably fine.
Alberto