[PATCH nbdkit v2 0/4] common/allocators: Always align mlock buffer
by Richard W.M. Jones
Earlier patch was reviewed here:
https://listman.redhat.com/archives/libguestfs/2022-January/msg00172.html
For this series I pulled out two smaller changes into separate commits
(patches 1 & 2).
I then decided to modify the vector library to add a new
<vector>_reserve_page_aligned function which always allocates
page-aligned memory (patch 3). Currently this is using
posix_memalign, maybe in future using mmap since there seems to be
some dispute about whether calling mlock on anonymous memory is safe.
This makes the change to common/allocators/malloc.c quite
straightforward (patch 4), at the cost of making the change to the
vector library rather headache-inducing. At least it is confined to a
new function and doesn't affect any existing callers!
Rich.
2 years, 9 months
- 2
- 12
[PATCH libnbd] copy: Implement destination preferred block size
by Richard W.M. Jones
[NB: I think this is a failed attempt, so shoudn't go upstream, and
doesn't need to be reviewed.]
When nbdcopy writes to an NBD server it ignores the server's
minimum/preferred block size. This hasn't caused a problem til now,
but it turns out that in at least one case we care about (writing to
qemu-nbd backed by a compressed qcow2 file) we must obey the minimum &
preferred block size of 64K.
For the longer story on that see this thread:
https://lists.nongnu.org/archive/html/qemu-devel/2022-01/threads.html#06108
This patch attempts to fix this. The uncontroversial part of this
patch adds a global "blocksize" variable which is the destination
preferred block size.
The tricky part of this patch tries to ensure that writes to the
destination obey this block size constraint.
Since nbdcopy is driven by the extent map read from the source, the
theory behind this implementation is that we read the extent map and
then try to "adjust" it so that extents which are not aligned to the
block size grow, shrink or are deleted. It proved to be very
difficult to get that right, but you can see the implementation in the
new function "adjust_extents_for_blocksize".
Unfortunately not only is this difficult to implement, but the theory
is wrong. Read requests are actually split up into smaller
subrequests on write (look for "copy_subcommand" in
multi-thread-copying.c). So this doesn't really solve the problem.
So I think in my second version I will look at adjusting the NBD
destination driver (nbd-ops.c) directly so that it turns unaligned
writes into buffered read/modify/write operations (which can be
optimized quite a lot because we understand the write pattern and know
that the main program doesn't go backwards within blocks).
Rich.
2 years, 9 months
- 3
- 14