On Sat, May 21, 2022 at 01:21:11PM +0100, Nikolaus Rath wrote:
Hi,
How does the blocksize filter take into account writes that end-up overlapping due to read-modify-write cycles?
Specifically, suppose there are two non-overlapping writes handled by two different threads, that, due to blocksize requirements, overlap when expanded. I think there is a risk that one thread may partially undo the work of the other here.
Looking at the code, it seems that writes of unaligned heads and tails are protected with a global lock., but writes of aligned data can occur concurrently.
I agree. Assuming the underlying plugin is NBDKIT_THREAD_MODEL_PARALLEL and no other filters impose thread model limits, the blocksize filter does not limit the thread model, so the thread model of nbdkit would also be NBDKIT_THREAD_MODEL_PARALLEL. That means that two writes either on different connections or pipelined on the same connection could happen at the same time. “blocksize_pwrite” would be called concurrently for the two requests.
However, does this not miss the case where there is one unaligned write that overlaps with an aligned one?
For example, with blocksize 10, we could have:
Thread 1: receives write request for offset=0, size=10 Thread 2: receives write request for offset=4, size=16 Thread 1: acquires lock, reads bytes 0-4 Thread 2: does aligned write (no locking needed), writes bytes 0-10 Thread 1: writes bytes 0-10, overwriting data from Thread 2
I believe this analysis is correct. (CC'd to Eric who knows a lot more about this.) However I don't think it's a bug. If a client doesn't want writes to squash each other, then it shouldn't send overlapping requests. I bet the same thing happens with an SSD. NBD_CMD_FLAG_FUA is provided for clients that wish to ensure that a write has been committed before sending another request. Do you have an example of a client which sends overlapping requests and depends on particular behaviour of the server? You may be able to get it to work by using nbdkit-noparallel-filter which can be used to serialize nbdkit. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org