On Thu, Oct 23, 2025 at 03:15:09PM +0200, Arsen Arsenović wrote:
"Richard W.M. Jones" <rjones(a)redhat.com> writes:
> I wonder if you can grab
'copy/copy-allocated-destination-zero-asynch.sh.log'
> and/or 'copy/copy-allocated-destination-zero-synch.sh.log' from the
> failing tests?
Sure thing. The contents of the logs of the synchronous version are:
+ requires cmp --version
+ requires truncate --version
+ requires stat --version
+ requires test -r /dev/zero
+ inp=copy-allocated-destination-zero-synch.in
+ out=copy-allocated-destination-zero-synch.out
+ cleanup_fn rm -f copy-allocated-destination-zero-synch.in
copy-allocated-destination-zero-synch.out
+ local _hook=_cleanup_hook0
+ declare -ag _cleanup_hook0
+ eval '_cleanup_hook0=("$@")'
++ _cleanup_hook0=("$@")
+ rm -f copy-allocated-destination-zero-synch.in
copy-allocated-destination-zero-synch.out
+ truncate -s 33554432 copy-allocated-destination-zero-synch.out
+ truncate -r copy-allocated-destination-zero-synch.out
copy-allocated-destination-zero-synch.in
+ nbdcopy --allocated --destination-is-zero --synchronous --request-size=32K
copy-allocated-destination-zero-synch.in copy-allocated-destination-zero-synch.out
libnbd: debug: nbd1: nbd_create: allocated the handle (9134 bytes)
libnbd: debug: nbd1: nbd_is_uri: enter:
uri="copy-allocated-destination-zero-synch.in"
libnbd: debug: nbd1: nbd_is_uri: leave: ret=0
libnbd: debug: nbd1: nbd_close: closing handle
libnbd: debug: nbd2: nbd_create: allocated the handle (9134 bytes)
libnbd: debug: nbd2: nbd_is_uri: enter:
uri="copy-allocated-destination-zero-synch.out"
libnbd: debug: nbd2: nbd_is_uri: leave: ret=0
libnbd: debug: nbd2: nbd_close: closing handle
+ echo Output:
Output:
+ ls -lsh copy-allocated-destination-zero-synch.out
512 -rw-r--r-- 1 portage portage 32M Oct 23 13:49
copy-allocated-destination-zero-synch.out
++ stat -c %s copy-allocated-destination-zero-synch.out
+ size=33554432
++ stat -c %b copy-allocated-destination-zero-synch.out
+ balloc=1
++ stat -c %B copy-allocated-destination-zero-synch.out
+ bsize=512
+ alloc=512
+ '[' 33554432 -ne 33554432 ']'
+ '[' 512 -ne 33554432 ']'
+ echo
'/var/tmp/portage/sys-libs/libnbd-1.23.9/work/libnbd-1.23.9/copy/copy-allocated-destination-zero-synch.sh:
allocated size is not 32M'
/var/tmp/portage/sys-libs/libnbd-1.23.9/work/libnbd-1.23.9/copy/copy-allocated-destination-zero-synch.sh:
allocated size is not 32M
+ exit 1
+ _run_cleanup_hooks
+ local _status=1 _i
+ set +e
+ trap '' INT QUIT TERM EXIT ERR
+ echo
/var/tmp/portage/sys-libs/libnbd-1.23.9/work/libnbd-1.23.9/copy/copy-allocated-destination-zero-synch.sh:
run cleanup hooks: exit code 1
/var/tmp/portage/sys-libs/libnbd-1.23.9/work/libnbd-1.23.9/copy/copy-allocated-destination-zero-synch.sh:
run cleanup hooks: exit code 1
+ (( _i = 0 ))
+ (( _i < 1 ))
+ eval '"${_cleanup_hook0[@]}"'
++ rm -f copy-allocated-destination-zero-synch.in
copy-allocated-destination-zero-synch.out
+ (( ++_i ))
+ (( _i < 1 ))
+ exit 1
FAIL copy-allocated-destination-zero-synch.sh (exit status: 1)
I suspect you might find the following interesting:
$ truncate -s 33554432 copy-allocated-destination-zero-synch.out
$ truncate -r copy-allocated-destination-zero-synch.out
copy-allocated-destination-zero-synch.in
$ ls -lsh copy*
512 -rw-r--r-- 1 arsen arsen 32M Oct 23 15:10 copy-allocated-destination-zero-synch.in
512 -rw-r--r-- 1 arsen arsen 32M Oct 23 15:10
copy-allocated-destination-zero-synch.out
$ sudo /var/tmp/portage/sys-libs/libnbd-1.23.9/work/libnbd-1.23.9/copy/nbdcopy
--allocated --destination-is-zero --synchronous --request-size=32K
copy-allocated-destination-zero-synch.in copy-allocated-destination-zero-synch.out
$ ls -lsh copy*
512 -rw-r--r-- 1 arsen arsen 32M Oct 23 15:13 copy-allocated-destination-zero-synch.in
512 -rw-r--r-- 1 arsen arsen 32M Oct 23 15:13
copy-allocated-destination-zero-synch.out
(forgive the 'sudo', portage has a strict umask so I can't access that
path from my user)
On tmpfs, I get the following instead:
/tmp/foo$ ls -lsh *
0 -rw-r--r-- 1 arsen arsen 32M Oct 23 15:12 copy-allocated-destination-zero-synch.in
0 -rw-r--r-- 1 arsen arsen 32M Oct 23 15:12 copy-allocated-destination-zero-synch.out
... and, after running nbdcopy:
/tmp/foo$ ls -lsh *
0 -rw-r--r-- 1 arsen arsen 32M Oct 23 13:58 copy-allocated-destination-zero-synch.in
32M -rw-r--r-- 1 arsen arsen 32M Oct 23 13:59 copy-allocated-destination-zero-synch.out
I don't have OpenZFS to test, but what seems to be happening on
regular filesystems is we call truncate on the destination before
doing the copy:
https://gitlab.com/nbdkit/libnbd/-/blob/6182a590ced685694574b74884c9cec53...
which eventually calls fallocate with mode == 0 here:
https://gitlab.com/nbdkit/libnbd/-/blob/6182a590ced685694574b74884c9cec53...
Confirmed in strace:
2745865 ftruncate(4, 0) = 0
2745865 ftruncate(4, 33554432) = 0
2745865 fallocate(4, 0, 0, 33554432) = 0
In Linux this is documented as:
Allocating disk space
The default operation (i.e., mode is zero) of fallocate() allocates the
disk space within the range specified by offset and len. [...]
which is the behaviour that we want here, allocating the space. We
make no further writes to the output disk in this test, but end up
with a 32M fully allocated, zeroed disk (what we want).
ZoL is probably ignoring fallocate and not allocating the space. It
may be returning ENOTSUP / EOPNOTSUPP, which we deliberately ignore.
I'm fairly sure this is a libnbd bug however, since we shouldn't
ignore unsupported fallocate (if ZoL returns that), if the --allocated
flag is set.
Eric, any thoughts?
The code path that this pair of flags takes is very convoluted ...
Rich.
--
Richard Jones, Virtualization Group, Red Hat
http://people.redhat.com/~rjones
Read my programming and virtualization blog:
http://rwmj.wordpress.com
nbdkit - Flexible, fast NBD server with plugins
https://gitlab.com/nbdkit/nbdkit