5:17 a.m.
Just noting this because I don't have time to look into it at the
moment. I suspect (but don't have any evidence for) that this might
be an issue in the cacheextents filter.
nbdkit-1.27.2-1.fc35.x86_64
libnbd-1.9.2-1.fc35.x86_64
nbdkit is running against a web server with this command line:
nbdkit --exit-with-parent --foreground --newstyle \
--pidfile /run/user/1000/v2vnbdkit.MHmNIZ/nbdkit1.pid \
--unix /tmp/v2v.wxXvoj/in0 \
--threads 16 \
--selinux-label system_u:object_r:svirt_socket_t:s0 \
-D nbdkit.backend.datapath=0 \
--exportname / --verbose \
--filter cow \
--filter cacheextents \
--filter retry \
curl timeout=2000 \
cookie-script=/tmp/v2vcse4d3ca.sh \
cookie-script-renew=1500 \
sslverify=false \
url=https://[redacted]
nbdinfo --map command failed:
$ nbdinfo --map nbd+unix:///?socket=in0
nbdinfo: nbd_block_status: block-status: command failed: Transport endpoint is not
connected
Looking at the nbdkit log:
nbdkit: debug: accepted connection
nbdkit: curl[7]: debug: cow: preconnect
nbdkit: curl[7]: debug: cacheextents: preconnect
nbdkit: curl[7]: debug: retry: preconnect
nbdkit: curl[7]: debug: curl: preconnect
nbdkit: curl[7]: debug: newstyle negotiation: flags: global 0x3
nbdkit: curl[7]: debug: newstyle negotiation: client flags: 0x3
nbdkit: curl[7]: debug: newstyle negotiation: NBD_OPT_STRUCTURED_REPLY: client requested
structured replies
nbdkit: curl[7]: debug: newstyle negotiation: NBD_OPT_SET_META_CONTEXT: client requested
export ''
nbdkit: curl[7]: debug: newstyle negotiation: NBD_OPT_SET_META_CONTEXT: set count: 1
nbdkit: curl[7]: debug: newstyle negotiation: NBD_OPT_SET_META_CONTEXT: set
base:allocation
nbdkit: curl[7]: debug: newstyle negotiation: NBD_OPT_SET_META_CONTEXT: replying with
base:allocation id 1
nbdkit: curl[7]: debug: newstyle negotiation: NBD_OPT_SET_META_CONTEXT: reply complete
nbdkit: curl[7]: debug: newstyle negotiation: NBD_OPT_GO: client requested export
''
nbdkit: curl[7]: debug: cow: open readonly=0 exportname="" tls=0
nbdkit: curl[7]: debug: cow: default_export readonly=0 tls=0
nbdkit: curl[7]: debug: cacheextents: default_export readonly=0 tls=0
nbdkit: curl[7]: debug: retry: default_export readonly=0 tls=0
nbdkit: curl[7]: debug: curl: default_export readonly=0 tls=0
nbdkit: curl[7]: debug: cacheextents: open readonly=1 exportname="" tls=0
nbdkit: curl[7]: debug: cacheextents: default_export readonly=1 tls=0
nbdkit: curl[7]: debug: retry: open readonly=1 exportname="" tls=0
nbdkit: curl[7]: debug: retry: default_export readonly=1 tls=0
nbdkit: curl[7]: debug: curl: open readonly=1 exportname="" tls=0
nbdkit: curl[7]: debug: curl: default_export readonly=1 tls=0
nbdkit: curl[7]: debug: content length: 12884901888
nbdkit: curl[7]: debug: accept range supported (for HTTP/HTTPS)
nbdkit: curl[7]: debug: curl: open returned handle 0x7ff9840012b0
nbdkit: curl[7]: debug: retry: open returned handle 0x7ff98400c370
nbdkit: curl[7]: debug: cacheextents: open returned handle 0x7ff992ffd4c0
nbdkit: curl[7]: debug: cow: open returned handle 0x7ff992ffd4c0
nbdkit: curl[7]: debug: curl: prepare readonly=1
nbdkit: curl[7]: debug: retry: prepare readonly=1
nbdkit: curl[7]: debug: cacheextents: prepare readonly=1
nbdkit: curl[7]: debug: cow: prepare readonly=0
nbdkit: curl[7]: debug: cacheextents: get_size
nbdkit: curl[7]: debug: retry: get_size
nbdkit: curl[7]: debug: curl: get_size
nbdkit: curl[7]: debug: cow: underlying file size: 12884901888
nbdkit: curl[7]: debug: bitmap resized to 786432 bytes
nbdkit: curl[7]: debug: cow: get_size
nbdkit: curl[7]: debug: cow: underlying file size: 12884901888
nbdkit: curl[7]: debug: bitmap resized to 786432 bytes
nbdkit: curl[7]: debug: cow: can_write
nbdkit: curl[7]: debug: cow: can_zero
nbdkit: curl[7]: debug: cacheextents: can_zero
nbdkit: curl[7]: debug: cow: can_fast_zero
nbdkit: curl[7]: debug: cow: can_trim
nbdkit: curl[7]: debug: cow: can_fua
nbdkit: curl[7]: debug: cow: can_flush
nbdkit: curl[7]: debug: cow: is_rotational
nbdkit: curl[7]: debug: cacheextents: is_rotational
nbdkit: curl[7]: debug: retry: is_rotational
nbdkit: curl[7]: debug: curl: is_rotational
nbdkit: curl[7]: debug: cow: can_multi_conn
nbdkit: curl[7]: debug: cow: can_cache
nbdkit: curl[7]: debug: cacheextents: can_cache
nbdkit: curl[7]: debug: retry: can_cache
nbdkit: curl[7]: debug: curl: can_cache
nbdkit: curl[7]: debug: cow: can_extents
nbdkit: curl[7]: debug: newstyle negotiation: flags: export 0x5ad
nbdkit: curl[7]: debug: newstyle negotiation: NBD_OPT_GO: ignoring NBD_INFO_* request 3
(NBD_INFO_BLOCK_SIZE)
nbdkit: curl[7]: debug: handshake complete, processing requests serially
nbdkit: curl[7]: debug: cacheextents: can_extents
nbdkit: curl[7]: debug: retry: can_extents
nbdkit: curl[7]: debug: curl: can_extents
nbdkit: cow.c:591: cow_extents: Assertion `count > 0' failed.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines. Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
5:30 a.m.
On Sat, Jul 24, 2021 at 11:17:43AM +0100, Richard W.M. Jones wrote:
Just noting this because I don't have time to look into it at
the
moment. I suspect (but don't have any evidence for) that this might
be an issue in the cacheextents filter.
I was wrong about cacheextents.
Here's the simplest reproducer I can come up with:
$ ./nbdkit curl
https://download.fedoraproject.org/pub/fedora/linux/releases/34/Cloud/x86...
--filter=cow --filter=xz --run 'nbdinfo --map $uri'
nbdkit: cow.c:591: cow_extents: Assertion `count > 0' failed.
It also happens if I download that cloud image and use the file plugin:
$ ./nbdkit file Fedora-Cloud-Base-34-1.2.x86_64.raw.xz --filter=cow --filter=xz --run
'nbdinfo --map $uri'
nbdkit: cow.c:591: cow_extents: Assertion `count > 0' failed.
Weird that we've not seen this before ...
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine. Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/
5:40 a.m.
I wonder if this is an overflow:
$ ./nbdkit sparse-random 3G --filter=cow --run 'nbdinfo --map $uri'
# OK
$ ./nbdkit sparse-random 4G --filter=cow --run 'nbdinfo --map $uri'
nbdkit: cow.c:591: cow_extents: Assertion `count > 0' failed.
Any size >= 4G seems to cause the crash.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines. Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
6:45 a.m.
On Sat, Jul 24, 2021 at 11:40:49AM +0100, Richard W.M. Jones wrote:
I wonder if this is an overflow:
$ ./nbdkit sparse-random 3G --filter=cow --run 'nbdinfo --map $uri'
# OK
$ ./nbdkit sparse-random 4G --filter=cow --run 'nbdinfo --map $uri'
nbdkit: cow.c:591: cow_extents: Assertion `count > 0' failed.
This is OK on my machine as well.
Any size >= 4G seems to cause the crash.
What struck me with this is:
cow.c:cow_extents:
/* To make this easier, align the requested extents to whole blocks.
* Note that count is a 64 bit variable containing at most a 32 bit
* value so rounding up is safe here.
*/
I started debugging the cow filter and the values for 4G image I got
are:
count=4294966784; offset=0 // Pre-rounding
end=4294967296; offset=0; count=4294967296; blknum=0 // Post-rounding
It looks like it is not asking for the whole size, but size - 1 block.
That gets rounded to 2^32 and sparse-extents then reports:
sparse-random: extents count=4294963200 offset=0 req_one=0
sparse-random: extents count=4096 offset=4294963200 req_one=0
Which is 2^32 - 4096. I hope I am not confusing anyone with this, but
it seems weird to me.
Martin
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines. Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
6:46 a.m.
On Mon, Jul 26, 2021 at 01:45:19PM +0200, Martin Kletzander wrote:
On Sat, Jul 24, 2021 at 11:40:49AM +0100, Richard W.M. Jones wrote:
>I wonder if this is an overflow:
>
>$ ./nbdkit sparse-random 3G --filter=cow --run 'nbdinfo --map $uri'
># OK
>
>$ ./nbdkit sparse-random 4G --filter=cow --run 'nbdinfo --map $uri'
>nbdkit: cow.c:591: cow_extents: Assertion `count > 0' failed.
>
This is OK on my machine as well.
>Any size >= 4G seems to cause the crash.
>
What struck me with this is:
cow.c:cow_extents:
/* To make this easier, align the requested extents to whole blocks.
* Note that count is a 64 bit variable containing at most a 32 bit
* value so rounding up is safe here.
*/
I started debugging the cow filter and the values for 4G image I got
are:
count=4294966784; offset=0 // Pre-rounding
end=4294967296; offset=0; count=4294967296; blknum=0 // Post-rounding
It looks like it is not asking for the whole size, but size - 1 block.
That gets rounded to 2^32 and sparse-extents then reports:
sparse-random: extents count=4294963200 offset=0 req_one=0
sparse-random: extents count=4096 offset=4294963200 req_one=0
Which is 2^32 - 4096. I hope I am not confusing anyone with this, but
it seems weird to me.
OK, it works for me because while catching up with e-mails post-vacation
I did not yet notice (when replying here) that you already fixed it.
Sorry for the noise.
Martin
>Rich.
>
>--
>Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
>Read my programming and virtualization blog: http://rwmj.wordpress.com
>virt-p2v converts physical machines to virtual machines. Boot with a
>live CD or over the network (PXE) and turn machines into KVM guests.
>http://libguestfs.org/virt-v2v
>
1029
days inactive
1031
days old
4 comments
2 participants
participants (2)
-
Martin Kletzander -
Richard W.M. Jones