On 11/21/18 4:39 AM, Richard W.M. Jones wrote:
> On Wed, Nov 21, 2018 at 09:25:05AM +0000, Richard W.M. Jones wrote:
>> nbdkit: file.2: error: invalid request: offset and count are out of range: offset=196608 count=512
>
> Actually what happens even more precisely is that the underlying file
> is not a multiple of 512 bytes (196624 ≡ 16 mod 512).
>
> qemu-img (the client) issues a request for the final "sector" of the
> file which goes beyond the end of the file.

I'm still hoping to fix that bug in qemu before the 3.1 release, but
that date is getting close :)

>
> You can "fix" this by using the truncate filter
> (https://github.com/libguestfs/nbdkit/blob/master/filters/truncate/nbdkit-truncate-filter.pod)
> telling it to round up the size to the next multiple of 512 bytes:
>
> $ nbdkit --filter=truncate file file=disk -f -v round-up=512
>
> and this allows qemu-img to work:

Yes, this is the right workaround for now.

>
> $ qemu-img info nbd://localhost
> image: nbd://localhost:10809
> file format: qcow2

Invoking qemu-img without '-f $format' means that qemu-img has to probe
the image to guess its format. Here, the probe succeeded, but in general
this is insecure (if the file was 'raw', but the probe returned anything
else, then a malicious guest could have tried to write metadata into the
initial sector of their disk to try and get qemu to perform something on
the guest's behalf due to the incorrect probe).  A more robust solution
pairs either:

qemu-nbd -f raw file.qcow2   <=>   qemu-img info -f qcow2 nbd://localhost
nbdkit file file.qcow2   <=>   qemu-img info -f qcow2 nbd://localhost

where NBD is exposing the qcow2 metadata over the network (whether via
qemu-nbd or nbdkit), and the client HAS to be aware of qcow2, or:

qemu-nbd -f qcow2 file.qcow2   <=>   qemu-img info -f raw nbd://localhost
qemu-nbd -f qcow2 file.qcow2   <=>   any other nbd client

where qemu-nbd is interpreting the qcow2 file, and only exposing the
guest-visible data over the network.

Or, repeating the above in a different formulation - when you are
interested in serving qcow2 data over the network, you HAVE to have
exactly one of your two endpoints be aware that the qcow2 format is
involved - either the server is agnostic and the client interprets qcow2
(the former setup), or the server interprets qcow2 and the client is
agnostic (the latter setup).

Then, as Rich pointed out, when you expose qcow2 over NBD (the former
setup), the current NBD protocol does NOT allow resizes - so while
reading the image works okay, writing to the image is likely to hit
ENOSPC unless you pre-allocated the .qcow2 file being served.  That is
likely to change upstream in the next six months (it's on my todo list
to get NBD resize implemented), but there's still a percolation delay
from upstream to using it in default OS installs.  On the other hand,
when you expose raw over NBD by using qemu-nbd to convert qcow2 to raw
on the server end, then the server can resize things as needed and the
client need not be aware that qcow2 is even involved.

As for why nbdkit doesn't have a plugin for exposing a local qcow2 file
as raw - we don't see the point in duplicating what qemu-nbd can already
do. Yes, someone could write such a plugin, but it will probably have
worse performance than what you can already get out of using qemu-nbd as
the server.

> However the fundamental problem remains that you're exporting qcow2
> bytes over the NBD connection which is not really what you wanted to
> do.

Maybe it is, if the qcow2 file is fully preallocated, but yeah, until
NBD resize exists, most people don't want that combination.

>
> You should use qemu-nbd instead since it understands qcow2 natively,
> or use raw format disks if you want to use nbdkit for its other
> features.
>
> Rich.
>

--
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org