On Tue, Apr 10, 2018 at 6:53 PM Richard W.M. Jones <rjones@redhat.com> wrote:

On Tue, Apr 10, 2018 at 03:25:47PM +0000, Nir Soffer wrote:
> On Tue, Apr 10, 2018 at 5:50 PM Richard W.M. Jones <rjones@redhat.com>
> wrote:
>
> > On Tue, Apr 10, 2018 at 02:07:33PM +0000, Nir Soffer wrote:
> > > This makes sense if the device is backed by a block device on oVirt side,
> > > and the NBD support efficient zeroing. But in this case the device is
> > backed
> > > by an empty sparse file on NFS, and oVirt does not support yet efficient
> > > zeroing, we just write zeros manually.
> > >
> > > I think should be handled on virt-v2v plugin side. When zeroing a file
> > raw
> > > image,
> > > you can ignore zero requests after the highest write offset, since the
> > > plugin
> > > created a new image, and we know that the image is empty.
> > >
> > > When the destination is a block device we cannot avoid zeroing since a
> > block
> > > device may contain junk data (we usually get dirty empty images from our
> > > local
> > > xtremio server).
> >
> > (Off topic for qemu-block but ...) We don't have enough information
> > at our end to know about any of this.
> >
>
> Can't use use this logic in the oVirt plugin?
>
> file based storage -> skip initial zeroing
> block based storage -> use initial zeroing
>
> Do you think that publishing disk capabilities in the sdk will solve this?

The plugin would have to do some complicated gymnastics. It would
have to keep track of which areas of the disk have been written and
ignore NBD_CMD_WRITE_ZEROES for other areas, except if block-based
storage is being used. And so yes we'd also need the imageio API to
publish that information to the plugin.

So it's possible but not trivial.

I think this should be fixed in qemu-img. The current zero optimization may

have insignificant improvement when the backend supports fast zeroing

(e.g fallocate, blkzeroout) by minimizing number of syscalls, but it has severe

performance issue when the desntiation does not support fast zero

(e.g. NFS < 4.2).

We plan to improve zero performance in 4.2.z, but this will not solve the issue

on NFS < 4.2.

By the way I think we're slowly reimplementing NBD in the imageio API.

You can also look at it as - slowly showing that HTTP can replace

NBD :-)

Dan Berrange pointed out earlier on that it might be easier if imageio
just exposed NBD, or if we found a way to tunnel NBD requests over web
sockets (in the format case nbdkit would not be needed, in the latter
case nbdkit could act as a bridge).

I'm not the more complex web sockets are needed for moving images

around. web sockets are good when the clients is a browser, and you need

to pass lot of small messages. If you have a real socket you don't need

web sockets. And the overhead of http headers is insignificant when you

move lot of data around.

Exposing NBD using the same authentication mechanism can be interesting,

but only if clients have an easy way to use this.

Will this allow client (e.g. backup vender) to download and upload images

using qemu-img?

qemu-img <- nbd-> ovirt-imageio <- ndb -> qemu/ndb-server

We also need streaming support - with http you can stream the data from

anywhere, while qemu-img needs a file. I don't think that backup vendors

or users will be happy to implement nbdkit plugins.

If qemu-img would support input and ouptut from stdin/stdout instead of

files, this model sounds much more interesting.

Nir