[PATCH] rhv-upload: Support qcow2 disks
When using oVirt >= 4.3, we can enable the NBD based backend in imageio by specifying that we transfer raw data when creating a transfer. With the NBD backend, we can specify qcow2 disk format. I think we need a way to expose the capabilities of the output to the user. For example, can we use qcow2 format. Issues: - I hacked qemu-img convert command line to always use -O raw instead of "-O t.target_format" since oVirt always starts qemu-nbd with the actual disk format. This works for this plugin, but may be wrong for other plugins. --- v2v/output_rhv_upload.ml | 5 ++--- v2v/rhv-upload-plugin.py | 19 +++++++++++++++++++ v2v/v2v.ml | 4 +++- 3 files changed, 24 insertions(+), 4 deletions(-) diff --git a/v2v/output_rhv_upload.ml b/v2v/output_rhv_upload.ml index 3514714b..481eb482 100644 --- a/v2v/output_rhv_upload.ml +++ b/v2v/output_rhv_upload.ml @@ -174,7 +174,7 @@ See also the virt-v2v-output-rhv(1) manual.") error (f_"nbdkit was compiled without SELinux support. You will have to recompile nbdkit with libselinux-devel installed, or else set SELinux to Permissive mode while doing the conversion.") in - (* Output format/sparse must be raw/sparse. We may be able to + (* Output sparse must be sparse. We may be able to * lift this limitation in future, but it requires changes on the * RHV side. See TODO file for details. XXX *) @@ -333,8 +333,7 @@ object let disk_format = match target_format with | "raw" as fmt -> fmt - | "qcow2" -> - error_current_limitation "-of raw" + | "qcow2" as fmt -> fmt | _ -> error (f_"rhv-upload: -of %s: Only output format ‘raw’ or ‘qcow2’ is supported. If the input is in a different format then force one of these output formats by adding either ‘-of raw’ or ‘-of qcow2’ on the command line.") target_format in diff --git a/v2v/rhv-upload-plugin.py b/v2v/rhv-upload-plugin.py index a2d09458..4272a89f 100644 --- a/v2v/rhv-upload-plugin.py +++ b/v2v/rhv-upload-plugin.py @@ -17,6 +17,7 @@ # 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. import builtins +import inspect import json import logging import socket @@ -456,6 +457,8 @@ def create_disk(connection): name = params['disk_name'], description = "Uploaded by virt-v2v", format = disk_format, + # XXX For qcow2 disk on block storage, we should use the estimated + # size, based on qemu-img measure of the overlay. initial_size = params['disk_size'], provisioned_size = params['disk_size'], # XXX Ignores params['output_sparse']. @@ -497,11 +500,16 @@ def create_transfer(connection, disk, host): system_service = connection.system_service() transfers_service = system_service.image_transfers_service() + extra = {} + if transfer_supports_format(): + extra["format"] = types.DiskFormat.RAW + transfer = transfers_service.add( types.ImageTransfer( disk = types.Disk(id = disk.id), host = host, inactivity_timeout = 3600, + **extra, ) ) @@ -531,6 +539,17 @@ def create_transfer(connection, disk, host): return transfer +def transfer_supports_format(): + """ + Return True if transfer supports the "format" argument, enabing the NBD + bakend on imageio side, which allows uploading to qcow2 images. + + This feature was added in ovirt 4.3. We assume that the SDK version matches + engine version. + """ + sig = inspect.signature(types.ImageTransfer) + return "format" in sig.parameters + # oVirt imageio operations def parse_transfer_url(transfer): diff --git a/v2v/v2v.ml b/v2v/v2v.ml index 03590c9e..58bb06c3 100644 --- a/v2v/v2v.ml +++ b/v2v/v2v.ml @@ -739,7 +739,9 @@ and copy_targets cmdline targets input output = | TargetURI uri -> uri in [ "qemu-img"; "convert" ] @ (if not (quiet ()) then [ "-p" ] else []) @ - [ "-n"; "-f"; "qcow2"; "-O"; t.target_format ] @ + (* XXX When using NBD we must use raw format, not the target_format + * which is the disk format. commpressed format will also not work. *) + [ "-n"; "-f"; "qcow2"; "-O"; "raw" ] @ (if cmdline.compressed then [ "-c" ] else []) @ [ "-S"; "64k" ] @ [ overlay_file; filename ] in -- 2.21.0
On Wed, Nov 20, 2019 at 03:06:55AM +0200, Nir Soffer wrote:
diff --git a/v2v/v2v.ml b/v2v/v2v.ml index 03590c9e..58bb06c3 100644 --- a/v2v/v2v.ml +++ b/v2v/v2v.ml @@ -739,7 +739,9 @@ and copy_targets cmdline targets input output = | TargetURI uri -> uri in [ "qemu-img"; "convert" ] @ (if not (quiet ()) then [ "-p" ] else []) @ - [ "-n"; "-f"; "qcow2"; "-O"; t.target_format ] @ + (* XXX When using NBD we must use raw format, not the target_format + * which is the disk format. commpressed format will also not work. *) + [ "-n"; "-f"; "qcow2"; "-O"; "raw" ] @
This is going to break all the other output modes. You probably want to define a new method output#override_output_format (see v2v.ml:get_target_formats and types.mli). Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW
On Mon, Nov 25, 2019, 11:15 Richard W.M. Jones <rjones@redhat.com> wrote:
On Wed, Nov 20, 2019 at 03:06:55AM +0200, Nir Soffer wrote:
diff --git a/v2v/v2v.ml b/v2v/v2v.ml index 03590c9e..58bb06c3 100644 --- a/v2v/v2v.ml +++ b/v2v/v2v.ml @@ -739,7 +739,9 @@ and copy_targets cmdline targets input output = | TargetURI uri -> uri in [ "qemu-img"; "convert" ] @ (if not (quiet ()) then [ "-p" ] else []) @ - [ "-n"; "-f"; "qcow2"; "-O"; t.target_format ] @ + (* XXX When using NBD we must use raw format, not the target_format + * which is the disk format. commpressed format will also not work. *) + [ "-n"; "-f"; "qcow2"; "-O"; "raw" ] @
This is going to break all the other output modes.
You probably want to define a new method output#override_output_format (see v2v.ml:get_target_formats and types.mli).
Maybe outpu#tranfer_format? Return output_format by default, rhv_output will override it to return raw?
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW
On Mon, Nov 25, 2019 at 04:27:11PM +0200, Nir Soffer wrote:
On Mon, Nov 25, 2019, 11:15 Richard W.M. Jones <rjones@redhat.com> wrote:
On Wed, Nov 20, 2019 at 03:06:55AM +0200, Nir Soffer wrote:
diff --git a/v2v/v2v.ml b/v2v/v2v.ml index 03590c9e..58bb06c3 100644 --- a/v2v/v2v.ml +++ b/v2v/v2v.ml @@ -739,7 +739,9 @@ and copy_targets cmdline targets input output = | TargetURI uri -> uri in [ "qemu-img"; "convert" ] @ (if not (quiet ()) then [ "-p" ] else []) @ - [ "-n"; "-f"; "qcow2"; "-O"; t.target_format ] @ + (* XXX When using NBD we must use raw format, not the target_format + * which is the disk format. commpressed format will also not work. *) + [ "-n"; "-f"; "qcow2"; "-O"; "raw" ] @
This is going to break all the other output modes.
You probably want to define a new method output#override_output_format (see v2v.ml:get_target_formats and types.mli).
Maybe outpu#tranfer_format? Return output_format by default, rhv_output will override it to return raw?
Isn't that essentially what #override_output_format virtual method does already? Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top
On Mon, Nov 25, 2019, 16:37 Richard W.M. Jones <rjones@redhat.com> wrote:
On Mon, Nov 25, 2019 at 04:27:11PM +0200, Nir Soffer wrote:
On Mon, Nov 25, 2019, 11:15 Richard W.M. Jones <rjones@redhat.com> wrote:
On Wed, Nov 20, 2019 at 03:06:55AM +0200, Nir Soffer wrote:
diff --git a/v2v/v2v.ml b/v2v/v2v.ml index 03590c9e..58bb06c3 100644 --- a/v2v/v2v.ml +++ b/v2v/v2v.ml @@ -739,7 +739,9 @@ and copy_targets cmdline targets input output = | TargetURI uri -> uri in [ "qemu-img"; "convert" ] @ (if not (quiet ()) then [ "-p" ] else []) @ - [ "-n"; "-f"; "qcow2"; "-O"; t.target_format ] @ + (* XXX When using NBD we must use raw format, not the target_format + * which is the disk format. commpressed format will also not work. *) + [ "-n"; "-f"; "qcow2"; "-O"; "raw" ] @
This is going to break all the other output modes.
You probably want to define a new method output#override_output_format (see v2v.ml:get_target_formats and types.mli).
Maybe outpu#tranfer_format? Return output_format by default, rhv_output will override it to return raw?
Isn't that essentially what #override_output_format virtual method does already?
The name confused me, I will check this then.
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top
On Mon, Nov 25, 2019 at 4:37 PM Richard W.M. Jones <rjones@redhat.com> wrote:
On Mon, Nov 25, 2019 at 04:27:11PM +0200, Nir Soffer wrote:
On Mon, Nov 25, 2019, 11:15 Richard W.M. Jones <rjones@redhat.com> wrote:
On Wed, Nov 20, 2019 at 03:06:55AM +0200, Nir Soffer wrote:
diff --git a/v2v/v2v.ml b/v2v/v2v.ml index 03590c9e..58bb06c3 100644 --- a/v2v/v2v.ml +++ b/v2v/v2v.ml @@ -739,7 +739,9 @@ and copy_targets cmdline targets input output = | TargetURI uri -> uri in [ "qemu-img"; "convert" ] @ (if not (quiet ()) then [ "-p" ] else []) @ - [ "-n"; "-f"; "qcow2"; "-O"; t.target_format ] @ + (* XXX When using NBD we must use raw format, not the target_format + * which is the disk format. commpressed format will also not work. *) + [ "-n"; "-f"; "qcow2"; "-O"; "raw" ] @
This is going to break all the other output modes.
You probably want to define a new method output#override_output_format (see v2v.ml:get_target_formats and types.mli).
Maybe outpu#tranfer_format? Return output_format by default, rhv_output will override it to return raw?
Isn't that essentially what #override_output_format virtual method does already?
No, it overrides target format (as I thought). If you implement: (* Force raw output, ignoring -of command line option. *) method override_output_format _ = Some "raw" And you run: vit-v2v -of qcow2 ... It creates the command: qemu-img convert -O raw ... But the rhv-upload-plugin gets these params: using params: {'disk_format': 'raw', ... And then we create a raw disk on oVirt side. What we need is to create qcow2 disk, and use -O raw. If this was written in python, I can think of this interface: class Output: def transfer_format(self): return self.target_format Outputs that want to always use raw format would implement: class OutputRHVUpload: def transfer_format(self): return "raw" So the v2v command line will be: ... [ "-n"; "-f"; "qcow2"; "-O"; t.transfer_format ] @ ... Nir
participants (3)
-
Nir Soffer -
Nir Soffer -
Richard W.M. Jones