[v2v PATCH v2] input_vmx: cleanly reject guests with snapshots when using "-it ssh"
For traversing a backing chain of VMDK descriptor files over ssh, two things are necessary: - qemu-nbd with the ssh block driver, rather than nbdkit-ssh-plugin, - a remote SSH URL (for qemu-nbd) without a query string appended, as qemu-nbd cannot update the last pathname component (for tracking the relative pathnames of VMDK descriptor files) if a query string is appended. Before commit 7a6f6113a25f ("v2v: -i vmx -it ssh: Replace qemu block ssh driver with nbdkit-ssh-plugin.", 2019-10-08), we passed the "?host_key_check=no" query string in the URL to qemu-nbd, so we can't just return to that, for accessing snapshotted guests with vmx+ssh. But, we shouldn't return to qemu-nbd for vmx+ssh even without a query string, as that would undo the other benefit(s) of commit 7a6f6113a25f. Instead, clearly document that snapshotted guests are not supported over vmx+ssh, and cleanly reject this situation in the code as well. Recommend the two alternative transports that allow the user to convert such guests. Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1774386 Signed-off-by: Laszlo Ersek <lersek@redhat.com> --- Notes: v2: - in the manual, centralize the description in a new NOTES section, and reference that section from "-i vmx -it ssh" and "INPUT FROM VMWARE VMX" [Rich] - in the code, bail out with error (f_"...") rather than failwith, for enabling translations [Rich] - in the error message printed from the code, recommend actions to the user (reference the new NOTES section of the manual) [Rich] input/input_vmx.ml | 27 ++++++++------------ docs/virt-v2v-input-vmware.pod | 18 ++++++++++++- 2 files changed, 27 insertions(+), 18 deletions(-) diff --git a/input/input_vmx.ml b/input/input_vmx.ml index 9921419b5a85..eed8a4335622 100644 --- a/input/input_vmx.ml +++ b/input/input_vmx.ml @@ -81,23 +81,16 @@ module VMX = struct let vmx_path = path_of_uri uri in let abs_path = absolute_path_from_other_file vmx_path filename in - let format = "vmdk" in + let flat_vmdk = PCRE.replace (PCRE.compile "\\.vmdk$") + "-flat.vmdk" abs_path in - (* XXX This is a hack to work around qemu / VMDK limitation - * "Cannot use relative extent paths with VMDK descriptor file" - * We can remove this if the above is fixed. - *) - let abs_path, format = - let flat_vmdk = - PCRE.replace (PCRE.compile "\\.vmdk$") "-flat.vmdk" abs_path in - if remote_file_exists uri flat_vmdk then (flat_vmdk, "raw") - else (abs_path, format) in - - (* XXX In virt-v2v 1.42+ importing from VMX over SSH - * was broken if the -flat.vmdk file did not exist. - * It is still broken here. - *) - ignore format; + (* RHBZ#1774386 *) + if not (remote_file_exists uri flat_vmdk) then + error (f_"This transport does not support guests with snapshots. \ + Either collapse the snapshots for this guest and try \ + the conversion again, or use one of the alternate \ + conversion methods described in \ + virt-v2v-input-vmware(1) section \"NOTES\"."); let server = server_of_uri uri in let port = Option.map string_of_int (port_of_uri uri) in @@ -110,7 +103,7 @@ module VMX = struct let cor = dir // "convert" in let bandwidth = options.bandwidth in let nbdkit = Nbdkit_ssh.create_ssh ?bandwidth ~cor ~password - ~server ?port ?user abs_path in + ~server ?port ?user flat_vmdk in let _, pid = Nbdkit.run_unix socket nbdkit in On_exit.kill pid ) filenames diff --git a/docs/virt-v2v-input-vmware.pod b/docs/virt-v2v-input-vmware.pod index 2c90e0afc7cd..66ca5f00d28a 100644 --- a/docs/virt-v2v-input-vmware.pod +++ b/docs/virt-v2v-input-vmware.pod @@ -51,6 +51,9 @@ to ESXi to read the F<GUEST.vmx> file and associated disks. This requires that you have enabled SSH access to the VMware ESXi hypervisor - in the default ESXi configuration this is turned off. +This transport is incompatible with guests that have snapshots; refer +to L</NOTES>. + =item B<-ic vpx://...> B<-it vddk> =item B<-ic esx://...> B<-it vddk> @@ -84,6 +87,18 @@ import a guest from VMware vCenter. This is the slowest method. =back +=head1 NOTES + +When accessing the F<I<guest>.vmx> file on ESXi over an SSH connection +(that is, when using the B<-i vmx> B<-it ssh> options), the conversion +will not work if the guest has snapshots (files called +F<I<guest>-000001.vmdk> and similar). Either collapse the snapshots +for the guest and retry the conversion with the same +B<-i vmx> B<-it ssh> options, or leave the snapshots intact and use a +transport different from SSH: just B<-i vmx>, or +B<-ic vpx://...> B<-it vddk> or B<-ic esx://...> B<-it vddk>. Refer +to L<https://bugzilla.redhat.com/1774386>. + =head1 INPUT FROM VMWARE VMX Virt-v2v is able to import guests from VMware’s vmx files. @@ -106,7 +121,8 @@ a C</vmfs/volumes> folder containing the virtual machines. If you find a folder of files called F<I<guest>.vmx>, F<I<guest>.vmxf>, F<I<guest>.nvram> and one or more F<.vmdk> disk -images, then you can use this method. +images, then you can use this method. The SSH transport is not usable +if the guest has snapshots; refer to L</NOTES>. =head2 VMX: Guest must be shut down -- 2.19.1.3.g30247aa5d201
On Sat, Apr 09, 2022 at 08:12:52AM +0200, Laszlo Ersek wrote:
For traversing a backing chain of VMDK descriptor files over ssh, two things are necessary:
- qemu-nbd with the ssh block driver, rather than nbdkit-ssh-plugin,
- a remote SSH URL (for qemu-nbd) without a query string appended, as qemu-nbd cannot update the last pathname component (for tracking the relative pathnames of VMDK descriptor files) if a query string is appended.
Before commit 7a6f6113a25f ("v2v: -i vmx -it ssh: Replace qemu block ssh driver with nbdkit-ssh-plugin.", 2019-10-08), we passed the "?host_key_check=no" query string in the URL to qemu-nbd, so we can't just return to that, for accessing snapshotted guests with vmx+ssh.
But, we shouldn't return to qemu-nbd for vmx+ssh even without a query string, as that would undo the other benefit(s) of commit 7a6f6113a25f.
Instead, clearly document that snapshotted guests are not supported over vmx+ssh, and cleanly reject this situation in the code as well. Recommend the two alternative transports that allow the user to convert such guests.
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1774386 Signed-off-by: Laszlo Ersek <lersek@redhat.com> ---
Notes: v2:
- in the manual, centralize the description in a new NOTES section, and reference that section from "-i vmx -it ssh" and "INPUT FROM VMWARE VMX" [Rich]
- in the code, bail out with error (f_"...") rather than failwith, for enabling translations [Rich]
And line wrapping too ..
- in the error message printed from the code, recommend actions to the user (reference the new NOTES section of the manual) [Rich]
input/input_vmx.ml | 27 ++++++++------------ docs/virt-v2v-input-vmware.pod | 18 ++++++++++++- 2 files changed, 27 insertions(+), 18 deletions(-)
diff --git a/input/input_vmx.ml b/input/input_vmx.ml index 9921419b5a85..eed8a4335622 100644 --- a/input/input_vmx.ml +++ b/input/input_vmx.ml @@ -81,23 +81,16 @@ module VMX = struct
let vmx_path = path_of_uri uri in let abs_path = absolute_path_from_other_file vmx_path filename in - let format = "vmdk" in + let flat_vmdk = PCRE.replace (PCRE.compile "\\.vmdk$") + "-flat.vmdk" abs_path in
- (* XXX This is a hack to work around qemu / VMDK limitation - * "Cannot use relative extent paths with VMDK descriptor file" - * We can remove this if the above is fixed. - *) - let abs_path, format = - let flat_vmdk = - PCRE.replace (PCRE.compile "\\.vmdk$") "-flat.vmdk" abs_path in - if remote_file_exists uri flat_vmdk then (flat_vmdk, "raw") - else (abs_path, format) in - - (* XXX In virt-v2v 1.42+ importing from VMX over SSH - * was broken if the -flat.vmdk file did not exist. - * It is still broken here. - *) - ignore format; + (* RHBZ#1774386 *) + if not (remote_file_exists uri flat_vmdk) then + error (f_"This transport does not support guests with snapshots. \ + Either collapse the snapshots for this guest and try \ + the conversion again, or use one of the alternate \ + conversion methods described in \ + virt-v2v-input-vmware(1) section \"NOTES\".");
let server = server_of_uri uri in let port = Option.map string_of_int (port_of_uri uri) in @@ -110,7 +103,7 @@ module VMX = struct let cor = dir // "convert" in let bandwidth = options.bandwidth in let nbdkit = Nbdkit_ssh.create_ssh ?bandwidth ~cor ~password - ~server ?port ?user abs_path in + ~server ?port ?user flat_vmdk in let _, pid = Nbdkit.run_unix socket nbdkit in On_exit.kill pid ) filenames diff --git a/docs/virt-v2v-input-vmware.pod b/docs/virt-v2v-input-vmware.pod index 2c90e0afc7cd..66ca5f00d28a 100644 --- a/docs/virt-v2v-input-vmware.pod +++ b/docs/virt-v2v-input-vmware.pod @@ -51,6 +51,9 @@ to ESXi to read the F<GUEST.vmx> file and associated disks. This requires that you have enabled SSH access to the VMware ESXi hypervisor - in the default ESXi configuration this is turned off.
+This transport is incompatible with guests that have snapshots; refer +to L</NOTES>. + =item B<-ic vpx://...> B<-it vddk>
=item B<-ic esx://...> B<-it vddk> @@ -84,6 +87,18 @@ import a guest from VMware vCenter. This is the slowest method.
=back
+=head1 NOTES + +When accessing the F<I<guest>.vmx> file on ESXi over an SSH connection +(that is, when using the B<-i vmx> B<-it ssh> options), the conversion
The convention -- not written down anywhere :-( -- is to use B<-option> when referencing the option first time in =item, and I<-option> elsewhere. See for example: https://gitlab.com/nbdkit/nbdkit/-/commit/bc0d4f23ce08052db4c9cd3a1a563f5497... So here it should be I<-i vmx> I<-it ssh> ...
+will not work if the guest has snapshots (files called +F<I<guest>-000001.vmdk> and similar). Either collapse the snapshots +for the guest and retry the conversion with the same +B<-i vmx> B<-it ssh> options, or leave the snapshots intact and use a +transport different from SSH: just B<-i vmx>, or +B<-ic vpx://...> B<-it vddk> or B<-ic esx://...> B<-it vddk>. Refer
... and here too.
+to L<https://bugzilla.redhat.com/1774386>. + =head1 INPUT FROM VMWARE VMX
Virt-v2v is able to import guests from VMware’s vmx files. @@ -106,7 +121,8 @@ a C</vmfs/volumes> folder containing the virtual machines.
If you find a folder of files called F<I<guest>.vmx>, F<I<guest>.vmxf>, F<I<guest>.nvram> and one or more F<.vmdk> disk -images, then you can use this method. +images, then you can use this method. The SSH transport is not usable +if the guest has snapshots; refer to L</NOTES>.
=head2 VMX: Guest must be shut down
With the B<> changed to I<> it's good to go, thanks, so: Reviewed-by: Richard W.M. Jones <rjones@redhat.com> Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v
On 04/09/22 08:30, Richard W.M. Jones wrote:
On Sat, Apr 09, 2022 at 08:12:52AM +0200, Laszlo Ersek wrote:
For traversing a backing chain of VMDK descriptor files over ssh, two things are necessary:
- qemu-nbd with the ssh block driver, rather than nbdkit-ssh-plugin,
- a remote SSH URL (for qemu-nbd) without a query string appended, as qemu-nbd cannot update the last pathname component (for tracking the relative pathnames of VMDK descriptor files) if a query string is appended.
Before commit 7a6f6113a25f ("v2v: -i vmx -it ssh: Replace qemu block ssh driver with nbdkit-ssh-plugin.", 2019-10-08), we passed the "?host_key_check=no" query string in the URL to qemu-nbd, so we can't just return to that, for accessing snapshotted guests with vmx+ssh.
But, we shouldn't return to qemu-nbd for vmx+ssh even without a query string, as that would undo the other benefit(s) of commit 7a6f6113a25f.
Instead, clearly document that snapshotted guests are not supported over vmx+ssh, and cleanly reject this situation in the code as well. Recommend the two alternative transports that allow the user to convert such guests.
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1774386 Signed-off-by: Laszlo Ersek <lersek@redhat.com> ---
Notes: v2:
- in the manual, centralize the description in a new NOTES section, and reference that section from "-i vmx -it ssh" and "INPUT FROM VMWARE VMX" [Rich]
- in the code, bail out with error (f_"...") rather than failwith, for enabling translations [Rich]
And line wrapping too ..
- in the error message printed from the code, recommend actions to the user (reference the new NOTES section of the manual) [Rich]
input/input_vmx.ml | 27 ++++++++------------ docs/virt-v2v-input-vmware.pod | 18 ++++++++++++- 2 files changed, 27 insertions(+), 18 deletions(-)
diff --git a/input/input_vmx.ml b/input/input_vmx.ml index 9921419b5a85..eed8a4335622 100644 --- a/input/input_vmx.ml +++ b/input/input_vmx.ml @@ -81,23 +81,16 @@ module VMX = struct
let vmx_path = path_of_uri uri in let abs_path = absolute_path_from_other_file vmx_path filename in - let format = "vmdk" in + let flat_vmdk = PCRE.replace (PCRE.compile "\\.vmdk$") + "-flat.vmdk" abs_path in
- (* XXX This is a hack to work around qemu / VMDK limitation - * "Cannot use relative extent paths with VMDK descriptor file" - * We can remove this if the above is fixed. - *) - let abs_path, format = - let flat_vmdk = - PCRE.replace (PCRE.compile "\\.vmdk$") "-flat.vmdk" abs_path in - if remote_file_exists uri flat_vmdk then (flat_vmdk, "raw") - else (abs_path, format) in - - (* XXX In virt-v2v 1.42+ importing from VMX over SSH - * was broken if the -flat.vmdk file did not exist. - * It is still broken here. - *) - ignore format; + (* RHBZ#1774386 *) + if not (remote_file_exists uri flat_vmdk) then + error (f_"This transport does not support guests with snapshots. \ + Either collapse the snapshots for this guest and try \ + the conversion again, or use one of the alternate \ + conversion methods described in \ + virt-v2v-input-vmware(1) section \"NOTES\".");
let server = server_of_uri uri in let port = Option.map string_of_int (port_of_uri uri) in @@ -110,7 +103,7 @@ module VMX = struct let cor = dir // "convert" in let bandwidth = options.bandwidth in let nbdkit = Nbdkit_ssh.create_ssh ?bandwidth ~cor ~password - ~server ?port ?user abs_path in + ~server ?port ?user flat_vmdk in let _, pid = Nbdkit.run_unix socket nbdkit in On_exit.kill pid ) filenames diff --git a/docs/virt-v2v-input-vmware.pod b/docs/virt-v2v-input-vmware.pod index 2c90e0afc7cd..66ca5f00d28a 100644 --- a/docs/virt-v2v-input-vmware.pod +++ b/docs/virt-v2v-input-vmware.pod @@ -51,6 +51,9 @@ to ESXi to read the F<GUEST.vmx> file and associated disks. This requires that you have enabled SSH access to the VMware ESXi hypervisor - in the default ESXi configuration this is turned off.
+This transport is incompatible with guests that have snapshots; refer +to L</NOTES>. + =item B<-ic vpx://...> B<-it vddk>
=item B<-ic esx://...> B<-it vddk> @@ -84,6 +87,18 @@ import a guest from VMware vCenter. This is the slowest method.
=back
+=head1 NOTES + +When accessing the F<I<guest>.vmx> file on ESXi over an SSH connection +(that is, when using the B<-i vmx> B<-it ssh> options), the conversion
The convention -- not written down anywhere :-( -- is to use B<-option> when referencing the option first time in =item, and I<-option> elsewhere. See for example:
https://gitlab.com/nbdkit/nbdkit/-/commit/bc0d4f23ce08052db4c9cd3a1a563f5497...
So here it should be I<-i vmx> I<-it ssh> ...
+will not work if the guest has snapshots (files called +F<I<guest>-000001.vmdk> and similar). Either collapse the snapshots +for the guest and retry the conversion with the same +B<-i vmx> B<-it ssh> options, or leave the snapshots intact and use a +transport different from SSH: just B<-i vmx>, or +B<-ic vpx://...> B<-it vddk> or B<-ic esx://...> B<-it vddk>. Refer
... and here too.
+to L<https://bugzilla.redhat.com/1774386>. + =head1 INPUT FROM VMWARE VMX
Virt-v2v is able to import guests from VMware’s vmx files. @@ -106,7 +121,8 @@ a C</vmfs/volumes> folder containing the virtual machines.
If you find a folder of files called F<I<guest>.vmx>, F<I<guest>.vmxf>, F<I<guest>.nvram> and one or more F<.vmdk> disk -images, then you can use this method. +images, then you can use this method. The SSH transport is not usable +if the guest has snapshots; refer to L</NOTES>.
=head2 VMX: Guest must be shut down
With the B<> changed to I<> it's good to go, thanks, so:
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Commit e0b08ee0213f. Thanks! Laszlo
participants (2)
-
Laszlo Ersek -
Richard W.M. Jones