[PATCH 0/4] v2v: Add a new tool virt-v2v-copy-to-local to handle Xen and ESXi
It turns out that RHEL 5 Xen conversions don't work if the source disk is located on a block device. See patch 1/4 for the gory details. This patch series proposes a new tool called virt-v2v-copy-to-local which essentially is a way to make new virt-v2v work like the old virt-v2v, ie. copy first, convert after. Of course this is very slow and would only be used as a last resort, but I identified two situations in which this would currently be useful (see explanation in patch 4/4). This is also available in my git repo: https://github.com/rwmjones/libguestfs/commits/master Rich.
If you try to convert a guest that has a zero-sized disk, it will currently fail in a rather strange way. Usually you will see errors in the debug output: [ 0.562714] sd 2:0:0:0: [sda] Read Capacity(16) failed: Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [ 0.563884] sd 2:0:0:0: [sda] Sense Key : Not Ready [current] [ 0.564587] sd 2:0:0:0: [sda] Add. Sense: Logical unit not ready, manual intervention required followed by virt-v2v failing with: libguestfs: trace: v2v: inspect_os = [] virt-v2v: error: inspection could not detect the source guest (or physical machine). Additionally, because of a problem with the ssh driver in qemu (or perhaps, with sftp-server on the remote side) it is not possible to use the ssh driver to open a block device path on the remote server. The drive will appear as zero-sized, triggering the above error. Therefore detect this situation, and emit an error (see below). Also add a section to the manual describing the workaround required for converting RHEL 5 Xen guests which are located on block devices. virt-v2v: error: guest disk sda appears to be zero bytes in size. There could be several reasons for this: Check that the guest doesn't really have a zero-sized disk. virt-v2v cannot convert such a guest. If you are converting a guest from an ssh source and the guest has a disk on a block device (eg. on a host partition or host LVM LV), then conversions of this type are not supported. See "XEN OR SSH CONVERSIONS FROM BLOCK DEVICES" in the virt-v2v(1) manual for a workaround. --- v2v/v2v.ml | 7 ++++++ v2v/virt-v2v.pod | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 83 insertions(+) diff --git a/v2v/v2v.ml b/v2v/v2v.ml index 4b04834..d633601 100644 --- a/v2v/v2v.ml +++ b/v2v/v2v.ml @@ -228,6 +228,13 @@ and create_overlays src_disks = let vsize = (open_guestfs ())#disk_virtual_size overlay_file in + (* If the virtual size is 0, then something went badly wrong. + * It could be RHBZ#1283588 or some other problem with qemu. + *) + if vsize = 0L then + error (f_"guest disk %s appears to be zero bytes in size.\n\nThere could be several reasons for this:\n\nCheck that the guest doesn't really have a zero-sized disk. virt-v2v cannot convert such a guest.\n\nIf you are converting a guest from an ssh source and the guest has a disk on a block device (eg. on a host partition or host LVM LV), then conversions of this type are not supported. See \"XEN OR SSH CONVERSIONS FROM BLOCK DEVICES\" in the virt-v2v(1) manual for a workaround.") + sd; + { ov_overlay_file = overlay_file; ov_sd = sd; ov_virtual_size = vsize; ov_source = source } ) src_disks diff --git a/v2v/virt-v2v.pod b/v2v/virt-v2v.pod index 2e83168..0b7be7d 100644 --- a/v2v/virt-v2v.pod +++ b/v2v/virt-v2v.pod @@ -1226,6 +1226,10 @@ B<If the above commands do not work, then virt-v2v is not going to work either>. Fix your libvirt configuration or the remote server before continuing. +B<If the guest disks are located on a host block device>, then the +conversion will fail. See L</XEN OR SSH CONVERSIONS FROM BLOCK DEVICES> +below for a workaround. + =head2 XEN: IMPORTING A GUEST To import a particular guest from a Xen server, do: @@ -1241,6 +1245,78 @@ In this case the output flags are set to write the converted guest to a temporary directory as this is just an example, but you can also write to libvirt or any other supported target. +=head2 XEN OR SSH CONVERSIONS FROM BLOCK DEVICES + +It is currently not possible to convert a Xen guest (or any guest +located remotely over ssh) if that guest's disks are located on host +block devices. + +To tell if a Xen guest uses host block devices, look at the guest XML. +You will see: + + <disk type='block' device='disk'> + ... + <source dev='/dev/VG/guest'/> + +where C<type='block'>, C<source dev=> and C</dev/...> are all +indications that the disk is located on a host block device. + +This happens because the qemu ssh block driver that we use to access +remote disks uses the ssh sftp protocol, and this protocol cannot +correctly detect the size of host block devices. + +The workaround is to copy the guest over to the conversion server. +You will need sufficient space on the conversion server to store a +full copy of the guest. + +=over 4 + +=item 1. + +From the conversion host, dump the guest XML to a local file: + + virsh -c xen+ssh://root@xen.example.com dumpxml guest > guest.xml + +=item 2. + +From the conversion host, copy the guest disk(s) over. You may need +to read the C<E<lt>diskE<gt>> sections from the guest XML to find the +names of the guest disks. + + ssh root@xen.example.com 'dd if=/dev/VG/guest' | dd conv=sparse of=guest-disk1 + +Notice C<conv=sparse> which adds sparseness, so the copied disk will +use as little space as possible. + +=item 3. + +Edit the guest XML file, replacing C<E<lt>diskE<gt>> section(s) that +refer to the remote disks with references to the local copies. + +Three changes have to be made. Firstly in: + + <disk type='block' device='disk'> + +the C<type> must be changed to C<file>: + + <disk type='file' device='disk'> + +The last two changes are in the C<E<lt>sourceE<gt>> line where: + + <source dev='/dev/VG/guest'/> + +C<source dev=> becomes C<source file=> pointing at the local file: + + <source file='guest-disk1'/> + +=item 4. + +Convert the guest using C<virt-v2v -i libvirtxml> mode like this: + + virt-v2v -i libvirtxml guest.xml [-o options as usual ...] + +=back + =head1 OUTPUT TO LIBVIRT The I<-o libvirt> option lets you upload the converted guest to -- 2.5.0
Split the map_source_to_uri function up so that we can get at just the https://... URI. This is almost refactoring. It doesn't quite handle the case where we are passed a source path from the libvirt VMware driver that doesn't match the expected "[datastore] path" format, but probably if we hit that case it's a bug anyway. --- v2v/vCenter.ml | 79 ++++++++++++++++++++++++++++++--------------------------- v2v/vCenter.mli | 6 +++++ 2 files changed, 48 insertions(+), 37 deletions(-) diff --git a/v2v/vCenter.ml b/v2v/vCenter.ml index 4c5ca1c..c411f6e 100644 --- a/v2v/vCenter.ml +++ b/v2v/vCenter.ml @@ -174,9 +174,9 @@ let guess_dcPath uri = function let source_re = Str.regexp "^\\[\\(.*\\)\\] \\(.*\\)\\.vmdk$" -let map_source_to_uri readahead dcPath password uri scheme server path = +let map_source_to_https dcPath uri server path = if not (Str.string_match source_re path 0) then - path + (path, true) else ( let datastore = Str.matched_group 1 path and path = Str.matched_group 2 path in @@ -208,39 +208,44 @@ let map_source_to_uri readahead dcPath password uri scheme server path = (* XXX only works if the query string is not URI-quoted *) String.find query "no_verify=1" = -1 in - (* Now we have to query the server to get the session cookie. *) - let session_cookie = get_session_cookie password scheme uri sslverify url in - - (* Construct the JSON parameters. *) - let json_params = [ - "file.driver", JSON.String "https"; - "file.url", JSON.String url; - (* https://bugzilla.redhat.com/show_bug.cgi?id=1146007#c10 *) - "file.timeout", JSON.Int 2000; - ] in - - let json_params = - match readahead with - | None -> json_params - | Some readahead -> - ("file.readahead", JSON.Int readahead) :: json_params in - - let json_params = - if sslverify then json_params - else ("file.sslverify", JSON.String "off") :: json_params in - - let json_params = - match session_cookie with - | None -> json_params - | Some cookie -> ("file.cookie", JSON.String cookie) :: json_params in - - if verbose () then - printf "vcenter: json parameters: %s\n" (JSON.string_of_doc json_params); - - (* Turn the JSON parameters into a 'json:' protocol string. - * Note this requires qemu-img >= 2.1.0. - *) - let qemu_uri = "json: " ^ JSON.string_of_doc json_params in - - qemu_uri + (url, sslverify) ) + +let map_source_to_uri readahead dcPath password uri scheme server path = + let url, sslverify = map_source_to_https dcPath uri server path in + + (* Now we have to query the server to get the session cookie. *) + let session_cookie = get_session_cookie password scheme uri sslverify url in + + (* Construct the JSON parameters. *) + let json_params = [ + "file.driver", JSON.String "https"; + "file.url", JSON.String url; + (* https://bugzilla.redhat.com/show_bug.cgi?id=1146007#c10 *) + "file.timeout", JSON.Int 2000; + ] in + + let json_params = + match readahead with + | None -> json_params + | Some readahead -> + ("file.readahead", JSON.Int readahead) :: json_params in + + let json_params = + if sslverify then json_params + else ("file.sslverify", JSON.String "off") :: json_params in + + let json_params = + match session_cookie with + | None -> json_params + | Some cookie -> ("file.cookie", JSON.String cookie) :: json_params in + + if verbose () then + printf "vcenter: json parameters: %s\n" (JSON.string_of_doc json_params); + + (* Turn the JSON parameters into a 'json:' protocol string. + * Note this requires qemu-img >= 2.1.0. + *) + let qemu_uri = "json: " ^ JSON.string_of_doc json_params in + + qemu_uri diff --git a/v2v/vCenter.mli b/v2v/vCenter.mli index 15b5143..fedddbd 100644 --- a/v2v/vCenter.mli +++ b/v2v/vCenter.mli @@ -52,3 +52,9 @@ val map_source_to_uri : int option -> string -> string option -> Xml.uri -> stri ["[datastore1] Fedora 20/Fedora 20.vmdk"] including those literal spaces in the string. *) + +val map_source_to_https : string -> Xml.uri -> string -> string -> string * bool +(** [map_source_to_https dcPath uri server path] is the same as + {!map_source_to_uri} but it produces a regular [https://...] URL. + The returned boolean is whether TLS certificate verification + should be done. *) -- 2.5.0
--- v2v/xml-c.c | 12 ++++++++++++ v2v/xml.ml | 3 +++ v2v/xml.mli | 3 +++ 3 files changed, 18 insertions(+) diff --git a/v2v/xml-c.c b/v2v/xml-c.c index 97b8a20..a6d9231 100644 --- a/v2v/xml-c.c +++ b/v2v/xml-c.c @@ -353,6 +353,18 @@ v2v_xml_node_ptr_set_prop (value nodev, value namev, value valv) } value +v2v_xml_node_ptr_unset_prop (value nodev, value namev) +{ + CAMLparam2 (nodev, namev); + xmlNodePtr node = (xmlNodePtr) nodev; + int r; + + r = xmlUnsetProp (node, BAD_CAST String_val (namev)); + + CAMLreturn (r == 0 ? Val_true : Val_false); +} + +value v2v_xml_node_ptr_unlink_node (value nodev) { CAMLparam1 (nodev); diff --git a/v2v/xml.ml b/v2v/xml.ml index dbb2b41..27c8566 100644 --- a/v2v/xml.ml +++ b/v2v/xml.ml @@ -95,6 +95,9 @@ let new_text_child (doc_ptr, node_ptr) name content = external node_ptr_set_prop : node_ptr -> string -> string -> unit = "v2v_xml_node_ptr_set_prop" let set_prop (doc_ptr, node_ptr) = node_ptr_set_prop node_ptr +external node_ptr_unset_prop : node_ptr -> string -> bool = "v2v_xml_node_ptr_unset_prop" +let unset_prop (doc_ptr, node_ptr) = node_ptr_unset_prop node_ptr + external node_ptr_unlink_node : node_ptr -> unit = "v2v_xml_node_ptr_unlink_node" let unlink_node (doc_ptr, node_ptr) = node_ptr_unlink_node node_ptr diff --git a/v2v/xml.mli b/v2v/xml.mli index b039d09..9664c73 100644 --- a/v2v/xml.mli +++ b/v2v/xml.mli @@ -73,6 +73,9 @@ val new_text_child : node -> string -> string -> node val set_prop : node -> string -> string -> unit (** xmlSetProp *) +val unset_prop : node -> string -> bool +(** xmlUnsetProp (returns [true] if the property was removed) *) + val unlink_node : node -> unit (** xmlUnlinkNode {b NB:} This frees the [node], do not use it afterwards. *) -- 2.5.0
This allows certain guests which virt-v2v cannot access to be copied off the remote hypervisor and converted. Essentially this just automates the process of copying the guest's disks and adjusting the libvirt XML. --- .gitignore | 3 + po/POTFILES-ml | 1 + v2v/Makefile.am | 65 ++++++++- v2v/copy_to_local.ml | 311 +++++++++++++++++++++++++++++++++++++++++ v2v/virt-v2v-copy-to-local.pod | 206 +++++++++++++++++++++++++++ v2v/virt-v2v.pod | 72 ++++------ 6 files changed, 606 insertions(+), 52 deletions(-) create mode 100644 v2v/copy_to_local.ml create mode 100644 v2v/virt-v2v-copy-to-local.pod diff --git a/.gitignore b/.gitignore index a43c243..11557b6 100644 --- a/.gitignore +++ b/.gitignore @@ -562,6 +562,7 @@ Makefile.in /v2v/rhel-6.5.img /v2v/rhel-7.0.img /v2v/stamp-virt-v2v.pod +/v2v/stamp-virt-v2v-copy-to-local.pod /v2v/test-harness/.depend /v2v/test-harness/META /v2v/test-harness/dllv2v_test_harness.so @@ -570,6 +571,8 @@ Makefile.in /v2v/v2v_unit_tests /v2v/virt-v2v /v2v/virt-v2v.1 +/v2v/virt-v2v-copy-to-local +/v2v/virt-v2v-copy-to-local.1 /website/*.html /website/README.txt /website/TODO.txt diff --git a/po/POTFILES-ml b/po/POTFILES-ml index c02ffc0..00a9d63 100644 --- a/po/POTFILES-ml +++ b/po/POTFILES-ml @@ -101,6 +101,7 @@ v2v/changeuid.ml v2v/cmdline.ml v2v/convert_linux.ml v2v/convert_windows.ml +v2v/copy_to_local.ml v2v/curl.ml v2v/domainxml.ml v2v/input_disk.ml diff --git a/v2v/Makefile.am b/v2v/Makefile.am index 5dfef6e..4b6cbcc 100644 --- a/v2v/Makefile.am +++ b/v2v/Makefile.am @@ -35,7 +35,8 @@ EXTRA_DIST = \ test-v2v-networks-and-bridges-expected.xml \ test-v2v-networks-and-bridges.xml \ test-v2v-sound.xml \ - virt-v2v.pod + virt-v2v.pod \ + virt-v2v-copy-to-local.pod CLEANFILES = *~ *.annot *.cmi *.cmo *.cmx *.cmxa *.o virt-v2v @@ -113,7 +114,7 @@ SOURCES_C = \ if HAVE_OCAML -bin_PROGRAMS = virt-v2v +bin_PROGRAMS = virt-v2v virt-v2v-copy-to-local virt_v2v_SOURCES = $(SOURCES_C) virt_v2v_CPPFLAGS = \ @@ -177,6 +178,46 @@ virt_v2v_LINK = \ $(OCAMLFIND) $(BEST) $(OCAMLFLAGS) $(OCAMLPACKAGES) $(OCAMLLINKFLAGS) \ $(OBJECTS) -o $@ +virt_v2v_copy_to_local_SOURCES = \ + domainxml-c.c \ + utils-c.c \ + xml-c.c +virt_v2v_copy_to_local_CPPFLAGS = \ + -I. \ + -I$(top_builddir) \ + -I$(shell $(OCAMLC) -where) \ + -I$(top_srcdir)/src +virt_v2v_copy_to_local_CFLAGS = \ + $(WARN_CFLAGS) $(WERROR_CFLAGS) \ + $(LIBXML2_CFLAGS) \ + $(LIBVIRT_CFLAGS) + +COPY_TO_LOCAL_BOBJECTS = \ + $(top_builddir)/mllib/guestfs_config.cmo \ + $(top_builddir)/mllib/common_gettext.cmo \ + $(top_builddir)/mllib/common_utils.cmo \ + $(top_builddir)/mllib/JSON.cmo \ + xml.cmo \ + utils.cmo \ + curl.cmo \ + vCenter.cmo \ + domainxml.cmo \ + copy_to_local.cmo +COPY_TO_LOCAL_XOBJECTS = $(COPY_TO_LOCAL_BOBJECTS:.cmo=.cmx) + +if !HAVE_OCAMLOPT +COPY_TO_LOCAL_OBJECTS = $(COPY_TO_LOCAL_BOBJECTS) +else +COPY_TO_LOCAL_OBJECTS = $(COPY_TO_LOCAL_XOBJECTS) +endif + +virt_v2v_copy_to_local_DEPENDENCIES = \ + $(COPY_TO_LOCAL_OBJECTS) $(top_srcdir)/ocaml-link.sh +virt_v2v_copy_to_local_LINK = \ + $(top_srcdir)/ocaml-link.sh -cclib '$(OCAMLCLIBS)' -- \ + $(OCAMLFIND) $(BEST) $(OCAMLFLAGS) $(OCAMLPACKAGES) $(OCAMLLINKFLAGS) \ + $(COPY_TO_LOCAL_OBJECTS) -o $@ + .mli.cmi: $(OCAMLFIND) ocamlc $(OCAMLFLAGS) $(OCAMLPACKAGES) -c $< -o $@ .ml.cmo: @@ -192,9 +233,11 @@ virttoolsdatadir = $(datadir)/virt-tools # Manual pages and HTML files for the website. -man_MANS = virt-v2v.1 +man_MANS = virt-v2v.1 virt-v2v-copy-to-local.1 -noinst_DATA = $(top_builddir)/website/virt-v2v.1.html +noinst_DATA = \ + $(top_builddir)/website/virt-v2v.1.html \ + $(top_builddir)/website/virt-v2v-copy-to-local.1.html virt-v2v.1 $(top_builddir)/website/virt-v2v.1.html: stamp-virt-v2v.pod @@ -206,9 +249,21 @@ stamp-virt-v2v.pod: virt-v2v.pod $< touch $@ +virt-v2v-copy-to-local.1 $(top_builddir)/website/virt-v2v-copy-to-local.1.html: stamp-virt-v2v-copy-to-local.pod + +stamp-virt-v2v-copy-to-local.pod: virt-v2v-copy-to-local.pod + $(PODWRAPPER) \ + --man virt-v2v-copy-to-local.1 \ + --html $(top_builddir)/website/virt-v2v-copy-to-local.1.html \ + --license GPLv2+ \ + $< + touch $@ + CLEANFILES += \ stamp-virt-v2v.pod \ - virt-v2v.1 + stamp-virt-v2v-copy-to-local.pod \ + virt-v2v.1 \ + virt-v2v-copy-to-local.1 # Tests. diff --git a/v2v/copy_to_local.ml b/v2v/copy_to_local.ml new file mode 100644 index 0000000..ed967be --- /dev/null +++ b/v2v/copy_to_local.ml @@ -0,0 +1,311 @@ +(* virt-v2v + * Copyright (C) 2009-2015 Red Hat Inc. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License along + * with this program; if not, write to the Free Software Foundation, Inc., + * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + *) + +(* The separate virt-v2v-copy-to-local tool. *) + +open Printf + +open Common_gettext.Gettext +open Common_utils + +open Utils + +type source_t = Xen_ssh | ESXi + +let rec main () = + let input_conn = ref None in + + let set_string_option_once optname optref arg = + match !optref with + | Some _ -> + error (f_"%s option used more than once on the command line") optname + | None -> + optref := Some arg + in + + (* Handle the command line. *) + let argspec = [ + "-ic", Arg.String (set_string_option_once "-ic" input_conn), + "uri " ^ s_"Libvirt URI"; + ] in + let argspec = set_standard_options argspec in + let args = ref [] in + let anon_fun s = args := s :: !args in + let usage_msg = + sprintf (f_"\ +%s: copy a remote guest to the local machine + +Copy the remote guest: + + virt-v2v-copy-to-local -ic xen+ssh://root@xen.example.com guest + + virt-v2v-copy-to-local -ic esx://esxi.example.com guest + +Then perform the conversion step: + + virt-v2v -i libvirtxml guest.xml -o local -os /var/tmp + +To clean up: + + rm guest.xml guest-disk* + +A short summary of the options is given below. For detailed help please +read the man page virt-v2v-copy-to-local(1). +") + prog in + Arg.parse argspec anon_fun usage_msg; + + let args = !args in + let input_conn = !input_conn in + + let input_conn = + match input_conn with + | None -> + error (f_"the -ic parameter is required") (* at the moment *) + | Some ic -> ic in + + (* Check this is a libvirt URI we can understand. *) + let parsed_uri = + try Xml.parse_uri input_conn + with Invalid_argument msg -> + error (f_"could not parse '-ic %s'. Original error message was: %s") + input_conn msg in + let source, server = + match parsed_uri.Xml.uri_server, parsed_uri.Xml.uri_scheme with + | Some server, Some "xen+ssh" -> (* Xen over SSH *) + Xen_ssh, server + | Some server, Some "esx" -> (* esxi over https *) + ESXi, server + + (* We can probably extend this list in future. *) + | _ -> + error (f_"only copies from VMware ESXi or Xen over SSH are supported. See the virt-v2v-copy-to-local(1) manual page.") in + + (* We expect a single extra argument, which is the guest name. *) + let guest_name = + match args with + | [] -> + error (f_"missing guest name. See the virt-v2v-copy-to-local(1) manual page.") + | [arg] -> arg + | _ -> + error (f_"too many command line parameters. See the virt-v2v-copy-to-local(1) manual page.") in + + (* Print the version, easier than asking users to tell us. *) + if verbose () then + printf "%s: %s %s (%s)\n%!" + prog Guestfs_config.package_name Guestfs_config.package_version Guestfs_config.host_cpu; + + (* Get the remote libvirt XML. *) + message (f_"Fetching the remote libvirt XML metadata ..."); + let xml = Domainxml.dumpxml ~conn:input_conn guest_name in + + if verbose () then + printf "libvirt XML from remote server:\n%s\n" xml; + + (* Get the disk remote paths from the XML. *) + message (f_"Parsing the remote libvirt XML metadata ..."); + let disks, dcpath, xml = parse_libvirt_xml guest_name xml in + + if verbose () then + printf "libvirt XML after modifying for local disks:\n%s\n" xml; + + (* For VMware ESXi source, we have to massage the disk path. *) + let disks = + match source with + | ESXi -> + List.map ( + fun (remote_disk, local_disk) -> + let url, sslverify = + VCenter.map_source_to_https dcpath parsed_uri + server remote_disk in + if verbose () then + printf "esxi: source disk %s (sslverify=%b)\n" url sslverify; + let cookie = + VCenter.get_session_cookie None "esx" parsed_uri sslverify url in + (url, local_disk, sslverify, cookie) + ) disks + | Xen_ssh -> + List.map (fun (remote_disk, local_disk) -> + (remote_disk, local_disk, false, None)) disks in + + (* Delete the disks on exit, unless we finish everything OK. *) + let delete_on_exit = ref true in + at_exit ( + fun () -> + if !delete_on_exit then ( + List.iter ( + fun (_, local_disk, _, _) -> + try Unix.unlink local_disk with _ -> () + ) disks + ) + ); + + (* Copy the disks. *) + let n = List.length disks in + iteri ( + fun i (remote_disk, local_disk, sslverify, cookie) -> + message (f_"Copying remote disk %d/%d to %s") + (i+1) n local_disk; + + (* How we copy it depends on the source. *) + match source with + | Xen_ssh -> + let { Xml.uri_server = server; uri_user = user; uri_port = port } = + parsed_uri in + + let cmd = + sprintf "set -o pipefail; ssh%s %s%s dd bs=1M if=%s | dd%s conv=sparse bs=1M of=%s" + (match port with + | n when n >= 1 -> sprintf " -p %d" n + | _ -> "") + (match user with + | None -> "" + | Some u -> sprintf "%s@" (quote u)) + (match server with + | None -> assert false + | Some server -> server) + (quote remote_disk) + (if quiet () then "" + else " status=progress") + (quote local_disk) in + if verbose () then + printf "%s\n%!" cmd; + if Sys.command cmd <> 0 then + error (f_"ssh copy command failed, see earlier errors"); + + | ESXi -> + let curl_args = [ + "url", Some remote_disk; + "output", Some local_disk; + ] in + let curl_args = + if sslverify then curl_args + else ("insecure", None) :: curl_args in + let curl_args = + match cookie with + | None -> curl_args + | Some cookie -> ("cookie", Some cookie) :: curl_args in + let curl_args = + if quiet () then ("silent", None) :: curl_args + else curl_args in + + if verbose () then + Curl.print_curl_command stdout curl_args; + ignore (Curl.run curl_args) + ) disks; + + let guest_xml = guest_name ^ ".xml" in + message (f_"Writing libvirt XML metadata to %s.xml ...") guest_xml; + let chan = open_out guest_xml in + output_string chan xml; + close_out chan; + + (* Finished, so don't delete the disks on exit. *) + message (f_"Finishing off"); + delete_on_exit := false + +(* This is a greatly simplified version of the parsing function + * in virt-v2v input_libvirtxml.ml:parse_libvirt_xml + * It also modifies the XML <disk> elements to point to local disks. + *) +and parse_libvirt_xml guest_name xml = + (* Parse the XML. *) + let doc = Xml.parse_memory xml in + let xpathctx = Xml.xpath_new_context doc in + Xml.xpath_register_ns xpathctx + "vmware" "http://libvirt.org/schemas/domain/vmware/1.0"; + let xpath_string = xpath_string xpathctx + and xpath_string_default = xpath_string_default xpathctx in + + (* Get the dcpath, only present for libvirt >= 1.2.20 so use a + * sensible default for older versions. + *) + let dcpath = + xpath_string_default "/domain/vmware:datacenterpath" "ha-datacenter" in + + (* Parse the disks. *) + let get_disks, add_disk = + let disks = ref [] and i = ref 0 in + let get_disks () = List.rev !disks in + let add_disk remote_disk = + (* Generate a unique name for each output disk. *) + incr i; + let local_disk = sprintf "%s-disk%d" guest_name !i in + + disks := (remote_disk, local_disk) :: !disks; + local_disk + in + get_disks, add_disk + in + + (* node is a <disk> node, containing a <source> element. Update the + * node to point to a local file. + *) + let update_disk_node node local_disk = + Xml.set_prop node "type" "file"; + let obj = Xml.xpath_eval_expression xpathctx "source" in + let nr_nodes = Xml.xpathobj_nr_nodes obj in + assert (nr_nodes >= 1); + for i = 0 to nr_nodes-1 do + let source_node = Xml.xpathobj_node obj i in + ignore (Xml.unset_prop source_node "dev"); + Xml.set_prop source_node "file" local_disk + done + in + + let obj = + Xml.xpath_eval_expression xpathctx + "/domain/devices/disk[@device='disk']" in + let nr_nodes = Xml.xpathobj_nr_nodes obj in + if nr_nodes < 1 then + error (f_"this guest has no non-removable disks"); + + for i = 0 to nr_nodes-1 do + let node = Xml.xpathobj_node obj i in + Xml.xpathctx_set_current_context xpathctx node; + + (* The <disk type='...'> attribute may be 'block' or 'file'. + * We ignore any other types. + *) + match xpath_string "@type" with + | None -> + warning (f_"<disk> element with no type attribute ignored") + + | Some "block" -> + (match xpath_string "source/@dev" with + | Some path -> + let local_disk = add_disk path in + update_disk_node node local_disk + | None -> () + ); + | Some "file" -> + (match xpath_string "source/@file" with + | Some path -> + let local_disk = add_disk path in + update_disk_node node local_disk + | None -> () + ); + + | Some disk_type -> + warning (f_"<disk type='%s'> was ignored") disk_type + done; + + let xml = Xml.to_string doc ~format:true in + get_disks (), dcpath, xml + +let () = run_main_and_handle_errors main diff --git a/v2v/virt-v2v-copy-to-local.pod b/v2v/virt-v2v-copy-to-local.pod new file mode 100644 index 0000000..85e6d4e --- /dev/null +++ b/v2v/virt-v2v-copy-to-local.pod @@ -0,0 +1,206 @@ +=head1 NAME + +virt-v2v-copy-to-local - Copy a remote guest to the local machine + +=head1 SYNOPSIS + + virt-v2v-copy-to-local -ic LIBVIRT_URI GUEST + + virt-v2v-copy-to-local -ic xen+ssh://root@xen.example.com xen_guest + + virt-v2v-copy-to-local -ic esx://root@esxi.example.com vmware_guest + +=head1 DESCRIPTION + +C<virt-v2v-copy-to-local> copies a guest from a remote hypervisor to +the local machine, in preparation for conversion by L<virt-v2v(1)>. +Note this tool alone B<does not> do the virt-v2v conversion. + +=head2 When to use this tool + +This tool is not usually necessary, but there are a few special cases +(see list below) where you might need it. + +If your case does not fit one of these special cases, then ignore this +tool and read L<virt-v2v(1)> instead. The virt-v2v-copy-to-local +process is slower than using virt-v2v directly, because it has to copy +unused parts of the guest disk. + +=over 4 + +=item * + +You have a Xen guest using host block devices. Virt-v2v cannot +convert such guests directly. + +See L<virt-v2v(1)/XEN OR SSH CONVERSIONS FROM BLOCK DEVICES>. + +=item * + +You have VMware ESXi hypervisors, and are not using VMware vCenter to +manage them. Virt-v2v cannot directly access ESXi hypervisor, so you +either have to export the guest as an OVA (eg. using VMware's +C<ovftool>); or you can use this tool to copy the guest to a local +file on the conversion server, from where virt-v2v will be able to +access it. + +=back + +=head2 How this tool works + +This tool uses libvirt to get the libvirt XML (metadata) of the remote +guest, essentially equivalent to running C<virsh dumpxml guest>. + +It then uses the XML to locate the remote guest disks, which are +copied over using a hypervisor-specific method. It uses ssh for +remote Xen hypervisors, and HTTPS (curl) for remote ESXi hypervisors. + +It then modifies the libvirt XML so that it points at the local copies +of the guest disks. + +The libvirt XML is output to a file called F<guest.xml> (where +I<guest> is the name of the guest). The disk(s) are output to file(s) +called F<guest-disk1>, F<guest-disk2> and so on. + +After copying the guest locally, you can convert it using: + + virt-v2v -i libvirtxml guest.xml [-o options ...] + +Virt-v2v finds the local copies of the disks by looking in the XML. + +=head1 EXAMPLES + +=head2 Copy and convert from Xen hypervisor that uses host block devices + + virt-v2v-copy-to-local -ic xen+ssh://root@xen.example.com xen_guest + virt-v2v -i libvirtxml xen_guest.xml -o local -os /var/tmp + rm xen_guest.xml xen_guest-disk* + +=head2 Copy and convert from ESXi hypervisor + + virt-v2v-copy-to-local -ic esx://root@esxi.example.com?no_verify=1 vmware_guest + virt-v2v -i libvirtxml vmware_guest.xml -o local -os /var/tmp + rm vmware_guest.xml vmware_guest-disk* + +=head1 COPYING FROM XEN HYPERVISOR + +=head2 XEN: LIBVIRT URI + +The libvirt URI for remote Xen hosts will look something like this: + + xen+ssh://root@xen.example.com + +The remote Xen server must allow root logins over ssh. + +To test it and list the remote guests available, use L<virsh(1)>: + + $ virsh -c xen+ssh://root@xen.example.com list --all + Id Name State + ---------------------------------------------------- + 0 Domain-0 running + - guest shut off + +=head2 XEN: COPY THE GUEST + +Using the libvirt URI as the I<-ic> option, copy one of the guests to +the local machine: + + $ virt-v2v-copy-to-local -ic xen+ssh://root@xen.example.com guest + +This creates F<guest.xml>, F<guest-disk1>, ... + +=head2 XEN: DO THE CONVERSION + +Perform the conversion of the guest using virt-v2v: + + $ virt-v2v -i libvirtxml guest.xml -o local -os /var/tmp + +=head2 XEN: CLEAN UP + +Remove the F<guest.xml> and F<guest-disk*> files. + +=head1 COPYING FROM VMware ESXi HYPERVISOR + +=head2 ESXi: LIBVIRT URI + +The libvirt URI for VMware ESXi hypervisors will look something like this: + + esx://root@esxi.example.com?no_verify=1 + +The C<?no_verify=1> parameter disables TLS certificate checking. + +To test it and list the remote guests available, use L<virsh(1)>: + + $ virsh -c esx://root@esxi.example.com?no_verify=1 list --all + Enter root's password for esxi.example.com: *** + Id Name State + ---------------------------------------------------- + - guest shut off + +=head2 ESXi: COPY THE GUEST + +Using the libvirt URI as the I<-ic> option, copy one of the guests to +the local machine: + + $ virt-v2v-copy-to-local -ic esx://root@esxi.example.com?no_verify=1 guest + +This creates F<guest.xml>, F<guest-disk1>, ... + +=head2 ESXi: DO THE CONVERSION + +Perform the conversion of the guest using virt-v2v: + + $ virt-v2v -i libvirtxml guest.xml -o local -os /var/tmp + +=head2 ESXi: CLEAN UP + +Remove the F<guest.xml> and F<guest-disk*> files. + +=head1 OPTIONS + +=over 4 + +=item B<--help> + +Display help. + +=item B<-ic> libvirtURI + +Specify a libvirt connection URI + +=item B<-q> + +=item B<--quiet> + +This disables progress bars and other unnecessary output. + +=item B<-v> + +=item B<--verbose> + +Enable verbose messages for debugging. + +=item B<-V> + +=item B<--version> + +Display version number and exit. + +=back + +=head1 SEE ALSO + +L<virt-v2v(1)>, +L<virsh(1)>, +L<http://libguestfs.org/>, +L<https://libvirt.org/uri.html>, +L<https://libvirt.org/remote.html>, +L<https://libvirt.org/drvesx.html>. + +=head1 AUTHORS + +Richard W.M. Jones L<http://people.redhat.com/~rjones/> + +=head1 COPYRIGHT + +Copyright (C) 2009-2015 Red Hat Inc. diff --git a/v2v/virt-v2v.pod b/v2v/virt-v2v.pod index 0b7be7d..0aa0c56 100644 --- a/v2v/virt-v2v.pod +++ b/v2v/virt-v2v.pod @@ -1177,6 +1177,19 @@ directory containing the files: $ virt-v2v -i ova /path/to/files -o local -os /var/tmp +=head1 INPUT FROM VMWARE ESXi HYPERVISOR + +Virt-v2v cannot access an ESXi hypervisor directly. + +However you can use the L<virt-v2v-copy-to-local(1)> tool to copy the +guest off the hypervisor into a local file, and then convert it: + + virt-v2v-copy-to-local -ic esx://esxi.example.com vmware_guest + virt-v2v -i libvirtxml vmware_guest.xml [-o options ...] + rm vmware_guest.xml vmware_guest-disk* + +See L<virt-v2v-copy-to-local(1)> for further information. + =head1 INPUT FROM RHEL 5 XEN Virt-v2v is able to import Xen guests from RHEL 5 Xen hosts. @@ -1269,53 +1282,9 @@ The workaround is to copy the guest over to the conversion server. You will need sufficient space on the conversion server to store a full copy of the guest. -=over 4 - -=item 1. - -From the conversion host, dump the guest XML to a local file: - - virsh -c xen+ssh://root@xen.example.com dumpxml guest > guest.xml - -=item 2. - -From the conversion host, copy the guest disk(s) over. You may need -to read the C<E<lt>diskE<gt>> sections from the guest XML to find the -names of the guest disks. - - ssh root@xen.example.com 'dd if=/dev/VG/guest' | dd conv=sparse of=guest-disk1 - -Notice C<conv=sparse> which adds sparseness, so the copied disk will -use as little space as possible. - -=item 3. - -Edit the guest XML file, replacing C<E<lt>diskE<gt>> section(s) that -refer to the remote disks with references to the local copies. - -Three changes have to be made. Firstly in: - - <disk type='block' device='disk'> - -the C<type> must be changed to C<file>: - - <disk type='file' device='disk'> - -The last two changes are in the C<E<lt>sourceE<gt>> line where: - - <source dev='/dev/VG/guest'/> - -C<source dev=> becomes C<source file=> pointing at the local file: - - <source file='guest-disk1'/> - -=item 4. - -Convert the guest using C<virt-v2v -i libvirtxml> mode like this: - - virt-v2v -i libvirtxml guest.xml [-o options as usual ...] - -=back + virt-v2v-copy-to-local -ic xen+ssh://root@xen.example.com guest + virt-v2v -i libvirtxml guest.xml [-o options ...] + rm guest.xml guest-disk* =head1 OUTPUT TO LIBVIRT @@ -1786,6 +1755,14 @@ For other environment variables, see L<guestfs(3)/ENVIRONMENT VARIABLES>. =over 4 +=item L<virt-v2v-copy-to-local(1)> + +There are some special cases where virt-v2v cannot directly access the +remote hypervisor. In that case you have to use +L<virt-v2v-copy-to-local(1)> to make a local copy of the guest first, +followed by running C<virt-v2v -i libvirtxml> to perform the +conversion. + =item L<engine-image-uploader(8)> Variously called C<engine-image-uploader>, C<ovirt-image-uploader> or @@ -1816,6 +1793,7 @@ L<guestfs(3)>, L<guestfish(1)>, L<qemu-img(1)>, L<fstrim(8)>, +L<virt-v2v-copy-to-local(1)>, L<virt-v2v-test-harness(1)>, L<engine-image-uploader(8)>, L<import-to-ovirt.pl|http://git.annexia.org/?p=import-to-ovirt.git>, -- 2.5.0
participants (1)
-
Richard W.M. Jones