May 2020 - Libguestfs - libguestfs List Archives

[PATCH 0/7] Support Windows BitLocker (RHBZ#1808977).
by Richard W.M. Jones 27 Jul '20

27 Jul '20

These commits, along with the associated changes to common: https://www.redhat.com/archives/libguestfs/2020-March/msg00286.html support the transparent decryption and inspection of Windows guests encrypted with BitLocker encryption. To do the BitLocker decryption requires cryptsetup 2.3.0 (although cryptsetup 2.3 is not required for existing LUKS use). It also requires a new-ish Linux kernel, I believe 5.3+ This starts by adding new APIs cryptsetup-open and cryptsetup-close. As well as dropping the "luks-" prefix, this allows specifying an optional type parameter, allowing you to select BitLocker encryption. Although also the new API does not require this parameter, and will autodetect the correct type of decryption to apply in every known case. The main work is then in updating list-filesystems and ensuring that inspection still works. This really needs but also lacks tests. See commit message of patch 1/7 for why. I believe it's impossible for us to prove unit testing right now, but hopefully this situation can improve in future. Rich.

2 9

[PATCH common 0/4] options: Support Windows BitLocker (RHBZ#1808977).
by Richard W.M. Jones 27 Jul '20

27 Jul '20

Support transparent decryption/inspection of Windows guests encrypted with BitLocker encryption. This won't make much sense without the associated libguestfs patches which I will post momentarily. (Submodules, ho hum) Rich.

2 6

[PATCH v3] python: Fix UnicodeError in inspect_list_applications2() (RHBZ#1684004)
by Sam Eiderman 06 Jul '20

06 Jul '20

The python3 bindings create PyUnicode objects from application strings on the guest (i.e. installed rpm, deb packages). It is documented that rpm package fields such as description should be utf8 encoded - however in some cases they are not a valid unicode string, on SLES11 SP4 the encoding of the description of the following packages is latin1 and they fail to be converted to unicode using guestfs_int_py_fromstring() (which invokes PyUnicode_FromString()): PackageKit aaa_base coreutils dejavu desktop-data-SLED gnome-utils hunspell hunspell-32bit hunspell-tools libblocxx6 libexif libgphoto2 libgtksourceview-2_0-0 libmpfr1 libopensc2 libopensc2-32bit liborc-0_4-0 libpackagekit-glib10 libpixman-1-0 libpixman-1-0-32bit libpoppler-glib4 libpoppler5 libsensors3 libtelepathy-glib0 m4 opensc opensc-32bit permissions pinentry poppler-tools python-gtksourceview splashy syslog-ng tar tightvnc xorg-x11 xorg-x11-xauth yast2-mouse Fix this by globally changing guestfs_int_py_fromstring() and guestfs_int_py_fromstringsize() to fallback to latin1 decoding if utf-8 decoding fails. Using the "strict" error handler doesn't matter in the case of latin1 and has the same effect of "replace": https://docs.python.org/3/library/codecs.html#error-handlers Signed-off-by: Sam Eiderman <sameid(a)google.com> --- python/handle.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/python/handle.c b/python/handle.c index 2fb8c18f0..fe89dc58a 100644 --- a/python/handle.c +++ b/python/handle.c @@ -387,7 +387,7 @@ guestfs_int_py_fromstring (const char *str) #if PY_MAJOR_VERSION < 3 return PyString_FromString (str); #else - return PyUnicode_FromString (str); + return guestfs_int_py_fromstringsize (str, strlen (str)); #endif } @@ -397,7 +397,12 @@ guestfs_int_py_fromstringsize (const char *str, size_t size) #if PY_MAJOR_VERSION < 3 return PyString_FromStringAndSize (str, size); #else - return PyUnicode_FromStringAndSize (str, size); + PyObject *s = PyUnicode_FromString (str); + if (s == NULL) { + PyErr_Clear (); + s = PyUnicode_Decode (str, strlen(str), "latin1", "strict"); + } + return s; #endif } -- 2.26.2.303.gf8c07b1a785-goog

3 11

[PATCH libguestfs-common 1/2] mlcustomize: Refactor SELinux_relabel code.
by Richard W.M. Jones 24 Jun '20

24 Jun '20

This shouldn't change the effect of this code. --- mlcustomize/SELinux_relabel.ml | 121 ++++++++++++++++++--------------- 1 file changed, 65 insertions(+), 56 deletions(-) diff --git a/mlcustomize/SELinux_relabel.ml b/mlcustomize/SELinux_relabel.ml index 44995df..5df1f08 100644 --- a/mlcustomize/SELinux_relabel.ml +++ b/mlcustomize/SELinux_relabel.ml @@ -28,65 +28,74 @@ module G = Guestfs let array_find a l = List.mem a (Array.to_list l) -let relabel (g : G.guestfs) = - (* Is the guest using SELinux? *) - if g#is_file ~followsymlinks:true "/usr/sbin/load_policy" && - g#is_file ~followsymlinks:true "/etc/selinux/config" then ( - (* Is setfiles / SELinux relabelling functionality available? *) - if g#feature_available [| "selinuxrelabel" |] then ( - (* Use Augeas to parse /etc/selinux/config. *) - g#aug_init "/" (16+32) (* AUG_SAVE_NOOP | AUG_NO_LOAD *); - (* See: https://bugzilla.redhat.com/show_bug.cgi?id=975412#c0 *) - ignore (g#aug_rm "/augeas/load/*[\"/etc/selinux/config/\" !~ regexp('^') + glob(incl) + regexp('/.*')]"); - g#aug_load (); - debug_augeas_errors g; - - (* Get the SELinux policy name, eg. "targeted", "minimum". - * Use "targeted" if not specified, just like libselinux does. +let rec relabel (g : G.guestfs) = + (* Is the guest using SELinux? (Otherwise this is a no-op). *) + if is_selinux_guest g then ( + try + use_setfiles g; + (* That worked, so we don't need to autorelabel. *) + g#rm_f "/.autorelabel" + with Failure _ -> + (* This is the fallback in case something in the setfiles + * method didn't work. That includes the case where a non-SELinux + * host is processing an SELinux guest, and other things. *) - let policy = - let config_path = "/files/etc/selinux/config" in - let selinuxtype_path = config_path ^ "/SELINUXTYPE" in - let keys = g#aug_ls config_path in - if array_find selinuxtype_path keys then - g#aug_get selinuxtype_path - else - "targeted" in + g#touch "/.autorelabel" + ) - g#aug_close (); +and is_selinux_guest g = + g#is_file ~followsymlinks:true "/usr/sbin/load_policy" && + g#is_file ~followsymlinks:true "/etc/selinux/config" - (* Get the spec file name. *) - let specfile = - sprintf "/etc/selinux/%s/contexts/files/file_contexts" policy in +and use_setfiles g = + (* Is setfiles / SELinux relabelling functionality available? *) + if not (g#feature_available [| "selinuxrelabel" |]) then + failwith "no selinux relabel feature"; - (* RHEL 6.2 - 6.5 had a malformed specfile that contained the - * invalid regular expression "/var/run/spice-vdagentd.\pid" - * (instead of "\.p"). This stops setfiles from working on - * the guest. - * - * Because an SELinux relabel writes all over the filesystem, - * it seems reasonable to fix this problem in the specfile - * at the same time. (RHBZ#1374232) - *) - if g#grep ~fixed:true "vdagentd.\\pid" specfile <> [||] then ( - debug "fixing invalid regular expression in %s" specfile; - let old_specfile = specfile ^ "~" in - g#mv specfile old_specfile; - let content = g#read_file old_specfile in - let content = - String.replace content "vdagentd.\\pid" "vdagentd\\.pid" in - g#write specfile content; - g#copy_attributes ~all:true old_specfile specfile - ); + (* Use Augeas to parse /etc/selinux/config. *) + g#aug_init "/" (16+32) (* AUG_SAVE_NOOP | AUG_NO_LOAD *); + (* See: https://bugzilla.redhat.com/show_bug.cgi?id=975412#c0 *) + ignore (g#aug_rm "/augeas/load/*[\"/etc/selinux/config/\" !~ regexp('^') + glob(incl) + regexp('/.*')]"); + g#aug_load (); + debug_augeas_errors g; - (* Relabel everything. *) - g#selinux_relabel ~force:true specfile "/"; + (* Get the SELinux policy name, eg. "targeted", "minimum". + * Use "targeted" if not specified, just like libselinux does. + *) + let policy = + let config_path = "/files/etc/selinux/config" in + let selinuxtype_path = config_path ^ "/SELINUXTYPE" in + let keys = g#aug_ls config_path in + if array_find selinuxtype_path keys then + g#aug_get selinuxtype_path + else + "targeted" in - (* If that worked, we don't need to autorelabel. *) - g#rm_f "/.autorelabel" - ) - else ( - (* SELinux guest, but not SELinux host. Fallback to this. *) - g#touch "/.autorelabel" - ) - ) + g#aug_close (); + + (* Get the spec file name. *) + let specfile = + sprintf "/etc/selinux/%s/contexts/files/file_contexts" policy in + + (* RHEL 6.2 - 6.5 had a malformed specfile that contained the + * invalid regular expression "/var/run/spice-vdagentd.\pid" + * (instead of "\.p"). This stops setfiles from working on + * the guest. + * + * Because an SELinux relabel writes all over the filesystem, + * it seems reasonable to fix this problem in the specfile + * at the same time. (RHBZ#1374232) + *) + if g#grep ~fixed:true "vdagentd.\\pid" specfile <> [||] then ( + debug "fixing invalid regular expression in %s" specfile; + let old_specfile = specfile ^ "~" in + g#mv specfile old_specfile; + let content = g#read_file old_specfile in + let content = + String.replace content "vdagentd.\\pid" "vdagentd\\.pid" in + g#write specfile content; + g#copy_attributes ~all:true old_specfile specfile + ); + + (* Relabel everything. *) + g#selinux_relabel ~force:true specfile "/" -- 2.24.1

2 5

[PATCH virt-v2v] v2v: Allow temporary directory to be set on a global basis.
by Richard W.M. Jones 17 Jun '20

17 Jun '20

Previously we placed large files in g#get_cachedir () (usually /var/tmp). However the problem is this ties the libguestfs appliance and the virt-v2v overlay files to the same location. When virt-v2v is run in a container, or any other situation where local storage is limited, it's helpful to be able to put the overlay files on an externally mounted PVC, which might be using NFS and shared between containers. But putting the libguestfs appliance on NFS in a shared location is certainly not recommended. This allows the two locations to be set separately: VIRT_V2V_TMPDIR - location of large temporary files, can use NFS and may be shared LIBGUESTFS_CACHEDIR - location of libguestfs appliance Another motivation for this patch is to allow more reliable cleanup of temporary files by an external process, as described in the updated documentation. --- docs/virt-v2v.pod | 21 +++++++++++++++++---- v2v/input_ova.ml | 4 ++-- v2v/input_vmx.ml | 3 +-- v2v/nbdkit.ml | 3 +-- v2v/output_glance.ml | 3 +-- v2v/output_null.ml | 5 ++--- v2v/output_rhv_upload.ml | 3 +-- v2v/parse_ova.ml | 5 ++--- v2v/python_script.ml | 3 +-- v2v/utils.ml | 8 ++++++-- v2v/utils.mli | 4 ++++ v2v/v2v.ml | 15 ++++++--------- 12 files changed, 44 insertions(+), 33 deletions(-) diff --git a/docs/virt-v2v.pod b/docs/virt-v2v.pod index ed95fdc8e..dbfd10cad 100644 --- a/docs/virt-v2v.pod +++ b/docs/virt-v2v.pod @@ -1206,8 +1206,9 @@ possible. =head3 Disk space -Virt-v2v places potentially large temporary files in C<$TMPDIR> (which -is F</var/tmp> if you don't set it). Using tmpfs is a bad idea. +Virt-v2v places potentially large temporary files in +C<$VIRT_V2V_TMPDIR> (or F</var/tmp>, see also L</ENVIRONMENT VARIBLES> +below). Using tmpfs is a bad idea. For each guest disk, an overlay is stored temporarily. This stores the changes made during conversion, and is used as a cache. The @@ -1554,10 +1555,22 @@ conversion. =over 4 +=item C<VIRT_V2V_TMPDIR> + +=item C<LIBGUESTFS_CACHEDIR> + =item C<TMPDIR> -Location of the temporary directory used for the potentially large -temporary overlay file. +Location of the temporary directory. This is used for the potentially +large temporary overlay files. Also for miscellaneous temporary +files. These environment variables are checked in order: +C<VIRT_V2V_TMPDIR>, C<LIBGUESTFS_CACHEDIR>, C<TMPDIR>. If none are +set then F</var/tmp> is used. + +To reliably ensure temporary files are cleaned up (for example in case +virt-v2v crashes) you should create a randomly named directory under +F</var/tmp>, set C<VIRT_V2V_TMPDIR> to point to this directory, then +when virt-v2v exits remove the directory. See the L</Disk space> section above. diff --git a/v2v/input_ova.ml b/v2v/input_ova.ml index 5d3bece18..86f81e1f9 100644 --- a/v2v/input_ova.ml +++ b/v2v/input_ova.ml @@ -132,8 +132,8 @@ class input_ova ova = object (* The spec allows the file to be gzip-compressed, in * which case we must uncompress it into a temporary. *) - let temp_dir = (open_guestfs ())#get_cachedir () in - let new_filename = Filename.temp_file ~temp_dir "ova" ".vmdk" in + let new_filename = + Filename.temp_file ~temp_dir:Utils.tmpdir "ova" ".vmdk" in unlink_on_exit new_filename; let cmd = sprintf "zcat %s > %s" (quote filename) (quote new_filename) in diff --git a/v2v/input_vmx.ml b/v2v/input_vmx.ml index f1d143e97..5a1804182 100644 --- a/v2v/input_vmx.ml +++ b/v2v/input_vmx.ml @@ -389,8 +389,7 @@ and find_nics vmx = class input_vmx input_password input_transport arg = let tmpdir = - let base_dir = (open_guestfs ())#get_cachedir () in - let t = Mkdtemp.temp_dir ~base_dir "vmx." in + let t = Mkdtemp.temp_dir ~base_dir:tmpdir "vmx." in rmdir_on_exit t; t in object diff --git a/v2v/nbdkit.ml b/v2v/nbdkit.ml index 65317f9b9..188647f65 100644 --- a/v2v/nbdkit.ml +++ b/v2v/nbdkit.ml @@ -105,8 +105,7 @@ let add_filter_if_available cmd filter = let run_unix cmd = (* Create a temporary directory where we place the socket and PID file. *) let tmpdir = - let base_dir = (open_guestfs ())#get_cachedir () in - let t = Mkdtemp.temp_dir ~base_dir "v2vnbdkit." in + let t = Mkdtemp.temp_dir ~base_dir:tmpdir "v2vnbdkit." in (* tmpdir must be readable (but not writable) by "other" so that * qemu can open the sockets. *) diff --git a/v2v/output_glance.ml b/v2v/output_glance.ml index 0a9e91818..fce223333 100644 --- a/v2v/output_glance.ml +++ b/v2v/output_glance.ml @@ -33,8 +33,7 @@ class output_glance () = * to write to a temporary file. XXX *) let tmpdir = - let base_dir = (open_guestfs ())#get_cachedir () in - let t = Mkdtemp.temp_dir ~base_dir "glance." in + let t = Mkdtemp.temp_dir ~base_dir:tmpdir "glance." in rmdir_on_exit t; t in object diff --git a/v2v/output_null.ml b/v2v/output_null.ml index 3528da50a..5ce30d557 100644 --- a/v2v/output_null.ml +++ b/v2v/output_null.ml @@ -50,7 +50,7 @@ let can_use_qemu_null_co_device () = (* We actually attempt to convert a raw file to the null-co device * using a JSON URL. *) - let tmp = Filename.temp_file "v2vqemunullcotst" ".img" in + let tmp = Filename.temp_file ~temp_dir:tmpdir "v2vqemunullcotst" ".img" in Unix.truncate tmp 1024; let json = [ @@ -75,8 +75,7 @@ class output_null = * the null-co device w/ a JSON URL. *) let tmpdir = - let base_dir = (open_guestfs ())#get_cachedir () in - let t = Mkdtemp.temp_dir ~base_dir "null." in + let t = Mkdtemp.temp_dir ~base_dir:tmpdir "null." in rmdir_on_exit t; t in object diff --git a/v2v/output_rhv_upload.ml b/v2v/output_rhv_upload.ml index 5c6c26110..4922a2040 100644 --- a/v2v/output_rhv_upload.ml +++ b/v2v/output_rhv_upload.ml @@ -155,8 +155,7 @@ class output_rhv_upload output_alloc output_conn rhv_options = (* Create a temporary directory which will be deleted on exit. *) let tmpdir = - let base_dir = (open_guestfs ())#get_cachedir () in - let t = Mkdtemp.temp_dir ~base_dir "rhvupload." in + let t = Mkdtemp.temp_dir ~base_dir:tmpdir "rhvupload." in rmdir_on_exit t; t in diff --git a/v2v/parse_ova.ml b/v2v/parse_ova.ml index 0b939ac43..1f332a82c 100644 --- a/v2v/parse_ova.ml +++ b/v2v/parse_ova.ml @@ -71,8 +71,7 @@ let rec parse_ova ova = if is_directory ova then ova, Directory else ( let tmpdir = - let base_dir = (open_guestfs ())#get_cachedir () in - let t = Mkdtemp.temp_dir ~base_dir "ova." in + let t = Mkdtemp.temp_dir ~base_dir:tmpdir "ova." in rmdir_on_exit t; t in @@ -222,7 +221,7 @@ and uncompress_head format file = and uncompressed_type format file = let head, headlen = uncompress_head format file in let tmpfile, chan = - Filename.open_temp_file "ova.file." "" in + Filename.open_temp_file ~temp_dir:tmpdir "ova.file." "" in output chan head 0 headlen; close_out chan; let ret = detect_file_type tmpfile in diff --git a/v2v/python_script.ml b/v2v/python_script.ml index 33c5e9a21..2d95f4bd5 100644 --- a/v2v/python_script.ml +++ b/v2v/python_script.ml @@ -35,8 +35,7 @@ let create ?(name = "script.py") ?tmpdir code = let tmpdir = match tmpdir with | None -> - let base_dir = (open_guestfs ())#get_cachedir () in - let t = Mkdtemp.temp_dir ~base_dir "v2v." in + let t = Mkdtemp.temp_dir ~base_dir:Utils.tmpdir "v2v." in rmdir_on_exit t; t | Some dir -> dir in diff --git a/v2v/utils.ml b/v2v/utils.ml index ccbb9d68a..8cc24d6a2 100644 --- a/v2v/utils.ml +++ b/v2v/utils.ml @@ -24,6 +24,10 @@ open Std_utils open Tools_utils open Common_gettext.Gettext +let tmpdir = + try Sys.getenv "VIRT_V2V_TMPDIR" + with Not_found -> (open_guestfs ())#get_cachedir () + (* Is SELinux enabled and enforcing on the host? *) let have_selinux = 0 = Sys.command "getenforce 2>/dev/null | grep -isq Enforcing" @@ -112,7 +116,8 @@ let qemu_img_supports_offset_and_size () = (* We actually attempt to create a qcow2 file with a raw backing * file that has an offset and size. *) - let tmp = Filename.temp_file "v2vqemuimgtst" ".img" in + let tmp = Filename.temp_file ~temp_dir:tmpdir "v2vqemuimgtst" ".img" in + unlink_on_exit tmp; Unix.truncate tmp 1024; let json = [ @@ -132,7 +137,6 @@ let qemu_img_supports_offset_and_size () = (if verbose () then "" else " 2>&1") in debug "%s" cmd; let r = 0 = Sys.command cmd in - Unix.unlink tmp; debug "qemu-img supports \"offset\" and \"size\" in json URLs: %b" r; r diff --git a/v2v/utils.mli b/v2v/utils.mli index 937e2b9b8..5afe2954f 100644 --- a/v2v/utils.mli +++ b/v2v/utils.mli @@ -18,6 +18,10 @@ (** Utilities used in virt-v2v only. *) +val tmpdir : string +(** [VIRT_V2V_TMPDIR] or [/var/tmp]. Create all temporary files + under this directory. *) + val have_selinux : bool (** True if SELinux is enabled and enforcing on the host. *) diff --git a/v2v/v2v.ml b/v2v/v2v.ml index 73edff2c4..6dd5fc693 100644 --- a/v2v/v2v.ml +++ b/v2v/v2v.ml @@ -264,8 +264,6 @@ and set_source_networks_and_bridges cmdline source = let nics = List.map (Networks.map cmdline.network_map) source.s_nics in { source with s_nics = nics } -and overlay_dir = (open_guestfs ())#get_cachedir () - (* Conversion can fail or hang if there is insufficient free space in * the temporary directory used to store overlays on the host * (RHBZ#1316479). Although only a few hundred MB is actually @@ -273,12 +271,12 @@ and overlay_dir = (open_guestfs ())#get_cachedir () * guestfs appliance which is also stored here. *) and check_host_free_space () = - let free_space = StatVFS.free_space (StatVFS.statvfs overlay_dir) in - debug "check_host_free_space: overlay_dir=%s free_space=%Ld" - overlay_dir free_space; + let free_space = StatVFS.free_space (StatVFS.statvfs tmpdir) in + debug "check_host_free_space: tmpdir=%s free_space=%Ld" + tmpdir free_space; if free_space < 1_073_741_824L then error (f_"insufficient free space in the conversion server temporary directory %s (%s).\n\nEither free up space in that directory, or set the LIBGUESTFS_CACHEDIR environment variable to point to another directory with more than 1GB of free space.\n\nSee also the virt-v2v(1) manual, section \"Minimum free space check in the host\".") - overlay_dir (human_size free_space) + tmpdir (human_size free_space) (* Create a qcow2 v3 overlay to protect the source image(s). *) and create_overlays source_disks = @@ -286,7 +284,7 @@ and create_overlays source_disks = List.mapi ( fun i ({ s_qemu_uri = qemu_uri; s_format = format } as source) -> let overlay_file = - Filename.temp_file ~temp_dir:overlay_dir "v2vovl" ".qcow2" in + Filename.temp_file ~temp_dir:tmpdir "v2vovl" ".qcow2" in unlink_on_exit overlay_file; (* There is a specific reason to use the newer qcow2 variant: @@ -822,8 +820,7 @@ and actual_target_size target_file disk_stats = and preserve_overlays overlays src_name = List.iter ( fun ov -> - let saved_filename = - sprintf "%s/%s-%s.qcow2" overlay_dir src_name ov.ov_sd in + let saved_filename = sprintf "%s/%s-%s.qcow2" tmpdir src_name ov.ov_sd in rename ov.ov_overlay_file saved_filename; info (f_"Overlay saved as %s [--debug-overlays]") saved_filename ) overlays -- 2.25.0

4 23

[PATCH] v2v: fix UEFI bootloader for linux guests
by Denis Plotnikov 06 Jun '20

06 Jun '20

Not all UEFI guests can survive conversion, because of lost bootloader information in UEFI NVRAM. But some guest can cope with this because they have a fallback bootloader and use UEFI Removable Media Boot Behavior. (see https://uefi.org/sites/default/files/resources/UEFI_Spec_2_8_A_Feb14.pdf 3.5.1.1 Removable Media Boot Behavior) to load. If UEFI firmware can't find a bootloader in its settings it uses the removable media boot behavior to find a bootloader. We can fix the guests which don't have such a fallback loader by providing a temporary one. This bootloader is used for the first boot only, then the conversion script restores the initial bootloader settings and removes the temporary loader. Signed-off-by: Denis Plotnikov <dplotnikov(a)virtuozzo.com> --- v2v/convert_linux.ml | 15 +++++ v2v/linux_bootloaders.ml | 149 ++++++++++++++++++++++++++++++++++++++++++++-- v2v/linux_bootloaders.mli | 2 + 3 files changed, 162 insertions(+), 4 deletions(-) diff --git a/v2v/convert_linux.ml b/v2v/convert_linux.ml index e91ae12..77a2555 100644 --- a/v2v/convert_linux.ml +++ b/v2v/convert_linux.ml @@ -1122,6 +1122,21 @@ shutdown -h -P +0 Linux.augeas_reload g ); + (* Some linux uefi setups can't boot after conversion because of + lost uefi boot entries. The uefi boot entries are stored in uefi + NVRAM. The NVRAM content isn't a part of vm disk content and + usualy can't be converted with a vm. If a vm doesn't have uefi + fallback path (/EFI/BOOT/BOOT<arch>.efi), the vm is unbootable + after conversion. The following function will try to make an uefi + fallback path if the vm being converted is an uefi setup. + *) + + let efi_fix_script = bootloader#fix_efi_boot () in + + if efi_fix_script <> "" then + Firstboot.add_firstboot_script g inspect.i_root + "fix uefi boot" efi_fix_script; + (* Delete blkid caches if they exist, since they will refer to the old * device names. blkid will rebuild these on demand. * diff --git a/v2v/linux_bootloaders.ml b/v2v/linux_bootloaders.ml index de3d107..cdab7bf 100644 --- a/v2v/linux_bootloaders.ml +++ b/v2v/linux_bootloaders.ml @@ -36,6 +36,7 @@ class virtual bootloader = object method virtual configure_console : unit -> unit method virtual remove_console : unit -> unit method update () = () + method virtual fix_efi_boot : unit -> string end (* Helper function for SUSE: remove (hdX,X) prefix from a path. *) @@ -43,6 +44,115 @@ let remove_hd_prefix = let rex = PCRE.compile "^\$hd.*\$" in PCRE.replace rex "" +(* Standard uefi fallback path *) +let uefi_fallback_path = + "/boot/efi/EFI/BOOT/" + +(* Helper function checks if 'source' contains 's' *) +let contains source s = + let re = Str.regexp_string s in + try + ignore (Str.search_forward re source 0); + true + with Not_found -> false + +(* Helper function to get architecture suffixes for uefi files *) +let get_uefi_arch_suffix arch = + let arch_suffix = + if contains arch "x86_64" then "X64" + else if contains arch "x86_32" then "X32" + else "" in + arch_suffix + +(* Function fixes uefi boot. It's used in both cases: legacy grub and grub2 *) +let fix_uefi g distro distro_ver grub_config arch = + let cant_fix_uefi () = ( + info (f_"Can't fix UEFI bootloader. VM may not boot."); + "" ) in + + let file_exists file = + if g#exists file then + true + else ( + info (f_"Can't find file: '%s' needed for UEFI bootloader fixing") file; + false ) in + + let grub_path = String.sub grub_config 0 (String.rindex grub_config '/') in + let uefi_fallback_name = + let arch_suffix = get_uefi_arch_suffix arch in + if arch_suffix <> "" then + String.concat "" [uefi_fallback_path; "BOOT"; arch_suffix; ".EFI"] + else + "" in + + if uefi_fallback_name = "" then ( + info (f_"Can't determine UEFI fallback path."); + cant_fix_uefi () ) + else if g#exists uefi_fallback_name then + (* don't do anything if uefi fallback exists *) + "" + else if uefi_fallback_name = "" then + cant_fix_uefi () + else ( + info (f_"Fixing UEFI bootloader."); + match distro, distro_ver with + | "centos", 6 -> + (* to make a bootable uefi centos 6 we need to + * copy grub.efi and grub.conf to UEFI fallback path + * and rename them to BOOT<arch>.efi and BOOT<arch>.conf + * correspondingly *) + let uefi_grub_name = String.concat "" [grub_path; "/grub.efi"] in + let uefi_grub_conf = String.concat "" [ + String.sub uefi_fallback_name 0 + (String.rindex uefi_fallback_name '.'); + ".conf" ] in + if file_exists uefi_grub_name && file_exists grub_config then ( + g#mkdir_p uefi_fallback_path; + g#cp uefi_grub_name uefi_fallback_name; + g#cp grub_config uefi_grub_conf; + let script = sprintf +"#!/bin/bash +efibootmgr -c -L \"CentOS 6\" +rm -rf %s" uefi_fallback_path in + script ) + else + cant_fix_uefi () + | "ubuntu", 14 -> + (* to make a bootable uefi ubuntu 14 we need to + * copy shim<arch>.efi to UEFI fallback path + * and rename it to BOOT<arch>.efi, also we copy + * grub.efi and grub.cfg to UEFI fallback path without renaming *) + let arch_suffix = + String.lowercase_ascii (get_uefi_arch_suffix arch) in + let shim = + String.concat "" [grub_path; "/shim"; arch_suffix; ".efi"] in + let uefi_grub_name = + String.concat "" [grub_path; "/grub"; arch_suffix; ".efi"] in + + if file_exists shim && file_exists uefi_grub_name + && file_exists grub_config then ( + g#mkdir_p uefi_fallback_path; + g#cp shim uefi_fallback_name; + g#cp uefi_grub_name uefi_fallback_path; + g#cp grub_config uefi_fallback_path; + (* if the shim is at the standard path, clean up uefi fixing + * if not, then just don't clean up and leave the temp loader + * at UEFI fallback path for simplicity + *) + if contains shim "/boot/efi/EFI/ubuntu/shim" then + sprintf +"#!/bin/bash +sudo efibootmgr -c -L ubuntu -l \\\\EFI\\\\ubuntu\\\\shim%s.efi +rm -rf %s" arch_suffix uefi_fallback_path + else + "") + else + cant_fix_uefi () + | _, _ -> + info (f_"No UEFI fix rule for %s %d") distro distro_ver; + cant_fix_uefi () + ) + (* Grub1 (AKA grub-legacy) representation. *) class bootloader_grub1 (g : G.guestfs) inspect grub_config = let () = @@ -60,6 +170,16 @@ class bootloader_grub1 (g : G.guestfs) inspect grub_config = fun path -> List.mem_assoc path mounts ) [ "/boot/grub"; "/boot" ] with Not_found -> "" in + + let uefi_active = + match inspect.i_firmware with + | I_UEFI _ -> true + | _ -> false in + + let arch = inspect.i_arch in + let distro = inspect.i_distro in + let distro_ver_major = inspect.i_major_version in + object inherit bootloader @@ -184,6 +304,12 @@ object loop paths; g#aug_save () + + method fix_efi_boot () = + if uefi_active then + fix_uefi g distro distro_ver_major grub_config arch + else + "" end (** The method used to get and set the default kernel in Grub2. *) @@ -193,7 +319,7 @@ type default_kernel_method = | MethodNone (** No known way. *) (* Grub2 representation. *) -class bootloader_grub2 (g : G.guestfs) grub_config = +class bootloader_grub2 (g : G.guestfs) inspect grub_config = let grub2_mkconfig_cmd = let elems = [ @@ -221,6 +347,15 @@ class bootloader_grub2 (g : G.guestfs) grub_config = MethodNone ) in + let uefi_active = + match inspect.i_firmware with + | I_UEFI _ -> true + | _ -> false in + + let arch = inspect.i_arch in + let distro = inspect.i_distro in + let distro_ver_major = inspect.i_major_version in + object (self) inherit bootloader @@ -340,8 +475,14 @@ object (self) method remove_console = self#grub2_update_console ~remove:true - method update () = - ignore (g#command [| grub2_mkconfig_cmd; "-o"; grub_config |]) + method update () = ( + ignore (g#command [| grub2_mkconfig_cmd; "-o"; grub_config |])) + + method fix_efi_boot () = + if uefi_active then + fix_uefi g distro distro_ver_major grub_config arch + else + "" end (* Helper type used in detect_bootloader. *) @@ -390,6 +531,6 @@ let detect_bootloader (g : G.guestfs) inspect = let bl = match typ with | Grub1 -> new bootloader_grub1 g inspect grub_config - | Grub2 -> new bootloader_grub2 g grub_config in + | Grub2 -> new bootloader_grub2 g inspect grub_config in debug "detected bootloader %s at %s" bl#name grub_config; bl diff --git a/v2v/linux_bootloaders.mli b/v2v/linux_bootloaders.mli index 30cdfe3..c4e1069 100644 --- a/v2v/linux_bootloaders.mli +++ b/v2v/linux_bootloaders.mli @@ -44,6 +44,8 @@ class virtual bootloader : object (** Update the bootloader: For grub2 only this runs the [grub2-mkconfig] command to rebuild the configuration. This is not necessary for grub-legacy. *) + method virtual fix_efi_boot : unit -> string + (** fix UEFI bootloader and return a clean up script. *) end (** Encapsulates a Linux boot loader as object. *) -- 1.8.3.1

2 1

[PATCH] v2v: convert-windows: remove installation reference for prl_strg driver
by Denis Plotnikov 05 Jun '20

05 Jun '20

There is an issue with removal of parallels disk paravirt driver. Without the patch, the VM being migrated may not boot on the desination with BSOD: 0x7b. The patch is a workaround to prevent the BSOD. Signed-off-by: Denis Plotnikov <dplotnikov(a)virtuozzo.com> --- v2v/convert_windows.ml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/v2v/convert_windows.ml b/v2v/convert_windows.ml index 4511c53c3..706202499 100644 --- a/v2v/convert_windows.ml +++ b/v2v/convert_windows.ml @@ -469,6 +469,8 @@ echo uninstalling Xen PV driver let fb_script = "\ @echo off +REG DELETE HKLM\\System\\CurrentControlSet\\Services\\prl_strg\\DriverInfo /v RefCount /f + echo uninstalling Parallels guest tools " ^ uninst ^ (* ERROR_SUCCESS_REBOOT_REQUIRED == 3010 is OK too *) -- 2.17.0

2 5

Re: [Libguestfs] Provide NBD via Browser over Websockets
by Eric Wheeler 01 Jun '20

01 Jun '20

On Mon, 15 Oct 2018, Nir Soffer wrote: > On Sat, Oct 13, 2018 at 9:45 PM Eric Wheeler <nbd(a)lists.ewheeler.net> wrote: > Hello all, > > It might be neat to attach ISOs to KVM guests via websockets. Basically > the browser would be the NBD "server" and an NBD client would run on the > hypervisor, then use `virsh change-media vm1 hdc --insert /dev/nbd0` could > use an ISO from my desk to boot from. > > Here's an HTML5 open file example: > https://stackoverflow.com/questions/3582671/how-to-open-a-local-disk-file-w… > > and the NBD protocol looks simple enough to implement in javascript: > https://stackoverflow.com/questions/17295140/where-is-the-network-block-dev… > > What do you think? Does anyone have an interest in doing this? > > > HTML File API is very limited: > - you cannot access any file except file provided by the user interactively > - no support for sparseness or underlying disk block size > > So it will be a pretty bad backend for NBD server. > > What are you trying to do? Hi Nir and Eric, I hope you are well! (I'm resurecting this old thread, not sure how I missed the replies.) We are interested in attaching a local ISO to a remote VM over http (maybe .qcow's, but ISO is the primary interest). This is common for remote KVM (iDRAC/iLO/iKVM/CIMC), so wondering about an http front-end for qemu to do the same. Combining that with a spice JS client or noVNC for VM console access would be neat. I also like Eric Blake's idea of direct NBD client integration with qemu instead of using /dev/nbd0. -Eric > > Nir > >

4 11

[PATCH v2v] v2v: -it vddk: Don't use nbdkit readahead filter with VDDK (RHBZ#1832805).
by Richard W.M. Jones 28 May '20

28 May '20

This is the simplest solution to this problem. There are two other possible fixes I considered: Increase the documented limit (see http://libguestfs.org/virt-v2v-input-vmware.1.html#vddk:-esxi-nfc-service-m…) However at the moment we know the current limit works through extensive testing (without readahead), plus I have no idea nor any way to test if larger limits are supported by all versions of VMware new and old. The limit we are recommending at the moment is the one documented by VMware. Also this would require users to change their VMware settings again, and no doubt introduce confusion for people who have already adjusted them who might not understand that they need to adjust them again for a v2v minor release. Or splitting large requests in nbdkit-vddk-plugin, but it's a bit silly to coalesce requests in a filter and then split them up at a later stage in the pipeline, not to mention error-prone when you consider multithreading etc. Rich.

2 2

[PATCH v2v] v2v: Remove extraneous '=' when setting --bandwidth/--bandwidth-file.
by Richard W.M. Jones 28 May '20

28 May '20

Trivial fix. We really need a regression test for all v2v inpus related to nbdkit. There is actually nothing at all at the moment. Of course if it was easy to test inputs over the network from a ‘make check’ rule then we'd be doing it already. I thought about adding something like a ‘-it file’ option which would use nbdkit-file-plugin for test only. However I'm rather cautious about adding new virt-v2v command line features, especially dubious do-nothing features, when my longer term plan is to break apart virt-v2v into more easily consumable pieces. That will only make the long term plan more difficult. We could probably test ‘-it ssh’ using the same trick that nbdkit uses (https://github.com/libguestfs/nbdkit/blob/master/tests/web-server.h#L33) but even that is not straightforward as ‘-it ssh’ also wants to run scp commands. So that's a long way of saying this fixes the code but doesn't add a regression test. Rich.

2 2