November 2015 - Libguestfs - libguestfs List Archives

guestunmount issues
by Maxim Perevedentsev 24 Nov '15

24 Nov '15

Hello everyone! I'm playing with guestunmount and got the following issues: 1) guestunmount never returns 3 (not mounted) - i guess it's a bug at guestunmount.c:223 2) I don't know whether it's a bug in guestunmount or fusermount, but on my machine while executing > guestmount > guestunmount > guestunmount on second "guestunmount" I get "guestunmount: failed to unmount {...}: fusermount: failed to unmount {...}: Invalid argument" while, according to guestunmount.c, guestunmount should return something like "entry for {} not found" 3) > fusermount -u {} gives me "Invalid argument" as well, while > umount {} handles this error successfully and returns "not mounted". (strace logs attached) I have a custom centos7-based distro, but I don't think it matters. I'd like someone to compare results with mine, and, possibly, give me some explanations about why it's going like this and how to distinguish non-mounted mountpoints from busy ones (I guess I'll have to parse umount {} output). Thanks in advance! -- Your sincerely, Maxim Perevedentsev

2 4

[PATCH] v2v: factor out bootloader handling
by Pino Toscano 20 Nov '15

20 Nov '15

Create an object hierarchy to represent different bootloaders for Linux guests, moving the separate handling of grub1 and grub2 in different classes. This will allow us to support more bootloaders in the future. This is mostly code refactoring, with no actual behaviour change. --- po/POTFILES-ml | 1 + v2v/Makefile.am | 2 + v2v/bootloaders.ml | 317 +++++++++++++++++++++++++++++++++++++++++++++++++++ v2v/bootloaders.mli | 44 +++++++ v2v/convert_linux.ml | 274 ++------------------------------------------ 5 files changed, 375 insertions(+), 263 deletions(-) create mode 100644 v2v/bootloaders.ml create mode 100644 v2v/bootloaders.mli diff --git a/po/POTFILES-ml b/po/POTFILES-ml index 00a9d63..a9b0b59 100644 --- a/po/POTFILES-ml +++ b/po/POTFILES-ml @@ -97,6 +97,7 @@ sysprep/sysprep_operation_utmp.ml sysprep/sysprep_operation_yum_uuid.ml v2v/DOM.ml v2v/OVF.ml +v2v/bootloaders.ml v2v/changeuid.ml v2v/cmdline.ml v2v/convert_linux.ml diff --git a/v2v/Makefile.am b/v2v/Makefile.am index f2919b7..db5aecd 100644 --- a/v2v/Makefile.am +++ b/v2v/Makefile.am @@ -42,6 +42,7 @@ EXTRA_DIST = \ CLEANFILES = *~ *.annot *.cmi *.cmo *.cmx *.cmxa *.o virt-v2v SOURCES_MLI = \ + bootloaders.mli \ cmdline.mli \ convert_linux.mli \ convert_windows.mli \ @@ -94,6 +95,7 @@ SOURCES_ML = \ input_libvirt_xen_ssh.ml \ input_libvirt.ml \ input_ova.ml \ + bootloaders.ml \ convert_linux.ml \ convert_windows.ml \ output_null.ml \ diff --git a/v2v/bootloaders.ml b/v2v/bootloaders.ml new file mode 100644 index 0000000..2c5f421 --- /dev/null +++ b/v2v/bootloaders.ml @@ -0,0 +1,317 @@ +(* virt-v2v + * Copyright (C) 2015 Red Hat Inc. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License along + * with this program; if not, write to the Free Software Foundation, Inc., + * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + *) + +open Printf + +open Common_gettext.Gettext +open Common_utils + +open Types +open Utils + +module G = Guestfs + +class virtual bootloader = object + method virtual name : string + method virtual augeas_device_patterns : string list + method virtual list_kernels : unit -> string list + method virtual set_default : string -> unit + method set_augeas_configuration () = false + method virtual configure_console : unit -> unit + method virtual remove_console : unit -> unit + method update () = () +end + +(* Helper type used in detect_bootloader. *) +type bootloader_type = + | Grub1 + | Grub2 + +(* Helper function for SUSE: remove (hdX,X) prefix from a path. *) +let remove_hd_prefix path = + let rex = Str.regexp "^(hd.*)\$.*\$" in + Str.replace_first rex "\\1" path + +(* Grub1 (AKA grub-legacy) representation. *) +class bootloader_grub1 g inspect grub_config = + let prefix = + let mounts = g#inspect_get_mountpoints inspect.i_root in + try + List.find ( + fun path -> List.mem_assoc path mounts + ) [ "/boot/grub"; "/boot" ] + with Not_found -> "" in +object + inherit bootloader + + method name = "grub1" + + method augeas_device_patterns = [ + "/files" ^ grub_config ^ "/*/kernel/root"; + "/files" ^ grub_config ^ "/*/kernel/resume"; + "/files/boot/grub/device.map/*[label() != \"#comment\"]"; + "/files/etc/sysconfig/grub/boot"; + ] + + method list_kernels () = + let paths = + let expr = sprintf "/files%s/title/kernel" grub_config in + let paths = g#aug_match expr in + let paths = Array.to_list paths in + + (* Remove duplicates. *) + let paths = remove_duplicates paths in + + (* Get the default kernel from grub if it's set. *) + let default = + let expr = sprintf "/files%s/default" grub_config in + try + let idx = g#aug_get expr in + let idx = int_of_string idx in + (* Grub indices are zero-based, augeas is 1-based. *) + let expr = + sprintf "/files%s/title[%d]/kernel" grub_config (idx+1) in + Some expr + with G.Error msg + when String.find msg "aug_get: no matching node" >= 0 -> + None in + + (* If a default kernel was set, put it at the beginning of the paths + * list. If not set, assume the first kernel always boots (?) + *) + match default with + | None -> paths + | Some p -> p :: List.filter ((<>) p) paths in + + (* Resolve the Augeas paths to kernel filenames. *) + let vmlinuzes = List.map g#aug_get paths in + + (* Make sure kernel does not begin with (hdX,X). *) + let vmlinuzes = List.map remove_hd_prefix vmlinuzes in + + (* Prepend grub filesystem. *) + List.map ((^) prefix) vmlinuzes + + method set_default kernel = + if not (String.is_prefix kernel prefix) then + error (f_"kernel %s is not under grub tree %s") + kernel prefix; + let kernel_under_grub_prefix = + let prefix_len = String.length prefix in + let kernel_len = String.length kernel in + String.sub kernel prefix_len (kernel_len - prefix_len) in + + (* Find the grub entry for the given kernel. *) + let paths = g#aug_match (sprintf "/files%s/title/kernel[. = '%s']" + grub_config kernel_under_grub_prefix) in + let paths = Array.to_list paths in + if paths = [] then + error (f_"didn't find grub entry for kernel %s") kernel; + let path = List.hd paths in + let rex = Str.regexp ".*/title\\[\$[1-9][0-9]*\$\\]/kernel" in + if not (Str.string_match rex path 0) then + error (f_"internal error: regular expression did not match '%s'") + path; + let index = int_of_string (Str.matched_group 1 path) - 1 in + g#aug_set (sprintf "/files%s/default" grub_config) (string_of_int index); + g#aug_save () + + method set_augeas_configuration () = + let incls = g#aug_match "/augeas/load/Grub/incl" in + let incls = Array.to_list incls in + let incls_contains_conf = + List.exists (fun incl -> g#aug_get incl = grub_config) incls in + if not incls_contains_conf then ( + g#aug_set "/augeas/load/Grub/incl[last()+1]" grub_config; + true + ) else false + + method configure_console () = + let rex = Str.regexp "\$.*\$\\b\$[xh]vc0\$\\b\$.*\$" in + let expr = sprintf "/files%s/title/kernel/console" grub_config in + + let paths = g#aug_match expr in + let paths = Array.to_list paths in + List.iter ( + fun path -> + let console = g#aug_get path in + if Str.string_match rex console 0 then ( + let console = Str.global_replace rex "\\1ttyS0\\3" console in + g#aug_set path console + ) + ) paths; + + g#aug_save () + + method remove_console () = + let rex = Str.regexp "\$.*\$\\b\$[xh]vc0\$\\b\$.*\$" in + let expr = sprintf "/files%s/title/kernel/console" grub_config in + + let rec loop = function + | [] -> () + | path :: paths -> + let console = g#aug_get path in + if Str.string_match rex console 0 then ( + ignore (g#aug_rm path); + (* All the paths are invalid, restart the loop. *) + let paths = g#aug_match expr in + let paths = Array.to_list paths in + loop paths + ) + else + loop paths + in + let paths = g#aug_match expr in + let paths = Array.to_list paths in + loop paths; + + g#aug_save () +end + +(* Grub2 representation. *) +class bootloader_grub2 g grub_config = + let grub2_update_console ~remove = + let rex = Str.regexp "\$.*\$\\bconsole=[xh]vc0\\b\$.*\$" in + + let paths = [ + "/files/etc/sysconfig/grub/GRUB_CMDLINE_LINUX"; + "/files/etc/default/grub/GRUB_CMDLINE_LINUX"; + "/files/etc/default/grub/GRUB_CMDLINE_LINUX_DEFAULT" + ] in + let paths = List.map g#aug_match paths in + let paths = List.map Array.to_list paths in + let paths = List.flatten paths in + match paths with + | [] -> + if not remove then + warning (f_"could not add grub2 serial console (ignored)") + else + warning (f_"could not remove grub2 serial console (ignored)") + | path :: _ -> + let grub_cmdline = g#aug_get path in + if Str.string_match rex grub_cmdline 0 then ( + let new_grub_cmdline = + if not remove then + Str.global_replace rex "\\1console=ttyS0\\3" grub_cmdline + else + Str.global_replace rex "\\1\\3" grub_cmdline in + g#aug_set path new_grub_cmdline; + g#aug_save (); + + try + ignore (g#command [| "grub2-mkconfig"; "-o"; grub_config |]) + with + G.Error msg -> + warning (f_"could not rebuild grub2 configuration file (%s). This may mean that grub output will not be sent to the serial port, but otherwise should be harmless. Original error message: %s") + grub_config msg + ) in +object + inherit bootloader + + method name = "grub2" + + method augeas_device_patterns = [ + "/files/etc/sysconfig/grub/GRUB_CMDLINE_LINUX"; + "/files/etc/default/grub/GRUB_CMDLINE_LINUX"; + "/files/etc/default/grub/GRUB_CMDLINE_LINUX_DEFAULT"; + "/files/boot/grub2/device.map/*[label() != \"#comment\"]"; + ] + + method list_kernels () = + let get_default_image () = + let cmd = + if g#exists "/sbin/grubby" then + [| "grubby"; "--default-kernel" |] + else + [| "/usr/bin/perl"; "-MBootloader::Tools"; "-e"; " + InitLibrary(); + my $default = Bootloader::Tools::GetDefaultSection(); + print $default->{image}; + " |] in + match g#command cmd with + | "" -> None + | k -> + let len = String.length k in + let k = + if len > 0 && k.[len-1] = '\n' then + String.sub k 0 (len-1) + else k in + Some (remove_hd_prefix k) + in + + let vmlinuzes = + (match get_default_image () with + | None -> [] + | Some k -> [k]) @ + (* This is how the grub2 config generator enumerates kernels. *) + Array.to_list (g#glob_expand "/boot/kernel-*") @ + Array.to_list (g#glob_expand "/boot/vmlinuz-*") @ + Array.to_list (g#glob_expand "/vmlinuz-*") in + let rex = Str.regexp ".*\\.\$dpkg-.*|rpmsave|rpmnew\$$" in + let vmlinuzes = List.filter ( + fun file -> not (Str.string_match rex file 0) + ) vmlinuzes in + vmlinuzes + + method set_default kernel = + let cmd = + if g#exists "/sbin/grubby" then + [| "grubby"; "--set-default"; kernel |] + else + [| "/usr/bin/perl"; "-MBootloader::Tools"; "-e"; sprintf " + InitLibrary(); + my @sections = GetSectionList(type=>image, image=>\"%s\"); + my $section = GetSection(@sections); + my $newdefault = $section->{name}; + SetGlobals(default, \"$newdefault\"); + " kernel |] in + ignore (g#command cmd) + + method configure_console () = + grub2_update_console ~remove:false + + method remove_console () = + grub2_update_console ~remove:true + + method update () = + ignore (g#command [| "grub2-mkconfig"; "-o"; grub_config |]) +end + +let detect_bootloader (g : Guestfs.guestfs) inspect = + let config_file, typ = + let locations = [ + "/boot/grub2/grub.cfg", Grub2; + "/boot/grub/menu.lst", Grub1; + "/boot/grub/grub.conf", Grub1; + ] in + let locations = + if inspect.i_uefi then + ("/boot/efi/EFI/redhat/grub.cfg", Grub2) :: locations + else + locations in + try + List.find ( + fun (config_file, _) -> g#is_file ~followsymlinks:true config_file + ) locations + with + Not_found -> + error (f_"no bootloader detected") in + + match typ with + | Grub1 -> new bootloader_grub1 g inspect config_file + | Grub2 -> new bootloader_grub2 g config_file diff --git a/v2v/bootloaders.mli b/v2v/bootloaders.mli new file mode 100644 index 0000000..b776a51 --- /dev/null +++ b/v2v/bootloaders.mli @@ -0,0 +1,44 @@ +(* virt-v2v + * Copyright (C) 2015 Red Hat Inc. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License along + * with this program; if not, write to the Free Software Foundation, Inc., + * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + *) + +class virtual bootloader : object + method virtual name : string + (** The name of the bootloader. *) + method virtual augeas_device_patterns : string list + (** A list of augeas patterns to search for device names. *) + method virtual list_kernels : unit -> string list + (** Lists all the kernels configured in the bootloader. *) + method virtual set_default : string -> unit + (** Sets the specified vmlinuz path as default bootloader entry. *) + method set_augeas_configuration : unit -> bool + (** Checks whether augeas is reading the configuration file + of the bootloader, and if not then add it. + + Returns whether augeas needs to be reloaded. *) + method virtual configure_console : unit -> unit + (** Sets up the console for the available kernels. *) + method virtual remove_console : unit -> unit + (** Removes the console in all the available kernels. *) + method update : unit -> unit + (** Update the bootloader. *) +end +(** Encapsulates a UNIX boot loader as object. *) + +val detect_bootloader : Guestfs.guestfs -> Types.inspect -> bootloader +(** Detects the bootloader on the guest, and creates the object + representing it. *) diff --git a/v2v/convert_linux.ml b/v2v/convert_linux.ml index 14f232f..81dcc14 100644 --- a/v2v/convert_linux.ml +++ b/v2v/convert_linux.ml @@ -89,37 +89,8 @@ let rec convert ~keep_serial_console (g : G.guestfs) inspect source = let dbfiles = Array.to_list dbfiles in List.iter g#rm_f dbfiles; - (* What grub is installed? *) - let grub_config, grub = - let locations = [ - "/boot/grub2/grub.cfg", `Grub2; - "/boot/grub/menu.lst", `Grub1; - "/boot/grub/grub.conf", `Grub1; - ] in - let locations = - if inspect.i_uefi then - ("/boot/efi/EFI/redhat/grub.cfg", `Grub2) :: locations - else - locations in - try - List.find ( - fun (grub_config, _) -> g#is_file ~followsymlinks:true grub_config - ) locations - with - Not_found -> - error (f_"no grub1/grub-legacy or grub2 configuration file was found") in - - (* Grub prefix? Usually "/boot". *) - let grub_prefix = - match grub with - | `Grub2 -> "" - | `Grub1 -> - let mounts = g#inspect_get_mountpoints inspect.i_root in - try - List.find ( - fun path -> List.mem_assoc path mounts - ) [ "/boot/grub"; "/boot" ] - with Not_found -> "" in + (* Detect the installed bootloader. *) + let bootloader = Bootloaders.detect_bootloader g inspect in (* What kernel/kernel-like packages are installed on the current guest? *) let installed_kernels : kernel_info list = @@ -270,88 +241,7 @@ let rec convert ~keep_serial_console (g : G.guestfs) inspect source = * list is the default booting kernel. *) let grub_kernels : kernel_info list = - (* Helper function for SUSE: remove (hdX,X) prefix from a path. *) - let remove_hd_prefix = - let rex = Str.regexp "^(hd.*)\$.*\$" in - Str.replace_first rex "\\1" - in - - let vmlinuzes = - match grub with - | `Grub1 -> - let paths = - let expr = sprintf "/files%s/title/kernel" grub_config in - let paths = g#aug_match expr in - let paths = Array.to_list paths in - - (* Remove duplicates. *) - let paths = remove_duplicates paths in - - (* Get the default kernel from grub if it's set. *) - let default = - let expr = sprintf "/files%s/default" grub_config in - try - let idx = g#aug_get expr in - let idx = int_of_string idx in - (* Grub indices are zero-based, augeas is 1-based. *) - let expr = - sprintf "/files%s/title[%d]/kernel" grub_config (idx+1) in - Some expr - with G.Error msg - when String.find msg "aug_get: no matching node" >= 0 -> - None in - - (* If a default kernel was set, put it at the beginning of the paths - * list. If not set, assume the first kernel always boots (?) - *) - match default with - | None -> paths - | Some p -> p :: List.filter ((<>) p) paths in - - (* Resolve the Augeas paths to kernel filenames. *) - let vmlinuzes = List.map g#aug_get paths in - - (* Make sure kernel does not begin with (hdX,X). *) - let vmlinuzes = List.map remove_hd_prefix vmlinuzes in - - (* Prepend grub filesystem. *) - List.map ((^) grub_prefix) vmlinuzes - - | `Grub2 -> - let get_default_image () = - let cmd = - if g#exists "/sbin/grubby" then - [| "grubby"; "--default-kernel" |] - else - [| "/usr/bin/perl"; "-MBootloader::Tools"; "-e"; " - InitLibrary(); - my $default = Bootloader::Tools::GetDefaultSection(); - print $default->{image}; - " |] in - match g#command cmd with - | "" -> None - | k -> - let len = String.length k in - let k = - if len > 0 && k.[len-1] = '\n' then - String.sub k 0 (len-1) - else k in - Some (remove_hd_prefix k) - in - - let vmlinuzes = - (match get_default_image () with - | None -> [] - | Some k -> [k]) @ - (* This is how the grub2 config generator enumerates kernels. *) - Array.to_list (g#glob_expand "/boot/kernel-*") @ - Array.to_list (g#glob_expand "/boot/vmlinuz-*") @ - Array.to_list (g#glob_expand "/vmlinuz-*") in - let rex = Str.regexp ".*\\.\$dpkg-.*|rpmsave|rpmnew\$$" in - let vmlinuzes = List.filter ( - fun file -> not (Str.string_match rex file 0) - ) vmlinuzes in - vmlinuzes in + let vmlinuzes = bootloader#list_kernels () in (* Map these to installed kernels. *) filter_map ( @@ -390,21 +280,8 @@ let rec convert ~keep_serial_console (g : G.guestfs) inspect source = (* Conversion step. *) let rec augeas_grub_configuration () = - match grub with - | `Grub1 -> - (* Ensure Augeas is reading the grub configuration file, and if not - * then add it. - *) - let incls = g#aug_match "/augeas/load/Grub/incl" in - let incls = Array.to_list incls in - let incls_contains_conf = - List.exists (fun incl -> g#aug_get incl = grub_config) incls in - if not incls_contains_conf then ( - g#aug_set "/augeas/load/Grub/incl[last()+1]" grub_config; - Linux.augeas_reload g; - ) - - | `Grub2 -> () (* Not necessary for grub2. *) + if bootloader#set_augeas_configuration () then + Linux.augeas_reload g and autorelabel () = (* Only do autorelabel if load_policy binary exists. Actually @@ -741,44 +618,7 @@ let rec convert ~keep_serial_console (g : G.guestfs) inspect source = best_kernel, virtio and grub_set_bootable kernel = - match grub with - | `Grub1 -> - if not (String.is_prefix kernel.ki_vmlinuz grub_prefix) then - error (f_"kernel %s is not under grub tree %s") - kernel.ki_vmlinuz grub_prefix; - let kernel_under_grub_prefix = - let prefix_len = String.length grub_prefix in - let kernel_len = String.length kernel.ki_vmlinuz in - String.sub kernel.ki_vmlinuz prefix_len (kernel_len - prefix_len) in - - (* Find the grub entry for the given kernel. *) - let paths = g#aug_match (sprintf "/files%s/title/kernel[. = '%s']" - grub_config kernel_under_grub_prefix) in - let paths = Array.to_list paths in - if paths = [] then - error (f_"didn't find grub entry for kernel %s") kernel.ki_vmlinuz; - let path = List.hd paths in - let rex = Str.regexp ".*/title\\[\$[1-9][0-9]*\$\\]/kernel" in - if not (Str.string_match rex path 0) then - error (f_"internal error: regular expression did not match '%s'") - path; - let index = int_of_string (Str.matched_group 1 path) - 1 in - g#aug_set (sprintf "/files%s/default" grub_config) (string_of_int index); - g#aug_save () - - | `Grub2 -> - let cmd = - if g#exists "/sbin/grubby" then - [| "grubby"; "--set-default"; kernel.ki_vmlinuz |] - else - [| "/usr/bin/perl"; "-MBootloader::Tools"; "-e"; sprintf " - InitLibrary(); - my @sections = GetSectionList(type=>image, image=>\"%s\"); - my $section = GetSection(@sections); - my $newdefault = $section->{name}; - SetGlobals(default, \"$newdefault\"); - " kernel.ki_vmlinuz |] in - ignore (g#command cmd) + bootloader#set_default kernel.ki_vmlinuz (* Even though the kernel was already installed (this version of * virt-v2v does not install new kernels), it could have an @@ -940,26 +780,7 @@ let rec convert ~keep_serial_console (g : G.guestfs) inspect source = g#aug_save () and grub_configure_console () = - match grub with - | `Grub1 -> - let rex = Str.regexp "\$.*\$\\b\$[xh]vc0\$\\b\$.*\$" in - let expr = sprintf "/files%s/title/kernel/console" grub_config in - - let paths = g#aug_match expr in - let paths = Array.to_list paths in - List.iter ( - fun path -> - let console = g#aug_get path in - if Str.string_match rex console 0 then ( - let console = Str.global_replace rex "\\1ttyS0\\3" console in - g#aug_set path console - ) - ) paths; - - g#aug_save () - - | `Grub2 -> - grub2_update_console ~remove:false + bootloader#configure_console () (* If the target doesn't support a serial console, we want to remove * all references to it instead. @@ -990,69 +811,8 @@ let rec convert ~keep_serial_console (g : G.guestfs) inspect source = g#aug_save () and grub_remove_console () = - match grub with - | `Grub1 -> - let rex = Str.regexp "\$.*\$\\b\$[xh]vc0\$\\b\$.*\$" in - let expr = sprintf "/files%s/title/kernel/console" grub_config in + bootloader#remove_console () - let rec loop = function - | [] -> () - | path :: paths -> - let console = g#aug_get path in - if Str.string_match rex console 0 then ( - ignore (g#aug_rm path); - (* All the paths are invalid, restart the loop. *) - let paths = g#aug_match expr in - let paths = Array.to_list paths in - loop paths - ) - else - loop paths - in - let paths = g#aug_match expr in - let paths = Array.to_list paths in - loop paths; - - g#aug_save () - - | `Grub2 -> - grub2_update_console ~remove:true - - and grub2_update_console ~remove = - let rex = Str.regexp "\$.*\$\\bconsole=[xh]vc0\\b\$.*\$" in - - let paths = [ - "/files/etc/sysconfig/grub/GRUB_CMDLINE_LINUX"; - "/files/etc/default/grub/GRUB_CMDLINE_LINUX"; - "/files/etc/default/grub/GRUB_CMDLINE_LINUX_DEFAULT" - ] in - let paths = List.map g#aug_match paths in - let paths = List.map Array.to_list paths in - let paths = List.flatten paths in - match paths with - | [] -> - if not remove then - warning (f_"could not add grub2 serial console (ignored)") - else - warning (f_"could not remove grub2 serial console (ignored)") - | path :: _ -> - let grub_cmdline = g#aug_get path in - if Str.string_match rex grub_cmdline 0 then ( - let new_grub_cmdline = - if not remove then - Str.global_replace rex "\\1console=ttyS0\\3" grub_cmdline - else - Str.global_replace rex "\\1\\3" grub_cmdline in - g#aug_set path new_grub_cmdline; - g#aug_save (); - - try - ignore (g#command [| "grub2-mkconfig"; "-o"; grub_config |]) - with - G.Error msg -> - warning (f_"could not rebuild grub2 configuration file (%s). This may mean that grub output will not be sent to the serial port, but otherwise should be harmless. Original error message: %s") - grub_config msg - ) and supports_acpi () = (* ACPI known to cause RHEL 3 to fail. *) @@ -1273,19 +1033,9 @@ let rec convert ~keep_serial_console (g : G.guestfs) inspect source = let paths = [ (* /etc/fstab *) "/files/etc/fstab/*/spec"; - - (* grub-legacy config *) - "/files" ^ grub_config ^ "/*/kernel/root"; - "/files" ^ grub_config ^ "/*/kernel/resume"; - "/files/boot/grub/device.map/*[label() != \"#comment\"]"; - "/files/etc/sysconfig/grub/boot"; - - (* grub2 config *) - "/files/etc/sysconfig/grub/GRUB_CMDLINE_LINUX"; - "/files/etc/default/grub/GRUB_CMDLINE_LINUX"; - "/files/etc/default/grub/GRUB_CMDLINE_LINUX_DEFAULT"; - "/files/boot/grub2/device.map/*[label() != \"#comment\"]"; ] in + (* Bootloader config *) + let paths = paths @ bootloader#augeas_device_patterns in (* Which of these paths actually exist? *) let paths = @@ -1361,9 +1111,7 @@ let rec convert ~keep_serial_console (g : G.guestfs) inspect source = if !changed then ( g#aug_save (); - (* If it's grub2, we have to regenerate the config files. *) - if grub = `Grub2 then - ignore (g#command [| "grub2-mkconfig"; "-o"; grub_config |]); + bootloader#update (); Linux.augeas_reload g ); -- 2.1.0

1 0

[PATCH] build: disable -Winline
by Pino Toscano 20 Nov '15

20 Nov '15

Older GCC versions (e.g. < 4.9) may complain about inline functions that cannot be inlined (growing the generated code). It is not really a problem for us, so just suppress again the warning. Followup of commit c37c554fc56151b709882b382bb5fa26d9083449. --- m4/guestfs_c.m4 | 1 + 1 file changed, 1 insertion(+) diff --git a/m4/guestfs_c.m4 b/m4/guestfs_c.m4 index 7250294..ea26202 100644 --- a/m4/guestfs_c.m4 +++ b/m4/guestfs_c.m4 @@ -48,6 +48,7 @@ nw="$nw -Wtraditional" # Warns on #elif which we use often nw="$nw -Wsystem-headers" # Don't let system headers trigger warnings nw="$nw -Wpadded" # Our structs are not padded nw="$nw -Wvla" # Allow variable length arrays. +nw="$nw -Winline" # inline functions in Python binding nw="$nw -Wshadow" # Not useful, as it applies to global vars nw="$nw -Wunsafe-loop-optimizations" # just a warning that an optimization # was not possible, safe to ignore -- 2.1.0

2 1

[PATCH 0/4] v2v: Add a new tool virt-v2v-copy-to-local to handle Xen and ESXi
by Richard W.M. Jones 19 Nov '15

19 Nov '15

It turns out that RHEL 5 Xen conversions don't work if the source disk is located on a block device. See patch 1/4 for the gory details. This patch series proposes a new tool called virt-v2v-copy-to-local which essentially is a way to make new virt-v2v work like the old virt-v2v, ie. copy first, convert after. Of course this is very slow and would only be used as a last resort, but I identified two situations in which this would currently be useful (see explanation in patch 4/4). This is also available in my git repo: https://github.com/rwmjones/libguestfs/commits/master Rich.

1 4

Re: [Libguestfs] [virt-tools-list] Paths with whitespace in guestfish
by Richard W.M. Jones 19 Nov '15

19 Nov '15

[I don't check this list for libguestfs questions, hence the late reply. There's a libguestfs mailing list: https://www.redhat.com/mailman/listinfo/libguestfs ] On Mon, Oct 12, 2015 at 08:06:03PM -0500, Robert Nichols wrote: > Is guestfish able to handle paths that include whitespace? I've > tried just about everything I can think of: > > ><fs> ls /Users/Owner/My\ Documents > ls should have 1 parameter(s) > type 'help ls' for help on ls > ><fs> ls "/Users/Owner/My Documents" > libguestfs: error: ls0: opendir: /Users/Owner/My Documents: No such > file or directory This syntax (single or double quoting) is the correct one and works fine for me: ><fs> ls "/Program Files/Windows NT/Accessories" mswrd6.wpc mswrd8.wpc wordpad.exe write.wpc I'm guessing it may be a case sensitivity problem. You could try prepending 'win:...' (a guestfish-specific hack for Windows) in front of the path, as described in the manual: http://libguestfs.org/guestfish.1.html#windows-paths For example: ><fs> ls "/program files/windows nt/accessories" libguestfs: error: ls0: opendir: /program files/windows nt/accessories: No such file or directory ><fs> ls "win:/program files/windows nt/accessories" mswrd6.wpc mswrd8.wpc wordpad.exe write.wpc ><fs> set-trace true ><fs> ls "win:/program files/windows nt/accessories" libguestfs: trace: case_sensitive_path "/program files/windows nt/accessories" libguestfs: trace: case_sensitive_path = "/Program Files/Windows NT/Accessories" libguestfs: trace: ls "/Program Files/Windows NT/Accessories" libguestfs: trace: ls0 "/Program Files/Windows NT/Accessories" "/tmp/libguestfse0H7nF/ls10" libguestfs: trace: ls0 = 0 libguestfs: trace: ls = ["mswrd6.wpc", "mswrd8.wpc", "wordpad.exe", "write.wpc"] mswrd6.wpc mswrd8.wpc wordpad.exe write.wpc (notice the extra call to case_sensitive_path in the trace). Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-builder quickly builds VMs from scratch http://libguestfs.org/virt-builder.1.html

2 1

[PATCH 0/3] v2v: windows: Use '*.inf' files to control how Windows drivers are installed.
by Richard W.M. Jones 18 Nov '15

18 Nov '15

https://github.com/rwmjones/libguestfs/tree/rewrite-virtio-copy-drivers Instead of trying to split and parse elements from virtio-win paths, use the '*.inf' files supplied with the drivers to control how Windows drivers are installed. The following emails best explain how this works: https://www.redhat.com/archives/libguestfs/2015-October/msg00352.html https://www.redhat.com/archives/libguestfs/2015-November/msg00065.html Currently the product variant (eg. client or server) is ignored (see email I sent earlier today). There's actually a fourth patch which I didn't include, which just adds the content of all of the *.inf files from the virtio-win 1.7.4 ISO and directory, to test-data/fake-virtio-win. I didn't include it because it's a big patch that's not very interesting, but you can get it from the repo above. Rich.

3 8

[PATCH 0/5] build: Enable some more warnings.
by Richard W.M. Jones 18 Nov '15

18 Nov '15

Add some warnings. Well, the first patch is a miscellaneous change, but patches 2-5 add some warnings. Rich.

2 8

[PATCH] p2v: Send physical server 'dmesg' output to debug dir on conversion server.
by Richard W.M. Jones 16 Nov '15

16 Nov '15

If the virt-p2v ISO doesn't contain the correct drivers for the hardware on the physical server, that can cause peculiar conversion problems. This was hard to diagnose because we could not see the 'dmesg' output from the physical server. Upload 'dmesg' output from the physical server (more precisely: from the virt-p2v ISO's kernel running on the physical server) to the virt-v2v conversion server debugging directory, so that this information is available for debugging later. Unfortunately this involves yet another copy of the 'read_whole_file' function. --- p2v/Makefile.am | 3 +- p2v/conversion.c | 32 ++++++++++++++++++- p2v/p2v.h | 5 ++- p2v/ssh.c | 21 ++++++++++++- p2v/virt-p2v.pod | 8 +++++ p2v/whole-file.c | 94 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ po/POTFILES | 1 + 7 files changed, 160 insertions(+), 4 deletions(-) create mode 100644 p2v/whole-file.c diff --git a/p2v/Makefile.am b/p2v/Makefile.am index 0fd9ef5..2401a07 100644 --- a/p2v/Makefile.am +++ b/p2v/Makefile.am @@ -60,7 +60,8 @@ virt_p2v_SOURCES = \ miniexpect.h \ p2v.h \ ssh.c \ - utils.c + utils.c \ + whole-file.c virt_p2v_CPPFLAGS = \ -DLOCALEBASEDIR=\""$(datadir)/locale"\" \ diff --git a/p2v/conversion.c b/p2v/conversion.c index e2f37cd..f781921 100644 --- a/p2v/conversion.c +++ b/p2v/conversion.c @@ -37,6 +37,8 @@ #include <libxml/xmlwriter.h> +#include "ignore-value.h" + #include "miniexpect.h" #include "p2v.h" @@ -129,6 +131,10 @@ start_conversion (struct config *config, time_t now; struct tm tm; mexp_h *control_h = NULL; + char dmesg_file[] = "/tmp/dmesg.XXXXXX"; + char dmesg_cmd[64]; + CLEANUP_FREE char *dmesg = NULL; + int fd, r; #if DEBUG_STDERR print_config (config, stderr); @@ -230,11 +236,35 @@ start_conversion (struct config *config, fprintf (stderr, "%s: libvirt XML:\n%s", guestfs_int_program_name, libvirt_xml); #endif + /* Get the output from the 'dmesg' command. We will store this + * on the remote server. + */ + fd = mkstemp (dmesg_file); + if (fd == -1) { + perror ("mkstemp"); + goto skip_dmesg; + } + close (fd); + snprintf (dmesg_cmd, sizeof dmesg_cmd, + "dmesg > %s", dmesg_file); + r = system (dmesg_cmd); + if (r == -1) { + perror ("system"); + goto skip_dmesg; + } + if (!WIFEXITED (r) || WEXITSTATUS (r) != 0) { + fprintf (stderr, "'dmesg' failed (ignored)\n"); + goto skip_dmesg; + } + + ignore_value (read_whole_file (dmesg_file, &dmesg, NULL)); + skip_dmesg: + /* Open the control connection and start conversion */ if (notify_ui) notify_ui (NOTIFY_STATUS, _("Setting up the control connection ...")); - control_h = start_remote_connection (config, remote_dir, libvirt_xml); + control_h = start_remote_connection (config, remote_dir, libvirt_xml, dmesg); if (control_h == NULL) { const char *err = get_ssh_error (); diff --git a/p2v/p2v.h b/p2v/p2v.h index 35b3f3c..f35200e 100644 --- a/p2v/p2v.h +++ b/p2v/p2v.h @@ -116,7 +116,7 @@ extern void cancel_conversion (void); /* ssh.c */ extern int test_connection (struct config *); extern mexp_h *open_data_connection (struct config *, int *local_port, int *remote_port); -extern mexp_h *start_remote_connection (struct config *, const char *remote_dir, const char *libvirt_xml); +extern mexp_h *start_remote_connection (struct config *, const char *remote_dir, const char *libvirt_xml, const char *dmesg); extern const char *get_ssh_error (void); /* utils.c */ @@ -124,6 +124,9 @@ extern char *get_if_addr (const char *if_name); extern char *get_if_vendor (const char *if_name, int truncate); extern void wait_network_online (const struct config *); +/* whole-file.c */ +extern int read_whole_file (const char *filename, char **data_r, size_t *size_r); + /* virt-v2v version and features (read from remote). */ extern int v2v_major; extern int v2v_minor; diff --git a/p2v/ssh.c b/p2v/ssh.c index cd059c5..6dc6854 100644 --- a/p2v/ssh.c +++ b/p2v/ssh.c @@ -848,7 +848,8 @@ wait_for_prompt (mexp_h *h) mexp_h * start_remote_connection (struct config *config, - const char *remote_dir, const char *libvirt_xml) + const char *remote_dir, const char *libvirt_xml, + const char *dmesg) { mexp_h *h; char magic[9]; @@ -904,6 +905,24 @@ start_remote_connection (struct config *config, if (wait_for_prompt (h) == -1) goto error; + if (dmesg != NULL) { + /* Upload the physical host dmesg to the remote directory. */ + if (mexp_printf (h, + "cat > '%s/dmesg' << '__%s__'\n" + "%s" + "\n" + "__%s__\n", + remote_dir, magic, + dmesg, + magic) == -1) { + set_ssh_error ("mexp_printf: %m"); + goto error; + } + + if (wait_for_prompt (h) == -1) + goto error; + } + return h; error: diff --git a/p2v/virt-p2v.pod b/p2v/virt-p2v.pod index be6b015..1d1b41c 100644 --- a/p2v/virt-p2v.pod +++ b/p2v/virt-p2v.pod @@ -628,6 +628,14 @@ Into this directory are written various files which include: =over 4 +=item F<dmesg> + +I<before conversion> + +The dmesg output from the physical machine. Useful for detecting +problems such as missing device drivers or firmware on the virt-p2v +ISO. + =item F<name> I<before conversion> diff --git a/p2v/whole-file.c b/p2v/whole-file.c new file mode 100644 index 0000000..a7e09b8 --- /dev/null +++ b/p2v/whole-file.c @@ -0,0 +1,94 @@ +/* libguestfs + * Copyright (C) 2011-2015 Red Hat Inc. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include <config.h> + +#include <stdio.h> +#include <stdlib.h> +#include <unistd.h> +#include <fcntl.h> +#include <sys/stat.h> +#include <libintl.h> + +#include "p2v.h" + +/* Read the whole file into a memory buffer and return it. The file + * should be a regular, local, trusted file. + */ +int +read_whole_file (const char *filename, char **data_r, size_t *size_r) +{ + int fd; + char *data; + off_t size; + off_t n; + ssize_t r; + struct stat statbuf; + + fd = open (filename, O_RDONLY|O_CLOEXEC); + if (fd == -1) { + fprintf (stderr, "open: %s: %m\n", filename); + return -1; + } + + if (fstat (fd, &statbuf) == -1) { + fprintf (stderr, "stat: %s: %m\n", filename); + close (fd); + return -1; + } + + size = statbuf.st_size; + data = malloc (size + 1); + if (data == NULL) { + perror ("malloc"); + return -1; + } + + n = 0; + while (n < size) { + r = read (fd, &data[n], size - n); + if (r == -1) { + fprintf (stderr, "read: %s: %m\n", filename); + free (data); + close (fd); + return -1; + } + if (r == 0) { + fprintf (stderr, "read: %s: unexpected end of file\n", filename); + free (data); + close (fd); + return -1; + } + n += r; + } + + if (close (fd) == -1) { + fprintf (stderr, "close: %s: %m\n", filename); + free (data); + return -1; + } + + /* For convenience of callers, \0-terminate the data. */ + data[size] = '\0'; + + *data_r = data; + if (size_r != NULL) + *size_r = size; + + return 0; +} diff --git a/po/POTFILES b/po/POTFILES index 09a8425..c6c277c 100644 --- a/po/POTFILES +++ b/po/POTFILES @@ -279,6 +279,7 @@ p2v/main.c p2v/miniexpect.c p2v/ssh.c p2v/utils.c +p2v/whole-file.c perl/bindtests.pl perl/lib/Sys/Guestfs.c perl/lib/Sys/Guestfs.pm -- 2.5.0

1 0

inotify* APIs
by Richard W.M. Jones 14 Nov '15

14 Nov '15

We have these APIs that cannot possibly work by design. Obviously adding them was a mistake, but here we are. http://libguestfs.org/guestfs.3.html#guestfs_inotify_add_watch The guestfs_inotify_add_watch API has started to fail in Rawhide. I'm trying to find the details, but the test error message is: libguestfs: error: inotify_add_watch: /inotify_add_watch: Invalid argument I wonder if we should just replace these APIs with ones that return -ENOTSUP, saving everyone the time and effort of fixing or using these APIs? Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/

1 0

[PATCH] po-docs: disable parallel build
by Richard Tollerton 14 Nov '15

14 Nov '15

Automating the master build from scratch requires some annoying hassles thanks to 1ea1646e6f224dbc5b3922bb3c65aab4d71a9cb9. This can be worked around by running the requisite commands first, i.e., rm -f po-docs/podfiles && make -C po-docs update-po && make But after that, there still appears to be a race in the po-docs build (reproduced on -j12 on an E5-1650 v2): make[2]: Entering directory 'libguestfs-git/src/libguestfs/po-docs' Making all in ja make[3]: Entering directory 'libguestfs-git/src/libguestfs/po-docs/ja' make[3]: *** No rule to make target 'guestfish-actions.pod', needed by 'guestfish.1'. Stop. This appears to go away if you set .NOTPARALLEL in all of po-docs. Signed-off-by: Richard Tollerton <rich.tollerton(a)ni.com> --- po-docs/Makefile.am | 2 ++ 1 file changed, 2 insertions(+) diff --git a/po-docs/Makefile.am b/po-docs/Makefile.am index a274898..617e8ad 100644 --- a/po-docs/Makefile.am +++ b/po-docs/Makefile.am @@ -88,3 +88,5 @@ podfiles: Makefile for f in `cat $(top_srcdir)/po/POTFILES-pl`; do echo $(top_srcdir)/$$f; done >> $@-t LC_ALL=C sort -o $@-t $@-t mv $@-t $@ + +.NOTPARALLEL: -- 2.6.2

2 1