June 2016 - Libguestfs - libguestfs List Archives

Re: [Libguestfs] libguestfs-tools and NTFS compression attribute
by Richard W.M. Jones 13 Jun '16

13 Jun '16

[Please follow up on libguestfs(a)redhat.com, there is no need to subscribe] On Mon, Jun 13, 2016 at 04:03:20PM +0200, Vlastimil Burián wrote: > Hello Richard, > > I need a little help with libguestfs-tools > > I used it as follows: > > |sudo guestmount -a thevirtualdisk.vhdx -i /mnt/anydirectory | > > > But so far I have no clue as to if it is possible and if yes, then how > to clear the NTFS compression attribute(?) supposing I have mounted a > virt. disk NTFS compressed. > > https://serverfault.com/questions/783548/hyper-v-how-to-connect-compressed-… If you just want to do it with libguestfs and don't need to automate the process (ie. you're only doing it one-off for a few disks), then it's better to use virt-rescue. Something like this: $ virt-rescue -a thedisk ><rescue> mount /dev/sda2 /sysroot ><rescue> cd /sysroot ><rescue> setfattr -h -v 0x00000000 -n system.ntfs_attrib directory-name (See also: http://www.tuxera.com/community/ntfs-3g-advanced/data-compression/) You can also use the regular getfattr command to list the attribute, although the format seems to need a bit of decoding: ><rescue> mkdir compressed ><rescue> setfattr -h -v 0x00080000 -n system.ntfs_attrib compressed ><rescue> mkdir uncompressed ><rescue> setfattr -h -v 0x00000000 -n system.ntfs_attrib uncompressed ><rescue> getfattr -n system.ntfs_attrib compressed # file: compressed system.ntfs_attrib=0sEAgAAA== ><rescue> getfattr -n system.ntfs_attrib uncompressed/ # file: uncompressed/ system.ntfs_attrib=0sEAAAAA== Unfortunately these attributes are not passed through FUSE. I suspect that is because they are in the system.* namespace which the kernel handles specially, but there are also a bunch of bugs associated with extended attributes & guestmount and maybe this is one of them. It should be possible to read and set the attributes through the API (eg. using guestfish). See the APIs guestfs_lsetxattr and guestfs_lgetxattr and related. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org

1 0

[PATCH] customize: Give an error if --truncate-recursive path does not exist (RHBZ#1345809).
by Richard W.M. Jones 13 Jun '16

13 Jun '16

You will now see an error such as: $ virt-customize -a centos-6.img --truncate-recursive /home/foo [ 0.0] Examining the guest ... [ 16.5] Setting a random seed [ 16.5] Recursively truncating: /home/foo virt-customize: error: libguestfs error: find0: /home/foo: No such file or directory Thanks: Xianghua Chen --- mllib/common_utils.ml | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/mllib/common_utils.ml b/mllib/common_utils.ml index dfffae3..9765a79 100644 --- a/mllib/common_utils.ml +++ b/mllib/common_utils.ml @@ -779,12 +779,10 @@ let rm_rf_only_files (g : Guestfs.guestfs) ?filter dir = ) let truncate_recursive (g : Guestfs.guestfs) dir = - if g#is_dir dir then ( - let files = Array.map (Filename.concat dir) (g#find dir) in - let files = Array.to_list files in - let files = List.filter g#is_file files in - List.iter g#truncate files - ) + let files = Array.map (Filename.concat dir) (g#find dir) in + let files = Array.to_list files in + let files = List.filter g#is_file files in + List.iter g#truncate files (* Detect type of a file. *) let detect_file_type filename = -- 2.7.4

2 1

[PATCH] sysprep: Enable the network, add --no-network to disable it
by Richard W.M. Jones 13 Jun '16

13 Jun '16

This is just one possible way to fix this bug, but the simplest. Others I considered: - Keeping the network disabled by default. Options like --install wouldn't work and there would be no actionable error message telling users what to do to fix it. - Modifying virt-customize to warn/error/suggest if operations like --install were used but the network was disabled. However it's not always wrong to use --install + --no-network, so getting the right message is difficult. - Disabling any customize operations that are not relevant to virt-sysprep. I think this is unnecessarily harsh. Rich.

2 2

[PATCH] v2v: Fix get_firmware_bootable_device.
by Richard W.M. Jones 13 Jun '16

13 Jun '16

I'm going to push this because it's a test blocker, but FYI. Rich.

1 1

[PATCH 0/2] Reset the graphics mode in UEFI BCD
by Pavel Butsykin 10 Jun '16

10 Jun '16

Guest tools for Windows in Parallels / Virtuozzo Server 6 contain a paravirtualized video driver, which wants to take full control of the video mode. To avoid excessive video mode switches in UEFI VMs it executes "bcdedit /set {current} graphicsmodedisabled true". When such a VM is imported into a QEMU/KVM-based hypervisor, the standard video drivers in Windows do not set the video mode as they expect it to have already been configured by the bootloader. As a result, the VM runs with black screen. To solve this issue, we need to try to lead the BCD to the default state. Pavel Butsykin (2): v2v: fill the list of the EFI system partitions v2v: remove the 'graphicsmodedisabled' entry in ESP BCD v2v/convert_linux.ml | 2 +- v2v/convert_windows.ml | 41 +++++++++++++++++++++++++++++++++++++++++ v2v/inspect_source.ml | 29 +++++++++++++++++++++-------- v2v/types.ml | 4 ++-- v2v/types.mli | 3 ++- v2v/v2v.ml | 2 +- v2v/v2v_unit_tests.ml | 2 +- 7 files changed, 69 insertions(+), 14 deletions(-) -- 2.8.3

2 5

[PATCH] v2v: OVF: Set <Origin/> field correctly based on source hypervisor.
by Richard W.M. Jones 10 Jun '16

10 Jun '16

https://bugzilla.redhat.com/show_bug.cgi?id=1342398#c6 --- v2v/OVF.ml | 138 ++++++++++++++++++++++++++++++++++--------------------------- 1 file changed, 76 insertions(+), 62 deletions(-) diff --git a/v2v/OVF.ml b/v2v/OVF.ml index 8a6f13e..0599306 100644 --- a/v2v/OVF.ml +++ b/v2v/OVF.ml @@ -184,6 +184,15 @@ and get_ostype = function typ distro major minor arch product; "Unassigned" +(* Set the Origin field based on the source hypervisor. + * https://bugzilla.redhat.com/show_bug.cgi?id=1342398#c6 + *) +let origin_of_source_hypervisor = function + | VMware -> Some 1 + | Xen -> Some 2 + | QEmu | KVM -> Some 7 + | _ -> None + (* Generate the .meta file associated with each volume. *) let create_meta_files output_alloc sd_uuid image_uuids targets = (* Note: Upper case in the .meta, mixed case in the OVF. *) @@ -238,11 +247,29 @@ let rec create_ovf source targets guestcaps inspect let vmtype = match vmtype with `Desktop -> "0" | `Server -> "1" in let ostype = get_ostype inspect in - let origin = - match source.s_hypervisor with - | VMware -> 1 - | Xen -> 2 - | _ -> 0 in + let content_fields = [ + e "Name" [] [PCData source.s_name]; + e "TemplateId" [] [PCData "00000000-0000-0000-0000-000000000000"]; + e "TemplateName" [] [PCData "Blank"]; + e "Description" [] [PCData generated_by]; + e "Domain" [] []; + e "CreationDate" [] [PCData iso_time]; + e "IsInitilized" (* sic *) [] [PCData "True"]; + e "IsAutoSuspend" [] [PCData "False"]; + e "TimeZone" [] []; + e "IsStateless" [] [PCData "False"]; + e "VmType" [] [PCData vmtype]; + (* See https://bugzilla.redhat.com/show_bug.cgi?id=1260590#c17 *) + e "DefaultDisplayType" [] [PCData "1"]; + ] in + + (* Add the <Origin> element if we can. *) + let content_fields = + match origin_of_source_hypervisor source.s_hypervisor with + | None -> content_fields + | Some origin -> + content_fields @ + [e "Origin" [] [PCData (string_of_int origin)]] in let ovf : doc = doc "ovf:Envelope" [ @@ -260,66 +287,53 @@ let rec create_ovf source targets guestcaps inspect e "Section" ["xsi:type", "ovf:DiskSection_Type"] [ e "Info" [] [PCData "List of Virtual Disks"] ]; - e "Content" ["ovf:id", "out"; "xsi:type", "ovf:VirtualSystem_Type"] [ - e "Name" [] [PCData source.s_name]; - e "TemplateId" [] [PCData "00000000-0000-0000-0000-000000000000"]; - e "TemplateName" [] [PCData "Blank"]; - e "Description" [] [PCData generated_by]; - e "Domain" [] []; - e "CreationDate" [] [PCData iso_time]; - e "IsInitilized" (* sic *) [] [PCData "True"]; - e "IsAutoSuspend" [] [PCData "False"]; - e "TimeZone" [] []; - e "IsStateless" [] [PCData "False"]; - e "Origin" [] [PCData (string_of_int origin)]; - e "VmType" [] [PCData vmtype]; - (* See https://bugzilla.redhat.com/show_bug.cgi?id=1260590#c17 *) - e "DefaultDisplayType" [] [PCData "1"]; - - e "Section" ["ovf:id", vm_uuid; "ovf:required", "false"; - "xsi:type", "ovf:OperatingSystemSection_Type"] [ - e "Info" [] [PCData inspect.i_product_name]; - e "Description" [] [PCData ostype]; - ]; - - e "Section" ["xsi:type", "ovf:VirtualHardwareSection_Type"] [ - e "Info" [] [PCData (sprintf "%d CPU, %Ld Memory" source.s_vcpu memsize_mb)]; - e "Item" [] [ - e "rasd:Caption" [] [PCData (sprintf "%d virtual cpu" source.s_vcpu)]; - e "rasd:Description" [] [PCData "Number of virtual CPU"]; - e "rasd:InstanceId" [] [PCData "1"]; - e "rasd:ResourceType" [] [PCData "3"]; - e "rasd:num_of_sockets" [] [PCData (string_of_int source.s_vcpu)]; - e "rasd:cpu_per_socket"[] [PCData "1"]; - ]; - e "Item" [] [ - e "rasd:Caption" [] [PCData (sprintf "%Ld MB of memory" memsize_mb)]; - e "rasd:Description" [] [PCData "Memory Size"]; - e "rasd:InstanceId" [] [PCData "2"]; - e "rasd:ResourceType" [] [PCData "4"]; - e "rasd:AllocationUnits" [] [PCData "MegaBytes"]; - e "rasd:VirtualQuantity" [] [PCData (Int64.to_string memsize_mb)]; + e "Content" ["ovf:id", "out"; "xsi:type", "ovf:VirtualSystem_Type"] ( + content_fields @ [ + e "Section" ["ovf:id", vm_uuid; "ovf:required", "false"; + "xsi:type", "ovf:OperatingSystemSection_Type"] [ + e "Info" [] [PCData inspect.i_product_name]; + e "Description" [] [PCData ostype]; ]; - e "Item" [] [ - e "rasd:Caption" [] [PCData "USB Controller"]; - e "rasd:InstanceId" [] [PCData "3"]; - e "rasd:ResourceType" [] [PCData "23"]; - e "rasd:UsbPolicy" [] [PCData "Disabled"]; - ]; - (* We always add a qxl device when outputting to RHEV. - * See RHBZ#1213701 and RHBZ#1211231 for the reasoning - * behind that. - *) - e "Item" [] [ - e "rasd:Caption" [] [PCData "Graphical Controller"]; - e "rasd:InstanceId" [] [PCData (uuidgen ())]; - e "rasd:ResourceType" [] [PCData "20"]; - e "Type" [] [PCData "video"]; - e "rasd:VirtualQuantity" [] [PCData "1"]; - e "rasd:Device" [] [PCData "qxl"]; + + e "Section" ["xsi:type", "ovf:VirtualHardwareSection_Type"] [ + e "Info" [] [PCData (sprintf "%d CPU, %Ld Memory" source.s_vcpu memsize_mb)]; + e "Item" [] [ + e "rasd:Caption" [] [PCData (sprintf "%d virtual cpu" source.s_vcpu)]; + e "rasd:Description" [] [PCData "Number of virtual CPU"]; + e "rasd:InstanceId" [] [PCData "1"]; + e "rasd:ResourceType" [] [PCData "3"]; + e "rasd:num_of_sockets" [] [PCData (string_of_int source.s_vcpu)]; + e "rasd:cpu_per_socket"[] [PCData "1"]; + ]; + e "Item" [] [ + e "rasd:Caption" [] [PCData (sprintf "%Ld MB of memory" memsize_mb)]; + e "rasd:Description" [] [PCData "Memory Size"]; + e "rasd:InstanceId" [] [PCData "2"]; + e "rasd:ResourceType" [] [PCData "4"]; + e "rasd:AllocationUnits" [] [PCData "MegaBytes"]; + e "rasd:VirtualQuantity" [] [PCData (Int64.to_string memsize_mb)]; + ]; + e "Item" [] [ + e "rasd:Caption" [] [PCData "USB Controller"]; + e "rasd:InstanceId" [] [PCData "3"]; + e "rasd:ResourceType" [] [PCData "23"]; + e "rasd:UsbPolicy" [] [PCData "Disabled"]; + ]; + (* We always add a qxl device when outputting to RHEV. + * See RHBZ#1213701 and RHBZ#1211231 for the reasoning + * behind that. + *) + e "Item" [] [ + e "rasd:Caption" [] [PCData "Graphical Controller"]; + e "rasd:InstanceId" [] [PCData (uuidgen ())]; + e "rasd:ResourceType" [] [PCData "20"]; + e "Type" [] [PCData "video"]; + e "rasd:VirtualQuantity" [] [PCData "1"]; + e "rasd:Device" [] [PCData "qxl"]; + ] ] ] - ] + ) ] in (* Add disks to the OVF XML. *) -- 2.7.4

2 1

[PATCH 1/2] ruby: Print exceptions thrown by event callbacks.
by Richard W.M. Jones 10 Jun '16

10 Jun '16

--- generator/ruby.ml | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/generator/ruby.ml b/generator/ruby.ml index 97ccfdc..0f71ccc 100644 --- a/generator/ruby.ml +++ b/generator/ruby.ml @@ -404,15 +404,17 @@ event_callback_wrapper_wrapper (VALUE argvv) return Qnil; } +/* Callbacks aren't supposed to throw exceptions. We just print the + * exception on stderr and hope for the best. + */ static VALUE event_callback_handle_exception (VALUE not_used, VALUE exn) { - /* Callbacks aren't supposed to throw exceptions. */ - fprintf (stderr, \"libguestfs: exception in callback!\\n\"); + volatile VALUE message; - /* XXX We could print the exception, but it's very difficult from - * a Ruby extension. - */ + message = rb_funcall (exn, rb_intern (\"to_s\"), 0); + fprintf (stderr, \"libguestfs: exception in callback: %%s\\n\", + StringValueCStr (message)); return Qnil; } -- 2.7.4

2 5

[PATCH] filesystems: don't try to get the size of btrfs subvolume
by Cédric Bosdonnat 09 Jun '16

09 Jun '16

virt-filesystem -l tries to get the size of btrfs subvolumes, which results in an error. Teach it to skip the subvolumes. --- cat/filesystems.c | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-) diff --git a/cat/filesystems.c b/cat/filesystems.c index f1c2852..f679517 100644 --- a/cat/filesystems.c +++ b/cat/filesystems.c @@ -476,9 +476,27 @@ do_output_filesystems (void) } } if ((columns & COLUMN_SIZE)) { - size = guestfs_blockdev_getsize64 (g, fses[i]); - if (size == -1) + CLEANUP_FREE char *device = guestfs_mountable_device (g, fses[i]); + CLEANUP_FREE char *subvolume = NULL; + + guestfs_push_error_handler (g, NULL, NULL); + + subvolume = guestfs_mountable_subvolume (g, fses[i]); + if (subvolume == NULL && guestfs_last_errno (g) != EINVAL) { + fprintf (stderr, + _("%s: cannot determine the subvolume for %s: %s (%d)\n"), + guestfs_int_program_name, fses[i], + guestfs_last_error (g), guestfs_last_errno (g)); exit (EXIT_FAILURE); + } + + guestfs_pop_error_handler (g); + + if (!device || !subvolume) { + size = guestfs_blockdev_getsize64 (g, fses[i]); + if (size == -1) + exit (EXIT_FAILURE); + } } if (is_md (fses[i])) -- 2.6.6

2 1

[PATCH] customize: Add --uninstall operation.
by Richard W.M. Jones 07 Jun '16

07 Jun '16

--- customize/customize_run.ml | 30 ++++++++++++++++++++++++++++++ generator/customize.ml | 17 +++++++++++++++-- 2 files changed, 45 insertions(+), 2 deletions(-) diff --git a/customize/customize_run.ml b/customize/customize_run.ml index c9d9d7d..b2506d1 100644 --- a/customize/customize_run.ml +++ b/customize/customize_run.ml @@ -151,6 +151,31 @@ exec >>%s 2>&1 | pm -> error_unimplemented_package_manager (s_"--update") pm + and guest_uninstall_command packages = + let quoted_args = String.concat " " (List.map quote packages) in + match g#inspect_get_package_management root with + | "apk" -> sprintf "apk del %s" quoted_args + | "apt" -> + (* http://unix.stackexchange.com/questions/22820 *) + sprintf " + export DEBIAN_FRONTEND=noninteractive + apt_opts='-q -y -o Dpkg::Options::=--force-confnew' + apt-get $apt_opts update + apt-get $apt_opts remove %s + " quoted_args + | "dnf" -> sprintf "dnf -y remove %s" quoted_args + | "pisi" -> sprintf "pisi rm %s" quoted_args + | "pacman" -> sprintf "pacman -R %s" quoted_args + | "urpmi" -> sprintf "urpme %s" quoted_args + | "xbps" -> sprintf "xbps-remove -Sy %s" quoted_args + | "yum" -> sprintf "yum -y remove %s" quoted_args + | "zypper" -> sprintf "zypper -n rm -l %s" quoted_args + + | "unknown" -> + error_unknown_package_manager (s_"--uninstall") + | pm -> + error_unimplemented_package_manager (s_"--uninstall") pm + (* Windows has package_management == "unknown". *) and error_unknown_package_manager flag = error (f_"cannot use '%s' because no package manager has been detected for this guest OS.\n\nIf this guest OS is a common one with ordinary package management then this may have been caused by a failure of libguestfs inspection.\n\nFor OSes such as Windows that lack package management, this is not possible. Try using one of the '--firstboot*' flags instead (described in the manual).") flag @@ -330,6 +355,11 @@ exec >>%s 2>&1 message (f_"Running touch: %s") path; g#touch path + | `UninstallPackages pkgs -> + message (f_"Uninstalling packages: %s") (String.concat " " pkgs); + let cmd = guest_uninstall_command pkgs in + do_run ~display:cmd cmd + | `Update -> message (f_"Updating packages"); let cmd = guest_update_command () in diff --git a/generator/customize.ml b/generator/customize.ml index 3d3f978..496077b 100644 --- a/generator/customize.ml +++ b/generator/customize.ml @@ -214,7 +214,7 @@ installed during the image build using the guest's package manager For an overview on the different ways to install packages, see L<virt-builder(1)/INSTALLING PACKAGES>. -See also I<--update>."; +See also I<--update>, I<--uninstall>."; }; { op_name = "link"; @@ -424,6 +424,19 @@ string like C<Europe/London>"; This command performs a L<touch(1)>-like operation on C<FILE>."; }; + { op_name = "uninstall"; + op_type = StringList "PKG,PKG.."; + op_discrim = "`UninstallPackages"; + op_shortdesc = "Uninstall package(s)"; + op_pod_longdesc = "\ +Uninstall the named packages (a comma-separated list). These are +removed during the image build using the guest's package manager +(eg. apt, yum, etc.). Dependent packages may also need to be +uninstalled to satisfy the request. + +See also I<--install>, I<--update>."; + }; + { op_name = "update"; op_type = Unit; op_discrim = "`Update"; @@ -433,7 +446,7 @@ Do the equivalent of C<yum update>, C<apt-get upgrade>, or whatever command is required to update the packages already installed in the template to their latest versions. -See also I<--install>."; +See also I<--install>, I<--uninstall>."; }; { op_name = "upload"; -- 2.7.4

1 1

[PATCH] v2v:windows: prevent Parallels drivers from loading at boot
by Roman Kagan 06 Jun '16

06 Jun '16

Parallels proprietary hypervisor uses RDPMC as the hypercall instruction. As this instruction is supported since early P6 family, the drivers didn't even bother to check for the presence of the corresponding feature in CPUID. In QEMU/KVM, however, this instruction triggers #GP unless the VM is run with PMU (performance monitoring unit) enabled, which is often not the case (due to its impact on perfromance and migratability). So, to prevent crashes upon conversion, stop respective drivers from loading by disabling the corresponding services. Note that the services are being disabled ("Start" value set to 4) rather than their node removed, to avoid confusing the uninstaller which is scheduled to run from a firstboot script. In addition, prl_strg (storage filter driver) is unlinked from the storage subsystem following the DelReg directive from its .inf file, otherwise Windows crashes with BSOD 0x7b due to missing dependency of the storage subsystem. Signed-off-by: Roman Kagan <rkagan(a)virtuozzo.com> --- v2v/convert_windows.ml | 46 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/v2v/convert_windows.ml b/v2v/convert_windows.ml index 49811a7..012f03d 100644 --- a/v2v/convert_windows.ml +++ b/v2v/convert_windows.ml @@ -318,6 +318,7 @@ if errorlevel 3010 exit /b 0 debug "current ControlSet is %s" current_cs; disable_services root current_cs; + disable_prl_drivers root current_cs; disable_autoreboot root current_cs; Windows_virtio.install_drivers g inspect systemroot root current_cs rcaps @@ -344,6 +345,51 @@ if errorlevel 3010 exit /b 0 ) ) disable + and disable_prl_drivers root current_cs = + (* Prevent Parallels drivers from loading at boot. *) + let services = Windows.get_node g root [current_cs; "Services"] in + let prl_svcs = [ "prl_boot"; "prl_dd"; "prl_eth5"; "prl_fs"; "prl_memdev"; + "prl_mouf"; "prl_pv32"; "prl_pv64"; "prl_scsi"; + "prl_sound"; "prl_strg"; "prl_tg"; "prl_time"; + "prl_uprof"; "prl_va" ] in + + match services with + | None -> () + | Some services -> + List.iter ( + fun svc -> + let svc_node = g#hivex_node_get_child services svc in + if svc_node <> 0L then ( + (* Disable the service rather than delete the node as it would + * confuse the uninstaller called from firstboot script. *) + g#hivex_node_set_value svc_node "Start" 4_L (le32_of_int 4_L) + ) + ) prl_svcs; + + (* perfrom the equivalent of DelReg from prl_strg.inf: + * HKLM, System\CurrentControlSet\Control\Class\{4d36e967-e325-11ce-bfc1-08002be10318}, LowerFilters, 0x00018002, prl_strg + *) + let strg_cls = Windows.get_node g root + [current_cs; "Control"; "Class"; + "{4d36e967-e325-11ce-bfc1-08002be10318}"] in + match strg_cls with + | None -> () + | Some strg_cls -> + let lfkey = "LowerFilters" in + let valueh = g#hivex_node_get_value strg_cls lfkey in + if valueh <> 0L then ( + let data = g#hivex_value_value valueh in + let filters = String.nsplit "\000" (Regedit.decode_utf16le data) in + let filters = List.filter ( + fun x -> x <> "prl_strg" && x <> "" + ) filters in + let filters = List.map ( + fun x -> Regedit.encode_utf16le x ^ "\000\000" + ) (filters @ [""]) in + let data = String.concat "" filters in + g#hivex_node_set_value strg_cls lfkey 7_L data + ) + and disable_autoreboot root current_cs = (* If the guest reboots after a crash, it's hard to see the original * error (eg. the infamous 0x0000007B). Turn off autoreboot. -- 2.5.5

2 2