January 2020 - Libguestfs - libguestfs List Archives

Re: [Libguestfs] [PATCH v2] mlcustomize: Trim whitespaces from commands read from file (RHBZ#1351000)
by Richard W.M. Jones 24 Feb '20

24 Feb '20

On Wed, Jan 29, 2020 at 03:07:12PM +0100, Martin Kletzander wrote: > From: Martin Kletzander <mkletzan(a)redhat.com> > > The first split does not care about the whole string, it is just trying to get > the command name in front, so triml is just right. > > Signed-off-by: Martin Kletzander <mkletzan(a)redhat.com> > --- > mlcustomize/customize_cmdline.ml | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/mlcustomize/customize_cmdline.ml b/mlcustomize/customize_cmdline.ml > index c062379879e2..abd21a4cbca5 100644 > --- a/mlcustomize/customize_cmdline.ml > +++ b/mlcustomize/customize_cmdline.ml > @@ -481,6 +481,7 @@ let rec argspec () = > ] in > let lines = read_whole_file filename in > let lines = String.lines_split lines in > + let lines = List.map String.triml lines in > let lines = List.filter ( > fun line -> > String.length line > 0 && line.[0] <> '#' Seems OK to me, so ACK. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v

2 2

[PATCH v2v v2 0/2] rhv-upload: Validate UUIDs and check they don't exist already
by Martin Kletzander 24 Feb '20

24 Feb '20

My stab v2 at fixing this: https://bugzilla.redhat.com/show_bug.cgi?id=1789279 It took me quite some time to go through the whole rfc 4122 just to realize we do not need to do anything with the versions. v2: - Use EEXIST instead of EINVAL - Put the colliding UUID into the error - Do not evaluate the PCRE needlessly multiple times v1: https://www.redhat.com/archives/libguestfs/2020-January/msg00184.html Martin Kletzander (2): rhv-upload: Validate UUIDs passed to -oo rhv-disk-uuid (RHBZ#1789279) rhv-upload: Check that rhv-disk-uuid is not already taken (RHBZ#1789279) v2v/output_rhv_upload.ml | 12 ++++++++++++ v2v/rhv-upload-plugin.py | 12 ++++++++++++ 2 files changed, 24 insertions(+) -- 2.25.0

2 7

[PATCH v2] add versioned directory for guest agent on EL8
by Tomáš Golembiovský 24 Feb '20

24 Feb '20

There was no source directory for EL8 guest agent (only EL6 and EL7). RHBZ#1791802 Signed-off-by: Tomáš Golembiovský <tgolembi(a)redhat.com> --- v2v/windows_virtio.ml | 1 + 1 file changed, 1 insertion(+) diff --git a/v2v/windows_virtio.ml b/v2v/windows_virtio.ml index 5ec7664b..86bc8c0a 100644 --- a/v2v/windows_virtio.ml +++ b/v2v/windows_virtio.ml @@ -192,6 +192,7 @@ and install_linux_tools g inspect = (match inspect.i_major_version with | 6 -> Some "el6" | 7 -> Some "el7" + | 8 -> Some "el8" | _ -> None) | "sles" | "suse-based" | "opensuse" -> Some "lp151" | _ -> None in -- 2.24.1

3 2

[PATCH 0/2] Delay firstboot scripts to some later time
by Tomáš Golembiovský 20 Feb '20

20 Feb '20

When firstboot is used from virt-v2v the scripts, if not fast enough, can get killed by Windows. After windows installs virtio drivers injected by virt-v2v it performs some internall reboot, stopping all the running services and killing any running firstboot script. This is problem mostly for MSI installs (like qemu-ga that was added recently) that can take several seconds to finish. This change is little bit controversial in fact that it relies on powershell. This is not available in early versions (without service packs) of XP and 2003 and our support matrix still mentions these Windows versions. The change can also be problem for those who really wish to perform an action early in the boot... if there are such users. Tomáš Golembiovský (2): firstboot: use absolute path to rhsrvany firstboot: schedule firstboot as delayed task mlcustomize/firstboot.ml | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) -- 2.24.0

3 9

[PATCH] appliance: Add ntfs-3g-system-compression (RHBZ#1703463).
by Richard W.M. Jones 14 Feb '20

14 Feb '20

This package in Fedora enables optional support for Windows 10 "CompactOS" (file-level compression), read-only, which is sufficient for inspecting Windows guests and doing certain types of modifications to them. Virt-v2v appears to work, but it may be that anything that involves modifying a compressed file might not work. I couldn't find the equivalent package in Debian or SUSE. It's available in Arch AUR although I didn't verify that part of the change actually works there (but should be safe because supermin ignores packages that are not known about on the target system). --- appliance/packagelist.in | 2 ++ 1 file changed, 2 insertions(+) diff --git a/appliance/packagelist.in b/appliance/packagelist.in index e1cd173f0..13c83d8e4 100644 --- a/appliance/packagelist.in +++ b/appliance/packagelist.in @@ -43,6 +43,7 @@ ifelse(REDHAT,1, nilfs-utils ntfsprogs ntfs-3g + ntfs-3g-system-compression openssh-clients pcre policycoreutils @@ -127,6 +128,7 @@ ifelse(ARCHLINUX,1, multipath-tools dnl for kpartx nilfs-utils ntfs-3g + ntfs-3g-system-compression pcre reiserfsprogs systemd -- 2.25.0

2 1

[PATCH libnbd 0/2] api: Add support for AF_VSOCK.
by Richard W.M. Jones 10 Feb '20

10 Feb '20

This is a series of patches to libnbd and nbdkit adding AF_VSOCK support. On the host side it allows you to start an nbdkit instance which listens on a virtio-vsock socket: $ ./nbdkit -fv --vsock memory 1G ... nbdkit: debug: bound to vsock 2:10809 On the guest side you can then use libnbd to connect to the server: $ ./run nbdsh -c 'h.connect_vsock(2, 10809)' -c 'print(h.get_size())' 1073741824 $ ./run nbdfuse mp --vsock 2 10809 & $ ll mp/ total 0 -rw-rw-rw-. 1 rjones rjones 1073741824 Oct 18 16:23 nbd $ dd if=/dev/random of=mp/nbd bs=1024 count=100 conv=notrunc,nocreat dd: warning: partial read (84 bytes); suggest iflag=fullblock 0+100 records in 0+100 records out 6851 bytes (6.9 kB, 6.7 KiB) copied, 0.013797 s, 497 kB/s (Performance of FUSE is not great, it should be better using raw libnbd.) I mainly wrote this to show that it can be done. It's unclear if this would be faster or slower than the usual way that NBD devices are exposed to guests via virtio-blk/-scsi. https://wiki.qemu.org/Features/VirtioVsock Thanks: Stefan Hajnoczi for help with debugging this.

3 10

[PATCH v2] launch: add support for autodetection of appliance image format
by Pavel Butsykin 06 Feb '20

06 Feb '20

This feature allows you to use different image formats for the fixed appliance. The raw format is used by default. Signed-off-by: Pavel Butsykin <pbutsykin(a)virtuozzo.com> --- lib/launch-direct.c | 2 ++ lib/launch-libvirt.c | 19 ++++++++++++------- m4/guestfs_appliance.m4 | 11 +++++++++++ 3 files changed, 25 insertions(+), 7 deletions(-) diff --git a/lib/launch-direct.c b/lib/launch-direct.c index 0be662e25..b9b54857a 100644 --- a/lib/launch-direct.c +++ b/lib/launch-direct.c @@ -592,7 +592,9 @@ launch_direct (guestfs_h *g, void *datav, const char *arg) append_list ("id=appliance"); append_list ("cache=unsafe"); append_list ("if=none"); +#ifndef APPLIANCE_FMT_AUTO append_list ("format=raw"); +#endif } end_list (); start_list ("-device") { append_list ("scsi-hd"); diff --git a/lib/launch-libvirt.c b/lib/launch-libvirt.c index 4adb2cfb3..030ea6911 100644 --- a/lib/launch-libvirt.c +++ b/lib/launch-libvirt.c @@ -212,9 +212,10 @@ get_source_format_or_autodetect (guestfs_h *g, struct drive *drv) /** * Create a qcow2 format overlay, with the given C<backing_drive> - * (file). The C<format> parameter, which must be non-NULL, is the - * backing file format. This is used to create the appliance overlay, - * and also for read-only drives. + * (file). The C<format> parameter is the backing file format. + * The C<format> parameter can be NULL, in this case the backing + * format will be determined automatically. This is used to create + * the appliance overlay, and also for read-only drives. */ static char * make_qcow2_overlay (guestfs_h *g, const char *backing_drive, @@ -223,8 +224,6 @@ make_qcow2_overlay (guestfs_h *g, const char *backing_drive, char *overlay; struct guestfs_disk_create_argv optargs; - assert (format != NULL); - if (guestfs_int_lazy_make_tmpdir (g) == -1) return NULL; @@ -232,8 +231,10 @@ make_qcow2_overlay (guestfs_h *g, const char *backing_drive, optargs.bitmask = GUESTFS_DISK_CREATE_BACKINGFILE_BITMASK; optargs.backingfile = backing_drive; - optargs.bitmask |= GUESTFS_DISK_CREATE_BACKINGFORMAT_BITMASK; - optargs.backingformat = format; + if (format) { + optargs.bitmask |= GUESTFS_DISK_CREATE_BACKINGFORMAT_BITMASK; + optargs.backingformat = format; + } if (guestfs_disk_create_argv (g, overlay, "qcow2", -1, &optargs) == -1) { free (overlay); @@ -461,7 +462,11 @@ launch_libvirt (guestfs_h *g, void *datav, const char *libvirt_uri) /* Note that appliance can be NULL if using the old-style appliance. */ if (appliance) { +#ifdef APPLIANCE_FMT_AUTO + params.appliance_overlay = make_qcow2_overlay (g, appliance, NULL); +#else params.appliance_overlay = make_qcow2_overlay (g, appliance, "raw"); +#endif if (!params.appliance_overlay) goto cleanup; } diff --git a/m4/guestfs_appliance.m4 b/m4/guestfs_appliance.m4 index 81c43879f..4e1ec8135 100644 --- a/m4/guestfs_appliance.m4 +++ b/m4/guestfs_appliance.m4 @@ -139,3 +139,14 @@ AC_SUBST([GUESTFS_DEFAULT_PATH]) AC_DEFINE_UNQUOTED([GUESTFS_DEFAULT_PATH], ["$GUESTFS_DEFAULT_PATH"], [Define guestfs default path.]) + +AC_ARG_ENABLE([appliance-fmt-auto], + [AS_HELP_STRING([--enable-appliance-fmt-auto], + [enable autodetection of appliance image format @<:@default=no@:>@])], + [ENABLE_APPLIANCE_FMT_AUTO="$enableval"], + [ENABLE_APPLIANCE_FMT_AUTO=no]) + +if test "x$ENABLE_APPLIANCE_FMT_AUTO" = "xyes"; then + AC_DEFINE([APPLIANCE_FMT_AUTO], [1], + [Define to 1 if enabled autodetection of appliance image format.]) +fi -- 2.13.0

2 4

[PATCH 0/2] Fixes and tweak to the installation of qemu-ga MSI
by Tomáš Golembiovský 05 Feb '20

05 Feb '20

This, together with the changes to common repo are fixes to the installation qemu-ga MSI. There is still an issue that I did not figure yet how to fix. On Windows 10 it fails to register the QEMU-GA service. Tomáš Golembiovský (2): windows: fix detection of qemu-ga installer on RHV windows: small tweaks of qemu-ga firstboot script v2v/convert_windows.ml | 8 +++++++- v2v/windows_virtio.ml | 5 ++--- 2 files changed, 9 insertions(+), 4 deletions(-) -- 2.24.0

3 8

[v2v PATCH v2 0/3] Use libosinfo for query device drivers
by Pino Toscano 03 Feb '20

03 Feb '20

This patch series integrates libosinfo in virt-v2v to get the list of files for Windows from libosinfo, if possible. The actual data is still from virtio-win, just unpacked. Changes from v1: - adapt to use the priority in libosinfo 1.7.0+ - filter out non-pre-installable drivers - collect all the drivers matching the requirements, not just the first, sorting them by priority like libosinfo does to get the best ones - actually upload the files by the lowercase file name - few minor fixes Pino Toscano (3): build: require libosinfo v2v: add a minimal libosinfo interface v2v: try to get windows driver files from libosinfo m4/guestfs-v2v.m4 | 3 + v2v/Makefile.am | 9 +- v2v/libosinfo-c.c | 237 ++++++++++++++++++++++++++++++++++++++++ v2v/libosinfo.ml | 53 +++++++++ v2v/libosinfo.mli | 48 ++++++++ v2v/libosinfo_utils.ml | 34 ++++++ v2v/libosinfo_utils.mli | 26 +++++ v2v/windows_virtio.ml | 78 ++++++++++++- 8 files changed, 483 insertions(+), 5 deletions(-) create mode 100644 v2v/libosinfo-c.c create mode 100644 v2v/libosinfo.ml create mode 100644 v2v/libosinfo.mli create mode 100644 v2v/libosinfo_utils.ml create mode 100644 v2v/libosinfo_utils.mli -- 2.24.1

2 4

[v2v PATCH 1/2] Add back guestcaps as parameter of output#prepare_targets
by Pino Toscano 03 Feb '20

03 Feb '20

It will be used to do extra checks in the output before copying the disks. Partially revert commit 3bafec4e693a25ef1c84abc0fd1bc3251862c7de. --- v2v/output_glance.ml | 2 +- v2v/output_json.ml | 2 +- v2v/output_libvirt.ml | 2 +- v2v/output_local.ml | 2 +- v2v/output_null.ml | 2 +- v2v/output_openstack.ml | 2 +- v2v/output_qemu.ml | 2 +- v2v/output_rhv.ml | 2 +- v2v/output_rhv_upload.ml | 2 +- v2v/output_vdsm.ml | 2 +- v2v/types.ml | 2 +- v2v/types.mli | 2 +- v2v/v2v.ml | 3 ++- 13 files changed, 14 insertions(+), 13 deletions(-) diff --git a/v2v/output_glance.ml b/v2v/output_glance.ml index 0eb8b5e6..21e8cc62 100644 --- a/v2v/output_glance.ml +++ b/v2v/output_glance.ml @@ -59,7 +59,7 @@ object method supported_firmware = [ TargetBIOS; TargetUEFI ] - method prepare_targets _ overlays = + method prepare_targets _ overlays _ = (* Write targets to a temporary local file - see above for reason. *) List.map (fun (_, ov) -> TargetFile (tmpdir // ov.ov_sd)) overlays diff --git a/v2v/output_json.ml b/v2v/output_json.ml index 29c4a1d0..b23420a7 100644 --- a/v2v/output_json.ml +++ b/v2v/output_json.ml @@ -71,7 +71,7 @@ class output_json dir json_options = object method as_options = sprintf "-o json -os %s" dir - method prepare_targets source_name overlays = + method prepare_targets source_name overlays _ = List.mapi ( fun i (_, ov) -> let outname = diff --git a/v2v/output_libvirt.ml b/v2v/output_libvirt.ml index dfd5d1a3..6c6b23ed 100644 --- a/v2v/output_libvirt.ml +++ b/v2v/output_libvirt.ml @@ -75,7 +75,7 @@ object (self) | None -> sprintf "-o libvirt -os %s" output_pool | Some uri -> sprintf "-o libvirt -oc %s -os %s" uri output_pool - method prepare_targets source_name overlays = + method prepare_targets source_name overlays _ = (* Get the capabilities from libvirt. *) let xml = try diff --git a/v2v/output_local.ml b/v2v/output_local.ml index 5eb743a4..48c29eeb 100644 --- a/v2v/output_local.ml +++ b/v2v/output_local.ml @@ -31,7 +31,7 @@ class output_local dir = object method as_options = sprintf "-o local -os %s" dir - method prepare_targets source_name overlays = + method prepare_targets source_name overlays _ = List.map ( fun (_, ov) -> TargetFile (dir // source_name ^ "-" ^ ov.ov_sd) ) overlays diff --git a/v2v/output_null.ml b/v2v/output_null.ml index df451f9f..36710e68 100644 --- a/v2v/output_null.ml +++ b/v2v/output_null.ml @@ -89,7 +89,7 @@ object (* Force raw output, ignoring -of command line option. *) method override_output_format _ = Some "raw" - method prepare_targets _ overlays = + method prepare_targets _ overlays _ = if can_use_qemu_null_co_device () then ( let json_params = [ "file.driver", JSON.String "null-co"; diff --git a/v2v/output_openstack.ml b/v2v/output_openstack.ml index caaedf45..81c55a43 100644 --- a/v2v/output_openstack.ml +++ b/v2v/output_openstack.ml @@ -400,7 +400,7 @@ object (* Create the Cinder volumes, wait for them to attach to the * appliance, and return the paths of the /dev devices. *) - method prepare_targets source_name overlays = + method prepare_targets source_name overlays _ = (* Set up an at-exit handler so we: * (1) Unconditionally detach volumes. * (2) Delete the volumes, but only if conversion was not successful. diff --git a/v2v/output_qemu.ml b/v2v/output_qemu.ml index 22e8581a..0156370e 100644 --- a/v2v/output_qemu.ml +++ b/v2v/output_qemu.ml @@ -36,7 +36,7 @@ object method as_options = sprintf "-o qemu -os %s%s" dir (if qemu_boot then " --qemu-boot" else "") - method prepare_targets source_name overlays = + method prepare_targets source_name overlays _ = List.map ( fun (_, ov) -> TargetFile (dir // source_name ^ "-" ^ ov.ov_sd) diff --git a/v2v/output_rhv.ml b/v2v/output_rhv.ml index a9102c06..133d03a8 100644 --- a/v2v/output_rhv.ml +++ b/v2v/output_rhv.ml @@ -151,7 +151,7 @@ object * 'os' is the output storage (-os nfs:/export). 'targets' describes * the destination files. We modify and return this list. *) - method prepare_targets _ overlays = + method prepare_targets _ overlays _ = let mp, uuid = mount_and_check_storage_domain (s_"Export Storage Domain") os in esd_mp <- mp; diff --git a/v2v/output_rhv_upload.ml b/v2v/output_rhv_upload.ml index 14153db3..01b1ce4a 100644 --- a/v2v/output_rhv_upload.ml +++ b/v2v/output_rhv_upload.ml @@ -290,7 +290,7 @@ object (* rhev-apt.exe will be installed (if available). *) method install_rhev_apt = true - method prepare_targets source_name overlays = + method prepare_targets source_name overlays _ = let uuids = match rhv_options.rhv_disk_uuids with | None -> diff --git a/v2v/output_vdsm.ml b/v2v/output_vdsm.ml index 1c84439e..bfb9856d 100644 --- a/v2v/output_vdsm.ml +++ b/v2v/output_vdsm.ml @@ -144,7 +144,7 @@ object * 'os' is the output storage domain (-os /rhv/data/<data center>/<data domain>) * this is already mounted path. *) - method prepare_targets _ overlays = + method prepare_targets _ overlays _ = if List.length vdsm_options.image_uuids <> List.length overlays || List.length vdsm_options.vol_uuids <> List.length overlays then error (f_"the number of ‘-oo vdsm-image-uuid’ and ‘-oo vdsm-vol-uuid’ parameters passed on the command line has to match the number of guest disk images (for this guest: %d)") diff --git a/v2v/types.ml b/v2v/types.ml index 8de48aec..a6a52294 100644 --- a/v2v/types.ml +++ b/v2v/types.ml @@ -530,7 +530,7 @@ class virtual output = object method check_target_firmware (_ : guestcaps) (_ : target_firmware) = () method override_output_format (_ : overlay) = (None : string option) method transfer_format t = t.target_format - method virtual prepare_targets : string -> (string * overlay) list -> target_file list + method virtual prepare_targets : string -> (string * overlay) list -> guestcaps -> target_file list method disk_create = (open_guestfs ())#disk_create method disk_copied (_ : target) (_ : int) (_ : int) = () method virtual create_metadata : source -> target list -> target_buses -> guestcaps -> inspect -> target_firmware -> unit diff --git a/v2v/types.mli b/v2v/types.mli index 9ceee8a2..483b8910 100644 --- a/v2v/types.mli +++ b/v2v/types.mli @@ -492,7 +492,7 @@ class virtual output : object (** Typically the transfer format is same as the target format, but some outputs may need to overide the transfer format, but create disk using target format. *) - method virtual prepare_targets : string -> (string * overlay) list -> target_file list + method virtual prepare_targets : string -> (string * overlay) list -> guestcaps -> target_file list (** Called after conversion but before copying to prepare (but {b not} create) the target file. The [(string * overlay list)] parameter is a list of the (format, overlay) for each target disk. If diff --git a/v2v/v2v.ml b/v2v/v2v.ml index dc1dedd7..9fd63442 100644 --- a/v2v/v2v.ml +++ b/v2v/v2v.ml @@ -173,7 +173,8 @@ let rec main () = let target_formats = get_target_formats cmdline output overlays in let target_files = output#prepare_targets source.s_name - (List.combine target_formats overlays) in + (List.combine target_formats overlays) + guestcaps in List.map ( fun (target_file, target_format, target_overlay) -> { target_file; target_format; target_overlay } -- 2.24.1

2 2