November 2018 - Libguestfs - libguestfs List Archives

[PATCH v2] launch: add support for autodetection of appliance image format
by Pavel Butsykin 06 Feb '20

06 Feb '20

This feature allows you to use different image formats for the fixed appliance. The raw format is used by default. Signed-off-by: Pavel Butsykin <pbutsykin(a)virtuozzo.com> --- lib/launch-direct.c | 2 ++ lib/launch-libvirt.c | 19 ++++++++++++------- m4/guestfs_appliance.m4 | 11 +++++++++++ 3 files changed, 25 insertions(+), 7 deletions(-) diff --git a/lib/launch-direct.c b/lib/launch-direct.c index 0be662e25..b9b54857a 100644 --- a/lib/launch-direct.c +++ b/lib/launch-direct.c @@ -592,7 +592,9 @@ launch_direct (guestfs_h *g, void *datav, const char *arg) append_list ("id=appliance"); append_list ("cache=unsafe"); append_list ("if=none"); +#ifndef APPLIANCE_FMT_AUTO append_list ("format=raw"); +#endif } end_list (); start_list ("-device") { append_list ("scsi-hd"); diff --git a/lib/launch-libvirt.c b/lib/launch-libvirt.c index 4adb2cfb3..030ea6911 100644 --- a/lib/launch-libvirt.c +++ b/lib/launch-libvirt.c @@ -212,9 +212,10 @@ get_source_format_or_autodetect (guestfs_h *g, struct drive *drv) /** * Create a qcow2 format overlay, with the given C<backing_drive> - * (file). The C<format> parameter, which must be non-NULL, is the - * backing file format. This is used to create the appliance overlay, - * and also for read-only drives. + * (file). The C<format> parameter is the backing file format. + * The C<format> parameter can be NULL, in this case the backing + * format will be determined automatically. This is used to create + * the appliance overlay, and also for read-only drives. */ static char * make_qcow2_overlay (guestfs_h *g, const char *backing_drive, @@ -223,8 +224,6 @@ make_qcow2_overlay (guestfs_h *g, const char *backing_drive, char *overlay; struct guestfs_disk_create_argv optargs; - assert (format != NULL); - if (guestfs_int_lazy_make_tmpdir (g) == -1) return NULL; @@ -232,8 +231,10 @@ make_qcow2_overlay (guestfs_h *g, const char *backing_drive, optargs.bitmask = GUESTFS_DISK_CREATE_BACKINGFILE_BITMASK; optargs.backingfile = backing_drive; - optargs.bitmask |= GUESTFS_DISK_CREATE_BACKINGFORMAT_BITMASK; - optargs.backingformat = format; + if (format) { + optargs.bitmask |= GUESTFS_DISK_CREATE_BACKINGFORMAT_BITMASK; + optargs.backingformat = format; + } if (guestfs_disk_create_argv (g, overlay, "qcow2", -1, &optargs) == -1) { free (overlay); @@ -461,7 +462,11 @@ launch_libvirt (guestfs_h *g, void *datav, const char *libvirt_uri) /* Note that appliance can be NULL if using the old-style appliance. */ if (appliance) { +#ifdef APPLIANCE_FMT_AUTO + params.appliance_overlay = make_qcow2_overlay (g, appliance, NULL); +#else params.appliance_overlay = make_qcow2_overlay (g, appliance, "raw"); +#endif if (!params.appliance_overlay) goto cleanup; } diff --git a/m4/guestfs_appliance.m4 b/m4/guestfs_appliance.m4 index 81c43879f..4e1ec8135 100644 --- a/m4/guestfs_appliance.m4 +++ b/m4/guestfs_appliance.m4 @@ -139,3 +139,14 @@ AC_SUBST([GUESTFS_DEFAULT_PATH]) AC_DEFINE_UNQUOTED([GUESTFS_DEFAULT_PATH], ["$GUESTFS_DEFAULT_PATH"], [Define guestfs default path.]) + +AC_ARG_ENABLE([appliance-fmt-auto], + [AS_HELP_STRING([--enable-appliance-fmt-auto], + [enable autodetection of appliance image format @<:@default=no@:>@])], + [ENABLE_APPLIANCE_FMT_AUTO="$enableval"], + [ENABLE_APPLIANCE_FMT_AUTO=no]) + +if test "x$ENABLE_APPLIANCE_FMT_AUTO" = "xyes"; then + AC_DEFINE([APPLIANCE_FMT_AUTO], [1], + [Define to 1 if enabled autodetection of appliance image format.]) +fi -- 2.13.0

2 4

1.39 proposal: Let's split up the libguestfs git repo and tarballs
by Richard W.M. Jones 29 Nov '19

29 Nov '19

My contention is that the libguestfs git repository is too large and unwieldy. There are too many separate, unrelated projects and as a result of that the source has too many dependencies and takes too long to build and test. The project divides (sort of) naturally into layers -- the library, the bindings, the various virt tools -- and could be split along those lines into separate projects which can then be released and evolve at their own pace. My suggested split would be something like this: * libguestfs: The library, daemon and appliance. That would include the following directories in a single project: appliance bash contrib daemon docs examples gnulib lib logo test-tool tmp utils website * 1 project for each language binding: csharp erlang gobject golang haskell java lua ocaml php perl python ruby * virt-customize and related tools, we'd probably call this subproject "virt-builder". It would include virt-builder, virt-customize and virt-sysprep, since they share a lot of common code. * 1 project for each of the following items: small tools written in C (virt-cat, virt-filesystems, virt-log, virt-ls, virt-tail, virt-diff, virt-edit, virt-format, guestmount, virt-inspector, virt-make-fs, virt-rescue) guestfish virt-alignment-scan and virt-df virt-dib virt-get-kernel virt-resize virt-sparsify virt-v2v and virt-p2v virt-win-reg * I'd be inclined to drop the legacy Perl tools virt-tar, virt-list-filesystems, virt-list-partitions unless someone especially wished to step forward to maintain them. * common code and generator: Off to the side we'd somehow need to package up the common code and the generator for use by all of the above projects. It wouldn't be a separate project for downstream packagers, but instead the code would be included (ie. duplicated) in tarballs and upstream available as a side git repo that you'd need to include when building (git submodule?). This is somewhat unspecified. M4, PO, and tests would be split between the projects as appropriate. My proposal would be to do this incrementally, rather than all at once, moving the easier things out first. Thoughts? Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org

4 18

[nbdkit PATCH 0/3] Fix %m usage on BSD
by Eric Blake 22 Jan '19

22 Jan '19

Our use of "%m" in various error messages is testament to the project's initial life on Linux - but other than Cygwin, I know of no other platforms supporting that glibc extension. We COULD audit the code and manually turn "%m" into "%s"/strerror(errno), but that's a lot of churn. Instead, let's fix the few outliers that can't be easily wrapped, then wrap the remainder. While I was able to test this on Linux (both that no wrapper is used by default, and that faking that %m fails causes the wrapper to do the right thing), I haven't actually tried it on a BSD box, hence I'll wait for review before pushing. Eric Blake (3): main: Avoid fprintf(%m) for BSD builds log: Ensure %m sees correct errno log: Guarantee the operation of %m in nbdkit_error() docs/nbdkit-filter.pod | 13 +++++++++---- docs/nbdkit-plugin.pod | 13 +++++++++---- configure.ac | 27 +++++++++++++++++++++++++++ src/internal.h | 6 ++++++ src/log-stderr.c | 4 ++-- src/log-syslog.c | 6 +++--- src/log.c | 21 +++++++++++++++++++++ src/main.c | 4 ++-- 8 files changed, 79 insertions(+), 15 deletions(-) -- 2.17.2

2 8

[hivex PATCH] Re-allocating unused blocks before assigning new blocks
by Shreyas Khare 03 Jan '19

03 Jan '19

Hello Richard As discussed in the IRC channel, when merging a moderately large reg file (~35MB) to a hiv file (~118 MB); hivex generates a huge hiv file (~580 MB). These changes address that by creating a list of unallocated blocks and reassigning unused blocks. I used https://github.com/msuhanov/regf/blob/master/Windows%20registry%20file%20fo… as a reference for the structure of the hiv file (in addition to the source code itself) Attaching the patch file. Shreyas -- NOTICE OF CONFIDENTIALITY: At Rapid7, the privacy of our customers, partners, and employees is paramount. If you received this email in error, please notify the sender and delete it from your inbox right away. Learn how Rapid7 handles privacy at rapid7.com/privacy-policy <https://www.rapid7.com/privacy-policy/>. To opt-out of Rapid7 marketing emails, please click here <https://information.rapid7.com/manage-subscription.html> or email privacy(a)rapid7.com <mailto:mailto:privacy@rapid7.com>.

2 3

[PATCH] v2v: -o openstack: Check openstack binary exists before running it.
by Richard W.M. Jones 05 Dec '18

05 Dec '18

Improves the error message when openstack is not installed: $ virt-v2v -i disk fedora-28.img -o openstack virt-v2v: error: no binary called ‘openstack’ was found on the $PATH. We use this program to communicate with OpenStack so it must be installed to use this output mode. --- v2v/output_openstack.ml | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/v2v/output_openstack.ml b/v2v/output_openstack.ml index b5bbc9195..238ec99a9 100644 --- a/v2v/output_openstack.ml +++ b/v2v/output_openstack.ml @@ -28,6 +28,9 @@ open Common_gettext.Gettext open Types open Utils +(* Name of the openstack CLI program (on $PATH). *) +let openstack_binary = "openstack" + (* Timeout waiting for new Cinder volumes to move to "available" state. * We assume this could be quite a long time on backends which want * to preallocate the storage. @@ -172,13 +175,20 @@ class output_openstack output_conn output_password output_storage object_get_string "uuid" json ) in + let error_unless_openstack_command_exists () = + try ignore (which openstack_binary) + with Executable_not_found _ -> + error (f_"no binary called ‘%s’ was found on the $PATH. We use this program to communicate with OpenStack so it must be installed to use this output mode.") + openstack_binary + in + (* We use this convenient wrapper around [Tools_utils.run_command] * for two reasons: (1) Because we want to run openstack with * extra_args. (2) OpenStack commands are noisy so we want to * direct stdout to /dev/null unless we're in verbose mode. *) let run_openstack_command args = - let cmd = [ "openstack" ] @ extra_args @ args in + let cmd = [ openstack_binary ] @ extra_args @ args in let stdout_fd = if verbose () then None else Some (openfile "/dev/null" [O_WRONLY] 0) in @@ -191,7 +201,7 @@ class output_openstack output_conn output_password output_storage * '-f json' to the args yourself. *) let run_openstack_command_capture_json args = - let cmd = [ "openstack" ] @ extra_args @ args in + let cmd = [ openstack_binary ] @ extra_args @ args in let json, chan = Filename.open_temp_file "v2vopenstack" ".json" in unlink_on_exit json; @@ -366,6 +376,9 @@ object inherit output method precheck () = + (* Check the openstack command exists. *) + error_unless_openstack_command_exists (); + (* Run the openstack command simply to check we can connect * with the provided authentication parameters/environment * variables. Issuing a token should have only a tiny -- 2.19.0.rc0

2 1

[PATCH REPOST] Introduce a wrapper around xmlParseURI.
by Richard W.M. Jones 30 Nov '18

30 Nov '18

Previously posted: https://www.redhat.com/archives/libguestfs/2017-December/msg00046.html Rich.

2 2

[PATCH 0/3] inspection: tweak limits of package manager files
by Pino Toscano 30 Nov '18

30 Nov '18

I got a recent report of virt-v2v (via virt-p2v) failing to migrate a Fedora guest. The issue was that its /var/lib/rpm/Packages was bigger than our current limit (~410M vs 300M), hence the inspection failed. I took the liberty to refactor the limits of the these files, bumping the problematic one, and reducing the ones of the others (as they are supposed to be way smaller than the limit). Pino Toscano (3): lib: move MAX_PKG_DB_SIZE to inspect-apps lib: inspect: split limits of package manager files lib: inspect: tweak limits of package manager files lib/guestfs-internal.h | 9 --------- lib/inspect-apps.c | 23 +++++++++++++++++++---- 2 files changed, 19 insertions(+), 13 deletions(-) -- 2.17.2

2 4

[PATCH] inspect: handle os-release "opensuse-tumbleweed" as opensuse
by Pino Toscano 30 Nov '18

30 Nov '18

Followup of commit 70407cd622dda6f088a0876e1e1ae669e9f8a281 for openSUSE Thumbleweed. --- daemon/inspect_fs_unix.ml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/daemon/inspect_fs_unix.ml b/daemon/inspect_fs_unix.ml index 4dd89fa8f..925db06a3 100644 --- a/daemon/inspect_fs_unix.ml +++ b/daemon/inspect_fs_unix.ml @@ -143,7 +143,7 @@ and distro_of_os_release_id = function | "kali" -> Some DISTRO_KALI_LINUX | "mageia" -> Some DISTRO_MAGEIA | "neokylin" -> Some DISTRO_NEOKYLIN - | "opensuse" | "opensuse-leap" -> Some DISTRO_OPENSUSE + | "opensuse" | "opensuse-leap" | "opensuse-tumbleweed" -> Some DISTRO_OPENSUSE | "pld" -> Some DISTRO_PLD_LINUX | "rhel" -> Some DISTRO_RHEL | "sles" | "sled" -> Some DISTRO_SLES -- 2.17.2

2 1

[PATCH] v2v: docs: qemu-kvm-rhev only applies to RHEL 7 (RHBZ#1651977).
by Richard W.M. Jones 30 Nov '18

30 Nov '18

The previous support documentation said that for UEFI guests, when using RHEL ≥ 7.3 host, you must use qemu-kvm-rhev. For RHEL 8 this is inaccurate since qemu-kvm-rhev no longer exists. Since 7.3 is now an obsolete version of RHEL 7 we can change the text to refer to RHEL 7 only: On RHEL 7, only qemu-kvm-rhev (not qemu-kvm) is supported. Thanks: Ming Xie. --- v2v/virt-v2v-support.pod | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/v2v/virt-v2v-support.pod b/v2v/virt-v2v-support.pod index ea9c79412..c0ec054ed 100644 --- a/v2v/virt-v2v-support.pod +++ b/v2v/virt-v2v-support.pod @@ -130,7 +130,7 @@ automatically, but note that the same version of OVMF must be installed on the conversion host as is installed on the target hypervisor, else you will have to adjust paths in the metadata. -On RHEL E<ge> 7.3, only qemu-kvm-rhev (not qemu-kvm) is supported. +On RHEL 7, only qemu-kvm-rhev (not qemu-kvm) is supported. =item UEFI on OpenStack -- 2.19.0.rc0

2 1

[PATCH] Replace -nodefconfig with -no-user-config.
by Richard W.M. Jones 30 Nov '18

30 Nov '18

This option was removed from qemu for no apparent reason except to break existing consumers. It does the same as -no-user-config, added in May 2012, so use that instead. --- builder/test-console.sh | 2 +- common/qemuopts/qemuopts-tests.c | 6 +++--- common/qemuopts/qemuopts.c | 2 +- contrib/p2v/build-p2v-iso.sh | 2 +- contrib/p2v/test-p2v-iso.sh | 2 +- customize/test-firstboot.sh | 2 +- customize/test-password.pl | 2 +- customize/test-settings.sh | 2 +- lib/launch-direct.c | 4 ++-- p2v/test-virt-p2v-pxe.sh | 2 +- utils/boot-analysis/boot-analysis-timeline.c | 4 ++-- 11 files changed, 15 insertions(+), 15 deletions(-) diff --git a/builder/test-console.sh b/builder/test-console.sh index 64e5b1788..4f204f3c8 100755 --- a/builder/test-console.sh +++ b/builder/test-console.sh @@ -84,7 +84,7 @@ rm -f "$out" # Boot the guest in qemu with a serial console. Allow it to run # for a set amount of time, and then kill it. $qemu \ - -nodefconfig \ + -no-user-config \ -display none \ -machine accel=kvm:tcg \ -m 1024 \ diff --git a/common/qemuopts/qemuopts-tests.c b/common/qemuopts/qemuopts-tests.c index bb7d24d59..f8e68aef4 100644 --- a/common/qemuopts/qemuopts-tests.c +++ b/common/qemuopts/qemuopts-tests.c @@ -70,7 +70,7 @@ main (int argc, char *argv[]) qemuopts_set_binary (qopts, "qemu-system-x86_64")); CHECK_ERROR (-1, "qemuopts_add_flag", - qemuopts_add_flag (qopts, "-nodefconfig")); + qemuopts_add_flag (qopts, "-no-user-config")); CHECK_ERROR (-1, "qemuopts_add_arg", qemuopts_add_arg (qopts, "-m", "1024")); CHECK_ERROR (-1, "qemuopts_add_arg_format", @@ -116,7 +116,7 @@ main (int argc, char *argv[]) const char *expected = "qemu-system-x86_64 \\\n" - " -nodefconfig \\\n" + " -no-user-config \\\n" " -m 1024 \\\n" " -smp 4 \\\n" " -drive file=/tmp/foo,if=ide \\\n" @@ -139,7 +139,7 @@ main (int argc, char *argv[]) actual_argv = qemuopts_to_argv (qopts)); const char *expected_argv[] = { "qemu-system-x86_64", - "-nodefconfig", + "-no-user-config", "-m", "1024", "-smp", "4", "-drive", "file=/tmp/foo,if=ide", diff --git a/common/qemuopts/qemuopts.c b/common/qemuopts/qemuopts.c index 3a8dcae70..53dc8c2a3 100644 --- a/common/qemuopts/qemuopts.c +++ b/common/qemuopts/qemuopts.c @@ -196,7 +196,7 @@ last_option (struct qemuopts *qopts) /** * Add a command line flag which has no argument. eg: * - * qemuopts_add_flag (qopts, "-nodefconfig"); + * qemuopts_add_flag (qopts, "-no-user-config"); * * Returns C<0> on success. Returns C<-1> on error, setting C<errno>. */ diff --git a/contrib/p2v/build-p2v-iso.sh b/contrib/p2v/build-p2v-iso.sh index ae25cebc8..2c6bd105b 100755 --- a/contrib/p2v/build-p2v-iso.sh +++ b/contrib/p2v/build-p2v-iso.sh @@ -139,7 +139,7 @@ virt-builder $osversion --arch $arch \ --selinux-relabel # Run the guest. -qemu-system-x86_64 -nodefconfig -nodefaults -nographic \ +qemu-system-x86_64 -no-user-config -nodefaults -nographic \ -no-reboot \ -machine accel=kvm:tcg \ -cpu host \ diff --git a/contrib/p2v/test-p2v-iso.sh b/contrib/p2v/test-p2v-iso.sh index cdc4a5412..ff6453b79 100755 --- a/contrib/p2v/test-p2v-iso.sh +++ b/contrib/p2v/test-p2v-iso.sh @@ -49,7 +49,7 @@ disk=$tmpdir/guest.img virt-builder rhel-6.8 --output $disk # Boot the guest as if running with virt-p2v ISO in the CD drive. -qemu-system-x86_64 -nodefconfig -nodefaults \ +qemu-system-x86_64 -no-user-config -nodefaults \ -no-reboot \ -machine accel=kvm:tcg \ -cpu host \ diff --git a/customize/test-firstboot.sh b/customize/test-firstboot.sh index 68f316dd9..bf26b2da6 100755 --- a/customize/test-firstboot.sh +++ b/customize/test-firstboot.sh @@ -116,7 +116,7 @@ rm -f $bat1 $bat2 # Adding a network device is not strictly necessary, but makes # the Debian 7 guest happier. $qemu \ - -nodefconfig \ + -no-user-config \ -display none \ -machine accel=kvm:tcg \ -m 2048 \ diff --git a/customize/test-password.pl b/customize/test-password.pl index f39d7b44f..b2fca6b2e 100755 --- a/customize/test-password.pl +++ b/customize/test-password.pl @@ -98,7 +98,7 @@ system ("virt-builder", $guestname, "--quiet", # Run qemu and make sure we get to the login prompt. my $exp = Expect->spawn ($qemu, - "-nodefconfig", "-display", "none", + "-no-user-config", "-display", "none", "-machine", "accel=kvm:tcg", "-m", "1024", "-boot", "c", "-drive", "file=$disk,format=raw,if=ide", diff --git a/customize/test-settings.sh b/customize/test-settings.sh index a86c9da9e..84e3a1fae 100755 --- a/customize/test-settings.sh +++ b/customize/test-settings.sh @@ -123,7 +123,7 @@ virt-builder "$guestname" \ # # Use IDE because some ancient guests don't support anything else. $qemu \ - -nodefconfig \ + -no-user-config \ -display none \ -machine accel=kvm:tcg \ -m 2048 \ diff --git a/lib/launch-direct.c b/lib/launch-direct.c index 496b333c1..1e51d2a45 100644 --- a/lib/launch-direct.c +++ b/lib/launch-direct.c @@ -480,8 +480,8 @@ launch_direct (guestfs_h *g, void *datav, const char *arg) */ arg ("-global", VIRTIO_DEVICE_NAME ("virtio-blk") ".scsi=off"); - if (guestfs_int_qemu_supports (g, data->qemu_data, "-nodefconfig")) - flag ("-nodefconfig"); + if (guestfs_int_qemu_supports (g, data->qemu_data, "-no-user-config")) + flag ("-no-user-config"); /* This oddly named option doesn't actually enable FIPS. It just * causes qemu to do the right thing if FIPS is enabled in the diff --git a/p2v/test-virt-p2v-pxe.sh b/p2v/test-virt-p2v-pxe.sh index ebb9ed853..115d1730f 100755 --- a/p2v/test-virt-p2v-pxe.sh +++ b/p2v/test-virt-p2v-pxe.sh @@ -78,7 +78,7 @@ cmdline="root=/dev/sda4 ro console=ttyS0 printk.time=1 p2v.server=10.0.2.2 p2v.p # Run virt-p2v inside qemu. $qemu \ - -nodefconfig \ + -no-user-config \ -display none \ -machine accel=kvm:tcg \ -m 2048 \ diff --git a/utils/boot-analysis/boot-analysis-timeline.c b/utils/boot-analysis/boot-analysis-timeline.c index 3ff031487..4b7da31fa 100644 --- a/utils/boot-analysis/boot-analysis-timeline.c +++ b/utils/boot-analysis/boot-analysis-timeline.c @@ -195,7 +195,7 @@ construct_timeline (void) /* Find where we run qemu. */ FIND_OPTIONAL ("qemu", LONG_ACTIVITY, data->events[j].source == GUESTFS_EVENT_APPLIANCE && - strstr (data->events[j].message, "-nodefconfig"), + strstr (data->events[j].message, "-no-user-config"), data->events[k].source == GUESTFS_EVENT_CLOSE); /* For the libvirt backend, connecting to libvirt, getting @@ -259,7 +259,7 @@ construct_timeline (void) /* From starting qemu up to entering the BIOS is the qemu overhead. */ FIND_OPTIONAL ("qemu:overhead", 0, data->events[j].source == GUESTFS_EVENT_APPLIANCE && - strstr (data->events[j].message, "-nodefconfig"), + strstr (data->events[j].message, "-no-user-config"), data->events[k].source == GUESTFS_EVENT_APPLIANCE && strstr (data->events[k].message, FIRST_FIRMWARE_MESSAGE)); -- 2.19.0.rc0

2 1