May 2016 - Libguestfs - libguestfs List Archives

[PATCH v2 0/2] lib: qemu: Memoize qemu feature detection.
by Richard W.M. Jones 18 May '16

18 May '16

v1 -> v2: - Rebase on top of Pino's version work. Two patches went upstream, these are the two remaining patches. Note the generation number is still inside the qemu.stat file. We could put it in the filename, I have no particular preference. Rich.

1 2

[PATCH 0/4] lib: qemu: Memoize qemu feature detection.
by Richard W.M. Jones 18 May '16

18 May '16

Doing qemu feature detection in the direct backend takes ~100ms because we need to run `qemu -help' and `qemu -devices ?', and each of those interacts with glibc's very slow link loader. Fixing the link loader is really hard. Instead memoize the output of those two commands. This patch series first separates all the code dealing with qemu into a separate module (src/qemu.c) and formalizes the interface to this module. Then I had to rearrange the code we use to create /var/tmp/.guestfs-<UID> for the supermin appliance to src/tmpdirs.c. The final patch adds the memoization, storing this in the supermin appliance cache directory. Rich.

2 7

[PATCH v2 0/2] src: introduce an helper version struct
by Pino Toscano 18 May '16

18 May '16

Hi, this adds an helper version struct, and uses it in the backends (for the libvirt and qemu versions) and inspection code. This also moves common code to that, so it is not repeated in many places. This should help with the small refactoring proposed with https://www.redhat.com/archives/libguestfs/2016-May/msg00070.html Thanks, Pino Toscano (2): src: start unifying version handling inspect: switch to version struct for os major/minor version src/Makefile.am | 1 + src/guestfs-internal.h | 22 ++++- src/inspect-fs-cd.c | 32 +++---- src/inspect-fs-unix.c | 220 +++++++++++++++++------------------------------ src/inspect-fs-windows.c | 20 ++--- src/inspect-fs.c | 33 ++----- src/inspect-icon.c | 8 +- src/inspect.c | 9 +- src/launch-direct.c | 28 +++--- src/launch-libvirt.c | 43 +++++---- src/version.c | 145 +++++++++++++++++++++++++++++++ 11 files changed, 313 insertions(+), 248 deletions(-) create mode 100644 src/version.c -- 2.5.5

2 3

[PATCH 0/2] src: introduce an helper version struct
by Pino Toscano 18 May '16

18 May '16

Hi, this adds an helper version struct, and uses it in the backends (for the libvirt and qemu versions) and inspection code. This also moves common code to that, so it is not repeated in many places. This should help with the small refactoring proposed with https://www.redhat.com/archives/libguestfs/2016-May/msg00070.html Thanks, Pino Toscano (2): src: start unifying version handling inspect: switch to version struct for os major/minor version src/Makefile.am | 1 + src/guestfs-internal.h | 22 ++++- src/inspect-fs-cd.c | 32 +++---- src/inspect-fs-unix.c | 220 +++++++++++++++++------------------------------ src/inspect-fs-windows.c | 20 ++--- src/inspect-fs.c | 33 ++----- src/inspect-icon.c | 8 +- src/inspect.c | 9 +- src/launch-direct.c | 28 +++--- src/launch-libvirt.c | 43 +++++---- src/version.c | 145 +++++++++++++++++++++++++++++++ 11 files changed, 313 insertions(+), 248 deletions(-) create mode 100644 src/version.c -- 2.5.5

2 5

[PATCH 0/2] Use -bios bios-fast.bin where supported.
by Richard W.M. Jones 17 May '16

17 May '16

NOTE: Not for upstream, yet. This depends on 3 non-upstream patches. - The qemu rework in libguestfs, which in turn depends on what Pino is up to. - Support for '-L ?' in qemu: https://lists.gnu.org/archive/html/qemu-devel/2016-05/threads.html#02596 - Support for bios-fast.bin in qemu: https://lists.gnu.org/archive/html/qemu-devel/2016-05/threads.html#02408 This commit uses -bios bios-fast.bin if available, which basically stops SeaBIOS from trying to do PCI probing during boot, which is a waste of time when using the -kernel option. Rich.

1 2

[PATCH v2] launch: direct: Add DAX root filesystem support.
by Richard W.M. Jones 17 May '16

17 May '16

NOTE: not for upstream, yet. v1 -> v2: - Remove the dependency on enabling ACPI, since ACPI is now enabled all the time. Rich.

1 1

[PATCH supermin] Add support for a DAX root filesystem.
by Richard W.M. Jones 17 May '16

17 May '16

DAX is explained in detail here: https://lwn.net/Articles/610174/ This patch adds support to supermin for using a DAX root filesystem. The corresponding libguestfs patches will be posted shortly once I've tested them a bit more. This requires qemu >= 2.6. Unfortunately it's not really a win for a few reasons: - Requires enabling ACPI, which slows everything down by hundreds of milliseconds. I need to work out exactly what's going on here. Since we never required ACPI before now, I never had the impetus to find out if this is easy to fix. - The extra modules in the supermin mini-initrd greatly increase its size, which also slows things down. qemu fw_cfg DMA should help here, if we ever manage to get it upstream. - vNVDIMM and DAX are not very widely tested or used yet. - Not supported at all by libvirt. Rich.

1 3

Can I specify qcow2 preallocation method with virt-builder?
by Yaniv Kaul 17 May '16

17 May '16

When creating qcow2 images using virt-builder, is there a way to specify the qcow2 preallocation, as possible with qemu-img create -o preallocation=metada , for example? TIA, Y.

2 6

[PATCH] launch: direct: Add DAX root filesystem support.
by Richard W.M. Jones 16 May '16

16 May '16

Allow the appliance / root filesystem to be placed on a virtual NVDIMM and accessed directly by the guest kernel (DAX). This requires corresponding changes in supermin. --- src/guestfs-internal.h | 1 + src/launch-direct.c | 68 ++++++++++++++++++++++++++++++++++++++++---------- src/launch.c | 8 +++++- 3 files changed, 63 insertions(+), 14 deletions(-) diff --git a/src/guestfs-internal.h b/src/guestfs-internal.h index d325f50..3655219 100644 --- a/src/guestfs-internal.h +++ b/src/guestfs-internal.h @@ -782,6 +782,7 @@ extern int64_t guestfs_int_timeval_diff (const struct timeval *x, const struct t extern void guestfs_int_launch_send_progress (guestfs_h *g, int perdozen); extern char *guestfs_int_appliance_command_line (guestfs_h *g, const char *appliance_dev, int flags); #define APPLIANCE_COMMAND_LINE_IS_TCG 1 +#define APPLIANCE_COMMAND_LINE_USE_ACPI 2 const char *guestfs_int_get_cpu_model (int kvm); int guestfs_int_create_socketname (guestfs_h *g, const char *filename, char (*sockname)[UNIX_PATH_MAX]); extern void guestfs_int_register_backend (const char *name, const struct backend_ops *); diff --git a/src/launch-direct.c b/src/launch-direct.c index 01b7e07..a005bda 100644 --- a/src/launch-direct.c +++ b/src/launch-direct.c @@ -234,6 +234,7 @@ launch_direct (guestfs_h *g, void *datav, const char *arg) struct hv_param *hp; bool has_kvm; int force_tcg; + bool dax; const char *cpu_model; /* At present you must add drives before starting the appliance. In @@ -371,15 +372,29 @@ launch_direct (guestfs_h *g, void *datav, const char *arg) warning (g, "qemu debugging is enabled, connect gdb to tcp::1234 to begin"); } + /* Can we use DAX? */ +#ifdef __x86_64__ + dax = guestfs_int_qemu_version_ge (data->qemu_data, 2, 6) && + guestfs_int_qemu_supports_device (g, data->qemu_data, "nvdimm"); +#else + dax = false; +#endif + ADD_CMDLINE ("-machine"); ADD_CMDLINE_PRINTF ( #ifdef MACHINE_TYPE MACHINE_TYPE "," + "%s" +#elif __x86_64__ + "pc,%s" +#else + "%s" #endif #ifdef __aarch64__ "gic-version=host," #endif "accel=%s", + dax ? "nvdimm," : "", !force_tcg ? "kvm:tcg" : "tcg"); cpu_model = guestfs_int_get_cpu_model (has_kvm && !force_tcg); @@ -394,7 +409,10 @@ launch_direct (guestfs_h *g, void *datav, const char *arg) } ADD_CMDLINE ("-m"); - ADD_CMDLINE_PRINTF ("%d", g->memsize); + if (dax) + ADD_CMDLINE_PRINTF ("%d,maxmem=32G,slots=32", g->memsize); + else + ADD_CMDLINE_PRINTF ("%d", g->memsize); /* Force exit instead of reboot on panic */ ADD_CMDLINE ("-no-reboot"); @@ -541,21 +559,43 @@ launch_direct (guestfs_h *g, void *datav, const char *arg) /* Add the ext2 appliance drive (after all the drives). */ if (has_appliance_drive) { - ADD_CMDLINE ("-drive"); - ADD_CMDLINE_PRINTF ("file=%s,snapshot=on,id=appliance," - "cache=unsafe,if=none,format=raw", - appliance); + if (dax) { + struct stat statbuf; - if (virtio_scsi) { - ADD_CMDLINE ("-device"); - ADD_CMDLINE ("scsi-hd,drive=appliance"); - } - else { + if (stat (appliance, &statbuf) == -1) { + perrorf (g, "stat: %s", appliance); + goto cleanup0; + } + + ADD_CMDLINE ("-object"); + /* share=off corresponds to mmap MAP_PRIVATE inside qemu, so + * this should not affect the underlying file. IOW parallel + * access should be fine. + */ + ADD_CMDLINE_PRINTF ("memory-backend-file,id=mem1,share=off," + "mem-path=%s,size=%" PRIu64 "b", + appliance, (uint64_t) statbuf.st_size); ADD_CMDLINE ("-device"); - ADD_CMDLINE (VIRTIO_BLK ",drive=appliance"); - } + ADD_CMDLINE ("nvdimm,memdev=mem1,id=nv1"); - appliance_dev = make_appliance_dev (g, virtio_scsi); + appliance_dev = safe_strdup (g, "/dev/pmem0"); + } else { + ADD_CMDLINE ("-drive"); + ADD_CMDLINE_PRINTF ("file=%s,snapshot=on,id=appliance," + "cache=unsafe,if=none,format=raw", + appliance); + + if (virtio_scsi) { + ADD_CMDLINE ("-device"); + ADD_CMDLINE ("scsi-hd,drive=appliance"); + } + else { + ADD_CMDLINE ("-device"); + ADD_CMDLINE (VIRTIO_BLK ",drive=appliance"); + } + + appliance_dev = make_appliance_dev (g, virtio_scsi); + } } /* Create the virtio serial bus. */ @@ -597,6 +637,8 @@ launch_direct (guestfs_h *g, void *datav, const char *arg) flags = 0; if (!has_kvm || force_tcg) flags |= APPLIANCE_COMMAND_LINE_IS_TCG; + if (dax) + flags |= APPLIANCE_COMMAND_LINE_USE_ACPI; ADD_CMDLINE_STRING_NODUP (guestfs_int_appliance_command_line (g, appliance_dev, flags)); diff --git a/src/launch.c b/src/launch.c index 72a8b29..49f0455 100644 --- a/src/launch.c +++ b/src/launch.c @@ -318,6 +318,10 @@ guestfs_impl_config (guestfs_h *g, * If we are launching a qemu TCG guest (ie. KVM is known to be * disabled or unavailable). If you don't know, don't pass this flag. * + * =item C<APPLIANCE_COMMAND_LINE_USE_ACPI> + * + * Use ACPI in the appliance. Normally disabled because it is slow. + * * =back * * Note that this function returns a newly allocated buffer which must @@ -331,6 +335,7 @@ guestfs_int_appliance_command_line (guestfs_h *g, const char *appliance_dev, char *term = getenv ("TERM"); char *ret; bool tcg = flags & APPLIANCE_COMMAND_LINE_IS_TCG; + bool use_acpi = flags & APPLIANCE_COMMAND_LINE_USE_ACPI; char lpj_s[64] = ""; if (appliance_dev) @@ -367,7 +372,7 @@ guestfs_int_appliance_command_line (guestfs_h *g, const char *appliance_dev, " udev.event-timeout=6000" /* for newer udevd */ " no_timer_check" /* fix for RHBZ#502058 */ "%s" /* lpj */ - " acpi=off" /* ACPI is slow - 150-200ms extra on my laptop */ + "%s" /* acpi=off: ACPI is slow, 150-200ms on my laptop */ " printk.time=1" /* display timestamp before kernel messages */ " cgroup_disable=memory" /* saves us about 5 MB of RAM */ " usbcore.nousb" /* disable USB, only saves about 1ms */ @@ -386,6 +391,7 @@ guestfs_int_appliance_command_line (guestfs_h *g, const char *appliance_dev, g->memsize, #endif lpj_s, + !use_acpi ? " acpi=off" : "", root, g->selinux ? "selinux=1 enforcing=0" : "selinux=0", g->verbose ? "guestfs_verbose=1" : "quiet", -- 2.7.4

1 0

[PATCH] btrfs_filesystem_show: work with btrfs < 4.3.1
by Pino Toscano 16 May '16

16 May '16

Versions older than 4.3.1 output also the version string after the output. Ignore such line, since it isn't relevant to what btrfs_filesystem_show needs. --- daemon/btrfs.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/daemon/btrfs.c b/daemon/btrfs.c index 62bdac7..9b52aa8 100644 --- a/daemon/btrfs.c +++ b/daemon/btrfs.c @@ -2270,6 +2270,12 @@ do_btrfs_filesystem_show (const char *device) } else if (STRPREFIX (lines[i], "\t*** Some devices missing")) { reply_with_error_errno (ENODEV, "%s: missing devices", device); return NULL; + } else if (STRPREFIX (lines[i], "btrfs-progs v")) { + /* Older versions of btrfs-progs output also the version string + * (the same as `btrfs --version`. This has been fixed upstream + * since v4.3.1, commit e29ec82e4e66042ca55bf8cd9ef609e3b21a7eb7. + * To support these older versions, ignore the version line. */ + continue; } else { reply_with_error ("unrecognized line in output from 'btrfs filesystem show': %s", lines[i]); return NULL; -- 2.5.5

2 1