October 2018 - Libguestfs - libguestfs List Archives

[PATCH v2] launch: add support for autodetection of appliance image format
by Pavel Butsykin 06 Feb '20

06 Feb '20

This feature allows you to use different image formats for the fixed appliance. The raw format is used by default. Signed-off-by: Pavel Butsykin <pbutsykin(a)virtuozzo.com> --- lib/launch-direct.c | 2 ++ lib/launch-libvirt.c | 19 ++++++++++++------- m4/guestfs_appliance.m4 | 11 +++++++++++ 3 files changed, 25 insertions(+), 7 deletions(-) diff --git a/lib/launch-direct.c b/lib/launch-direct.c index 0be662e25..b9b54857a 100644 --- a/lib/launch-direct.c +++ b/lib/launch-direct.c @@ -592,7 +592,9 @@ launch_direct (guestfs_h *g, void *datav, const char *arg) append_list ("id=appliance"); append_list ("cache=unsafe"); append_list ("if=none"); +#ifndef APPLIANCE_FMT_AUTO append_list ("format=raw"); +#endif } end_list (); start_list ("-device") { append_list ("scsi-hd"); diff --git a/lib/launch-libvirt.c b/lib/launch-libvirt.c index 4adb2cfb3..030ea6911 100644 --- a/lib/launch-libvirt.c +++ b/lib/launch-libvirt.c @@ -212,9 +212,10 @@ get_source_format_or_autodetect (guestfs_h *g, struct drive *drv) /** * Create a qcow2 format overlay, with the given C<backing_drive> - * (file). The C<format> parameter, which must be non-NULL, is the - * backing file format. This is used to create the appliance overlay, - * and also for read-only drives. + * (file). The C<format> parameter is the backing file format. + * The C<format> parameter can be NULL, in this case the backing + * format will be determined automatically. This is used to create + * the appliance overlay, and also for read-only drives. */ static char * make_qcow2_overlay (guestfs_h *g, const char *backing_drive, @@ -223,8 +224,6 @@ make_qcow2_overlay (guestfs_h *g, const char *backing_drive, char *overlay; struct guestfs_disk_create_argv optargs; - assert (format != NULL); - if (guestfs_int_lazy_make_tmpdir (g) == -1) return NULL; @@ -232,8 +231,10 @@ make_qcow2_overlay (guestfs_h *g, const char *backing_drive, optargs.bitmask = GUESTFS_DISK_CREATE_BACKINGFILE_BITMASK; optargs.backingfile = backing_drive; - optargs.bitmask |= GUESTFS_DISK_CREATE_BACKINGFORMAT_BITMASK; - optargs.backingformat = format; + if (format) { + optargs.bitmask |= GUESTFS_DISK_CREATE_BACKINGFORMAT_BITMASK; + optargs.backingformat = format; + } if (guestfs_disk_create_argv (g, overlay, "qcow2", -1, &optargs) == -1) { free (overlay); @@ -461,7 +462,11 @@ launch_libvirt (guestfs_h *g, void *datav, const char *libvirt_uri) /* Note that appliance can be NULL if using the old-style appliance. */ if (appliance) { +#ifdef APPLIANCE_FMT_AUTO + params.appliance_overlay = make_qcow2_overlay (g, appliance, NULL); +#else params.appliance_overlay = make_qcow2_overlay (g, appliance, "raw"); +#endif if (!params.appliance_overlay) goto cleanup; } diff --git a/m4/guestfs_appliance.m4 b/m4/guestfs_appliance.m4 index 81c43879f..4e1ec8135 100644 --- a/m4/guestfs_appliance.m4 +++ b/m4/guestfs_appliance.m4 @@ -139,3 +139,14 @@ AC_SUBST([GUESTFS_DEFAULT_PATH]) AC_DEFINE_UNQUOTED([GUESTFS_DEFAULT_PATH], ["$GUESTFS_DEFAULT_PATH"], [Define guestfs default path.]) + +AC_ARG_ENABLE([appliance-fmt-auto], + [AS_HELP_STRING([--enable-appliance-fmt-auto], + [enable autodetection of appliance image format @<:@default=no@:>@])], + [ENABLE_APPLIANCE_FMT_AUTO="$enableval"], + [ENABLE_APPLIANCE_FMT_AUTO=no]) + +if test "x$ENABLE_APPLIANCE_FMT_AUTO" = "xyes"; then + AC_DEFINE([APPLIANCE_FMT_AUTO], [1], + [Define to 1 if enabled autodetection of appliance image format.]) +fi -- 2.13.0

2 4

1.39 proposal: Let's split up the libguestfs git repo and tarballs
by Richard W.M. Jones 29 Nov '19

29 Nov '19

My contention is that the libguestfs git repository is too large and unwieldy. There are too many separate, unrelated projects and as a result of that the source has too many dependencies and takes too long to build and test. The project divides (sort of) naturally into layers -- the library, the bindings, the various virt tools -- and could be split along those lines into separate projects which can then be released and evolve at their own pace. My suggested split would be something like this: * libguestfs: The library, daemon and appliance. That would include the following directories in a single project: appliance bash contrib daemon docs examples gnulib lib logo test-tool tmp utils website * 1 project for each language binding: csharp erlang gobject golang haskell java lua ocaml php perl python ruby * virt-customize and related tools, we'd probably call this subproject "virt-builder". It would include virt-builder, virt-customize and virt-sysprep, since they share a lot of common code. * 1 project for each of the following items: small tools written in C (virt-cat, virt-filesystems, virt-log, virt-ls, virt-tail, virt-diff, virt-edit, virt-format, guestmount, virt-inspector, virt-make-fs, virt-rescue) guestfish virt-alignment-scan and virt-df virt-dib virt-get-kernel virt-resize virt-sparsify virt-v2v and virt-p2v virt-win-reg * I'd be inclined to drop the legacy Perl tools virt-tar, virt-list-filesystems, virt-list-partitions unless someone especially wished to step forward to maintain them. * common code and generator: Off to the side we'd somehow need to package up the common code and the generator for use by all of the above projects. It wouldn't be a separate project for downstream packagers, but instead the code would be included (ie. duplicated) in tarballs and upstream available as a side git repo that you'd need to include when building (git submodule?). This is somewhat unspecified. M4, PO, and tests would be split between the projects as appropriate. My proposal would be to do this incrementally, rather than all at once, moving the easier things out first. Thoughts? Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org

4 18

[hivex PATCH] Re-allocating unused blocks before assigning new blocks
by Shreyas Khare 03 Jan '19

03 Jan '19

Hello Richard As discussed in the IRC channel, when merging a moderately large reg file (~35MB) to a hiv file (~118 MB); hivex generates a huge hiv file (~580 MB). These changes address that by creating a list of unallocated blocks and reassigning unused blocks. I used https://github.com/msuhanov/regf/blob/master/Windows%20registry%20file%20fo… as a reference for the structure of the hiv file (in addition to the source code itself) Attaching the patch file. Shreyas -- NOTICE OF CONFIDENTIALITY: At Rapid7, the privacy of our customers, partners, and employees is paramount. If you received this email in error, please notify the sender and delete it from your inbox right away. Learn how Rapid7 handles privacy at rapid7.com/privacy-policy <https://www.rapid7.com/privacy-policy/>. To opt-out of Rapid7 marketing emails, please click here <https://information.rapid7.com/manage-subscription.html> or email privacy(a)rapid7.com <mailto:mailto:privacy@rapid7.com>.

2 3

[PATCH v2 0/3] v2v: -o rhv-upload: Add a test.
by Richard W.M. Jones 21 Nov '18

21 Nov '18

v1 was here: https://www.redhat.com/archives/libguestfs/2018-September/msg00121.html v2: - Rewrote patch 2 from scratch so it incorporates Nir's suggestions. - Add fake module to EXTRA_DIST. - Retested. Unfortunately I am no longer able to test the ordinary conversion path because ovirtsdk4 is incompatible with Fedora 29 / Python 3.7: https://bugzilla.redhat.com/show_bug.cgi?id=1622043 so I'm unable to verify that the changes (esp patch 2) don't break this. Rich.

3 15

[PATCH v2 0/4] common/utils: Move libxml2 writer macros to a common header file.
by Richard W.M. Jones 02 Nov '18

02 Nov '18

v1 was here: https://www.redhat.com/archives/libguestfs/2018-October/msg00047.html However it was broken in a few ways. First of all the documentation was broken because "/**" enhanced comments were not permitted on macros. This is fixed in the new 1/4 patch. Secondly we didn't use single_element() everywhere possible, which is fixed in the new 4/4 patch. Lastly I've enhanced the documentation so it's clear what precise XML is generated. Rich.

2 6

[PATCH nbdkit 0/4] Add floppy plugin.
by Richard W.M. Jones 30 Oct '18

30 Oct '18

Add nbdkit-floppy-plugin, “inspired” by qemu's VVFAT driver, but without the ability to handle writes. The implementation is pretty complete, supporting FAT32, LFNs, volume labels, timestamps, etc, and it passes both ‘make check’ and ‘make check-valgrind’. Usage is simple; to serve the current directory: $ nbdkit floppy . Then using guestfish (or any NBD client): $ guestfish --ro --format=raw -a nbd://localhost -m /dev/sda1 Welcome to guestfish, the guest filesystem shell for editing virtual machine filesystems and disk images. Type: ‘help’ for help on commands ‘man’ to read the manual ‘quit’ to quit the shell ><fs> ll / total 2420 drwxr-xr-x 14 root root 16384 Jan 1 1970 . drwxr-xr-x 19 root root 4096 Oct 28 10:07 .. -rwxr-xr-x 1 root root 40 Sep 17 21:23 .dir-locals.el -rwxr-xr-x 1 root root 879 Oct 27 21:10 .gdb_history drwxr-xr-x 8 root root 16384 Oct 28 10:05 .git -rwxr-xr-x 1 root root 1383 Sep 17 21:23 .gitignore -rwxr-xr-x 1 root root 1453 Sep 17 21:23 LICENSE -rwxr-xr-x 1 root root 34182 Oct 28 10:04 Makefile -rwxr-xr-x 1 root root 2568 Oct 27 22:17 Makefile.am -rwxr-xr-x 1 root root 32085 Oct 27 22:18 Makefile.in -rwxr-xr-x 1 root root 620 Sep 17 21:23 OTHER_PLUGINS -rwxr-xr-x 1 root root 4628 Oct 16 22:36 README -rwxr-xr-x 1 root root 4007 Sep 17 21:23 TODO -rwxr-xr-x 1 root root 54733 Oct 27 22:18 aclocal.m4 drwxr-xr-x 2 root root 16384 Oct 27 22:18 autom4te.cache drwxr-xr-x 2 root root 16384 Oct 28 10:04 bash drwxr-xr-x 5 root root 16384 Oct 27 18:07 common [etc] ><fs> ll /plugins/floppy/ total 560 drwxr-xr-x 4 root root 16384 Oct 28 10:04 . drwxr-xr-x 34 root root 16384 Oct 28 10:04 .. drwxr-xr-x 2 root root 16384 Oct 28 10:04 .deps drwxr-xr-x 2 root root 16384 Oct 28 10:04 .libs -rwxr-xr-x 1 root root 34238 Oct 28 10:04 Makefile -rwxr-xr-x 1 root root 2347 Oct 27 22:17 Makefile.am -rwxr-xr-x 1 root root 33158 Oct 27 22:18 Makefile.in -rwxr-xr-x 1 root root 18181 Oct 28 09:36 directory-lfn.c -rwxr-xr-x 1 root root 5003 Oct 27 22:17 floppy.c -rwxr-xr-x 1 root root 6724 Oct 28 10:04 nbdkit-floppy-plugin.1 -rwxr-xr-x 1 root root 996 Oct 28 10:04 nbdkit-floppy-plugin.la -rwxr-xr-x 1 root root 2547 Oct 27 22:17 nbdkit-floppy-plugin.pod -rwxr-xr-x 1 root root 363 Oct 28 10:04 nbdkit_floppy_plugin_la-directory-lfn.lo -rwxr-xr-x 1 root root 90096 Oct 28 10:04 nbdkit_floppy_plugin_la-directory-lfn.o -rwxr-xr-x 1 root root 342 Oct 28 10:04 nbdkit_floppy_plugin_la-floppy.lo -rwxr-xr-x 1 root root 31720 Oct 28 10:04 nbdkit_floppy_plugin_la-floppy.o -rwxr-xr-x 1 root root 366 Oct 28 10:04 nbdkit_floppy_plugin_la-virtual-floppy.lo -rwxr-xr-x 1 root root 57320 Oct 28 10:04 nbdkit_floppy_plugin_la-virtual-floppy.o -rwxr-xr-x 1 root root 24514 Oct 28 09:56 virtual-floppy.c -rwxr-xr-x 1 root root 9184 Oct 28 09:33 virtual-floppy.h Rich.

2 9

Re: [Libguestfs] Multiple convertion from hyper-v VM to RHEV 4.1
by Richard W.M. Jones 30 Oct '18

30 Oct '18

On Tue, Oct 30, 2018 at 10:27:49AM -0300, HUGO DANIEL MEDINA SOSA wrote: > Hi Richard, how are you!! ... I find you mail address in a mailing list > when i search for a tool to convert multiple VMs allocated in Hyper-V > plattform to RHEV 4.1 plattform. > The issue presented for me is that i have almost fifty (~50) linux VMs [Red > Hat 6.x-7.x , CentOS 6.x-7.x] in Hyper-V 2012 R2 and i wan to migrate > thoose to RHEV. > I already been migrate a few VMs with virt2virt , but the work is hard , > because my servers are in production enviroment , and is tricky to open a > windows for maintenance every week in the late night. > Is there a tool ,application , or terciary software who can do what i need?. Work is being done to add virt-v2v support to CloudForms 5.x. https://bugzilla.redhat.com/show_bug.cgi?id=1601090 Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top

2 1

guestfs launch failed in CentOS 7.5
by Sarbajit Chatterjee 30 Oct '18

30 Oct '18

Hello, I am using https://github.com/clalancette/oz/tree/v0.16.0 to customise qcow2 images. In CentOS 7.5.1804 (yum updated to latest) the image customisation fails during guestfs launch. Following is the output of libguestfs-test-tool run. Please help in debugging the problem - ************************************************************ * IMPORTANT NOTICE * * When reporting bugs, include the COMPLETE, UNEDITED * output below in your bug report. * ************************************************************ LIBGUESTFS_BACKEND=direct PATH=/home/ciuser/gradle-2.4/bin:/home/ciuser/JAVA/jdk1.8.0_77/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/etc/oz:/home/ciuser/.local/bin:/home/ciuser/bin XDG_RUNTIME_DIR=/run/user/1000 SELinux: Enforcing guestfs_get_append: (null) guestfs_get_autosync: 1 guestfs_get_backend: direct guestfs_get_backend_settings: [] guestfs_get_cachedir: /var/tmp guestfs_get_hv: /usr/libexec/qemu-kvm guestfs_get_memsize: 500 guestfs_get_network: 0 guestfs_get_path: /usr/lib64/guestfs guestfs_get_pgroup: 0 guestfs_get_program: libguestfs-test-tool guestfs_get_recovery_proc: 1 guestfs_get_smp: 1 guestfs_get_sockdir: /run/user/1000 guestfs_get_tmpdir: /tmp guestfs_get_trace: 0 guestfs_get_verbose: 1 host_cpu: x86_64 Launching appliance, timeout set to 600 seconds. libguestfs: launch: program=libguestfs-test-tool libguestfs: launch: version=1.36.10rhel=7,release=6.el7_5.2,libvirt libguestfs: launch: backend registered: unix libguestfs: launch: backend registered: uml libguestfs: launch: backend registered: libvirt libguestfs: launch: backend registered: direct libguestfs: launch: backend=direct libguestfs: launch: tmpdir=/tmp/libguestfsIHeKLD libguestfs: launch: umask=0002 libguestfs: launch: euid=1000 libguestfs: begin building supermin appliance libguestfs: run supermin libguestfs: command: run: /usr/bin/supermin5 libguestfs: command: run: \ --build libguestfs: command: run: \ --verbose libguestfs: command: run: \ --if-newer libguestfs: command: run: \ --lock /var/tmp/.guestfs-1000/lock libguestfs: command: run: \ --copy-kernel libguestfs: command: run: \ -f ext2 libguestfs: command: run: \ --host-cpu x86_64 libguestfs: command: run: \ /usr/lib64/guestfs/supermin.d libguestfs: command: run: \ -o /var/tmp/.guestfs-1000/appliance.d supermin: version: 5.1.19 supermin: rpm: detected RPM version 4.11 supermin: package handler: fedora/rpm supermin: acquiring lock on /var/tmp/.guestfs-1000/lock supermin: if-newer: output does not need rebuilding libguestfs: finished building supermin appliance libguestfs: begin testing qemu features libguestfs: checking for previously cached test results of /usr/libexec/qemu-kvm, in /var/tmp/.guestfs-1000 libguestfs: loading previously cached test results libguestfs: QMP parse error: parse error: premature EOF\n \n (right here) ------^\n (ignored) libguestfs: qemu version: 1.5 libguestfs: qemu mandatory locking: no libguestfs: finished testing qemu features [00034ms] /usr/libexec/qemu-kvm \ -global virtio-blk-pci.scsi=off \ -nodefconfig \ -enable-fips \ -nodefaults \ -display none \ -machine accel=kvm:tcg \ -cpu host \ -m 500 \ -no-reboot \ -rtc driftfix=slew \ -no-hpet \ -global kvm-pit.lost_tick_policy=discard \ -kernel /var/tmp/.guestfs-1000/appliance.d/kernel \ -initrd /var/tmp/.guestfs-1000/appliance.d/initrd \ -object rng-random,filename=/dev/urandom,id=rng0 \ -device virtio-rng-pci,rng=rng0 \ -device virtio-scsi-pci,id=scsi \ -drive file=/tmp/libguestfsIHeKLD/scratch1.img,cache=unsafe,format=raw,id=hd0,if=none \ -device scsi-hd,drive=hd0 \ -drive file=/var/tmp/.guestfs-1000/appliance.d/root,snapshot=on,id=appliance,cache=unsafe,if=none,format=raw \ -device scsi-hd,drive=appliance \ -device virtio-serial-pci \ -serial stdio \ -device sga \ -chardev socket,path=/run/user/1000/libguestfsoBqGwV/guestfsd.sock,id=channel0 \ -device virtserialport,chardev=channel0,name=org.libguestfs.channel.0 \ -append 'panic=1 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=xterm' \x1b[1;256r\x1b[256;256H\x1b[6n Google, Inc. Serial Graphics Adapter 06/09/14 SGABIOS $Id: sgabios.S 8 2010-04-22 00:03:40Z nlaredo $ (mockbuild@) Mon Jun 9 21:33:48 UTC 2014 Term: 80x24 4 0 SeaBIOS (version 1.11.0-2.el7) Booting from ROM... \x1b[2J[ 0.000000] Initializing cgroup subsys cpuset [ 0.000000] Initializing cgroup subsys cpu [ 0.000000] Initializing cgroup subsys cpuacct [ 0.000000] Linux version 3.10.0-862.14.4.el7.x86_64 ( mockbuild(a)kbuilder.bsys.centos.org) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-28) (GCC) ) #1 SMP Wed Sep 26 15:12:11 UTC 2018 [ 0.000000] Command line: panic=1 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=xterm [ 0.000000] e820: BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009f7ff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009f800-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000001f3fbfff] usable [ 0.000000] BIOS-e820: [mem 0x000000001f3fc000-0x000000001f3fffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.4 present. [ 0.000000] Hypervisor detected: KVM [ 0.000000] e820: last_pfn = 0x1f3fc max_arch_pfn = 0x400000000 [ 0.000000] PAT configuration [0-7]: WB WC UC- UC WB WP UC- UC [ 0.000000] found SMP MP-table at [mem 0x000f63b0-0x000f63bf] mapped at [ffffffffff2003b0] [ 0.000000] RAMDISK: [mem 0x1f0c0000-0x1f3effff] [ 0.000000] Early table checksum verification disabled [ 0.000000] ACPI: RSDP 00000000000f6210 00014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 000000001f3ffad7 00030 (v01 BOCHS BXPCRSDT 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 000000001f3ff177 00074 (v01 BOCHS BXPCFACP 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 000000001f3fe040 01137 (v01 BOCHS BXPCDSDT 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 000000001f3fe000 00040 [ 0.000000] ACPI: SSDT 000000001f3ff1eb 00874 (v01 BOCHS BXPCSSDT 00000001 BXPC 00000001) [ 0.000000] ACPI: APIC 000000001f3ffa5f 00078 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001) [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x000000001f3fbfff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1f099000-0x1f0bffff] [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: cpu 0, msr 0:1f049001, primary cpu clock [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x00001000-0x00ffffff] [ 0.000000] DMA32 [mem 0x01000000-0xffffffff] [ 0.000000] Normal empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x00001000-0x0009efff] [ 0.000000] node 0: [mem 0x00100000-0x1f3fbfff] [ 0.000000] Initmem setup node 0 [mem 0x00001000-0x1f3fbfff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] smpboot: Allowing 1 CPUs, 0 hotplug CPUs [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] e820: [mem 0x1f400000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] setup_percpu: NR_CPUS:5120 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1 [ 0.000000] PERCPU: Embedded 35 pages/cpu @ffff93c95ee00000 s104920 r8192 d30248 u2097152 [ 0.000000] KVM setup async PF for cpu 0 [ 0.000000] kvm-stealtime: cpu 0, msr 1ee13500 [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes) [ 0.000000] Built 1 zonelists in Node order, mobility grouping on. Total pages: 125877 [ 0.000000] Policy zone: DMA32 [ 0.000000] Kernel command line: panic=1 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=xterm [ 0.000000] Disabling memory control group subsystem [ 0.000000] PID hash table entries: 2048 (order: 2, 16384 bytes) [ 0.000000] x86/fpu: xstate_offset[2]: 0240, xstate_sizes[2]: 0100 [ 0.000000] xsave: enabled xstate_bv 0x7, cntxt size 0x340 using standard form [ 0.000000] Memory: 480520k/511984k available (7348k kernel code, 392k absent, 31072k reserved, 6281k data, 1844k init) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 [ 0.000000] x86/pti: Unmapping kernel while in userspace [ 0.000000] Hierarchical RCU implementation. [ 0.000000] \tRCU restricting CPUs from NR_CPUS=5120 to nr_cpu_ids=1. [ 0.000000] NR_IRQS:327936 nr_irqs:256 0 [ 0.000000] Console: colour *CGA 80x25 [ 0.000000] console [ttyS0] enabled [ 0.000000] tsc: Detected 2494.218 MHz processor [ 0.098000] Calibrating delay loop (skipped) preset value.. 4988.43 BogoMIPS (lpj=2494218) [ 0.104230] pid_max: default: 32768 minimum: 301 [ 0.110254] Security Framework initialized [ 0.116237] SELinux: Disabled at boot. [ 0.117234] Yama: becoming mindful. [ 0.131312] Dentry cache hash table entries: 65536 (order: 7, 524288 bytes) [ 0.141311] Inode-cache hash table entries: 32768 (order: 6, 262144 bytes) [ 0.153253] Mount-cache hash table entries: 1024 (order: 1, 8192 bytes) [ 0.157238] Mountpoint-cache hash table entries: 1024 (order: 1, 8192 bytes) [ 0.162358] Initializing cgroup subsys memory [ 0.168236] Initializing cgroup subsys devices [ 0.192231] Initializing cgroup subsys freezer [ 0.211226] Initializing cgroup subsys net_cls [ 0.229237] Initializing cgroup subsys blkio [ 0.240227] Initializing cgroup subsys perf_event [ 0.242232] Initializing cgroup subsys hugetlb [ 0.256002] random: fast init done [ 0.281241] Initializing cgroup subsys pids [ 0.297210] Initializing cgroup subsys net_prio [ 0.857224] mce: CPU supports 10 MCE banks [ 1.161221] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 1.175212] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 1.189209] tlb_flushall_shift: 6 [ 1.198225] Speculative Store Bypass: Vulnerable [ 1.206223] FEATURE SPEC_CTRL Not Present [ 1.218208] FEATURE IBPB_SUPPORT Not Present [ 4.853233] Spectre V2 : Vulnerable: Retpoline without IBPB [ 13.376012] random: crng init done [ 101.375237] Freeing SMP alternatives: 24k freed [ 110.720007] BUG: unable to handle kernel NULL pointer dereference at 0000000000000102 [ 110.721000] IP: [<ffffffffad8b3ef2>] __queue_work+0x32/0x3e0 [ 110.721000] PGD 0 [ 110.721000] Oops: 0000 [#1] SMP [ 110.721000] Modules linked in: [ 110.721000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.10.0-862.14.4.el7.x86_64 #1 [ 110.721000] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 [ 110.721000] task: ffffffffae416480 ti: ffffffffae400000 task.ti: ffffffffae400000 [ 110.721000] RIP: 0010:[<ffffffffad8b3ef2>] [<ffffffffad8b3ef2>] __queue_work+0x32/0x3e0 [ 110.721000] RSP: 0000:ffff93c95ee03e20 EFLAGS: 00010046 [ 110.721000] RAX: 0000000000000082 RBX: 0000000000000087 RCX: 0000000000000000 [ 110.721000] RDX: ffffffffae4e0340 RSI: 0000000000000000 RDI: 0000000000001400 [ 110.721000] RBP: ffff93c95ee03e58 R08: 0000000000000000 R09: 0000000000004000 [ 110.721000] R10: ffffffffaea17998 R11: 0000000000007ffe R12: ffffffffae4e0340 [ 110.721000] R13: 0000000000001400 R14: 0000000000000000 R15: ffffffffae2b3a62 [ 110.721000] FS: 0000000000000000(0000) GS:ffff93c95ee00000(0000) knlGS:0000000000000000 [ 110.721000] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 110.721000] CR2: 0000000000000102 CR3: 000000001200e000 CR4: 00000000001606f0 [ 110.721000] Call Trace: [ 110.721000] <IRQ> [ 110.721000] [<ffffffffad8b44f5>] queue_work_on+0x45/0x50 [ 110.721000] [<ffffffffadc564d6>] credit_entropy_bits+0x1c6/0x290 [ 110.721000] [<ffffffffadc571c4>] ? add_interrupt_randomness+0x1c4/0x230 [ 110.721000] [<ffffffffadc571c4>] add_interrupt_randomness+0x1c4/0x230 [ 110.721000] [<ffffffffad943f7f>] handle_irq_event_percpu+0x3f/0x80 [ 110.721000] [<ffffffffad943ffc>] handle_irq_event+0x3c/0x60 [ 110.721000] [<ffffffffad9470a3>] handle_level_irq+0x73/0xd0 [ 110.721000] [<ffffffffad82e504>] handle_irq+0xe4/0x1a0 [ 110.721000] [<ffffffffad89cf08>] ? __local_bh_enable+0x28/0x90 [ 110.721000] [<ffffffffadf29f9d>] do_IRQ+0x4d/0xf0 [ 110.721000] [<ffffffffadf28c83>] ? simd_coprocessor_error+0x3/0x30 [ 110.721000] [<ffffffffadf1c362>] common_interrupt+0x162/0x162 [ 110.721000] <EOI> [ 110.721000] [<ffffffffad877b80>] ? __native_set_fixmap+0x40/0x40 [ 110.721000] [<ffffffffad86b14c>] ? end_pv_cpu_ops_usergs_sysret32+0x3/0x3 [ 110.721000] [<ffffffffadf28c83>] ? simd_coprocessor_error+0x3/0x30 [ 110.721000] [<ffffffffad86a4b6>] ? native_restore_fl+0x6/0x10 [ 110.721000] [<ffffffffad833126>] text_poke_early+0x36/0x40 [ 110.721000] [<ffffffffad833383>] apply_paravirt+0xb3/0xe0 [ 110.721000] [<ffffffffad8973f4>] ? vprintk_emit+0x3c4/0x510 [ 110.721000] [<ffffffffad9a00e6>] ? free_hot_cold_page+0x106/0x160 [ 110.721000] [<ffffffffad99bc6d>] ? adjust_managed_page_count+0x5d/0x80 [ 110.721000] [<ffffffffad86fc6a>] ? free_init_pages+0x15a/0x1b0 [ 110.721000] [<ffffffffae579537>] alternative_instructions+0xc3/0xca [ 110.721000] [<ffffffffae57b0cb>] check_bugs+0x25f/0x2b6 [ 110.721000] [<ffffffffae56c19d>] start_kernel+0x422/0x46c [ 110.721000] [<ffffffffae56bb7b>] ? repair_env_string+0x5c/0x5c [ 110.721000] [<ffffffffae56b120>] ? early_idt_handler_array+0x120/0x120 [ 110.721000] [<ffffffffae56b72f>] x86_64_start_reservations+0x24/0x26 [ 110.721000] [<ffffffffae56b885>] x86_64_start_kernel+0x154/0x177 [ 110.721000] [<ffffffffad8000d5>] start_cpu+0x5/0x14 [ 110.721000] Code: 89 e5 41 57 41 56 49 89 f6 41 55 41 89 fd 41 54 49 89 d4 53 48 83 ec 10 89 7d d4 ff 14 25 00 07 43 ae f6 c4 02 0f 85 de 02 00 00 <41> f6 86 02 01 00 00 01 0f 85 78 02 00 00 49 c7 c7 08 5a 01 00 [ 110.721000] RIP [<ffffffffad8b3ef2>] __queue_work+0x32/0x3e0 [ 110.721000] RSP <ffff93c95ee03e20> [ 110.721000] CR2: 0000000000000102 [ 110.721000] ---[ end trace aaa07a7d70f6c07b ]--- [ 110.721000] Kernel panic - not syncing: Fatal exception in interrupt [ 110.721000] Rebooting in 1 seconds..libguestfs: error: appliance closed the connection unexpectedly, see earlier error messages libguestfs: child_cleanup: 0x55690ed44c60: child process died libguestfs: sending SIGTERM to process 31847 libguestfs: qemu maxrss 96256K libguestfs: error: guestfs_launch failed, see earlier error messages libguestfs: closing guestfs handle 0x55690ed44c60 (state 0) libguestfs: command: run: rm libguestfs: command: run: \ -rf /tmp/libguestfsIHeKLD libguestfs: command: run: rm libguestfs: command: run: \ -rf /run/user/1000/libguestfsoBqGwV This is running on a VM with nested virtualization enabled. rpm -qa | grep kernel kernel-tools-3.10.0-862.14.4.el7.x86_64 kernel-tools-libs-3.10.0-862.14.4.el7.x86_64 kernel-3.10.0-862.14.4.el7.x86_64 kernel-3.10.0-862.el7.x86_64 kernel-3.10.0-862.9.1.el7.x86_64 rpm -qa | grep supermin supermin5-5.1.19-1.el7.x86_64 Possibly similar issue reported in CentOS https://bugzilla.redhat.com/show_bug.cgi?id=1535973 ? Thanks, Sarbajit

2 3

How does guestmount actually work?
by Sergey Ivanov 29 Oct '18

29 Oct '18

Is there any articles that explain in detail how actually guestmount works internally? My main question is how does guestfs deal with backing disk changes. So, I have a libvirt/qemu-kvm setup. Each vm has a lvm2 volume for a disk. For example, I want to mount one of these volumes and read some data. 1. I mount the volume using guestmount (guestmount --ro -d <libvirt domain> -m /dev/sda1 /mnt) 2. Files become available in /mnt 3. If I make changes to the filesystem from inside the VM, no changes are visible in /mnt In guestfs-faq, there is a diagram that shows how guestfs interacts with the backing disk, and the description says that you can't see what's happening in the underlying filesystem because of vm/appliance caches, and it should work this way. Now, I fill my host's RAM with garbage until I end up with like ~50mb available. Then I 1. Mount the LV again 2. Go into guest OS and do some I/O operations like extracting some big .tar.gz files, and then do fdatasync and drop all the caches 3. Read all the data from /mnt, where the VM's disk is mounted 4. See no changes again Swap is not used. Overcommit is not enabled either. All the memory allocated is actually in use. There is virtually no RAM available at this moment, but the oom killer still doesn't invoke. Buffers/cache column in `free` output shows that approximately 2gb is used, which, I assume, are used not exclusively by guestfs. The LV is 20gb in size, 16 of which are used. How does guestfs manage to preserve filesystem's state after mount? Is there any chance that the changes made to the backing file will somehow become visible in /mnt? Why is that supermin appliance needed to mount a disk? How is it used and how does it interact with every other part? I hope someone explains this to me, or at least that there is already an article on that matter. Thanks.

2 1

[PATCH v2 0/2] v2v: machine type for oVirt
by Tomáš Golembiovský 18 Oct '18

18 Oct '18

changes in v2: - split patch in two - changed as per suggestions Tomáš Golembiovský (2): v2v: ovf: add firmware and machine type element v2v: enable UEFI for oVirt/RHV outputs v2v/create_ovf.ml | 12 +++++++++++- v2v/create_ovf.mli | 2 +- v2v/output_rhv.ml | 6 ++---- v2v/output_rhv_upload.ml | 4 ++-- v2v/output_vdsm.ml | 6 ++---- v2v/test-v2v-o-rhv.ovf.expected | 1 + v2v/test-v2v-o-vdsm-options.ovf.expected | 1 + 7 files changed, 20 insertions(+), 12 deletions(-) -- 2.19.0

3 8