October 2014 - Libguestfs - libguestfs List Archives

make install of ocaml libs fails
by Olaf Hering 21 Oct '14

21 Oct '14

Today I did it, for the very first time: Did not build a package. Just ran "make install". Twice! ... ocamlfind install \ -ldconf ignore -destdir /usr/lib64/ocaml \ guestfs \ META *.so *.a *.cma *.cmi ./*.mli *.cmx *.cmxa ocamlfind: Package guestfs is already installed - (file /usr/lib64/ocaml/guestfs/META already exists) Makefile:2053: recipe for target 'install-data-hook' failed make[3]: *** [install-data-hook] Error 2 I wonder if that is failing just for my version of ocamlfind or if this is a common failure. How do you guys handle that situation? Olaf

2 2

[PATCH V5 0/4] virt-resize: add support for resizing logical
by Hu Tao 21 Oct '14

21 Oct '14

Hi Rich, This is v5 series to add support for resizing MBR logical partitions. please review. Thanks! changes to v4: 1. add support to resize extended partition (--resize or --expand extended partition) 2. fix the problem of deficit of 512 bytes when expanding a logical partition (this problem can be reproduced in v4 by only expanding a logical partition, without resizing any other partitions) 3. update the test script to support logical partitions and extended partition changes to v3: 1. merge patch 1 and patch 3 in v3 2. let mbr_part_type return 'primary' for GPT partitions 3. add test for resizing logical partitions 4. fix extending the extended partition (yet). see patch 7. changes to v2: 1. remove p_part_num 2. remove filter_parts 3. name the function calculate_target_partitions 4. remove the code to restart guest introduced in v2 changes to v1: 1. spit the patches so it's easier to review 2. fix the parted error caused by unaligned logical partitions 3. extend the content of logical partitions 4. refactor to make logical partitions a seperate list Hu Tao (3): resize: add partition type LogicalPartition resize: add support to resize logical partitions resize: support resize extended partition root (1): resize: test: add support for resizing extended and logical partitions resize/resize.ml | 118 +++++++++++++++++++++++++++++++++++++++------ resize/test-virt-resize.pl | 32 ++---------- 2 files changed, 107 insertions(+), 43 deletions(-) -- 1.9.3

2 6

[PATCH] diff: do not pad uid/gid in CSV mode
by Pino Toscano 20 Oct '14

20 Oct '14

--- diff/diff.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/diff/diff.c b/diff/diff.c index f4b25e9..ad371b0 100644 --- a/diff/diff.c +++ b/diff/diff.c @@ -1108,7 +1108,7 @@ output_int64_uid (int64_t i) { next_field (); /* csv doesn't need escaping */ - if (printf ("%4" PRIi64, i) < 0) { + if (printf (csv ? "%" PRIi64 : "%4" PRIi64, i) < 0) { perror ("printf"); exit (EXIT_FAILURE); } -- 1.9.3

2 1

GIT: [PATCH 0/5] v2v: Multiple fixes for handling semi-standard OVA files (RHBZ#1152998).
by Richard W.M. Jones 18 Oct '14

18 Oct '14

OVA not a real standard. Colour me surprised ...

1 5

ANNOUNCE: libguestfs 1.28 released
by Richard W.M. Jones 18 Oct '14

18 Oct '14

I'm pleased to announce libguestfs 1.28, a library and set of tools for accessing and modifying virtual machine disk images. This release took 7 months of work by a considerable number of people, and has many new features (see release notes below), including new 'virt-v2v' and 'virt-p2v' tools for importing guests from foreign hypervisors to KVM, and 'virt-log' which is a tool for extracting log files from guests (including Windows). You can get libguestfs 1.28 here: Main website: http://libguestfs.org/ Source: http://libguestfs.org/download/1.28-stable/ You will also need latest supermin from here: http://libguestfs.org/download/supermin/ Fedora 21: http://koji.fedoraproject.org/koji/packageinfo?packageID=8391 It will appear as an update for F21 in about a week. Debian/experimental coming soon, see: https://packages.debian.org/experimental/libguestfs0 [ From http://libguestfs.org/guestfs-release-notes.1.html ] RELEASE NOTES FOR LIBGUESTFS 1.28 New features Tools virt-v2v(1) and virt-p2v(1) are tools for converting from foreign hypervisors (such as VMware or Xen) or physical machines, to KVM. These tools were previously a separate project. The code for both tools has been integrated into the libguestfs tree. As part of the refactoring, virt-p2v has been rewritten (from Ruby, now C), and virt-v2v now uses several modern features of qemu which make it run many times faster than before. virt-v2v(1) can import from: VMware vCenter, VMware OVA files, local disk files, libvirt, RHEL 5 Xen. It can output to: oVirt or Red Hat Enterprise Virtualization (RHEV-M), OpenStack Glance, libvirt, qemu, local disk files. virt-log(1) is a new tool for listing log files from guests. It supports a variety of guests including Linux traditional, Linux using journal, and Windows event log. virt-builder(1) has a new --machine-readable option to make it simpler to consume virt-builder from other programs. New virt-builder(1) and virt-customize(1) --chmod option. virt-sysprep(1) will now remove: systemd journals, anaconda logs, lightdm logs, debian-installer files, apt logs, exim logs, ConsoleKit logs, firewalld logs, grubby logs, proftpd logs, libvirt/libxml logs, and several other log files. The virt-sysprep user-account operation is now able to remove or keep particular user accounts. (Pino Toscano). New guestmount(1) --fd option, allowing you to run guestmount captive under another process. virt-sparsify(1) has a new --tmp option allowing you to precisely control where temporary files are stored. virt-sparsify(1) can now write to a block device. virt-customize(1) and virt-sysprep(1) now support adding firstboot scripts to Windows guests. virt-cat(1), virt-edit(1), virt-ls(1) now support the -m option, allowing you to override automatic inspection of guests and instead specify where to mount partitions (Pino Toscano). The virt tools now have coloured output: green for OK, red for errors, blue for warnings. Language bindings You can build a Python distribution using make -C python sdist Ruby bindings have switched from Test::Unit to MiniTest. The Java bindings are now compatible with OpenJDK 8 (Pino Toscano). Inspection Oracle Linux is returned as oraclelinux (Nikos Skalkotos). Linux guests which do not have /etc/fstab can now be handled (Pino Toscano). Minix is returned as minix (Pino Toscano). Architectures Aarch64 (64 bit ARM) support has been added. Inspection and the guestfs_file_architecture call can now recognize arm (32 bit) and aarch64 guests. Security CVE-2014-0191 Network usage and entity resolution in XML parsing Libguestfs previously used unsafe libxml2 APIs for parsing libvirt XML. These APIs defaulted to allowing network connections to be made when certain XML documents were presented. Using a malformed XML document it was also possible to exhaust all CPU, memory or file descriptors on the machine. Since the libvirt XML comes from a trusted source (the libvirt daemon) it is not thought that this could have been exploitable. This was fixed in libguestfs ≥ 1.27.9 and the fix was backported to stable versions ≥ 1.26.2, ≥ 1.24.9, ≥ 1.22.10 and ≥ 1.20.13. (Thanks: Dan Berrange, Pino Toscano). Shellshock (bash CVE-2014-6271) This bash bug indirectly affects libguestfs. For more information see: https://www.redhat.com/archives/libguestfs/2014-September/msg00252.html API New APIs guestfs_clear_backend_setting guestfs_get_backend_setting guestfs_set_backend_setting Use these APIs to set individual backend settings. guestfs_cpio_out Convert a directory within the disk image to cpio format. guestfs_journal_get_realtime_usec Fetch precise journal times, see sd_journal_get_realtime_usec(3). guestfs_lstatns guestfs_lstatnslist guestfs_statns Enhanced versions of guestfs_lstat, guestfs_lstatlist, guestfs_stat which return nanosecond timestamps. Build changes Libguestfs can now be built with qemu 2.x. Add support for packaging the appliance on Mageia (Joseph Wang, Thierry Vignaud). Augeas is now a required dependency. Flex and bison are now required dependencies. Internals User-Mode Linux (UML) is no longer a supported backend (although still available). It will be supported again once someone steps in and fixes various regressions and brokenness in the UML (Linux) kernel. You can now connect gdb to qemu by specifying LIBGUESTFS_BACKEND_SETTINGS=gdb Serial BIOS (sgabios) is now used by the libvirt backend (on x86 only) so that we see early BIOS messages. For the OCaml virt tools, exceptions are now handled and printed by a common function. A GCC warning about large stack frames is now enabled by default. Several places that used large arrays and structures on the stack have been fixed. There is now a test for booting the appliance repeatedly. Useful for finding kernel leaks. See: tests/qemu/qemu-boot.c There is a test for testing the speed of various qemu features such as virtio-serial uploads and block device writes. See: tests/qemu/qemu-speed-test.c GCC warnings are now enabled for OCaml-C bindings in the OCaml virt tools. The code for editing files used by virt-edit(1), guestfish(1) edit command, virt-customize(1) --edit option and more is now shared between all these utilities (thanks Pino Toscano). The FUSE tests were rewritten in C to ensure finer control over how system calls are tested. The update-bugs.sh script has been fixed so it should no longer create an empty BUGS file if the Bugzilla server is unavailable. The virt-resize(1) tests now use a stochastic method to ensure much deeper and broader testing. When the network is enabled, the appliance now uses DHCP to acquire an IP address. If libvirt is being used then the appliance will connect to virbr0 (can be overridden by setting LIBGUESTFS_BACKEND_SETTINGS=network_bridge=<some_bridge>). This enables full-featured network connections, with working ICMP, ping and so on. Bugs fixed https://bugzilla.redhat.com/1153515 empty (but set) LIBGUESTFS_BACKEND prints libguestfs: error: invalid backend: https://bugzilla.redhat.com/1151766 libguestfs-bash-completion package contains file /usr/share/bash-completion/completions/virt-log, but no _virt_log() function in that file https://bugzilla.redhat.com/1151738 typo error in man page of virt-edit https://bugzilla.redhat.com/1151033 virt-v2v conversions from VMware vCenter server run slowly https://bugzilla.redhat.com/1150880 virt-v2v: warning: display <graphics type='sdl'> was ignored when converting rhel5 guest with sdl graphics https://bugzilla.redhat.com/1150867 'copy-file-to-file /src /dest' create file /dest even if command 'copy-file-to-file /src /dest' failed https://bugzilla.redhat.com/1150815 Can not find "RECURSIVE LONG LISTING" in 'man virt-diff' https://bugzilla.redhat.com/1150701 virt-v2v does not fstrim data disks and non-mounted filesystems https://bugzilla.redhat.com/1150475 Exporting 2+ disk guest to RHEV, only one disk shows after import https://bugzilla.redhat.com/1148355 virt-filesystems: incorrect LVM vg name when listing Fedora 21 Alpha Atomic image filesystems https://bugzilla.redhat.com/1148072 Option missing --no-selinux-relabel https://bugzilla.redhat.com/1148012 RFE: Allow qemu-bridge-helper to be used to implement guestfs_set_network https://bugzilla.redhat.com/1146815 virt-v2v prints warning: /files/etc/fstab/8/spec references unknown device "cdrom" https://bugzilla.redhat.com/1146275 regression setting root password with virt-builder/virt-customize https://bugzilla.redhat.com/1146017 virt-v2v -v -x during windows guest conversion will hang at hivex: hivex_open: used block id …… https://bugzilla.redhat.com/1145995 Improve the error info when converting windows guest with unclean file system https://bugzilla.redhat.com/1145916 virt-v2v fails to convert win7 guest https://bugzilla.redhat.com/1144891 RFE: stat calls do not return nanosecond timestamps https://bugzilla.redhat.com/1144766 futimens() is a no-op via guestmount https://bugzilla.redhat.com/1144201 guestfish can not restore terminal's output colour when exit guestfish, if the terminal's background colour is black then it will make a inconvenient https://bugzilla.redhat.com/1143949 virt-customize option '--password-crypto' do not work https://bugzilla.redhat.com/1143887 Warning shows when converting guests to rhev:chown: changing ownership of ‘/tmp/v2v.u48xag/*.ovf’: Invalid argument https://bugzilla.redhat.com/1143883 warning shows when converting rhel7 guest:virt-v2v:could not update grub2 console: aug_get: no matching node (ignored) https://bugzilla.redhat.com/1143866 virt-v2v fails with error: cannot open Packages index using db5 https://bugzilla.redhat.com/1142416 part-get-name give 'libguestfs: error: part_get_name: parted does not support the machine output (-m)' error message when run 'part-get-name /dev/sda 1' https://bugzilla.redhat.com/1142186 virt-sysprep option '--mount-options' don't work well https://bugzilla.redhat.com/1142158 Illegal command 'part-get-name /dev/sda1 1' cause libguestfs appliance crashed https://bugzilla.redhat.com/1142004 virt-v2v prints waring:WARNING:/files/boot/grub/device.map references unknown device "xvda" https://bugzilla.redhat.com/1141723 virt-v2v: error: disk sda has no defined format shows when converting xen hvm guest https://bugzilla.redhat.com/1141680 [RFE] virt-v2 should support convert a domain with using domain's UUID instead of domain name https://bugzilla.redhat.com/1141631 [RFE] virt-v2 should support convert a guest to a dir-pool with using pool's uuid https://bugzilla.redhat.com/1141157 virt-sysprep option '--user-accounts' don't work well https://bugzilla.redhat.com/1141145 virt-v2v fails to convert xen pv guests. https://bugzilla.redhat.com/1141113 virt-v2v fails to convert esx guests https://bugzilla.redhat.com/1140946 qemu-img shows error message for backing file twice https://bugzilla.redhat.com/1140894 No error messages output if append '--format qcow2' after '-a guest.img', guest.img is a raw format image file https://bugzilla.redhat.com/1140547 virt-builder option '--format' don't work well https://bugzilla.redhat.com/1140156 RFE: Export to RHEV data domain https://bugzilla.redhat.com/1140050 No error shows when multiple conflicting options used with virt-v2v https://bugzilla.redhat.com/1139973 Improve the error info when converting xen guest with no passwordless SSH access configured https://bugzilla.redhat.com/1139543 Improve the error info when converting guest with no space left https://bugzilla.redhat.com/1138586 No error shows when converting running guest with virt-v2v https://bugzilla.redhat.com/1138184 virt-v2v will fail when converting guests with initramfs-*kdump.img under /boot https://bugzilla.redhat.com/1138182 xen guest will be kernel panic after converted by virt-v2v https://bugzilla.redhat.com/1130189 Annoying message about permissions of /dev/kvm https://bugzilla.redhat.com/1123007 libguestfs 'direct' backend should close file descriptors before exec-ing qemu to avoid leaking !O_CLOEXEC fds https://bugzilla.redhat.com/1122557 virt-sparsify overwrites block devices if used as output files https://bugzilla.redhat.com/1113156 Empty fstab breaks libguestfs inspection https://bugzilla.redhat.com/1111662 Guestfs.Error("vgchange_uuid_all: Volume group has active logical volumes") https://bugzilla.redhat.com/1109174 virt-win-reg manual page is corrupted https://bugzilla.redhat.com/1106548 root gets an error accessing to a non-root dir on a snapshot guestmount VMDK img https://bugzilla.redhat.com/1103877 These APIs also need to add to `guestfish -h` command list https://bugzilla.redhat.com/1102448 mageia package list is incorrect https://bugzilla.redhat.com/1102447 libguestfs 1.27.13 does not build due to libxml library order https://bugzilla.redhat.com/1100498 RFE: non-polling mechanism to detect guestmount --no-fork readiness https://bugzilla.redhat.com/1099490 scrub-file can't handle link file https://bugzilla.redhat.com/1099284 typo errors in man pages https://bugzilla.redhat.com/1098718 RFE: virt-builder aliases https://bugzilla.redhat.com/1096465 virt-builder "proxy=off" setting doesn't turn off the proxy for downloads https://bugzilla.redhat.com/1094746 virt-sparsify man failes to mention what happens to snapshots within a qcow2 image (they are discarded) https://bugzilla.redhat.com/1092753 virt-builder: If several repos contain the same os-version images, then they are duplicated in -l output https://bugzilla.redhat.com/1091803 tar-in-opts execute failed that cause libguestfs appliance crashed https://bugzilla.redhat.com/1088424 virt-resize: libguestfs error: part_set_name: parted: /dev/sdb: Error during translation: Invalid or incomplete multibyte or wide character https://bugzilla.redhat.com/1088262 virt-builder cannot write to a block device https://bugzilla.redhat.com/1085029 Ruby tests fail with latest Ruby https://bugzilla.redhat.com/1079210 virt-sparsify --in-place cannot sparsify as much as it should https://bugzilla.redhat.com/1079182 virt-df cannot report used disk space of windows guest when updated to 6.5 https://bugzilla.redhat.com/1077817 virt-builder error: "syntax error at line 3: syntax error" https://bugzilla.redhat.com/1056290 virt-sparsify overwrites block devices if used as output files https://bugzilla.redhat.com/812970 virt-rescue cannot set ≥<rescue> prompt, on Ubuntu 12.04 -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v

1 1

[PATCH] btrfs: list only subvolumes below path
by Olaf Hering 16 Oct '14

16 Oct '14

The current code does not cover all subvolume variants. If a subvolume exists which covers the entire root filesystem the result is two mountables that have is_root set. This leads to failures in virt-ls and other tools because they work only with a single root. Update the btrfs subvolume call to use the "-o" option. This fixes two cases: first it skips the subvolume for the root filesystem, so the virt-ls case is covered. Second case is failure of mount(1) in guestfs___check_for_filesystem_on. Without this change the subvol would be a relative path, which causes mount(1) to return with ENOENT. The option "-o" changes the output to print <path> as absolute: ><rescue> btrfs subvolume list /sysroot/ ID 257 gen 135 top level 5 path @ ID 258 gen 124 top level 257 path boot/grub2/i386-pc vs. ><rescue> btrfs subvolume list -o /sysroot/ ID 258 gen 124 top level 257 path @/boot/grub2/i386-pc mount -osubvol=boot/grub2/i386-pc $dev /sysroot will fail, while mount -osubvol=@/boot/grub2/i386-pc $dev /sysroot will succeed. Signed-off-by: Olaf Hering <olaf(a)aepfle.de> --- daemon/btrfs.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/daemon/btrfs.c b/daemon/btrfs.c index 7a4d43d..3aceffc 100644 --- a/daemon/btrfs.c +++ b/daemon/btrfs.c @@ -356,6 +356,7 @@ do_btrfs_subvolume_list (const mountable_t *fs) ADD_ARG (argv, i, str_btrfs); ADD_ARG (argv, i, "subvolume"); ADD_ARG (argv, i, "list"); + ADD_ARG (argv, i, "-o"); ADD_ARG (argv, i, fs_buf); ADD_ARG (argv, i, NULL); @@ -399,7 +400,7 @@ do_btrfs_subvolume_list (const mountable_t *fs) * updated. * "top level <n>" is the top level subvolume ID. * "path <str>" is the subvolume path, relative to the top of the - * filesystem. + * filesystem. Using the option "-o" will print the absolute path. * * Note that the order that each of the above is fixed, but * different versions of btrfs may display different sets of data.

1 0

Re: [Libguestfs] Virt-v2v conversion issue
by Richard W.M. Jones 16 Oct '14

16 Oct '14

[Please keep replies on the list] On Mon, Oct 13, 2014 at 07:47:34AM +0000, VONDRA Alain wrote: > I give you some news about the conversion issue, I don't have the > same problem, but I can't finalize the conversion, the process > freezes randomly, sometimes during the conversion of the first disk > or the third... Can you run virt-v2v with the -x and -v options and post the complete output. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v

2 30

[PATCH nbdkit] streaming plugin: Add support for a sliding window
by Richard W.M. Jones 15 Oct '14

15 Oct '14

[Sorry, can't use git-send-email at the moment ...] These patches implement a sliding window for the streaming plugin[1] in nbdkit. I would like to be able to stream a filesystem from tools such as 'virt-make-fs'[2]. This is a fairly frequently requested feature. Unfortunately: (a) The patches make the code significantly more complex and therefore likely to have bugs. (b) They are not practically useful. 'parted' likes to write to the beginning and end of a disk, even when creating a simple MBR, and of course 'mkfs' scribbles the group headers across the whole disk when creating a filesystem. A simple window approach is obviously not sufficient. A better approach might be something like a sparse, size-limited map recording writes at any point in the disk. But that has the problem that you don't know when you can commit a write to the stream -- some heuristic would have to be used. I'm posting them to the mailing list for the record and in case anyone has any better ideas. Rich. [1] http://rwmj.wordpress.com/2014/10/14/streaming-nbd-server/#content [2] http://libguestfs.org/virt-make-fs.1.html -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW

1 0

[PATCH] cat, diff: avoid double slashes in paths (RHBZ#1151910).
by Pino Toscano 13 Oct '14

13 Oct '14

In full_path, skip the trailing slash in the base directory when different than "/", as the slash will eventually be added when building the resulting path. --- cat/visit.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/cat/visit.c b/cat/visit.c index 963beb8..49b779e 100644 --- a/cat/visit.c +++ b/cat/visit.c @@ -138,13 +138,18 @@ full_path (const char *dir, const char *name) { int r; char *path; + int len; + + len = strlen (dir); + if (len > 0 && dir[len - 1] == '/') + --len; if (STREQ (dir, "/")) r = asprintf (&path, "/%s", name ? name : ""); else if (name) - r = asprintf (&path, "%s/%s", dir, name); + r = asprintf (&path, "%.*s/%s", len, dir, name); else - r = asprintf (&path, "%s", dir); + r = asprintf (&path, "%.*s", len, dir); if (r == -1) { perror ("asprintf"); -- 1.9.3

1 0

[PATCH] ls: in CSV mode, always have a checksum field (RHBZ#1151900).
by Pino Toscano 13 Oct '14

13 Oct '14

Make sure to output the field for checksum even for non-regular files, as empty, in CSV output mode. This ensures each line has the same number of fields, regardless of the file type. --- cat/ls.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/cat/ls.c b/cat/ls.c index de8248e..2cb4e74 100644 --- a/cat/ls.c +++ b/cat/ls.c @@ -524,12 +524,15 @@ show_file (const char *dir, const char *name, path = full_path (dir, name); - if (checksum && is_reg (stat->st_mode)) { - csum = guestfs_checksum (g, checksum, path); - if (!csum) - exit (EXIT_FAILURE); + if (checksum) { + if (is_reg (stat->st_mode)) { + csum = guestfs_checksum (g, checksum, path); + if (!csum) + exit (EXIT_FAILURE); - output_string (csum); + output_string (csum); + } else if (csv) + output_string (""); } output_string (path); -- 1.9.3

1 0