July 2017 - Libguestfs - libguestfs List Archives

[PATCH v2] launch: add support for autodetection of appliance image format
by Pavel Butsykin 06 Feb '20

06 Feb '20

This feature allows you to use different image formats for the fixed appliance. The raw format is used by default. Signed-off-by: Pavel Butsykin <pbutsykin(a)virtuozzo.com> --- lib/launch-direct.c | 2 ++ lib/launch-libvirt.c | 19 ++++++++++++------- m4/guestfs_appliance.m4 | 11 +++++++++++ 3 files changed, 25 insertions(+), 7 deletions(-) diff --git a/lib/launch-direct.c b/lib/launch-direct.c index 0be662e25..b9b54857a 100644 --- a/lib/launch-direct.c +++ b/lib/launch-direct.c @@ -592,7 +592,9 @@ launch_direct (guestfs_h *g, void *datav, const char *arg) append_list ("id=appliance"); append_list ("cache=unsafe"); append_list ("if=none"); +#ifndef APPLIANCE_FMT_AUTO append_list ("format=raw"); +#endif } end_list (); start_list ("-device") { append_list ("scsi-hd"); diff --git a/lib/launch-libvirt.c b/lib/launch-libvirt.c index 4adb2cfb3..030ea6911 100644 --- a/lib/launch-libvirt.c +++ b/lib/launch-libvirt.c @@ -212,9 +212,10 @@ get_source_format_or_autodetect (guestfs_h *g, struct drive *drv) /** * Create a qcow2 format overlay, with the given C<backing_drive> - * (file). The C<format> parameter, which must be non-NULL, is the - * backing file format. This is used to create the appliance overlay, - * and also for read-only drives. + * (file). The C<format> parameter is the backing file format. + * The C<format> parameter can be NULL, in this case the backing + * format will be determined automatically. This is used to create + * the appliance overlay, and also for read-only drives. */ static char * make_qcow2_overlay (guestfs_h *g, const char *backing_drive, @@ -223,8 +224,6 @@ make_qcow2_overlay (guestfs_h *g, const char *backing_drive, char *overlay; struct guestfs_disk_create_argv optargs; - assert (format != NULL); - if (guestfs_int_lazy_make_tmpdir (g) == -1) return NULL; @@ -232,8 +231,10 @@ make_qcow2_overlay (guestfs_h *g, const char *backing_drive, optargs.bitmask = GUESTFS_DISK_CREATE_BACKINGFILE_BITMASK; optargs.backingfile = backing_drive; - optargs.bitmask |= GUESTFS_DISK_CREATE_BACKINGFORMAT_BITMASK; - optargs.backingformat = format; + if (format) { + optargs.bitmask |= GUESTFS_DISK_CREATE_BACKINGFORMAT_BITMASK; + optargs.backingformat = format; + } if (guestfs_disk_create_argv (g, overlay, "qcow2", -1, &optargs) == -1) { free (overlay); @@ -461,7 +462,11 @@ launch_libvirt (guestfs_h *g, void *datav, const char *libvirt_uri) /* Note that appliance can be NULL if using the old-style appliance. */ if (appliance) { +#ifdef APPLIANCE_FMT_AUTO + params.appliance_overlay = make_qcow2_overlay (g, appliance, NULL); +#else params.appliance_overlay = make_qcow2_overlay (g, appliance, "raw"); +#endif if (!params.appliance_overlay) goto cleanup; } diff --git a/m4/guestfs_appliance.m4 b/m4/guestfs_appliance.m4 index 81c43879f..4e1ec8135 100644 --- a/m4/guestfs_appliance.m4 +++ b/m4/guestfs_appliance.m4 @@ -139,3 +139,14 @@ AC_SUBST([GUESTFS_DEFAULT_PATH]) AC_DEFINE_UNQUOTED([GUESTFS_DEFAULT_PATH], ["$GUESTFS_DEFAULT_PATH"], [Define guestfs default path.]) + +AC_ARG_ENABLE([appliance-fmt-auto], + [AS_HELP_STRING([--enable-appliance-fmt-auto], + [enable autodetection of appliance image format @<:@default=no@:>@])], + [ENABLE_APPLIANCE_FMT_AUTO="$enableval"], + [ENABLE_APPLIANCE_FMT_AUTO=no]) + +if test "x$ENABLE_APPLIANCE_FMT_AUTO" = "xyes"; then + AC_DEFINE([APPLIANCE_FMT_AUTO], [1], + [Define to 1 if enabled autodetection of appliance image format.]) +fi -- 2.13.0

2 4

Plan for libguestfs 1.36
by Richard W.M. Jones 05 Nov '17

05 Nov '17

Libguestfs 1.34 was released on 2016-08-08, which is about 6 months ago. I'd would like to think about what needs work for the next stable 1.36 release, which might happen at the end of this month. Any new APIs added in the 1.35 cycle will become supported and guaranteed when we release 1.36, so we need to check those. See: git diff v1.34.0 -- generator/actions.ml It looks like there are only 3 new public APIs. Please follow-up also if there are features / blockers / bugs that need to be addressed for 1.36. As usual, bugs which have "1.36" (without quotes) in the "Devel Whiteboard" field in Bugzilla are nominated as blockers for the release. You can see a list of those here (currently empty): https://bugzilla.redhat.com/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&… For a list of all upstream bugs, see: https://bugzilla.redhat.com/buglist.cgi?component=libguestfs&product=Virtua… Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-builder quickly builds VMs from scratch http://libguestfs.org/virt-builder.1.html

1 2

libguestfs binaries missing when built from source code
by emlyn.jose＠wipro.com 01 Sep '17

01 Sep '17

Hello, This is regarding the binary missing from libguestfs built from the source code(v 1.30.3). Here there are some binaries missing when compared to the libguestfs rpm installed from the rhel6-server repo. Below are the list of binaries >From the os rpm libguestfs-tools-c-1.20.11-2.el6.x86_64: >From the one built from source code(v 1.30.3) is : guestfish guestfish guestmount guestmount virt-alignment-scan virt-alignment-scan virt-cat virt-cat virt-copy-in virt-copy-in virt-copy-out virt-copy-out virt-df virt-df virt-edit virt-edit virt-filesystems virt-filesystems virt-format virt-format virt-inspector2 virt-inspector virt-ls virt-ls virt-rescue virt-rescue virt-resize virt-sparsify virt-sysprep virt-tar-in virt-tar-in virt-tar-out virt-tar-out The highlighted binaries are missing from the v 1.30.3. Below is the content inside the tar file: [root(a)rhel6.3 SOURCES]# ls appliance-1.30.1.tar.xz gdk-pixbuf-2.21.7.tar.gz libguestfs-1.30.3 linsee.env atk-1.29.2.tar.gz glib-2.27.3.tar.gz libguestfs-1.30.3.tar.gz seehelp.content augeas-1.0.0.tar.gz gtk+-2.24.4.tar.gz libguestfs-post.env seehelp.template [root(a)rhel6.3 SOURCES]# cd libguestfs-1.30.3 [root(a)rhel6.3 libguestfs-1.30.3]# ls aclocal.m4 configure.ac gnulib Makefile.in sparsify align contrib GNUmakefile make-fs src appliance COPYING gobject mllib subdir-rules.mk AUTHORS COPYING.LIB golang ocaml sysprep autogen.sh csharp guestfs-release-notes.pod ocaml-link.sh tests bash customize guestfs-release-notes.txt p2v test-tool bindtests daemon HACKING perl tmp bootstrap df haskell php TODO BUGS dib html pick-guests.pl.in tools bugs-in-changelog.sh diff inspector po update-bugs.sh build-aux edit installcheck.sh.in po-docs v2v builder erlang java podwrapper.pl.in valgrind-suppressions cat examples libtool-kill-dependency_libs.sh python zanata-pull.sh cfg.mk fish logo README zanata.xml ChangeLog format lua rescue common-rules.mk fuse m4 resize config.h.in generator maint.mk ruby configure get-kernel Makefile.am run.in What is missing here that results in missing those binaries? Regards, Emlyn Jose The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com

3 15

[PATCH v11 00/10] Reimplement inspection in the daemon.
by Richard W.M. Jones 08 Aug '17

08 Aug '17

v10: https://www.redhat.com/archives/libguestfs/2017-July/msg00245.html No actual change here, but I rebased and retested. Also this series now does not depend on any other patch series since everything else needed is upstream. Rich.

2 18

read/write performance through mount point by guestmount
by lampahome 03 Aug '17

03 Aug '17

I create a *4x256GB-SSD RAID0(/dev/md1)* and I test the performance through fio. fio config: ioengine=libaiodirect=1time_basedruntime=120ramp_time=30size=100g The sequential read/write performance is: > > *read: 2000MB/swrite: 1800MB/s* Now I make a ext4 on the RAID0(/dev/md1) and mount on /home/. ANd I create a 100G-disk.qcow2 by guestfish. > $guestfish > ><fish>: disk-create disk.qcow2 qcow2 100*1024*1024*1024 > ><fish>: run > ><fish>: mkfs ext4 /dev/sda > ><fish>: shutdown > ><fish>: quit No errors showed. I mount the disk.qcow2 on the /home/test/, and create a 50GB file. > $dd if=/dev/zero of=/home/test/file bs=1M count=51280 Test sequential read/write again through fio and config almost the same: ioengine=libaiodirect=1time_basedruntime=120ramp_time=30size=50g and the testing filename is /home/test/file and the performance is : > > *read: 86MB/swrite: 43MB/s* how come the performance is so bad? almost down 95%. My machine is: 4.4.0-83-generic, Ubuntu 16.04, x86_64 GNU/Linux

2 6

[PATCH 1/1] New partition API: part_resize
by Nikos Skalkotos 01 Aug '17

01 Aug '17

This can be used to enlarge or shrink an existing partition. --- daemon/parted.c | 31 +++++++++++++++++++++++++++++++ generator/actions_core.ml | 22 ++++++++++++++++++++++ generator/proc_nr.ml | 1 + lib/MAX_PROC_NR | 2 +- 4 files changed, 55 insertions(+), 1 deletion(-) diff --git a/daemon/parted.c b/daemon/parted.c index 72e1b8420..f1205cadf 100644 --- a/daemon/parted.c +++ b/daemon/parted.c @@ -178,6 +178,37 @@ do_part_del (const char *device, int partnum) } int +do_part_resize (const char *device, int partnum, int64_t endsect) +{ + int r; + CLEANUP_FREE char *err = NULL; + char endstr[32]; + char partnum_str[16]; + + if (partnum <= 0) { + reply_with_error ("partition number must be >= 1"); + return -1; + } + + snprintf (partnum_str, sizeof partnum_str, "%d", partnum); + snprintf (endstr, sizeof endstr, "%" PRIi64 "s", endsect); + + udev_settle (); + + r = commandf (NULL, &err, COMMAND_FLAG_FOLD_STDOUT_ON_STDERR, + "parted", "-s", "--", device, "resizepart", partnum_str, + endstr, NULL); + if (r == -1) { + reply_with_error ("parted: %s: %s:", device, err); + return -1; + } + + udev_settle(); + + return 0; +} + +int do_part_disk (const char *device, const char *parttype) { int r; diff --git a/generator/actions_core.ml b/generator/actions_core.ml index ea0735676..939aec7ee 100644 --- a/generator/actions_core.ml +++ b/generator/actions_core.ml @@ -9646,4 +9646,26 @@ be mountable but require special options. Filesystems may not all belong to a single logical operating system (use C<guestfs_inspect_os> to look for OSes)." }; + { defaults with + name = "part_resize"; added = (1, 37, 20); + style = RErr, [String (Device, "device"); Int "partnum"; Int64 "endsect"], []; + tests = [ + InitEmpty, Always, TestRun ( + [["part_init"; "/dev/sda"; "mbr"]; + ["part_add"; "/dev/sda"; "primary"; "1"; "-1025"]; + ["part_resize"; "/dev/sda"; "1"; "-1"]]), [] + ]; + shortdesc = "resize a partition"; + longdesc = "\ +This command resizes the partition numbered C<partnum> on C<device> +by moving the end position. + +Note that this does not modify any filesystem present in the partition. +If you wish to do this, you will need to use filesystem resizing +commands like C<guestfs_resize2fs>. + +When growing a partition you will want to grow the filesystem +afterwards, but when shrinking, you need to shrink the filesystem +before the partition." }; + ] diff --git a/generator/proc_nr.ml b/generator/proc_nr.ml index dec02f5fa..7895063b6 100644 --- a/generator/proc_nr.ml +++ b/generator/proc_nr.ml @@ -484,6 +484,7 @@ let proc_nr = [ 474, "internal_yara_scan"; 475, "file_architecture"; 476, "list_filesystems"; +477, "part_resize"; ] (* End of list. If adding a new entry, add it at the end of the list diff --git a/lib/MAX_PROC_NR b/lib/MAX_PROC_NR index b86395733..bf2c10d23 100644 --- a/lib/MAX_PROC_NR +++ b/lib/MAX_PROC_NR @@ -1 +1 @@ -476 +477 -- 2.13.3

4 6

[PATCH] build: require ocaml-hivex for the daemon
by Pino Toscano 31 Jul '17

31 Jul '17

Since commit d5b6f1df5ff2d387a5dfc89b8316c0dff67ce2c9, the daemon requires ocaml-hivex; OTOH, nothing checks it is actually available, so the build of the daemon will fail with a semi-cryptic error about the lack of a directory in the OCaml install prefix. As fix, check for the presence of the hivex module at build time, failing earlier if not present. The check is performed only when the daemon is enabled, as ocaml-hivex is not used for anything else than the daemon. --- m4/guestfs_ocaml.m4 | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/m4/guestfs_ocaml.m4 b/m4/guestfs_ocaml.m4 index d0f3349..e577b31 100644 --- a/m4/guestfs_ocaml.m4 +++ b/m4/guestfs_ocaml.m4 @@ -56,6 +56,14 @@ AM_CONDITIONAL([HAVE_OCAMLOPT], AM_CONDITIONAL([HAVE_OCAMLDOC], [test "x$OCAMLDOC" != "xno"]) +if test "x$enable_daemon" = "xyes"; then + OCAML_PKG_hivex=no + AC_CHECK_OCAML_PKG(hivex) + if test "x$OCAML_PKG_hivex" = "xno"; then + AC_MSG_ERROR([the OCaml module 'hivex' is required]) + fi +fi + OCAML_PKG_gettext=no OCAML_PKG_libvirt=no OCAML_PKG_oUnit=no -- 2.9.4

3 3

cannot find any suitable libguestfs supermin
by lampahome 30 Jul '17

30 Jul '17

I download libguestfs from github and fetch to the *origin master*. I tried to build libguestfs in my own linux distro and I followd the http://libguestfs.org/guestfs-building.1.html#using-a-prebuilt-binary-appli… to replace with supermin I download the appliance-1.30.1.tar.xz and appliance-1.30.1.tar.xz.sig and extract the tarball to /home/user/appliance and export LIBGUESTFS_PATH=/*home/user/appliance* Then I build libguestfs below: > ./autogen.sh --disable-appliance --disable-daemon > make make install DESTDIR=/tmp It builds fine. And guestfish version is 1.37.18 But I got error when I execute guestfish: > guestfish > add --rw disk.qcow2 > run > error: libguestfs: error: cannot find any suitable libguestfs supermin, > fixed or old-style appliance on LIBGUESTFS_PATH But I echo $LIBGUESTFS_PATH, it's */home/user/appliance* I don't know how come going wrong

2 6

[PATCH 0/1] New partition API: part_resize
by Nikos Skalkotos 29 Jul '17

29 Jul '17

Hello, I've added a new API method to expose parted's resizepart command. It's been there for some years now and can be used to change the ending boundaries of a partition. This command does always retain the partition number and flags. There are cases where simulating this behavior by removing and re-creating the partition is not easy. One such case is when the partition numbering contains gaps. If we have a partition table with 2 partitions with numbers 1 and 3, removing and re-creating the latter with parted will change its number to 2. Nikos Skalkotos (1): New partition API: part_resize daemon/parted.c | 31 +++++++++++++++++++++++++++++++ generator/actions_core.ml | 22 ++++++++++++++++++++++ generator/proc_nr.ml | 1 + lib/MAX_PROC_NR | 2 +- 4 files changed, 55 insertions(+), 1 deletion(-) -- 2.13.3

1 0

where to create localenv?
by lampahome 29 Jul '17

29 Jul '17

I don't want prefix 'git' and I watched this http://libguestfs.org/guestfs-building.1.html#local-files . localenv ./autogen.sh \ -C \ --enable-werror \ "$@" but I cannot execute . localenv and I saw Note that *localenv* is included by the top Makefile (so it’s a Makefile fragment). But if it is also sourced by your *localconfigure* script then it is used as a shell script. But I don't see any file named localenv after execute cmds and make: > ./autogen.sh > make how to solve it?

2 2