October 2017 - Libguestfs - libguestfs List Archives

[PATCH 0/2] lib: Allow db_dump package to be a weak dependency.
by Richard W.M. Jones 06 Oct '17

06 Oct '17

Fix for: https://bugzilla.redhat.com/show_bug.cgi?id=1409024

2 7

Re: [Libguestfs] --with-qemu option in libguestfs
by Richard W.M. Jones 06 Oct '17

06 Oct '17

[PLEASE maintain libguestfs mailing list. I'm not paid to give free private help] On Fri, Oct 06, 2017 at 10:51:06AM +0000, emlyn.jose(a)wipro.com wrote: > >Don't use ‘--with-qemu=...qemu-img’, it is simply wrong. The --with-qemu parameter is intended to specify the hypervisor, and qemu-img is not a hypervisor. > Yes this is clear now. Thank you! > > > You just said above that you set the PATH to contain both qemu-kvm and qemu-img. > >Anyway, I suggest you enable debugging which will show you exactly what libguestfs is doing. > > Yes, I exported PATH correctly and is pointing to where I wanted but look at the log below after enabling the debugging that shows path of qemu-kvm and qemu-img. They are different. > > libguestfs: command: run: qemu-img > libguestfs: command: run: \ create > libguestfs: command: run: \ -f qcow2 > libguestfs: command: run: \ -o backing_file=/root/image123.tmp > libguestfs: command: run: \ /tmp/libguestfs123/overlay1 > Formatting '/tmp/…… > ----- > ------ > libguestfs: [00000ms] begin testing qemu features > libguestfs: command: run: /opt/qemu/bin/qemu-kvm > libguestfs: command: run: \ -display none > libguestfs: command: run: \ -help > > Does this mean that they are both taken from the same location? The first message indicates that qemu-img was found and run from $PATH, so it depends what $PATH is set to. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v

1 0

[PATCH nbdkit 0/3] misc fixes
by Pino Toscano 06 Oct '17

06 Oct '17

Hi, few miscellaneous fixes for nbdkit. Thanks, Pino Toscano (3): build: use AC_MSG_RESULT for $PERL_LDOPTS crypto: add missing '{' tests: fix generation of PKI test data configure.ac | 1 + src/crypto.c | 2 +- tests/Makefile.am | 10 +++++----- 3 files changed, 7 insertions(+), 6 deletions(-) -- 2.13.6

2 4

Re: [Libguestfs] --with-qemu option in libguestfs
by Richard W.M. Jones 06 Oct '17

06 Oct '17

[Please keep replies on the mailing list so that others can benefit from following the discussion] On Fri, Oct 06, 2017 at 06:04:46AM +0000, emlyn.jose(a)wipro.com wrote: > > qemu is a hypervisor, qemu-img is a program for manipulating disk > > images. They are very different things, and it makes no sense to > > substitute qemu-img for qemu. > > I do not want to substitute qemu-img instead of qemu : > > • virt-resize uses qemu-kvm as well as qemu-img. > > • My qemu binaries come from a different path(instead of /usr/bin/) > which is set in $PATH already > > • Providing "export PATH=/path/to/qemu-kvm/and/qemu-img" (both > binaries are in same location) and using "--with > qemu=qemu-kvm qemu-img" takes only the qemu-kvm correctly Don't use ‘--with-qemu=...qemu-img’, it is simply wrong. The --with-qemu parameter is intended to specify the hypervisor, and qemu-img is not a hypervisor. > but qemu-img is still not taken from the $PATH. Why? qemu-img comes from the PATH, see: https://github.com/libguestfs/libguestfs/blob/master/lib/create.c#L317 https://github.com/libguestfs/libguestfs/blob/master/lib/info.c#L194 External commands run through the guestfs_int_cmd_* mechanism can be displayed by enabling debugging: http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs so I suggest you enable this to see exactly what's going on. > • I used "--with qemu=qemu-kvm qemu-img" as well as “export > LIBGUESTFS_HV=/path/to/qemu-img” which will obviously > override qemu-kvm to qemu-img > > Hope you got what I am saying now? > > I do not want to substitute qemu-img instead of qemu-kvm. Both has > to be taken from the specified path i.e., $PATH. > > > P.S: Its surprising to see that even after uninstalling the available > > qemu-img package from my host, libguestfs takes it without any > > issues!! Where does it take that "qemu-img" binary from? > > > From $PATH. > > How is that possible when there is no qemu-img installed which in > turn has no such binary available in $PATH. That is weird!!! I just > want to know if libguestfs itself has some qemu-img binary inbuilt > to use it for virt-resize. You just said above that you set the PATH to contain both qemu-kvm and qemu-img. Anyway, I suggest you enable debugging which will show you exactly what libguestfs is doing. Also don't put qemu-img into the ‘./configure --with-qemu’ parameter. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/

1 0

[PATCH] build: Don't test for qemu virtio-serial at configure time.
by Richard W.M. Jones 05 Oct '17

05 Oct '17

qemu has supported virtio-serial since forever, and we depend on much newer features now, so testing for virtio-serial is pointless. In fact allowing a set of possible qemu binaries to be supplied is also pointless, since packagers or even people building from source know what qemu binary they want to use. Now the possible options for --with-qemu are: * --with-qemu=search (or --with-qemu omitted) Search for various qemu binaries on the path. * --with-qemu=/path/to/qemu Use the supplied qemu binary without question. No checks on the binary are done. * --with-qemu=no or --without-qemu Disable the default hypervisor. You have to set $LIBGUESTFS_HV before running programs. --- docs/guestfs-building.pod | 7 +-- m4/guestfs_qemu.m4 | 137 ++++++++++++++++++---------------------------- 2 files changed, 56 insertions(+), 88 deletions(-) diff --git a/docs/guestfs-building.pod b/docs/guestfs-building.pod index 187da37be..9b3f37901 100644 --- a/docs/guestfs-building.pod +++ b/docs/guestfs-building.pod @@ -686,11 +686,10 @@ development libraries are installed. This option forces virt-p2v to be built against Gtk 2, which is currently the most widely tested configuration. -=item B<--with-qemu=">bin1 bin2 ...B<"> +=item B<--with-qemu=>/usr/bin/qemu-system-x86_64 -Provide an alternate qemu binary (or list of binaries). This can be -overridden at runtime by setting the C<LIBGUESTFS_HV> environment -variable. +Provide an alternate qemu binary. This can be overridden at runtime +by setting the C<LIBGUESTFS_HV> environment variable. =item B<--with-supermin-packager-config=>I<yum.conf> diff --git a/m4/guestfs_qemu.m4 b/m4/guestfs_qemu.m4 index e07e9800f..350bb980d 100644 --- a/m4/guestfs_qemu.m4 +++ b/m4/guestfs_qemu.m4 @@ -17,6 +17,7 @@ dnl Check for QEMU for running binaries on this $host_cpu, fall dnl back to basic 'qemu'. Allow the user to override it. + AS_CASE([$host_cpu], [i@<:@456@:>@86],[qemu_cpu=i386], [arm*],[qemu_cpu=arm], @@ -24,110 +25,78 @@ AS_CASE([$host_cpu], [powerpc64 | ppc64le | powerpc64le],[qemu_cpu=ppc64], [qemu_cpu=$host_cpu]) default_qemu="qemu-kvm kvm qemu-system-$qemu_cpu qemu" + AC_ARG_WITH([qemu], - [AS_HELP_STRING([--with-qemu="bin1 bin2 ..."], - [set default QEMU binary @<:@default="[qemu-kvm] qemu-system-<host> qemu"@:>@])], + [AS_HELP_STRING([--with-qemu=/path/to/qemu], + [set default QEMU binary @<:@default=search@:>@])], dnl --with-qemu or --without-qemu: [], dnl neither option was given: - [with_qemu="$default_qemu"] + [with_qemu=search] ) -AS_IF([test "x$with_qemu" = "xno"],[ - AC_MSG_WARN([qemu was disabled, libguestfs may not work at all]) - QEMU=no -],[ - AC_PATH_PROGS([QEMU],[$with_qemu],[no], - [$PATH$PATH_SEPARATOR/usr/sbin$PATH_SEPARATOR/sbin$PATH_SEPARATOR/usr/libexec]) - test "x$QEMU" = "xno" && AC_MSG_ERROR([qemu must be installed]) +AS_CASE([$with_qemu], + [no],[ + AC_MSG_WARN([qemu was disabled, libguestfs may not work at all]) + QEMU=no + ], + [search],[ + AC_PATH_PROGS([QEMU],[$default_qemu],[no], + [$PATH$PATH_SEPARATOR/usr/sbin$PATH_SEPARATOR/sbin$PATH_SEPARATOR/usr/libexec]) + test "x$QEMU" = "xno" && AC_MSG_ERROR([qemu not found]) AC_DEFINE_UNQUOTED([QEMU],["$QEMU"],[Location of qemu binary.]) - dnl Check that the chosen qemu has virtio-serial support. - dnl For historical reasons this can be disabled by setting - dnl vmchannel_test=no. - if test "x$vmchannel_test" != "xno"; then - AC_MSG_CHECKING([that $QEMU -help works]) - if $QEMU -help >&AS_MESSAGE_LOG_FD 2>&1; then - AC_MSG_RESULT([yes]) - else - AC_MSG_RESULT([no]) - AC_MSG_FAILURE( + # Only check that -help and -version are available and look + # reasonable. Real checks will be performed at runtime. + + AC_MSG_CHECKING([that $QEMU -help works]) + if $QEMU -help >&AS_MESSAGE_LOG_FD 2>&1; then + AC_MSG_RESULT([yes]) + else + AC_MSG_RESULT([no]) + AC_MSG_FAILURE( [$QEMU -help: command failed. This could be a very old version of qemu, or qemu might not be working. ]) - fi + fi - AC_MSG_CHECKING([that $QEMU -version works]) - if $QEMU -version >&AS_MESSAGE_LOG_FD 2>&1; then - AC_MSG_RESULT([yes]) - else - AC_MSG_RESULT([no]) - AC_MSG_FAILURE( + AC_MSG_CHECKING([that $QEMU -version works]) + if $QEMU -version >&AS_MESSAGE_LOG_FD 2>&1; then + AC_MSG_RESULT([yes]) + else + AC_MSG_RESULT([no]) + AC_MSG_FAILURE( [$QEMU -version: command failed. This could be a very old version of qemu, or qemu might not be working. ]) - fi - - AC_MSG_CHECKING([for $QEMU version >= 1]) - if $QEMU -version | grep -sq 'version @<:@1-9@:>@'; then - AC_MSG_RESULT([yes]) - else - AC_MSG_RESULT([no]) - AC_MSG_FAILURE([$QEMU version must be >= 1.0.]) - fi - - dnl Unfortunately $QEMU -device \? won't just work. We probably - dnl need to add a cocktail of different arguments which differ - dnl on the various emulators. Thanks, qemu. - AC_MSG_CHECKING([what extra options we need to use for qemu feature tests]) - QEMU_OPTIONS_FOR_CONFIGURE= - # Note: the order we test these matters. - for opt in "-machine virt" "-machine accel=kvm:tcg" "-display none"; do - if $QEMU $QEMU_OPTIONS_FOR_CONFIGURE $opt -device \? >&AS_MESSAGE_LOG_FD 2>&1; then - QEMU_OPTIONS_FOR_CONFIGURE="$QEMU_OPTIONS_FOR_CONFIGURE $opt" - fi - done - AC_MSG_RESULT([$QEMU_OPTIONS_FOR_CONFIGURE]) - - AC_MSG_CHECKING([that $QEMU $QEMU_OPTIONS_FOR_CONFIGURE -device ? works]) - if $QEMU $QEMU_OPTIONS_FOR_CONFIGURE -device \? >&AS_MESSAGE_LOG_FD 2>&1; then - AC_MSG_RESULT([yes]) - else - AC_MSG_RESULT([no]) - AC_MSG_FAILURE([$QEMU $QEMU_OPTIONS_FOR_CONFIGURE -device ? doesn't work.]) - fi - - AC_MSG_CHECKING([for virtio-serial support in $QEMU]) - if $QEMU $QEMU_OPTIONS_FOR_CONFIGURE -device \? 2>&1 | grep -sq virtio-serial; then - AC_MSG_RESULT([yes]) - else - AC_MSG_RESULT([no]) - AC_MSG_FAILURE( -[I did not find virtio-serial support in -$QEMU. - -virtio-serial support in qemu or KVM is essential for libguestfs -to operate. - -Usually this means that you have to install a newer version of qemu -and/or KVM. Please read the relevant section in the README file for -more information about this. - -You can override this test by setting the environment variable -vmchannel_test=no - -However if you don't have the right support in your qemu, then this -just delays the pain. + fi -If I am using the wrong qemu or you want to compile qemu from source -and install it in another location, then you should configure with -the --with-qemu option. -]) - fi + AC_MSG_CHECKING([for $QEMU version >= 1]) + if $QEMU -version | grep -sq 'version @<:@1-9@:>@'; then + AC_MSG_RESULT([yes]) + else + AC_MSG_RESULT([no]) + AC_MSG_FAILURE([$QEMU version must be >= 1.0.]) fi + ],[ + # AS_CASE default: + # Use the binary supplied without question, but check + # that a full path was given. + QEMU="$with_qemu" + AC_MSG_CHECKING([that $QEMU is an absolute path]) + case "$QEMU" in + /*) + AC_MSG_RESULT([yes]) + ;; + *) + AC_MSG_RESULT([no]) + AC_MSG_ERROR([--with-qemu=$QEMU must be an absolute path]) + ;; + esac + AC_DEFINE_UNQUOTED([QEMU],["$QEMU"],[Location of qemu binary.]) ]) -- 2.13.2

1 0

Re: [Libguestfs] --with-qemu option in libguestfs
by Richard W.M. Jones 05 Oct '17

05 Oct '17

On Thu, Oct 05, 2017 at 10:32:09AM +0000, emlyn.jose(a)wipro.com wrote: > Hi Richard, > > The --with-qemu option description is given as below: > > --with-qemu="bin1 bin2 ..." > set default QEMU binary [default="[qemu-kvm] > qemu-system-<host> qemu"] > > Does specifying this option takes only these 3 binaries? The ‘./configure --with-qemu=...’ option takes a list of possible qemu binary names. It searches the $PATH for each until it finds that name. Libguestfs uses this for various things: - As a default setting for the hypervisor in the handle, which you can override at runtime: http://libguestfs.org/guestfs.3.html#guestfs_set_hv - At configure time we need to test various features of qemu, although generally speaking we are trying to reduce the need to test qemu features at compile time, preferring instead to test at runtime. In fact most of the configure-time tests could now be dropped if we just assumed "new enough" qemu. As a special case you can also do ‘./configure --without-qemu’ (or ‘./configure --with-qemu=no’ which is the same thing) which disables qemu, although that means for libguestfs to work at all you would need to supply a qemu binary at runtime through an environment variable (‘LIBGUESTFS_HV=/path/to/qemu program’). The logic for all this is here: https://github.com/libguestfs/libguestfs/blob/master/m4/guestfs_qemu.m4#L26 > I would like libguestfs to take even the "qemu-img" binary from the > installed qemu package. > How to enable that? qemu is a hypervisor, qemu-img is a program for manipulating disk images. They are very different things, and it makes no sense to substitute qemu-img for qemu. However libguestfs does use qemu-img as well, and it gets it from $PATH when run. There is no configuration required, you just have to make sure that qemu-img is on the $PATH when running any libguestfs program that requires it. > P.S: Its surprising to see that even after uninstalling the > available qemu-img package from my host, libguestfs takes it without > any issues!! Where does it take that "qemu-img" binary from? >From $PATH. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW

1 0

[PATCH 0/9] build: Require OCaml >= 4.02.
by Richard W.M. Jones 05 Oct '17

05 Oct '17

Per my previous email: https://www.redhat.com/archives/libguestfs/2017-September/msg00203.html I'd like to talk about requiring a more modern version of the OCaml compiler. These commits show some of the code changes which would be possible with OCaml >= 3.12 [which it turns out we already require by accident] and also with OCaml >= 4.02. The latter is my favoured option. Rich.

2 12

[PATCH v2 0/2] builder: Choose better weights in the planner.
by Richard W.M. Jones 04 Oct '17

04 Oct '17

v1 -> v2: - Removed the f_type field from StatVFS.statvfs structure. - New function StatVFS.filesystem_is_remote, written in C. [Thinking about it, this should probably be called ‘is_network_filesystem’, but I can change that before pushing]. - Use statvfs instead of fstatvfs, and statfs instead of fstatfs. - Rejigged the comments in builder/builder.ml to make them simpler and clearer. Rich.

2 6

[PATCH v3 0/2] builder: Choose better weights in the planner.
by Richard W.M. Jones 04 Oct '17

04 Oct '17

v2 -> v3: - Drop gnulib fallback.

1 2

[PATCH] build: improve and simplify distro detection
by Pino Toscano 04 Oct '17

04 Oct '17

Add a --with-distro=ID argument for configure, so it is possible to manually specify the distro to use for the packages (in case os-release does not provide ID=.., or the ID is not recognized yet). In the case when --with-distro is not set, keep doing the autodetection, but using os-release only, i.e. dropping the checks for all the other -release files -- since there is --with-distro, older distros with no os-release can still be used. --- m4/guestfs_appliance.m4 | 44 +++++++++++++++++++------------------------- 1 file changed, 19 insertions(+), 25 deletions(-) diff --git a/m4/guestfs_appliance.m4 b/m4/guestfs_appliance.m4 index fbba3373f..4993f57a1 100644 --- a/m4/guestfs_appliance.m4 +++ b/m4/guestfs_appliance.m4 @@ -94,31 +94,25 @@ dnl names vary slightly across distros. (See dnl appliance/packagelist.in, appliance/excludefiles.in, dnl appliance/hostfiles.in) AC_MSG_CHECKING([which Linux distro for package names]) -if test -f /etc/os-release; then - ( . /etc/os-release && echo $ID | tr '@<:@:lower:@:>@' '@<:@:upper:@:>@' ) >&AS_MESSAGE_LOG_FD - DISTRO="`. /etc/os-release && echo $ID | tr '@<:@:lower:@:>@' '@<:@:upper:@:>@'`" - AS_CASE([$DISTRO], - [FEDORA | RHEL | CENTOS],[DISTRO=REDHAT], - [OPENSUSE | SLED | SLES],[DISTRO=SUSE], - [ARCH],[DISTRO=ARCHLINUX]) -elif test -f /etc/debian_version; then - DISTRO=DEBIAN - if grep -q 'DISTRIB_ID=Ubuntu' /etc/lsb-release 2>&AS_MESSAGE_LOG_FD; then - DISTRO=UBUNTU - fi -elif test -f /etc/arch-release; then - DISTRO=ARCHLINUX -elif test -f /etc/SuSE-release; then - DISTRO=SUSE -elif test -f /etc/frugalware-release; then - DISTRO=FRUGALWARE -elif test -f /etc/mageia-release; then - DISTRO=MAGEIA -else -dnl fallback option - DISTRO=REDHAT -fi -AC_MSG_RESULT([$DISTRO]) +AC_ARG_WITH([distro], + [AS_HELP_STRING([--with-distro="DISTRO_ID"], + [distro ID @<:@default=ID in /etc/os-release@:>@])],[ + DISTRO="$withval" + AC_MSG_RESULT([$DISTRO (manually specified)]) + ],[ + if test -f /etc/os-release; then + ( . /etc/os-release && echo $ID | tr '@<:@:lower:@:>@' '@<:@:upper:@:>@' ) >&AS_MESSAGE_LOG_FD + DISTRO="`. /etc/os-release && echo $ID | tr '@<:@:lower:@:>@' '@<:@:upper:@:>@'`" + AS_CASE([$DISTRO], + [FEDORA | RHEL | CENTOS],[DISTRO=REDHAT], + [OPENSUSE | SLED | SLES],[DISTRO=SUSE], + [ARCH],[DISTRO=ARCHLINUX]) + AC_MSG_RESULT([$DISTRO (from /etc/os-release)]) + else + AC_MSG_ERROR([/etc/os-release not available, please specify the distro using --with-distro=DISTRO]) + fi + ] +) AC_SUBST([DISTRO]) dnl Add extra packages to the appliance. -- 2.13.5

2 2