November 2017 - Libguestfs - libguestfs List Archives

[PATCH v2] launch: add support for autodetection of appliance image format
by Pavel Butsykin 06 Feb '20

06 Feb '20

This feature allows you to use different image formats for the fixed appliance. The raw format is used by default. Signed-off-by: Pavel Butsykin <pbutsykin(a)virtuozzo.com> --- lib/launch-direct.c | 2 ++ lib/launch-libvirt.c | 19 ++++++++++++------- m4/guestfs_appliance.m4 | 11 +++++++++++ 3 files changed, 25 insertions(+), 7 deletions(-) diff --git a/lib/launch-direct.c b/lib/launch-direct.c index 0be662e25..b9b54857a 100644 --- a/lib/launch-direct.c +++ b/lib/launch-direct.c @@ -592,7 +592,9 @@ launch_direct (guestfs_h *g, void *datav, const char *arg) append_list ("id=appliance"); append_list ("cache=unsafe"); append_list ("if=none"); +#ifndef APPLIANCE_FMT_AUTO append_list ("format=raw"); +#endif } end_list (); start_list ("-device") { append_list ("scsi-hd"); diff --git a/lib/launch-libvirt.c b/lib/launch-libvirt.c index 4adb2cfb3..030ea6911 100644 --- a/lib/launch-libvirt.c +++ b/lib/launch-libvirt.c @@ -212,9 +212,10 @@ get_source_format_or_autodetect (guestfs_h *g, struct drive *drv) /** * Create a qcow2 format overlay, with the given C<backing_drive> - * (file). The C<format> parameter, which must be non-NULL, is the - * backing file format. This is used to create the appliance overlay, - * and also for read-only drives. + * (file). The C<format> parameter is the backing file format. + * The C<format> parameter can be NULL, in this case the backing + * format will be determined automatically. This is used to create + * the appliance overlay, and also for read-only drives. */ static char * make_qcow2_overlay (guestfs_h *g, const char *backing_drive, @@ -223,8 +224,6 @@ make_qcow2_overlay (guestfs_h *g, const char *backing_drive, char *overlay; struct guestfs_disk_create_argv optargs; - assert (format != NULL); - if (guestfs_int_lazy_make_tmpdir (g) == -1) return NULL; @@ -232,8 +231,10 @@ make_qcow2_overlay (guestfs_h *g, const char *backing_drive, optargs.bitmask = GUESTFS_DISK_CREATE_BACKINGFILE_BITMASK; optargs.backingfile = backing_drive; - optargs.bitmask |= GUESTFS_DISK_CREATE_BACKINGFORMAT_BITMASK; - optargs.backingformat = format; + if (format) { + optargs.bitmask |= GUESTFS_DISK_CREATE_BACKINGFORMAT_BITMASK; + optargs.backingformat = format; + } if (guestfs_disk_create_argv (g, overlay, "qcow2", -1, &optargs) == -1) { free (overlay); @@ -461,7 +462,11 @@ launch_libvirt (guestfs_h *g, void *datav, const char *libvirt_uri) /* Note that appliance can be NULL if using the old-style appliance. */ if (appliance) { +#ifdef APPLIANCE_FMT_AUTO + params.appliance_overlay = make_qcow2_overlay (g, appliance, NULL); +#else params.appliance_overlay = make_qcow2_overlay (g, appliance, "raw"); +#endif if (!params.appliance_overlay) goto cleanup; } diff --git a/m4/guestfs_appliance.m4 b/m4/guestfs_appliance.m4 index 81c43879f..4e1ec8135 100644 --- a/m4/guestfs_appliance.m4 +++ b/m4/guestfs_appliance.m4 @@ -139,3 +139,14 @@ AC_SUBST([GUESTFS_DEFAULT_PATH]) AC_DEFINE_UNQUOTED([GUESTFS_DEFAULT_PATH], ["$GUESTFS_DEFAULT_PATH"], [Define guestfs default path.]) + +AC_ARG_ENABLE([appliance-fmt-auto], + [AS_HELP_STRING([--enable-appliance-fmt-auto], + [enable autodetection of appliance image format @<:@default=no@:>@])], + [ENABLE_APPLIANCE_FMT_AUTO="$enableval"], + [ENABLE_APPLIANCE_FMT_AUTO=no]) + +if test "x$ENABLE_APPLIANCE_FMT_AUTO" = "xyes"; then + AC_DEFINE([APPLIANCE_FMT_AUTO], [1], + [Define to 1 if enabled autodetection of appliance image format.]) +fi -- 2.13.0

2 4

failure to virt-sysprep (FC27?)
by Yaniv Kaul 14 Dec '17

14 Dec '17

Since I upgrading to FC27, I *sometimes* fail to virt-sysprep. The debug messages: libguestfs: trace: set_verbose true libguestfs: trace: set_verbose = 0 libguestfs: create: flags = 0, handle = 0x7f4600005dd0, program = python2 libguestfs: trace: set_program "lago" libguestfs: trace: set_program = 0 libguestfs: trace: add_drive_ro "/home/ykaul/ovirt-system-tests/deployment-basic-suite-master/default/images/lago-basic-suite-master-host-0_root.qcow2" libguestfs: trace: add_drive "/home/ykaul/ovirt-system-tests/deployment-basic-suite-master/default/images/lago-basic-suite-master-host-0_root.qcow2" "readonly:true" libguestfs: creating COW overlay to protect original drive content libguestfs: trace: disk_format "/home/ykaul/ovirt-system-tests/deployment-basic-suite-master/default/images/lago-basic-suite-master-host-0_root.qcow2" libguestfs: command: run: qemu-img libguestfs: command: run: \ info libguestfs: command: run: \ --output json libguestfs: command: run: \ /dev/fd/7 qemu-img: Could not open '/dev/fd/7': Failed to get shared "write" lock Is another process using the image? libguestfs: trace: disk_format = NULL (error) libguestfs: trace: add_drive = -1 (error) libguestfs: trace: add_drive_ro = -1 (error) Any ideas? TIA, Y.

3 11

[PATCH] builder: use the template arch when caching all templates
by Pino Toscano 07 Dec '17

07 Dec '17

When caching all the templates, use the architecture of each template, instead of the architecture passed as --arch (or the host architecture, as default). This way, the right destination filename will be used. Fixes commit b1cf6246f3c80762cf27dbdb24168589a34daf00. Thanks to: Erik Skultety. --- builder/Makefile.am | 2 + builder/builder.ml | 4 +- builder/test-virt-builder-cacheall.sh | 88 +++++++++++++++++++++++++++++++++++ 3 files changed, 92 insertions(+), 2 deletions(-) create mode 100755 builder/test-virt-builder-cacheall.sh diff --git a/builder/Makefile.am b/builder/Makefile.am index 979726bff..4be9bd617 100644 --- a/builder/Makefile.am +++ b/builder/Makefile.am @@ -32,6 +32,7 @@ EXTRA_DIST = \ test-simplestreams/streams/v1/net.cirros-cloud_released_download.json \ test-virt-builder.sh \ test-docs.sh \ + test-virt-builder-cacheall.sh \ test-virt-builder-list.sh \ test-virt-builder-list-simplestreams.sh \ test-virt-builder-planner.sh \ @@ -386,6 +387,7 @@ index_parser_tests_LINK = \ TESTS = \ test-docs.sh \ + test-virt-builder-cacheall.sh \ test-virt-builder-list.sh \ test-virt-index-validate.sh \ $(SLOW_TESTS) diff --git a/builder/builder.ml b/builder/builder.ml index 41c0a4ccc..c84352fec 100644 --- a/builder/builder.ml +++ b/builder/builder.ml @@ -251,8 +251,8 @@ let main () = | Some _ -> List.iter ( fun (name, - { Index.revision; file_uri; proxy }) -> - let template = name, Index.Arch cmdline.arch, revision in + { Index.revision; file_uri; proxy; arch }) -> + let template = name, arch, revision in message (f_"Downloading: %s") file_uri; let progress_bar = not (quiet ()) in ignore (Downloader.download downloader ~template ~progress_bar diff --git a/builder/test-virt-builder-cacheall.sh b/builder/test-virt-builder-cacheall.sh new file mode 100755 index 000000000..c80d9ecd2 --- /dev/null +++ b/builder/test-virt-builder-cacheall.sh @@ -0,0 +1,88 @@ +#!/bin/bash - +# libguestfs virt-builder test script +# Copyright (C) 2017 Red Hat Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + +set -e + +$TEST_FUNCTIONS +skip_if_skipped + +tmpdir="$(mktemp -d)" +echo "tmpdir= $tmpdir" +reposdir="$tmpdir/virt-builder/repos.d" +repodir="$tmpdir/repo" +indexfile="$repodir/index" +cachedir="$tmpdir/cache" + +mkdir -p "$reposdir" +mkdir -p "$repodir" + +# Create some fake images. +img1_path="$repodir/img1.raw" +img1_size=10485760 # 10G +qemu-img create -f raw "$img1_path" $img1_size +img1_csum=`do_sha256 "$img1_path"` + +img2_path="$repodir/img2.qcow2" +img2_size=5242880 # 5G +qemu-img create -f qcow2 "$img2_path" $img2_size +img2_csum=`do_sha256 "$img2_path"` + +# Create an index for the images. +cat > "$indexfile" <<EOF +[img1] +name=img1 +file=$(basename "$img1_path") +arch=x86_64 +size=$img1_size +checksum[sha512]=$img1_csum +revision=1 + +[img2] +name=img2 +file=$(basename "$img2_path") +arch=aarch64 +size=$img2_size +checksum[sha512]=$img2_csum +revision=3 +EOF + +# Create the repository. +cat > "$reposdir/repo1.conf" <<EOF +[repo1] +uri=$indexfile +EOF + +export XDG_CONFIG_HOME= +export XDG_CONFIG_DIRS="$tmpdir" +export XDG_CACHE_HOME="$cachedir" + +short_list=$($VG virt-builder --no-check-signature --no-cache --list) + +if [ "$short_list" != "img1 x86_64 img1 +img2 aarch64 img2" ]; then + echo "$0: unexpected --list output:" + echo "$short_list" + exit 1 +fi + +$VG virt-builder --no-check-signature --cache-all-templates +ls -lh "$cachedir/virt-builder" +test -f "$cachedir/virt-builder/img1.x86_64.1" +test -f "$cachedir/virt-builder/img2.aarch64.3" + +rm -rf "$tmpdir" -- 2.14.3

2 1

virt-customize kernel question
by ANDRES POZO MUNOZ 29 Nov '17

29 Nov '17

Hi, I’m facing some problem trying to install some software in an image with virt-customize. The package I’m trying to install uses some script to check dependencies and verifies the linux-headers-4.xxxxx are installed in the guest system. The problem is that it uses the ‘uname –r’ command to obtain the kernel version, and that command seems to be replying with the host kernel version instead the guest one. Could that be the scenario? When the image is booted, the default kernel used is the host’s? Is there any option (via config or something like that) to boot the image kernel so ‘uname –r’ can provide the right info? (both host and guest are the same architecture). Thanks in advance! Regards, Andrés ________________________________ Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener información privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilización, divulgación y/o copia sin autorización puede estar prohibida en virtud de la legislación vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma vía y proceda a su destrucción. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinatário, pode conter informação privilegiada ou confidencial e é para uso exclusivo da pessoa ou entidade de destino. Se não é vossa senhoria o destinatário indicado, fica notificado de que a leitura, utilização, divulgação e/ou cópia sem autorização pode estar proibida em virtude da legislação vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destruição

2 1

virt-customize networking question
by ANDRES POZO MUNOZ 24 Nov '17

24 Nov '17

Hi, A quick question about a problem I’m facing. I want to customize some image running a customization script with virt-customize. Everything works fine, except the fact that I don’t have a DHCP server in the network (virt-customize relies on DHCP to assign the IP when it boots the image, right?) and all the commands accessing the network are failing. Is there any option (via config or something like that) to configure an static IP in the image? Thanks in advance! Regards, Andrés ________________________________ Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener información privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilización, divulgación y/o copia sin autorización puede estar prohibida en virtud de la legislación vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma vía y proceda a su destrucción. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinatário, pode conter informação privilegiada ou confidencial e é para uso exclusivo da pessoa ou entidade de destino. Se não é vossa senhoria o destinatário indicado, fica notificado de que a leitura, utilização, divulgação e/ou cópia sem autorização pode estar proibida em virtude da legislação vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destruição

2 5

[PATCH 0/1] RFC: switch from YAJL to Jansson
by Pino Toscano 23 Nov '17

23 Nov '17

Hi, recently, there was a discussion in the development list of libvirt on switching to a different JSON library than YAJL [1]. Since we use YAJL, and the points there IMHO apply to libguestfs as well, I decided to give a try in switching to Jansson [2]. The result IMHO is nice, with the additional APIs of Jansson that simplify some of our code. Unlike with YAJL, I did not set a minimum required version, as I did not use any feature in newer versions (simply because there was no need for them), so even older versions should work. Of course the test suite works for me, although it does not cover the LDM APIs. What do you think? [1] https://www.redhat.com/archives/libvir-list/2017-November/msg00225.html [2] http://www.digip.org/jansson/ Pino Toscano (1): Switch from YAJL to Jansson appliance/packagelist.in | 13 ++-- builder/Makefile.am | 4 +- builder/yajl-c.c | 66 ++++++++++----------- builder/yajl_tests.ml | 4 +- contrib/p2v/aux-scripts/do-build.sh | 8 +-- contrib/p2v/build-p2v-iso.sh | 3 +- daemon/Makefile.am | 4 +- daemon/ldm.c | 115 ++++++++++++++++-------------------- docs/guestfs-building.pod | 2 +- lib/Makefile.am | 4 +- lib/info.c | 113 ++++++++++++++--------------------- lib/qemu.c | 64 +++++++------------- m4/guestfs-libraries.m4 | 4 +- 13 files changed, 170 insertions(+), 234 deletions(-) -- 2.13.6

4 6

RFE: ctime byte-for-byte reproducible qcow2 ext2/3/4 FS
by Marc Pawlowsky 23 Nov '17

23 Nov '17

Problem: Want to be able to produce a qcow2 file with multiple ext4 File Systems. Days later want to reproduce the production of the qcow2 and have the exact same byte-for-byte file, to prove my build is reproducible. Currently the ctime attributes of the inodes will differ and thus the qcow2 files will differ. Since the file times are subsecond the trick of setting the system time and chmoding the files will not work. In https://stackoverflow.com/questions/47440618/changing-qcow2-st-ctime I mention how debugfs can be used to change the ctimes. Two possible usages come to mind: 1. A global flag that would indicate the ctime that should be used for all file operations, e.g. a fake system clock. 2. A new command to specifically change the ctime, like debugfs's set_inode_field. Thanks Marc

2 1

[PATCH] launch: direct: Omit locking option for non-file disks (RHBZ#1516094)
by Lars Seipel 23 Nov '17

23 Nov '17

QEMU does not accept options unrecognized by the block driver in use. Disable locking only for read-only disks that are file-backed, as that's the only block driver it is supported with. Signed-off-by: Lars Seipel <ls(a)slrz.net> --- lib/launch-direct.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/lib/launch-direct.c b/lib/launch-direct.c index 87ac121c7558..678c8c68d59d 100644 --- a/lib/launch-direct.c +++ b/lib/launch-direct.c @@ -260,7 +260,11 @@ add_drive_standard_params (guestfs_h *g, struct backend_direct_data *data, append_list ("cache=unsafe"); if (drv->disk_label) append_list_format ("serial=%s", drv->disk_label); - if (data->qemu_mandatory_locking) + + /* Add the file-specific locking option only for files, as qemu + * won't accept options unknown to the block driver in use. + */ + if (data->qemu_mandatory_locking && drv->src.protocol == drive_protocol_file) append_list ("file.backing.file.locking=off"); } -- 2.14.3

1 0

[nbdkit PATCH 0/2] more nbd tweaks
by Eric Blake 22 Nov '17

22 Nov '17

I tried reproducing the testsuite failure on test-parallel-*.sh using the same machine Rich posted a log from, but did not quickly hit it on a loop of make -j20 check TESTS=test-parallel-{file,nbd}.sh But I still think this series can't hurt. Eric Blake (2): nbd: Don't advertise writes if nbdkit is readonly tests: Make parallel tests more robust plugins/nbd/nbd.c | 2 ++ tests/test-parallel-file.sh | 30 +++++++++++++++--------------- tests/test-parallel-nbd.sh | 30 +++++++++++++++--------------- 3 files changed, 32 insertions(+), 30 deletions(-) -- 2.13.6

2 3

[nbdkit PATCH] file: Diagnose a missing file earlier
by Eric Blake 22 Nov '17

22 Nov '17

If a user typos the filename for the file plugin, they do not find out about it until later when the first client gets an abrupt hangup, when file_open() fails to open things. Better is to avoid starting a server at all if the file doesn't exist yet. While at it, avoid a memory leak if the caller passes a file= argument more than once. Signed-off-by: Eric Blake <eblake(a)redhat.com> --- plugins/file/file.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/plugins/file/file.c b/plugins/file/file.c index ef5da3d..4a91251 100644 --- a/plugins/file/file.c +++ b/plugins/file/file.c @@ -117,6 +117,7 @@ file_config (const char *key, const char *value) { if (strcmp (key, "file") == 0) { /* See FILENAMES AND PATHS in nbdkit-plugin(3). */ + free (filename); filename = nbdkit_absolute_path (value); if (!filename) return -1; @@ -147,6 +148,10 @@ file_config_complete (void) nbdkit_error ("you must supply the file=<FILENAME> parameter after the plugin name on the command line"); return -1; } + if (access (filename, F_OK) < 0) { + nbdkit_error ("access '%s': %m", filename); + return -1; + } return 0; } -- 2.13.6

2 1