March 2018 - Libguestfs - libguestfs List Archives

[PATCH v2] launch: add support for autodetection of appliance image format
by Pavel Butsykin 06 Feb '20

06 Feb '20

This feature allows you to use different image formats for the fixed appliance. The raw format is used by default. Signed-off-by: Pavel Butsykin <pbutsykin(a)virtuozzo.com> --- lib/launch-direct.c | 2 ++ lib/launch-libvirt.c | 19 ++++++++++++------- m4/guestfs_appliance.m4 | 11 +++++++++++ 3 files changed, 25 insertions(+), 7 deletions(-) diff --git a/lib/launch-direct.c b/lib/launch-direct.c index 0be662e25..b9b54857a 100644 --- a/lib/launch-direct.c +++ b/lib/launch-direct.c @@ -592,7 +592,9 @@ launch_direct (guestfs_h *g, void *datav, const char *arg) append_list ("id=appliance"); append_list ("cache=unsafe"); append_list ("if=none"); +#ifndef APPLIANCE_FMT_AUTO append_list ("format=raw"); +#endif } end_list (); start_list ("-device") { append_list ("scsi-hd"); diff --git a/lib/launch-libvirt.c b/lib/launch-libvirt.c index 4adb2cfb3..030ea6911 100644 --- a/lib/launch-libvirt.c +++ b/lib/launch-libvirt.c @@ -212,9 +212,10 @@ get_source_format_or_autodetect (guestfs_h *g, struct drive *drv) /** * Create a qcow2 format overlay, with the given C<backing_drive> - * (file). The C<format> parameter, which must be non-NULL, is the - * backing file format. This is used to create the appliance overlay, - * and also for read-only drives. + * (file). The C<format> parameter is the backing file format. + * The C<format> parameter can be NULL, in this case the backing + * format will be determined automatically. This is used to create + * the appliance overlay, and also for read-only drives. */ static char * make_qcow2_overlay (guestfs_h *g, const char *backing_drive, @@ -223,8 +224,6 @@ make_qcow2_overlay (guestfs_h *g, const char *backing_drive, char *overlay; struct guestfs_disk_create_argv optargs; - assert (format != NULL); - if (guestfs_int_lazy_make_tmpdir (g) == -1) return NULL; @@ -232,8 +231,10 @@ make_qcow2_overlay (guestfs_h *g, const char *backing_drive, optargs.bitmask = GUESTFS_DISK_CREATE_BACKINGFILE_BITMASK; optargs.backingfile = backing_drive; - optargs.bitmask |= GUESTFS_DISK_CREATE_BACKINGFORMAT_BITMASK; - optargs.backingformat = format; + if (format) { + optargs.bitmask |= GUESTFS_DISK_CREATE_BACKINGFORMAT_BITMASK; + optargs.backingformat = format; + } if (guestfs_disk_create_argv (g, overlay, "qcow2", -1, &optargs) == -1) { free (overlay); @@ -461,7 +462,11 @@ launch_libvirt (guestfs_h *g, void *datav, const char *libvirt_uri) /* Note that appliance can be NULL if using the old-style appliance. */ if (appliance) { +#ifdef APPLIANCE_FMT_AUTO + params.appliance_overlay = make_qcow2_overlay (g, appliance, NULL); +#else params.appliance_overlay = make_qcow2_overlay (g, appliance, "raw"); +#endif if (!params.appliance_overlay) goto cleanup; } diff --git a/m4/guestfs_appliance.m4 b/m4/guestfs_appliance.m4 index 81c43879f..4e1ec8135 100644 --- a/m4/guestfs_appliance.m4 +++ b/m4/guestfs_appliance.m4 @@ -139,3 +139,14 @@ AC_SUBST([GUESTFS_DEFAULT_PATH]) AC_DEFINE_UNQUOTED([GUESTFS_DEFAULT_PATH], ["$GUESTFS_DEFAULT_PATH"], [Define guestfs default path.]) + +AC_ARG_ENABLE([appliance-fmt-auto], + [AS_HELP_STRING([--enable-appliance-fmt-auto], + [enable autodetection of appliance image format @<:@default=no@:>@])], + [ENABLE_APPLIANCE_FMT_AUTO="$enableval"], + [ENABLE_APPLIANCE_FMT_AUTO=no]) + +if test "x$ENABLE_APPLIANCE_FMT_AUTO" = "xyes"; then + AC_DEFINE([APPLIANCE_FMT_AUTO], [1], + [Define to 1 if enabled autodetection of appliance image format.]) +fi -- 2.13.0

2 4

1.39 proposal: Let's split up the libguestfs git repo and tarballs
by Richard W.M. Jones 29 Nov '19

29 Nov '19

My contention is that the libguestfs git repository is too large and unwieldy. There are too many separate, unrelated projects and as a result of that the source has too many dependencies and takes too long to build and test. The project divides (sort of) naturally into layers -- the library, the bindings, the various virt tools -- and could be split along those lines into separate projects which can then be released and evolve at their own pace. My suggested split would be something like this: * libguestfs: The library, daemon and appliance. That would include the following directories in a single project: appliance bash contrib daemon docs examples gnulib lib logo test-tool tmp utils website * 1 project for each language binding: csharp erlang gobject golang haskell java lua ocaml php perl python ruby * virt-customize and related tools, we'd probably call this subproject "virt-builder". It would include virt-builder, virt-customize and virt-sysprep, since they share a lot of common code. * 1 project for each of the following items: small tools written in C (virt-cat, virt-filesystems, virt-log, virt-ls, virt-tail, virt-diff, virt-edit, virt-format, guestmount, virt-inspector, virt-make-fs, virt-rescue) guestfish virt-alignment-scan and virt-df virt-dib virt-get-kernel virt-resize virt-sparsify virt-v2v and virt-p2v virt-win-reg * I'd be inclined to drop the legacy Perl tools virt-tar, virt-list-filesystems, virt-list-partitions unless someone especially wished to step forward to maintain them. * common code and generator: Off to the side we'd somehow need to package up the common code and the generator for use by all of the above projects. It wouldn't be a separate project for downstream packagers, but instead the code would be included (ie. duplicated) in tarballs and upstream available as a side git repo that you'd need to include when building (git submodule?). This is somewhat unspecified. M4, PO, and tests would be split between the projects as appropriate. My proposal would be to do this incrementally, rather than all at once, moving the easier things out first. Thoughts? Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org

4 18

[nbdkit PATCH v3 00/15] Add FUA support to nbdkit
by Eric Blake 11 Apr '18

11 Apr '18

After more than a month since v2 [1], I've finally got my FUA support series polished. This is all of my outstanding patches, even though some of them were originally posted in separate threads from the original FUA post [2], [3] [1] https://www.redhat.com/archives/libguestfs/2018-January/msg00113.html [2] https://www.redhat.com/archives/libguestfs/2018-January/msg00219.html [3] https://www.redhat.com/archives/libguestfs/2018-February/msg00000.html Still to go: figure out how we want to expose flags through the language bindings (there, we can break API if needed, but hopefully we can instead exploit languages with function-overloading and/or optional parameters to make it feel like a natural extension). This exercise has been good; I've found a couple of tweaks needed in qemu for corner cases explored while writing these nbdkit patches. Also, the qemu list reminded me that even though the NBD spec says FUA on trim is required to wait until the trim effects have hit the disk, no sane client will ever send trim+FUA because trim is advisory, and the client has no sane way to tell if trim had an effect in the first place. It feels pretty much like a rewrite, according to: $ git backport-diff -u fua-v2 -r origin.. Key: [----] : patches are identical [####] : number of functional differences between upstream/downstream patch [down] : patch is downstream-only The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively 001/15:[down] 'src: Let internal.h handle common includes' 002/15:[down] 'backend: Rework internal error return semantics' 003/15:[down] 'filters: Adjust callback API for flags/errors' 004/15:[0133] [FC] 'filters: Add log filter' 005/15:[0157] [FC] 'filters: Add blocksize filter' 006/15:[down] 'backend: Add .can_zero/.can_fua helpers' 007/15:[down] 'filters: Expose new .can_zero callback' 008/15:[0125] [FC] 'filters: Add nozero filter' 009/15:[down] 'filters: Expose new .can_fua callback' 010/15:[down] 'filters: Add fua filter' 011/15:[down] 'plugins: Expose new FUA callbacks' 012/15:[0043] [FC] 'nbd: Wire up FUA flag passthrough' 013/15:[down] 'null: Wire up FUA flag support' 014/15:[down] 'todo: Mention possibility of caching .can_FOO callbacks' 015/15:[0130] [FC] 'RFC: plugins: Add back-compat for new plugin with old nbdkit' Eric Blake (15): src: Let internal.h handle common includes backend: Rework internal error return semantics filters: Adjust callback API for flags/errors filters: Add log filter filters: Add blocksize filter backend: Add .can_zero/.can_fua helpers filters: Expose new .can_zero callback filters: Add nozero filter filters: Expose new .can_fua callback filters: Add fua filter plugins: Expose new FUA callbacks nbd: Wire up FUA flag passthrough null: Wire up FUA flag support todo: Mention possibility of caching .can_FOO callbacks RFC: plugins: Add back-compat for new plugin with old nbdkit TODO | 22 +- docs/nbdkit-filter.pod | 173 ++++++++++-- docs/nbdkit-plugin.pod | 151 +++++++++-- docs/nbdkit.pod | 9 +- filters/blocksize/nbdkit-blocksize-filter.pod | 141 ++++++++++ filters/fua/nbdkit-fua-filter.pod | 119 +++++++++ filters/log/nbdkit-log-filter.pod | 115 ++++++++ filters/nozero/nbdkit-nozero-filter.pod | 99 +++++++ configure.ac | 6 +- include/nbdkit-common.h | 7 + include/nbdkit-filter.h | 36 ++- include/nbdkit-plugin.h | 89 ++++++- src/internal.h | 24 +- src/cleanup.c | 1 - src/connections.c | 71 +++-- src/errors.c | 1 - src/filters.c | 137 ++++++---- src/main.c | 1 - src/plugins.c | 216 ++++++++++----- src/sockets.c | 1 - src/threadlocal.c | 1 - src/utils.c | 1 - plugins/nbd/nbd.c | 42 ++- plugins/null/null.c | 42 ++- filters/blocksize/blocksize.c | 370 ++++++++++++++++++++++++++ filters/cache/cache.c | 87 ++++-- filters/cow/cow.c | 55 +++- filters/delay/delay.c | 15 +- filters/fua/fua.c | 251 +++++++++++++++++ filters/log/log.c | 366 +++++++++++++++++++++++++ filters/nozero/nozero.c | 106 ++++++++ filters/offset/offset.c | 20 +- filters/partition/partition.c | 26 +- filters/Makefile.am | 4 + filters/blocksize/Makefile.am | 62 +++++ filters/fua/Makefile.am | 62 +++++ filters/log/Makefile.am | 62 +++++ filters/nozero/Makefile.am | 62 +++++ tests/Makefile.am | 16 ++ tests/test-blocksize.sh | 156 +++++++++++ tests/test-fua.sh | 153 +++++++++++ tests/test-log.sh | 88 ++++++ tests/test-nozero.sh | 145 ++++++++++ 43 files changed, 3318 insertions(+), 293 deletions(-) create mode 100644 filters/blocksize/nbdkit-blocksize-filter.pod create mode 100644 filters/fua/nbdkit-fua-filter.pod create mode 100644 filters/log/nbdkit-log-filter.pod create mode 100644 filters/nozero/nbdkit-nozero-filter.pod create mode 100644 filters/blocksize/blocksize.c create mode 100644 filters/fua/fua.c create mode 100644 filters/log/log.c create mode 100644 filters/nozero/nozero.c create mode 100644 filters/blocksize/Makefile.am create mode 100644 filters/fua/Makefile.am create mode 100644 filters/log/Makefile.am create mode 100644 filters/nozero/Makefile.am create mode 100755 tests/test-blocksize.sh create mode 100755 tests/test-fua.sh create mode 100755 tests/test-log.sh create mode 100755 tests/test-nozero.sh -- 2.14.3

2 23

[PATCH] virt-builder.pod: Update Fedora versions
by Kashyap Chamarthy 10 Apr '18

10 Apr '18

This is just a mechanincal change, so that the public documentation[*] refers to the latest release Fedora versions, instead of the EOLed versions. While at it, also update the `virt-cutomize` Makefile.am [*] http://libguestfs.org/virt-builder.1.html Signed-off-by: Kashyap Chamarthy <kchamart(a)redhat.com> --- Maybe I missed more places. (Me wonders if upstream libguestfs uses a more systematic approach to do this "purge" of EOLed distributions from the source, where applicable.) --- builder/virt-builder.pod | 54 ++++++++++++++++++++++++------------------------ customize/Makefile.am | 6 +++--- 2 files changed, 30 insertions(+), 30 deletions(-) diff --git a/builder/virt-builder.pod b/builder/virt-builder.pod index 1ed18a7c7..c82a08b4d 100644 --- a/builder/virt-builder.pod +++ b/builder/virt-builder.pod @@ -59,11 +59,11 @@ your own too (see below). After choosing a guest from the list, you may want to see if there are any installation notes: - virt-builder --notes fedora-25 + virt-builder --notes fedora-27 =head2 Build a virtual machine - virt-builder fedora-25 + virt-builder fedora-27 will build a Fedora 25 image for the same architecture as virt-builder (so running it from an i686 installation will try to build an i686 @@ -77,31 +77,31 @@ The first time this runs it has to download the template over the network, but this gets cached (see L</CACHING>). The name of the output file is derived from the template name, so -above it will be F<fedora-25.img>. You can change the output filename +above it will be F<fedora-27.img>. You can change the output filename using the I<-o> option: - virt-builder fedora-25 -o mydisk.img + virt-builder fedora-27 -o mydisk.img You can also use the I<-o> option to write to existing devices or logical volumes. - virt-builder fedora-25 --format qcow2 + virt-builder fedora-27 --format qcow2 -As above, but write the output in qcow2 format to F<fedora-25.qcow2>. +As above, but write the output in qcow2 format to F<fedora-27.qcow2>. - virt-builder fedora-25 --size 20G + virt-builder fedora-27 --size 20G As above, but the output size will be 20 GB. The guest OS is resized as it is copied to the output (automatically, using L<virt-resize(1)>). - virt-builder fedora-25 --arch i686 + virt-builder fedora-27 --arch i686 As above, but using an i686 template, if available. =head2 Setting the root password - virt-builder fedora-25 --root-password file:/tmp/rootpw + virt-builder fedora-27 --root-password file:/tmp/rootpw Create a Fedora 25 image. The root password is taken from the file F</tmp/rootpw>. @@ -113,7 +113,7 @@ You can also create user accounts. See L</USERS AND PASSWORDS> below. =head2 Set the hostname - virt-builder fedora-25 --hostname virt.example.com + virt-builder fedora-27 --hostname virt.example.com Set the hostname to C<virt.example.com>. @@ -122,7 +122,7 @@ Set the hostname to C<virt.example.com>. To install packages from the ordinary (guest) software repository (eg. dnf or apt): - virt-builder fedora-25 --install "inkscape,@Xfce Desktop" + virt-builder fedora-27 --install "inkscape,@Xfce Desktop" (In Fedora, C<@> is used to install groups of packages. On Debian you would install a meta-package instead.) @@ -135,7 +135,7 @@ For guests which use SELinux, like Fedora and Red Hat Enterprise Linux, you may need to do SELinux relabelling after installing or updating packages (see L</SELINUX> below): - virt-builder fedora-25 --update --selinux-relabel + virt-builder fedora-27 --update --selinux-relabel =head2 Customizing the installation @@ -153,18 +153,18 @@ For example: dnf -y --best update EOF - virt-builder fedora-25 --firstboot /tmp/dnf-update.sh + virt-builder fedora-27 --firstboot /tmp/dnf-update.sh or simply: - virt-builder fedora-25 --firstboot-command 'dnf -y --best update' + virt-builder fedora-27 --firstboot-command 'dnf -y --best update' which makes the L<dnf(8)> C<update> command run once the first time the guest boots. Or: - virt-builder fedora-25 \ + virt-builder fedora-27 \ --edit '/etc/dnf/dnf.conf: s/gpgcheck=1/gpgcheck=0/' @@ -557,7 +557,7 @@ If the guest OS you are installing is similar to the host OS (eg. both are Linux), and if libguestfs supports network connections, then you can use I<--install> to install packages like this: - virt-builder fedora-25 --install inkscape + virt-builder fedora-27 --install inkscape This uses the guest��s package manager and the host��s network connection. @@ -566,7 +566,7 @@ connection. To update the installed packages in the template at build time: - virt-builder fedora-25 --update + virt-builder fedora-27 --update Most of the templates that ship with virt-builder come with a very minimal selection of packages (known as a "JEOS" or "Just Enough @@ -578,7 +578,7 @@ OS from the template. This option updates those template packages. Another option is to install the packages when the guest first boots: - virt-builder fedora-25 --firstboot-install inkscape + virt-builder fedora-27 --firstboot-install inkscape This uses the guest��s package manager and the guest��s network connection. @@ -626,7 +626,7 @@ For apt, create /tmp/install.sh containing: Use the I<--attach> option to attach the CD / disk image and the I<--run> option to run the script: - virt-builder fedora-25 \ + virt-builder fedora-27 \ --attach extra-packages.iso \ --run /tmp/install.sh @@ -705,7 +705,7 @@ keyboard for some common Linux distributions. For distros that use systemd C<localectl>, use a command like this: - virt-builder fedora-25 \ + virt-builder fedora-27 \ --firstboot-command 'localectl set-keymap uk' See L<localectl(1)> and @@ -749,7 +749,7 @@ This section contains examples for some common Linux distributions. =head3 Setting Japanese in Fedora 25 - virt-builder fedora-25 \ + virt-builder fedora-27 \ --size 20G \ --update \ --install @japanese-support \ @@ -977,7 +977,7 @@ I<--import> option. virt-install --import \ --name guest --ram 2048 \ - --disk path=disk.img,format=raw --os-variant fedora25 + --disk path=disk.img,format=raw --os-variant fedora27 Notes: @@ -1012,7 +1012,7 @@ tools probably work differently as well. Import the image into Glance (the OpenStack image store) by doing: - glance image-create --name fedora-25-image --file fedora-25.img \ + glance image-create --name fedora-27-image --file fedora-27.img \ --disk-format raw --container-format bare \ --is-public True @@ -1022,12 +1022,12 @@ parameter should match virt-builder��s I<--format> option (or C<raw> if you didn't use that option). The I<--container-format> should always be C<bare> since virt-builder doesn't put images into containers. -You can use the S<C<glance image-show I<fedora-25-image>>> command to +You can use the S<C<glance image-show I<fedora-27-image>>> command to display the properties of the image. To boot up an instance of your image on a Nova compute node, do: - nova boot fedora-25-server --image fedora-25-image \ + nova boot fedora-27-server --image fedora-27-image \ --flavor m1.medium Use S<C<nova flavor-list>> to list possible machine flavors. Use @@ -1058,7 +1058,7 @@ at boot. A typical virt-builder command would be: - virt-builder fedora-25 \ + virt-builder fedora-27 \ --hostname client.example.com \ --update \ --install puppet \ @@ -1583,7 +1583,7 @@ package manager at that. To install a Fedora guest using a local mirror: - virt-builder fedora-25 \ + virt-builder fedora-27 \ --edit '/etc/yum.repos.d/fedora.repo: s{.*baseurl=.*}{baseurl=http://example.com/mirror/}; s{.*metalink=.*}{}; diff --git a/customize/Makefile.am b/customize/Makefile.am index 5fa176341..0d1d5aa6c 100644 --- a/customize/Makefile.am +++ b/customize/Makefile.am @@ -237,8 +237,8 @@ firstboot_test_scripts := \ test-firstboot-debian-6.sh \ test-firstboot-debian-7.sh \ test-firstboot-debian-8.sh \ - test-firstboot-fedora-25.sh \ test-firstboot-fedora-26.sh \ + test-firstboot-fedora-27.sh \ test-firstboot-ubuntu-10.04.sh \ test-firstboot-ubuntu-12.04.sh \ test-firstboot-ubuntu-14.04.sh \ @@ -258,7 +258,7 @@ password_test_scripts := \ test-password-debian-6.sh \ test-password-debian-7.sh \ test-password-debian-8.sh \ - test-password-fedora-25.sh \ + test-password-fedora-27.sh \ test-password-rhel-3.9.sh \ test-password-rhel-4.9.sh \ test-password-rhel-5.11.sh \ @@ -284,7 +284,7 @@ settings_test_scripts := \ test-settings-debian-6.sh \ test-settings-debian-7.sh \ test-settings-debian-8.sh \ - test-settings-fedora-25.sh \ + test-settings-fedora-27.sh \ test-settings-ubuntu-10.04.sh \ test-settings-ubuntu-12.04.sh \ test-settings-ubuntu-14.04.sh \ -- 2.13.6

2 1

Re: [Libguestfs] Change in ovirt-imageio[master]: Document the random I/O APIs
by Daniel Erez 09 Apr '18

09 Apr '18

Hi Richard, We've added zero and flush functionality to imageio-daemon. You can download and test the latest build (for el7/fc) from: - http://jenkins.ovirt.org/job/ovirt-imageio_master_build-artifacts-el7-x86_6… - http://jenkins.ovirt.org/job/ovirt-imageio_master_build-artifacts-fc27-x86_… - http://jenkins.ovirt.org/job/ovirt-imageio_master_build-artifacts-fcraw-x86… The latest documentation and examples for Random I/O: - https://github.com/oVirt/ovirt-imageio/blob/master/docs/random-io.md - https://github.com/oVirt/ovirt-imageio/tree/master/examples Notes: - We plan to introduce the functionality also to the proxy soon. - We plan to implement trim later during ovirt 4.2 release: https://gerrit.ovirt.org/89470 - Additional pending patches: https://gerrit.ovirt.org/#/q/status:open+project:ovirt-imageio+branch:maste… Thanks! Nir & Daniel On Wed, Mar 28, 2018 at 12:39 PM Code Review <gerrit(a)ovirt.org> wrote: > gerrit-hooks *posted comments* on this change. > > View Change <https://gerrit.ovirt.org/89022> > > Patch set 7: > > - Set MODIFIED::IGNORE, not relevant for branch: master > - Update Tracker::#1560242::OK, tracker status: MERGED > > > > To view, visit change 89022 <https://gerrit.ovirt.org/89022>. To > unsubscribe, visit settings <https://gerrit.ovirt.org/settings>. > Gerrit-Project: ovirt-imageio > Gerrit-Branch: master > Gerrit-MessageType: comment > Gerrit-Change-Id: I98fbc20e1c05ab9fe627d33034034a47e1740da8 > Gerrit-Change-Number: 89022 > Gerrit-PatchSet: 7 > Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com> > Gerrit-Reviewer: Allon Mureinik <amureini(a)redhat.com> > Gerrit-Reviewer: Daniel Erez <derez(a)redhat.com> > Gerrit-Reviewer: Jenkins CI > Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com> > Gerrit-Reviewer: Roy Golan <rgolan(a)redhat.com> > Gerrit-Reviewer: Tomas Golembiovsky <tgolembi(a)redhat.com> > Gerrit-Reviewer: Yaniv Kaul <ykaul(a)redhat.com> > Gerrit-Reviewer: gerrit-hooks <automation(a)ovirt.org> > Gerrit-Comment-Date: Wed, 28 Mar 2018 09:39:47 +0000 > Gerrit-HasComments: No >

2 1

[nbdkit PATCH v2] plugin: add and use nbdkit_realpath
by Pino Toscano 06 Apr '18

06 Apr '18

Introduce a new helper function to resolve a path name, calling nbdkit_error on failure: other than doing what nbdkit_absolute_path does, it also checks that the file exists (and thus avoids errors later on). To help distinguish it from nbdkit_absolute_path, improve the documentation of the latter. Apply it where an existing path is required, both in nbdkit itself and in plugins. Related to: https://bugzilla.redhat.com/show_bug.cgi?id=1527334 --- docs/nbdkit-plugin.pod | 18 +++++++++++++++++- include/nbdkit-common.h | 1 + plugins/example2/example2.c | 2 +- plugins/file/file.c | 6 +----- plugins/gzip/gzip.c | 2 +- plugins/split/split.c | 2 +- plugins/vddk/vddk.c | 2 +- plugins/xz/xz.c | 2 +- src/plugins.c | 2 +- src/utils.c | 19 +++++++++++++++++++ 10 files changed, 44 insertions(+), 12 deletions(-) diff --git a/docs/nbdkit-plugin.pod b/docs/nbdkit-plugin.pod index 44822fc..5faba1d 100644 --- a/docs/nbdkit-plugin.pod +++ b/docs/nbdkit-plugin.pod @@ -200,13 +200,29 @@ descriptor. char *nbdkit_absolute_path (const char *filename); The utility function C<nbdkit_absolute_path> converts any path to an -absolute path. +absolute path: if it is relative, then all this function does is +prepending the current working directory to the path, with no extra +checks. If conversion was not possible, this calls C<nbdkit_error> and returns C<NULL>. Note that this function does not check that the file exists. The returned string must be freed by the caller. +=head2 C<nbdkit_realpath> + + char *nbdkit_realpath (const char *filename); + +The utility function C<nbdkit_realpath> converts any path to an +absolute path, resolving symlinks. Under the hood it uses the +C<realpath> function, and thus it fails if the path does not exist, +or it is not possible to access to any of the components of the path. + +If the path resolution was not possible, this calls C<nbdkit_error> +and returns C<NULL>. + +The returned string must be freed by the caller. + =head1 CALLBACKS =head2 C<.name> diff --git a/include/nbdkit-common.h b/include/nbdkit-common.h index 5e69579..693213f 100644 --- a/include/nbdkit-common.h +++ b/include/nbdkit-common.h @@ -60,6 +60,7 @@ extern void nbdkit_vdebug (const char *msg, va_list args); extern char *nbdkit_absolute_path (const char *path); extern int64_t nbdkit_parse_size (const char *str); extern int nbdkit_read_password (const char *value, char **password); +extern char *nbdkit_realpath (const char *path); #ifdef __cplusplus } diff --git a/plugins/example2/example2.c b/plugins/example2/example2.c index 5bc4f94..a2d6fca 100644 --- a/plugins/example2/example2.c +++ b/plugins/example2/example2.c @@ -78,7 +78,7 @@ example2_config (const char *key, const char *value) { if (strcmp (key, "file") == 0) { /* See FILENAMES AND PATHS in nbdkit-plugin(3). */ - filename = nbdkit_absolute_path (value); + filename = nbdkit_realpath (value); if (!filename) return -1; } diff --git a/plugins/file/file.c b/plugins/file/file.c index f8cb3d3..b6e33de 100644 --- a/plugins/file/file.c +++ b/plugins/file/file.c @@ -65,7 +65,7 @@ file_config (const char *key, const char *value) if (strcmp (key, "file") == 0) { /* See FILENAMES AND PATHS in nbdkit-plugin(3). */ free (filename); - filename = nbdkit_absolute_path (value); + filename = nbdkit_realpath (value); if (!filename) return -1; } @@ -90,10 +90,6 @@ file_config_complete (void) nbdkit_error ("you must supply the file=<FILENAME> parameter after the plugin name on the command line"); return -1; } - if (access (filename, F_OK) < 0) { - nbdkit_error ("access '%s': %m", filename); - return -1; - } return 0; } diff --git a/plugins/gzip/gzip.c b/plugins/gzip/gzip.c index e9dbfdb..09dd629 100644 --- a/plugins/gzip/gzip.c +++ b/plugins/gzip/gzip.c @@ -62,7 +62,7 @@ gzip_config (const char *key, const char *value) { if (strcmp (key, "file") == 0) { /* See FILENAMES AND PATHS in nbdkit-plugin(3). */ - filename = nbdkit_absolute_path (value); + filename = nbdkit_realpath (value); if (!filename) return -1; } diff --git a/plugins/split/split.c b/plugins/split/split.c index 47c366d..bdcdcf7 100644 --- a/plugins/split/split.c +++ b/plugins/split/split.c @@ -76,7 +76,7 @@ split_config (const char *key, const char *value) return -1; } filenames = new_filenames; - filenames[nr_files] = nbdkit_absolute_path (value); + filenames[nr_files] = nbdkit_realpath (value); if (filenames[nr_files] == NULL) return -1; nr_files++; diff --git a/plugins/vddk/vddk.c b/plugins/vddk/vddk.c index 1c15127..8bc1517 100644 --- a/plugins/vddk/vddk.c +++ b/plugins/vddk/vddk.c @@ -153,7 +153,7 @@ vddk_config (const char *key, const char *value) if (strcmp (key, "config") == 0) { /* See FILENAMES AND PATHS in nbdkit-plugin(3). */ free (config); - config = nbdkit_absolute_path (value); + config = nbdkit_realpath (value); if (!config) return -1; } diff --git a/plugins/xz/xz.c b/plugins/xz/xz.c index 437f798..f45e489 100644 --- a/plugins/xz/xz.c +++ b/plugins/xz/xz.c @@ -67,7 +67,7 @@ xz_config (const char *key, const char *value) { if (strcmp (key, "file") == 0) { /* See FILENAMES AND PATHS in nbdkit-plugin(3). */ - filename = nbdkit_absolute_path (value); + filename = nbdkit_realpath (value); if (!filename) return -1; } diff --git a/src/plugins.c b/src/plugins.c index dba3e24..595b632 100644 --- a/src/plugins.c +++ b/src/plugins.c @@ -134,7 +134,7 @@ plugin_dump_fields (struct backend *b) struct backend_plugin *p = container_of (b, struct backend_plugin, backend); char *path; - path = nbdkit_absolute_path (p->filename); + path = nbdkit_realpath (p->filename); printf ("path=%s\n", path); free (path); diff --git a/src/utils.c b/src/utils.c index 0083370..c6c8003 100644 --- a/src/utils.c +++ b/src/utils.c @@ -228,3 +228,22 @@ nbdkit_read_password (const char *value, char **password) return 0; } + +char * +nbdkit_realpath (const char *path) +{ + char *ret; + + if (path == NULL || *path == '\0') { + nbdkit_error ("cannot resolve a null or empty path"); + return NULL; + } + + ret = realpath (path, NULL); + if (ret == NULL) { + nbdkit_error ("realpath(%s): %m", path); + return NULL; + } + + return ret; +} -- 2.14.3

2 4

[PATCH FOR DISCUSSION ONLY v2] v2v: Add -o kubevirt output mode.
by Richard W.M. Jones 05 Apr '18

05 Apr '18

Fixes some of the more egregious problems with v1, and also applies properly to the head of git without needing any other patches. Rich.

3 5

[PATCH v7 0/6] v2v: Add -o rhv-upload output mode (RHBZ#1557273).
by Richard W.M. Jones 30 Mar '18

30 Mar '18

v6 was here: https://www.redhat.com/archives/libguestfs/2018-March/msg00126.html This makes a number of significant changes: - Input and output options now use a uniform set of -io and -oo parameters. - For -o rhv-upload, we use ‘-oo rhv-cafile=/tmp/ca.pem’ etc. The ‘--rhv*’ options have been dropped. - Rearranges the documentation. - As before includes (untested) support for zero, trim and flush, and falls back to emulating these with current imageio. - Includes suggested changes from Nir Soffer. Rich.

5 33

Re: [Libguestfs] Compilation on OL7.4 - Tests fail (possibly) due to libyajl
by Richard W.M. Jones 29 Mar '18

29 Mar '18

[Let's keep replies on the list please] On Thu, Mar 29, 2018 at 02:16:31PM +0200, Claudio Sasso wrote: > Hi Richard, > > thanks for the suggestions; I think I have a more fundamental > problem here, as after I added yajl, it complained about hivex, then > about augeas. But for now I would be content with a working build, > and later address the underling root cause if any, so for now I am > adding them all; but I am stuck with libiconv as I have built it > manually, so it's not a package. What could be a "proper" way to add > it to the supermin image? > > $ make quickcheck > > ... > > + guestfsd --verbose > guestfsd: error while loading shared libraries: libiconv.so.2: > cannot open shared object file: No such file or directory > + sync > + test '' = 1 > + reboot -f > > $ ldd /tmp/supermin.d/usr/sbin/guestfsd > ... > > libiconv.so.2 => /home/vm/install/lib/libiconv.so.2 > (0x00007fc03464d000) > > I could not check the linkage with ./run virt-rescue --scratch, as > the shell appears to be blocked whatever command I type. Here's the > output (some of the errors may be the root cause of all that? on the > same system build 1.34 works, built following the same recipe): > > $ ./run virt-rescue --scratch > [ 0.168448] usbserial: usb_serial_init - usb_register failed > [ 0.169093] usbserial: usb_serial_init - returning with error -19 > supermin: mounting /proc > supermin: ext2 mini initrd starting up: 5.1.19 glibc > Starting /init script ... > [/usr/lib/tmpfiles.d/systemd.conf:11] Unknown group 'utmp'. > [/usr/lib/tmpfiles.d/systemd.conf:19] Unknown user 'systemd-network'. > [/usr/lib/tmpfiles.d/systemd.conf:20] Unknown user 'systemd-network'. > [/usr/lib/tmpfiles.d/systemd.conf:21] Unknown user 'systemd-network'. > [/usr/lib/tmpfiles.d/systemd.conf:25] Unknown group 'systemd-journal'. > [/usr/lib/tmpfiles.d/systemd.conf:26] Unknown group 'systemd-journal'. > starting version 219 > specified group 'input' unknown > /init: line 120: ip: command not found > /init: line 121: ip: command not found > mdadm: No arrays found in config file or automatically > WARNING: Failed to connect to lvmetad. Falling back to device > scanning. > mdadm: No arrays found in config file or automatically > [ > ] > guestfsd: error while loading shared libraries: libiconv.so.2: > cannot open shared object file: No such file or directory > > ------------------------------------------------------------ > > Welcome to virt-rescue, the libguestfs rescue shell. > > Note: The contents of / (root) are the rescue appliance. > You have to mount the guest’s partitions under /sysroot > before you can examine them. > > ><rescue> ldd /usr/sbin/guestfsd > .... wait forever ... There's obviously some big problem going on. You could try this: supermin5 --build \ /path/to/libguestfs/appliance/supermin.d \ -o /tmp/root -f chroot and see if /tmp/root/usr/lib64/libiconv.so.2 is being copied from the host. To find out why it's not working, keep adding -v options to the supermin command line to enable more and more debug, and compare it to the source: https://github.com/libguestfs/supermin/blob/master/src/mode_build.ml#L85 Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v

1 0

Compilation on OL7.4 - Tests fail (possibly) due to libyajl
by Claudio Sasso 29 Mar '18

29 Mar '18

Hello list, short description: in 1.38, after manual compilation on OL7.4, tests fail; the closest error in logs is guestfsd --verbose failing for libyajl not found, even if it appears correctly linked by the guestfsd executable. Any help/suggestion? My environment is a bit tricky as I work on Oracle Linux 7.4; in my experience it is 99.9% assimilable to RHEL. The server I am using is dedicated to an application embedding libguestfs to analyse the content of VMs, and other things. I build libguestfs from source, instead of relying on the version installed by yum (I also use VirtualBox in this server, I need to use force_tcg=1 and I had issues with the pre-compiled version when I tried). I install all the dependencies which I have to compile manually in ~/install and prepend the following env to anything (configure / make / make install and tests). env LIBGUESTFS_BACKEND_SETTINGS="force_tcg=1" LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1 CPPFLAGS="-I/home/vm/install/include -I/home/vm/install/usr/include" LDFLAGS="-L/home/vm/install/lib -L/home/vm/install/usr/lib" PATH="/home/vm/install/bin:$PATH" After compilation, make check fails with a crash of the appliance; same happens with libguestfs-test-tool. Here is the final part of the log + cmd='guestfsd --verbose' + test '' = 1 + false + test '' = 1 + echo guestfsd --verbose guestfsd --verbose + guestfsd --verbose guestfsd: error while loading shared libraries: libyajl.so.2: cannot open shared object file: No such file or directory + sync + test '' = 1 + reboot -f Rebooting. [ 6.194831] kvm: exiting hardware virtualization [ 6.195551] sd 2:0:1:0: [sdb] Synchronizing SCSI cache [ 6.196217] sd 2:0:0:0: [sda] Synchronizing SCSI cache [ 6.197375] reboot: Restarting system [ 6.197484] reboot: machine restart libguestfs: error: appliance closed the connection unexpectedly, see earlier error messages libguestfs: child_cleanup: 0xf4a160: child process died libguestfs: sending SIGTERM to process 20925 libguestfs: qemu maxrss 342340K libguestfs: error: guestfs_launch failed, see earlier error messages libguestfs: trace: launch = -1 (error) libguestfs: trace: close libguestfs: closing guestfs handle 0xf4a160 (state 0) libguestfs: command: run: rm libguestfs: command: run: \ -rf /tmp/libguestfsGxWqKQ libguestfs: command: run: rm libguestfs: command: run: \ -rf /run/user/500/libguestfssLfpo6 My guess (not much sustained by data, other than message proximity) is that guestfsd cannot find libyajl.so.2, the verbose mode fails, and even if it finishes, perhaps an error message in the log makes the test fail. I am using yajl in yum, but I also tried to compile version 2.1.0 manually and it did not change $ sudo yum install yajl yajl-devel Loaded plugins: ulninfo Package yajl-2.0.4-4.el7.x86_64 already installed and latest version Package yajl-devel-2.0.4-4.el7.x86_64 already installed and latest version The guestfsd executable in appliance/supermin.d/ seems to link yajl correctly $ cp -r libguestfs-1.38.0/appliance/supermin.d/*tar.gz /tmp/supermin.d/ $ cd /tmp/supermin.d/ $ for i in *tar.gz; do tar -xzf $i; done $ ldd ./usr/sbin/guestfsd linux-vdso.so.1 => (0x00007fff4392d000) libacl.so.1 => /lib64/libacl.so.1 (0x00007f1a8919e000) libcap.so.2 => /lib64/libcap.so.2 (0x00007f1a88f99000) libyajl.so.2 => /lib64/libyajl.so.2 (0x00007f1a88d8e000) libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f1a88b67000) libaugeas.so.0 => /lib64/libaugeas.so.0 (0x00007f1a8891b000) libhivex.so.0 => /home/vm/install/lib/libhivex.so.0 (0x00007f1a88708000) libiconv.so.2 => /home/vm/install/lib/libiconv.so.2 (0x00007f1a88422000) libsystemd.so.0 => /lib64/libsystemd.so.0 (0x00007f1a883fa000) libpcre.so.1 => /lib64/libpcre.so.1 (0x00007f1a88197000) libdl.so.2 => /lib64/libdl.so.2 (0x00007f1a87f93000) libm.so.6 => /lib64/libm.so.6 (0x00007f1a87c91000) libc.so.6 => /lib64/libc.so.6 (0x00007f1a878cd000) libattr.so.1 => /lib64/libattr.so.1 (0x00007f1a876c8000) /lib64/ld-linux-x86-64.so.2 (0x000055c87ee30000) libfa.so.1 => /lib64/libfa.so.1 (0x00007f1a874b5000) libxml2.so.2 => /lib64/libxml2.so.2 (0x00007f1a8714a000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f1a86f34000) librt.so.1 => /lib64/librt.so.1 (0x00007f1a86d2c000) liblzma.so.5 => /lib64/liblzma.so.5 (0x00007f1a86b05000) libgcrypt.so.11 => /lib64/libgcrypt.so.11 (0x00007f1a86884000) libgpg-error.so.0 => /lib64/libgpg-error.so.0 (0x00007f1a8667f000) libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f1a86464000) libdw.so.1 => /lib64/libdw.so.1 (0x00007f1a8621d000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f1a86001000) libz.so.1 => /lib64/libz.so.1 (0x00007f1a85dea000) libelf.so.1 => /lib64/libelf.so.1 (0x00007f1a85bd2000) libbz2.so.1 => /lib64/libbz2.so.1 (0x00007f1a859c1000) $ file /lib64/libyajl.so.2.0.4 /lib64/libyajl.so.2 /lib64/libyajl.so.2.0.4: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=b568a12b3f954f8a7fc2be567df484ccc5b1e058, stripped /lib64/libyajl.so.2: symbolic link to `libyajl.so.2.0.4' Attached the output of ./libguestfs-test-tool run using this line $ env LIBGUESTFS_BACKEND_SETTINGS="force_tcg=1" LIBGUESTFS_PATH=/home/vm/building7.4/deps4libguestfs/libguestfs-1.38.0/appliance LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1 CPPFLAGS="-I/home/vm/install/include -I/home/vm/install/usr/include" LDFLAGS="-L/home/vm/install/lib -L/home/vm/install/usr/lib" PATH="/home/vm/install/bin:$PATH" ./libguestfs-test-tool > libyajl.test-fail.out 2>&1 I am either missing something trivial OR this is not the cause of failure. In both cases, any help is greatly appreciated Thanks in advance

2 1