June 2022 - Libguestfs - libguestfs List Archives

RFC for NBD protocol extension: extended headers
by Eric Blake 04 Oct '22

04 Oct '22

In response to this mail, I will be cross-posting a series of patches to multiple projects as a proof-of-concept implementation and request for comments on a new NBD protocol extension, called NBD_OPT_EXTENDED_HEADERS. With this in place, it will be possible for clients to request 64-bit zero, trim, cache, and block status operations when supported by the server. Not yet complete: an implementation of this in nbdkit. I also plan to tweak libnbd's 'nbdinfo --map' and 'nbdcopy' to take advantage of the larger block status results. Also, with 64-bit commands, we may want to also make it easier to let servers advertise an actual maximum size they are willing to accept for the commands in question (for example, a server may be happy with a full 64-bit block status, but still want to limit non-fast zero and cache to a smaller limit to avoid denial of service). -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org

4 46

[PATCH] always 'max' for the appliance CPU model on all targes except ppc
by Daniel P. Berrangé 20 Jul '22

20 Jul '22

The current logic for selecting CPU model to use with the appliance selects 'max' for all architectures except for ppc64le and aarch64. On aarch64, it selects 'host' if KVM is available which is identical to 'max'. For TCG it selects 'cortex-a57'. The 'max' model is a superset of 'cortex-a57', so it is reasonable to use 'max' for TCG too. On ppc64, it selects no CPU model due to a historical bug where using 'host' would break ability to fallback to TCG. Unfortunately we can't use 'max' on PPC as this is actually an old G4 vintage 32-bit model, rather than a synonym for 'host' / all-TCG-features as on other targets. We can at least simplify the code to use 'max' in all scenarios for appliance CPU model, and simply skip a CPU model for PPC. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- docs/C_SOURCE_FILES | 1 - lib/Makefile.am | 1 - lib/appliance-cpu.c | 93 ------------------------------------------ lib/guestfs-internal.h | 3 -- lib/launch-direct.c | 25 +++++------- lib/launch-libvirt.c | 40 ++++++++---------- po/POTFILES | 1 - 7 files changed, 27 insertions(+), 137 deletions(-) delete mode 100644 lib/appliance-cpu.c diff --git a/docs/C_SOURCE_FILES b/docs/C_SOURCE_FILES index a367c93a0..a26487d99 100644 --- a/docs/C_SOURCE_FILES +++ b/docs/C_SOURCE_FILES @@ -283,7 +283,6 @@ lib/actions-6.c lib/actions-support.c lib/actions-variants.c lib/alloc.c -lib/appliance-cpu.c lib/appliance-kcmdline.c lib/appliance-uefi.c lib/appliance.c diff --git a/lib/Makefile.am b/lib/Makefile.am index 212bcb94a..40a4c9ac3 100644 --- a/lib/Makefile.am +++ b/lib/Makefile.am @@ -71,7 +71,6 @@ libguestfs_la_SOURCES = \ actions-variants.c \ alloc.c \ appliance.c \ - appliance-cpu.c \ appliance-kcmdline.c \ appliance-uefi.c \ available.c \ diff --git a/lib/appliance-cpu.c b/lib/appliance-cpu.c deleted file mode 100644 index 54ac6e2e3..000000000 --- a/lib/appliance-cpu.c +++ /dev/null @@ -1,93 +0,0 @@ -/* libguestfs - * Copyright (C) 2009-2020 Red Hat Inc. - * - * This library is free software; you can redistribute it and/or - * modify it under the terms of the GNU Lesser General Public - * License as published by the Free Software Foundation; either - * version 2 of the License, or (at your option) any later version. - * - * This library is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public - * License along with this library; if not, write to the Free Software - * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA - */ - -/** - * The appliance choice of CPU model. - */ - -#include <config.h> - -#include <stdio.h> -#include <stdlib.h> - -#include "guestfs.h" -#include "guestfs-internal.h" - -/** - * Return the right CPU model to use as the qemu C<-cpu> parameter or - * its equivalent in libvirt. This returns: - * - * =over 4 - * - * =item C<"host"> - * - * The literal string C<"host"> means use C<-cpu host>. - * - * =item C<"max"> - * - * The literal string C<"max"> means use C<-cpu max> (the best - * possible). This requires awkward translation for libvirt. - * - * =item some string - * - * Some string such as C<"cortex-a57"> means use C<-cpu cortex-a57>. - * - * =item C<NULL> - * - * C<NULL> means no C<-cpu> option at all. Note returning C<NULL> - * does not indicate an error. - * - * =back - * - * This is made unnecessarily hard and fragile because of two stupid - * choices in QEMU: - * - * =over 4 - * - * =item * - * - * The default for C<qemu-system-aarch64 -M virt> is to emulate a - * C<cortex-a15> (WTF?). - * - * =item * - * - * We don't know for sure if KVM will work, but C<-cpu host> is broken - * with TCG, so we almost always pass a broken C<-cpu> flag if KVM is - * semi-broken in any way. - * - * =back - */ -const char * -guestfs_int_get_cpu_model (int kvm) -{ -#if defined(__aarch64__) - /* With -M virt, the default -cpu is cortex-a15. Stupid. */ - if (kvm) - return "host"; - else - return "cortex-a57"; -#elif defined(__powerpc64__) - /* See discussion in https://bugzilla.redhat.com/show_bug.cgi?id=1605071 */ - return NULL; -#else - /* On most architectures we can use "max" to get the best possible CPU. - * For recent qemu this should work even on TCG. - */ - return "max"; -#endif -} diff --git a/lib/guestfs-internal.h b/lib/guestfs-internal.h index 16755cfb3..33037a26d 100644 --- a/lib/guestfs-internal.h +++ b/lib/guestfs-internal.h @@ -718,9 +718,6 @@ extern const char *guestfs_int_drive_protocol_to_string (enum drive_protocol pro /* appliance.c */ extern int guestfs_int_build_appliance (guestfs_h *g, char **kernel, char **initrd, char **appliance); -/* appliance-cpu.c */ -const char *guestfs_int_get_cpu_model (int kvm); - /* appliance-kcmdline.c */ extern char *guestfs_int_appliance_command_line (guestfs_h *g, const char *appliance, int flags); #define APPLIANCE_COMMAND_LINE_IS_TCG 1 diff --git a/lib/launch-direct.c b/lib/launch-direct.c index ff0eaeb62..f41711353 100644 --- a/lib/launch-direct.c +++ b/lib/launch-direct.c @@ -354,7 +354,6 @@ launch_direct (guestfs_h *g, void *datav, const char *arg) int force_tcg; int force_kvm; const char *accel_val = "kvm:tcg"; - const char *cpu_model; CLEANUP_FREE char *append = NULL; CLEANUP_FREE_STRING_LIST char **argv = NULL; @@ -517,20 +516,18 @@ launch_direct (guestfs_h *g, void *datav, const char *arg) #endif } end_list (); - cpu_model = guestfs_int_get_cpu_model (has_kvm && !force_tcg); - if (cpu_model) { -#if defined(__x86_64__) - /* Temporary workaround for RHBZ#2082806 */ - if (STREQ (cpu_model, "max")) { - start_list ("-cpu") { - append_list (cpu_model); - append_list ("la57=off"); - } end_list (); - } - else + /* + * Can't use 'max' on ppc64 since it is an old G4 model + * Also can't use 'host' due to TCG fallback issues + * https://bugzilla.redhat.com/show_bug.cgi?id=1605071 + */ +#if !defined(__powerpc64__) +# if defined(__x86_64__) + arg ("-cpu", "max,la57=off"); +# else + arg ("-cpu", "max"); +# endif #endif - arg ("-cpu", cpu_model); - } if (g->smp > 1) arg_format ("-smp", "%d", g->smp); diff --git a/lib/launch-libvirt.c b/lib/launch-libvirt.c index 03d69e027..b97c91566 100644 --- a/lib/launch-libvirt.c +++ b/lib/launch-libvirt.c @@ -1161,8 +1161,6 @@ construct_libvirt_xml_cpu (guestfs_h *g, const struct libvirt_xml_params *params, xmlTextWriterPtr xo) { - const char *cpu_model; - start_element ("memory") { attribute ("unit", "MiB"); string_format ("%d", g->memsize); @@ -1173,30 +1171,24 @@ construct_libvirt_xml_cpu (guestfs_h *g, string_format ("%d", g->memsize); } end_element (); - cpu_model = guestfs_int_get_cpu_model (params->data->is_kvm); - if (cpu_model) { - start_element ("cpu") { - if (STREQ (cpu_model, "host")) { - attribute ("mode", "host-passthrough"); - start_element ("model") { - attribute ("fallback", "allow"); - } end_element (); - } - else if (STREQ (cpu_model, "max")) { - /* https://bugzilla.redhat.com/show_bug.cgi?id=1935572#c11 */ - attribute ("mode", "maximum"); + /* + * Can't use 'max' on ppc64 since it is an old G4 model + * Also can't use 'host' due to TCG fallback issues + * https://bugzilla.redhat.com/show_bug.cgi?id=1605071 + */ +#if !defined(__powerpc64__) + start_element ("cpu") { + /* https://bugzilla.redhat.com/show_bug.cgi?id=1935572#c11 */ + attribute ("mode", "maximum"); #if defined(__x86_64__) - /* Temporary workaround for RHBZ#2082806 */ - start_element ("feature") { - attribute ("policy", "disable"); - attribute ("name", "la57"); - } end_element (); -#endif - } - else - single_element ("model", cpu_model); + /* Temporary workaround for RHBZ#2082806 */ + start_element ("feature") { + attribute ("policy", "disable"); + attribute ("name", "la57"); } end_element (); - } +#endif + } end_element (); +#endif single_element_format ("vcpu", "%d", g->smp); diff --git a/po/POTFILES b/po/POTFILES index 32b975a04..67cdffb29 100644 --- a/po/POTFILES +++ b/po/POTFILES @@ -329,7 +329,6 @@ lib/actions-6.c lib/actions-support.c lib/actions-variants.c lib/alloc.c -lib/appliance-cpu.c lib/appliance-kcmdline.c lib/appliance-uefi.c lib/appliance.c -- 2.36.1

5 14

[PATCH libnbd v4 0/2] copy: Use preferred block size for copying
by Richard W.M. Jones 08 Jul '22

08 Jul '22

v3 was: https://listman.redhat.com/archives/libguestfs/2022-June/029336.html v4 makes almost all of the recommended changes from Laszlo's reviews. I have dropped the third patch (the one which stablises copy/copy-nbd-error.sh test). That patch is still necessary, but it's not so urgent because the null: destination no longer advertises a 4M preferred block size, so the test fails only very rarely as before. Also I'd like to get Eric's input on that test when he's back. Rich.

2 7

LUKS decryption with Clevis+Tang | CVE-2022-2211
by Laszlo Ersek 01 Jul '22

01 Jul '22

Hi, * in response to this cover letter, I'm going to post four series (one for each of libguestfs-common, libguestfs, guestfs-tools, virt-v2v). These four series implement LUKS decryption with Clevis+Tang: https://bugzilla.redhat.com/show_bug.cgi?id=1809453 * The first patch in the libguestfs-common series fixes a bug that I'd found while working on the feature, and ended up receiving a CVE number (CVE-2022-2211): https://bugzilla.redhat.com/show_bug.cgi?id=2100862 This patch is an integral part of the larger Clevis+Tang feature. However, it can be backported easily to stable branches that only want the bugfix. * Correspondingly, the first patch in the libguestfs series documents the new CVE (and updates the common submodule just enough to get the CVE fix). This patch should also be easy to backport to stable branches. A later patch in the libguestfs series updates the "common" submodule checkout to the end of the libguestfs-common series. * In each of the guestfs-tools and virt-v2v series, the full "common" submodule series is consumed right in the first patch, covering both the CVE fix and the new stuff needed for the Clevis feature. Thanks, Laszlo

3 60

[libguestfs PATCH v2 0/3] LUKS decryption with Clevis+Tang
by Laszlo Ersek 01 Jul '22

01 Jul '22

v1: https://listman.redhat.com/archives/libguestfs/2022-June/029290.html Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1809453 Please see the Notes section on each patch for the updates in this version. (The documentation of the CVE fix has been merged: 99844660b48e.) Thanks Laszlo Laszlo Ersek (3): use local git repo for common submodule introduce the "clevis_luks_unlock" API guestfish, guestmount: enable networking for "--key ID:clevis" .gitmodules | 3 +- appliance/packagelist.in | 4 ++ common | 2 +- daemon/Makefile.am | 1 + daemon/clevis-luks.c | 58 ++++++++++++++++++++ fish/fish.c | 3 + fuse/guestmount.c | 4 ++ generator/actions_core.ml | 40 ++++++++++++++ generator/proc_nr.ml | 1 + lib/MAX_PROC_NR | 2 +- lib/guestfs.pod | 19 +++++-- 11 files changed, 130 insertions(+), 7 deletions(-) create mode 100644 daemon/clevis-luks.c -- 2.19.1.3.g30247aa5d201

2 7

[libguestfs-common PATCH v2 00/12] LUKS decryption with Clevis+Tang
by Laszlo Ersek 01 Jul '22

01 Jul '22

v1: https://listman.redhat.com/archives/libguestfs/2022-June/029277.html Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1809453 Please see the Notes section on each patch for the updates in this version. (The CVE fix has been merged: 35467027f657.) Thanks, Laszlo Laszlo Ersek (12): options: fix UUID comparison logic bug in get_keys() mltools/tools_utils: remove unused function "key_store_to_cli" mltools/tools_utils: allow multiple "--key" options for OCaml tools too options: replace NULL-termination with number-of-elements in get_keys() options: wrap each passphrase from get_keys() into a struct options: add back-end for LUKS decryption with Clevis+Tang options: introduce selector type "key_clevis" options: generalize "--key" selector parsing for C-language utilities mltools/tools_utils-c: handle internal type error with abort() mltools/tools_utils: generalize "--key" selector parsing for OCaml utils options, mltools/tools_utils: parse "--key ID:clevis" options options, mltools/tools_utils: add helper for network dependency mltools/tools_utils-c.c | 43 ++++--- mltools/tools_utils.ml | 51 ++++---- mltools/tools_utils.mli | 12 +- options/decrypt.c | 30 +++-- options/key-option.pod | 9 ++ options/keys.c | 124 ++++++++++++++------ options/options.h | 19 ++- 7 files changed, 191 insertions(+), 97 deletions(-) -- 2.19.1.3.g30247aa5d201

2 22

[PATCH libnbd v3 0/3] copy: Use preferred block size for copying
by Richard W.M. Jones 01 Jul '22

01 Jul '22

v2 was: https://listman.redhat.com/archives/libguestfs/2022-June/029321.html v3: - adds a test - force sparseness detection size to be at least as large as the target preferred size (otherwise you end up creating sub-requests which are smaller than the preferred size) Rich.

2 10

libnbd | Failed pipeline for master | a8fa05ff
by GitLab 30 Jun '22

30 Jun '22

Pipeline #577440597 has failed! Project: libnbd ( https://gitlab.com/nbdkit/libnbd ) Branch: master ( https://gitlab.com/nbdkit/libnbd/-/commits/master ) Commit: a8fa05ff ( https://gitlab.com/nbdkit/libnbd/-/commit/a8fa05ffb8b85f41276ffb52498e4528c… ) Commit Message: dump/dump-data.sh: Test requires nbdkit 1.22 U... Commit Author: Richard W_M_ Jones ( https://gitlab.com/rwmjones ) Pipeline #577440597 ( https://gitlab.com/nbdkit/libnbd/-/pipelines/577440597 ) triggered by Richard W_M_ Jones ( https://gitlab.com/rwmjones ) had 2 failed jobs. Job #2664717877 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2664717877/raw ) Stage: builds Name: x86_64-freebsd-current Job #2664717864 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2664717864/raw ) Stage: builds Name: x86_64-opensuse-leap-153 -- You're receiving this email because of your account on gitlab.com.

1 0

libnbd | Failed pipeline for master | 315a637d
by GitLab 30 Jun '22

30 Jun '22

Pipeline #577418984 has failed! Project: libnbd ( https://gitlab.com/nbdkit/libnbd ) Branch: master ( https://gitlab.com/nbdkit/libnbd/-/commits/master ) Commit: 315a637d ( https://gitlab.com/nbdkit/libnbd/-/commit/315a637d3eae003c1d84eb1b88a7b47b5… ) Commit Message: dump: Visually separate columns 0-7 and 8-15 B... Commit Author: Richard W_M_ Jones ( https://gitlab.com/rwmjones ) Pipeline #577418984 ( https://gitlab.com/nbdkit/libnbd/-/pipelines/577418984 ) triggered by Richard W_M_ Jones ( https://gitlab.com/rwmjones ) had 6 failed jobs. Job #2664589626 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2664589626/raw ) Stage: builds Name: x86_64-opensuse-leap-153 Job #2664589635 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2664589635/raw ) Stage: builds Name: x86_64-macos-11 Job #2664589630 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2664589630/raw ) Stage: builds Name: i686-debian-10 Job #2664589629 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2664589629/raw ) Stage: builds Name: x86_64-ubuntu-2004 Job #2664589631 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2664589631/raw ) Stage: builds Name: i686-debian-11 Job #2664589617 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2664589617/raw ) Stage: builds Name: x86_64-debian-10 -- You're receiving this email because of your account on gitlab.com.

1 0

libnbd | Failed pipeline for master | 788f0591
by GitLab 30 Jun '22

30 Jun '22

Pipeline #576956721 has failed! Project: libnbd ( https://gitlab.com/nbdkit/libnbd ) Branch: master ( https://gitlab.com/nbdkit/libnbd/-/commits/master ) Commit: 788f0591 ( https://gitlab.com/nbdkit/libnbd/-/commit/788f0591eb42640de251b604080613f02… ) Commit Message: Version 1.13.4. Commit Author: Richard W_M_ Jones ( https://gitlab.com/rwmjones ) Pipeline #576956721 ( https://gitlab.com/nbdkit/libnbd/-/pipelines/576956721 ) triggered by Richard W_M_ Jones ( https://gitlab.com/rwmjones ) had 6 failed jobs. Job #2661872840 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2661872840/raw ) Stage: builds Name: x86_64-opensuse-leap-153 Job #2661872821 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2661872821/raw ) Stage: builds Name: x86_64-debian-10 Job #2661872862 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2661872862/raw ) Stage: builds Name: x86_64-macos-11 Job #2661872850 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2661872850/raw ) Stage: builds Name: x86_64-ubuntu-2004 Job #2661872851 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2661872851/raw ) Stage: builds Name: i686-debian-10 Job #2661872853 ( https://gitlab.com/nbdkit/libnbd/-/jobs/2661872853/raw ) Stage: builds Name: i686-debian-11 -- You're receiving this email because of your account on gitlab.com.

1 0