January 2019 - Libguestfs - libguestfs List Archives

[PATCH] v2v: -o openstack: Don't echo full commands (RHBZ#1664310).
by Richard W.M. Jones 08 Jan '19

08 Jan '19

They can contain passwords or tokens if for example the ‘-oo os-password’ option is used. Thanks: Tomáš Golembiovský, Brett Thurber. --- v2v/output_openstack.ml | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/v2v/output_openstack.ml b/v2v/output_openstack.ml index 95c42cbed..3ff1e1269 100644 --- a/v2v/output_openstack.ml +++ b/v2v/output_openstack.ml @@ -177,8 +177,10 @@ class output_openstack output_conn output_password output_storage let stdout_fd = if verbose () then None else Some (openfile "/dev/null" [O_WRONLY] 0) in - (* Note that run_command will close stdout_fd if defined. *) - Tools_utils.run_command ?stdout_fd cmd + (* Note that run_command will close stdout_fd if defined. + * Don't echo the command because it can contain passwords. + *) + Tools_utils.run_command ~echo_cmd:false ?stdout_fd cmd in (* Similar to above, run the openstack command and capture the @@ -192,8 +194,10 @@ class output_openstack output_conn output_password output_storage unlink_on_exit json; let fd = descr_of_out_channel chan in - (* Note that Tools_utils.run_command closes fd. *) - if Tools_utils.run_command ~stdout_fd:fd cmd <> 0 then + (* Note that Tools_utils.run_command closes fd. + * Don't echo the command because it can contain passwords. + *) + if Tools_utils.run_command ~echo_cmd:false ~stdout_fd:fd cmd <> 0 then None else ( let json = json_parser_tree_parse_file json in -- 2.19.2

2 1

[PATCH] v2v: -o rhv-upload: decouple name of nbdkit python plugin
by Pino Toscano 07 Jan '19

07 Jan '19

Do not assume that the Python plugin of nbdkit has the same name of the Python interpreter. Use the default upstream name of nbdkit to identify it; downstream distributions must adjust this variable, in case they rename the Python plugin of nbdkit. --- v2v/output_rhv_upload.ml | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/v2v/output_rhv_upload.ml b/v2v/output_rhv_upload.ml index f5e0e6b1c..d8f608cff 100644 --- a/v2v/output_rhv_upload.ml +++ b/v2v/output_rhv_upload.ml @@ -78,6 +78,7 @@ let parse_output_options options = { rhv_cafile; rhv_cluster; rhv_direct; rhv_verifypeer } +let nbdkit_python_plugin = "python" let pidfile_timeout = 30 let finalization_timeout = 5*60 @@ -139,14 +140,14 @@ class output_rhv_upload output_alloc output_conn *) let error_unless_nbdkit_python_plugin_working () = let cmd = sprintf "nbdkit %s %s --dump-plugin >/dev/null" - Python_script.python + nbdkit_python_plugin (quote (Python_script.path plugin_script)) in debug "%s" cmd; if Sys.command cmd <> 0 then error (f_"nbdkit %s plugin is not installed or not working. It is required if you want to use ‘-o rhv-upload’. See also the virt-v2v-output-rhv(1) manual.") - Python_script.python + nbdkit_python_plugin in (* Check that nbdkit was compiled with SELinux support (for the @@ -209,7 +210,7 @@ See also the virt-v2v-output-rhv(1) manual.") "--newstyle"; (* use newstyle NBD protocol *) "--exportname"; "/"; - Python_script.python; (* use the nbdkit Python 3 plugin *) + nbdkit_python_plugin; (* use the nbdkit Python plugin *) Python_script.path plugin_script; (* Python plugin script *) ] in let args = if verbose () then args @ ["--verbose"] else args in -- 2.17.2

2 5

[PATCH v2] Revert "launch: libvirt: Use qemu-bridge-helper to implement a full network (RHBZ#1148012)."
by Richard W.M. Jones 07 Jan '19

07 Jan '19

Let's actually compile and test the patch this time, rather than trusting the RHEL 7.6 patch to apply directly to head ... Rich.

2 3

[PATCH nbdkit v2 01/11] server: Implement NBD_FLAG_CAN_MULTI_CONN.
by Richard W.M. Jones 06 Jan '19

06 Jan '19

For existing commits, this is almost identical to v1, except that I updated some commit messages and reordered the commits in a somewhat more logical sequence. The main changes are the extra commits: [06/11] plugins: Return NBD_FLAG_CAN_MULTI_CONN from some readonly plugins. - Readonly plugins that can set the flag unconditionally. [09/11] partitioning: Return NBD_FLAG_CAN_MULTI_CONN. [10/11] data, memory: Return NBD_FLAG_CAN_MULTI_CONN. - Writeable plugins where it should be safe to set the flag always. [11/11] memory, data: Use fine-grained locking and - Let's play with locking. That last commit should be "data, memory: ..." Rich.

2 16

[PATCH nbdkit v5 3/3] cache: Implement cache-max-size and cache space reclaim.
by Richard W.M. Jones 04 Jan '19

04 Jan '19

v4: https://www.redhat.com/archives/libguestfs/2019-January/msg00032.html v5: - Now we set the block size at run time. I'd like to say that I was able to test this change, but unfortunately I couldn't find any easy way to create a filesystem on x86-64 with a block size > 4K. Ext4 doesn't support it at all, and XFS doesn't support block size > page size (and I recently gave away my aarch64 machine that ran RHEL, so ..) - Addressed a few other minor issues that Eric found in v4. - Retested on Linux, FreeBSD and OpenBSD. I'm including the minmax patch which I sent before. That's because it isn't independent as I thought it was - it depends on the LRU patch. Rich.

2 5

[PATCH nbdkit] common/include: Add generic MIN and MAX macros.
by Richard W.M. Jones 04 Jan '19

04 Jan '19

The preferred implementation uses __auto_type, a GCC extension also now supported by Clang. Unfortunately OpenBSD ships with GCC 4.2.1 (from 2007!) which predates this extension by quite a few years, so we have to be able to fall back to a plain macro. --- configure.ac | 20 ++++++++++- common/include/minmax.h | 63 +++++++++++++++++++++++++++++++++++ filters/blocksize/blocksize.c | 3 +- filters/cache/lru.c | 3 +- filters/nozero/nozero.c | 3 +- plugins/pattern/pattern.c | 3 +- common/include/Makefile.am | 1 + filters/blocksize/Makefile.am | 3 +- filters/nozero/Makefile.am | 3 +- 9 files changed, 93 insertions(+), 9 deletions(-) diff --git a/configure.ac b/configure.ac index 606e632..ee1ff0d 100644 --- a/configure.ac +++ b/configure.ac @@ -156,9 +156,27 @@ you may be using a C compiler which does not support this attribute, or the configure test may be wrong. This code requires the attribute to work for proper locking between threads.])]) -dnl restore CFLAGS CFLAGS="${acx_nbdkit_save_CFLAGS}" +dnl Check for __auto_type (GCC extension). +AC_MSG_CHECKING([if __auto_type is available in this compiler]) +AC_COMPILE_IFELSE([ +AC_LANG_SOURCE([[ +static int +test (int a) +{ + __auto_type at = a; + return at; +} +]]) + ],[ + AC_MSG_RESULT([yes]) + AC_DEFINE([HAVE_AUTO_TYPE],[1],[__auto_type is available]) + ],[ + AC_MSG_RESULT([no]) + ] +) + dnl Check for other headers, all optional. AC_CHECK_HEADERS([\ alloca.h \ diff --git a/common/include/minmax.h b/common/include/minmax.h new file mode 100644 index 0000000..dc01679 --- /dev/null +++ b/common/include/minmax.h @@ -0,0 +1,63 @@ +/* nbdkit + * Copyright (C) 2019 Red Hat Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * * Neither the name of Red Hat nor the names of its contributors may be + * used to endorse or promote products derived from this software without + * specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, + * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A + * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR + * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF + * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND + * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, + * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT + * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#ifndef NBDKIT_MINMAX_H +#define NBDKIT_MINMAX_H + +#include <config.h> + +#ifdef HAVE_AUTO_TYPE + +/* __auto_type is a GCC extension, added in GCC 4.9 in 2014, and to + * Clang in 2015. + */ + +#define MIN(x, y) ({ \ + __auto_type _x = (x); \ + __auto_type _y = (y); \ + _x < _y ? _x : _y; \ + }) +#define MAX(x, y) ({ \ + __auto_type _x = (x); \ + __auto_type _y = (y); \ + _x > _y ? _x : _y; \ + }) + +#else + +#define MIN(x, y) ((x) < (y) ? (x) : (y)) +#define MAX(x, y) ((x) > (y) ? (x) : (y)) + +#endif + +#endif /* NBDKIT_MINMAX_H */ diff --git a/filters/blocksize/blocksize.c b/filters/blocksize/blocksize.c index 6acd9b6..34f58c9 100644 --- a/filters/blocksize/blocksize.c +++ b/filters/blocksize/blocksize.c @@ -44,11 +44,12 @@ #include <nbdkit-filter.h> +#include "minmax.h" + /* XXX See design comment in filters/cow/cow.c. */ #define THREAD_MODEL NBDKIT_THREAD_MODEL_SERIALIZE_ALL_REQUESTS #define BLOCKSIZE_MIN_LIMIT (64U * 1024) -#define MIN(a, b) ((a) < (b) ? (a) : (b)) /* As long as we don't have parallel requests, we can reuse a common * buffer for alignment purposes; size it to the maximum we allow for diff --git a/filters/cache/lru.c b/filters/cache/lru.c index c828968..9e20408 100644 --- a/filters/cache/lru.c +++ b/filters/cache/lru.c @@ -46,13 +46,12 @@ #include <nbdkit-filter.h> #include "bitmap.h" +#include "minmax.h" #include "cache.h" #include "blk.h" #include "lru.h" -#define MAX(a, b) ((a) > (b) ? (a) : (b)) - /* LRU bitmaps. These bitmaps implement a simple, fast LRU structure. * * bm[0] bm[1] blocks not in either bitmap diff --git a/filters/nozero/nozero.c b/filters/nozero/nozero.c index dac47ad..6a22f9b 100644 --- a/filters/nozero/nozero.c +++ b/filters/nozero/nozero.c @@ -42,9 +42,10 @@ #include <nbdkit-filter.h> +#include "minmax.h" + #define THREAD_MODEL NBDKIT_THREAD_MODEL_PARALLEL -#define MIN(a, b) ((a) < (b) ? (a) : (b)) #define MAX_WRITE (64 * 1024 * 1024) static char buffer[MAX_WRITE]; diff --git a/plugins/pattern/pattern.c b/plugins/pattern/pattern.c index 1d1b234..32db381 100644 --- a/plugins/pattern/pattern.c +++ b/plugins/pattern/pattern.c @@ -45,6 +45,7 @@ #include <nbdkit-plugin.h> #include "byte-swapping.h" +#include "minmax.h" /* The size of disk in bytes (initialized by size=<SIZE> parameter). */ static int64_t size = 0; @@ -87,8 +88,6 @@ pattern_get_size (void *handle) return size; } -#define MIN(a, b) ((a) < (b) ? (a) : (b)) - /* Read data. */ static int pattern_pread (void *handle, void *buf, uint32_t count, uint64_t offset, diff --git a/common/include/Makefile.am b/common/include/Makefile.am index 3e7f056..7df4855 100644 --- a/common/include/Makefile.am +++ b/common/include/Makefile.am @@ -41,6 +41,7 @@ EXTRA_DIST = \ isaligned.h \ ispowerof2.h \ iszero.h \ + minmax.h \ nextnonzero.h \ random.h \ rounding.h diff --git a/filters/blocksize/Makefile.am b/filters/blocksize/Makefile.am index b49fc3f..231c9b5 100644 --- a/filters/blocksize/Makefile.am +++ b/filters/blocksize/Makefile.am @@ -41,7 +41,8 @@ nbdkit_blocksize_filter_la_SOURCES = \ $(top_srcdir)/include/nbdkit-filter.h nbdkit_blocksize_filter_la_CPPFLAGS = \ - -I$(top_srcdir)/include + -I$(top_srcdir)/include \ + -I$(top_srcdir)/common/include nbdkit_blocksize_filter_la_CFLAGS = \ $(WARNINGS_CFLAGS) nbdkit_blocksize_filter_la_LDFLAGS = \ diff --git a/filters/nozero/Makefile.am b/filters/nozero/Makefile.am index b1fd4a8..fe88233 100644 --- a/filters/nozero/Makefile.am +++ b/filters/nozero/Makefile.am @@ -41,7 +41,8 @@ nbdkit_nozero_filter_la_SOURCES = \ $(top_srcdir)/include/nbdkit-filter.h nbdkit_nozero_filter_la_CPPFLAGS = \ - -I$(top_srcdir)/include + -I$(top_srcdir)/include \ + -I$(top_srcdir)/common/include nbdkit_nozero_filter_la_CFLAGS = \ $(WARNINGS_CFLAGS) nbdkit_nozero_filter_la_LDFLAGS = \ -- 2.19.2

1 0

Libguestfs virt-builder network
by Andrzej Gretkowski 04 Jan '19

04 Jan '19

Hi, I am trying to install a package inside a VM with virt-builder, and it seems I cannot connect to any repo, which means I probably have no network inside the guest. I installed it with: *sudo apt-get install libguestfs-tools* My OS-version: *Ubuntu 18.04* Command I used: *virt-builder ubuntu-16.04 --network --install git * I attached the complete output. I have also tried to install it from a tarball, 1.38.6, and the error persists. Should I somehow configure my network? Best regards, AG

2 1

[PATCH nbdkit v4 0/2] cache: Implement cache-max-size and method of
by Richard W.M. Jones 03 Jan '19

03 Jan '19

v3 was broken by a bad rebase, so let's forget about that one. Compared to v2: - Patch 1 is the same except for a minor comment change. - Patch 2 splits the reclaim code into a separate file (filters/cache/reclaim.c) - Addressed Eric's comments from his review of v2. - Retested on Linux and FreeBSD.

2 5

[PATCH nbdkit v3 0/2] cache: Implement cache-max-size and method of reclaiming space from the cache.
by Richard W.M. Jones 03 Jan '19

03 Jan '19

Patch 1 is the same as last time, except for a minor comment fix. Patch 2 should address everything that Eric mentioned in his review, and has been retested. Rich.

1 3

[PATCH nbdkit v2 0/4] cache: Implement cache-max-size etc.
by Richard W.M. Jones 03 Jan '19

03 Jan '19

These are essentially identical to what was previously posted as patches 6/9 through 9/9 here: https://www.redhat.com/archives/libguestfs/2018-December/msg00145.html except that it has been rebased onto the current git master and retested thoroughly. Rich.

2 10