October 2017 - Libguestfs - libguestfs List Archives

[PATCH] p2v: Test for sudo requiring a password first (RHBZ#1500673).
by Richard W.M. Jones 11 Oct '17

11 Oct '17

When testing if sudo -n requires a password, we tested for the prompt earlier than testing for the magic sudo message ‘a password is required’. Since the shell will print the prompt just after the sudo message: prompt$ sudo -n virt-v2v --version sudo: a password is required prompt$ the prompt nearly always matched and we missed the magic sudo message. (The exception is in the case where we are running everything on localhost where the sudo message could be read in a single call to read(2) without seeing the prompt immediately afterwards. Even this exception was non-deterministic.) By swapping the priority of the sudo message and prompt we avoid this. Looking at the debug output (enabled by editing common/miniexpect) makes this clearer: DEBUG: writing: sudo -n virt-v2v --version DEBUG: buffer content: sudo: a password is required ESC]0;rjones@hamr:~^G###bphcxtq5### Thanks: Ming Xie. --- p2v/ssh.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/p2v/ssh.c b/p2v/ssh.c index bfeb80661..991888348 100644 --- a/p2v/ssh.c +++ b/p2v/ssh.c @@ -792,8 +792,8 @@ test_connection (struct config *config) switch (mexp_expect (h, (mexp_regexp[]) { { 100, .re = version_re }, - { 101, .re = prompt_re }, - { 102, .re = sudo_password_re }, + { 101, .re = sudo_password_re }, + { 102, .re = prompt_re }, { 0 } }, ovector, ovecsize)) { case 100: /* Got version string. */ @@ -805,15 +805,15 @@ test_connection (struct config *config) #endif break; - case 101: /* Got the prompt. */ - goto end_of_version; - - case 102: + case 101: set_ssh_error ("sudo for user \"%s\" requires a password. Edit /etc/sudoers on the conversion server to ensure the \"NOPASSWD:\" option is set for this user.", config->username); mexp_close (h); return -1; + case 102: /* Got the prompt. */ + goto end_of_version; + case MEXP_EOF: set_ssh_unexpected_eof ("\"virt-v2v --version\" output"); mexp_close (h); -- 2.13.2

2 1

[PATCH 0/2] v2v: -i vmx: Allow deviceType field to be completely omitted.
by Richard W.M. Jones 11 Oct '17

11 Oct '17

A colleague found some VMX files which omit the deviceType field. This allows -i vmx mode to parse them. Rich.

2 3

[PATCH] build: build mlaugeas with -Wno-shift-negative-value
by Pino Toscano 09 Oct '17

09 Oct '17

The embedded copy of ocaml-augeas does Val_int(-1), which in turns triggers warnings in newer GCC versions about "left shift of negative value". The issue actually lies in the OCaml headers (mlvalues.h in particular), and it was fixed in newer OCaml versions. Since the code is actually correct, disable -Wshift-negative-value with -Wno-shift-negative-value (checking whether the compiler has it). --- common/mlaugeas/Makefile.am | 2 +- m4/guestfs-c.m4 | 6 ++++++ 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/common/mlaugeas/Makefile.am b/common/mlaugeas/Makefile.am index d71d0074d..0f3cea820 100644 --- a/common/mlaugeas/Makefile.am +++ b/common/mlaugeas/Makefile.am @@ -53,7 +53,7 @@ libmlaugeas_a_CPPFLAGS = \ -I$(top_builddir) \ -I$(shell $(OCAMLC) -where) libmlaugeas_a_CFLAGS = \ - $(WARN_CFLAGS) $(WERROR_CFLAGS) \ + $(WARN_CFLAGS) $(NO_SNV_CFLAGS) $(WERROR_CFLAGS) \ $(LIBVIRT_CFLAGS) $(LIBXML2_CFLAGS) \ -fPIC diff --git a/m4/guestfs-c.m4 b/m4/guestfs-c.m4 index 6621a2747..13310d5db 100644 --- a/m4/guestfs-c.m4 +++ b/m4/guestfs-c.m4 @@ -105,6 +105,12 @@ gl_WARN_ADD([-Wformat-truncation=1]) AC_SUBST([WARN_CFLAGS]) +NO_SNV_CFLAGS= +gl_COMPILER_OPTION_IF([-Wno-shift-negative-value],[ + NO_SNV_CFLAGS="-Wno-shift-negative-value" +]) +AC_SUBST([NO_SNV_CFLAGS]) + AC_DEFINE([lint], [1], [Define to 1 if the compiler is checking for lint.]) AC_DEFINE([GNULIB_PORTCHECK], [1], [Enable some gnulib portability checks.]) AH_VERBATIM([FORTIFY_SOURCE],[ -- 2.13.6

2 1

[PATCH v11 0/6] virt-builder-repository
by Cédric Bosdonnat 09 Oct '17

09 Oct '17

Hi there, This is an update of the series. Just to rebase it on top of Rich's latest changes. Cédric Bosdonnat (5): builder: rename docs test script builder: add a template parameter to get_index builder: add Index.write_entry function mllib: add XPath helper xpath_get_nodes() New tool: virt-builder-repository Pino Toscano (1): builder: add simple OCaml osinfo-db reader .gitignore | 5 + builder/Makefile.am | 128 ++++- builder/builder.ml | 2 +- builder/index.mli | 3 + builder/index_parser.ml | 80 ++- builder/index_parser.mli | 9 +- builder/index_parser_tests.ml | 129 +++++ builder/osinfo.ml | 80 +++ builder/osinfo.mli | 22 + builder/repository_main.ml | 597 +++++++++++++++++++++ .../{test-virt-builder-docs.sh => test-docs.sh} | 2 + builder/virt-builder-repository.pod | 213 ++++++++ common/mltools/xpath_helpers.ml | 9 + common/mltools/xpath_helpers.mli | 4 + 14 files changed, 1268 insertions(+), 15 deletions(-) create mode 100644 builder/index_parser_tests.ml create mode 100644 builder/osinfo.ml create mode 100644 builder/osinfo.mli create mode 100644 builder/repository_main.ml rename builder/{test-virt-builder-docs.sh => test-docs.sh} (93%) create mode 100644 builder/virt-builder-repository.pod -- 2.13.2

2 16

[PATCH] Fully initialize the custom_operations structs
by Pino Toscano 09 Oct '17

09 Oct '17

Use also custom_compare_ext_default for the compare_ext field. According to the git logs, this was introduced in OCaml 3.12.1, which is earlier than out minimum required version. mlaugeas is not touched by this change, since it is a copy of a 3rd party library (and thus it will be fixed there first). --- common/mlpcre/pcre-c.c | 3 ++- common/mlprogress/progress-c.c | 3 ++- common/mlxml/xml-c.c | 9 ++++++--- ocaml/guestfs-c.c | 3 ++- v2v/qemuopts-c.c | 3 ++- 5 files changed, 14 insertions(+), 7 deletions(-) diff --git a/common/mlpcre/pcre-c.c b/common/mlpcre/pcre-c.c index 61af616be..da982025f 100644 --- a/common/mlpcre/pcre-c.c +++ b/common/mlpcre/pcre-c.c @@ -97,7 +97,8 @@ static struct custom_operations custom_operations = { custom_compare_default, custom_hash_default, custom_serialize_default, - custom_deserialize_default + custom_deserialize_default, + custom_compare_ext_default, }; static value diff --git a/common/mlprogress/progress-c.c b/common/mlprogress/progress-c.c index ac6bbfa31..f4a83fd77 100644 --- a/common/mlprogress/progress-c.c +++ b/common/mlprogress/progress-c.c @@ -52,7 +52,8 @@ static struct custom_operations progress_bar_custom_operations = { custom_compare_default, custom_hash_default, custom_serialize_default, - custom_deserialize_default + custom_deserialize_default, + custom_compare_ext_default, }; value diff --git a/common/mlxml/xml-c.c b/common/mlxml/xml-c.c index 92388d940..45475f8f7 100644 --- a/common/mlxml/xml-c.c +++ b/common/mlxml/xml-c.c @@ -49,7 +49,8 @@ static struct custom_operations docptr_custom_operations = { custom_compare_default, custom_hash_default, custom_serialize_default, - custom_deserialize_default + custom_deserialize_default, + custom_compare_ext_default, }; value @@ -71,7 +72,8 @@ static struct custom_operations xpathctxptr_custom_operations = { custom_compare_default, custom_hash_default, custom_serialize_default, - custom_deserialize_default + custom_deserialize_default, + custom_compare_ext_default, }; value @@ -93,7 +95,8 @@ static struct custom_operations xpathobjptr_custom_operations = { custom_compare_default, custom_hash_default, custom_serialize_default, - custom_deserialize_default + custom_deserialize_default, + custom_compare_ext_default, }; value diff --git a/ocaml/guestfs-c.c b/ocaml/guestfs-c.c index d4ada3165..925f6188c 100644 --- a/ocaml/guestfs-c.c +++ b/ocaml/guestfs-c.c @@ -96,7 +96,8 @@ static struct custom_operations guestfs_custom_operations = { custom_compare_default, custom_hash_default, custom_serialize_default, - custom_deserialize_default + custom_deserialize_default, + custom_compare_ext_default, }; static value diff --git a/v2v/qemuopts-c.c b/v2v/qemuopts-c.c index 40b23f956..0f076d77d 100644 --- a/v2v/qemuopts-c.c +++ b/v2v/qemuopts-c.c @@ -51,7 +51,8 @@ static struct custom_operations qemuopts_custom_operations = { custom_compare_default, custom_hash_default, custom_serialize_default, - custom_deserialize_default + custom_deserialize_default, + custom_compare_ext_default, }; value -- 2.13.6

2 1

[PATCH nbdkit] tests/tls: fix qemu-img check
by Pino Toscano 09 Oct '17

09 Oct '17

Old qemu-img versions return 1 instead of 0 for --help, failing the presence check: hence, which to `command` for checking the presence of qemu-img. Also, check that qemu-img actually has the --object option, since the test uses it. --- tests/test-tls.sh | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/tests/test-tls.sh b/tests/test-tls.sh index 73d44fa..7a6c949 100755 --- a/tests/test-tls.sh +++ b/tests/test-tls.sh @@ -40,10 +40,14 @@ if ! ss --version; then echo "$0: 'ss' command not available" exit 77 fi -if ! qemu-img --help; then +if ! command -v qemu-img > /dev/null; then echo "$0: 'qemu-img' command not available" exit 77 fi +if ! qemu-img --help | grep -- --object; then + echo "$0: 'qemu-img' command does not have the --object option" + exit 77 +fi # Does the nbdkit binary support TLS? if ! nbdkit --dump-config | grep -sq tls=yes; then -- 2.13.6

2 1

Virtualbox vdi Input Format and man pages
by stef204 08 Oct '17

08 Oct '17

Hi, I am new to v2v/libguestfs. I need to convert a 30 GB virtual machine running Windows7 64 bit (a guest on a Linux system) from Virtualbox vdi format to qcow2 (or raw/img--another debate in itself) so I can use libvirt/qemu/kvm to run it and completely migrate away from Virtualbox. The vdi machine is a mission critical production environment and I cannot afford to mess it up, etc. Will keep original vdi intact until have successfully converted and tested. I have done some research and come upon v2v which seems to be just what the doctor ordered except that, in "local" man pages of version 1.36.4 which I have installed, there is NO mention of vdi compatibility as input format. However, on these online man pages <https://linux.die.net/man/1/virt-v2v> I see a section called "Local VirtualBox guests" which seems to indicate that vdi is indeed accepted as input format. Can anyone provide feedback as to the above? v2v seems to be a tool I could use to more easily deal with this conversion/migration than just using qemu-img convert and having to then deal with the other bits and pieces such as virtio drivers, etc. Thanks for any helpful feedback and pointers.

2 16

[PATCH 0/3] common/mlstdutils: Add Std_utils List and Option modules.
by Richard W.M. Jones 08 Oct '17

08 Oct '17

In Std_utils we already extend Char and String. These commits take it a little further by extending List and adding a new Option submodule. All basically simple refactoring. Rich.

1 4

[PATCH] lib: Pick up qemu-img path at build time.
by Richard W.M. Jones 06 Oct '17

06 Oct '17

The main purpose of this change is two-fold: (1) Ensure that we run the same version of qemu-img that we are built against. (2) Allow the qemu-img path to be overridden at build time in case it's on a nonstandard path or (like RHV) has a nonstandard name. --- docs/guestfs-building.pod | 6 ++++++ lib/command.c | 4 ++-- lib/create.c | 4 ++-- lib/info.c | 2 +- m4/guestfs-qemu.m4 | 4 ++++ 5 files changed, 15 insertions(+), 5 deletions(-) diff --git a/docs/guestfs-building.pod b/docs/guestfs-building.pod index d85483a0b..94df49158 100644 --- a/docs/guestfs-building.pod +++ b/docs/guestfs-building.pod @@ -716,6 +716,12 @@ binary to find the version of Python, the location of Python libraries and so on. See L</BUILDING PYTHON 2 AND PYTHON 3 BINDINGS> below. +=item B<QEMU_IMG> + +This environment variable may be set to point to the full path of the +L<qemu-img(1)> program, in case it is on a nonstandard path or has a +nonstandard name. + =item B<SUPERMIN> This environment variable can be set to choose an alternative diff --git a/lib/command.c b/lib/command.c index bfec76f19..3d8bc7dbf 100644 --- a/lib/command.c +++ b/lib/command.c @@ -35,7 +35,7 @@ * * I<Either> add arguments: * - * guestfs_int_cmd_add_arg (cmd, "qemu-img"); + * guestfs_int_cmd_add_arg (cmd, QEMU_IMG); * guestfs_int_cmd_add_arg (cmd, "info"); * guestfs_int_cmd_add_arg (cmd, filename); * @@ -48,7 +48,7 @@ * shell commands, with the added safety of allowing args to be quoted * properly). * - * guestfs_int_cmd_add_string_unquoted (cmd, "qemu-img info "); + * guestfs_int_cmd_add_string_unquoted (cmd, "file info "); * guestfs_int_cmd_add_string_quoted (cmd, filename); * * =item 4. diff --git a/lib/create.c b/lib/create.c index fff5cf332..b4865172c 100644 --- a/lib/create.c +++ b/lib/create.c @@ -314,7 +314,7 @@ disk_create_qcow2 (guestfs_h *g, const char *orig_filename, int64_t size, } /* Assemble the qemu-img command line. */ - guestfs_int_cmd_add_arg (cmd, "qemu-img"); + guestfs_int_cmd_add_arg (cmd, QEMU_IMG); guestfs_int_cmd_add_arg (cmd, "create"); guestfs_int_cmd_add_arg (cmd, "-f"); guestfs_int_cmd_add_arg (cmd, "qcow2"); @@ -347,7 +347,7 @@ disk_create_qcow2 (guestfs_h *g, const char *orig_filename, int64_t size, r = guestfs_int_cmd_run (cmd); if (!WIFEXITED (r) || WEXITSTATUS (r) != 0) { - guestfs_int_external_command_failed (g, r, "qemu-img", orig_filename); + guestfs_int_external_command_failed (g, r, QEMU_IMG, orig_filename); return -1; } diff --git a/lib/info.c b/lib/info.c index f7378adfd..4464df994 100644 --- a/lib/info.c +++ b/lib/info.c @@ -191,7 +191,7 @@ get_json_output (guestfs_h *g, const char *filename) snprintf (fdpath, sizeof fdpath, "/dev/fd/%d", fd); guestfs_int_cmd_clear_close_files (cmd); - guestfs_int_cmd_add_arg (cmd, "qemu-img"); + guestfs_int_cmd_add_arg (cmd, QEMU_IMG); guestfs_int_cmd_add_arg (cmd, "info"); guestfs_int_cmd_add_arg (cmd, "--output"); guestfs_int_cmd_add_arg (cmd, "json"); diff --git a/m4/guestfs-qemu.m4 b/m4/guestfs-qemu.m4 index 350bb980d..6ea7ef3d7 100644 --- a/m4/guestfs-qemu.m4 +++ b/m4/guestfs-qemu.m4 @@ -100,3 +100,7 @@ working. esac AC_DEFINE_UNQUOTED([QEMU],["$QEMU"],[Location of qemu binary.]) ]) + +dnl qemu-img (required). +AC_PATH_PROGS([QEMU_IMG],[qemu-img],[no]) +AC_DEFINE_UNQUOTED([QEMU_IMG],["$QEMU_IMG"],[Path to qemu-img program]) -- 2.13.2

2 2

[PATCH] inspector: Fix virt-inspector on *BSD guests (RHBZ#1144138).
by Richard W.M. Jones 06 Oct '17

06 Oct '17

--- inspector/inspector.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/inspector/inspector.c b/inspector/inspector.c index 3583c61df..30d279987 100644 --- a/inspector/inspector.c +++ b/inspector/inspector.c @@ -347,6 +347,7 @@ output_root (xmlTextWriterPtr xo, char *root) char buf[32]; char *canonical_root; size_t size; + int is_bsd; XMLERROR (-1, xmlTextWriterStartElement (xo, BAD_CAST "operatingsystem")); @@ -362,6 +363,10 @@ output_root (xmlTextWriterPtr xo, char *root) if (STRNEQ (str, "unknown")) XMLERROR (-1, xmlTextWriterWriteElement (xo, BAD_CAST "name", BAD_CAST str)); + is_bsd = + STREQ (str, "freebsd") || + STREQ (str, "netbsd") || + STREQ (str, "openbsd"); free (str); str = guestfs_inspect_get_arch (g, root); @@ -451,8 +456,13 @@ output_root (xmlTextWriterPtr xo, char *root) /* We need to mount everything up in order to read out the list of * applications and the icon, ie. everything below this point. + * + * XXX As a workaround for BSD guests, because the Linux kernel + * driver cannot just mount a UFS filesystem, we must disable this + * for all *BSD operating systems. We cannot read the apps or icon + * from *BSD anyway. */ - if (inspect_apps || inspect_icon) { + if ((inspect_apps || inspect_icon) && !is_bsd) { inspect_mount_root (g, root); if (inspect_apps) -- 2.13.2

2 6