January 2019 - Libguestfs - libguestfs List Archives

[PATCH] build: integrate ocaml-link.sh with automake silent rules
by Pino Toscano 18 Jan '19

18 Jan '19

In case either the silent rules are disabled, or V=1 is set, print the full command line that is executed. --- ocaml-link.sh.in | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/ocaml-link.sh.in b/ocaml-link.sh.in index 129c6fb0e..ba990f809 100755 --- a/ocaml-link.sh.in +++ b/ocaml-link.sh.in @@ -43,6 +43,14 @@ while true ; do esac done +# Integration with silent rules of automake: print the full command +# line option in verbose mode. +if [ x"${V:-@AM_DEFAULT_VERBOSITY@}" = x1 ]; then + echo "$@" \ + @OCAML_RUNTIME_VARIANT_PIC_OPTION@ \ + -linkpkg \ + -cclib "'@LDFLAGS@ $cclib'" +fi # NB -cclib must come last. exec "$@" \ @OCAML_RUNTIME_VARIANT_PIC_OPTION@ \ -- 2.20.1

2 1

ANNOUNCE: libguestfs 1.40 released
by Richard W.M. Jones 17 Jan '19

17 Jan '19

I'm pleased to announce libguestfs 1.40, a library and a set of tools for accessing and modifying virtual machine disk images. This release represents about a year of work by many contributors. I'd like to call out in particular substantial contributions from: Pino Toscano, Tomáš Golembiovský, Nir Soffer and Nikolay Ivanets. See the release notes below for full details. You can get libguestfs 1.40 from here: Main website: http://libguestfs.org/ [not updated yet] Source: http://libguestfs.org/download/1.40-stable/ Fedora: https://koji.fedoraproject.org/koji/packageinfo?packageID=8391 Debian/experimental: https://packages.debian.org/libguestfs0 Rich. ---------------------------------------------------------------------- Release notes for libguestfs 1.40 These are also available online at: http://libguestfs.org/guestfs-release-notes.1.html New features New features in existing tools Virt-inspector now displays the libosinfo short ID for guests (Pino Toscano). Guestfish -N will now generate 1G disks instead of 100M disks by default. Virt-resize supports f2fs filesystems (Pino Toscano). libguestfs-test-tool now supports bash tab completion (Pino Toscano). The --machine-readable option now supports sending output to files or stdout/stderr. This works uniformly across most OCaml-based virt tools, specifically: virt-builder, virt-builder-repository, virt-dib, virt-get-kernel, virt-resize, virt-sparsify, and virt-v2v (Pino Toscano). Virt-builder --uninstall option now works on SUSE (Sebastian Meyer). Virt-builder now supports Windows. We are not able to ship Windows templates in the public respository for obvious licensing reasons, but if you are an MSDN subscriber you may build your own. See https://rwmj.wordpress.com/2018/09/13/creating-windows-templates-for-virt-b… Many tools now support a --key option allowing you to pass in decryption keys for filesystems stored in local files on the host rather than having to feed them in over stdin (Pino Toscano). virt-v2v and virt-p2v New -o rhv-upload mode for directly uploading images to RHV, bypassing the Export Storage Domain (Tomáš Golembiovský, Nir Soffer, Daniel Erez, Pino Toscano). New -o openstack mode for directly uploading images to OpenStack and Cinder using OpenStack APIs. Virt-v2v now has a general mechanism for input and output options: -io and -oo. Virt-v2v can now install the RHV tools or QEMU GA in guests (Tomáš Golembiovský). The huge manual has now been split into several smaller documents and is more focused on helping users to accomplish specific v2v tasks. Multiple improvements to the OVF metadata when converting to RHV (Tomáš Golembiovský, Pino Toscano, Arik Hadas). Virt-v2v can now convert Linux guests with split kernel packages, especially Ubuntu 18.04 (Pino Toscano). Virt-v2v old --password-file option has been replaced by -ip (the old option remains for backwards compatibility). Virt-v2v now preserves the VM Generation ID. Virt-v2v has a new --mac option allowing specific NICs to be mapped precisely to networks or bridges on the target. New virt-v2v --print-estimate option to estimate the size of data that virt-v2v will copy. Virt-v2v is now usually able to remove open-vm-tools and VMware Tools from the Linux guest during conversion (Pino Toscano). Virt-v2v can now support UEFI conversions to RHV (Tomáš Golembiovský). Virt-p2v now prefers you to shut down instead of rebooting the machine after conversion has finished (Pino Toscano). Virt-v2v now writes the libosinfo short ID to the libvirt metadata when using -o local and -o libvirt output modes (Martin Kletzander). Language bindings OpenJDK 10+ is supported (Pino Toscano). Java bindings fixed for Gentoo host (Martin Kletzander). Inspection Support Kali Linux (Pino Toscano). When inspecting mountpoints, look for /etc/mdadm/mdadm.conf as well as /etc/mdadm.conf (Nikolay Ivanets). Improved support for OpenSUSE Tumbleweed (Pino Toscano). Architectures and platforms Miscellaneous macOS build fixes (Adam Robinson). Other Multiple documentation typos fixed (Yuri Chornoivan). Security See also guestfs-security(1). CVE-2018-11806 Qemu's slirp (userspace networking) had several buffer overflows which could be triggered from the guest or network side. API New APIs "f2fs_expand" Expand an f2fs filesystem (Pino Toscano). "inspect_get_osinfo" Get the libosinfo short ID for the inspected guest (Pino Toscano). "lvm_scan" This rescans all PVs, VGs and LVs, optionally activating them. Other API changes "part_get_mbr_part_type" was reimplemented in OCaml (Nikolay Ivanets). "list_filesystems" now filters out MBR extended partitions (Nikolay Ivanets). "vgscan" has been deprecated. Use the new "lvm_scan" API in preference. Build changes The JSON library switched from yajl to Jansson ≥ 2.7 (Pino Toscano). Support for The Sleuth Kit 4.5 (Pino Toscano). Multiple improvements in the "./configure" script (Lin Ma). More out-of-tree build fixes (Hilko Bengen). Some work on reproducible builds (Bernhard M. Wiedemann). Internals lvmetad is now used unconditionally, and is started much earlier in the appliance boot process. This fixes support for certain PV types. On Debian, dash is installed inside the appliance and in virt-rescue (Pino Toscano). The amount of RAM assigned to the appliance has been increased from 500M to 768M (on x86_64, other architectures have different limits). This enables use of up to 255 disks when using recent Linux kernels. Virt-v2v now models the source machine type (eg. i440FX, Q35 or virt). Virt-p2v now generates configuration code automatically, also code for parsing the kernel command line. Use "LT_INIT" instead of "AC_PROG_LIBTOOL". The OCaml "JSON" code for parsing and serializing has been unified into a single abstract data type. Most OCaml tools are now built using the PIC runtime. This has a performance penalty on i686, but is relatively free on other architectures and has security benefits. The direct backend now queries the qemu binary for availability of KVM, instead of using a heuristic based on /dev/kvm (Andrea Bolognani). Our "clever" libxml2 writer macros are now used consistently across all parts of the code base. Qemu dropped -nodefconfig without telling us. The equivalent -no-user-config option is now used instead. Qemu also moved the "serial" option from -drive to -device, again without any notice, and this is also fixed. Bugs fixed https://bugzilla.redhat.com/1664310 [RHEL 7.6 LP] openstack output leaks passwords https://bugzilla.redhat.com/1661038 virt-inspector fails with "error: int_of_string" on a Linux image when /etc/fstab contains a partionless device https://bugzilla.redhat.com/1651432 v2v - support for -insecure option to support OSP SSL connection for VM migrations https://bugzilla.redhat.com/1642044 [RFE] virt-p2v UI should give the option to shutdown after conversion is finished https://bugzilla.redhat.com/1634248 virt-v2v cannot convert opensuse15 guest https://bugzilla.redhat.com/1624902 Fix rhev-apt command that virt-v2v runs in Windows guests on first boot https://bugzilla.redhat.com/1624878 Update v2v docs to describe support for SHA 2 certs required for converting Windows 7 and 2008 R2 guests https://bugzilla.redhat.com/1615885 Minor log issue of virt-v2v https://bugzilla.redhat.com/1614276 VixDiskLib: VixDiskLib_Read: Read 4096 sectors at 57516160 failed. Error 2 (Memory allocation failed. Out of memory.) (DiskLib error 802: NBD_ERR_INSUFFICIENT_RESOURCES) at 5240. https://bugzilla.redhat.com/1612785 "warning: <target dev='sr0'> was ignored because the device name could not be recognized" should be hidden since convert cdrom is supported by virt-v2v https://bugzilla.redhat.com/1611690 part_to_dev "/dev/sdp1" returns "/dev/sd" instead of "/dev/sdp" https://bugzilla.redhat.com/1608131 inspection fails when swap partition in /etc/fstab has incorrect UUID compared to actual swap partition https://bugzilla.redhat.com/1605071 On machines where /dev/kvm exists but KVM doesn't work, libguestfs will not fall back to TCG https://bugzilla.redhat.com/1602353 virt-inspector can't inspect LUKS-encrypted RHEL7 guest image https://bugzilla.redhat.com/1601943 Improve error "No module named ovirtsdk4" in v2v rhv-upload conversion https://bugzilla.redhat.com/1598715 Adding QXL device in OVF causes Cirrus Logic device to be added to guest https://bugzilla.redhat.com/1598350 RFE: virt-v2v should preserve <genid> https://bugzilla.redhat.com/1596851 Transfer fails if local host is in maintenance mode https://bugzilla.redhat.com/1596810 Transfer fails if local host belongs to another DC https://bugzilla.redhat.com/1592468 v2v to RHV transfer fails with: error: [empty name]: cannot read '//*/disksection' with value: null https://bugzilla.redhat.com/1592061 virt-v2v convert rhel5.3 failure when executing rpm -ql kernel-2.6.18-128.el5 command https://bugzilla.redhat.com/1591789 p2v: error: XML error: CPU vendor specified without CPU model https://bugzilla.redhat.com/1590220 Hide rhv-upload option in virt-p2v client https://bugzilla.redhat.com/1588451 Don't use relative socket paths for NBD https://bugzilla.redhat.com/1588088 RFE: Support for -o rhv-upload via Unix domain socket https://bugzilla.redhat.com/1586198 ovirt-imageio-daemon times out during migration https://bugzilla.redhat.com/1580309 virt-v2v rhel7.6 build can't convert guest to null with qemu-kvm rhel7.5.z build https://bugzilla.redhat.com/1580292 Virt-v2v rhel7.6 build can't convert guest with qemu-kvm rhel7.5.z build https://bugzilla.redhat.com/1579047 virt-builder: ubuntu-18.04 template has stray HTTP proxy configured for apt https://bugzilla.redhat.com/1575640 virt-builder: error: host cpu (x86_64) and guest arch (unknown) are not compatible https://bugzilla.redhat.com/1570407 virt-v2v -i ova gives checksum error when missing files are mentioned in .mf https://bugzilla.redhat.com/1567763 guestfsd: error: /Windows/Drivers/VirtIO: Read-only file system https://bugzilla.redhat.com/1564983 The mount-vfs failed to mount the logical volume https://bugzilla.redhat.com/1561828 RHV tools not properly installed in migrated Windows VM's causing boot failure https://bugzilla.redhat.com/1559027 virt-v2v does not honor --network with -o rhv-upload https://bugzilla.redhat.com/1557273 [RFE] Upload images directly to oVirt (virt-v2v -o rhv-upload) https://bugzilla.redhat.com/1544842 [RFE] Add libosinfo short-id to virt-inspector output https://bugzilla.redhat.com/1544674 virt-inspector --help differs from manpage https://bugzilla.redhat.com/1544227 RFE: recognize the Kali Linux distribution https://bugzilla.redhat.com/1532224 Resume=/dev/sdaX can't be updated to resume=/dev/vdaX in rhel7 guest's grub file after v2v conversion https://bugzilla.redhat.com/1530967 Change categories of "Guest Operating system management by VIX API" in v2v man page https://bugzilla.redhat.com/1527334 Various -it vddk -io vddk* options should be documented, removed or fixed https://bugzilla.redhat.com/1148012 Reverted: RFE: Allow qemu-bridge-helper to be used to implement guestfs_set_network -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW

1 0

[PATCH 0/5] [RFC] builder: handle unavailable repos
by Pino Toscano 17 Jan '19

17 Jan '19

In case a repository of virt-builder references files (e.g. the index) that cannot be downloaded (network issues, 404, etc) then virt-builder errors out on this situation. This is not a nice situation, from an user POV. This series does some refactoring to allow to better handle downloading failures, and handle the failures gracefully in virt-builder. RFC because I'm not yet too convinced the approach I used (especially for the changes in the Curl module) is optimal, so looking for feedback on this. Pino Toscano (5): mltools: split helper do_check_exitcode mltools: create a new external_command_code mltools: add simple tests for external_command mltools: curl: turn Curl.run to raise exceptions builder: ignore repositories with download failures builder/builder.ml | 39 ++++++++++++++++++++--------- common/mltools/curl.ml | 15 ++++++++++- common/mltools/curl.mli | 3 +++ common/mltools/tools_utils.ml | 26 +++++++++---------- common/mltools/tools_utils.mli | 8 ++++++ common/mltools/tools_utils_tests.ml | 22 ++++++++++++++++ v2v/copy_to_local.ml | 9 ++++++- v2v/v2v.ml | 9 ++++++- 8 files changed, 103 insertions(+), 28 deletions(-) -- 2.20.1

2 13

[hivex PATCH] ruby: improve test functions
by Pino Toscano 17 Jan '19

17 Jan '19

Use better functions to check for proper values instead of assert: - refute_nil for non-null functions - assert_equal for checking equality Also, make sure that the parameters for assert_equal are correct: expected value, then got value. --- ruby/tests/tc_120_rlenvalue.rb | 4 ++-- ruby/tests/tc_130_special.rb | 12 ++++++------ ruby/tests/tc_200_write.rb | 4 ++-- ruby/tests/tc_210_setvalue.rb | 20 ++++++++++---------- 4 files changed, 20 insertions(+), 20 deletions(-) diff --git a/ruby/tests/tc_120_rlenvalue.rb b/ruby/tests/tc_120_rlenvalue.rb index 6c03f43..6a2fb72 100644 --- a/ruby/tests/tc_120_rlenvalue.rb +++ b/ruby/tests/tc_120_rlenvalue.rb @@ -37,7 +37,7 @@ class TestRLenValue < MiniTest::Unit::TestCase moderate_value_value = h.node_get_value(moderate_value_node, "33Bytes") r = h.value_data_cell_offset(moderate_value_value) - assert_equal(r[:len], 37) - assert_equal(r[:off], 8712) + assert_equal(37, r[:len]) + assert_equal(8712, r[:off]) end end diff --git a/ruby/tests/tc_130_special.rb b/ruby/tests/tc_130_special.rb index 282b288..7570824 100644 --- a/ruby/tests/tc_130_special.rb +++ b/ruby/tests/tc_130_special.rb @@ -13,17 +13,17 @@ class TestRLenValue < MiniTest::Unit::TestCase nodes = h.node_children (root) node = nodes.find { |n| h.node_name(n) == "abcd_äöüß" } - assert node != nil + refute_nil(node) value = h.node_values(node).find { |v| h.value_key(v) == "abcd_äöüß" } - assert value != nil + refute_nil(value) node = nodes.find { |n| h.node_name(n) == "zero\0key" } - assert node != nil + refute_nil(node) value = h.node_values(node).find { |v| h.value_key(v) == "zero\0val" } - assert value != nil + refute_nil(value) node = nodes.find { |n| h.node_name(n) == "weird™" } - assert node != nil + refute_nil(node) value = h.node_values(node).find { |v| h.value_key(v) == "symbols $£₤₧€" } - assert value != nil + refute_nil(value) end end diff --git a/ruby/tests/tc_200_write.rb b/ruby/tests/tc_200_write.rb index 327d0a8..463283e 100644 --- a/ruby/tests/tc_200_write.rb +++ b/ruby/tests/tc_200_write.rb @@ -23,12 +23,12 @@ class TestWrite < MiniTest::Unit::TestCase refute_nil (h) root = h.root() - assert (root) + refute_nil (root) h.node_add_child(root, "A") h.node_add_child(root, "B") b = h.node_get_child(root, "B") - assert (b) + refute_nil (b) values = [ { :key => "Key1", :type => 3, :value => "ABC" }, diff --git a/ruby/tests/tc_210_setvalue.rb b/ruby/tests/tc_210_setvalue.rb index 62ffd99..736b073 100644 --- a/ruby/tests/tc_210_setvalue.rb +++ b/ruby/tests/tc_210_setvalue.rb @@ -23,7 +23,7 @@ class TestSetValue < MiniTest::Unit::TestCase refute_nil (h) root = h.root() - assert (root) + refute_nil (root) h.node_add_child(root, "B") b = h.node_get_child(root, "B") @@ -42,21 +42,21 @@ class TestSetValue < MiniTest::Unit::TestCase val = h.node_get_value(b, "Key1") hash = h.value_value(val) - assert (hash[:type] == 3) - assert (hash[:value] == "JKL") - assert (hash[:len] == 3) + assert_equal(3, hash[:type]) + assert_equal("JKL", hash[:value]) + assert_equal(3, hash[:len]) val = h.node_get_value(b, "Key2") hash = h.value_value(val) - assert (hash[:type] == 2) - assert (hash[:value] == "DEF") - assert (hash[:len] == 3) + assert_equal(2, hash[:type]) + assert_equal("DEF", hash[:value]) + assert_equal(3, hash[:len]) val = h.node_get_value(b, "Key3") hash = h.value_value(val) - assert (hash[:type] == 3) - assert (hash[:value] == "GHI") - assert (hash[:len] == 3) + assert_equal(3, hash[:type]) + assert_equal("GHI", hash[:value]) + assert_equal(3, hash[:len]) # Don't actually commit here because that would overwrite # the original file. -- 2.20.1

2 1

[hivex PATCH] perl: fix format of croak for RLenValue
by Pino Toscano 17 Jan '19

17 Jan '19

When calling croak on failure in the Perl code for RLenValue, add the %s placeholder for the strerror result. This makes the croak call just like all the others. Fixes commit 87e1be8ff3431ac81d721d5f647fc85d02e045dc. --- generator/generator.ml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/generator/generator.ml b/generator/generator.ml index 0401217..f2cb627 100755 --- a/generator/generator.ml +++ b/generator/generator.ml @@ -2743,7 +2743,7 @@ DESTROY (h) name (String.concat ", " c_params); free_args (); pr " if (r == 0 && errno)\n"; - pr " croak (\"%%s: \", \"%s\", strerror (errno));\n" + pr " croak (\"%%s: %%s\", \"%s\", strerror (errno));\n" name; pr " EXTEND (SP, 2);\n"; pr " PUSHs (sv_2mortal (newSViv (len)));\n"; -- 2.20.1

2 1

[PATCH v2] v2v: -o rhv-upload: Allow configure to set the nbdkit Python version.
by Richard W.M. Jones 16 Jan '19

16 Jan '19

No functional change, but it does allow downstream distributions to adjust the nbdkit Python plugin used by virt-v2v -o rhv-upload mode: ./configure --with-virt-v2v-nbdkit-python-plugin=... --- .gitignore | 1 + configure.ac | 5 +++++ m4/guestfs-v2v.m4 | 28 ++++++++++++++++++++++++++++ v2v/Makefile.am | 4 ++++ v2v/config.ml.in | 20 ++++++++++++++++++++ v2v/config.mli | 26 ++++++++++++++++++++++++++ v2v/output_rhv_upload.ml | 2 +- 7 files changed, 85 insertions(+), 1 deletion(-) diff --git a/.gitignore b/.gitignore index fe5aa6d70..637bf7765 100644 --- a/.gitignore +++ b/.gitignore @@ -678,6 +678,7 @@ Makefile.in /utils/qemu-boot/qemu-boot /utils/qemu-speed-test/qemu-speed-test /v2v/.depend +/v2v/config.ml /v2v/oUnit-* /v2v/output_rhv_upload_*_source.ml /v2v/real-*.d/ diff --git a/configure.ac b/configure.ac index 143435b36..a94f3abab 100644 --- a/configure.ac +++ b/configure.ac @@ -162,6 +162,10 @@ m4_include([m4/guestfs-golang.m4]) HEADING([Checking for GObject Introspection]) m4_include([m4/guestfs-gobject.m4]) +dnl virt-v2v, virt-p2v. +HEADING([Checking the virt-v2v and virt-p2v dependencies]) +m4_include([m4/guestfs-v2v.m4]) + dnl Bash completion. HEADING([Checking for bash completion]) m4_include([m4/guestfs-bash-completion.m4]) @@ -365,6 +369,7 @@ AC_CONFIG_FILES([Makefile utils/qemu-boot/Makefile utils/qemu-speed-test/Makefile v2v/Makefile + v2v/config.ml v2v/test-harness/Makefile v2v/test-harness/META website/index.html]) diff --git a/m4/guestfs-v2v.m4 b/m4/guestfs-v2v.m4 new file mode 100644 index 000000000..dff8f124e --- /dev/null +++ b/m4/guestfs-v2v.m4 @@ -0,0 +1,28 @@ +# libguestfs +# Copyright (C) 2009-2019 Red Hat Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + +dnl Virt-v2v and virt-p2v. + +dnl nbdkit python plugin. +AC_MSG_CHECKING([for the nbdkit python plugin name]) +AC_ARG_WITH([virt-v2v-nbdkit-python-plugin], + [AS_HELP_STRING([--with-virt-v2v-nbdkit-python-plugin="python|..."], + [set nbdkit python plugin name used by virt-v2v @<:@default=python@:>@])], + [VIRT_V2V_NBDKIT_PYTHON_PLUGIN="$withval"], + [VIRT_V2V_NBDKIT_PYTHON_PLUGIN=python]) +AC_MSG_RESULT([$VIRT_V2V_NBDKIT_PYTHON_PLUGIN]) +AC_SUBST([VIRT_V2V_NBDKIT_PYTHON_PLUGIN]) diff --git a/v2v/Makefile.am b/v2v/Makefile.am index a156524ae..2312812fb 100644 --- a/v2v/Makefile.am +++ b/v2v/Makefile.am @@ -23,6 +23,7 @@ generator_built = \ BUILT_SOURCES = \ $(generator_built) \ + config.ml \ output_rhv_upload_createvm_source.ml \ output_rhv_upload_plugin_source.ml \ output_rhv_upload_precheck_source.ml @@ -48,6 +49,7 @@ EXTRA_DIST = \ SOURCES_MLI = \ changeuid.mli \ cmdline.mli \ + config.mli \ convert_linux.mli \ convert_windows.mli \ create_libvirt_xml.mli \ @@ -100,6 +102,7 @@ SOURCES_MLI = \ windows_virtio.mli SOURCES_ML = \ + config.ml \ types.ml \ uefi.ml \ utils.ml \ @@ -697,6 +700,7 @@ v2v_unit_tests_LINK = \ .depend: \ $(srcdir)/*.mli \ $(srcdir)/*.ml \ + config.ml \ output_rhv_upload_*_source.ml $(top_builddir)/ocaml-dep.sh $^ -include .depend diff --git a/v2v/config.ml.in b/v2v/config.ml.in new file mode 100644 index 000000000..240f60868 --- /dev/null +++ b/v2v/config.ml.in @@ -0,0 +1,20 @@ +(* virt-v2v + * Copyright (C) 2019 Red Hat Inc. + * @configure_input@ + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + *) + +let virt_v2v_nbdkit_python_plugin = "@VIRT_V2V_NBDKIT_PYTHON_PLUGIN@" diff --git a/v2v/config.mli b/v2v/config.mli new file mode 100644 index 000000000..b9f4d6225 --- /dev/null +++ b/v2v/config.mli @@ -0,0 +1,26 @@ +(* virt-v2v + * Copyright (C) 2019 Red Hat Inc. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + *) + +val virt_v2v_nbdkit_python_plugin : string +(** Return the name of the nbdkit python plugin used by + [virt-v2v -o rhv-upload]. + + As above this must also be the Python 3 version of the plugin, + unless you change it. The configure command to change this is: + + [./configure --with-virt-v2v-nbdkit-python-plugin=...] *) diff --git a/v2v/output_rhv_upload.ml b/v2v/output_rhv_upload.ml index c309e31e3..77c39107e 100644 --- a/v2v/output_rhv_upload.ml +++ b/v2v/output_rhv_upload.ml @@ -78,7 +78,7 @@ let parse_output_options options = { rhv_cafile; rhv_cluster; rhv_direct; rhv_verifypeer } -let nbdkit_python_plugin = "python" +let nbdkit_python_plugin = Config.virt_v2v_nbdkit_python_plugin let pidfile_timeout = 30 let finalization_timeout = 5*60 -- 2.20.1

2 1

[PATCH nbdkit incomplete 0/5] Port to Windows.
by Richard W.M. Jones 15 Jan '19

15 Jan '19

This is an incomplete port to Windows. Currently the server compiles and starts up successfully, but goes into an infinite loop when you connect to it. Nevertheless I think the approach is ready for feedback. This being Windows the changes go quite deep. Rich.

1 6

[PATCH] inspect: fix inspection of partition-less devices (RHBZ#1661038)
by Pino Toscano 14 Jan '19

14 Jan '19

When parsing "xdev"-kind devices, do not assume that the partition number can be converted to integer: re_xdev accepts an empty part of the partition number, so just handle as it is, as string. This fixes a regression due to the conversion of the inspection code to OCaml, as the old C version did not have this issue. --- daemon/inspect_fs_unix_fstab.ml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/daemon/inspect_fs_unix_fstab.ml b/daemon/inspect_fs_unix_fstab.ml index 9f9478144..02956ae94 100644 --- a/daemon/inspect_fs_unix_fstab.ml +++ b/daemon/inspect_fs_unix_fstab.ml @@ -350,7 +350,7 @@ and resolve_fstab_device spec md_map os_type = debug_matching "xdev"; let typ = PCRE.sub 1 and disk = PCRE.sub 2 - and part = int_of_string (PCRE.sub 3) in + and part = PCRE.sub 3 in resolve_xdev typ disk part default ) @@ -467,7 +467,7 @@ and resolve_fstab_device spec md_map os_type = debug_matching "Hurd"; let typ = PCRE.sub 1 and disk = int_of_string (PCRE.sub 2) - and part = int_of_string (PCRE.sub 3) in + and part = PCRE.sub 3 in (* Hurd disk devices are like /dev/hdNsM, where hdN is the * N-th disk and M is the M-th partition on that disk. @@ -504,7 +504,7 @@ and resolve_xdev typ disk part default = let i = drive_index disk in if i >= 0 && i < Array.length devices then ( let dev = Array.get devices i in - let dev = dev ^ string_of_int part in + let dev = dev ^ part in if is_partition dev then Mountable.of_device dev else -- 2.20.1

2 1

[PATCH 1/3] mlstdutils: add a very simple test for Std_utils.which
by Pino Toscano 14 Jan '19

14 Jan '19

--- common/mlstdutils/std_utils_tests.ml | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/common/mlstdutils/std_utils_tests.ml b/common/mlstdutils/std_utils_tests.ml index 81f512cbf..f7b0247a4 100644 --- a/common/mlstdutils/std_utils_tests.ml +++ b/common/mlstdutils/std_utils_tests.ml @@ -29,6 +29,11 @@ let assert_equal_int = assert_equal ~printer:(fun x -> string_of_int x) let assert_equal_int64 = assert_equal ~printer:(fun x -> Int64.to_string x) let assert_equal_stringlist = assert_equal ~printer:(fun x -> "(" ^ (String.escaped (String.concat "," x)) ^ ")") let assert_equal_stringpair = assert_equal ~printer:(fun (x, y) -> sprintf "%S, %S" x y) +let assert_nonempty_string str = + if str = "" then + assert_failure (sprintf "Expected empty string, got '%s'" str) +let assert_raises_executable_not_found exe = + assert_raises (Executable_not_found exe) (fun () -> which exe) (* Test Std_utils.int_of_X and Std_utils.X_of_int byte swapping * functions. @@ -140,6 +145,12 @@ let test_string_chomp ctx = assert_equal_string "" (String.chomp "\n"); assert_equal_string "\n" (String.chomp "\n\n") (* only removes one *) +(* Test Std_utils.which. *) +let test_which ctx = + assert_nonempty_string (which "true"); + assert_raises_executable_not_found "this-command-does-not-really-exist"; + () + (* Suites declaration. *) let suite = "mllib Std_utils" >::: @@ -154,6 +165,7 @@ let suite = "strings.lines_split" >:: test_string_lines_split; "strings.span" >:: test_string_span; "strings.chomp" >:: test_string_chomp; + "which" >:: test_which; ] let () = -- 2.17.2

2 3

[PATCH] v2v: -o rhv-upload: Allow configure commands to set the Python version.
by Richard W.M. Jones 12 Jan '19

12 Jan '19

No functional change, but it does allow downstream distributions to adjust the Python interpreter and nbdkit Python plugin used by virt-v2v -o rhv-upload mode: ./configure --with-virt-v2v-python-interpreter=... ./configure --with-virt-v2v-nbdkit-python-plugin=... If you don't set the configure parameters then this change makes no difference. --- .gitignore | 1 + configure.ac | 5 +++++ m4/guestfs-v2v.m4 | 38 ++++++++++++++++++++++++++++++++++++++ v2v/Makefile.am | 4 ++++ v2v/config.ml.in | 21 +++++++++++++++++++++ v2v/config.mli | 37 +++++++++++++++++++++++++++++++++++++ v2v/output_rhv_upload.ml | 18 +++++++++++++++--- v2v/python_script.ml | 15 +++++---------- v2v/python_script.mli | 14 +++++--------- 9 files changed, 131 insertions(+), 22 deletions(-) diff --git a/.gitignore b/.gitignore index fe5aa6d70..637bf7765 100644 --- a/.gitignore +++ b/.gitignore @@ -678,6 +678,7 @@ Makefile.in /utils/qemu-boot/qemu-boot /utils/qemu-speed-test/qemu-speed-test /v2v/.depend +/v2v/config.ml /v2v/oUnit-* /v2v/output_rhv_upload_*_source.ml /v2v/real-*.d/ diff --git a/configure.ac b/configure.ac index 143435b36..a94f3abab 100644 --- a/configure.ac +++ b/configure.ac @@ -162,6 +162,10 @@ m4_include([m4/guestfs-golang.m4]) HEADING([Checking for GObject Introspection]) m4_include([m4/guestfs-gobject.m4]) +dnl virt-v2v, virt-p2v. +HEADING([Checking the virt-v2v and virt-p2v dependencies]) +m4_include([m4/guestfs-v2v.m4]) + dnl Bash completion. HEADING([Checking for bash completion]) m4_include([m4/guestfs-bash-completion.m4]) @@ -365,6 +369,7 @@ AC_CONFIG_FILES([Makefile utils/qemu-boot/Makefile utils/qemu-speed-test/Makefile v2v/Makefile + v2v/config.ml v2v/test-harness/Makefile v2v/test-harness/META website/index.html]) diff --git a/m4/guestfs-v2v.m4 b/m4/guestfs-v2v.m4 new file mode 100644 index 000000000..3b85d6d54 --- /dev/null +++ b/m4/guestfs-v2v.m4 @@ -0,0 +1,38 @@ +# libguestfs +# Copyright (C) 2009-2019 Red Hat Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + +dnl Virt-v2v and virt-p2v. + +dnl Python interpreter and nbdkit python plugin. +dnl Note the name "python3" is defined by PEP 394. +AC_MSG_CHECKING([for the python interpreter used by virt-v2v]) +AC_ARG_WITH([virt-v2v-python-interpreter], + [AS_HELP_STRING([--with-virt-v2v-python-interpreter="python3|..."], + [set Python interpreter used by virt-v2v @<:@default=python3@:>@])], + [VIRT_V2V_PYTHON_INTERPRETER="$withval"], + [VIRT_V2V_PYTHON_INTERPRETER=python3]) +AC_MSG_RESULT([$VIRT_V2V_PYTHON_INTERPRETER]) +AC_SUBST([VIRT_V2V_PYTHON_INTERPRETER]) + +AC_MSG_CHECKING([for the nbdkit python plugin name]) +AC_ARG_WITH([virt-v2v-nbdkit-python-plugin], + [AS_HELP_STRING([--with-virt-v2v-nbdkit-python-plugin="python|..."], + [set nbdkit python plugin name used by virt-v2v @<:@default=python@:>@])], + [VIRT_V2V_NBDKIT_PYTHON_PLUGIN="$withval"], + [VIRT_V2V_NBDKIT_PYTHON_PLUGIN=python]) +AC_MSG_RESULT([$VIRT_V2V_NBDKIT_PYTHON_PLUGIN]) +AC_SUBST([VIRT_V2V_NBDKIT_PYTHON_PLUGIN]) diff --git a/v2v/Makefile.am b/v2v/Makefile.am index a156524ae..2312812fb 100644 --- a/v2v/Makefile.am +++ b/v2v/Makefile.am @@ -23,6 +23,7 @@ generator_built = \ BUILT_SOURCES = \ $(generator_built) \ + config.ml \ output_rhv_upload_createvm_source.ml \ output_rhv_upload_plugin_source.ml \ output_rhv_upload_precheck_source.ml @@ -48,6 +49,7 @@ EXTRA_DIST = \ SOURCES_MLI = \ changeuid.mli \ cmdline.mli \ + config.mli \ convert_linux.mli \ convert_windows.mli \ create_libvirt_xml.mli \ @@ -100,6 +102,7 @@ SOURCES_MLI = \ windows_virtio.mli SOURCES_ML = \ + config.ml \ types.ml \ uefi.ml \ utils.ml \ @@ -697,6 +700,7 @@ v2v_unit_tests_LINK = \ .depend: \ $(srcdir)/*.mli \ $(srcdir)/*.ml \ + config.ml \ output_rhv_upload_*_source.ml $(top_builddir)/ocaml-dep.sh $^ -include .depend diff --git a/v2v/config.ml.in b/v2v/config.ml.in new file mode 100644 index 000000000..f24cd1629 --- /dev/null +++ b/v2v/config.ml.in @@ -0,0 +1,21 @@ +(* virt-v2v + * Copyright (C) 2019 Red Hat Inc. + * @configure_input@ + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + *) + +let virt_v2v_python_interpreter = "@VIRT_V2V_PYTHON_INTERPRETER@" +let virt_v2v_nbdkit_python_plugin = "@VIRT_V2V_NBDKIT_PYTHON_PLUGIN@" diff --git a/v2v/config.mli b/v2v/config.mli new file mode 100644 index 000000000..327a68016 --- /dev/null +++ b/v2v/config.mli @@ -0,0 +1,37 @@ +(* virt-v2v + * Copyright (C) 2019 Red Hat Inc. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + *) + +val virt_v2v_python_interpreter : string +(** Return the name of the Python interpreter used to run scripts by + [virt-v2v -o rhv-upload]. + + This must normally be a Python 3 interpreter. However some + downstream distributions replace the Python 3 scripts with Python + 2 scripts and use the following configure option to adjust this + interpreter accordingly: + + [./configure --with-virt-v2v-python-interpreter=...] *) + +val virt_v2v_nbdkit_python_plugin : string +(** Return the name of the nbdkit python plugin used by + [virt-v2v -o rhv-upload]. + + As above this must also be the Python 3 version of the plugin, + unless you change it. The configure command to change this is: + + [./configure --with-virt-v2v-nbdkit-python-plugin=...] *) diff --git a/v2v/output_rhv_upload.ml b/v2v/output_rhv_upload.ml index c309e31e3..e0416a842 100644 --- a/v2v/output_rhv_upload.ml +++ b/v2v/output_rhv_upload.ml @@ -78,7 +78,8 @@ let parse_output_options options = { rhv_cafile; rhv_cluster; rhv_direct; rhv_verifypeer } -let nbdkit_python_plugin = "python" +let python_intepreter = Config.virt_v2v_python_interpreter +let nbdkit_python_plugin = Config.virt_v2v_nbdkit_python_plugin let pidfile_timeout = 30 let finalization_timeout = 5*60 @@ -97,21 +98,32 @@ class output_rhv_upload output_alloc output_conn (* Create Python scripts for precheck, plugin and create VM. *) let precheck_script = Python_script.create ~name:"rhv-upload-precheck.py" + ~python:python_intepreter Output_rhv_upload_precheck_source.code in let plugin_script = Python_script.create ~name:"rhv-upload-plugin.py" + ~python:python_intepreter Output_rhv_upload_plugin_source.code in let createvm_script = Python_script.create ~name:"rhv-upload-createvm.py" + ~python:python_intepreter Output_rhv_upload_createvm_source.code in (* Is SELinux enabled and enforcing on the host? *) let have_selinux = 0 = Sys.command "getenforce 2>/dev/null | grep -isq Enforcing" in + (* Check Python interpreter found. *) + let error_unless_python_interpreter_found () = + try ignore (which python_intepreter) + with Executable_not_found _ -> + error (f_"no python binary called ‘%s’ can be found on the $PATH") + python_intepreter + in + (* Check that the 'ovirtsdk4' Python module is available. *) let error_unless_ovirtsdk4_module_available () = - let res = run_command [ Python_script.python; "-c"; "import ovirtsdk4" ] in + let res = run_command [ python_intepreter; "-c"; "import ovirtsdk4" ] in if res <> 0 then error (f_"the Python module ‘ovirtsdk4’ could not be loaded, is it installed? See previous messages for problems.") in @@ -225,7 +237,7 @@ object inherit output method precheck () = - Python_script.error_unless_python_interpreter_found (); + error_unless_python_interpreter_found (); error_unless_ovirtsdk4_module_available (); error_unless_nbdkit_working (); error_unless_nbdkit_python_plugin_working (); diff --git a/v2v/python_script.ml b/v2v/python_script.ml index 3159373a1..9b7525b95 100644 --- a/v2v/python_script.ml +++ b/v2v/python_script.ml @@ -24,14 +24,15 @@ open Unix_utils open Common_gettext.Gettext -let python = "python3" (* Defined by PEP 394 *) +let python = Config.virt_v2v_python_interpreter type script = { tmpdir : string; (* Temporary directory. *) + python : string; (* Python interpreter. *) path : string; (* Path to script. *) } -let create ?(name = "script.py") code = +let create ?(name = "script.py") ~python code = let tmpdir = let base_dir = (open_guestfs ())#get_cachedir () in let t = Mkdtemp.temp_dir ~base_dir "v2v." in @@ -39,10 +40,10 @@ let create ?(name = "script.py") code = t in let path = tmpdir // name in with_open_out path (fun chan -> output_string chan code); - { tmpdir; path } + { tmpdir; python; path } let run_command ?echo_cmd ?stdout_fd ?stderr_fd - { tmpdir; path } params args = + { tmpdir; python; path } params args = let param_file = tmpdir // sprintf "params%d.json" (unique ()) in with_open_out param_file @@ -51,9 +52,3 @@ let run_command ?echo_cmd ?stdout_fd ?stderr_fd (python :: path :: param_file :: args) let path { path } = path - -let error_unless_python_interpreter_found () = - try ignore (which python) - with Executable_not_found _ -> - error (f_"no python binary called ‘%s’ can be found on the $PATH") - python diff --git a/v2v/python_script.mli b/v2v/python_script.mli index c008eec1a..580d16381 100644 --- a/v2v/python_script.mli +++ b/v2v/python_script.mli @@ -20,12 +20,15 @@ type script -val create : ?name:string -> string -> script +val create : ?name:string -> python:string -> string -> script (** Create a Python script object. The optional parameter [?name] is a hint for the name of the script. - The parameter is the Python code. Usually this is + The named parameter is the Python interpreter to use. Usually + you would use {!Config.virt_v2v_python_interpreter}. + + The unnamed parameter is the Python code. Usually this is [Some_source.code] where [some_source.ml] is generated from the Python file by [v2v/embed.sh] (see also [v2v/Makefile.am]). *) @@ -54,10 +57,3 @@ val path : script -> string try using/storing it beyond the lifetime of the program. This is used only where {!run_command} is not suitable. *) - -val python : string -(** Return the name of the Python interpreter. *) - -val error_unless_python_interpreter_found : unit -> unit -(** Check if the Python interpreter can be found on the path, and - call [error] (which is fatal) if not. *) -- 2.19.2

2 2