April 2019 - Libguestfs - libguestfs List Archives

[PATCH] v2v: start reading the new libvirt firmware autoselect
by Pino Toscano 10 Apr '19

10 Apr '19

Starting with 5.2.0, libvirt has a way to select automatically the firmware for a guest using an attribute of the <os> tag. Hence, use this information (when available, of course) to flag the firmware used by the guest. --- v2v/parse_libvirt_xml.ml | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/v2v/parse_libvirt_xml.ml b/v2v/parse_libvirt_xml.ml index d5d78d367..9cf4c496b 100644 --- a/v2v/parse_libvirt_xml.ml +++ b/v2v/parse_libvirt_xml.ml @@ -476,6 +476,13 @@ let parse_libvirt_xml ?conn xml = done; List.rev !nics in + (* Firmware. *) + let firmware = + match xpath_string "/domain/os/@firmware" with + | Some "bios" -> BIOS + | Some "efi" -> UEFI + | None | Some _ -> UnknownFirmware in + (* Check for hostdev devices. (RHBZ#1472719) *) let () = let obj = Xml.xpath_eval_expression xpathctx "/domain/devices/hostdev" in @@ -502,7 +509,7 @@ let parse_libvirt_xml ?conn xml = s_cpu_model = cpu_model; s_cpu_topology = cpu_topology; s_features = features; - s_firmware = UnknownFirmware; (* XXX until RHBZ#1217444 is fixed *) + s_firmware = firmware; s_display = display; s_video = video; s_sound = sound; -- 2.20.1

2 1

[PATCH] inspect: get icon of OpenMandriva guests
by Pino Toscano 10 Apr '19

10 Apr '19

Followup of commit a4ef6716b4119e476949c55f1bc12d81e735f0c0. --- lib/inspect-icon.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/lib/inspect-icon.c b/lib/inspect-icon.c index 5c7da0476..725af574b 100644 --- a/lib/inspect-icon.c +++ b/lib/inspect-icon.c @@ -67,6 +67,7 @@ static char *icon_cirros (guestfs_h *g, size_t *size_r); static char *icon_voidlinux (guestfs_h *g, size_t *size_r); static char *icon_altlinux (guestfs_h *g, size_t *size_r); static char *icon_gentoo (guestfs_h *g, size_t *size_r); +static char *icon_openmandriva (guestfs_h *g, size_t *size_r); #if CAN_DO_WINDOWS static char *icon_windows (guestfs_h *g, const char *root, size_t *size_r); #endif @@ -170,6 +171,9 @@ guestfs_impl_inspect_get_icon (guestfs_h *g, const char *root, size_t *size_r, else if (STREQ (distro, "gentoo")) { r = icon_gentoo (g, &size); } + else if (STREQ (distro, "openmandriva")) { + r = icon_openmandriva (g, &size); + } } else if (STREQ (type, "windows")) { #if CAN_DO_WINDOWS @@ -453,6 +457,18 @@ icon_gentoo (guestfs_h *g, size_t *size_r) return get_png (g, GENTOO_ICON, size_r, 10240); } +static char * +icon_openmandriva (guestfs_h *g, size_t *size_r) +{ + const char *icons[] = { + "/usr/share/icons/large/mandriva.png", + "/usr/share/icons/mandriva.png", + NULL + }; + + return find_png (g, icons, size_r, 10240); +} + #if CAN_DO_WINDOWS /* Windows, as usual, has to be much more complicated and stupid than -- 2.20.1

2 1

ANNOUNCE: nbdkit 1.12 - an NBD server toolkit with stable plugin API and permissive license
by Richard W.M. Jones 10 Apr '19

10 Apr '19

I’m pleased to announce the next stable release of nbdkit. This release concentrates on numerous feature enhancements - see the release notes below. NBD — Network Block Device — is a protocol for accessing Block Devices (hard disks and disk-like things) over a Network. nbdkit is a toolkit for creating NBD servers. The key features are: * Multithreaded NBD server written in C with good performance. * Minimal dependencies for the basic server. * Liberal license (BSD) allows nbdkit to be linked to proprietary libraries or included in proprietary code. * Well-documented, simple plugin API with a stable ABI guarantee. Lets you export “unconventional” block devices easily. * You can write plugins in C, Lua, Perl, Python, OCaml, Ruby, [new!] Rust, shell script or Tcl. * Filters can be stacked in front of plugins to transform the output. Clone source: https://github.com/libguestfs/nbdkit Download tarball: http://download.libguestfs.org/nbdkit/1.12-stable/ Other downloads: http://download.libguestfs.org/nbdkit/ Fedora packages: https://koji.fedoraproject.org/koji/packageinfo?packageID=16469 Release notes: New ‘ssh’ plugin which lets you turn any disk image on a remote Unix server accessible by ssh into an NBD source. New ‘rate’ filter for bandwidth-limiting connections. Either single connections or the whole server can be bandwidth-limited, and the limit can be dynamically changed at runtime. New ‘readahead’ filter improves performance when reading sequentially from plugins with a high request overhead like curl. New ‘linuxdisk’ plugin lets you create complete ext2, ext3 or ext4 filesystems in a GPT partitioned disk image, from a local directory. This can be attached to existing Linux VMs or used to create new ones from scratch. New ‘noextents’ filter can make sparse disks appear fully allocated. Plugins can now be written in the Rust programming language. The old ‘xz’ plugin has been removed. Use ‘nbdkit --filter=xz file file.xz’ as a replacement. This filter can be placed on top of other plugins such as curl (for decompressing a remote web URL). NBD_CMD_BLOCK_STATUS has been implemented allowing clients to query which parts of the disk are allocated, holes or zeroes. Plugins and filters have been extended to supply this information in many cases. This required adding minimal support for NBD protocol Structured Replies, NBD_CMD_FLAG_REQ_ONE and NBD_CMD_FLAG_DF. (Thanks to Eric Blake for extensive help with this one.) The ‘vddk’ plugin has been updated to VDDK 6.7, but now also works back as far as VDDK 5.1.1. It supports extents, flush and FUA. Also added: new flags single-link and unbuffered. (Thanks to Martin Kletzander). The line "All rights reserved." was removed from the license, with the agreement of all nbdkit contributors. Note this does not change the license. ‘nbdkit_error’ errors are printed in red when writing to the terminal, making them much easier to see. ‘nbdkit_parse_size’ rejects negative values, eg. ‘nbdkit memory size=-100’ (Nikolay Ivanets). The ‘curl’ plugin now supports cookies, password auth and proxies. It can also limit URLs to whitelisted protocols. This makes it broadly feature equivalent to and a replacement for the qemu curl block driver. The ‘partitioning’ plugin can now create MBR logical partitions. Previously it only supported 4 MBR primary partitions and if you wanted more than that you had to use GPT. The ‘partition’ filter also supports MBR logical partitions. The ‘python’ plugin creates plugins which are compatible with Python >= 3.7. Tests are now run under MALLOC_CHECK_=1 and MALLOC_PERTURB_=<random>. This causes some tests to crash (silently) because of a shutdown race which needs to be fixed. Test shell scripts now have a generic ‘requires’ function to skip tests if features are not present. C99 type ‘bool’ is now used in most places where appropriate. The protocol and connections code in the server has been split up because the single file had grown very large. (Thanks to Eric Blake for helping with protocol issues). Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW

1 0

[PATCH] v2v: Implement the --bandwidth* options to control network bandwidth.
by Richard W.M. Jones 09 Apr '19

09 Apr '19

This is built on top of the following patch series: https://www.redhat.com/archives/libguestfs/2019-April/msg00054.html Rich.

1 1

[PATCH nbdkit] vddk: Add support for VIXDISKLIB_FLAG_OPEN_SINGLE_LINK
by Richard W.M. Jones 06 Apr '19

06 Apr '19

From: Martin Kletzander <mkletzan(a)redhat.com> Signed-off-by: Martin Kletzander <mkletzan(a)redhat.com> --- plugins/vddk/nbdkit-vddk-plugin.pod | 7 ++++++- plugins/vddk/vddk-structs.h | 1 + plugins/vddk/vddk.c | 10 ++++++++++ 3 files changed, 17 insertions(+), 1 deletion(-) diff --git a/plugins/vddk/nbdkit-vddk-plugin.pod b/plugins/vddk/nbdkit-vddk-plugin.pod index 586aab4..25a6511 100644 --- a/plugins/vddk/nbdkit-vddk-plugin.pod +++ b/plugins/vddk/nbdkit-vddk-plugin.pod @@ -8,7 +8,7 @@ nbdkit-vddk-plugin - nbdkit VMware VDDK plugin [vm=moref=ID] [server=HOSTNAME] [user=USERNAME] [password=PASSWORD | password=- | password=+FILENAME] [cookie=COOKIE] [thumbprint=THUMBPRINT] - [port=PORT] [nfchostport=PORT] + [port=PORT] [nfchostport=PORT] [single-link=true] [snapshot=MOREF] [transports=MODE:MODE:...] nbdkit vddk --dump-plugin @@ -128,6 +128,11 @@ Optional. The port on the VCenter/ESXi host. Defaults to 443. Optional (required for remote connections). The hostname or IP address of VCenter or ESXi host. +=item B<single-link=true> + +Optional. Open the current link, not the entire chain. This +corresponds to the C<VIXDISKLIB_FLAG_OPEN_SINGLE_LINK> flag. + =item B<snapshot=>MOREF Optional. The Managed Object Reference of the snapshot. diff --git a/plugins/vddk/vddk-structs.h b/plugins/vddk/vddk-structs.h index 24b3259..bc68ac6 100644 --- a/plugins/vddk/vddk-structs.h +++ b/plugins/vddk/vddk-structs.h @@ -43,6 +43,7 @@ typedef uint64_t VixError; #define VIX_OK 0 +#define VIXDISKLIB_FLAG_OPEN_SINGLE_LINK 2 #define VIXDISKLIB_FLAG_OPEN_READ_ONLY 4 #define VIXDISKLIB_SECTOR_SIZE 512 diff --git a/plugins/vddk/vddk.c b/plugins/vddk/vddk.c index 8ea923a..78a4c88 100644 --- a/plugins/vddk/vddk.c +++ b/plugins/vddk/vddk.c @@ -91,6 +91,7 @@ static int nfc_host_port = 0; /* nfchostport */ static char *password = NULL; /* password */ static int port = 0; /* port */ static const char *server_name = NULL; /* server */ +static bool single_link = false; /* single-link */ static const char *snapshot_moref = NULL; /* snapshot */ static const char *thumb_print = NULL; /* thumbprint */ static const char *transport_modes = NULL; /* transports */ @@ -272,6 +273,13 @@ vddk_config (const char *key, const char *value) else if (strcmp (key, "vm") == 0) { vmx_spec = value; } + else if (strcmp (key, "single-link") == 0) { + int r = nbdkit_parse_bool (value); + + if (r == -1) + return -1; + single_link = r; + } else { nbdkit_error ("unknown parameter '%s'", key); return -1; @@ -464,6 +472,8 @@ vddk_open (int readonly) flags = 0; if (readonly) flags |= VIXDISKLIB_FLAG_OPEN_READ_ONLY; + if (single_link) + flags |= VIXDISKLIB_FLAG_OPEN_SINGLE_LINK; DEBUG_CALL ("VixDiskLib_Open", "connection, %s, %d, &handle", filename, flags); -- 2.20.1

2 3

[PATCH nbdkit] vddk: Add support for VIXDISKLIB_FLAG_OPEN_UNBUFFERED.
by Richard W.M. Jones 05 Apr '19

05 Apr '19

I suppose we may as well implement the only other flag too ... It's not clear what this does, something like O_DIRECT I imagine. Rich.

1 1

cloud-init in virt-builder images
by Jonathan Wright 03 Apr '19

03 Apr '19

I've noticed that none of the images from virt-builder have cloud-init in them by default. The documentation reads to me as if the OS vendor's official cloud images are the ones used as the sources for the generated images but I'm assuming this is wrong. I know for a fact CentOS's cloud images from https://cloud.centos.org/centos/7/images/ and Ubuntu's from https://cloud-images.ubuntu.com/bionic/ have cloud-init installed and enabled already. Where are the images sourced from if not these official images? I understand I can install cloud-init with virt-builder but I'm trying to figure out what OS-distributed images are in use so I may determine what else is different from expectations. -- Jonathan Wright KnownHost, LLC https://www.knownhost.com

2 2

[PATCH] Add missing python bindings tests
by Hiroyuki Katsura 03 Apr '19

03 Apr '19

Signed-off-by: Hiroyuki Katsura <hiroyuki.katsura.0513(a)gmail.com> --- python/t/test020Create.py | 25 ++++++++++++++ python/t/test030CreateFlags.py | 27 +++++++++++++++ python/t/test040CreateMultiple.py | 32 ++++++++++++++++++ python/t/test050HandleProperties.py | 50 +++++++++++++++++++++++++++ python/t/test430ProgressMessages.py | 52 +++++++++++++++++++++++++++++ 5 files changed, 186 insertions(+) create mode 100644 python/t/test020Create.py create mode 100644 python/t/test030CreateFlags.py create mode 100644 python/t/test040CreateMultiple.py create mode 100644 python/t/test050HandleProperties.py create mode 100644 python/t/test430ProgressMessages.py diff --git a/python/t/test020Create.py b/python/t/test020Create.py new file mode 100644 index 000000000..476ffd059 --- /dev/null +++ b/python/t/test020Create.py @@ -0,0 +1,25 @@ +# libguestfs Python bindings +# Copyright (C) 2019 Red Hat Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License along +# with this program; if not, write to the Free Software Foundation, Inc., +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. +#) + +import unittest +import guestfs + + +class Test020Create(unittest.TestCase): + def test_create(self): + _ = guestfs.GuestFS(python_return_dict=True) diff --git a/python/t/test030CreateFlags.py b/python/t/test030CreateFlags.py new file mode 100644 index 000000000..2e9b48203 --- /dev/null +++ b/python/t/test030CreateFlags.py @@ -0,0 +1,27 @@ +# libguestfs Python bindings +# Copyright (C) 2019 Red Hat Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License along +# with this program; if not, write to the Free Software Foundation, Inc., +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. +#) + +import unittest +import guestfs + + +class Test030CreateFlags(unittest.TestCase): + def test_create_flags(self): + g = guestfs.GuestFS(python_return_dict=True, + environment=False) + g.parse_environment() diff --git a/python/t/test040CreateMultiple.py b/python/t/test040CreateMultiple.py new file mode 100644 index 000000000..cc0050695 --- /dev/null +++ b/python/t/test040CreateMultiple.py @@ -0,0 +1,32 @@ +# libguestfs Python bindings +# Copyright (C) 2019 Red Hat Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License along +# with this program; if not, write to the Free Software Foundation, Inc., +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. +#) + +import unittest +import guestfs + + +def ignore(_): + pass + + +class Test040CreateMultiple(unittest.TestCase): + def test_create_multiple(self): + g1 = guestfs.GuestFS(python_return_dict=True) + g2 = guestfs.GuestFS(python_return_dict=True) + g3 = guestfs.GuestFS(python_return_dict=True) + ignore((g1, g2, g3)) diff --git a/python/t/test050HandleProperties.py b/python/t/test050HandleProperties.py new file mode 100644 index 000000000..fc053d779 --- /dev/null +++ b/python/t/test050HandleProperties.py @@ -0,0 +1,50 @@ + +# libguestfs Python bindings +# Copyright (C) 2019 Red Hat Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License along +# with this program; if not, write to the Free Software Foundation, Inc., +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. +#) + +import unittest +import guestfs + + +class Test050HandleProperties(unittest.TestCase): + def test_verbose(self): + g = guestfs.GuestFS(python_return_dict=True) + g.set_verbose(1) + self.assertEqual(g.get_verbose(), 1) + g.set_verbose(0) + self.assertEqual(g.get_verbose(), 0) + + def test_autosync(self): + g = guestfs.GuestFS(python_return_dict=True) + g.set_autosync(1) + self.assertEqual(g.get_autosync(), 1) + g.set_autosync(0) + self.assertEqual(g.get_autosync(), 0) + + def test_path(self): + g = guestfs.GuestFS(python_return_dict=True) + g.set_path(".") + self.assertEqual(g.get_path(), ".") + + def test_add_drive(self): + g = guestfs.GuestFS(python_return_dict=True) + g.add_drive("/dev/null") + + def test_add_cdrom(self): + g = guestfs.GuestFS(python_return_dict=True) + g.add_cdrom("/dev/zero") diff --git a/python/t/test430ProgressMessages.py b/python/t/test430ProgressMessages.py new file mode 100644 index 000000000..79e0491ff --- /dev/null +++ b/python/t/test430ProgressMessages.py @@ -0,0 +1,52 @@ +# libguestfs Python bindings +# Copyright (C) 2019 Red Hat Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + +import unittest +import os +import guestfs + +callback_invoked = 0 + + +def callback(ev, eh, buf, array): + global callback_invoked + callback_invoked += 1 + + +class Test430ProgressMessages(unittest.TestCase): + def test_progress_messages(self): + global callback_invoked + g = guestfs.GuestFS(python_return_dict=True) + g.add_drive('/dev/null') + g.launch() + + events = guestfs.EVENT_PROGRESS + + eh = g.set_event_callback(callback, events) + g.debug('progress', ['5']) + self.assertTrue(callback_invoked > 0) + + callback_invoked = 0 + g.delete_event_callback(eh) + g.debug('progress', ['5']) + self.assertEqual(callback_invoked, 0) + + g.set_event_callback(callback, events) + g.debug('progress', ['5']) + self.assertTrue(callback_invoked > 0) + + g.close() -- 2.17.1

2 1

[PATCH 0/5] Small inspection improvements
by Pino Toscano 03 Apr '19

03 Apr '19

Few improvements to the results of the inspection on some distros. Pino Toscano (5): inspect: factorize list of rolling distros inspect: detect Gentoo from os-release inspect: fully detect Arch Linux from os-release inspect: return osinfo short IDs for rolling distros inspect: correct osinfo ID for ALT Linux >= 8 daemon/inspect_fs_unix.ml | 15 +++++++++------ inspector/expected-archlinux.img.xml | 1 + lib/inspect-osinfo.c | 8 ++++++++ 3 files changed, 18 insertions(+), 6 deletions(-) -- 2.20.1

2 6

[PATCH] UNTESTED v2v: openstack: Read server-id from metadata service.
by Richard W.M. Jones 03 Apr '19

03 Apr '19

Random old patch that I had in my queue. Posting it as a backup, it is still untested. Rich.

1 1