[PATCH v2v] -o rhv-upload: Give a nicer error if the storage domain
by Richard W.M. Jones
https://bugzilla.redhat.com/show_bug.cgi?id=1986386
My RHV instance is dead at the moment so I didn't do much more than
check this compiles and passes the one test we have. Also I want to
spend as little time as possible on RHV outputs for virt-v2v since the
RHV product will be discontinued soon.
I did want to point out some things:
- The preceeding code is probably wrong.
https://github.com/libguestfs/virt-v2v/blob/master/output/rhv-upload-tran...
It attempts to search for the output storage using:
storage_domains = system_service.storage_domains_service().list(
search='name=%s' % params['output_storage'],
case_sensitive=True,
)
I couldn't find any documentation about what can go into that
search string, but it's clearly a lot more complicated than just
pasting in the literal name after "name=". At the very least,
spaces are not permitted, see:
https://github.com/libguestfs/virt-v2v/blob/master/output/rhv-upload-tran...
- The bug reporter used "data*" as the name and I suspect that is
parsed in some way (wildcard? regexp? I've no idea).
- Probably for the same reason, the preceeding code ought to fail
with an error if the output storage domain doesn't exist. The fact
we reach the code patched here at all also indicates some bug,
maybe in the search string.
As I say above, I don't especially care about any of this.
Rich.
1 year, 9 months
[p2v PATCH 0/4] RHEL9 build host tweaks
by Laszlo Ersek
In preparation for some GUI investigation / work, I've run "make
run-virt-p2v-in-a-vm" for the first time on my new RHEL-9.1 laptop. Some
tweaks appear to be necessary (or at least useful).
Laszlo
Laszlo Ersek (4):
ssh: shield virt-v2v from bash RC files
make-disk: restrict kernel update bug work-around
make-disk: replace here-docs with printfs for better indentation
make-disk: inject EPEL9 repo in RHEL9 disk image
ssh.c | 2 +-
test-virt-p2v-ssh.sh | 2 +-
virt-p2v-make-disk.in | 57 +++++++++++++++++++++++++++++++------------
3 files changed, 44 insertions(+), 17 deletions(-)
1 year, 9 months
[PATCH nbdkit] New plugin: blkio [incomplete]
by Richard W.M. Jones
This is an incomplete outline implementation for a libblkio plugin for
nbdkit. At the moment it only supports reading the same ("capacity")
of the device, and not even reading or writing. I have some questions
about the libblkio API before I can complete the plugin (see below).
The idea here is that by connecting libblkio to NBD we can use the
existing set of scripting tools to script access to devices. For
example you could use Python to read or modify a device:
----------------------------------------------------------------------
$ nbdsh -c - <<'EOF'
from subprocess import *
# Run nbdkit-blkio-plugin.
h.connect_systemd_socket_activation ("nbdkit" "blkio",
"virtio-blk-vhost-user",
"path=unix.sock")
print("device size=%d", h.get_size())
# Dump the boot sector.
bootsect = h.pread(512, 0)
p = Popen("hexdump -C", shell=True, stdin=PIPE)
p.stdin.write(bootsect)
EOF
----------------------------------------------------------------------
So my questions and comments about libblkio:
(1) There is no way to know which properties are readable, writable,
and those which need to be set before or after blkio_connect (see
is_preconnect_property in the plugin). It should be possible to
introspect this information. Also might be nice to be able read a
list of all available properties.
(2) It would be nice if libblkio had a way to enable debugging and
call the caller back for debug messages. We could then redirect the
callbacks into the nbdkit debug API (nbdkit_debug()) where they would
eventually end up on stderr or syslog.
However don't send debug messages to stderr, or at least allow that
behaviour to be overridden.
(3) It seems like some drivers require pre-allocated memory regions,
and since some do that means we might as well implement this. It
also seems like some drivers require file-backed pre-allocated
memory regions, and so we might as well implement that too.
However what is not clear: does memfd_create(2) always allocate
sufficiently aligned memory regions, such that we never need to bother
reading the mem-region-alignment property?
I notice that the example:
https://gitlab.com/libblkio/libblkio/-/blob/main/examples/blkio-copy.c
just passes on this and calls blkio_alloc_mem_region(). Is that the
safest and easiest thing to do which will always work?
(4) As a first pass, I only want to bother implementing blocking mode.
It'll be slow, but it doesn't matter for this application. Also I've
chosen nbdkit's NBDKIT_THREAD_MODEL_SERIALIZE_ALL_REQUESTS so nbdkit
will serialise all requests (again, only for a very simple first pass).
Looking at: https://libblkio.gitlab.io/libblkio/blkio.html#performing-i-o
seems simple enough but:
(4a) What is the queue number? Always 0? Is it affected by num-entries?
(4b) It's unclear how completions work. If I set min=max=1, will it
return after the whole operation has completed? Do I need to
call it again? What about if the request is very large, can it
get split?
Reading the example
https://gitlab.com/libblkio/libblkio/-/blob/main/examples/blkio-copy.c
it appears that requests cannot ever be split?
Rich.
1 year, 9 months
[nbdkit PATCH] data: don't ignore SIGPIPE
by Thomas Weißschuh
If all of the requested data has been read, the call to popen_close will
close the scripts output pipe and on the next write a SIGPIPE is
delivered.
When the scripts inherited signalhandler is ignoring the singal, it
won't be aborted by default.
This happens in the test suite, as make 4.4 seems to ignore this signal.
Therefore the executed scripts never stop and the testsuite never
finishes.
---
plugins/data/format.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/plugins/data/format.c b/plugins/data/format.c
index 3667ead6790f..2319ea09475c 100644
--- a/plugins/data/format.c
+++ b/plugins/data/format.c
@@ -40,6 +40,7 @@
#include <stdarg.h>
#include <string.h>
#include <assert.h>
+#include <signal.h>
#define NBDKIT_API_VERSION 2
#include <nbdkit-plugin.h>
@@ -1951,6 +1952,9 @@ store_script_len (struct allocator *a,
char buf[BUFSIZ];
size_t n;
+ /* Restore SIGPIPE back to SIG_DFL, since shell can't undo SIG_IGN */
+ signal (SIGPIPE, SIG_DFL);
+
pp = popen (script, "r");
if (pp == NULL) {
nbdkit_error ("popen: %m");
base-commit: 09a61a0a00d3036ebfbc2fbadb0ba3e10a778ff3
--
2.39.0
1 year, 9 months
Cargo edition problem with Debian 11
by Richard W.M. Jones
https://gitlab.com/nbdkit/nbdkit/-/jobs/3598390121
----------------------------------------------------------------------
cargo build --release --example ramdisk
Downloading crates ...
Downloaded float-cmp v0.9.0
Downloaded downcast v0.11.0
Downloaded mockall_derive v0.11.3
Downloaded itertools v0.10.5
Downloaded aho-corasick v0.7.20
Downloaded predicates-tree v1.0.7
error: failed to parse manifest at `/root/.cargo/registry/src/github.com-1ecc6299db9ec823/predicates-tree-1.0.7/Cargo.toml`
Caused by:
failed to parse the `edition` key
Caused by:
this version of Cargo is older than the `2021` edition, and only supports `2015` and `2018` editions.
----------------------------------------------------------------------
nbdkit itself uses edition = 2018, but this seems to affects one of
the dependencies.
I'm not sure how to solve this, but one ideas I had is in ./configure
to check if the cargo/rust we're trying to use doesn't support some
base edition (eg. latest edition supported < 2021) then we would
disable rust bindings.
Unfortunately actually getting the latest supported edition seems
hard. The best I could find is parsing this which doesn't seem ideal:
$ rustc --help |& grep -- --edition
--edition 2015|2018|2021|2024
What do you think?
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines. Supports shell scripting,
bindings from many languages. http://libguestfs.org
1 year, 9 months
[v2v PATCH 0/2] tests: spell out btrfs and ntfs feature group dependencies
by Laszlo Ersek
A few test cases fail if the libguestfs appliance lacks the btrfs or the
ntfs feature group. Let us skip these tests in those situations instead.
Laszlo Ersek (2):
test-v2v-fedora-btrfs-conversion: spell out btrfs feature group
dependency
test-v2v-i-ova: spell out ntfs feature group dependency
tests/test-v2v-fedora-btrfs-conversion.sh | 1 +
tests/test-v2v-i-ova-directory.sh | 4 +++-
tests/test-v2v-i-ova.sh | 4 +++-
3 files changed, 7 insertions(+), 2 deletions(-)
1 year, 9 months
Re: [Libguestfs] [PATCH nbdkit] New plugin: blkio [incomplete]
by Alberto Faria
> This is upstream in nbdkit now:
> https://gitlab.com/nbdkit/nbdkit/-/tree/master/plugins/blkio
>
> Another question:
>
> (6) vhost-user + "read-only" property acts a bit strangely. After
> opening virtio-blk-vhost-user it throws an EROFS error if you try to
> "blkio_start" it. However if you set the "read-only" property to true
> then it's OK to start it, and subsequent read operations work too.
>
> I would expect that any device can be started without needing to set
> this property, and in general I don't understand why vhost-user is r/o
> since everything I read about it seems to indicate it's a general
> purpose writable protocol.
read-only has POSIX file open-like semantics: if the device is itself
read only but the read-only property is false, blkio_start() fails.
With qemu-storage-daemon, you must set writable=on on the
vhost-user-blk export for the device to be writable.
Alberto
1 year, 9 months
[PATCH] inspect: check presence of BIOS boot partition to handle BIOS+GPT setup
by Andrey Drobyshev
If the guest uses BIOS firmware but GPT partitioned disk, and in
addition has "bios, esp" flag enabled on its "/boot" partition, then
inspection wrongly detects UEFI firmware and v2v ends up with the error:
"error: no bootloader detected".
Let's fix this by checking for the presence of BIOS boot partition (0xef02
type for gdisk, "bios_grub" flag for parted), which is used to store a
bootloader code in BIOS+GPT configurations. If such a partition is
present, then it's likely a BIOS+GPT setup.
Signed-off-by: Andrey Drobyshev <andrey.drobyshev(a)virtuozzo.com>
---
convert/inspect_source.ml | 27 +++++++++++++++++++++++----
1 file changed, 23 insertions(+), 4 deletions(-)
diff --git a/convert/inspect_source.ml b/convert/inspect_source.ml
index 056d0bca..6650e136 100644
--- a/convert/inspect_source.ml
+++ b/convert/inspect_source.ml
@@ -219,6 +219,9 @@ and list_applications g root = function
(* See if this guest could use UEFI to boot. It should use GPT and
* it should have an EFI System Partition (ESP).
*
+ * If it has a BIOS boot partition (BIOS+GPT setup), then [BIOS] is
+ * returned.
+ *
* If it has ESP(s), then [UEFI devs] is returned where [devs] is the
* list of at least one ESP.
*
@@ -226,9 +229,13 @@ and list_applications g root = function
*)
and get_firmware_bootable_device g =
let rec uefi_ESP_guid = "C12A7328-F81F-11D2-BA4B-00A0C93EC93B"
+ and bios_boot_guid = "21686148-6449-6E6F-744E-656564454649"
and is_uefi_ESP dev part =
let partnum = g#part_to_partnum part in
g#part_get_gpt_type dev partnum = uefi_ESP_guid
+ and is_bios_boot dev part =
+ let partnum = g#part_to_partnum part in
+ g#part_get_gpt_type dev partnum = bios_boot_guid
and parttype_is_gpt dev =
try g#part_get_parttype dev = "gpt"
with G.Error msg as exn ->
@@ -239,14 +246,26 @@ and get_firmware_bootable_device g =
and is_uefi_bootable_part part =
let dev = g#part_to_dev part in
parttype_is_gpt dev && is_uefi_ESP dev part
+ and is_bios_gpt_part part =
+ let dev = g#part_to_dev part in
+ parttype_is_gpt dev && is_bios_boot dev part
in
let partitions = Array.to_list (g#list_partitions ()) in
- let partitions = List.filter is_uefi_bootable_part partitions in
+ let esp_partitions = List.filter is_uefi_bootable_part partitions in
- match partitions with
- | [] -> I_BIOS
- | partitions -> I_UEFI partitions
+ (* If there's a BIOS boot partition present (0xef02 type for gdisk,
+ * "bios_grub" flag for parted), then this is likely a BIOS+GPT setup.
+ * Note that if a source VM is using UEFI firmware and has a secondary
+ * non-bootable disk attached which contains such a partition, the
+ * firmware detection will detect I_BIOS wrongly. But this can only be
+ * done manually, and we assume that there's no point doing it on purpose.
+ *)
+ if List.exists is_bios_gpt_part partitions then I_BIOS
+ else
+ match esp_partitions with
+ | [] -> I_BIOS
+ | esp_partitions -> I_UEFI esp_partitions
(* If some inspection fields are "unknown", then that indicates a
* failure in inspection, and we shouldn't continue. For an example
--
2.31.1
1 year, 9 months
