9:17 a.m.
If the only CD-ROM in "s_removables" is on an IDE controller, and the
guest kernel represents it with a /dev/hdX device node, then convert
references to this device node, in the boot config files, to /dev/cdrom.
On the destination (after conversion), /dev/cdrom will point to whataver
node we converted the CD-ROM to, masking a potential i440fx -> q35 (IDE ->
SATA) board change.
If the only CD-ROM is not on an IDE controller, or the guest is modern
enough to represent the IDE CD-ROM as /dev/sr0, then perform no
translation. Namely, /dev/sr0 survives a potential i440fx -> q35 (IDE ->
SATA) board change intact.
When multiple CD-ROMs exist, emit a warning, and attempt the conversion on
the first CD-ROM, as a guess. This may be inexact, but we can't do better,
because:
- SATA, SCSI, and (on modern guests) IDE CD-ROMs are lumped together in
the /dev/sr* namespace, on the source side, and "s_removable_slot" is
useless for telling them apart, as we don't know the exact controller
topology (and OS enumeration order);
- after conversion: some OSes don't create /dev/cdrom* symlinks to all
CD-ROMs, and even if multiple such symlinks are created, their order is
basically random.
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1637857
Signed-off-by: Laszlo Ersek <lersek(a)redhat.com>
---
convert/convert_linux.ml | 25 +++++++++++++++++++++++++
tests/test-v2v-cdrom.expected | 2 +-
tests/test-v2v-cdrom.xml.in | 4 +++-
tests/test-v2v-i-ova.xml | 2 +-
4 files changed, 30 insertions(+), 3 deletions(-)
diff --git a/convert/convert_linux.ml b/convert/convert_linux.ml
index 8dc648169dcb..9ea94b75bfc9 100644
--- a/convert/convert_linux.ml
+++ b/convert/convert_linux.ml
@@ -1020,6 +1020,31 @@ let convert (g : G.guestfs) source inspect keep_serial_console _ =
"xvd" ^ drive_name i, block_prefix_after_conversion ^ drive_name i
) source.s_disks in
+ (* Check the first CD-ROM. If its controller is IDE, and the OS is RHEL<=5,
+ * then translate the CD-ROM from "/dev/hd[SLOT]" to
"/dev/cdrom". See
+ * RHBZ#1637857 for details.
+ *)
+ let cdroms = List.filter
+ (fun removable -> removable.s_removable_type = CDROM)
+ source.s_removables in
+
+ match cdroms with
+ | _ :: _ :: _ -> warning (f_"multiple CD-ROMs found; translation of \
+ CD-ROM references may be inexact")
+ | _ -> ();
+
+ let map = map @
+ (match cdroms with
+ | cdrom :: _ ->
+ (match (cdrom.s_removable_controller, cdrom.s_removable_slot,
+ family, inspect.i_major_version) with
+ | Some Source_IDE, Some slot, `RHEL_family, v when v <= 5 ->
+ [("hd" ^ drive_name slot, "cdrom")]
+ | _ -> []
+ )
+ | [] -> []
+ ) in
+
if verbose () then (
eprintf "block device map:\n";
List.iter (
diff --git a/tests/test-v2v-cdrom.expected b/tests/test-v2v-cdrom.expected
index 34d2bf5961b0..17bd152d8e64 100644
--- a/tests/test-v2v-cdrom.expected
+++ b/tests/test-v2v-cdrom.expected
@@ -4,5 +4,5 @@
</disk>
<disk device='cdrom' type='file'>
<driver name='qemu' type='raw'/>
- <target dev='hdc' bus='ide'/>
+ <target dev='sdc' bus='sata'/>
</disk>
diff --git a/tests/test-v2v-cdrom.xml.in b/tests/test-v2v-cdrom.xml.in
index 6bad5eab1cd4..a6e1e3f514d5 100644
--- a/tests/test-v2v-cdrom.xml.in
+++ b/tests/test-v2v-cdrom.xml.in
@@ -35,7 +35,9 @@ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301
USA.
<disk type='file' device='cdrom'>
<driver name='qemu' type='raw'/>
<source
file='@abs_top_builddir(a)/test-data/phony-guests/blank-disk.img'/>
- <!-- virt-v2v should preserve the device name and bus -->
+ <!-- virt-v2v should change the bus to sata, due to Windows 7
+ triggering a machine type change from i440fx to q35. Beyond that,
+ virt-v2v should preserve the on-bus index. -->
<target dev='hdc' bus='ide'/>
</disk>
</devices>
diff --git a/tests/test-v2v-i-ova.xml b/tests/test-v2v-i-ova.xml
index d7383905fdc0..9f3c1974243f 100644
--- a/tests/test-v2v-i-ova.xml
+++ b/tests/test-v2v-i-ova.xml
@@ -28,7 +28,7 @@
</disk>
<disk device='cdrom' type='file'>
<driver name='qemu' type='raw'/>
- <target dev='hda' bus='ide'/>
+ <target dev='sda' bus='sata'/>
</disk>
<disk device='floppy' type='file'>
<driver name='qemu' type='raw'/>
--
2.19.1.3.g30247aa5d201
2:46 p.m.
On Tue, Dec 14, 2021 at 04:17:49PM +0100, Laszlo Ersek wrote:
+ match cdroms with
+ | _ :: _ :: _ -> warning (f_"multiple CD-ROMs found; translation of \
+ CD-ROM references may be inexact")
+ | _ -> ();
+
+ let map = map @
This part is either wrong or indented incorrectly. The "let map ..."
part is part of the second branch of the match statement and doesn't
run if the warning is printed.
In any case it's probably better to replace the confusing first match
with something simpler such as:
if List.length cdroms > 2 then
warning (f_"multiple CD-ROMs found; translation of CD-ROM references may be
inexact");
The \ is also not necessary in the original since OCaml lets you have
arbitrary length strings over multiple lines (a good thing), plus we
automatically wrap warning and error strings on whitespace.
+ (match cdroms with
+ | cdrom :: _ ->
+ (match (cdrom.s_removable_controller, cdrom.s_removable_slot,
+ family, inspect.i_major_version) with
+ | Some Source_IDE, Some slot, `RHEL_family, v when v <= 5 ->
You can just match to arbitrary depth in single match statements, so
the whole match would become something like:
let map = map @
match cdroms with
| { Some Source_IDE, Some slot, `RHEL_family v } :: _ when v <= 5 ->
[("hd" ^ drive_name slot, "cdrom")]
| _ -> [] in
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines. Supports shell scripting,
bindings from many languages. http://libguestfs.org
2:48 p.m.
On Tue, Dec 14, 2021 at 08:46:45PM +0000, Richard W.M. Jones wrote:
On Tue, Dec 14, 2021 at 04:17:49PM +0100, Laszlo Ersek wrote:
> + match cdroms with
> + | _ :: _ :: _ -> warning (f_"multiple CD-ROMs found; translation of \
> + CD-ROM references may be inexact")
> + | _ -> ();
> +
> + let map = map @
This part is either wrong or indented incorrectly. The "let map ..."
part is part of the second branch of the match statement and doesn't
run if the warning is printed.
In any case it's probably better to replace the confusing first match
with something simpler such as:
if List.length cdroms > 2 then
">=" ? I said it was confusing :-)
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine. Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/
2:51 p.m.
On Tue, Dec 14, 2021 at 08:46:45PM +0000, Richard W.M. Jones wrote:
On Tue, Dec 14, 2021 at 04:17:49PM +0100, Laszlo Ersek wrote:
> + match cdroms with
> + | _ :: _ :: _ -> warning (f_"multiple CD-ROMs found; translation of \
> + CD-ROM references may be inexact")
> + | _ -> ();
> +
> + let map = map @
This part is either wrong or indented incorrectly. The "let map ..."
part is part of the second branch of the match statement and doesn't
run if the warning is printed.
In any case it's probably better to replace the confusing first match
with something simpler such as:
if List.length cdroms > 2 then
warning (f_"multiple CD-ROMs found; translation of CD-ROM references may be
inexact");
The \ is also not necessary in the original since OCaml lets you have
arbitrary length strings over multiple lines (a good thing), plus we
automatically wrap warning and error strings on whitespace.
> + (match cdroms with
> + | cdrom :: _ ->
> + (match (cdrom.s_removable_controller, cdrom.s_removable_slot,
> + family, inspect.i_major_version) with
> + | Some Source_IDE, Some slot, `RHEL_family, v when v <= 5 ->
You can just match to arbitrary depth in single match statements, so
the whole match would become something like:
let map = map @
match cdroms with
| { Some Source_IDE, Some slot, `RHEL_family v } :: _ when v <= 5 ->
This line should have been:
| { s_removable_slot = Some Source_IDE; s_removable_slot = Some slot } :: _
when family = `RHEL_family && inspect.i_major_version <= 5 ->
Rich.
[("hd" ^ drive_name slot, "cdrom")]
| _ -> [] in
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines. Supports shell scripting,
bindings from many languages. http://libguestfs.org
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
4:19 a.m.
On 12/14/21 21:51, Richard W.M. Jones wrote:
On Tue, Dec 14, 2021 at 08:46:45PM +0000, Richard W.M. Jones wrote:
> On Tue, Dec 14, 2021 at 04:17:49PM +0100, Laszlo Ersek wrote:
>> + match cdroms with
>> + | _ :: _ :: _ -> warning (f_"multiple CD-ROMs found; translation of
\
>> + CD-ROM references may be inexact")
>> + | _ -> ();
>> +
>> + let map = map @
>
> This part is either wrong or indented incorrectly. The "let map ..."
> part is part of the second branch of the match statement and doesn't
> run if the warning is printed.
>
> In any case it's probably better to replace the confusing first match
> with something simpler such as:
>
> if List.length cdroms > 2 then
> warning (f_"multiple CD-ROMs found; translation of CD-ROM references may be
inexact");
>
> The \ is also not necessary in the original since OCaml lets you have
> arbitrary length strings over multiple lines (a good thing), plus we
> automatically wrap warning and error strings on whitespace.
>
>> + (match cdroms with
>> + | cdrom :: _ ->
>> + (match (cdrom.s_removable_controller, cdrom.s_removable_slot,
>> + family, inspect.i_major_version) with
>> + | Some Source_IDE, Some slot, `RHEL_family, v when v <= 5 ->
>
> You can just match to arbitrary depth in single match statements, so
> the whole match would become something like:
>
> let map = map @
> match cdroms with
> | { Some Source_IDE, Some slot, `RHEL_family v } :: _ when v <= 5 ->
This line should have been:
| { s_removable_slot = Some Source_IDE; s_removable_slot = Some slot } :: _
when family = `RHEL_family && inspect.i_major_version <= 5 ->
Yes, in an earlier version, I used some structure field matching, and
just found it impossible to remember the syntax for that. Had to look it
up in the book... So, now I'm even less sure that this is more readable
:) Please advise, I'll heed it.
Thanks,
Laszlo
Rich.
> [("hd" ^ drive_name slot, "cdrom")]
> | _ -> [] in
>
> Rich.
>
> --
> Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
> Read my programming and virtualization blog: http://rwmj.wordpress.com
> libguestfs lets you edit virtual machines. Supports shell scripting,
> bindings from many languages. http://libguestfs.org
4:17 a.m.
On 12/14/21 21:46, Richard W.M. Jones wrote:
On Tue, Dec 14, 2021 at 04:17:49PM +0100, Laszlo Ersek wrote:
> + match cdroms with
> + | _ :: _ :: _ -> warning (f_"multiple CD-ROMs found; translation of \
> + CD-ROM references may be inexact")
> + | _ -> ();
> +
> + let map = map @
This part is either wrong or indented incorrectly. The "let map ..."
part is part of the second branch of the match statement and doesn't
run if the warning is printed.
*groan* I admit the binding strengths of these operators keep confusing me.
(Also, I meant to post this as an RFC in the first round...)
In any case it's probably better to replace the confusing first match
with something simpler such as:
if List.length cdroms > 2 then
warning (f_"multiple CD-ROMs found; translation of CD-ROM references may be
inexact");
Yes, this was my original version, but I figured not counting the full
length (until the end of the list) is more welcome. I can update it though.
The \ is also not necessary in the original since OCaml lets you have
arbitrary length strings over multiple lines (a good thing),
I'm aware, however. :)
I *really* dislike the one aspect of the current multi-line messages
that they are nailed to column #0. It disrupts the code flow for me. For
example, consider the following snippet from this very file
[convert/convert_linux.ml]:
g#cp grub_config uefi_grub_conf;
let fix_script = sprintf
"#!/bin/bash
efibootmgr -c -L \"CentOS 6\"
rm -rf %s" uefi_fallback_path in
Firstboot.add_firstboot_script
g inspect.i_root "fix uefi boot" fix_script)
I find it less than ideal.
I'd really like to indent such strings logically. And that's exactly
what the backslash is for (I had checked the ocaml documentation): after
an escaped newline, leading spaces and tabs are dropped. I think that's
a fantastic feature of OCaml; I used it intentionally.
plus we
automatically wrap warning and error strings on whitespace.
That sounds very user friendly (I didn't expect it!), but does not
contradict my point about formatting the source code, AFAICT.
> + (match cdroms with
> + | cdrom :: _ ->
> + (match (cdrom.s_removable_controller, cdrom.s_removable_slot,
> + family, inspect.i_major_version) with
> + | Some Source_IDE, Some slot, `RHEL_family, v when v <= 5 ->
You can just match to arbitrary depth in single match statements, so
the whole match would become something like:
let map = map @
match cdroms with
| { Some Source_IDE, Some slot, `RHEL_family v } :: _ when v <= 5 ->
[("hd" ^ drive_name slot, "cdrom")]
| _ -> [] in
Does this make the code easier to read to you? (Honest question.) To me,
this does not separate the steps "take first" and "investigate
first",
and so it's harder to read. But if that's only because I don't have
enough OCaml experience yet, I'm happy to change it. (Because, in
comparison, you might find the two "match" expressions to be the
distraction.)
Thanks!
Laszlo
4:57 a.m.
On Wed, Dec 15, 2021 at 11:17:51AM +0100, Laszlo Ersek wrote:
I *really* dislike the one aspect of the current multi-line messages
that they are nailed to column #0. It disrupts the code flow for me. For
example, consider the following snippet from this very file
[convert/convert_linux.ml]:
g#cp grub_config uefi_grub_conf;
let fix_script = sprintf
"#!/bin/bash
efibootmgr -c -L \"CentOS 6\"
rm -rf %s" uefi_fallback_path in
Firstboot.add_firstboot_script
g inspect.i_root "fix uefi boot" fix_script)
I find it less than ideal.
I'd really like to indent such strings logically. And that's exactly
what the backslash is for (I had checked the ocaml documentation): after
an escaped newline, leading spaces and tabs are dropped. I think that's
a fantastic feature of OCaml; I used it intentionally.
Right, this is indeed a reason to use the \, as it drops the following
whitespace.
> You can just match to arbitrary depth in single match
statements, so
> the whole match would become something like:
>
> let map = map @
> match cdroms with
> | { Some Source_IDE, Some slot, `RHEL_family v } :: _ when v <= 5 ->
> [("hd" ^ drive_name slot, "cdrom")]
> | _ -> [] in
Does this make the code easier to read to you? (Honest question.) To me,
this does not separate the steps "take first" and "investigate
first",
and so it's harder to read. But if that's only because I don't have
enough OCaml experience yet, I'm happy to change it. (Because, in
comparison, you might find the two "match" expressions to be the
distraction.)
For me it's less code => better, within reason. The above (after
fixing the bugs - see later emails) is more natural.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
9:14 a.m.
On 12/15/21 11:57, Richard W.M. Jones wrote:
On Wed, Dec 15, 2021 at 11:17:51AM +0100, Laszlo Ersek wrote:
> I *really* dislike the one aspect of the current multi-line messages
> that they are nailed to column #0. It disrupts the code flow for me. For
> example, consider the following snippet from this very file
> [convert/convert_linux.ml]:
>
> g#cp grub_config uefi_grub_conf;
> let fix_script = sprintf
> "#!/bin/bash
> efibootmgr -c -L \"CentOS 6\"
> rm -rf %s" uefi_fallback_path in
> Firstboot.add_firstboot_script
> g inspect.i_root "fix uefi boot" fix_script)
>
> I find it less than ideal.
>
> I'd really like to indent such strings logically. And that's exactly
> what the backslash is for (I had checked the ocaml documentation): after
> an escaped newline, leading spaces and tabs are dropped. I think that's
> a fantastic feature of OCaml; I used it intentionally.
Right, this is indeed a reason to use the \, as it drops the following
whitespace.
>> You can just match to arbitrary depth in single match statements, so
>> the whole match would become something like:
>>
>> let map = map @
>> match cdroms with
>> | { Some Source_IDE, Some slot, `RHEL_family v } :: _ when v <= 5 ->
>> [("hd" ^ drive_name slot, "cdrom")]
>> | _ -> [] in
>
> Does this make the code easier to read to you? (Honest question.) To me,
> this does not separate the steps "take first" and "investigate
first",
> and so it's harder to read. But if that's only because I don't have
> enough OCaml experience yet, I'm happy to change it. (Because, in
> comparison, you might find the two "match" expressions to be the
> distraction.)
For me it's less code => better, within reason. The above (after
fixing the bugs - see later emails) is more natural.
Before posting v2, I intended to test it :) , so I installed a new RHEL5 guest, and tried
to run virt-v2v (with the v2 patch) on it:
virt-v2v -i libvirt -o local -os ~/output rhel5.11
Unfortunately, things break before my code gets a chance to run; here:
(* Fail early if i_apps is empty. Certain steps such as kernel
* detection won't work without this. If the list is empty it
* likely indicates that libguestfs inspection is broken for
* this guest. See for example RHBZ#1965147.
*)
if inspect.i_apps = [] then
error (f_"inspection of the package database failed for ...
This seems to come from an empty apps list, from "convert/inspect_source.ml",
which in turn invokes "g#inspect_list_applications2".
I turned to guestfish, and ran the following manually:
inspect-os
inspect-get-mountpoints /dev/VolGroup00/LogVol00
inspect-list-applications2 /dev/VolGroup00/LogVol00
Indeed the last command does not return anything. I turned on "verbose" and
"trace" then, and then I see a failure in the daemon:
libguestfs: trace: inspect_list_applications2 "/dev/VolGroup00/LogVol00"
libguestfs: trace: inspect_get_type "/dev/VolGroup00/LogVol00"
libguestfs: trace: inspect_get_type = "linux"
libguestfs: trace: inspect_get_package_format "/dev/VolGroup00/LogVol00"
libguestfs: trace: inspect_get_package_format = "rpm"
libguestfs: trace: internal_list_rpm_applications
\x1b[31;1merror: \x1b[0m\x1b[1mUnable to open /usr/lib/rpm/rpmrc for reading: No such file
or directory.
\x1b[0m\x1b[31;1merror: \x1b[0m\x1b[1mno dbpath has been set
\x1b[0m\x1b[31;1merror: \x1b[0m\x1b[1mcannot open Packages database in
\x1b[0mlibrpm returned 0 installed packages
libguestfs: trace: internal_list_rpm_applications = <struct
guestfs_application2_list(0)>
libguestfs: trace: inspect_list_applications2 = <struct
guestfs_application2_list(0)>
The file "/usr/lib/rpm/rpmrc" absolutely exists in the guest (I booted it up
separately, and checked it).
It seems that "internal_list_rpm_applications" [daemon/rpm.ml] calls chroot, and
uses a child project -- can that be related to the issue?
Anyway, I think I'll post v2, just saying that I could not actually test it.
Thanks,
Laszlo
9:16 a.m.
On 12/15/21 16:14, Laszlo Ersek wrote:
Before posting v2, I intended to test it :) , so I installed a new
RHEL5 guest, and tried to run virt-v2v (with the v2 patch) on it:
virt-v2v -i libvirt -o local -os ~/output rhel5.11
Unfortunately, things break before my code gets a chance to run; here:
(* Fail early if i_apps is empty. Certain steps such as kernel
* detection won't work without this. If the list is empty it
* likely indicates that libguestfs inspection is broken for
* this guest. See for example RHBZ#1965147.
*)
if inspect.i_apps = [] then
error (f_"inspection of the package database failed for ...
This seems to come from an empty apps list, from "convert/inspect_source.ml",
which in turn invokes "g#inspect_list_applications2".
I turned to guestfish, and ran the following manually:
inspect-os
inspect-get-mountpoints /dev/VolGroup00/LogVol00
inspect-list-applications2 /dev/VolGroup00/LogVol00
Indeed the last command does not return anything. I turned on "verbose" and
"trace" then, and then I see a failure in the daemon:
libguestfs: trace: inspect_list_applications2 "/dev/VolGroup00/LogVol00"
libguestfs: trace: inspect_get_type "/dev/VolGroup00/LogVol00"
libguestfs: trace: inspect_get_type = "linux"
libguestfs: trace: inspect_get_package_format "/dev/VolGroup00/LogVol00"
libguestfs: trace: inspect_get_package_format = "rpm"
libguestfs: trace: internal_list_rpm_applications
\x1b[31;1merror: \x1b[0m\x1b[1mUnable to open /usr/lib/rpm/rpmrc for reading: No such
file or directory.
\x1b[0m\x1b[31;1merror: \x1b[0m\x1b[1mno dbpath has been set
\x1b[0m\x1b[31;1merror: \x1b[0m\x1b[1mcannot open Packages database in
\x1b[0mlibrpm returned 0 installed packages
libguestfs: trace: internal_list_rpm_applications = <struct
guestfs_application2_list(0)>
libguestfs: trace: inspect_list_applications2 = <struct
guestfs_application2_list(0)>
The file "/usr/lib/rpm/rpmrc" absolutely exists in the guest (I booted it up
separately, and checked it).
It seems that "internal_list_rpm_applications" [daemon/rpm.ml] calls chroot,
and uses a child project -- can that be related to the issue?
Anyway, I think I'll post v2, just saying that I could not actually test it.
BTW my libguestfs has been last built at commit d829f9ff9ae0
("daemon/listfs: don't call "sgdisk -i" on bogus MBR partition table
entry", 2021-11-26).
Thanks
Laszlo
9:33 a.m.
On Wed, Dec 15, 2021 at 04:14:42PM +0100, Laszlo Ersek wrote:
Before posting v2, I intended to test it :) , so I installed a new
RHEL5 guest, and tried to run virt-v2v (with the v2 patch) on it:
virt-v2v -i libvirt -o local -os ~/output rhel5.11
As a general comment, you might want to check virt-v2v 1.44 branch to
see if these problems are regressions in virt-v2v or something else.
In this case ...
Unfortunately, things break before my code gets a chance to run;
here:
(* Fail early if i_apps is empty. Certain steps such as kernel
* detection won't work without this. If the list is empty it
* likely indicates that libguestfs inspection is broken for
* this guest. See for example RHBZ#1965147.
*)
if inspect.i_apps = [] then
error (f_"inspection of the package database failed for ...
This seems to come from an empty apps list, from "convert/inspect_source.ml",
which in turn invokes "g#inspect_list_applications2".
I turned to guestfish, and ran the following manually:
inspect-os
inspect-get-mountpoints /dev/VolGroup00/LogVol00
inspect-list-applications2 /dev/VolGroup00/LogVol00
Indeed the last command does not return anything. I turned on "verbose" and
"trace" then, and then I see a failure in the daemon:
libguestfs: trace: inspect_list_applications2 "/dev/VolGroup00/LogVol00"
libguestfs: trace: inspect_get_type "/dev/VolGroup00/LogVol00"
libguestfs: trace: inspect_get_type = "linux"
libguestfs: trace: inspect_get_package_format "/dev/VolGroup00/LogVol00"
libguestfs: trace: inspect_get_package_format = "rpm"
libguestfs: trace: internal_list_rpm_applications
\x1b[31;1merror: \x1b[0m\x1b[1mUnable to open /usr/lib/rpm/rpmrc for reading: No such
file or directory.
\x1b[0m\x1b[31;1merror: \x1b[0m\x1b[1mno dbpath has been set
\x1b[0m\x1b[31;1merror: \x1b[0m\x1b[1mcannot open Packages database in
\x1b[0mlibrpm returned 0 installed packages
libguestfs: trace: internal_list_rpm_applications = <struct
guestfs_application2_list(0)>
libguestfs: trace: inspect_list_applications2 = <struct
guestfs_application2_list(0)>
... I think this is potentially an instance of:
[sorry, BZ is down - I will follow up with the bug number]
The new thing here (not really related to virt-v2v) is that RPM itself
changed from Berkley DB to sqlite. However old libguestfs actually
accessed the BDB directly and could not read the RPM databases of new
guests. So we switched to using librpm (instead of direct access),
but this meant for a time we couldn't access the RPM DBs of *old*
guests. However that was eventually fixed in RPM.
https://github.com/libguestfs/libguestfs/commit/c9ee831affed55abe0f928134...
Rich.
The file "/usr/lib/rpm/rpmrc" absolutely exists in the
guest (I booted it up separately, and checked it).
It seems that "internal_list_rpm_applications" [daemon/rpm.ml] calls chroot,
and uses a child project -- can that be related to the issue?
Anyway, I think I'll post v2, just saying that I could not actually test it.
Thanks,
Laszlo
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
9:34 a.m.
On Wed, Dec 15, 2021 at 03:33:00PM +0000, Richard W.M. Jones wrote:
On Wed, Dec 15, 2021 at 04:14:42PM +0100, Laszlo Ersek wrote:
> Before posting v2, I intended to test it :) , so I installed a new RHEL5 guest, and
tried to run virt-v2v (with the v2 patch) on it:
>
> virt-v2v -i libvirt -o local -os ~/output rhel5.11
As a general comment, you might want to check virt-v2v 1.44 branch to
see if these problems are regressions in virt-v2v or something else.
In this case ...
> Unfortunately, things break before my code gets a chance to run; here:
>
> (* Fail early if i_apps is empty. Certain steps such as kernel
> * detection won't work without this. If the list is empty it
> * likely indicates that libguestfs inspection is broken for
> * this guest. See for example RHBZ#1965147.
> *)
> if inspect.i_apps = [] then
> error (f_"inspection of the package database failed for ...
>
> This seems to come from an empty apps list, from
"convert/inspect_source.ml", which in turn invokes
"g#inspect_list_applications2".
>
> I turned to guestfish, and ran the following manually:
>
> inspect-os
> inspect-get-mountpoints /dev/VolGroup00/LogVol00
> inspect-list-applications2 /dev/VolGroup00/LogVol00
>
> Indeed the last command does not return anything. I turned on "verbose"
and "trace" then, and then I see a failure in the daemon:
>
> libguestfs: trace: inspect_list_applications2 "/dev/VolGroup00/LogVol00"
> libguestfs: trace: inspect_get_type "/dev/VolGroup00/LogVol00"
> libguestfs: trace: inspect_get_type = "linux"
> libguestfs: trace: inspect_get_package_format "/dev/VolGroup00/LogVol00"
> libguestfs: trace: inspect_get_package_format = "rpm"
> libguestfs: trace: internal_list_rpm_applications
> \x1b[31;1merror: \x1b[0m\x1b[1mUnable to open /usr/lib/rpm/rpmrc for reading: No
such file or directory.
> \x1b[0m\x1b[31;1merror: \x1b[0m\x1b[1mno dbpath has been set
> \x1b[0m\x1b[31;1merror: \x1b[0m\x1b[1mcannot open Packages database in
> \x1b[0mlibrpm returned 0 installed packages
> libguestfs: trace: internal_list_rpm_applications = <struct
guestfs_application2_list(0)>
> libguestfs: trace: inspect_list_applications2 = <struct
guestfs_application2_list(0)>
... I think this is potentially an instance of:
[sorry, BZ is down - I will follow up with the bug number]
https://bugzilla.redhat.com/show_bug.cgi?id=1965147
The new thing here (not really related to virt-v2v) is that RPM
itself
changed from Berkley DB to sqlite. However old libguestfs actually
accessed the BDB directly and could not read the RPM databases of new
guests. So we switched to using librpm (instead of direct access),
but this meant for a time we couldn't access the RPM DBs of *old*
guests. However that was eventually fixed in RPM.
https://github.com/libguestfs/libguestfs/commit/c9ee831affed55abe0f928134...
Rich.
> The file "/usr/lib/rpm/rpmrc" absolutely exists in the guest (I booted it
up separately, and checked it).
>
> It seems that "internal_list_rpm_applications" [daemon/rpm.ml] calls
chroot, and uses a child project -- can that be related to the issue?
>
> Anyway, I think I'll post v2, just saying that I could not actually test it.
>
> Thanks,
> Laszlo
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
9:40 a.m.
On 12/15/21 16:34, Richard W.M. Jones wrote:
On Wed, Dec 15, 2021 at 03:33:00PM +0000, Richard W.M. Jones wrote:
> On Wed, Dec 15, 2021 at 04:14:42PM +0100, Laszlo Ersek wrote:
>> Before posting v2, I intended to test it :) , so I installed a new RHEL5 guest,
and tried to run virt-v2v (with the v2 patch) on it:
>>
>> virt-v2v -i libvirt -o local -os ~/output rhel5.11
>
> As a general comment, you might want to check virt-v2v 1.44 branch to
> see if these problems are regressions in virt-v2v or something else.
>
> In this case ...
>
>> Unfortunately, things break before my code gets a chance to run; here:
>>
>> (* Fail early if i_apps is empty. Certain steps such as kernel
>> * detection won't work without this. If the list is empty it
>> * likely indicates that libguestfs inspection is broken for
>> * this guest. See for example RHBZ#1965147.
>> *)
>> if inspect.i_apps = [] then
>> error (f_"inspection of the package database failed for ...
>>
>> This seems to come from an empty apps list, from
"convert/inspect_source.ml", which in turn invokes
"g#inspect_list_applications2".
>>
>> I turned to guestfish, and ran the following manually:
>>
>> inspect-os
>> inspect-get-mountpoints /dev/VolGroup00/LogVol00
>> inspect-list-applications2 /dev/VolGroup00/LogVol00
>>
>> Indeed the last command does not return anything. I turned on "verbose"
and "trace" then, and then I see a failure in the daemon:
>>
>> libguestfs: trace: inspect_list_applications2
"/dev/VolGroup00/LogVol00"
>> libguestfs: trace: inspect_get_type "/dev/VolGroup00/LogVol00"
>> libguestfs: trace: inspect_get_type = "linux"
>> libguestfs: trace: inspect_get_package_format
"/dev/VolGroup00/LogVol00"
>> libguestfs: trace: inspect_get_package_format = "rpm"
>> libguestfs: trace: internal_list_rpm_applications
>> \x1b[31;1merror: \x1b[0m\x1b[1mUnable to open /usr/lib/rpm/rpmrc for reading: No
such file or directory.
>> \x1b[0m\x1b[31;1merror: \x1b[0m\x1b[1mno dbpath has been set
>> \x1b[0m\x1b[31;1merror: \x1b[0m\x1b[1mcannot open Packages database in
>> \x1b[0mlibrpm returned 0 installed packages
>> libguestfs: trace: internal_list_rpm_applications = <struct
guestfs_application2_list(0)>
>> libguestfs: trace: inspect_list_applications2 = <struct
guestfs_application2_list(0)>
>
> ... I think this is potentially an instance of:
>
> [sorry, BZ is down - I will follow up with the bug number]
https://bugzilla.redhat.com/show_bug.cgi?id=1965147
Yep, that's exactly the BZ number that's highlighted in the code comment
above. I had read the BZ quickly before posting my findings. However,
the RPM error messages in the BZ are different. (I can't open the BZ
ticket again because bugzilla is down (yay!), but I seem to remember
that the librpm complaints were not about failures to open existent
files. I could be wrong though, from memory.)
Anyway, I guess this indicates that my Fedora 34 host does not have a
recent enough librpm (and so the appliance too has an out-of-date librpm).
Thanks,
Laszlo
> The new thing here (not really related to virt-v2v) is that RPM itself
> changed from Berkley DB to sqlite. However old libguestfs actually
> accessed the BDB directly and could not read the RPM databases of new
> guests. So we switched to using librpm (instead of direct access),
> but this meant for a time we couldn't access the RPM DBs of *old*
> guests. However that was eventually fixed in RPM.
>
>
https://github.com/libguestfs/libguestfs/commit/c9ee831affed55abe0f928134...
>
> Rich.
>
>> The file "/usr/lib/rpm/rpmrc" absolutely exists in the guest (I booted
it up separately, and checked it).
>>
>> It seems that "internal_list_rpm_applications" [daemon/rpm.ml] calls
chroot, and uses a child project -- can that be related to the issue?
>>
>> Anyway, I think I'll post v2, just saying that I could not actually test it.
>>
>> Thanks,
>> Laszlo
>
> --
> Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
> Read my programming and virtualization blog: http://rwmj.wordpress.com
> Fedora Windows cross-compiler. Compile Windows programs, test, and
> build Windows installers. Over 100 libraries supported.
> http://fedoraproject.org/wiki/MinGW
9:51 a.m.
On Wed, Dec 15, 2021 at 04:40:22PM +0100, Laszlo Ersek wrote:
On 12/15/21 16:34, Richard W.M. Jones wrote:
> On Wed, Dec 15, 2021 at 03:33:00PM +0000, Richard W.M. Jones wrote:
>> On Wed, Dec 15, 2021 at 04:14:42PM +0100, Laszlo Ersek wrote:
>>> Before posting v2, I intended to test it :) , so I installed a new RHEL5
guest, and tried to run virt-v2v (with the v2 patch) on it:
>>>
>>> virt-v2v -i libvirt -o local -os ~/output rhel5.11
>>
>> As a general comment, you might want to check virt-v2v 1.44 branch to
>> see if these problems are regressions in virt-v2v or something else.
>>
>> In this case ...
>>
>>> Unfortunately, things break before my code gets a chance to run; here:
>>>
>>> (* Fail early if i_apps is empty. Certain steps such as kernel
>>> * detection won't work without this. If the list is empty it
>>> * likely indicates that libguestfs inspection is broken for
>>> * this guest. See for example RHBZ#1965147.
>>> *)
>>> if inspect.i_apps = [] then
>>> error (f_"inspection of the package database failed for ...
>>>
>>> This seems to come from an empty apps list, from
"convert/inspect_source.ml", which in turn invokes
"g#inspect_list_applications2".
>>>
>>> I turned to guestfish, and ran the following manually:
>>>
>>> inspect-os
>>> inspect-get-mountpoints /dev/VolGroup00/LogVol00
>>> inspect-list-applications2 /dev/VolGroup00/LogVol00
>>>
>>> Indeed the last command does not return anything. I turned on
"verbose" and "trace" then, and then I see a failure in the daemon:
>>>
>>> libguestfs: trace: inspect_list_applications2
"/dev/VolGroup00/LogVol00"
>>> libguestfs: trace: inspect_get_type "/dev/VolGroup00/LogVol00"
>>> libguestfs: trace: inspect_get_type = "linux"
>>> libguestfs: trace: inspect_get_package_format
"/dev/VolGroup00/LogVol00"
>>> libguestfs: trace: inspect_get_package_format = "rpm"
>>> libguestfs: trace: internal_list_rpm_applications
>>> \x1b[31;1merror: \x1b[0m\x1b[1mUnable to open /usr/lib/rpm/rpmrc for
reading: No such file or directory.
>>> \x1b[0m\x1b[31;1merror: \x1b[0m\x1b[1mno dbpath has been set
>>> \x1b[0m\x1b[31;1merror: \x1b[0m\x1b[1mcannot open Packages database in
>>> \x1b[0mlibrpm returned 0 installed packages
>>> libguestfs: trace: internal_list_rpm_applications = <struct
guestfs_application2_list(0)>
>>> libguestfs: trace: inspect_list_applications2 = <struct
guestfs_application2_list(0)>
>>
>> ... I think this is potentially an instance of:
>>
>> [sorry, BZ is down - I will follow up with the bug number]
>
> https://bugzilla.redhat.com/show_bug.cgi?id=1965147
Yep, that's exactly the BZ number that's highlighted in the code comment
above. I had read the BZ quickly before posting my findings. However,
the RPM error messages in the BZ are different. (I can't open the BZ
ticket again because bugzilla is down (yay!), but I seem to remember
that the librpm complaints were not about failures to open existent
files. I could be wrong though, from memory.)
Anyway, I guess this indicates that my Fedora 34 host does not have a
recent enough librpm (and so the appliance too has an out-of-date librpm).
The bottom line is still that it's librpm's problem now to handle any
guest rpmdb, no matter how ancient (well, at least RHEL >= 5). If you
have the latest rpm installed and it still doesn't work then it could
be a bug in librpm, although I'm pretty sure I tested this case.
Easy test is something like:
$ virt-builder rhel-5.10
$ virt-inspector -a rhel-5.10.img
(I'll send you privately some links to make those commands work)
Rich.
Thanks,
Laszlo
>
>> The new thing here (not really related to virt-v2v) is that RPM itself
>> changed from Berkley DB to sqlite. However old libguestfs actually
>> accessed the BDB directly and could not read the RPM databases of new
>> guests. So we switched to using librpm (instead of direct access),
>> but this meant for a time we couldn't access the RPM DBs of *old*
>> guests. However that was eventually fixed in RPM.
>>
>>
https://github.com/libguestfs/libguestfs/commit/c9ee831affed55abe0f928134...
>>
>> Rich.
>>
>>> The file "/usr/lib/rpm/rpmrc" absolutely exists in the guest (I
booted it up separately, and checked it).
>>>
>>> It seems that "internal_list_rpm_applications" [daemon/rpm.ml]
calls chroot, and uses a child project -- can that be related to the issue?
>>>
>>> Anyway, I think I'll post v2, just saying that I could not actually test
it.
>>>
>>> Thanks,
>>> Laszlo
>>
>> --
>> Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
>> Read my programming and virtualization blog: http://rwmj.wordpress.com
>> Fedora Windows cross-compiler. Compile Windows programs, test, and
>> build Windows installers. Over 100 libraries supported.
>> http://fedoraproject.org/wiki/MinGW
>
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine. Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/
2:35 a.m.
On 12/15/21 16:51, Richard W.M. Jones wrote:
The bottom line is still that it's librpm's problem now to
handle any
guest rpmdb, no matter how ancient (well, at least RHEL >= 5). If you
have the latest rpm installed and it still doesn't work then it could
be a bug in librpm, although I'm pretty sure I tested this case.
The latest "rpm" package in Fedora 34, rpm-4.16.1.3-1.fc34:
https://koji.fedoraproject.org/koji/buildinfo?buildID=1726304
built on 22 Mar 2021, does not (cannot) have the fix. The upstream rpm
fix is:
https://github.com/rpm-software-management/rpm/commit/ea2754b4f14e0a195f4...
which is between tags "rpm-4.17.0-alpha" and "rpm-4.17.0-beta1", and
has
commit date 2021-06-11. The fix has been backported to RHEL-9 (1965147):
https://brewweb.engineering.redhat.com/brew/buildinfo?buildID=1824746
but not to Fedora 34.
Fedora 34 is supposed to be maintained until 2022-05-17, according to
Wikipedia, so this is definitely a Fedora bug. I guess I should update
my workstation to Fedora 35 regardless; that should be faster.
(Fedora 35 has rpm-4.17.0-1.fc35, so it should have the fix.)
Thanks
Laszlo
3:11 a.m.
On Thu, Dec 16, 2021 at 09:35:31AM +0100, Laszlo Ersek wrote:
On 12/15/21 16:51, Richard W.M. Jones wrote:
> The bottom line is still that it's librpm's problem now to handle any
> guest rpmdb, no matter how ancient (well, at least RHEL >= 5). If you
> have the latest rpm installed and it still doesn't work then it could
> be a bug in librpm, although I'm pretty sure I tested this case.
The latest "rpm" package in Fedora 34, rpm-4.16.1.3-1.fc34:
https://koji.fedoraproject.org/koji/buildinfo?buildID=1726304
built on 22 Mar 2021, does not (cannot) have the fix. The upstream rpm
fix is:
https://github.com/rpm-software-management/rpm/commit/ea2754b4f14e0a195f4...
which is between tags "rpm-4.17.0-alpha" and "rpm-4.17.0-beta1", and
has
commit date 2021-06-11. The fix has been backported to RHEL-9 (1965147):
https://brewweb.engineering.redhat.com/brew/buildinfo?buildID=1824746
but not to Fedora 34.
Fedora 34 is supposed to be maintained until 2022-05-17, according to
Wikipedia, so this is definitely a Fedora bug. I guess I should update
my workstation to Fedora 35 regardless; that should be faster.
(Fedora 35 has rpm-4.17.0-1.fc35, so it should have the fix.)
You might find you can just install the Fedora 35 package on F34, eg:
dnf update rpm --releasever=35
Small possibility it could do Bad Stuff to your system -- since it is
a rather fundamental package ...
Rich.
Thanks
Laszlo
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
1071
days inactive
1073
days old
14 comments
2 participants
participants (2)
-
Laszlo Ersek -
Richard W.M. Jones