9:31 a.m.
If the only CD-ROM in "s_removables" is on an IDE controller, and the
guest kernel represents it with a /dev/hdX device node, then convert
references to this device node, in the boot config files, to /dev/cdrom.
On the destination (after conversion), /dev/cdrom will point to whataver
node we converted the CD-ROM to, masking a potential i440fx -> q35 (IDE ->
SATA) board change.
If the only CD-ROM is not on an IDE controller, or the guest is modern
enough to represent the IDE CD-ROM as /dev/sr0, then perform no
translation. Namely, /dev/sr0 survives a potential i440fx -> q35 (IDE ->
SATA) board change intact.
When multiple CD-ROMs exist, emit a warning, and attempt the conversion on
the first CD-ROM, as a guess. This may be inexact, but we can't do better,
because:
- SATA, SCSI, and (on modern guests) IDE CD-ROMs are lumped together in
the /dev/sr* namespace, on the source side, and "s_removable_slot" is
useless for telling them apart, as we don't know the exact controller
topology (and OS enumeration order);
- after conversion: some OSes don't create /dev/cdrom* symlinks to all
CD-ROMs, and even if multiple such symlinks are created, their order is
basically random.
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1637857
Signed-off-by: Laszlo Ersek <lersek(a)redhat.com>
---
Notes:
v2:
- Use List.length >= 2 rather than matching with three cons operations.
[Rich]
- At the same time, eliminate wrong precedence between "->" in a match,
and ";". [Rich]
- Match the first element of the cdrom list, and the contents of that
element, in a single "match" expression. [Rich]
- Not tested, due to the issue I described in
<8b3ce08b-ea47-dc1c-f441-c8b91708bd6f(a)redhat.com> (cannot provide a
mailing list URL because the archive seems to have stopped refreshing
itself?)
convert/convert_linux.ml | 19 +++++++++++++++++++
tests/test-v2v-cdrom.expected | 2 +-
tests/test-v2v-cdrom.xml.in | 4 +++-
tests/test-v2v-i-ova.xml | 2 +-
4 files changed, 24 insertions(+), 3 deletions(-)
diff --git a/convert/convert_linux.ml b/convert/convert_linux.ml
index 8dc648169dcb..d49ecec03aeb 100644
--- a/convert/convert_linux.ml
+++ b/convert/convert_linux.ml
@@ -1020,6 +1020,25 @@ let convert (g : G.guestfs) source inspect keep_serial_console _ =
"xvd" ^ drive_name i, block_prefix_after_conversion ^ drive_name i
) source.s_disks in
+ (* Check the first CD-ROM. If its controller is IDE, and the OS is RHEL<=5,
+ * then translate the CD-ROM from "/dev/hd[SLOT]" to
"/dev/cdrom". See
+ * RHBZ#1637857 for details.
+ *)
+ let cdroms = List.filter
+ (fun removable -> removable.s_removable_type = CDROM)
+ source.s_removables in
+ if List.length cdroms >= 2 then
+ warning (f_"multiple CD-ROMs found; translation of CD-ROM references \
+ may be inexact");
+ let map = map @
+ (match cdroms with
+ | { s_removable_controller = Some Source_IDE;
+ s_removable_slot = Some slot } :: _
+ when family = `RHEL_family && inspect.i_major_version <= 5 ->
+ [("hd" ^ drive_name slot, "cdrom")]
+ | _ -> []
+ ) in
+
if verbose () then (
eprintf "block device map:\n";
List.iter (
diff --git a/tests/test-v2v-cdrom.expected b/tests/test-v2v-cdrom.expected
index 34d2bf5961b0..17bd152d8e64 100644
--- a/tests/test-v2v-cdrom.expected
+++ b/tests/test-v2v-cdrom.expected
@@ -4,5 +4,5 @@
</disk>
<disk device='cdrom' type='file'>
<driver name='qemu' type='raw'/>
- <target dev='hdc' bus='ide'/>
+ <target dev='sdc' bus='sata'/>
</disk>
diff --git a/tests/test-v2v-cdrom.xml.in b/tests/test-v2v-cdrom.xml.in
index 6bad5eab1cd4..a6e1e3f514d5 100644
--- a/tests/test-v2v-cdrom.xml.in
+++ b/tests/test-v2v-cdrom.xml.in
@@ -35,7 +35,9 @@ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301
USA.
<disk type='file' device='cdrom'>
<driver name='qemu' type='raw'/>
<source
file='@abs_top_builddir(a)/test-data/phony-guests/blank-disk.img'/>
- <!-- virt-v2v should preserve the device name and bus -->
+ <!-- virt-v2v should change the bus to sata, due to Windows 7
+ triggering a machine type change from i440fx to q35. Beyond that,
+ virt-v2v should preserve the on-bus index. -->
<target dev='hdc' bus='ide'/>
</disk>
</devices>
diff --git a/tests/test-v2v-i-ova.xml b/tests/test-v2v-i-ova.xml
index d7383905fdc0..9f3c1974243f 100644
--- a/tests/test-v2v-i-ova.xml
+++ b/tests/test-v2v-i-ova.xml
@@ -28,7 +28,7 @@
</disk>
<disk device='cdrom' type='file'>
<driver name='qemu' type='raw'/>
- <target dev='hda' bus='ide'/>
+ <target dev='sda' bus='sata'/>
</disk>
<disk device='floppy' type='file'>
<driver name='qemu' type='raw'/>
--
2.19.1.3.g30247aa5d201
3:45 a.m.
On Wed, Dec 15, 2021 at 04:31:53PM +0100, Laszlo Ersek wrote:
If the only CD-ROM in "s_removables" is on an IDE
controller, and the
guest kernel represents it with a /dev/hdX device node, then convert
references to this device node, in the boot config files, to /dev/cdrom.
On the destination (after conversion), /dev/cdrom will point to whataver
node we converted the CD-ROM to, masking a potential i440fx -> q35 (IDE ->
SATA) board change.
If the only CD-ROM is not on an IDE controller, or the guest is modern
enough to represent the IDE CD-ROM as /dev/sr0, then perform no
translation. Namely, /dev/sr0 survives a potential i440fx -> q35 (IDE ->
SATA) board change intact.
When multiple CD-ROMs exist, emit a warning, and attempt the conversion on
the first CD-ROM, as a guess. This may be inexact, but we can't do better,
because:
- SATA, SCSI, and (on modern guests) IDE CD-ROMs are lumped together in
the /dev/sr* namespace, on the source side, and "s_removable_slot" is
useless for telling them apart, as we don't know the exact controller
topology (and OS enumeration order);
- after conversion: some OSes don't create /dev/cdrom* symlinks to all
CD-ROMs, and even if multiple such symlinks are created, their order is
basically random.
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1637857
Signed-off-by: Laszlo Ersek <lersek(a)redhat.com>
---
Notes:
v2:
- Use List.length >= 2 rather than matching with three cons operations.
[Rich]
- At the same time, eliminate wrong precedence between "->" in a match,
and ";". [Rich]
- Match the first element of the cdrom list, and the contents of that
element, in a single "match" expression. [Rich]
- Not tested, due to the issue I described in
<8b3ce08b-ea47-dc1c-f441-c8b91708bd6f(a)redhat.com> (cannot provide a
mailing list URL because the archive seems to have stopped refreshing
itself?)
convert/convert_linux.ml | 19 +++++++++++++++++++
tests/test-v2v-cdrom.expected | 2 +-
tests/test-v2v-cdrom.xml.in | 4 +++-
tests/test-v2v-i-ova.xml | 2 +-
4 files changed, 24 insertions(+), 3 deletions(-)
diff --git a/convert/convert_linux.ml b/convert/convert_linux.ml
index 8dc648169dcb..d49ecec03aeb 100644
--- a/convert/convert_linux.ml
+++ b/convert/convert_linux.ml
@@ -1020,6 +1020,25 @@ let convert (g : G.guestfs) source inspect keep_serial_console _
=
"xvd" ^ drive_name i, block_prefix_after_conversion ^ drive_name i
) source.s_disks in
+ (* Check the first CD-ROM. If its controller is IDE, and the OS is RHEL<=5,
+ * then translate the CD-ROM from "/dev/hd[SLOT]" to
"/dev/cdrom". See
+ * RHBZ#1637857 for details.
+ *)
+ let cdroms = List.filter
+ (fun removable -> removable.s_removable_type = CDROM)
+ source.s_removables in
+ if List.length cdroms >= 2 then
+ warning (f_"multiple CD-ROMs found; translation of CD-ROM references \
+ may be inexact");
+ let map = map @
+ (match cdroms with
+ | { s_removable_controller = Some Source_IDE;
+ s_removable_slot = Some slot } :: _
+ when family = `RHEL_family && inspect.i_major_version <= 5 ->
+ [("hd" ^ drive_name slot, "cdrom")]
+ | _ -> []
+ ) in
+
if verbose () then (
eprintf "block device map:\n";
List.iter (
diff --git a/tests/test-v2v-cdrom.expected b/tests/test-v2v-cdrom.expected
index 34d2bf5961b0..17bd152d8e64 100644
--- a/tests/test-v2v-cdrom.expected
+++ b/tests/test-v2v-cdrom.expected
@@ -4,5 +4,5 @@
</disk>
<disk device='cdrom' type='file'>
<driver name='qemu' type='raw'/>
- <target dev='hdc' bus='ide'/>
+ <target dev='sdc' bus='sata'/>
</disk>
diff --git a/tests/test-v2v-cdrom.xml.in b/tests/test-v2v-cdrom.xml.in
index 6bad5eab1cd4..a6e1e3f514d5 100644
--- a/tests/test-v2v-cdrom.xml.in
+++ b/tests/test-v2v-cdrom.xml.in
@@ -35,7 +35,9 @@ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
02110-1301 USA.
<disk type='file' device='cdrom'>
<driver name='qemu' type='raw'/>
<source
file='@abs_top_builddir(a)/test-data/phony-guests/blank-disk.img'/>
- <!-- virt-v2v should preserve the device name and bus -->
+ <!-- virt-v2v should change the bus to sata, due to Windows 7
+ triggering a machine type change from i440fx to q35. Beyond that,
+ virt-v2v should preserve the on-bus index. -->
<target dev='hdc' bus='ide'/>
</disk>
</devices>
diff --git a/tests/test-v2v-i-ova.xml b/tests/test-v2v-i-ova.xml
index d7383905fdc0..9f3c1974243f 100644
--- a/tests/test-v2v-i-ova.xml
+++ b/tests/test-v2v-i-ova.xml
@@ -28,7 +28,7 @@
</disk>
<disk device='cdrom' type='file'>
<driver name='qemu' type='raw'/>
- <target dev='hda' bus='ide'/>
+ <target dev='sda' bus='sata'/>
</disk>
<disk device='floppy' type='file'>
<driver name='qemu' type='raw'/>
--
2.19.1.3.g30247aa5d201
The patch looks fine, although I think it would be better to
wait until it's tested, if you can fix the librpm problem.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
4:50 a.m.
On 12/15/21 16:31, Laszlo Ersek wrote:
If the only CD-ROM in "s_removables" is on an IDE
controller, and the
guest kernel represents it with a /dev/hdX device node, then convert
references to this device node, in the boot config files, to /dev/cdrom.
On the destination (after conversion), /dev/cdrom will point to whataver
node we converted the CD-ROM to, masking a potential i440fx -> q35 (IDE ->
SATA) board change.
If the only CD-ROM is not on an IDE controller, or the guest is modern
enough to represent the IDE CD-ROM as /dev/sr0, then perform no
translation. Namely, /dev/sr0 survives a potential i440fx -> q35 (IDE ->
SATA) board change intact.
When multiple CD-ROMs exist, emit a warning, and attempt the conversion on
the first CD-ROM, as a guess. This may be inexact, but we can't do better,
because:
- SATA, SCSI, and (on modern guests) IDE CD-ROMs are lumped together in
the /dev/sr* namespace, on the source side, and "s_removable_slot" is
useless for telling them apart, as we don't know the exact controller
topology (and OS enumeration order);
- after conversion: some OSes don't create /dev/cdrom* symlinks to all
CD-ROMs, and even if multiple such symlinks are created, their order is
basically random.
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1637857
Signed-off-by: Laszlo Ersek <lersek(a)redhat.com>
---
Notes:
v2:
- Use List.length >= 2 rather than matching with three cons operations.
[Rich]
- At the same time, eliminate wrong precedence between "->" in a match,
and ";". [Rich]
- Match the first element of the cdrom list, and the contents of that
element, in a single "match" expression. [Rich]
- Not tested, due to the issue I described in
<8b3ce08b-ea47-dc1c-f441-c8b91708bd6f(a)redhat.com> (cannot provide a
mailing list URL because the archive seems to have stopped refreshing
itself?)
convert/convert_linux.ml | 19 +++++++++++++++++++
tests/test-v2v-cdrom.expected | 2 +-
tests/test-v2v-cdrom.xml.in | 4 +++-
tests/test-v2v-i-ova.xml | 2 +-
4 files changed, 24 insertions(+), 3 deletions(-)
diff --git a/convert/convert_linux.ml b/convert/convert_linux.ml
index 8dc648169dcb..d49ecec03aeb 100644
--- a/convert/convert_linux.ml
+++ b/convert/convert_linux.ml
@@ -1020,6 +1020,25 @@ let convert (g : G.guestfs) source inspect keep_serial_console _
=
"xvd" ^ drive_name i, block_prefix_after_conversion ^ drive_name i
) source.s_disks in
+ (* Check the first CD-ROM. If its controller is IDE, and the OS is RHEL<=5,
+ * then translate the CD-ROM from "/dev/hd[SLOT]" to
"/dev/cdrom". See
+ * RHBZ#1637857 for details.
+ *)
+ let cdroms = List.filter
+ (fun removable -> removable.s_removable_type = CDROM)
+ source.s_removables in
+ if List.length cdroms >= 2 then
+ warning (f_"multiple CD-ROMs found; translation of CD-ROM references \
+ may be inexact");
+ let map = map @
+ (match cdroms with
+ | { s_removable_controller = Some Source_IDE;
+ s_removable_slot = Some slot } :: _
+ when family = `RHEL_family && inspect.i_major_version <= 5 ->
+ [("hd" ^ drive_name slot, "cdrom")]
+ | _ -> []
+ ) in
+
if verbose () then (
eprintf "block device map:\n";
List.iter (
After updating my workstation to F35 and rebuilding everything, this
patch seems to do the right thing in a synthetic test.
Namely, in the input guest, I added an entry to /etc/fstab, mounting
/dev/hda (an IDE CD-ROM) as /mnt2. After conversion with
virt-v2v -i libvirt -o libvirt -on rhel5.11.out rhel5.11
I checked the output disk manually with guestfish, and /dev/hda had
indeed been replaced with /dev/cdrom in the same /etc/fstab entry.
Additionally, when passing "-v -x" to the above command line, the debug
output contains:
block device map:
hda -> cdrom
vda -> vda
xvda -> vda
So this looks good.
However... the converted guest does not boot at all. Please see the
attached screenshot (it's not big).
To see whether the issue was introduced by my patch, I rebuilt virt-v2v
with my patch removed (that is, at current master = 1673fc4b640f), and
repeated the conversion (first undefining the "rhel5.11.out" domain).
The boot failure persists, so it's unrelated to the patch.
Is this a known issue?
Does it block merging the patch?
Thanks!
Laszlo
diff --git a/tests/test-v2v-cdrom.expected
b/tests/test-v2v-cdrom.expected
index 34d2bf5961b0..17bd152d8e64 100644
--- a/tests/test-v2v-cdrom.expected
+++ b/tests/test-v2v-cdrom.expected
@@ -4,5 +4,5 @@
</disk>
<disk device='cdrom' type='file'>
<driver name='qemu' type='raw'/>
- <target dev='hdc' bus='ide'/>
+ <target dev='sdc' bus='sata'/>
</disk>
diff --git a/tests/test-v2v-cdrom.xml.in b/tests/test-v2v-cdrom.xml.in
index 6bad5eab1cd4..a6e1e3f514d5 100644
--- a/tests/test-v2v-cdrom.xml.in
+++ b/tests/test-v2v-cdrom.xml.in
@@ -35,7 +35,9 @@ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
02110-1301 USA.
<disk type='file' device='cdrom'>
<driver name='qemu' type='raw'/>
<source
file='@abs_top_builddir(a)/test-data/phony-guests/blank-disk.img'/>
- <!-- virt-v2v should preserve the device name and bus -->
+ <!-- virt-v2v should change the bus to sata, due to Windows 7
+ triggering a machine type change from i440fx to q35. Beyond that,
+ virt-v2v should preserve the on-bus index. -->
<target dev='hdc' bus='ide'/>
</disk>
</devices>
diff --git a/tests/test-v2v-i-ova.xml b/tests/test-v2v-i-ova.xml
index d7383905fdc0..9f3c1974243f 100644
--- a/tests/test-v2v-i-ova.xml
+++ b/tests/test-v2v-i-ova.xml
@@ -28,7 +28,7 @@
</disk>
<disk device='cdrom' type='file'>
<driver name='qemu' type='raw'/>
- <target dev='hda' bus='ide'/>
+ <target dev='sda' bus='sata'/>
</disk>
<disk device='floppy' type='file'>
<driver name='qemu' type='raw'/>
5:08 a.m.
On 12/16/21 11:50, Laszlo Ersek wrote:
However... the converted guest does not boot at all. Please see the
attached screenshot (it's not big).
Some more details. During the conversion, I get a warning as follows:
virt-v2v: warning: mount: mount exited with status 32: mount:
/sysroot/mnt2: /dev/sda already mounted or mount point busy. (ignored)
The issue seems to be that *before* the code in my patch runs, there is a separate
mapping, from /dev/hda (the CD-ROM) to /dev/sda. However, /dev/sda is already taken: it
represents the (originally virtio-blk) system disk.
Here's the contents of the original /etc/fstab file:
/dev/VolGroup00/LogVol00 / ext3 defaults
1 1
LABEL=/boot /boot ext3 defaults 1 2
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
/dev/VolGroup00/LogVol01 swap swap defaults 0 0
/dev/hda /mnt2 iso9660 noauto 0 0
And here's a bit larger snippet from the log:
libguestfs: trace: v2v: mount "/dev/hda" "/mnt2"
guestfsd: => mount (0x1) took 0.13 secs
guestfsd: <= mount (0x1) request length 64 bytes
device name translated: /dev/hda -> /dev/sda
This seems to come from "daemon/device-name-translation.c" in libguestfs.
But, I'm attaching the full conversion log too. (NB this log was generated with my
patch applied; however, the patch itself makes no difference regarding the boot failure,
as stated before.)
FWIW, I don't know if this warning (i.e., the failed attempt to mount /dev/sda) has
anything to do with the boot failure in the converted guest.
Thanks,
Laszlo
5:12 a.m.
On 12/16/21 12:08, Laszlo Ersek wrote:
But, I'm attaching the full conversion log too. (NB this log was
generated with my patch applied; however, the patch itself makes no difference regarding
the boot failure, as stated before.)
FWIW, I don't know if this warning (i.e., the failed attempt to mount /dev/sda) has
anything to do with the boot failure in the converted guest.
Apologies for conversing with myself :/
Noticed this (a bunch of it) in the log:
WARNING: PV /dev/sda2 in VG VolGroup00 is using an old PV header,
modify the VG to update.
I don't know if we ultimately perform any operation that modifies the volume group,
but if we do, it seems plausible that that update prevents the LVM driver in RHEL5 from
recognizing the volume group again :(
Laszlo
7:10 a.m.
On Thu, Dec 16, 2021 at 12:12:33PM +0100, Laszlo Ersek wrote:
On 12/16/21 12:08, Laszlo Ersek wrote:
> But, I'm attaching the full conversion log too. (NB this log was generated with
my patch applied; however, the patch itself makes no difference regarding the boot
failure, as stated before.)
>
> FWIW, I don't know if this warning (i.e., the failed attempt to mount /dev/sda)
has anything to do with the boot failure in the converted guest.
Apologies for conversing with myself :/
Noticed this (a bunch of it) in the log:
> WARNING: PV /dev/sda2 in VG VolGroup00 is using an old PV header, modify the VG to
update.
I don't know if we ultimately perform any operation that modifies the volume group,
but if we do, it seems plausible that that update prevents the LVM driver in RHEL5 from
recognizing the volume group again :(
This does indeed seem like a new problem (in LVM). I don't think
we've seen this one before so it needs a bug to track it.
About the patch: The fact that there's an existing bug does not block
this patch from going upstream. It doesn't seem as if this patch is
the cause.
About device name translation:
This is something of a mess at the moment. There's a lot of history ...
Originally I didn't think very deeply about how device names
should be represented in the API. For example if you had an
API like
int guestfs_mount (guestfs_h *g, char *device, char *mp)
how should the device name be represented? In the original version it
was basically the QEMU / kernel name, probably "/dev/vda1". When we
switched to using virtio-scsi the device names changed ("/dev/sda1"),
but that was fine I just added some code which spotted
/dev/[hsv]... and translated it inside the daemon. (This is what
"device name translation" does).
Informally I said that "/dev/sda" in the API means the "first disk",
(in the sense of "added first using guestfs_add_drive") "/dev/sdb"
means second disk, "/dev/sda4" means the fourth partition of the first
disk and so on. This was a convention but it wasn't encoded or
enforced anywhere.
That all worked for quite a long time, but then the kernel started to
enumerate devices in parallel, so now if you two disks, you could no
longer assume that the "first disk" might randomly appear as /dev/sda
or /dev/sdb.
We're still sending these strings across from the library to the
daemon, but now we do some additional and very hokey translation using
device serial numbers which you can check in the code
(daemon/device-name-translation.c)
I think what we _should_ be doing is translating the strings into an
internal representation, which we'd use inside the library and
serialise into the daemon. This is, naturally, an awful lot of work.
Note that the ABI cannot be changed, so whatever happens we're still
going to be passing strings to guestfs_* APIs, this is all about what
happens internally after that point in the generated code.
There's the additional complication that not everything mountable is a
disk (eg. LVs, /dev/md*). Matt Booth already separated that out with
a distinction between disks and "mountables".
There's also the issue that we always add a hidden disk for the
appliance, which is added after the other drives. We need to ignore
that.
There's also disk hotplugging which we should deprecate - it was a
failed experiment, and indeed has never worked for the direct backend
which is enough justification to replace those APIs with ones which
print an error and return -1.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines. Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
10:53 a.m.
On 12/16/21 14:10, Richard W.M. Jones wrote:
On Thu, Dec 16, 2021 at 12:12:33PM +0100, Laszlo Ersek wrote:
> On 12/16/21 12:08, Laszlo Ersek wrote:
>
>> But, I'm attaching the full conversion log too. (NB this log was
>> generated with my patch applied; however, the patch itself makes no
>> difference regarding the boot failure, as stated before.)
>>
>> FWIW, I don't know if this warning (i.e., the failed attempt to
>> mount /dev/sda) has anything to do with the boot failure in the
>> converted guest.
>
> Apologies for conversing with myself :/
>
> Noticed this (a bunch of it) in the log:
>
>> WARNING: PV /dev/sda2 in VG VolGroup00 is using an old PV header,
>> modify the VG to update.
>
> I don't know if we ultimately perform any operation that modifies the
> volume group, but if we do, it seems plausible that that update
> prevents the LVM driver in RHEL5 from recognizing the volume group
> again :(
This does indeed seem like a new problem (in LVM). I don't think
we've seen this one before so it needs a bug to track it.
[... snipping the other explanations -- I've read them and they're
great, but there's only so much I can internalize so quickly, and right
now something else seems to be the issue after all ...]
So, I reinstalled the RHEL-5.11 guest from scratch, and made sure that
it did not use LVM at all, only an MBR partition table and three primary
partitions (/boot, swap, and /). The issue reproduces just the same
(just with different names).
Then, it dawned on me. RHEL-5 does not have virtio-1.0 drivers, only
virtio-0.9 drivers. And a QEMU monitor command (info qtree) confirmed
that, for all virtio devices in the output domain, the "disable_legacy"
property was "on". Meaning, these were not transitional virtio devices,
but purely virtio-1.0 devices.
I've looked at both the domain XML directly generated by virt-v2v, and
the variant that libvirtd actually maintains after the "virsh define".
In the former, there is no PCI topology information (no <address>
elements), and also no "virtio-transitional" model specifications:
https://libvirt.org/formatdomain.html#virtio-transitional-devices
(1)
<disk type='volume' device='disk'>
<driver name='qemu' type='raw'/>
<source pool='default' volume='rhel5.11.out-sda'/>
<target dev='vda' bus='virtio'/>
</disk>
(2)
<interface type='bridge'>
<source bridge='virbr0'/>
<model type='virtio'/>
<mac address='52:54:00:6c:97:64'/>
</interface>
(3)
<memballoon model='virtio'/>
Accordingly, because the Q35 machine type is a PCI Express, not
traditional PCI, board, libvirtd generates the PCIe topology such that
each virtio device lands in a separate PCI Express Root Port:
(1)
<disk type='volume' device='disk'>
<driver name='qemu' type='raw'/>
<source pool='default' volume='rhel5.11.out-sda'/>
<target dev='vda' bus='virtio'/>
<boot order='2'/>
<address type='pci' domain='0x0000' bus='0x05'
slot='0x00' function='0x0'/>
^^^^^^^^^^
</disk>
with
<controller type='pci' index='5'
model='pcie-root-port'>
^^^^^^^^^
(2)
<interface type='bridge'>
<mac address='52:54:00:6c:97:64'/>
<source bridge='virbr0'/>
<model type='virtio'/>
<address type='pci' domain='0x0000' bus='0x02'
slot='0x00' function='0x0'/>
^^^^^^^^^^
</interface>
with
<controller type='pci' index='2'
model='pcie-root-port'>
^^^^^^^^^
(3)
<memballoon model='virtio'>
<address type='pci' domain='0x0000' bus='0x06'
slot='0x00' function='0x0'/>
^^^^^^^^^^
</memballoon>
with
<controller type='pci' index='6'
model='pcie-root-port'>
^^^^^^^^^
In turn, the virtio-pci code in QEMU works such that, if the
"disable_legacy" device property is not explicitly specified, then the
virtio device's placement in the PCI(e) topology decides whether the
device is transitional, or purely virtio-1.0. See function
virtio_pci_realize() in "hw/virtio/virtio-pci.c":
bool pcie_port = pci_bus_is_express(pci_get_bus(pci_dev)) &&
!pci_bus_is_root(pci_get_bus(pci_dev));
...
if (proxy->disable_legacy == ON_OFF_AUTO_AUTO) {
proxy->disable_legacy = pcie_port ? ON_OFF_AUTO_ON : ON_OFF_AUTO_OFF;
}
The particular "pcie_port" initialization means that PCI Express Root
Ports will set "pcie_port" to true, but the PCI Express *root complex*
-- that is, when the virtio devices are endpoints integrated into the
PCIe root complex -- will set "pcie_port" to "false". (The difference
is
made by pci_bus_is_root() -- returns "false" for the root ports, but
"true" for the root complex.) In other words, if all these virtio
devices had been placed on:
<controller type='pci' index='0' model='pcie-root'/>
^^^^^^^^^
then the devices would have been transitional.
In order to verify this theory, I updated the addresses like that
(moving each virtio device to a different slot of bus='0x00') in the
output domain. This way, the RHEL5 guest booted -- no kernel panic. (The
X11 server did not start up, but I chose to ignore that.)
At this point, I managed to verify my patch -- the /dev/hda entry in
/etc/fstab had indeed been replaced with /dev/cdrom, and the /dev/cdrom
symlink pointed to "sr0". So mounting the "/mnt2" mount point worked
fine. I think this is enough evidence to push the patch.
Anyway, back to the larger non-transitional problem. How can we convince
libvirtd to set up transitional devices?
Based on this article again:
<https://libvirt.org/formatdomain.html#virtio-transitional-devices>;, I
went back to the XML generated by virt-v2v, and tried to update every
mention of "virtio" to "virtio-transitional", as follows:
@@ -21,7 +21,7 @@
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<devices>
- <disk type='volume' device='disk'>
+ <disk type='volume' device='disk'
model='virtio-transitional'>
<driver name='qemu' type='raw'/>
<source pool='default' volume='rhel5.11.out-sda'/>
<target dev='vda' bus='virtio'/>
@@ -32,7 +32,7 @@
</disk>
<interface type='bridge'>
<source bridge='virbr0'/>
- <model type='virtio'/>
+ <model type='virtio-transitional'/>
<mac address='52:54:00:6c:97:64'/>
</interface>
<video>
@@ -40,7 +40,7 @@
</video>
<graphics type='spice' autoport='yes' port='-1'/>
<sound model='ich6'/>
- <memballoon model='virtio'/>
+ <memballoon model='virtio-transitional'/>
<viosock model='none'/>
<input type='tablet' bus='usb'/>
<input type='mouse' bus='ps2'/>
When running "virsh define" on this, the effect is that libvirtd places
the virtio devices on the separate "pcie-to-pci-bridge" controller (not
on the root complex, as integrated endpoints, but still good -- this
way, the pci_bus_is_express() call retval is what makes the difference):
(1)
<disk type='volume' device='disk'
model='virtio-transitional'>
^^^^^^^^^^^^^^^^^^^^^^^^^^^
<driver name='qemu' type='raw'/>
<source pool='default' volume='rhel5.11.out-sda'
index='2'/>
<backingStore/>
<target dev='vda' bus='virtio'/>
<alias name='virtio-disk0'/>
<address type='pci' domain='0x0000' bus='0x02'
slot='0x03' function='0x0'/>
^^^^^^^^^^
</disk>
with
<controller type='pci' index='2'
model='pcie-to-pci-bridge'>
^^^^^^^^^
(2)
<interface type='bridge'>
<mac address='52:54:00:6c:97:64'/>
<source bridge='virbr0'/>
<target dev='tap0'/>
<model type='virtio-transitional'/>
^^^^^^^^^^^^^^^^^^^^^
<alias name='net0'/>
<address type='pci' domain='0x0000' bus='0x02'
slot='0x01' function='0x0'/>
^^^^^^^^^^
</interface>
(3)
<memballoon model='virtio-transitional'>
^^^^^^^^^^^^^^^^^^^^^^^^^^
<alias name='balloon0'/>
<address type='pci' domain='0x0000' bus='0x02'
slot='0x04' function='0x0'/>
^^^^^^^^^^
</memballoon>
This way, the guest boots as well.
Summary: libosinfo should have told virt-v2v that RHEL-5.11 does not
have virtio-1.0 drivers, only virtio-0.9 drivers. Accordingly, virt-v2v
should have generated "virtio-transitional" model attributes, so that
libvirtd would position the virtio devices in traditional PCI bus slots.
Do we need a new BZ about this?
Thanks!
Laszlo
11:32 a.m.
On Thu, Dec 16, 2021 at 05:53:11PM +0100, Laszlo Ersek wrote:
Summary: libosinfo should have told virt-v2v that RHEL-5.11 does not
have virtio-1.0 drivers, only virtio-0.9 drivers. Accordingly, virt-v2v
should have generated "virtio-transitional" model attributes, so that
libvirtd would position the virtio devices in traditional PCI bus slots.
Do we need a new BZ about this?
Isn't it this one (albeit the title should be <= 6):
https://bugzilla.redhat.com/show_bug.cgi?id=1942325
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
2:55 a.m.
On 12/16/21 18:32, Richard W.M. Jones wrote:
On Thu, Dec 16, 2021 at 05:53:11PM +0100, Laszlo Ersek wrote:
> Summary: libosinfo should have told virt-v2v that RHEL-5.11 does not
> have virtio-1.0 drivers, only virtio-0.9 drivers. Accordingly, virt-v2v
> should have generated "virtio-transitional" model attributes, so that
> libvirtd would position the virtio devices in traditional PCI bus slots.
>
> Do we need a new BZ about this?
Isn't it this one (albeit the title should be <= 6):
https://bugzilla.redhat.com/show_bug.cgi?id=1942325
Yes, that's it exactly!
Comment 3 in the BZ touches on a question that I've had in mind myself, BTW:
"BTW, rhel6 guest is not supported on rhel9 any more."
I'm a bit confused as to how far back in time we plan to support guest
OSes, and exactly how. Some example thoughts:
- If we decide that we want to drop conversion support (even up-stream)
up to and including RHEL6, then the virtio-transitional question falls
away -- and *with it*, the very patch in this whole thread. :)
- From commit ac39fa292c31 ("v2v: Set machine type explicitly for
outputs which support it (RHBZ#1581428).", 2020-12-04), we have the
following comment:
+ (* Pivot on the year 2007. Any Linux distro from earlier than
+ * 2007 should use i440fx, anything 2007 or newer should use q35.
+ * XXX Look up this information in libosinfo in future.
+ *)
In case we pivoted -- based on any factor really -- such that RHEL5
ended up associated with the i440fx machine type, then it would not be
affected by the virtio-transitional problem at all.
I *think* that the motivation for using Q35 at all, even for guest OSes
like RHEL5 and RHEL6, is motivated by RHEL downstream. (Upstream QEMU is
unlikely to ever remove i440fx, in my opinion.) However, if we consider
RHEL downstream a guiding principle, then we should also consider that
RHEL6 (and earlier) are not supported on RHEL9+ hosts at all!
So, while I'm admittedly confused, I feel like we're trying to fill a
gap here that nobody really cares about. Personally I'd just keep RHEL5
and RHEL6 guests on i440fx, upstream, sidestepping the whole
virtio-transitional problem, and allow RHEL9 hosts to forget about these
guests in peace.
Please enlighten me :)
Back to this patch: after I (unknowingly) worked around RHBZ#1942325,
manually, I managed to test the patch successfully. Can you please ACK it?
Thanks,
Laszlo
4:21 a.m.
On Fri, Dec 17, 2021 at 09:55:56AM +0100, Laszlo Ersek wrote:
On 12/16/21 18:32, Richard W.M. Jones wrote:
> On Thu, Dec 16, 2021 at 05:53:11PM +0100, Laszlo Ersek wrote:
>> Summary: libosinfo should have told virt-v2v that RHEL-5.11 does not
>> have virtio-1.0 drivers, only virtio-0.9 drivers. Accordingly, virt-v2v
>> should have generated "virtio-transitional" model attributes, so that
>> libvirtd would position the virtio devices in traditional PCI bus slots.
>>
>> Do we need a new BZ about this?
>
> Isn't it this one (albeit the title should be <= 6):
>
> https://bugzilla.redhat.com/show_bug.cgi?id=1942325
Yes, that's it exactly!
Comment 3 in the BZ touches on a question that I've had in mind myself, BTW:
"BTW, rhel6 guest is not supported on rhel9 any more."
There was definitely discussion about this at Red Hat. I'm not sure
what the current situation really is. But if we can make it work with
a simple change (like in this bug) then we probably ought to.
I'm a bit confused as to how far back in time we plan to support
guest
OSes, and exactly how. Some example thoughts:
- If we decide that we want to drop conversion support (even up-stream)
up to and including RHEL6, then the virtio-transitional question falls
away -- and *with it*, the very patch in this whole thread. :)
- From commit ac39fa292c31 ("v2v: Set machine type explicitly for
outputs which support it (RHBZ#1581428).", 2020-12-04), we have the
following comment:
+ (* Pivot on the year 2007. Any Linux distro from earlier than
+ * 2007 should use i440fx, anything 2007 or newer should use q35.
+ * XXX Look up this information in libosinfo in future.
+ *)
In case we pivoted -- based on any factor really -- such that RHEL5
ended up associated with the i440fx machine type, then it would not be
affected by the virtio-transitional problem at all.
I *think* that the motivation for using Q35 at all, even for guest OSes
like RHEL5 and RHEL6, is motivated by RHEL downstream. (Upstream QEMU is
unlikely to ever remove i440fx, in my opinion.) However, if we consider
RHEL downstream a guiding principle, then we should also consider that
RHEL6 (and earlier) are not supported on RHEL9+ hosts at all!
So, while I'm admittedly confused, I feel like we're trying to fill a
gap here that nobody really cares about. Personally I'd just keep RHEL5
and RHEL6 guests on i440fx, upstream, sidestepping the whole
virtio-transitional problem, and allow RHEL9 hosts to forget about these
guests in peace.
Please enlighten me :)
What RHEL supports, what virt-v2v offers and what customers do are
often in conflict.
No one is using virt-v2v when they have a way to ansible-deploy their
entire cloud infrastructure of "cattle" from a single button press.
They're using virt-v2v (and even more so, virt-p2v) because they have
very special pet machines that they have no idea how to rebuild
because the sysadmin who installed it left the company / the obscure
CNC machine software isn't supported on newer OSes and the
manufacturer went out of business / they have some regulatory
requirement to use RHEL 5 / they don't want to touch that thing in the
corner which is working.
So we need to keep things going for them even if RHEL doesn't
necessarily support their use case. That means making conversions of
RHEL 5/6 work on RHEL 9, especially if it doesn't really require much
effort as in this case.
Back to this patch: after I (unknowingly) worked around
RHBZ#1942325,
manually, I managed to test the patch successfully. Can you please ACK it?
Sure, ACK to that patch, thanks!
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines. Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
4:31 a.m.
On Fri, Dec 17, 2021 at 10:21:39AM +0000, Richard W.M. Jones wrote:
So we need to keep things going for them even if RHEL doesn't
necessarily support their use case. That means making conversions of
RHEL 5/6 work on RHEL 9
I mean "on RHEL 9 with Q35", my assumption being that either RHEL 9
has already dropped i440fx, or that using i440fx will produce some big
scary deprecation warnings (not sure which is going to happen).
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
4:34 a.m.
On Fri, Dec 17, 2021 at 10:31:40AM +0000, Richard W.M. Jones wrote:
On Fri, Dec 17, 2021 at 10:21:39AM +0000, Richard W.M. Jones wrote:
> So we need to keep things going for them even if RHEL doesn't
> necessarily support their use case. That means making conversions of
> RHEL 5/6 work on RHEL 9
I mean "on RHEL 9 with Q35", my assumption being that either RHEL 9
has already dropped i440fx, or that using i440fx will produce some big
scary deprecation warnings (not sure which is going to happen).
i440fx will exist for the purpose of incoming migrations from RHEL-8
because OpenStack still used i440fx for the lifetime of 8.
It should not be used for newly provisioned VMs on 9, and is likely
to be marked as deprecated when querying from QEMU with QMP.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
6:28 a.m.
On 12/17/21 11:21, Richard W.M. Jones wrote:
On Fri, Dec 17, 2021 at 09:55:56AM +0100, Laszlo Ersek wrote:
> Back to this patch: after I (unknowingly) worked around
RHBZ#1942325,
> manually, I managed to test the patch successfully. Can you please ACK it?
Sure, ACK to that patch, thanks!
Commit eb7af78bea30.
Thanks!
Laszlo
885
days inactive
887
days old
12 comments
3 participants
participants (3)
-
Daniel P. Berrangé -
Laszlo Ersek -
Richard W.M. Jones