[v2v PATCH v2] convert_linux: translate the first CD-ROM's references in boot conf files
If the only CD-ROM in "s_removables" is on an IDE controller, and the guest kernel represents it with a /dev/hdX device node, then convert references to this device node, in the boot config files, to /dev/cdrom. On the destination (after conversion), /dev/cdrom will point to whataver node we converted the CD-ROM to, masking a potential i440fx -> q35 (IDE -> SATA) board change. If the only CD-ROM is not on an IDE controller, or the guest is modern enough to represent the IDE CD-ROM as /dev/sr0, then perform no translation. Namely, /dev/sr0 survives a potential i440fx -> q35 (IDE -> SATA) board change intact. When multiple CD-ROMs exist, emit a warning, and attempt the conversion on the first CD-ROM, as a guess. This may be inexact, but we can't do better, because: - SATA, SCSI, and (on modern guests) IDE CD-ROMs are lumped together in the /dev/sr* namespace, on the source side, and "s_removable_slot" is useless for telling them apart, as we don't know the exact controller topology (and OS enumeration order); - after conversion: some OSes don't create /dev/cdrom* symlinks to all CD-ROMs, and even if multiple such symlinks are created, their order is basically random. Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1637857 Signed-off-by: Laszlo Ersek <lersek@redhat.com> --- Notes: v2: - Use List.length >= 2 rather than matching with three cons operations. [Rich] - At the same time, eliminate wrong precedence between "->" in a match, and ";". [Rich] - Match the first element of the cdrom list, and the contents of that element, in a single "match" expression. [Rich] - Not tested, due to the issue I described in <8b3ce08b-ea47-dc1c-f441-c8b91708bd6f@redhat.com> (cannot provide a mailing list URL because the archive seems to have stopped refreshing itself?) convert/convert_linux.ml | 19 +++++++++++++++++++ tests/test-v2v-cdrom.expected | 2 +- tests/test-v2v-cdrom.xml.in | 4 +++- tests/test-v2v-i-ova.xml | 2 +- 4 files changed, 24 insertions(+), 3 deletions(-) diff --git a/convert/convert_linux.ml b/convert/convert_linux.ml index 8dc648169dcb..d49ecec03aeb 100644 --- a/convert/convert_linux.ml +++ b/convert/convert_linux.ml @@ -1020,6 +1020,25 @@ let convert (g : G.guestfs) source inspect keep_serial_console _ = "xvd" ^ drive_name i, block_prefix_after_conversion ^ drive_name i ) source.s_disks in + (* Check the first CD-ROM. If its controller is IDE, and the OS is RHEL<=5, + * then translate the CD-ROM from "/dev/hd[SLOT]" to "/dev/cdrom". See + * RHBZ#1637857 for details. + *) + let cdroms = List.filter + (fun removable -> removable.s_removable_type = CDROM) + source.s_removables in + if List.length cdroms >= 2 then + warning (f_"multiple CD-ROMs found; translation of CD-ROM references \ + may be inexact"); + let map = map @ + (match cdroms with + | { s_removable_controller = Some Source_IDE; + s_removable_slot = Some slot } :: _ + when family = `RHEL_family && inspect.i_major_version <= 5 -> + [("hd" ^ drive_name slot, "cdrom")] + | _ -> [] + ) in + if verbose () then ( eprintf "block device map:\n"; List.iter ( diff --git a/tests/test-v2v-cdrom.expected b/tests/test-v2v-cdrom.expected index 34d2bf5961b0..17bd152d8e64 100644 --- a/tests/test-v2v-cdrom.expected +++ b/tests/test-v2v-cdrom.expected @@ -4,5 +4,5 @@ </disk> <disk device='cdrom' type='file'> <driver name='qemu' type='raw'/> - <target dev='hdc' bus='ide'/> + <target dev='sdc' bus='sata'/> </disk> diff --git a/tests/test-v2v-cdrom.xml.in b/tests/test-v2v-cdrom.xml.in index 6bad5eab1cd4..a6e1e3f514d5 100644 --- a/tests/test-v2v-cdrom.xml.in +++ b/tests/test-v2v-cdrom.xml.in @@ -35,7 +35,9 @@ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='@abs_top_builddir@/test-data/phony-guests/blank-disk.img'/> - <!-- virt-v2v should preserve the device name and bus --> + <!-- virt-v2v should change the bus to sata, due to Windows 7 + triggering a machine type change from i440fx to q35. Beyond that, + virt-v2v should preserve the on-bus index. --> <target dev='hdc' bus='ide'/> </disk> </devices> diff --git a/tests/test-v2v-i-ova.xml b/tests/test-v2v-i-ova.xml index d7383905fdc0..9f3c1974243f 100644 --- a/tests/test-v2v-i-ova.xml +++ b/tests/test-v2v-i-ova.xml @@ -28,7 +28,7 @@ </disk> <disk device='cdrom' type='file'> <driver name='qemu' type='raw'/> - <target dev='hda' bus='ide'/> + <target dev='sda' bus='sata'/> </disk> <disk device='floppy' type='file'> <driver name='qemu' type='raw'/> -- 2.19.1.3.g30247aa5d201
On Wed, Dec 15, 2021 at 04:31:53PM +0100, Laszlo Ersek wrote:
If the only CD-ROM in "s_removables" is on an IDE controller, and the guest kernel represents it with a /dev/hdX device node, then convert references to this device node, in the boot config files, to /dev/cdrom. On the destination (after conversion), /dev/cdrom will point to whataver node we converted the CD-ROM to, masking a potential i440fx -> q35 (IDE -> SATA) board change.
If the only CD-ROM is not on an IDE controller, or the guest is modern enough to represent the IDE CD-ROM as /dev/sr0, then perform no translation. Namely, /dev/sr0 survives a potential i440fx -> q35 (IDE -> SATA) board change intact.
When multiple CD-ROMs exist, emit a warning, and attempt the conversion on the first CD-ROM, as a guess. This may be inexact, but we can't do better, because:
- SATA, SCSI, and (on modern guests) IDE CD-ROMs are lumped together in the /dev/sr* namespace, on the source side, and "s_removable_slot" is useless for telling them apart, as we don't know the exact controller topology (and OS enumeration order);
- after conversion: some OSes don't create /dev/cdrom* symlinks to all CD-ROMs, and even if multiple such symlinks are created, their order is basically random.
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1637857 Signed-off-by: Laszlo Ersek <lersek@redhat.com> ---
Notes: v2:
- Use List.length >= 2 rather than matching with three cons operations. [Rich]
- At the same time, eliminate wrong precedence between "->" in a match, and ";". [Rich]
- Match the first element of the cdrom list, and the contents of that element, in a single "match" expression. [Rich]
- Not tested, due to the issue I described in <8b3ce08b-ea47-dc1c-f441-c8b91708bd6f@redhat.com> (cannot provide a mailing list URL because the archive seems to have stopped refreshing itself?)
convert/convert_linux.ml | 19 +++++++++++++++++++ tests/test-v2v-cdrom.expected | 2 +- tests/test-v2v-cdrom.xml.in | 4 +++- tests/test-v2v-i-ova.xml | 2 +- 4 files changed, 24 insertions(+), 3 deletions(-)
diff --git a/convert/convert_linux.ml b/convert/convert_linux.ml index 8dc648169dcb..d49ecec03aeb 100644 --- a/convert/convert_linux.ml +++ b/convert/convert_linux.ml @@ -1020,6 +1020,25 @@ let convert (g : G.guestfs) source inspect keep_serial_console _ = "xvd" ^ drive_name i, block_prefix_after_conversion ^ drive_name i ) source.s_disks in
+ (* Check the first CD-ROM. If its controller is IDE, and the OS is RHEL<=5, + * then translate the CD-ROM from "/dev/hd[SLOT]" to "/dev/cdrom". See + * RHBZ#1637857 for details. + *) + let cdroms = List.filter + (fun removable -> removable.s_removable_type = CDROM) + source.s_removables in + if List.length cdroms >= 2 then + warning (f_"multiple CD-ROMs found; translation of CD-ROM references \ + may be inexact"); + let map = map @ + (match cdroms with + | { s_removable_controller = Some Source_IDE; + s_removable_slot = Some slot } :: _ + when family = `RHEL_family && inspect.i_major_version <= 5 -> + [("hd" ^ drive_name slot, "cdrom")] + | _ -> [] + ) in + if verbose () then ( eprintf "block device map:\n"; List.iter ( diff --git a/tests/test-v2v-cdrom.expected b/tests/test-v2v-cdrom.expected index 34d2bf5961b0..17bd152d8e64 100644 --- a/tests/test-v2v-cdrom.expected +++ b/tests/test-v2v-cdrom.expected @@ -4,5 +4,5 @@ </disk> <disk device='cdrom' type='file'> <driver name='qemu' type='raw'/> - <target dev='hdc' bus='ide'/> + <target dev='sdc' bus='sata'/> </disk> diff --git a/tests/test-v2v-cdrom.xml.in b/tests/test-v2v-cdrom.xml.in index 6bad5eab1cd4..a6e1e3f514d5 100644 --- a/tests/test-v2v-cdrom.xml.in +++ b/tests/test-v2v-cdrom.xml.in @@ -35,7 +35,9 @@ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='@abs_top_builddir@/test-data/phony-guests/blank-disk.img'/> - <!-- virt-v2v should preserve the device name and bus --> + <!-- virt-v2v should change the bus to sata, due to Windows 7 + triggering a machine type change from i440fx to q35. Beyond that, + virt-v2v should preserve the on-bus index. --> <target dev='hdc' bus='ide'/> </disk> </devices> diff --git a/tests/test-v2v-i-ova.xml b/tests/test-v2v-i-ova.xml index d7383905fdc0..9f3c1974243f 100644 --- a/tests/test-v2v-i-ova.xml +++ b/tests/test-v2v-i-ova.xml @@ -28,7 +28,7 @@ </disk> <disk device='cdrom' type='file'> <driver name='qemu' type='raw'/> - <target dev='hda' bus='ide'/> + <target dev='sda' bus='sata'/> </disk> <disk device='floppy' type='file'> <driver name='qemu' type='raw'/> -- 2.19.1.3.g30247aa5d201
The patch looks fine, although I think it would be better to wait until it's tested, if you can fix the librpm problem. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top
On 12/15/21 16:31, Laszlo Ersek wrote:
If the only CD-ROM in "s_removables" is on an IDE controller, and the guest kernel represents it with a /dev/hdX device node, then convert references to this device node, in the boot config files, to /dev/cdrom. On the destination (after conversion), /dev/cdrom will point to whataver node we converted the CD-ROM to, masking a potential i440fx -> q35 (IDE -> SATA) board change.
If the only CD-ROM is not on an IDE controller, or the guest is modern enough to represent the IDE CD-ROM as /dev/sr0, then perform no translation. Namely, /dev/sr0 survives a potential i440fx -> q35 (IDE -> SATA) board change intact.
When multiple CD-ROMs exist, emit a warning, and attempt the conversion on the first CD-ROM, as a guess. This may be inexact, but we can't do better, because:
- SATA, SCSI, and (on modern guests) IDE CD-ROMs are lumped together in the /dev/sr* namespace, on the source side, and "s_removable_slot" is useless for telling them apart, as we don't know the exact controller topology (and OS enumeration order);
- after conversion: some OSes don't create /dev/cdrom* symlinks to all CD-ROMs, and even if multiple such symlinks are created, their order is basically random.
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1637857 Signed-off-by: Laszlo Ersek <lersek@redhat.com> ---
Notes: v2:
- Use List.length >= 2 rather than matching with three cons operations. [Rich]
- At the same time, eliminate wrong precedence between "->" in a match, and ";". [Rich]
- Match the first element of the cdrom list, and the contents of that element, in a single "match" expression. [Rich]
- Not tested, due to the issue I described in <8b3ce08b-ea47-dc1c-f441-c8b91708bd6f@redhat.com> (cannot provide a mailing list URL because the archive seems to have stopped refreshing itself?)
convert/convert_linux.ml | 19 +++++++++++++++++++ tests/test-v2v-cdrom.expected | 2 +- tests/test-v2v-cdrom.xml.in | 4 +++- tests/test-v2v-i-ova.xml | 2 +- 4 files changed, 24 insertions(+), 3 deletions(-)
diff --git a/convert/convert_linux.ml b/convert/convert_linux.ml index 8dc648169dcb..d49ecec03aeb 100644 --- a/convert/convert_linux.ml +++ b/convert/convert_linux.ml @@ -1020,6 +1020,25 @@ let convert (g : G.guestfs) source inspect keep_serial_console _ = "xvd" ^ drive_name i, block_prefix_after_conversion ^ drive_name i ) source.s_disks in
+ (* Check the first CD-ROM. If its controller is IDE, and the OS is RHEL<=5, + * then translate the CD-ROM from "/dev/hd[SLOT]" to "/dev/cdrom". See + * RHBZ#1637857 for details. + *) + let cdroms = List.filter + (fun removable -> removable.s_removable_type = CDROM) + source.s_removables in + if List.length cdroms >= 2 then + warning (f_"multiple CD-ROMs found; translation of CD-ROM references \ + may be inexact"); + let map = map @ + (match cdroms with + | { s_removable_controller = Some Source_IDE; + s_removable_slot = Some slot } :: _ + when family = `RHEL_family && inspect.i_major_version <= 5 -> + [("hd" ^ drive_name slot, "cdrom")] + | _ -> [] + ) in + if verbose () then ( eprintf "block device map:\n"; List.iter (
After updating my workstation to F35 and rebuilding everything, this patch seems to do the right thing in a synthetic test. Namely, in the input guest, I added an entry to /etc/fstab, mounting /dev/hda (an IDE CD-ROM) as /mnt2. After conversion with virt-v2v -i libvirt -o libvirt -on rhel5.11.out rhel5.11 I checked the output disk manually with guestfish, and /dev/hda had indeed been replaced with /dev/cdrom in the same /etc/fstab entry. Additionally, when passing "-v -x" to the above command line, the debug output contains:
block device map: hda -> cdrom vda -> vda xvda -> vda
So this looks good. However... the converted guest does not boot at all. Please see the attached screenshot (it's not big). To see whether the issue was introduced by my patch, I rebuilt virt-v2v with my patch removed (that is, at current master = 1673fc4b640f), and repeated the conversion (first undefining the "rhel5.11.out" domain). The boot failure persists, so it's unrelated to the patch. Is this a known issue? Does it block merging the patch? Thanks! Laszlo
diff --git a/tests/test-v2v-cdrom.expected b/tests/test-v2v-cdrom.expected index 34d2bf5961b0..17bd152d8e64 100644 --- a/tests/test-v2v-cdrom.expected +++ b/tests/test-v2v-cdrom.expected @@ -4,5 +4,5 @@ </disk> <disk device='cdrom' type='file'> <driver name='qemu' type='raw'/> - <target dev='hdc' bus='ide'/> + <target dev='sdc' bus='sata'/> </disk> diff --git a/tests/test-v2v-cdrom.xml.in b/tests/test-v2v-cdrom.xml.in index 6bad5eab1cd4..a6e1e3f514d5 100644 --- a/tests/test-v2v-cdrom.xml.in +++ b/tests/test-v2v-cdrom.xml.in @@ -35,7 +35,9 @@ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='@abs_top_builddir@/test-data/phony-guests/blank-disk.img'/> - <!-- virt-v2v should preserve the device name and bus --> + <!-- virt-v2v should change the bus to sata, due to Windows 7 + triggering a machine type change from i440fx to q35. Beyond that, + virt-v2v should preserve the on-bus index. --> <target dev='hdc' bus='ide'/> </disk> </devices> diff --git a/tests/test-v2v-i-ova.xml b/tests/test-v2v-i-ova.xml index d7383905fdc0..9f3c1974243f 100644 --- a/tests/test-v2v-i-ova.xml +++ b/tests/test-v2v-i-ova.xml @@ -28,7 +28,7 @@ </disk> <disk device='cdrom' type='file'> <driver name='qemu' type='raw'/> - <target dev='hda' bus='ide'/> + <target dev='sda' bus='sata'/> </disk> <disk device='floppy' type='file'> <driver name='qemu' type='raw'/>
On 12/16/21 11:50, Laszlo Ersek wrote:
However... the converted guest does not boot at all. Please see the attached screenshot (it's not big).
Some more details. During the conversion, I get a warning as follows:
virt-v2v: warning: mount: mount exited with status 32: mount: /sysroot/mnt2: /dev/sda already mounted or mount point busy. (ignored)
The issue seems to be that *before* the code in my patch runs, there is a separate mapping, from /dev/hda (the CD-ROM) to /dev/sda. However, /dev/sda is already taken: it represents the (originally virtio-blk) system disk. Here's the contents of the original /etc/fstab file:
/dev/VolGroup00/LogVol00 / ext3 defaults 1 1 LABEL=/boot /boot ext3 defaults 1 2 tmpfs /dev/shm tmpfs defaults 0 0 devpts /dev/pts devpts gid=5,mode=620 0 0 sysfs /sys sysfs defaults 0 0 proc /proc proc defaults 0 0 /dev/VolGroup00/LogVol01 swap swap defaults 0 0 /dev/hda /mnt2 iso9660 noauto 0 0
And here's a bit larger snippet from the log:
libguestfs: trace: v2v: mount "/dev/hda" "/mnt2" guestfsd: => mount (0x1) took 0.13 secs guestfsd: <= mount (0x1) request length 64 bytes device name translated: /dev/hda -> /dev/sda
This seems to come from "daemon/device-name-translation.c" in libguestfs. But, I'm attaching the full conversion log too. (NB this log was generated with my patch applied; however, the patch itself makes no difference regarding the boot failure, as stated before.) FWIW, I don't know if this warning (i.e., the failed attempt to mount /dev/sda) has anything to do with the boot failure in the converted guest. Thanks, Laszlo
On 12/16/21 12:08, Laszlo Ersek wrote:
But, I'm attaching the full conversion log too. (NB this log was generated with my patch applied; however, the patch itself makes no difference regarding the boot failure, as stated before.)
FWIW, I don't know if this warning (i.e., the failed attempt to mount /dev/sda) has anything to do with the boot failure in the converted guest.
Apologies for conversing with myself :/ Noticed this (a bunch of it) in the log:
WARNING: PV /dev/sda2 in VG VolGroup00 is using an old PV header, modify the VG to update.
I don't know if we ultimately perform any operation that modifies the volume group, but if we do, it seems plausible that that update prevents the LVM driver in RHEL5 from recognizing the volume group again :( Laszlo
On Thu, Dec 16, 2021 at 12:12:33PM +0100, Laszlo Ersek wrote:
On 12/16/21 12:08, Laszlo Ersek wrote:
But, I'm attaching the full conversion log too. (NB this log was generated with my patch applied; however, the patch itself makes no difference regarding the boot failure, as stated before.)
FWIW, I don't know if this warning (i.e., the failed attempt to mount /dev/sda) has anything to do with the boot failure in the converted guest.
Apologies for conversing with myself :/
Noticed this (a bunch of it) in the log:
WARNING: PV /dev/sda2 in VG VolGroup00 is using an old PV header, modify the VG to update.
I don't know if we ultimately perform any operation that modifies the volume group, but if we do, it seems plausible that that update prevents the LVM driver in RHEL5 from recognizing the volume group again :(
This does indeed seem like a new problem (in LVM). I don't think we've seen this one before so it needs a bug to track it. About the patch: The fact that there's an existing bug does not block this patch from going upstream. It doesn't seem as if this patch is the cause. About device name translation: This is something of a mess at the moment. There's a lot of history ... Originally I didn't think very deeply about how device names should be represented in the API. For example if you had an API like int guestfs_mount (guestfs_h *g, char *device, char *mp) how should the device name be represented? In the original version it was basically the QEMU / kernel name, probably "/dev/vda1". When we switched to using virtio-scsi the device names changed ("/dev/sda1"), but that was fine I just added some code which spotted /dev/[hsv]... and translated it inside the daemon. (This is what "device name translation" does). Informally I said that "/dev/sda" in the API means the "first disk", (in the sense of "added first using guestfs_add_drive") "/dev/sdb" means second disk, "/dev/sda4" means the fourth partition of the first disk and so on. This was a convention but it wasn't encoded or enforced anywhere. That all worked for quite a long time, but then the kernel started to enumerate devices in parallel, so now if you two disks, you could no longer assume that the "first disk" might randomly appear as /dev/sda or /dev/sdb. We're still sending these strings across from the library to the daemon, but now we do some additional and very hokey translation using device serial numbers which you can check in the code (daemon/device-name-translation.c) I think what we _should_ be doing is translating the strings into an internal representation, which we'd use inside the library and serialise into the daemon. This is, naturally, an awful lot of work. Note that the ABI cannot be changed, so whatever happens we're still going to be passing strings to guestfs_* APIs, this is all about what happens internally after that point in the generated code. There's the additional complication that not everything mountable is a disk (eg. LVs, /dev/md*). Matt Booth already separated that out with a distinction between disks and "mountables". There's also the issue that we always add a hidden disk for the appliance, which is added after the other drives. We need to ignore that. There's also disk hotplugging which we should deprecate - it was a failed experiment, and indeed has never worked for the direct backend which is enough justification to replace those APIs with ones which print an error and return -1. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v
On 12/16/21 14:10, Richard W.M. Jones wrote:
On Thu, Dec 16, 2021 at 12:12:33PM +0100, Laszlo Ersek wrote:
On 12/16/21 12:08, Laszlo Ersek wrote:
But, I'm attaching the full conversion log too. (NB this log was generated with my patch applied; however, the patch itself makes no difference regarding the boot failure, as stated before.)
FWIW, I don't know if this warning (i.e., the failed attempt to mount /dev/sda) has anything to do with the boot failure in the converted guest.
Apologies for conversing with myself :/
Noticed this (a bunch of it) in the log:
WARNING: PV /dev/sda2 in VG VolGroup00 is using an old PV header, modify the VG to update.
I don't know if we ultimately perform any operation that modifies the volume group, but if we do, it seems plausible that that update prevents the LVM driver in RHEL5 from recognizing the volume group again :(
This does indeed seem like a new problem (in LVM). I don't think we've seen this one before so it needs a bug to track it.
[... snipping the other explanations -- I've read them and they're great, but there's only so much I can internalize so quickly, and right now something else seems to be the issue after all ...] So, I reinstalled the RHEL-5.11 guest from scratch, and made sure that it did not use LVM at all, only an MBR partition table and three primary partitions (/boot, swap, and /). The issue reproduces just the same (just with different names). Then, it dawned on me. RHEL-5 does not have virtio-1.0 drivers, only virtio-0.9 drivers. And a QEMU monitor command (info qtree) confirmed that, for all virtio devices in the output domain, the "disable_legacy" property was "on". Meaning, these were not transitional virtio devices, but purely virtio-1.0 devices. I've looked at both the domain XML directly generated by virt-v2v, and the variant that libvirtd actually maintains after the "virsh define". In the former, there is no PCI topology information (no <address> elements), and also no "virtio-transitional" model specifications: https://libvirt.org/formatdomain.html#virtio-transitional-devices (1) <disk type='volume' device='disk'> <driver name='qemu' type='raw'/> <source pool='default' volume='rhel5.11.out-sda'/> <target dev='vda' bus='virtio'/> </disk> (2) <interface type='bridge'> <source bridge='virbr0'/> <model type='virtio'/> <mac address='52:54:00:6c:97:64'/> </interface> (3) <memballoon model='virtio'/> Accordingly, because the Q35 machine type is a PCI Express, not traditional PCI, board, libvirtd generates the PCIe topology such that each virtio device lands in a separate PCI Express Root Port: (1) <disk type='volume' device='disk'> <driver name='qemu' type='raw'/> <source pool='default' volume='rhel5.11.out-sda'/> <target dev='vda' bus='virtio'/> <boot order='2'/> <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/> ^^^^^^^^^^ </disk> with <controller type='pci' index='5' model='pcie-root-port'> ^^^^^^^^^ (2) <interface type='bridge'> <mac address='52:54:00:6c:97:64'/> <source bridge='virbr0'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> ^^^^^^^^^^ </interface> with <controller type='pci' index='2' model='pcie-root-port'> ^^^^^^^^^ (3) <memballoon model='virtio'> <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> ^^^^^^^^^^ </memballoon> with <controller type='pci' index='6' model='pcie-root-port'> ^^^^^^^^^ In turn, the virtio-pci code in QEMU works such that, if the "disable_legacy" device property is not explicitly specified, then the virtio device's placement in the PCI(e) topology decides whether the device is transitional, or purely virtio-1.0. See function virtio_pci_realize() in "hw/virtio/virtio-pci.c": bool pcie_port = pci_bus_is_express(pci_get_bus(pci_dev)) && !pci_bus_is_root(pci_get_bus(pci_dev)); ... if (proxy->disable_legacy == ON_OFF_AUTO_AUTO) { proxy->disable_legacy = pcie_port ? ON_OFF_AUTO_ON : ON_OFF_AUTO_OFF; } The particular "pcie_port" initialization means that PCI Express Root Ports will set "pcie_port" to true, but the PCI Express *root complex* -- that is, when the virtio devices are endpoints integrated into the PCIe root complex -- will set "pcie_port" to "false". (The difference is made by pci_bus_is_root() -- returns "false" for the root ports, but "true" for the root complex.) In other words, if all these virtio devices had been placed on: <controller type='pci' index='0' model='pcie-root'/> ^^^^^^^^^ then the devices would have been transitional. In order to verify this theory, I updated the addresses like that (moving each virtio device to a different slot of bus='0x00') in the output domain. This way, the RHEL5 guest booted -- no kernel panic. (The X11 server did not start up, but I chose to ignore that.) At this point, I managed to verify my patch -- the /dev/hda entry in /etc/fstab had indeed been replaced with /dev/cdrom, and the /dev/cdrom symlink pointed to "sr0". So mounting the "/mnt2" mount point worked fine. I think this is enough evidence to push the patch. Anyway, back to the larger non-transitional problem. How can we convince libvirtd to set up transitional devices? Based on this article again: <https://libvirt.org/formatdomain.html#virtio-transitional-devices>, I went back to the XML generated by virt-v2v, and tried to update every mention of "virtio" to "virtio-transitional", as follows: @@ -21,7 +21,7 @@ <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> - <disk type='volume' device='disk'> + <disk type='volume' device='disk' model='virtio-transitional'> <driver name='qemu' type='raw'/> <source pool='default' volume='rhel5.11.out-sda'/> <target dev='vda' bus='virtio'/> @@ -32,7 +32,7 @@ </disk> <interface type='bridge'> <source bridge='virbr0'/> - <model type='virtio'/> + <model type='virtio-transitional'/> <mac address='52:54:00:6c:97:64'/> </interface> <video> @@ -40,7 +40,7 @@ </video> <graphics type='spice' autoport='yes' port='-1'/> <sound model='ich6'/> - <memballoon model='virtio'/> + <memballoon model='virtio-transitional'/> <viosock model='none'/> <input type='tablet' bus='usb'/> <input type='mouse' bus='ps2'/> When running "virsh define" on this, the effect is that libvirtd places the virtio devices on the separate "pcie-to-pci-bridge" controller (not on the root complex, as integrated endpoints, but still good -- this way, the pci_bus_is_express() call retval is what makes the difference): (1) <disk type='volume' device='disk' model='virtio-transitional'> ^^^^^^^^^^^^^^^^^^^^^^^^^^^ <driver name='qemu' type='raw'/> <source pool='default' volume='rhel5.11.out-sda' index='2'/> <backingStore/> <target dev='vda' bus='virtio'/> <alias name='virtio-disk0'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x03' function='0x0'/> ^^^^^^^^^^ </disk> with <controller type='pci' index='2' model='pcie-to-pci-bridge'> ^^^^^^^^^ (2) <interface type='bridge'> <mac address='52:54:00:6c:97:64'/> <source bridge='virbr0'/> <target dev='tap0'/> <model type='virtio-transitional'/> ^^^^^^^^^^^^^^^^^^^^^ <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x0'/> ^^^^^^^^^^ </interface> (3) <memballoon model='virtio-transitional'> ^^^^^^^^^^^^^^^^^^^^^^^^^^ <alias name='balloon0'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x04' function='0x0'/> ^^^^^^^^^^ </memballoon> This way, the guest boots as well. Summary: libosinfo should have told virt-v2v that RHEL-5.11 does not have virtio-1.0 drivers, only virtio-0.9 drivers. Accordingly, virt-v2v should have generated "virtio-transitional" model attributes, so that libvirtd would position the virtio devices in traditional PCI bus slots. Do we need a new BZ about this? Thanks! Laszlo
On Thu, Dec 16, 2021 at 05:53:11PM +0100, Laszlo Ersek wrote:
Summary: libosinfo should have told virt-v2v that RHEL-5.11 does not have virtio-1.0 drivers, only virtio-0.9 drivers. Accordingly, virt-v2v should have generated "virtio-transitional" model attributes, so that libvirtd would position the virtio devices in traditional PCI bus slots.
Do we need a new BZ about this?
Isn't it this one (albeit the title should be <= 6): https://bugzilla.redhat.com/show_bug.cgi?id=1942325 Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top
On 12/16/21 18:32, Richard W.M. Jones wrote:
On Thu, Dec 16, 2021 at 05:53:11PM +0100, Laszlo Ersek wrote:
Summary: libosinfo should have told virt-v2v that RHEL-5.11 does not have virtio-1.0 drivers, only virtio-0.9 drivers. Accordingly, virt-v2v should have generated "virtio-transitional" model attributes, so that libvirtd would position the virtio devices in traditional PCI bus slots.
Do we need a new BZ about this?
Isn't it this one (albeit the title should be <= 6):
Yes, that's it exactly! Comment 3 in the BZ touches on a question that I've had in mind myself, BTW: "BTW, rhel6 guest is not supported on rhel9 any more." I'm a bit confused as to how far back in time we plan to support guest OSes, and exactly how. Some example thoughts: - If we decide that we want to drop conversion support (even up-stream) up to and including RHEL6, then the virtio-transitional question falls away -- and *with it*, the very patch in this whole thread. :) - From commit ac39fa292c31 ("v2v: Set machine type explicitly for outputs which support it (RHBZ#1581428).", 2020-12-04), we have the following comment: + (* Pivot on the year 2007. Any Linux distro from earlier than + * 2007 should use i440fx, anything 2007 or newer should use q35. + * XXX Look up this information in libosinfo in future. + *) In case we pivoted -- based on any factor really -- such that RHEL5 ended up associated with the i440fx machine type, then it would not be affected by the virtio-transitional problem at all. I *think* that the motivation for using Q35 at all, even for guest OSes like RHEL5 and RHEL6, is motivated by RHEL downstream. (Upstream QEMU is unlikely to ever remove i440fx, in my opinion.) However, if we consider RHEL downstream a guiding principle, then we should also consider that RHEL6 (and earlier) are not supported on RHEL9+ hosts at all! So, while I'm admittedly confused, I feel like we're trying to fill a gap here that nobody really cares about. Personally I'd just keep RHEL5 and RHEL6 guests on i440fx, upstream, sidestepping the whole virtio-transitional problem, and allow RHEL9 hosts to forget about these guests in peace. Please enlighten me :) Back to this patch: after I (unknowingly) worked around RHBZ#1942325, manually, I managed to test the patch successfully. Can you please ACK it? Thanks, Laszlo
On Fri, Dec 17, 2021 at 09:55:56AM +0100, Laszlo Ersek wrote:
On 12/16/21 18:32, Richard W.M. Jones wrote:
On Thu, Dec 16, 2021 at 05:53:11PM +0100, Laszlo Ersek wrote:
Summary: libosinfo should have told virt-v2v that RHEL-5.11 does not have virtio-1.0 drivers, only virtio-0.9 drivers. Accordingly, virt-v2v should have generated "virtio-transitional" model attributes, so that libvirtd would position the virtio devices in traditional PCI bus slots.
Do we need a new BZ about this?
Isn't it this one (albeit the title should be <= 6):
Yes, that's it exactly!
Comment 3 in the BZ touches on a question that I've had in mind myself, BTW:
"BTW, rhel6 guest is not supported on rhel9 any more."
There was definitely discussion about this at Red Hat. I'm not sure what the current situation really is. But if we can make it work with a simple change (like in this bug) then we probably ought to.
I'm a bit confused as to how far back in time we plan to support guest OSes, and exactly how. Some example thoughts:
- If we decide that we want to drop conversion support (even up-stream) up to and including RHEL6, then the virtio-transitional question falls away -- and *with it*, the very patch in this whole thread. :)
- From commit ac39fa292c31 ("v2v: Set machine type explicitly for outputs which support it (RHBZ#1581428).", 2020-12-04), we have the following comment:
+ (* Pivot on the year 2007. Any Linux distro from earlier than + * 2007 should use i440fx, anything 2007 or newer should use q35. + * XXX Look up this information in libosinfo in future. + *)
In case we pivoted -- based on any factor really -- such that RHEL5 ended up associated with the i440fx machine type, then it would not be affected by the virtio-transitional problem at all.
I *think* that the motivation for using Q35 at all, even for guest OSes like RHEL5 and RHEL6, is motivated by RHEL downstream. (Upstream QEMU is unlikely to ever remove i440fx, in my opinion.) However, if we consider RHEL downstream a guiding principle, then we should also consider that RHEL6 (and earlier) are not supported on RHEL9+ hosts at all!
So, while I'm admittedly confused, I feel like we're trying to fill a gap here that nobody really cares about. Personally I'd just keep RHEL5 and RHEL6 guests on i440fx, upstream, sidestepping the whole virtio-transitional problem, and allow RHEL9 hosts to forget about these guests in peace.
Please enlighten me :)
What RHEL supports, what virt-v2v offers and what customers do are often in conflict. No one is using virt-v2v when they have a way to ansible-deploy their entire cloud infrastructure of "cattle" from a single button press. They're using virt-v2v (and even more so, virt-p2v) because they have very special pet machines that they have no idea how to rebuild because the sysadmin who installed it left the company / the obscure CNC machine software isn't supported on newer OSes and the manufacturer went out of business / they have some regulatory requirement to use RHEL 5 / they don't want to touch that thing in the corner which is working. So we need to keep things going for them even if RHEL doesn't necessarily support their use case. That means making conversions of RHEL 5/6 work on RHEL 9, especially if it doesn't really require much effort as in this case.
Back to this patch: after I (unknowingly) worked around RHBZ#1942325, manually, I managed to test the patch successfully. Can you please ACK it?
Sure, ACK to that patch, thanks! Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v
On Fri, Dec 17, 2021 at 10:21:39AM +0000, Richard W.M. Jones wrote:
So we need to keep things going for them even if RHEL doesn't necessarily support their use case. That means making conversions of RHEL 5/6 work on RHEL 9
I mean "on RHEL 9 with Q35", my assumption being that either RHEL 9 has already dropped i440fx, or that using i440fx will produce some big scary deprecation warnings (not sure which is going to happen). Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW
On Fri, Dec 17, 2021 at 10:31:40AM +0000, Richard W.M. Jones wrote:
On Fri, Dec 17, 2021 at 10:21:39AM +0000, Richard W.M. Jones wrote:
So we need to keep things going for them even if RHEL doesn't necessarily support their use case. That means making conversions of RHEL 5/6 work on RHEL 9
I mean "on RHEL 9 with Q35", my assumption being that either RHEL 9 has already dropped i440fx, or that using i440fx will produce some big scary deprecation warnings (not sure which is going to happen).
i440fx will exist for the purpose of incoming migrations from RHEL-8 because OpenStack still used i440fx for the lifetime of 8. It should not be used for newly provisioned VMs on 9, and is likely to be marked as deprecated when querying from QEMU with QMP. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 12/17/21 11:21, Richard W.M. Jones wrote:
On Fri, Dec 17, 2021 at 09:55:56AM +0100, Laszlo Ersek wrote:
Back to this patch: after I (unknowingly) worked around RHBZ#1942325, manually, I managed to test the patch successfully. Can you please ACK it?
Sure, ACK to that patch, thanks!
Commit eb7af78bea30. Thanks! Laszlo
participants (3)
-
Daniel P. Berrangé -
Laszlo Ersek -
Richard W.M. Jones