Re: Can't boot UEFI VM after migration: "error: can't find command `linux16`"
Hi Richard, This is a more comprehensive description of the problem, where I respond to the specific issue and questions that you brought up during the discussion last week. I also cover the proposed solution towards the end. Kindly bear with the somewhat longish one - *[Comment 1] - Specific behaviors of the GRUB bootloader when encountering "Linux 16" versus "Linux EFI" directives.* - linux16 - Directive in GRUB boot loader to load the Linux Kernel image. It is documented in the GNU GRUB documentation - linux16 <https://doc.guix.gnu.org/grub/latest/en/grub.html#linux16> - The linux16 command forces GRUB load Linux Kernel image by using legacy *16-bit real-mode boot protocol*. - GRUB hands control to the kernel *while the CPU is still in 16-bit Real Mode*. - This allows the kernel's real-mode setup code to perform early hardware initialization using BIOS services (e.g., querying video modes, memory map)" - The Linux Kernel switches itself into 32-bit protected mode. - initrd16 loads the initramfs and is paired with linux16 - linuxefi - downstream GRUB2 bootloader directives specific to RHEL 7 for UEFI systems. Documented in RHEL 7 System Administrator Guide <https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/7/html/system_administrators_guide/part-kernel_customization_with_bootloader> - Directive specific to RHEL 7. These directives are deprecated in RHEL 8 upwards which use standard GRUB directives linux, initrd instead and these work uniformly across BIOS or UEFI systems. - linuxefi and initrdefi directives use UEFI boot services to load the Linux Kernel image and initramfs respectively. - Based on the boot mode of the system - These GRUB configuration files are created using scripts in /etc/grub.d/ - primary script which generates the kernel image, initramfs boot loader commands is /etc/grub.d/10_linux. - RHEL 7.9 VM in BIOS Mode - GRUB reads the file /boot/grub2/grub.cfg which has the directives {linux16, initrd16} [root@localhost ~]# cat /boot/grub2/grub.cfg | egrep "linux16|initrd16" *linux16* /vmlinuz-3.10.0-1160.el7.x86_64 root=UUID=b635e330-0173-4866-9e5e-693086760507 ro crashkernel=auto rhgb quiet LANG=en_US.UTF-8 *initrd16* /initramfs-3.10.0-1160.el7.x86_64.img *linux16* /vmlinuz-0-rescue-bb246b817b814a44b5e960404fdc7ae3 root=UUID=b635e330-0173-4866 -9e5e-693086760507 ro crashkernel=auto rhgb quiet *initrd16* /initramfs-0-rescue-bb246b817b814a44b5e960404fdc7ae3.img - RHEL 7.9 VM in UEFI Mode - GRUB reads the file /boot/efi/EFI/redhat/grub.cfg which has the directives {linux16, initrd16} [root@localhost ~]# cat /boot/efi/EFI/redhat/grub.cfg | egrep "linuxefi|initrdefi" *linuxefi* /vmlinuz-3.10.0-1160.el7.x86_64 root=/dev/mapper/rhel-root ro crashkernel=auto rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap rhgb quiet *initrdefi* /initramfs-3.10.0-1160.el7.x86_64.img *linuxefi* /vmlinuz-0-rescue-6e4e30972e924630b169d78059b932d2 root=/dev/mapper/rhel-root ro crashkernel=auto rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap rhgb quiet *initrdefi* /initramfs-0-rescue-6e4e30972e924630b169d78059b932d2.img - Code snippet from /etc/grub.d/10_linux for RHEL9 and RHEL7. The root device UUID passed to the kernel (e.g., root=UUID=...) is dynamically queried and formatted by helper routines in /usr/share/grub/grub-mkconfig_lib using grub-probe . - Code snippet from */etc/grub.d/10_linux in RHEL9* if [ -d /sys/firmware/efi ]; then bootefi_device="`${grub_probe} --target=device /boot/efi/`" prepare_grub_to_access_device_with_variable boot ${bootefi_device} else boot_device="`${grub_probe} --target=device /boot/`" prepare_grub_to_access_device_with_variable boot ${boot_device} fi - Code snippet from */etc/grub.d/10_linux in RHEL7* if [ -d /sys/firmware/efi ]; then sed "s/^/$submenu_indentation/" << EOF ${linuxefi} ${rel_dirname}/${basename} root=${linux_root_device_thisversion} ro ${args} EOF else sed "s/^/$submenu_indentation/" << EOF linux${sixteenbit} ${rel_dirname}/${basename} root=${linux_root_device_thisversion} ro ${args} EOF fi - *[Comment 2] Confirming that the virtual machine Linux file format is fully compatible with the new directive* - Based on the analysis done for the previous comment, we can safely conclude that this problem is very specific to RHEL 7 only. Further, the problem occurs when following conditions are met - Source VM must be in UEFI boot mode - In the /etc/fstab, the /boot and /boot/efi partitions must be identified by the corresponding /dev/sd* files instead of the UUIDs (default) generated during FS creation. - The second condition forces virt-v2v conversion to configure GRUB. - As the grub configuration executes in context of libguestfs appliance (BIOS mode), it is generated with linux16, initrd16 whereas the VM is configured to boot in UEFI. - The VM powers up with UEFI firmware, but the linux16 directive switches the CPU into 16-bit real mode and look for BIOS services (/boot/grub2/i386-pc) which are absent. - *[Comment 3] Regarding the potential modification of file system table entries and disk device naming conventions, Richard Jones insisted that any statements regarding potential disk access problems must be accompanied by a detailed explanation of the "why" and "what" behind those risks* - The kernel device discovery (names /dev/sd*) is non-deterministic as it is based on order of driver initialization at the boot time. The order variation is due to asynchronous driver probe, kernel update or VM configuration changes. - By "hard-coding" the device names inside /etc/fstab, there could be a possibility of an unintended device being mounted onto an incorrect mount point, thus causing the system to fail or corrupt. - UUIDs (written into FS) are more reliable - where order of initialization would not matter making it more deterministic. - *The proposed solution* - Convert the references to /dev/sd* files to corresponding FS UUIDs *[Implemented and tested]* - Following change applicable only if OS is RHEL 7 and Boot mode is UEFI - Replace {linux16, initrd16} with {linuxefi, initrdefi} *[Work in progress to change from using sed to using APIs]* Thanks Srihari
On Mon, Aug 03, 2026 at 01:45:44PM +0530, Srihari Parimi via Libguestfs wrote:
Hi Richard,
This is a more comprehensive description of the problem, where I respond to the specific issue and questions that you brought up during the discussion last week. I also cover the proposed solution towards the end. Kindly bear with the somewhat longish one
What thread is this all in response to ? AFAICT, there's nothing in the list archives with this subjct.
- *[Comment 2] Confirming that the virtual machine Linux file format is fully compatible with the new directive* - Based on the analysis done for the previous comment, we can safely conclude that this problem is very specific to RHEL 7 only. Further, the problem occurs when following conditions are met - Source VM must be in UEFI boot mode - In the /etc/fstab, the /boot and /boot/efi partitions must be identified by the corresponding /dev/sd* files instead of the UUIDs (default) generated during FS creation. - The second condition forces virt-v2v conversion to configure GRUB. - As the grub configuration executes in context of libguestfs appliance (BIOS mode), it is generated with linux16, initrd16 whereas the VM is configured to boot in UEFI. - The VM powers up with UEFI firmware, but the linux16 directive switches the CPU into 16-bit real mode and look for BIOS services (/boot/grub2/i386-pc) which are absent.
- *[Comment 3] Regarding the potential modification of file system table entries and disk device naming conventions, Richard Jones insisted that any statements regarding potential disk access problems must be accompanied by a detailed explanation of the "why" and "what" behind those risks* - The kernel device discovery (names /dev/sd*) is non-deterministic as it is based on order of driver initialization at the boot time. The order variation is due to asynchronous driver probe, kernel update or VM configuration changes.
Conceptually I agree, but in practice this is does not look like an actual problem. If the original guest before v2v was using device names already, then it must have been OK, unless the guest was already broken in some way.
- By "hard-coding" the device names inside /etc/fstab, there could be a possibility of an unintended device being mounted onto an incorrect mount point, thus causing the system to fail or corrupt. - UUIDs (written into FS) are more reliable - where order of initialization would not matter making it more deterministic.
- *The proposed solution* - Convert the references to /dev/sd* files to corresponding FS UUIDs *[Implemented and tested]*
IMHO the use of device names vs UUIDs in an image is a end user choice, and v2v shouldn't really override that.
- Following change applicable only if OS is RHEL 7 and Boot mode is UEFI - Replace {linux16, initrd16} with {linuxefi, initrdefi} *[Work in progress to change from using sed to using APIs]*
Rather than hardcoding that specific logic, why not query the original grub.conf and preserve what it used in the new config? Or not even re-generate grub.conf at all, instead replace the device paths inline ? With regards, Daniel -- |: https://berrange.com ~~ https://hachyderm.io/@berrange :| |: https://libvirt.org ~~ https://entangle-photo.org :| |: https://pixelfed.art/berrange ~~ https://fstop138.berrange.com :|
On Mon, Aug 03, 2026 at 11:54:44AM +0100, Daniel P. Berrangé via Libguestfs wrote:
On Mon, Aug 03, 2026 at 01:45:44PM +0530, Srihari Parimi via Libguestfs wrote:
Hi Richard,
This is a more comprehensive description of the problem, where I respond to the specific issue and questions that you brought up during the discussion last week. I also cover the proposed solution towards the end. Kindly bear with the somewhat longish one
What thread is this all in response to ? AFAICT, there's nothing in the list archives with this subjct.
I believe it's related to this confusing upstream pull request: https://github.com/libguestfs/virt-v2v/pull/160
- The kernel device discovery (names /dev/sd*) is non-deterministic as it is based on order of driver initialization at the boot time. The order variation is due to asynchronous driver probe, kernel update or VM configuration changes.
Conceptually I agree, but in practice this is does not look like an actual problem. If the original guest before v2v was using device names already, then it must have been OK, unless the guest was already broken in some way.
[Note this is largely unrelated to the PR above.] We've been discussing if it's a good idea to simply replace any /dev/XdX device reference with a UUID, where possible, for a few reasons: (1) A kernel upgrade can break an existing working guest. (2) Admins use device references wrongly, and it mostly works but occasionally breaks. For example it can work fine until you add a second disk and then it becomes intermittently flakey. (3) It allows you to move devices around or change drivers in future without breaking the guest. Basically it's just a good idea to use UUIDs to reference filesystems. The con is that we want to make as few changes as possible, because every change we make in v2v can have unintended consequences when used at the scale of conversions we are doing now. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v
On Wed, Aug 05, 2026 at 10:29:16AM +0100, Richard W.M. Jones wrote:
On Mon, Aug 03, 2026 at 11:54:44AM +0100, Daniel P. Berrangé via Libguestfs wrote:
On Mon, Aug 03, 2026 at 01:45:44PM +0530, Srihari Parimi via Libguestfs wrote:
Hi Richard,
This is a more comprehensive description of the problem, where I respond to the specific issue and questions that you brought up during the discussion last week. I also cover the proposed solution towards the end. Kindly bear with the somewhat longish one
What thread is this all in response to ? AFAICT, there's nothing in the list archives with this subjct.
I believe it's related to this confusing upstream pull request:
https://github.com/libguestfs/virt-v2v/pull/160
- The kernel device discovery (names /dev/sd*) is non-deterministic as it is based on order of driver initialization at the boot time. The order variation is due to asynchronous driver probe, kernel update or VM configuration changes.
Conceptually I agree, but in practice this is does not look like an actual problem. If the original guest before v2v was using device names already, then it must have been OK, unless the guest was already broken in some way.
[Note this is largely unrelated to the PR above.]
We've been discussing if it's a good idea to simply replace any /dev/XdX device reference with a UUID, where possible, for a few reasons:
(1) A kernel upgrade can break an existing working guest.
(2) Admins use device references wrongly, and it mostly works but occasionally breaks. For example it can work fine until you add a second disk and then it becomes intermittently flakey.
(3) It allows you to move devices around or change drivers in future without breaking the guest.
Basically it's just a good idea to use UUIDs to reference filesystems.
I agree, but I'm very sceptical that v2v should be forcing that decision on users during a VM conversion.
The con is that we want to make as few changes as possible, because every change we make in v2v can have unintended consequences when used at the scale of conversions we are doing now.
Yep, there may be reasons an org chose to use device names that we are not aware of. If an organization has a corporate standard provisioning workflow that uses device names, we should honour that in VMs we convert IMHO. With regards, Daniel -- |: https://berrange.com ~~ https://hachyderm.io/@berrange :| |: https://libvirt.org ~~ https://entangle-photo.org :| |: https://pixelfed.art/berrange ~~ https://fstop138.berrange.com :|
On Wed, Aug 05, 2026 at 10:47:34AM +0100, Daniel P. Berrangé wrote:
On Wed, Aug 05, 2026 at 10:29:16AM +0100, Richard W.M. Jones wrote:
On Mon, Aug 03, 2026 at 11:54:44AM +0100, Daniel P. Berrangé via Libguestfs wrote:
On Mon, Aug 03, 2026 at 01:45:44PM +0530, Srihari Parimi via Libguestfs wrote:
Hi Richard,
This is a more comprehensive description of the problem, where I respond to the specific issue and questions that you brought up during the discussion last week. I also cover the proposed solution towards the end. Kindly bear with the somewhat longish one
What thread is this all in response to ? AFAICT, there's nothing in the list archives with this subjct.
I believe it's related to this confusing upstream pull request:
https://github.com/libguestfs/virt-v2v/pull/160
- The kernel device discovery (names /dev/sd*) is non-deterministic as it is based on order of driver initialization at the boot time. The order variation is due to asynchronous driver probe, kernel update or VM configuration changes.
Conceptually I agree, but in practice this is does not look like an actual problem. If the original guest before v2v was using device names already, then it must have been OK, unless the guest was already broken in some way.
[Note this is largely unrelated to the PR above.]
We've been discussing if it's a good idea to simply replace any /dev/XdX device reference with a UUID, where possible, for a few reasons:
(1) A kernel upgrade can break an existing working guest.
(2) Admins use device references wrongly, and it mostly works but occasionally breaks. For example it can work fine until you add a second disk and then it becomes intermittently flakey.
(3) It allows you to move devices around or change drivers in future without breaking the guest.
Basically it's just a good idea to use UUIDs to reference filesystems.
I agree, but I'm very sceptical that v2v should be forcing that decision on users during a VM conversion.
The con is that we want to make as few changes as possible, because every change we make in v2v can have unintended consequences when used at the scale of conversions we are doing now.
Yep, there may be reasons an org chose to use device names that we are not aware of.
If an organization has a corporate standard provisioning workflow that uses device names, we should honour that in VMs we convert IMHO.
Sure, I agree. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com nbdkit - Flexible, fast NBD server with plugins https://gitlab.com/nbdkit/nbdkit
Hi All, First, my apologies for the rather delayed response. Also, once the approach is finalized and approved, I will push any new commits and include the justification and description in the final commit message. On Mon, Aug 3, 2026 at 4:24 PM Daniel P. Berrangé <berrange@redhat.com> wrote:
On Mon, Aug 03, 2026 at 01:45:44PM +0530, Srihari Parimi via Libguestfs wrote:
Hi Richard,
This is a more comprehensive description of the problem, where I respond to the specific issue and questions that you brought up during the discussion last week. I also cover the proposed solution towards the end. Kindly bear with the somewhat longish one
What thread is this all in response to ? AFAICT, there's nothing in the list archives with this subjct.
As Richard mentioned - https://github.com/libguestfs/virt-v2v/pull/160
- *[Comment 2] Confirming that the virtual machine Linux file format is fully compatible with the new directive* - Based on the analysis done for the previous comment, we can safely conclude that this problem is very specific to RHEL 7 only. Further, the problem occurs when following conditions are met - Source VM must be in UEFI boot mode - In the /etc/fstab, the /boot and /boot/efi partitions must be identified by the corresponding /dev/sd* files instead of the UUIDs (default) generated during FS creation. - The second condition forces virt-v2v conversion to configure GRUB. - As the grub configuration executes in context of libguestfs appliance (BIOS mode), it is generated with linux16, initrd16 whereas the VM is configured to boot in UEFI. - The VM powers up with UEFI firmware, but the linux16 directive switches the CPU into 16-bit real mode and look for BIOS services (/boot/grub2/i386-pc) which are absent.
- *[Comment 3] Regarding the potential modification of file system table entries and disk device naming conventions, Richard Jones insisted that any statements regarding potential disk access problems must be accompanied by a detailed explanation of the "why" and "what" behind those risks* - The kernel device discovery (names /dev/sd*) is non-deterministic as it is based on order of driver initialization at the boot time. The order variation is due to asynchronous driver probe, kernel update or VM configuration changes.
Conceptually I agree, but in practice this is does not look like an actual problem. If the original guest before v2v was using device names already, then it must have been OK, unless the guest was already broken in some way.
- By "hard-coding" the device names inside /etc/fstab, there could be a possibility of an unintended device being mounted onto an incorrect mount point, thus causing the system to fail or corrupt. - UUIDs (written into FS) are more reliable - where order of initialization would not matter making it more deterministic.
- *The proposed solution* - Convert the references to /dev/sd* files to corresponding FS UUIDs *[Implemented and tested]*
IMHO the use of device names vs UUIDs in an image is a end user choice, and v2v shouldn't really override that.
It looks like we must not convert device files to their corresponding UUIDs. Yes this seems to be safer approach.
- Following change applicable only if OS is RHEL 7 and Boot mode
is UEFI
- Replace {linux16, initrd16} with {linuxefi, initrdefi} *[Work in progress to change from using sed to using APIs]*
Rather than hardcoding that specific logic, why not query the original grub.conf and preserve what it used in the new config? Or not even re-generate grub.conf at all, instead replace the device paths inline ?
- There seems to be two ways to address - - Like you suggest, do not regenerate GRUB config - Avoid running grub2-mkconfig - if the source VM has {linuxefi, initrdefi}. This is possible if and only if source VM is RHEL 7 (verified in RHEL 7.9, RHEL 7.6) - virt-v2v in convert_linux.ml is already making a map of {source VM device files, Virtio Storage device files}. This could be used in the next step - This step Needs to be done: Inline replacement of device paths in /boot/efi/EFI/redhat/grub.cfg with corresponding virtio storage device (E.g. /dev/sda1 → /dev/vda1) - Problems: Directly parsing and editing grub.cfg - can cause missed references or path inconsistencies across custom kernel parameters. There are multiple modes of configuring GRUB - use of tools like grubby, anaconda kick start configuration, /etc/grub.d/<scripts> etc. - Let the grub2-mkconfig run - Check if the source VM uses {linuxefi, initrdef} in the grub.cfg and mark it so - Run the grub2-mkconfig (runs in libguestfs appliance context, BIOS boot mode) - If it was marked, then replace the {linux16, initrd16} with {linuxefi, initrdefi} - This approach is more deterministic for GRUB configuration. But we need to replace the GRUB directives (conditionally)
With regards, Daniel -- |: https://berrange.com ~~ https://hachyderm.io/@berrange :| |: https://libvirt.org ~~ https://entangle-photo.org :| |: https://pixelfed.art/berrange ~~ https://fstop138.berrange.com :|
On Mon, Aug 03, 2026 at 01:45:44PM +0530, Srihari Parimi via Libguestfs wrote:
Hi Richard,
This is a more comprehensive description of the problem, where I respond to the specific issue and questions that you brought up during the discussion last week. I also cover the proposed solution towards the end. Kindly bear with the somewhat longish one
• [Comment 1] - Specific behaviors of the GRUB bootloader when encountering "Linux 16" versus "Linux EFI" directives. □ linux16 - Directive in GRUB boot loader to load the Linux Kernel image. It is documented in the GNU GRUB documentation - linux16 ☆ The linux16 command forces GRUB load Linux Kernel image by using legacy 16-bit real-mode boot protocol. ☆ GRUB hands control to the kernel while the CPU is still in 16-bit Real Mode. ☆ This allows the kernel's real-mode setup code to perform early hardware initialization using BIOS services (e.g., querying video modes, memory map)" ☆ The Linux Kernel switches itself into 32-bit protected mode. ☆ initrd16 loads the initramfs and is paired with linux16 □ linuxefi - downstream GRUB2 bootloader directives specific to RHEL 7 for UEFI systems. Documented in RHEL 7 System Administrator Guide ☆ Directive specific to RHEL 7. These directives are deprecated in RHEL 8 upwards which use standard GRUB directives linux, initrd instead and these work uniformly across BIOS or UEFI systems. ☆ linuxefi and initrdefi directives use UEFI boot services to load the Linux Kernel image and initramfs respectively.
I think this was the context I was missing. So these are RHEL 7 only directives that allow a kernel/initramfs to be loaded either in 16 bit mode or as an EFI binary. Are all RHEL 7 kernels compatible with either boot method? If these directives were deprecated or removed in RHEL 8+ then that is a justification for only implementing the change specifically in RHEL 7 guests.
□ Based on the boot mode of the system ☆ These GRUB configuration files are created using scripts in /etc/ grub.d/ - primary script which generates the kernel image, initramfs boot loader commands is /etc/grub.d/10_linux. ○ RHEL 7.9 VM in BIOS Mode - GRUB reads the file /boot/grub2/ grub.cfg which has the directives {linux16, initrd16}
[root@localhost ~]# cat /boot/grub2/grub.cfg | egrep "linux16|initrd16" linux16 /vmlinuz-3.10.0-1160.el7.x86_64 root=UUID=b635e330-0173-4866-9e5e-693086760507 ro crashkernel=auto rhgb quiet LANG=en_US.UTF-8 initrd16 /initramfs-3.10.0-1160.el7.x86_64.img linux16 /vmlinuz-0-rescue-bb246b817b814a44b5e960404fdc7ae3 root=UUID=b635e330-0173-4866 -9e5e-693086760507 ro crashkernel=auto rhgb quiet initrd16 /initramfs-0-rescue-bb246b817b814a44b5e960404fdc7ae3.img
• RHEL 7.9 VM in UEFI Mode - GRUB reads the file /boot/efi/EFI/redhat/ grub.cfg which has the directives {linux16, initrd16}
ITYM linuxefi, initrdefi?
[root@localhost ~]# cat /boot/efi/EFI/redhat/grub.cfg | egrep "linuxefi|initrdefi" linuxefi /vmlinuz-3.10.0-1160.el7.x86_64 root=/dev/mapper/rhel-root ro crashkernel=auto rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap rhgb quiet initrdefi /initramfs-3.10.0-1160.el7.x86_64.img linuxefi /vmlinuz-0-rescue-6e4e30972e924630b169d78059b932d2 root=/dev/mapper/rhel-root ro crashkernel=auto rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap rhgb quiet initrdefi /initramfs-0-rescue-6e4e30972e924630b169d78059b932d2.img
• Code snippet from /etc/grub.d/10_linux for RHEL9 and RHEL7. The root device UUID passed to the kernel (e.g., root=UUID=...) is dynamically queried and formatted by helper routines in /usr/share/grub/grub-mkconfig_lib using grub-probe . □ Code snippet from /etc/grub.d/10_linux in RHEL9
if [ -d /sys/firmware/efi ]; then bootefi_device="`${grub_probe} --target=device /boot/efi/`" prepare_grub_to_access_device_with_variable boot ${bootefi_device} else boot_device="`${grub_probe} --target=device /boot/`" prepare_grub_to_access_device_with_variable boot ${boot_device} fi
• Code snippet from /etc/grub.d/10_linux in RHEL7
if [ -d /sys/firmware/efi ]; then sed "s/^/$submenu_indentation/" << EOF ${linuxefi} ${rel_dirname}/${basename} root=${linux_root_device_thisversion} ro ${args} EOF else sed "s/^/$submenu_indentation/" << EOF linux${sixteenbit} ${rel_dirname}/${basename} root=${linux_root_device_thisversion} ro ${args} EOF fi
Jeeez ... Grub is so complicated & over-engineered ...
• [Comment 2] Confirming that the virtual machine Linux file format is fully compatible with the new directive
I'm not clear that the text below does confirm this.
□ Based on the analysis done for the previous comment, we can safely conclude that this problem is very specific to RHEL 7 only. Further, the problem occurs when following conditions are met ☆ Source VM must be in UEFI boot mode ☆ In the /etc/fstab, the /boot and /boot/efi partitions must be identified by the corresponding /dev/sd* files instead of the UUIDs (default) generated during FS creation. □ The second condition forces virt-v2v conversion to configure GRUB. ☆ As the grub configuration executes in context of libguestfs appliance (BIOS mode), it is generated with linux16, initrd16 whereas the VM is configured to boot in UEFI. ☆ The VM powers up with UEFI firmware, but the linux16 directive switches the CPU into 16-bit real mode and look for BIOS services (/boot/grub2/i386-pc) which are absent.
As ever I can only review what's in the pull request, and currently the pull request is the same as it always was, so we're no closer to actually merging anything. Also, all the justifications must be in the final commit message, not random comments spread over github and mailing lists. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
On Wed, Aug 5, 2026 at 3:06 PM Richard W.M. Jones <rjones@redhat.com> wrote:
On Mon, Aug 03, 2026 at 01:45:44PM +0530, Srihari Parimi via Libguestfs wrote:
Hi Richard,
This is a more comprehensive description of the problem, where I respond to the specific issue and questions that you brought up during the discussion last week. I also cover the proposed solution towards the end. Kindly bear with the somewhat longish one
• [Comment 1] - Specific behaviors of the GRUB bootloader when encountering "Linux 16" versus "Linux EFI" directives. □ linux16 - Directive in GRUB boot loader to load the Linux Kernel image. It is documented in the GNU GRUB documentation - linux16 ☆ The linux16 command forces GRUB load Linux Kernel image by using legacy 16-bit real-mode boot protocol. ☆ GRUB hands control to the kernel while the CPU is still in 16-bit Real Mode. ☆ This allows the kernel's real-mode setup code to perform early hardware initialization using BIOS services (e.g., querying video modes, memory map)" ☆ The Linux Kernel switches itself into 32-bit protected mode. ☆ initrd16 loads the initramfs and is paired with linux16 □ linuxefi - downstream GRUB2 bootloader directives specific to RHEL 7 for UEFI systems. Documented in RHEL 7 System Administrator Guide ☆ Directive specific to RHEL 7. These directives are deprecated in RHEL 8 upwards which use standard GRUB directives linux, initrd instead and these work uniformly across BIOS or UEFI systems. ☆ linuxefi and initrdefi directives use UEFI boot services to load the Linux Kernel image and initramfs respectively.
I think this was the context I was missing. So these are RHEL 7 only directives that allow a kernel/initramfs to be loaded either in 16 bit mode or as an EFI binary.
Are all RHEL 7 kernels compatible with either boot method?
I verified this in RHEL 7.6 too - below is snapshot. If needed we can verify it in prior RHEL 7 versions too. [root@dhcp-6-172-238 ~]# cat /etc/os-release | grep VERSION VERSION="7.6 (Maipo)" [root@dhcp-6-172-238 ~]# cat /boot/efi/EFI/redhat/grub.cfg | grep -E "linuxefi|initrdefi" linuxefi /vmlinuz-3.10.0-957.el7.x86_64 root=/dev/mapper/rhel-root ro crashkernel=auto rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap rhgb quiet LANG=en_US.UTF-8 initrdefi /initramfs-3.10.0-957.el7.x86_64.img linuxefi /vmlinuz-0-rescue-9d978a092fa6417ca4c724320cc009f2 root=/dev/mapper/rhel-root ro crashkernel=auto rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap rhgb quiet initrdefi /initramfs-0-rescue-9d978a092fa6417ca4c724320cc009f2.img
If these directives were deprecated or removed in RHEL 8+ then that is a justification for only implementing the change specifically in RHEL 7 guests.
□ Based on the boot mode of the system ☆ These GRUB configuration files are created using scripts in
/etc/
grub.d/ - primary script which generates the kernel image, initramfs boot loader commands is /etc/grub.d/10_linux. ○ RHEL 7.9 VM in BIOS Mode - GRUB reads the file
/boot/grub2/
grub.cfg which has the directives {linux16, initrd16}
[root@localhost ~]# cat /boot/grub2/grub.cfg | egrep "linux16|initrd16" linux16 /vmlinuz-3.10.0-1160.el7.x86_64
root=UUID=b635e330-0173-4866-9e5e-693086760507
ro crashkernel=auto rhgb quiet LANG=en_US.UTF-8 initrd16 /initramfs-3.10.0-1160.el7.x86_64.img linux16 /vmlinuz-0-rescue-bb246b817b814a44b5e960404fdc7ae3 root=UUID=b635e330-0173-4866 -9e5e-693086760507 ro crashkernel=auto rhgb quiet initrd16 /initramfs-0-rescue-bb246b817b814a44b5e960404fdc7ae3.img
• RHEL 7.9 VM in UEFI Mode - GRUB reads the file /boot/efi/EFI/redhat/ grub.cfg which has the directives {linux16, initrd16}
ITYM linuxefi, initrdefi?
Yes - I meant linuxefi, initrdefi
[root@localhost ~]# cat /boot/efi/EFI/redhat/grub.cfg | egrep "linuxefi|initrdefi" linuxefi /vmlinuz-3.10.0-1160.el7.x86_64 root=/dev/mapper/rhel-root ro crashkernel=auto rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap rhgb quiet initrdefi /initramfs-3.10.0-1160.el7.x86_64.img linuxefi /vmlinuz-0-rescue-6e4e30972e924630b169d78059b932d2 root=/dev/mapper/rhel-root ro crashkernel=auto rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap rhgb quiet initrdefi /initramfs-0-rescue-6e4e30972e924630b169d78059b932d2.img
• Code snippet from /etc/grub.d/10_linux for RHEL9 and RHEL7. The root device UUID passed to the kernel (e.g., root=UUID=...) is dynamically queried and formatted by helper routines in /usr/share/grub/grub-mkconfig_lib using grub-probe . □ Code snippet from /etc/grub.d/10_linux in RHEL9
if [ -d /sys/firmware/efi ]; then bootefi_device="`${grub_probe} --target=device /boot/efi/`" prepare_grub_to_access_device_with_variable boot ${bootefi_device} else boot_device="`${grub_probe} --target=device /boot/`" prepare_grub_to_access_device_with_variable boot ${boot_device} fi
• Code snippet from /etc/grub.d/10_linux in RHEL7
if [ -d /sys/firmware/efi ]; then sed "s/^/$submenu_indentation/" << EOF ${linuxefi} ${rel_dirname}/${basename} root=${linux_root_device_thisversion} ro ${args} EOF else sed "s/^/$submenu_indentation/" << EOF linux${sixteenbit} ${rel_dirname}/${basename} root=${linux_root_device_thisversion} ro ${args} EOF fi
Jeeez ... Grub is so complicated & over-engineered ...
• [Comment 2] Confirming that the virtual machine Linux file format is fully compatible with the new directive
I'm not clear that the text below does confirm this.
- We are not using any new directive; rather, we are replacing incompatible directives {linux16, initrd16} with the compatible directives {linuxefi, initrdefi} that were natively used by the Source VM prior to virt-v2v conversion. Below is the snippet of /boot/efi/EFI/redhat/grub.cfg in the source VM before conversion. [root@localhost ~]# virt-what vmware [root@localhost ~]# cat /boot/efi/EFI/redhat/grub.cfg | egrep "linuxefi|initrdefi" linuxefi /vmlinuz-3.10.0-1160.el7.x86_64 root=/dev/mapper/rhel-root ro crashkernel=auto rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap rhgb quiet initrdefi /initramfs-3.10.0-1160.el7.x86_64.img linuxefi /vmlinuz-0-rescue-6e4e30972e924630b169d78059b932d2 root=/dev/mapper/rhel-root ro crashkernel=auto rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap rhgb quiet initrdefi /initramfs-0-rescue-6e4e30972e924630b169d78059b932d2.img [root@localhost ~]# - Summary of the issue - Source VM is RHEL 7.9 EFI based, grub configuration has {linuxefi, initrdefi} - virt-v2v executes grub2-mkconfig because /dev/sdX is present in /etc/fstab. - grub2-mkconfig runs in the libguestfs appliance which is in BIOS boot mode, so it ends up generating the GRUB 2 bootloader configuration{linux16, initrd16} - The VM fails to boot post-conversion because these directives require BIOS firmware - I still need to update the PR to avoid using sed; I will work on that once the approach is accepted. - The unit test which I performed with the current PR - In the Source VM, I changed /etc/fstab to use device files instead of default UUIDs - Post virt-v2v conversion, the VM booted successfully with these changes.
□ Based on the analysis done for the previous comment, we can
safely conclude that this problem is very specific to RHEL 7 only. Further, the problem occurs when following conditions are met
☆ Source VM must be in UEFI boot mode ☆ In the /etc/fstab, the /boot and /boot/efi partitions must
be identified by the corresponding /dev/sd* files instead of the UUIDs (default) generated during FS creation.
□ The second condition forces virt-v2v conversion to configure
GRUB.
☆ As the grub configuration executes in context of libguestfs
appliance (BIOS mode), it is generated with linux16, initrd16 whereas the VM is configured to boot in UEFI.
☆ The VM powers up with UEFI firmware, but the linux16
directive switches the CPU into 16-bit real mode and look for BIOS services (/boot/grub2/i386-pc) which are absent.
As ever I can only review what's in the pull request, and currently the pull request is the same as it always was, so we're no closer to actually merging anything.
As I mentioned earlier, I will update the PR once the approach is accepted.
Also, all the justifications must be in the final commit message, not random comments spread over github and mailing lists.
I agree and will put the justifications and approach in the commit message.
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
participants (3)
-
Daniel P. Berrangé -
Richard W.M. Jones -
Srihari Parimi