6:19 a.m.
Already pushed as:
https://github.com/libguestfs/virt-v2v/commit/9f12b95bbe7bc2850ce4ba134c4...
I'm posting to the list for awareness.
Rich.
6:19 a.m.
RHEL >= 9 and compatible distros like Rocky >= 9 will not boot using
the default qemu CPU. You will see an error at boot:
Fatal glibc error: CPU does not support x86-64-v2
Instead you need to use -cpu host.
In commit f28757c6d1 ("convert_linux: set "gcaps_default_cpu = false"
for x86_64 RHEL-9.0+ guests") we fixed this specifically for RHEL >= 9.
This commit extends the same fix to all RHEL family distros.
Updates: commit f28757c6d100060c65212ea55cfa59d308dcb850
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2166619
Reported-by: Ming Xie
Thanks: Laszlo Ersek
---
convert/convert_linux.ml | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/convert/convert_linux.ml b/convert/convert_linux.ml
index e20cafa551..460cbff0ed 100644
--- a/convert/convert_linux.ml
+++ b/convert/convert_linux.ml
@@ -201,9 +201,9 @@ let convert (g : G.guestfs) source inspect i_firmware
keep_serial_console _ =
(* RHEL >= 9.0 on x86_64 requires the processor to support the
"x86-64-v2"
* microarchitecture level, which the default QEMU VCPU model does not
- * satisfy. Refer to RHBZ#2076013.
+ * satisfy. Refer to RHBZ#2076013 RHBZ#2166619.
*)
- let default_cpu_suffices = inspect.i_distro <> "rhel" ||
+ let default_cpu_suffices = family <> `RHEL_family ||
inspect.i_arch <> "x86_64" ||
inspect.i_major_version < 9 in
--
2.39.0
6:37 a.m.
On Mon, Feb 06, 2023 at 12:19:40PM +0000, Richard W.M. Jones wrote:
RHEL >= 9 and compatible distros like Rocky >= 9 will not boot
using
the default qemu CPU. You will see an error at boot:
Fatal glibc error: CPU does not support x86-64-v2
Instead you need to use -cpu host.
In commit f28757c6d1 ("convert_linux: set "gcaps_default_cpu = false"
for x86_64 RHEL-9.0+ guests") we fixed this specifically for RHEL >= 9.
This commit extends the same fix to all RHEL family distros.
If v2v is going to use host CPU for a bunch of distros, why not use
it for all distros ? What's the downside in -cpu host, instead of
the default qemu64 CPU ?
Even if the distro doesn't require x86_64-v2 ABI, they'll pretty much
all benefit from NOT using the default QEMU CPU model, if only for
the fact that they gain the 'aesni' feature which increases crypto
performance massively.
Updates: commit f28757c6d100060c65212ea55cfa59d308dcb850
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2166619
Reported-by: Ming Xie
Thanks: Laszlo Ersek
---
convert/convert_linux.ml | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/convert/convert_linux.ml b/convert/convert_linux.ml
index e20cafa551..460cbff0ed 100644
--- a/convert/convert_linux.ml
+++ b/convert/convert_linux.ml
@@ -201,9 +201,9 @@ let convert (g : G.guestfs) source inspect i_firmware
keep_serial_console _ =
(* RHEL >= 9.0 on x86_64 requires the processor to support the
"x86-64-v2"
* microarchitecture level, which the default QEMU VCPU model does not
- * satisfy. Refer to RHBZ#2076013.
+ * satisfy. Refer to RHBZ#2076013 RHBZ#2166619.
*)
- let default_cpu_suffices = inspect.i_distro <> "rhel" ||
+ let default_cpu_suffices = family <> `RHEL_family ||
inspect.i_arch <> "x86_64" ||
inspect.i_major_version < 9 in
--
2.39.0
_______________________________________________
Libguestfs mailing list
Libguestfs(a)redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
6:49 a.m.
On Mon, Feb 06, 2023 at 12:37:30PM +0000, Daniel P. Berrangé wrote:
On Mon, Feb 06, 2023 at 12:19:40PM +0000, Richard W.M. Jones wrote:
> RHEL >= 9 and compatible distros like Rocky >= 9 will not boot using
> the default qemu CPU. You will see an error at boot:
>
> Fatal glibc error: CPU does not support x86-64-v2
>
> Instead you need to use -cpu host.
>
> In commit f28757c6d1 ("convert_linux: set "gcaps_default_cpu =
false"
> for x86_64 RHEL-9.0+ guests") we fixed this specifically for RHEL >= 9.
>
> This commit extends the same fix to all RHEL family distros.
If v2v is going to use host CPU for a bunch of distros, why not use
it for all distros ? What's the downside in -cpu host, instead of
the default qemu64 CPU ?
Even if the distro doesn't require x86_64-v2 ABI, they'll pretty much
all benefit from NOT using the default QEMU CPU model, if only for
the fact that they gain the 'aesni' feature which increases crypto
performance massively.
I don't know what we should do, but the current code does this:
(1) If a particular CPU model is specified by the source hypervisor
(eg VMware), use that.
(2) If the guest is RHEL family >= 9, use -cpu host / host-passthrough
(3) Otherwise don't specify any -cpu or <cpu> section
The third option seems to cause qemu / libvirt to use qemu64. However
it has the advantage that the guest is migratable afterwards.
There may be a case for changing this so:
(a) Rule (2) is changed so we use host-model (for the libvirt target).
(b) Rule (3) is changed so we always specify a minimum CPU, eg.
for x86-64 guests, choose x86-64-v2 (is that an actual CPU model??)
(a) seems like a no-brainer. Not sure why we chose host-passthrough
instead of host-model? The explanation in the commit message is not
very convincing:
https://github.com/libguestfs/virt-v2v/commit/a3e45b3ea5b45e682cb4b7ad3a5...
I think (b) is fraught because it is my understanding that there is no
good choice for a minimum CPU since we don't know anything about the
target hardware (eg. if it's Intel or AMD). And on !x86-64 we really
have no idea at all what's a good choice.
Other management systems like RHV, KubeVirt are probably ignoring our
CPU metadata anyway ...
Rich.
>
> Updates: commit f28757c6d100060c65212ea55cfa59d308dcb850
> Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2166619
> Reported-by: Ming Xie
> Thanks: Laszlo Ersek
> ---
> convert/convert_linux.ml | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/convert/convert_linux.ml b/convert/convert_linux.ml
> index e20cafa551..460cbff0ed 100644
> --- a/convert/convert_linux.ml
> +++ b/convert/convert_linux.ml
> @@ -201,9 +201,9 @@ let convert (g : G.guestfs) source inspect i_firmware
keep_serial_console _ =
>
> (* RHEL >= 9.0 on x86_64 requires the processor to support the
"x86-64-v2"
> * microarchitecture level, which the default QEMU VCPU model does not
> - * satisfy. Refer to RHBZ#2076013.
> + * satisfy. Refer to RHBZ#2076013 RHBZ#2166619.
> *)
> - let default_cpu_suffices = inspect.i_distro <> "rhel" ||
> + let default_cpu_suffices = family <> `RHEL_family ||
> inspect.i_arch <> "x86_64" ||
> inspect.i_major_version < 9 in
>
> --
> 2.39.0
>
> _______________________________________________
> Libguestfs mailing list
> Libguestfs(a)redhat.com
> https://listman.redhat.com/mailman/listinfo/libguestfs
>
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
1:23 a.m.
On 2/6/23 13:49, Richard W.M. Jones wrote:
On Mon, Feb 06, 2023 at 12:37:30PM +0000, Daniel P. Berrangé wrote:
> On Mon, Feb 06, 2023 at 12:19:40PM +0000, Richard W.M. Jones wrote:
>> RHEL >= 9 and compatible distros like Rocky >= 9 will not boot using
>> the default qemu CPU. You will see an error at boot:
>>
>> Fatal glibc error: CPU does not support x86-64-v2
>>
>> Instead you need to use -cpu host.
>>
>> In commit f28757c6d1 ("convert_linux: set "gcaps_default_cpu =
false"
>> for x86_64 RHEL-9.0+ guests") we fixed this specifically for RHEL >= 9.
>>
>> This commit extends the same fix to all RHEL family distros.
>
> If v2v is going to use host CPU for a bunch of distros, why not use
> it for all distros ? What's the downside in -cpu host, instead of
> the default qemu64 CPU ?
>
> Even if the distro doesn't require x86_64-v2 ABI, they'll pretty much
> all benefit from NOT using the default QEMU CPU model, if only for
> the fact that they gain the 'aesni' feature which increases crypto
> performance massively.
I don't know what we should do, but the current code does this:
(1) If a particular CPU model is specified by the source hypervisor
(eg VMware), use that.
(2) If the guest is RHEL family >= 9, use -cpu host / host-passthrough
(3) Otherwise don't specify any -cpu or <cpu> section
The third option seems to cause qemu / libvirt to use qemu64. However
it has the advantage that the guest is migratable afterwards.
There may be a case for changing this so:
(a) Rule (2) is changed so we use host-model (for the libvirt target).
(b) Rule (3) is changed so we always specify a minimum CPU, eg.
for x86-64 guests, choose x86-64-v2 (is that an actual CPU model??)
(a) seems like a no-brainer. Not sure why we chose host-passthrough
instead of host-model? The explanation in the commit message is not
very convincing:
https://github.com/libguestfs/virt-v2v/commit/a3e45b3ea5b45e682cb4b7ad3a5...
I think it's the result of the previous discussion we had about it. I
don't remember it -- nor do I feel very strongly about it.
I think (b) is fraught because it is my understanding that there is no
good choice for a minimum CPU since we don't know anything about the
target hardware (eg. if it's Intel or AMD). And on !x86-64 we really
have no idea at all what's a good choice.
Other management systems like RHV, KubeVirt are probably ignoring our
CPU metadata anyway ...
Yes, that's why gcaps_default_cpu is only considered in the QEMU and
libvirt outputs.
Laszlo
Rich.
>>
>> Updates: commit f28757c6d100060c65212ea55cfa59d308dcb850
>> Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2166619
>> Reported-by: Ming Xie
>> Thanks: Laszlo Ersek
>> ---
>> convert/convert_linux.ml | 4 ++--
>> 1 file changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/convert/convert_linux.ml b/convert/convert_linux.ml
>> index e20cafa551..460cbff0ed 100644
>> --- a/convert/convert_linux.ml
>> +++ b/convert/convert_linux.ml
>> @@ -201,9 +201,9 @@ let convert (g : G.guestfs) source inspect i_firmware
keep_serial_console _ =
>>
>> (* RHEL >= 9.0 on x86_64 requires the processor to support the
"x86-64-v2"
>> * microarchitecture level, which the default QEMU VCPU model does not
>> - * satisfy. Refer to RHBZ#2076013.
>> + * satisfy. Refer to RHBZ#2076013 RHBZ#2166619.
>> *)
>> - let default_cpu_suffices = inspect.i_distro <> "rhel" ||
>> + let default_cpu_suffices = family <> `RHEL_family ||
>> inspect.i_arch <> "x86_64" ||
>> inspect.i_major_version < 9 in
>>
>> --
>> 2.39.0
>>
>> _______________________________________________
>> Libguestfs mailing list
>> Libguestfs(a)redhat.com
>> https://listman.redhat.com/mailman/listinfo/libguestfs
>>
>
> With regards,
> Daniel
> --
> |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org -o- https://fstop138.berrange.com :|
> |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
2:56 a.m.
On Mon, Feb 06, 2023 at 12:49:41PM +0000, Richard W.M. Jones wrote:
On Mon, Feb 06, 2023 at 12:37:30PM +0000, Daniel P. Berrangé wrote:
> On Mon, Feb 06, 2023 at 12:19:40PM +0000, Richard W.M. Jones wrote:
> > RHEL >= 9 and compatible distros like Rocky >= 9 will not boot using
> > the default qemu CPU. You will see an error at boot:
> >
> > Fatal glibc error: CPU does not support x86-64-v2
> >
> > Instead you need to use -cpu host.
> >
> > In commit f28757c6d1 ("convert_linux: set "gcaps_default_cpu =
false"
> > for x86_64 RHEL-9.0+ guests") we fixed this specifically for RHEL >=
9.
> >
> > This commit extends the same fix to all RHEL family distros.
>
> If v2v is going to use host CPU for a bunch of distros, why not use
> it for all distros ? What's the downside in -cpu host, instead of
> the default qemu64 CPU ?
>
> Even if the distro doesn't require x86_64-v2 ABI, they'll pretty much
> all benefit from NOT using the default QEMU CPU model, if only for
> the fact that they gain the 'aesni' feature which increases crypto
> performance massively.
I don't know what we should do, but the current code does this:
(1) If a particular CPU model is specified by the source hypervisor
(eg VMware), use that.
Reasonable, if there's a match
(2) If the guest is RHEL family >= 9, use -cpu host /
host-passthrough
(3) Otherwise don't specify any -cpu or <cpu> section
The third option seems to cause qemu / libvirt to use qemu64. However
it has the advantage that the guest is migratable afterwards.
I vaguely recall in previous discussion that we decided not to
care if the guest is migratable or not, on the basis that wasn't
really domain knowledge of v2v nor an user request. Thus the
use of 'host-passthrough' rather than 'host-model' to give a
"perfect" match for the host, rather than approximation.
There may be a case for changing this so:
(a) Rule (2) is changed so we use host-model (for the libvirt target).
All hinges on whether users expect the output of v2v to be
migratable or not.
(b) Rule (3) is changed so we always specify a minimum CPU, eg.
for x86-64 guests, choose x86-64-v2 (is that an actual CPU model??)
In the case of x86-64-v2 it is the QEMU Nehalem model is a perfect
match, and is capable of running on both Intel and AMD hosts. This
is mostly luck though. For x86-64-v3 / -v4 ABIs there is no viable
model that will run on both AMD and Intel. So once there's a guest
OS that mandates -v3, we'll be back to wanting host-model/passthrough.
(a) seems like a no-brainer. Not sure why we chose host-passthrough
instead of host-model? The explanation in the commit message is not
very convincing:
https://github.com/libguestfs/virt-v2v/commit/a3e45b3ea5b45e682cb4b7ad3a5...
I think (b) is fraught because it is my understanding that there is no
good choice for a minimum CPU since we don't know anything about the
target hardware (eg. if it's Intel or AMD). And on !x86-64 we really
have no idea at all what's a good choice.
Yeah, I'd recommend against (b).
IMHO mgmt apps / provisioning tools should only ever choose host-model
or host-passthrough for KVM. Use of an explicit named CPU model should
be something left to admins to decide based on their global knowledge of
what machines they need to migrate across. Ignoring named models also
nicely avoids the problem of needing knowledge across all arches.
If you care about TCG, you could use 'maximum' instead of
'host-passthrough'.
They are identical in the case of KVM, and for TCG 'maximum' just gives you
all possible features.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
4:20 a.m.
On Tue, Feb 07, 2023 at 08:56:06AM +0000, Daniel P. Berrangé wrote:
On Mon, Feb 06, 2023 at 12:49:41PM +0000, Richard W.M. Jones wrote:
> (2) If the guest is RHEL family >= 9, use -cpu host / host-passthrough
>
> (3) Otherwise don't specify any -cpu or <cpu> section
>
> The third option seems to cause qemu / libvirt to use qemu64. However
> it has the advantage that the guest is migratable afterwards.
I vaguely recall in previous discussion that we decided not to
care if the guest is migratable or not, on the basis that wasn't
really domain knowledge of v2v nor an user request. Thus the
use of 'host-passthrough' rather than 'host-model' to give a
"perfect" match for the host, rather than approximation.
Discussion on the orignal patch:
https://listman.redhat.com/archives/libguestfs/2022-April/thread.html#28711
Maybe we discussed migratability on IRC or on some previous thread,
but all I can find there is the same commit message. (OTOH I did
review the patch and didn't query it, so ...)
I just posted a patch to change this from host-passthrough to
host-model, because I think we don't need to hinder live migration if
there's an easy way to do that.
> There may be a case for changing this so:
>
> (a) Rule (2) is changed so we use host-model (for the libvirt target).
All hinges on whether users expect the output of v2v to be
migratable or not.
> (b) Rule (3) is changed so we always specify a minimum CPU, eg.
> for x86-64 guests, choose x86-64-v2 (is that an actual CPU model??)
In the case of x86-64-v2 it is the QEMU Nehalem model is a perfect
match, and is capable of running on both Intel and AMD hosts. This
is mostly luck though. For x86-64-v3 / -v4 ABIs there is no viable
model that will run on both AMD and Intel. So once there's a guest
OS that mandates -v3, we'll be back to wanting host-model/passthrough.
Is there any easy way to query libvirt to find out what is the best
x86-64-vX compatible CPU available? Sometimes we are connected to
libvirt on the output side.
> (a) seems like a no-brainer. Not sure why we chose
host-passthrough
> instead of host-model? The explanation in the commit message is not
> very convincing:
>
https://github.com/libguestfs/virt-v2v/commit/a3e45b3ea5b45e682cb4b7ad3a5...
>
> I think (b) is fraught because it is my understanding that there is no
> good choice for a minimum CPU since we don't know anything about the
> target hardware (eg. if it's Intel or AMD). And on !x86-64 we really
> have no idea at all what's a good choice.
Yeah, I'd recommend against (b).
IMHO mgmt apps / provisioning tools should only ever choose host-model
or host-passthrough for KVM. Use of an explicit named CPU model should
be something left to admins to decide based on their global knowledge of
what machines they need to migrate across. Ignoring named models also
nicely avoids the problem of needing knowledge across all arches.
If you care about TCG, you could use 'maximum' instead of
'host-passthrough'.
They are identical in the case of KVM, and for TCG 'maximum' just gives you
all possible features.
OK.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines. Supports shell scripting,
bindings from many languages. http://libguestfs.org
4:27 a.m.
On Tue, Feb 07, 2023 at 10:20:14AM +0000, Richard W.M. Jones wrote:
On Tue, Feb 07, 2023 at 08:56:06AM +0000, Daniel P. Berrangé wrote:
> On Mon, Feb 06, 2023 at 12:49:41PM +0000, Richard W.M. Jones wrote:
> > (2) If the guest is RHEL family >= 9, use -cpu host / host-passthrough
> >
> > (3) Otherwise don't specify any -cpu or <cpu> section
> >
> > The third option seems to cause qemu / libvirt to use qemu64. However
> > it has the advantage that the guest is migratable afterwards.
>
> I vaguely recall in previous discussion that we decided not to
> care if the guest is migratable or not, on the basis that wasn't
> really domain knowledge of v2v nor an user request. Thus the
> use of 'host-passthrough' rather than 'host-model' to give a
> "perfect" match for the host, rather than approximation.
Discussion on the orignal patch:
https://listman.redhat.com/archives/libguestfs/2022-April/thread.html#28711
Maybe we discussed migratability on IRC or on some previous thread,
but all I can find there is the same commit message. (OTOH I did
review the patch and didn't query it, so ...)
I just posted a patch to change this from host-passthrough to
host-model, because I think we don't need to hinder live migration if
there's an easy way to do that.
> > There may be a case for changing this so:
> >
> > (a) Rule (2) is changed so we use host-model (for the libvirt target).
>
> All hinges on whether users expect the output of v2v to be
> migratable or not.
>
> > (b) Rule (3) is changed so we always specify a minimum CPU, eg.
> > for x86-64 guests, choose x86-64-v2 (is that an actual CPU model??)
>
> In the case of x86-64-v2 it is the QEMU Nehalem model is a perfect
> match, and is capable of running on both Intel and AMD hosts. This
> is mostly luck though. For x86-64-v3 / -v4 ABIs there is no viable
> model that will run on both AMD and Intel. So once there's a guest
> OS that mandates -v3, we'll be back to wanting host-model/passthrough.
Is there any easy way to query libvirt to find out what is the best
x86-64-vX compatible CPU available? Sometimes we are connected to
libvirt on the output side.
No, that's not a concept we expose. The x86-64-vX things are really
software ABIs aka the GCC -march / -mcpu flag arguments, and so
reporting about that kind of thing isn't especially in scope for
libvirt.
For QEMU we do however generate docs which show which CPUs can
support each x86 ABI - see the big colourful table here:
https://www.qemu.org/docs/master/system/i386/cpu.html
and the script for that is here
https://gitlab.com/qemu-project/qemu/-/blob/master/scripts/cpu-x86-uarch-...
I think it probably would be possible to query libvirt to ask for
the full expansion of the hypervisor host CPU featureset, because
ultimately the ABI is just declaring that features X, Y & Z must
exist as a minimum.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
1:23 a.m.
On 2/6/23 13:19, Richard W.M. Jones wrote:
RHEL >= 9 and compatible distros like Rocky >= 9 will not boot
using
the default qemu CPU. You will see an error at boot:
Fatal glibc error: CPU does not support x86-64-v2
Instead you need to use -cpu host.
In commit f28757c6d1 ("convert_linux: set "gcaps_default_cpu = false"
for x86_64 RHEL-9.0+ guests") we fixed this specifically for RHEL >= 9.
This commit extends the same fix to all RHEL family distros.
Updates: commit f28757c6d100060c65212ea55cfa59d308dcb850
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2166619
Reported-by: Ming Xie
Thanks: Laszlo Ersek
---
convert/convert_linux.ml | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/convert/convert_linux.ml b/convert/convert_linux.ml
index e20cafa551..460cbff0ed 100644
--- a/convert/convert_linux.ml
+++ b/convert/convert_linux.ml
@@ -201,9 +201,9 @@ let convert (g : G.guestfs) source inspect i_firmware
keep_serial_console _ =
(* RHEL >= 9.0 on x86_64 requires the processor to support the
"x86-64-v2"
* microarchitecture level, which the default QEMU VCPU model does not
- * satisfy. Refer to RHBZ#2076013.
+ * satisfy. Refer to RHBZ#2076013 RHBZ#2166619.
*)
- let default_cpu_suffices = inspect.i_distro <> "rhel" ||
+ let default_cpu_suffices = family <> `RHEL_family ||
inspect.i_arch <> "x86_64" ||
inspect.i_major_version < 9 in
Thanks!
1211
days inactive
1212
days old
8 comments
3 participants
participants (3)
-
Daniel P. Berrangé -
Laszlo Ersek -
Richard W.M. Jones