6:09 a.m.
We currently don't generate any @check attribute for the /domain/cpu
element, which causes the following libvirtd behavior
<https://libvirt.org/formatdomain.html#cpu-model-and-topology>;:
Once the domain starts, libvirt will automatically change the check
attribute to the best supported value to ensure the virtual CPU does not
change when the domain is migrated to another host
Vera Wu reports that in practice, at least when the source CPU model is
explicitly "qemu64", libvirtd sets @check='partial'. That's defined
as:
Libvirt will check the guest CPU specification before starting a
domain
This is a problem: the default "qemu64" CPU model exposes the SVM CPU
flag, and that's unsupportable on Intel hosts. SVM is the AMD counterpart
of VT-x; IOW, the flag effectively advertizes AMD-specific nesting to
guests.
With @check='partial', libvirt prevents the converted domain from starting
on Intel hosts; but with @check='none',
Libvirt does no checking and it is up to the hypervisor to refuse to
start the domain if it cannot provide the requested CPU. With QEMU this
means no checking is done at all since the default behavior of QEMU is
to emit warnings, but start the domain anyway.
We don't care about the migratability of the converted domain, so relax
the libvirtd checks, by generating the @check='none' attribute.
Consider adding @check='none' in two cases:
(1) When the source domain specifies a CPU model.
Generating @check='none' in this case fixes the issue reported by Vera.
(2) When the source domain does not specify a CPU model, and the guest OS
is assumed to work well with the default CPU model.
Generating @check='none' in this case is actually a no-op. Going from "no
CPU element" to "<cpu check='none'/>" does not change how
libvirtd
augments the domain config. Namely,
(2.1) for x86 guests, we get
<cpu mode='custom' match='exact' check='none'>
<model fallback='forbid'>qemu64</model>
</cpu>
either way,
(2.2) for aarch64 guests, we get
<cpu mode='custom' match='exact' check='none'>
<model fallback='forbid'>cortex-a15</model>
</cpu>
either way.
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2107503
Signed-off-by: Laszlo Ersek <lersek(a)redhat.com>
---
output/create_libvirt_xml.ml | 1 +
1 file changed, 1 insertion(+)
diff --git a/output/create_libvirt_xml.ml b/output/create_libvirt_xml.ml
index 531a4f75bf3e..0343d3194268 100644
--- a/output/create_libvirt_xml.ml
+++ b/output/create_libvirt_xml.ml
@@ -192,6 +192,7 @@ let create_libvirt_xml ?pool source inspect
List.push_back cpu_attrs ("mode", "host-passthrough");
| Some model ->
List.push_back cpu_attrs ("match", "minimum");
+ List.push_back cpu_attrs ("check", "none");
(match source.s_cpu_vendor with
| None -> ()
| Some vendor ->
--
2.19.1.3.g30247aa5d201
7 a.m.
On Wed, Jul 20, 2022 at 01:09:13PM +0200, Laszlo Ersek wrote:
We currently don't generate any @check attribute for the
/domain/cpu
element, which causes the following libvirtd behavior
<https://libvirt.org/formatdomain.html#cpu-model-and-topology>;:
> Once the domain starts, libvirt will automatically change the check
> attribute to the best supported value to ensure the virtual CPU does not
> change when the domain is migrated to another host
Vera Wu reports that in practice, at least when the source CPU model is
explicitly "qemu64", libvirtd sets @check='partial'. That's defined
as:
I think my main confusion is why is the _source_ model "qemu64"?
Doesn't that imply the source hypervisor is qemu and so we shouldn't
use virt-v2v at all? In fact from the BZ:
# virt-v2v -i libvirt -ic qemu:///system esx6.7-rhel8.6-nvme-disk ...
(Vera is doing this for a good reason - she wants to test the -oo
compressed option, and using a local qemu guest is the quickest way.)
Anyway let's go with the assumption this is a bad idea, but for
testing purposes we often do this so we should try to make it work.
> Libvirt will check the guest CPU specification before starting a
domain
This is a problem: the default "qemu64" CPU model exposes the SVM CPU
flag, and that's unsupportable on Intel hosts. SVM is the AMD counterpart
of VT-x; IOW, the flag effectively advertizes AMD-specific nesting to
guests.
With @check='partial', libvirt prevents the converted domain from starting
on Intel hosts; but with @check='none',
> Libvirt does no checking and it is up to the hypervisor to refuse to
> start the domain if it cannot provide the requested CPU. With QEMU this
> means no checking is done at all since the default behavior of QEMU is
> to emit warnings, but start the domain anyway.
We don't care about the migratability of the converted domain, so relax
the libvirtd checks, by generating the @check='none' attribute.
For testing we don't, but in general -o libvirt conversions we might
care about migratability.
Consider adding @check='none' in two cases:
(1) When the source domain specifies a CPU model.
Generating @check='none' in this case fixes the issue reported by Vera.
(2) When the source domain does not specify a CPU model, and the guest OS
is assumed to work well with the default CPU model.
Generating @check='none' in this case is actually a no-op. Going from "no
CPU element" to "<cpu check='none'/>" does not change how
libvirtd
augments the domain config. Namely,
(2.1) for x86 guests, we get
<cpu mode='custom' match='exact' check='none'>
<model fallback='forbid'>qemu64</model>
</cpu>
either way,
(2.2) for aarch64 guests, we get
<cpu mode='custom' match='exact' check='none'>
<model fallback='forbid'>cortex-a15</model>
</cpu>
either way.
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2107503
Signed-off-by: Laszlo Ersek <lersek(a)redhat.com>
---
output/create_libvirt_xml.ml | 1 +
1 file changed, 1 insertion(+)
diff --git a/output/create_libvirt_xml.ml b/output/create_libvirt_xml.ml
index 531a4f75bf3e..0343d3194268 100644
--- a/output/create_libvirt_xml.ml
+++ b/output/create_libvirt_xml.ml
@@ -192,6 +192,7 @@ let create_libvirt_xml ?pool source inspect
List.push_back cpu_attrs ("mode", "host-passthrough");
| Some model ->
List.push_back cpu_attrs ("match", "minimum");
+ List.push_back cpu_attrs ("check", "none");
So should we make this attribute conditional on the source CPU model
being qemu64, just so the change is minimal and we don't break
migratability for non testcases?
Rich.
(match source.s_cpu_vendor with
| None -> ()
| Some vendor ->
--
2.19.1.3.g30247aa5d201
_______________________________________________
Libguestfs mailing list
Libguestfs(a)redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines. Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
7:55 a.m.
On 07/20/22 14:00, Richard W.M. Jones wrote:
On Wed, Jul 20, 2022 at 01:09:13PM +0200, Laszlo Ersek wrote:
> We currently don't generate any @check attribute for the /domain/cpu
> element, which causes the following libvirtd behavior
> <https://libvirt.org/formatdomain.html#cpu-model-and-topology>;:
>
>> Once the domain starts, libvirt will automatically change the check
>> attribute to the best supported value to ensure the virtual CPU does
>> not change when the domain is migrated to another host
>
> Vera Wu reports that in practice, at least when the source CPU model
> is explicitly "qemu64", libvirtd sets @check='partial'. That's
> defined as:
I think my main confusion is why is the _source_ model "qemu64"?
Doesn't that imply the source hypervisor is qemu and so we shouldn't
use virt-v2v at all? In fact from the BZ:
# virt-v2v -i libvirt -ic qemu:///system esx6.7-rhel8.6-nvme-disk ...
(Vera is doing this for a good reason - she wants to test the -oo
compressed option, and using a local qemu guest is the quickest way.)
Anyway let's go with the assumption this is a bad idea, but for
testing purposes we often do this so we should try to make it work.
Indeed this is for that "-i libvirt -ic qemu://..." use case.
I commented similarly to you, in
<https://bugzilla.redhat.com/show_bug.cgi?id=2047660#c11>;:
Anyway I defer to Rich on whether we need a new bug for this. I
don't
think that we absolutely need to investigate this, as I'd never expect
libvirt-to-libvirt conversion to be used in production.
And then Vera filed 2107503.
Continuing:
On 07/20/22 14:00, Richard W.M. Jones wrote:
On Wed, Jul 20, 2022 at 01:09:13PM +0200, Laszlo Ersek wrote:
>> Libvirt will check the guest CPU specification before starting a
>> domain
>
> This is a problem: the default "qemu64" CPU model exposes the SVM CPU
> flag, and that's unsupportable on Intel hosts. SVM is the AMD
> counterpart of VT-x; IOW, the flag effectively advertizes
> AMD-specific nesting to guests.
>
> With @check='partial', libvirt prevents the converted domain from
> starting on Intel hosts; but with @check='none',
>
>> Libvirt does no checking and it is up to the hypervisor to refuse to
>> start the domain if it cannot provide the requested CPU. With QEMU
>> this means no checking is done at all since the default behavior of
>> QEMU is to emit warnings, but start the domain anyway.
>
> We don't care about the migratability of the converted domain, so
> relax the libvirtd checks, by generating the @check='none' attribute.
For testing we don't, but in general -o libvirt conversions we might
care about migratability.
Without @check='none', we may not even be able to start the converted
domain in-place (that is, on the libvirt host that the domain was first
converted to -- even before we attempt migration). See below.
> Consider adding @check='none' in two cases:
>
> (1) When the source domain specifies a CPU model.
>
> Generating @check='none' in this case fixes the issue reported by
> Vera.
>
> (2) When the source domain does not specify a CPU model, and the
> guest OS is assumed to work well with the default CPU model.
>
> Generating @check='none' in this case is actually a no-op. Going from
> "no CPU element" to "<cpu check='none'/>" does not
change how
> libvirtd augments the domain config. Namely,
>
> (2.1) for x86 guests, we get
>
> <cpu mode='custom' match='exact' check='none'>
> <model fallback='forbid'>qemu64</model>
> </cpu>
>
> either way,
>
> (2.2) for aarch64 guests, we get
>
> <cpu mode='custom' match='exact' check='none'>
> <model fallback='forbid'>cortex-a15</model>
> </cpu>
>
> either way.
>
> Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2107503
> Signed-off-by: Laszlo Ersek <lersek(a)redhat.com>
> ---
> output/create_libvirt_xml.ml | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/output/create_libvirt_xml.ml b/output/create_libvirt_xml.ml
> index 531a4f75bf3e..0343d3194268 100644
> --- a/output/create_libvirt_xml.ml
> +++ b/output/create_libvirt_xml.ml
> @@ -192,6 +192,7 @@ let create_libvirt_xml ?pool source inspect
> List.push_back cpu_attrs ("mode",
"host-passthrough");
> | Some model ->
> List.push_back cpu_attrs ("match", "minimum");
> + List.push_back cpu_attrs ("check", "none");
So should we make this attribute conditional on the source CPU model
being qemu64, just so the change is minimal and we don't break
migratability for non testcases?
In my opinion, having SVM in qemu64 is actually a bug in the qemu64 CPU
model. Qemu64 is supposed to be the most compatible ("runs everywhere")
CPU model, per
<https://gitlab.com/qemu-project/qemu/-/blob/master/docs/system/cpu-models...;.
So why does it include a CPU feature that's impossible to satisfy with
KVM on an Intel host? Libvirt is totally right to complain about that.
With this patch, *at first sight*, we're working around the qemu64 CPU
model breakage, by asking libvirt to ignore its own (otherwise
justified) checks.
However... assume the source virt host is a genuine AMD one, and the
source VCPU model is (say) EPYC
<https://gitlab.com/qemu-project/qemu/-/blob/master/docs/system/cpu-models...;.
Then the "svm" CPU feature (and some other AMD-only CPU flags) will show
up just the same, and the libvirtd-side checks will prevent the
converted domain from launching on an Intel host (with KVM) just the
same.
From all perspectives *except* virt-v2v's, this is arguably the correct
behavior. But from virt-v2v's perspective, it is *still* wrong. We
arguably don't care if we need to tweak the CPU model "incompatibly", we
just want the converted guest to boot *somehow*.
Now, one could argue that "-ic libvirtURI" and "-o libvirt" only
support
local libvirt connections, so it's impossible to have an AMD "source
host" and an Intel "target host", when those hosts are actually the
same. In other words, if the host were Intel, then EPYC would not be
bootable in the first place (and indeed "qemu64" is only bootable in the
first place because the original domain does include @check='none').
However, we also have the "-i libvirtxml" input mode, where the original
domain XML may come from anywhere.
In short, I don't know what to do about this. Modeling the *entire*
<cpu> element in guestcaps (for the sake of transferring @check='none'
along with the qemu64 model) would be totally overkill; this situation
is never encountered in production where we convert mostly from vmware,
i.e., where neither "qemu64" or "@check='none'" are relevant.
I can spin a version 2 of this patch, restricting the hack to the qemu64
VCPU model, but then the commit message can only say, "this is yet
another band-aid, namely for the *very specific* test environment -i
libvirt -ic qemu://... -o libvirt".
We could be best off with just producing "host-passthrough"
indiscriminately in the libvirt output module (and explicitly writing
off migratability).
... Compare how much simpler the QEMU output is:
(match source.s_cpu_model, guestcaps.gcaps_default_cpu with
| None, true -> ()
| None, false -> arg "-cpu" "host"
| Some model, _ -> arg "-cpu" model
);
The first two patterns match what we currently do in the libvirt output
module, but the last pattern doesn't. If QEMU cannot satisfy the
particular -cpu model, it will emit some warnings (see the docs quote
above), but will start the domain anyway. That's where libvirt is
stricter, with the default @check='partial'. Adding
"@check='none'"
would relax the libvirt output to the level of the QEMU output, if you
will.
Consider where "s_cpu_model" is used among all the output modules. It is
consumed by the libvirt, qemu, and RHV (lib/create_ovf.ml) ones.
QEMU does a very simple thing (see above).
The OVF stuff is even simpler (this is what the QEMU logic was based
upon):
(match source.s_cpu_model with
| None -> ()
| Some model ->
List.push_back content_subnodes (e "CustomCpuName" [] [PCData
model])
);
It's only the libvirt output that's super hairy. I think @check='none'
relaxes it to the level of the other two.
Laszlo
Rich.
> (match source.s_cpu_vendor with
> | None -> ()
> | Some vendor ->
> --
> 2.19.1.3.g30247aa5d201
>
> _______________________________________________
> Libguestfs mailing list
> Libguestfs(a)redhat.com
> https://listman.redhat.com/mailman/listinfo/libguestfs
4:34 a.m.
Sorry for the delayed response to this. I see you've posted an
updated patch, so this is just a bit of FYI.
I originally added CPU modelling in commit 11505e4b84 (March 2017):
https://github.com/libguestfs/virt-v2v/commit/11505e4b84ce8d7eda4e2a275fd...
What we were actually trying to achieve here was to preserve the CPU
topology. I believe the request came from Bill Helgerson who was
working on v2v in the proto-IMS product, and was working a lot with
customers.
You can see in the code before the patch is applied we only modelled
the number of vCPUs. Afterwards we have:
* number of vCPUs
* vendor (eg. AMD)
* model (eg. EPYC)
* sockets
* cores per socket
* threads per core
I think here only the first 1 and last 3 (#vCPUS, topology) are really
important. I believe I added the vendor and model just because they
were there, without necessarily thinking too deeply about the
implications.
As you covered in your email, what is the real meaning of converting a
source guest using eg AMD/EPYC with virt-v2v to some target? Does it
mean that the target must be able to emulate all EPYC feature (likely
impossible if the target is Intel)? I would say it's not that
important. This isn't live migration, and almost all guests can be
booted interchangably on different x86_64 hardware.
Is topology important? I would say yes, or at least it's much more
important than vendor/model. Workloads may expect not just a number
of vCPUs, but a particular layout, especially the larger and more
complex ones.
So ... my question now is, should we simply remove the vendor and
model fields completely?
I'll ACK the v2 patch anyway (in a minute).
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
4:42 a.m.
On Fri, Jul 22, 2022 at 10:34:44AM +0100, Richard W.M. Jones wrote:
Sorry for the delayed response to this. I see you've posted an
updated patch, so this is just a bit of FYI.
I originally added CPU modelling in commit 11505e4b84 (March 2017):
https://github.com/libguestfs/virt-v2v/commit/11505e4b84ce8d7eda4e2a275fd...
What we were actually trying to achieve here was to preserve the CPU
topology. I believe the request came from Bill Helgerson who was
working on v2v in the proto-IMS product, and was working a lot with
customers.
You can see in the code before the patch is applied we only modelled
the number of vCPUs. Afterwards we have:
* number of vCPUs
* vendor (eg. AMD)
* model (eg. EPYC)
* sockets
* cores per socket
* threads per core
I think here only the first 1 and last 3 (#vCPUS, topology) are really
important. I believe I added the vendor and model just because they
were there, without necessarily thinking too deeply about the
implications.
As you covered in your email, what is the real meaning of converting a
source guest using eg AMD/EPYC with virt-v2v to some target? Does it
mean that the target must be able to emulate all EPYC feature (likely
impossible if the target is Intel)? I would say it's not that
important. This isn't live migration, and almost all guests can be
booted interchangably on different x86_64 hardware.
Is topology important? I would say yes, or at least it's much more
important than vendor/model. Workloads may expect not just a number
of vCPUs, but a particular layout, especially the larger and more
complex ones.
In terms of topology, if you have NOT set pCPU:vCPU 1:1 pinning,
then NEVER set threads > 1. There's a choice of sockets vs cores
for non-pinned scenario, and generally I'd recommends 'cores'
always because high core counts are common in real world, and
'sockets' mostly maxes out at 2/4 in real world (ignoring wierd
high end hardware), also some OS restrict you based on sockets,
but not cores. So IMHO the only compelling reason to use
sockets > 1 is you want to have virtual NUMA topology, but
even that's dubious unless pinning.
If you have set pCPU:vCPU 1:1 pinning, then set topology to
try to match what you've pinned to.
So ... my question now is, should we simply remove the vendor and
model fields completely?
Removing 'model' is not a good idea, as you'll get the default
CPU model.
If you don't have to pick a particular CPU, then IMHO either
use host-model or host-passthrough depending on whether you
think live migration is important or not.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
4:50 a.m.
On Fri, Jul 22, 2022 at 10:42:48AM +0100, Daniel P. Berrangé wrote:
On Fri, Jul 22, 2022 at 10:34:44AM +0100, Richard W.M. Jones wrote:
> Sorry for the delayed response to this. I see you've posted an
> updated patch, so this is just a bit of FYI.
>
> I originally added CPU modelling in commit 11505e4b84 (March 2017):
>
>
https://github.com/libguestfs/virt-v2v/commit/11505e4b84ce8d7eda4e2a275fd...
>
> What we were actually trying to achieve here was to preserve the CPU
> topology. I believe the request came from Bill Helgerson who was
> working on v2v in the proto-IMS product, and was working a lot with
> customers.
>
> You can see in the code before the patch is applied we only modelled
> the number of vCPUs. Afterwards we have:
>
> * number of vCPUs
> * vendor (eg. AMD)
> * model (eg. EPYC)
> * sockets
> * cores per socket
> * threads per core
>
> I think here only the first 1 and last 3 (#vCPUS, topology) are really
> important. I believe I added the vendor and model just because they
> were there, without necessarily thinking too deeply about the
> implications.
>
> As you covered in your email, what is the real meaning of converting a
> source guest using eg AMD/EPYC with virt-v2v to some target? Does it
> mean that the target must be able to emulate all EPYC feature (likely
> impossible if the target is Intel)? I would say it's not that
> important. This isn't live migration, and almost all guests can be
> booted interchangably on different x86_64 hardware.
>
> Is topology important? I would say yes, or at least it's much more
> important than vendor/model. Workloads may expect not just a number
> of vCPUs, but a particular layout, especially the larger and more
> complex ones.
In terms of topology, if you have NOT set pCPU:vCPU 1:1 pinning,
then NEVER set threads > 1. There's a choice of sockets vs cores
for non-pinned scenario, and generally I'd recommends 'cores'
always because high core counts are common in real world, and
'sockets' mostly maxes out at 2/4 in real world (ignoring wierd
high end hardware), also some OS restrict you based on sockets,
but not cores. So IMHO the only compelling reason to use
sockets > 1 is you want to have virtual NUMA topology, but
even that's dubious unless pinning.
If you have set pCPU:vCPU 1:1 pinning, then set topology to
try to match what you've pinned to.
> So ... my question now is, should we simply remove the vendor and
> model fields completely?
Removing 'model' is not a good idea, as you'll get the default
CPU model.
If you don't have to pick a particular CPU, then IMHO either
use host-model or host-passthrough depending on whether you
think live migration is important or not.
I mean remove them from virt-v2v's internal source model [confusing
terminology here - modelling the source != CPU model]. On targets
we'd choose something like cpu=host-model to get the best possible
migratable CPU.
The point is we're not copying the Intel / Nehalem, AMD / EPYC etc of
the guest from the source to the destination hypervisor.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
nbdkit - Flexible, fast NBD server with plugins
https://gitlab.com/nbdkit/nbdkit
6:49 a.m.
On 07/22/22 11:50, Richard W.M. Jones wrote:
On Fri, Jul 22, 2022 at 10:42:48AM +0100, Daniel P. Berrangé wrote:
> On Fri, Jul 22, 2022 at 10:34:44AM +0100, Richard W.M. Jones wrote:
>> Sorry for the delayed response to this. I see you've posted an
>> updated patch, so this is just a bit of FYI.
>>
>> I originally added CPU modelling in commit 11505e4b84 (March 2017):
>>
>>
https://github.com/libguestfs/virt-v2v/commit/11505e4b84ce8d7eda4e2a275fd...
>>
>> What we were actually trying to achieve here was to preserve the CPU
>> topology. I believe the request came from Bill Helgerson who was
>> working on v2v in the proto-IMS product, and was working a lot with
>> customers.
>>
>> You can see in the code before the patch is applied we only modelled
>> the number of vCPUs. Afterwards we have:
>>
>> * number of vCPUs
>> * vendor (eg. AMD)
>> * model (eg. EPYC)
>> * sockets
>> * cores per socket
>> * threads per core
>>
>> I think here only the first 1 and last 3 (#vCPUS, topology) are really
>> important. I believe I added the vendor and model just because they
>> were there, without necessarily thinking too deeply about the
>> implications.
>>
>> As you covered in your email, what is the real meaning of converting a
>> source guest using eg AMD/EPYC with virt-v2v to some target? Does it
>> mean that the target must be able to emulate all EPYC feature (likely
>> impossible if the target is Intel)? I would say it's not that
>> important. This isn't live migration, and almost all guests can be
>> booted interchangably on different x86_64 hardware.
>>
>> Is topology important? I would say yes, or at least it's much more
>> important than vendor/model. Workloads may expect not just a number
>> of vCPUs, but a particular layout, especially the larger and more
>> complex ones.
>
> In terms of topology, if you have NOT set pCPU:vCPU 1:1 pinning,
> then NEVER set threads > 1. There's a choice of sockets vs cores
> for non-pinned scenario, and generally I'd recommends 'cores'
> always because high core counts are common in real world, and
> 'sockets' mostly maxes out at 2/4 in real world (ignoring wierd
> high end hardware), also some OS restrict you based on sockets,
> but not cores. So IMHO the only compelling reason to use
> sockets > 1 is you want to have virtual NUMA topology, but
> even that's dubious unless pinning.
>
> If you have set pCPU:vCPU 1:1 pinning, then set topology to
> try to match what you've pinned to.
>
>> So ... my question now is, should we simply remove the vendor and
>> model fields completely?
>
> Removing 'model' is not a good idea, as you'll get the default
> CPU model.
>
> If you don't have to pick a particular CPU, then IMHO either
> use host-model or host-passthrough depending on whether you
> think live migration is important or not.
I mean remove them from virt-v2v's internal source model [confusing
terminology here - modelling the source != CPU model]. On targets
we'd choose something like cpu=host-model to get the best possible
migratable CPU.
The point is we're not copying the Intel / Nehalem, AMD / EPYC etc of
the guest from the source to the destination hypervisor.
I think producing host-passthrough indiscriminately on output (which we
already do in the particular case only when the source does not specify
a model and we know an OS does not run on qemu64) would be best. I don't
think it would be a very difficult patch or patch set, but I dread the
testing of it. :/
Let me go ahead and commit v2; and let's remember this discussion for
the next time a CPU model related problem pops up. If switching to
host-passthrough solves that problem then, we should implement it then.
(And then ask QE to test it as comprehensively as they can...)
Thanks!
Laszlo
6:53 a.m.
On Fri, Jul 22, 2022 at 01:49:21PM +0200, Laszlo Ersek wrote:
On 07/22/22 11:50, Richard W.M. Jones wrote:
> On Fri, Jul 22, 2022 at 10:42:48AM +0100, Daniel P. Berrangé wrote:
>> On Fri, Jul 22, 2022 at 10:34:44AM +0100, Richard W.M. Jones wrote:
>>> Sorry for the delayed response to this. I see you've posted an
>>> updated patch, so this is just a bit of FYI.
>>>
>>> I originally added CPU modelling in commit 11505e4b84 (March 2017):
>>>
>>>
https://github.com/libguestfs/virt-v2v/commit/11505e4b84ce8d7eda4e2a275fd...
>>>
>>> What we were actually trying to achieve here was to preserve the CPU
>>> topology. I believe the request came from Bill Helgerson who was
>>> working on v2v in the proto-IMS product, and was working a lot with
>>> customers.
>>>
>>> You can see in the code before the patch is applied we only modelled
>>> the number of vCPUs. Afterwards we have:
>>>
>>> * number of vCPUs
>>> * vendor (eg. AMD)
>>> * model (eg. EPYC)
>>> * sockets
>>> * cores per socket
>>> * threads per core
>>>
>>> I think here only the first 1 and last 3 (#vCPUS, topology) are really
>>> important. I believe I added the vendor and model just because they
>>> were there, without necessarily thinking too deeply about the
>>> implications.
>>>
>>> As you covered in your email, what is the real meaning of converting a
>>> source guest using eg AMD/EPYC with virt-v2v to some target? Does it
>>> mean that the target must be able to emulate all EPYC feature (likely
>>> impossible if the target is Intel)? I would say it's not that
>>> important. This isn't live migration, and almost all guests can be
>>> booted interchangably on different x86_64 hardware.
>>>
>>> Is topology important? I would say yes, or at least it's much more
>>> important than vendor/model. Workloads may expect not just a number
>>> of vCPUs, but a particular layout, especially the larger and more
>>> complex ones.
>>
>> In terms of topology, if you have NOT set pCPU:vCPU 1:1 pinning,
>> then NEVER set threads > 1. There's a choice of sockets vs cores
>> for non-pinned scenario, and generally I'd recommends 'cores'
>> always because high core counts are common in real world, and
>> 'sockets' mostly maxes out at 2/4 in real world (ignoring wierd
>> high end hardware), also some OS restrict you based on sockets,
>> but not cores. So IMHO the only compelling reason to use
>> sockets > 1 is you want to have virtual NUMA topology, but
>> even that's dubious unless pinning.
>>
>> If you have set pCPU:vCPU 1:1 pinning, then set topology to
>> try to match what you've pinned to.
>>
>>> So ... my question now is, should we simply remove the vendor and
>>> model fields completely?
>>
>> Removing 'model' is not a good idea, as you'll get the default
>> CPU model.
>>
>> If you don't have to pick a particular CPU, then IMHO either
>> use host-model or host-passthrough depending on whether you
>> think live migration is important or not.
>
> I mean remove them from virt-v2v's internal source model [confusing
> terminology here - modelling the source != CPU model]. On targets
> we'd choose something like cpu=host-model to get the best possible
> migratable CPU.
>
> The point is we're not copying the Intel / Nehalem, AMD / EPYC etc of
> the guest from the source to the destination hypervisor.
I think producing host-passthrough indiscriminately on output (which we
already do in the particular case only when the source does not specify
a model and we know an OS does not run on qemu64) would be best. I don't
think it would be a very difficult patch or patch set, but I dread the
testing of it. :/
Let me go ahead and commit v2; and let's remember this discussion for
the next time a CPU model related problem pops up. If switching to
host-passthrough solves that problem then, we should implement it then.
(And then ask QE to test it as comprehensively as they can...)
Remember, 'host-passthrough' is only possible with KVM, not TCG,
'host-model' works with both. If you have newish libvirt + QEMU
you can use 'maximum' which is equiv to 'host-pasthrough' on
KVM, or "all implemented features" on TCG.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
6:57 a.m.
On Fri, Jul 22, 2022 at 12:53:00PM +0100, Daniel P. Berrangé wrote:
On Fri, Jul 22, 2022 at 01:49:21PM +0200, Laszlo Ersek wrote:
> On 07/22/22 11:50, Richard W.M. Jones wrote:
> > On Fri, Jul 22, 2022 at 10:42:48AM +0100, Daniel P. Berrangé wrote:
> >> On Fri, Jul 22, 2022 at 10:34:44AM +0100, Richard W.M. Jones wrote:
> >>> Sorry for the delayed response to this. I see you've posted an
> >>> updated patch, so this is just a bit of FYI.
> >>>
> >>> I originally added CPU modelling in commit 11505e4b84 (March 2017):
> >>>
> >>>
https://github.com/libguestfs/virt-v2v/commit/11505e4b84ce8d7eda4e2a275fd...
> >>>
> >>> What we were actually trying to achieve here was to preserve the CPU
> >>> topology. I believe the request came from Bill Helgerson who was
> >>> working on v2v in the proto-IMS product, and was working a lot with
> >>> customers.
> >>>
> >>> You can see in the code before the patch is applied we only modelled
> >>> the number of vCPUs. Afterwards we have:
> >>>
> >>> * number of vCPUs
> >>> * vendor (eg. AMD)
> >>> * model (eg. EPYC)
> >>> * sockets
> >>> * cores per socket
> >>> * threads per core
> >>>
> >>> I think here only the first 1 and last 3 (#vCPUS, topology) are really
> >>> important. I believe I added the vendor and model just because they
> >>> were there, without necessarily thinking too deeply about the
> >>> implications.
> >>>
> >>> As you covered in your email, what is the real meaning of converting a
> >>> source guest using eg AMD/EPYC with virt-v2v to some target? Does it
> >>> mean that the target must be able to emulate all EPYC feature (likely
> >>> impossible if the target is Intel)? I would say it's not that
> >>> important. This isn't live migration, and almost all guests can
be
> >>> booted interchangably on different x86_64 hardware.
> >>>
> >>> Is topology important? I would say yes, or at least it's much
more
> >>> important than vendor/model. Workloads may expect not just a number
> >>> of vCPUs, but a particular layout, especially the larger and more
> >>> complex ones.
> >>
> >> In terms of topology, if you have NOT set pCPU:vCPU 1:1 pinning,
> >> then NEVER set threads > 1. There's a choice of sockets vs cores
> >> for non-pinned scenario, and generally I'd recommends 'cores'
> >> always because high core counts are common in real world, and
> >> 'sockets' mostly maxes out at 2/4 in real world (ignoring wierd
> >> high end hardware), also some OS restrict you based on sockets,
> >> but not cores. So IMHO the only compelling reason to use
> >> sockets > 1 is you want to have virtual NUMA topology, but
> >> even that's dubious unless pinning.
> >>
> >> If you have set pCPU:vCPU 1:1 pinning, then set topology to
> >> try to match what you've pinned to.
> >>
> >>> So ... my question now is, should we simply remove the vendor and
> >>> model fields completely?
> >>
> >> Removing 'model' is not a good idea, as you'll get the default
> >> CPU model.
> >>
> >> If you don't have to pick a particular CPU, then IMHO either
> >> use host-model or host-passthrough depending on whether you
> >> think live migration is important or not.
> >
> > I mean remove them from virt-v2v's internal source model [confusing
> > terminology here - modelling the source != CPU model]. On targets
> > we'd choose something like cpu=host-model to get the best possible
> > migratable CPU.
> >
> > The point is we're not copying the Intel / Nehalem, AMD / EPYC etc of
> > the guest from the source to the destination hypervisor.
>
> I think producing host-passthrough indiscriminately on output (which we
> already do in the particular case only when the source does not specify
> a model and we know an OS does not run on qemu64) would be best. I don't
> think it would be a very difficult patch or patch set, but I dread the
> testing of it. :/
>
> Let me go ahead and commit v2; and let's remember this discussion for
> the next time a CPU model related problem pops up. If switching to
> host-passthrough solves that problem then, we should implement it then.
> (And then ask QE to test it as comprehensively as they can...)
Remember, 'host-passthrough' is only possible with KVM, not TCG,
'host-model' works with both. If you have newish libvirt + QEMU
you can use 'maximum' which is equiv to 'host-pasthrough' on
KVM, or "all implemented features" on TCG.
host-model is the only one which allows migration?
This choice of CPU model only really matters for local conversions (-o
libvirt, -o local, -o qemu). For conversions to target hypervisors we
can probably let them choose.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
nbdkit - Flexible, fast NBD server with plugins
https://gitlab.com/nbdkit/nbdkit
854
days inactive
856
days old
8 comments
3 participants
participants (3)
-
Daniel P. Berrangé -
Laszlo Ersek -
Richard W.M. Jones