1:10 a.m.
I am using a debian 9 guest, hosted on a ESXi platform with nested
virtualisation enabled.
On this debian 9 guest when I run libguesfs-test-tool, it fails with an
error:
"qemu-system-x86_64:
/build/qemu-DqynNa/qemu-2.8+dfsg/target-i386/kvm.c:1805: kvm_put_msrs:
Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed."
Instead when I use a wrapper script and hook it with the env
variable LIBGUESTFS_HV=/tmp/qemu.wrapper, things work fine, as in the
script I just spawn the guest with the specific machine type pc-i440fx-2.1
over KVM.
exec qemu-system-x86_64 -enable-kvm -machine *pc-i440fx-2.**1*,accel
=kvm,dump-guest-core=off "${args[@]}"
The script is attached as well as the libguestfs-test-tool logs.
Can anybody help me on this. Why is the latest machine type, i.e.
pc-i440fx-2.8 on my debian 9 guest not working out? If I do a force_tcg it
works even with the latest machine type, but that is not what I desire, I
don't intend libguestfs to run on emulation.
=========================================================================================================================
System configuration:
*$> uname -a*
Linux XXXX 4.9.0-6-amd64 #1 SMP Debian 4.9.82-1+deb9u3 (2018-03-02) x86_64
GNU/Linux
*$> kvm -machine help*
Supported machines are:
pc Standard PC (i440FX + PIIX, 1996) (alias of
pc-i440fx-2.8)
pc-i440fx-2.8 Standard PC (i440FX + PIIX, 1996) (default)
pc-i440fx-2.7 Standard PC (i440FX + PIIX, 1996)
pc-i440fx-2.6 Standard PC (i440FX + PIIX, 1996)
pc-i440fx-2.5 Standard PC (i440FX + PIIX, 1996)
pc-i440fx-2.4 Standard PC (i440FX + PIIX, 1996)
pc-i440fx-2.3 Standard PC (i440FX + PIIX, 1996)
pc-i440fx-2.2 Standard PC (i440FX + PIIX, 1996)
pc-i440fx-2.1 Standard PC (i440FX + PIIX, 1996)
pc-i440fx-2.0 Standard PC (i440FX + PIIX, 1996)
pc-i440fx-1.7 Standard PC (i440FX + PIIX, 1996)
pc-i440fx-1.6 Standard PC (i440FX + PIIX, 1996)
pc-i440fx-1.5 Standard PC (i440FX + PIIX, 1996)
pc-i440fx-1.4 Standard PC (i440FX + PIIX, 1996)
pc-1.3 Standard PC (i440FX + PIIX, 1996)
pc-1.2 Standard PC (i440FX + PIIX, 1996)
pc-1.1 Standard PC (i440FX + PIIX, 1996)
pc-1.0 Standard PC (i440FX + PIIX, 1996)
pc-0.15 Standard PC (i440FX + PIIX, 1996)
pc-0.14 Standard PC (i440FX + PIIX, 1996)
pc-0.13 Standard PC (i440FX + PIIX, 1996)
pc-0.12 Standard PC (i440FX + PIIX, 1996)
pc-0.11 Standard PC (i440FX + PIIX, 1996)
pc-0.10 Standard PC (i440FX + PIIX, 1996)
q35 Standard PC (Q35 + ICH9, 2009) (alias of pc-q35-2.8)
pc-q35-2.8 Standard PC (Q35 + ICH9, 2009)
pc-q35-2.7 Standard PC (Q35 + ICH9, 2009)
pc-q35-2.6 Standard PC (Q35 + ICH9, 2009)
pc-q35-2.5 Standard PC (Q35 + ICH9, 2009)
pc-q35-2.4 Standard PC (Q35 + ICH9, 2009)
isapc ISA-only PC
none empty machine
xenfv Xen Fully-virtualized PC
xenpv Xen Para-virtualized PC
*$> kvm -cpu help*
Available CPUs:
x86 486
x86 Broadwell-noTSX Intel Core Processor (Broadwell, no TSX)
x86 Broadwell Intel Core Processor (Broadwell)
x86 Conroe Intel Celeron_4x0 (Conroe/Merom Class Core 2)
x86 Haswell-noTSX Intel Core Processor (Haswell, no TSX)
x86 Haswell Intel Core Processor (Haswell)
x86 IvyBridge Intel Xeon E3-12xx v2 (Ivy Bridge)
x86 Nehalem Intel Core i7 9xx (Nehalem Class Core i7)
x86 Opteron_G1 AMD Opteron 240 (Gen 1 Class Opteron)
x86 Opteron_G2 AMD Opteron 22xx (Gen 2 Class Opteron)
x86 Opteron_G3 AMD Opteron 23xx (Gen 3 Class Opteron)
x86 Opteron_G4 AMD Opteron 62xx class CPU
x86 Opteron_G5 AMD Opteron 63xx class CPU
x86 Penryn Intel Core 2 Duo P9xxx (Penryn Class Core 2)
x86 SandyBridge Intel Xeon E312xx (Sandy Bridge)
x86 Skylake-Client Intel Core Processor (Skylake)
x86 Westmere Westmere E56xx/L56xx/X56xx (Nehalem-C)
x86 athlon QEMU Virtual CPU version 2.5+
x86 core2duo Intel(R) Core(TM)2 Duo CPU T7700 @ 2.40GHz
x86 coreduo Genuine Intel(R) CPU T2600 @ 2.16GHz
x86 kvm32 Common 32-bit KVM processor
x86 kvm64 Common KVM processor
x86 n270 Intel(R) Atom(TM) CPU N270 @ 1.60GHz
x86 pentium
x86 pentium2
x86 pentium3
x86 phenom AMD Phenom(tm) 9550 Quad-Core Processor
x86 qemu32 QEMU Virtual CPU version 2.5+
x86 qemu64 QEMU Virtual CPU version 2.5+
x86 host KVM processor with all supported host features (only
available in KVM mode)
Recognized CPUID flags:
fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36
pn clflush ds acpi mmx fxsr sse sse2 ss ht tm ia64 pbe
pni pclmulqdq dtes64 monitor ds-cpl vmx smx est tm2 ssse3 cid fma cx16
xtpr pdcm pcid dca sse4.1 sse4.2 x2apic movbe popcnt tsc-deadline aes xsave
osxsave avx f16c rdrand hypervisor
fsgsbase tsc-adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f
avx512dq rdseed adx smap avx512ifma pcommit clflushopt clwb avx512pf
avx512er avx512cd avx512bw avx512vl
avx512vbmi umip pku ospke rdpid
avx512-4vnniw avx512-4fmaps
syscall nx mmxext fxsr-opt pdpe1gb rdtscp lm 3dnowext 3dnow
lahf-lm cmp-legacy svm extapic cr8legacy abm sse4a misalignsse
3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid-msr tbm topoext
perfctr-core perfctr-nb
invtsc
xstore xstore-en xcrypt xcrypt-en ace2 ace2-en phe phe-en pmm pmm-en
kvmclock kvm-nopiodelay kvm-mmu kvmclock kvm-asyncpf kvm-steal-time
kvm-pv-eoi kvm-pv-unhalt kvmclock-stable-bit
npt lbrv svm-lock nrip-save tsc-scale vmcb-clean flushbyasid
decodeassists pause-filter pfthreshold
xsaveopt xsavec xgetbv1 xsaves
arat
4:07 a.m.
On Friday, 23 March 2018 07:10:00 CET Tanmoy Sinha wrote:
I am using a debian 9 guest, hosted on a ESXi platform with nested
virtualisation enabled.
On this debian 9 guest when I run libguesfs-test-tool, it fails with an
error:
"qemu-system-x86_64:
/build/qemu-DqynNa/qemu-2.8+dfsg/target-i386/kvm.c:1805: kvm_put_msrs:
Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed."
See also https://bugzilla.redhat.com/show_bug.cgi?id=1456407
This is basically a bug in VMware and the qemu bug
https://bugs.launchpad.net/qemu/+bug/1661386
has a couple of hints about what it can be changed in VMware to avoid
this situation.
--
Pino Toscano
12:35 p.m.
Thanks. Reading the defect and the associated thread in
https://bugs.launchpad.net/qemu/+bug/1661386, I enabled performance
counters in VMWare guest settings. Now the kvm assertion `ret ==
cpu->kvm_msr_buf->nmsrs' is gone. But the qemu/kvm hangs after printing
"SeaBIOS (version 1.10.2-1)"
I presume this too is a qemu/kvm issue for vmware platforms. Is there any
existing issue reported?
Regards,
Tanmoy Sinha
=============================================
libguestfs: command: run: \ --verbose
libguestfs: command: run: \ --if-newer
libguestfs: command: run: \ --lock /var/tmp/.guestfs-0/lock
libguestfs: command: run: \ --copy-kernel
libguestfs: command: run: \ -f ext2
libguestfs: command: run: \ --host-cpu x86_64
libguestfs: command: run: \ /usr/lib/x86_64-linux-gnu/guestfs/supermin.d
libguestfs: command: run: \ -o /var/tmp/.guestfs-0/appliance.d
supermin: version: 5.1.17
supermin: package handler: debian/dpkg
supermin: acquiring lock on /var/tmp/.guestfs-0/lock
supermin: if-newer: output does not need rebuilding
libguestfs: finished building supermin appliance
libguestfs: begin testing qemu features
libguestfs: checking for previously cached test results of
/usr/bin/qemu-system-x86_64, in /var/tmp/.guestfs-0
libguestfs: loading previously cached test results
libguestfs: qemu version 2.8
libguestfs: finished testing qemu features
[00012ms] /usr/bin/qemu-system-x86_64 \
-global virtio-blk-pci.scsi=off \
-nodefconfig \
-enable-fips \
-nodefaults \
-display none \
-machine accel=kvm:tcg \
-cpu host \
-m 500 \
-no-reboot \
-rtc driftfix=slew \
-no-hpet \
-global kvm-pit.lost_tick_policy=discard \
-kernel /var/tmp/.guestfs-0/appliance.d/kernel \
-initrd /var/tmp/.guestfs-0/appliance.d/initrd \
-object rng-random,filename=/dev/urandom,id=rng0 \
-device virtio-rng-pci,rng=rng0 \
-device virtio-scsi-pci,id=scsi \
-drive
file=/tmp/libguestfswJEdlG/scratch.1,cache=unsafe,format=raw,id=hd0,if=none
\
-device scsi-hd,drive=hd0 \
-drive
file=/var/tmp/.guestfs-0/appliance.d/root,snapshot=on,id=appliance,cache=unsafe,if=none,format=raw
\
-device scsi-hd,drive=appliance \
-device virtio-serial-pci \
-serial stdio \
-device sga \
-chardev socket,path=/tmp/libguestfsj95RNX/guestfsd.sock,id=channel0 \
-device virtserialport,chardev=channel0,name=org.libguestfs.channel.0 \
-append 'panic=1 console=ttyS0 edd=off udevtimeout=6000
udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory
usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=/dev/sdb
selinux=0 guestfs_verbose=1 TERM=screen'
libguestfs: responding to serial console Device Status Report
\x1b[1;256r\x1b[256;256H\x1b[6n
Google, Inc.
Serial Graphics Adapter 12/07/13
SGABIOS $Id: sgabios.S 8 2010-04-22 00:03:40Z nlaredo $ (mjt@gandalf) Sat
Dec 7 15:39:53 UTC 2013
Term: 80x24
4 0
SeaBIOS (version 1.10.2-1)
On Fri, 23 Mar 2018 at 14:37 Pino Toscano <ptoscano(a)redhat.com> wrote:
On Friday, 23 March 2018 07:10:00 CET Tanmoy Sinha wrote:
> I am using a debian 9 guest, hosted on a ESXi platform with nested
> virtualisation enabled.
>
> On this debian 9 guest when I run libguesfs-test-tool, it fails with an
> error:
>
> "qemu-system-x86_64:
> /build/qemu-DqynNa/qemu-2.8+dfsg/target-i386/kvm.c:1805: kvm_put_msrs:
> Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed."
See also https://bugzilla.redhat.com/show_bug.cgi?id=1456407
This is basically a bug in VMware and the qemu bug
https://bugs.launchpad.net/qemu/+bug/1661386
has a couple of hints about what it can be changed in VMware to avoid
this situation.
--
Pino Toscano
6:07 p.m.
On Fri, Mar 23, 2018 at 05:35:38PM +0000, Tanmoy Sinha wrote:
Thanks. Reading the defect and the associated thread in
https://bugs.launchpad.net/qemu/+bug/1661386, I enabled performance
counters in VMWare guest settings. Now the kvm assertion `ret ==
cpu->kvm_msr_buf->nmsrs' is gone. But the qemu/kvm hangs after printing
"SeaBIOS (version 1.10.2-1)"
I presume this too is a qemu/kvm issue for vmware platforms. Is there any
existing issue reported?
Does it work if you set:
export LIBGUESTFS_BACKEND_SETTINGS=force_tcg
?
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
11:16 p.m.
Yes it works if I use force_tcg env variable
On Sat, 24 Mar 2018 at 4:37 AM, Richard W.M. Jones <rjones(a)redhat.com>
wrote:
On Fri, Mar 23, 2018 at 05:35:38PM +0000, Tanmoy Sinha wrote:
> Thanks. Reading the defect and the associated thread in
> https://bugs.launchpad.net/qemu/+bug/1661386, I enabled performance
> counters in VMWare guest settings. Now the kvm assertion `ret ==
> cpu->kvm_msr_buf->nmsrs' is gone. But the qemu/kvm hangs after printing
> "SeaBIOS (version 1.10.2-1)"
>
> I presume this too is a qemu/kvm issue for vmware platforms. Is there any
> existing issue reported?
Does it work if you set:
export LIBGUESTFS_BACKEND_SETTINGS=force_tcg
?
Rich.
--
Richard Jones, Virtualization Group, Red Hat
http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
10:08 a.m.
Even though force_tcg works, I intend not to run it on emulation. Is there
way I can run it over kvm? The other observation is, without force_tcg if I
use the machine type as *pc-i440fx-2.**1*,accel=kvm it works fine. The
default machine type for my host *pc-i440fx-2.8, *which seems to crib.
On Sat, 24 Mar 2018 at 09:46 Tanmoy Sinha <tanmoy.sinha(a)gmail.com> wrote:
Yes it works if I use force_tcg env variable
On Sat, 24 Mar 2018 at 4:37 AM, Richard W.M. Jones <rjones(a)redhat.com>
wrote:
>
> On Fri, Mar 23, 2018 at 05:35:38PM +0000, Tanmoy Sinha wrote:
> > Thanks. Reading the defect and the associated thread in
> > https://bugs.launchpad.net/qemu/+bug/1661386, I enabled performance
> > counters in VMWare guest settings. Now the kvm assertion `ret ==
> > cpu->kvm_msr_buf->nmsrs' is gone. But the qemu/kvm hangs after
printing
> > "SeaBIOS (version 1.10.2-1)"
> >
> > I presume this too is a qemu/kvm issue for vmware platforms. Is there
> any
> > existing issue reported?
>
> Does it work if you set:
>
> export LIBGUESTFS_BACKEND_SETTINGS=force_tcg
>
> ?
>
> Rich.
>
> --
> Richard Jones, Virtualization Group, Red Hat
> http://people.redhat.com/~rjones
> Read my programming and virtualization blog: http://rwmj.wordpress.com
> virt-builder quickly builds VMs from scratch
> http://libguestfs.org/virt-builder.1.html
>
10:11 a.m.
On Sat, Mar 24, 2018 at 03:08:16PM +0000, Tanmoy Sinha wrote:
Even though force_tcg works, I intend not to run it on emulation. Is
there
way I can run it over kvm? The other observation is, without force_tcg if I
use the machine type as *pc-i440fx-2.**1*,accel=kvm it works fine. The
default machine type for my host *pc-i440fx-2.8, *which seems to crib.
I don't know, but this is basically a bug in VMware, so you need
to ask them to fix their nested KVM-on-ESXi use case.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines. Supports shell scripting,
bindings from many languages. http://libguestfs.org
10:49 a.m.
Thanks will write to VMWare forums.
Btw, is there a way I can change the qemu default machine type? I mean when
I specify *-machine pc *it picks up pc-i440fx-2.8 by default.
Or is it tied to the underlying host hardware?
*$> *qemu-system-x86_64* -machine help*
Supported machines are:
*pc* Standard PC (i440FX + PIIX, 1996) (alias of
pc-i440fx-2.8)
*pc-i440fx-2.8* Standard PC (i440FX + PIIX, 1996) (default)
pc-i440fx-2.7 Standard PC (i440FX + PIIX, 1996)
pc-i440fx-2.6 Standard PC (i440FX + PIIX, 1996)
pc-i440fx-2.5 Standard PC (i440FX + PIIX, 1996)
pc-i440fx-2.4 Standard PC (i440FX + PIIX, 1996)
pc-i440fx-2.3 Standard PC (i440FX + PIIX, 1996)
pc-i440fx-2.2 Standard PC (i440FX + PIIX, 1996)
*pc-i440fx-2.1* Standard PC (i440FX + PIIX, 1996)
.....
On Sat, 24 Mar 2018 at 20:41 Richard W.M. Jones <rjones(a)redhat.com> wrote:
On Sat, Mar 24, 2018 at 03:08:16PM +0000, Tanmoy Sinha wrote:
> Even though force_tcg works, I intend not to run it on emulation. Is
there
> way I can run it over kvm? The other observation is, without force_tcg
if I
> use the machine type as *pc-i440fx-2.**1*,accel=kvm it works fine. The
> default machine type for my host *pc-i440fx-2.8, *which seems to crib.
I don't know, but this is basically a bug in VMware, so you need
to ask them to fix their nested KVM-on-ESXi use case.
Rich.
--
Richard Jones, Virtualization Group, Red Hat
http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines. Supports shell scripting,
bindings from many languages. http://libguestfs.org
12:42 p.m.
On Sat, Mar 24, 2018 at 03:49:24PM +0000, Tanmoy Sinha wrote:
Thanks will write to VMWare forums.
Btw, is there a way I can change the qemu default machine type? I mean when
I specify *-machine pc *it picks up pc-i440fx-2.8 by default.
qemu maps aliases like 'pc' to some emulated machine type. You
can see the mapping by doing:
$ qemu-system-x86_64 -machine \?
Or is it tied to the underlying host hardware?
No, it's something that qemu emulates and is nothing to do with
the host hardware.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
1:13 a.m.
On Sat, Mar 24, 2018 at 03:11:12PM +0000, Richard W.M. Jones wrote:
On Sat, Mar 24, 2018 at 03:08:16PM +0000, Tanmoy Sinha wrote:
> Even though force_tcg works, I intend not to run it on emulation. Is there
> way I can run it over kvm? The other observation is, without force_tcg if I
> use the machine type as *pc-i440fx-2.**1*,accel=kvm it works fine. The
> default machine type for my host *pc-i440fx-2.8, *which seems to crib.
I don't know, but this is basically a bug in VMware, so you need
to ask them to fix their nested KVM-on-ESXi use case.
We've encountered this problem, too.
Strictly speaking, the bug is not in VMWare, but rather in KVM: on
EPT_MISCONFIG vmexits it assumed the processor to set the instuction
length field. This wasn't mandated by the spec but the real processors
did that. OTOH some hypervisors (VMWare, Hyper-V) didn't do that for
the nested hypervisor. As a result, when handling MMIO the guest
instruction pointer didn't get advanced, i.e. the guest got stuck in an
infinite loop.
The difference between the old and the new machine type is that the
latter turns on newer virtio protocol version employing MMIO, exposing
this bug.
The fix is commit d391f1207067268261add0485f0f34503539c5b0 which went
into 4.16-rc1.
Roman.
2:30 a.m.
New subject: Nested KVM on VMware ESXi failure (was: Re: Issue with libguestfs-test-tool on a guest hosted on VMWare ESXi)
[Adding Paolo and Vitaly, but FYI only as the bug seems to have an
upstream fix already.]
On Mon, Mar 26, 2018 at 09:13:45AM +0300, Roman Kagan wrote:
On Sat, Mar 24, 2018 at 03:11:12PM +0000, Richard W.M. Jones wrote:
> On Sat, Mar 24, 2018 at 03:08:16PM +0000, Tanmoy Sinha wrote:
> > Even though force_tcg works, I intend not to run it on emulation. Is there
> > way I can run it over kvm? The other observation is, without force_tcg if I
> > use the machine type as *pc-i440fx-2.**1*,accel=kvm it works fine. The
> > default machine type for my host *pc-i440fx-2.8, *which seems to crib.
>
> I don't know, but this is basically a bug in VMware, so you need
> to ask them to fix their nested KVM-on-ESXi use case.
We've encountered this problem, too.
Strictly speaking, the bug is not in VMWare, but rather in KVM: on
EPT_MISCONFIG vmexits it assumed the processor to set the instuction
length field. This wasn't mandated by the spec but the real processors
did that. OTOH some hypervisors (VMWare, Hyper-V) didn't do that for
the nested hypervisor. As a result, when handling MMIO the guest
instruction pointer didn't get advanced, i.e. the guest got stuck in an
infinite loop.
The difference between the old and the new machine type is that the
latter turns on newer virtio protocol version employing MMIO, exposing
this bug.
The fix is commit d391f1207067268261add0485f0f34503539c5b0 which went
into 4.16-rc1.
Can you (Tanmoy) please try a newer kernel inside the VMware guest?
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines. Supports shell scripting,
bindings from many languages. http://libguestfs.org
6:22 a.m.
New subject: Nested KVM on VMware ESXi failure (was: Re: Issue with libguestfs-test-tool on a guest hosted on VMWare ESXi)
I am trying that out. Debian 9 (stretch) backports still doesn't have 4.16
kernel.
So I building the kernel from the source. Will keep you posted.
On Mon, 26 Mar 2018 at 13:00 Richard W.M. Jones <rjones(a)redhat.com> wrote:
[Adding Paolo and Vitaly, but FYI only as the bug seems to have an
upstream fix already.]
On Mon, Mar 26, 2018 at 09:13:45AM +0300, Roman Kagan wrote:
> On Sat, Mar 24, 2018 at 03:11:12PM +0000, Richard W.M. Jones wrote:
> > On Sat, Mar 24, 2018 at 03:08:16PM +0000, Tanmoy Sinha wrote:
> > > Even though force_tcg works, I intend not to run it on emulation. Is
there
> > > way I can run it over kvm? The other observation is, without
force_tcg if I
> > > use the machine type as *pc-i440fx-2.**1*,accel=kvm it works fine.
The
> > > default machine type for my host *pc-i440fx-2.8, *which seems to
crib.
> >
> > I don't know, but this is basically a bug in VMware, so you need
> > to ask them to fix their nested KVM-on-ESXi use case.
>
> We've encountered this problem, too.
>
> Strictly speaking, the bug is not in VMWare, but rather in KVM: on
> EPT_MISCONFIG vmexits it assumed the processor to set the instuction
> length field. This wasn't mandated by the spec but the real processors
> did that. OTOH some hypervisors (VMWare, Hyper-V) didn't do that for
> the nested hypervisor. As a result, when handling MMIO the guest
> instruction pointer didn't get advanced, i.e. the guest got stuck in an
> infinite loop.
>
> The difference between the old and the new machine type is that the
> latter turns on newer virtio protocol version employing MMIO, exposing
> this bug.
>
> The fix is commit d391f1207067268261add0485f0f34503539c5b0 which went
> into 4.16-rc1.
Can you (Tanmoy) please try a newer kernel inside the VMware guest?
Rich.
--
Richard Jones, Virtualization Group, Red Hat
http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines. Supports shell scripting,
bindings from many languages. http://libguestfs.org
10:18 a.m.
New subject: Nested KVM on VMware ESXi failure (was: Re: Issue with libguestfs-test-tool on a guest hosted on VMWare ESXi)
*It works*.
I built and deployed the linux 4.16.0-rc7 kernel on my debian 9 guest
running over VMWare. Now libguestfs-test-tool just works fine when qemu
picks the default machine type pc-i440fx-2.8
$> uname -a
Linux tanmoy-deb-host 4.16.0-rc7 #1 SMP Mon Mar 26 09:52:54 UTC 2018 x86_64
GNU/Linux
On Mon, 26 Mar 2018 at 16:52 Tanmoy Sinha <tanmoy.sinha(a)gmail.com> wrote:
I am trying that out. Debian 9 (stretch) backports still doesn't
have 4.16
kernel.
So I building the kernel from the source. Will keep you posted.
On Mon, 26 Mar 2018 at 13:00 Richard W.M. Jones <rjones(a)redhat.com> wrote:
> [Adding Paolo and Vitaly, but FYI only as the bug seems to have an
> upstream fix already.]
>
> On Mon, Mar 26, 2018 at 09:13:45AM +0300, Roman Kagan wrote:
> > On Sat, Mar 24, 2018 at 03:11:12PM +0000, Richard W.M. Jones wrote:
> > > On Sat, Mar 24, 2018 at 03:08:16PM +0000, Tanmoy Sinha wrote:
> > > > Even though force_tcg works, I intend not to run it on emulation.
> Is there
> > > > way I can run it over kvm? The other observation is, without
> force_tcg if I
> > > > use the machine type as *pc-i440fx-2.**1*,accel=kvm it works fine.
> The
> > > > default machine type for my host *pc-i440fx-2.8, *which seems to
> crib.
> > >
> > > I don't know, but this is basically a bug in VMware, so you need
> > > to ask them to fix their nested KVM-on-ESXi use case.
> >
> > We've encountered this problem, too.
> >
> > Strictly speaking, the bug is not in VMWare, but rather in KVM: on
> > EPT_MISCONFIG vmexits it assumed the processor to set the instuction
> > length field. This wasn't mandated by the spec but the real processors
> > did that. OTOH some hypervisors (VMWare, Hyper-V) didn't do that for
> > the nested hypervisor. As a result, when handling MMIO the guest
> > instruction pointer didn't get advanced, i.e. the guest got stuck in an
> > infinite loop.
> >
> > The difference between the old and the new machine type is that the
> > latter turns on newer virtio protocol version employing MMIO, exposing
> > this bug.
> >
> > The fix is commit d391f1207067268261add0485f0f34503539c5b0 which went
> > into 4.16-rc1.
>
> Can you (Tanmoy) please try a newer kernel inside the VMware guest?
>
> Rich.
>
> --
> Richard Jones, Virtualization Group, Red Hat
> http://people.redhat.com/~rjones
> Read my programming and virtualization blog: http://rwmj.wordpress.com
> libguestfs lets you edit virtual machines. Supports shell scripting,
> bindings from many languages. http://libguestfs.org
>
_______________________________________________
Libguestfs mailing list
Libguestfs(a)redhat.com
https://www.redhat.com/mailman/listinfo/libguestfs
2434
days inactive
2437
days old
12 comments
4 participants
participants (4)
-
Pino Toscano -
Richard W.M. Jones -
Roman Kagan -
Tanmoy Sinha