7:44 a.m.
=> Next, is to try: L1 w/o nesting, VMCS shadowing disabled.
----------------------------------------------------------------------
# On L0
$ vi /etc/modprobe.d/dist.conf
- s/options kvm-intel nested=y/options kvm-intel nested=n/
# For the test purpose, remove the XML fragment (which adds VMX
# extensions) for L1 CPU
$ virsh edit regular-guest
- Remove the below fragment, save it:
<cpu mode='custom' match='exact'>
<model fallback='allow'>Haswell</model>
<feature policy='require' name='vmx'/>
</cpu>
# Reboot
$ init 6
# Check for the nested parameter on L0
$ cat /sys/module/kvm_intel/parameters/nested
N
# Remove any old appliances
$ rm -rf /var/tmp/.guestfs-0
# Run:
$ for i in {1..10}; do time guestfish -a /dev/null run; done | tee \
guestfish-timings-L1-no-nested.txt
----------------------------------------------------------------------
Results for the above case -- no nesting, guestfish run in 1st level guest:
----------------------------------------------------------------------
$ for i in {1..10}; do time guestfish -a /dev/null run; done | tee
guestfish-timings-L1-no-nesting.txt
real 1m10.899s
user 0m13.558s
sys 0m8.167s
real 0m29.899s
user 0m0.066s
sys 0m0.522s
real 0m25.310s
user 0m0.056s
sys 0m0.516s
real 0m35.267s
user 0m0.053s
sys 0m0.515s
real 0m24.872s
user 0m0.051s
sys 0m0.545s
real 0m24.827s
user 0m0.044s
sys 0m0.538s
real 0m41.053s
user 0m0.060s
sys 0m0.531s
real 0m29.107s
user 0m0.049s
sys 0m0.534s
real 0m30.196s
user 0m0.051s
sys 0m0.532s
real 0m52.815s
user 0m0.052s
sys 0m0.520s
----------------------------------------------------------------------
To summarize:
L0 (bare-metal):
- guestfish run: ~ 3 seconds
L1 (guest):
- w/o nesting, guestfish run: ~ 52 seconds
- w/ nesting, guestfish run: ~ 8 seconds (Almost 6 1/2 times speed improvement)
For Reference:
--------------
1/ Level-1 guest qemu-kvm command-line, indicating that I *did* remove <cpu>
fragment
gracefully, i.e. no nesting.
===================================
$ ps -ef | grep -i qemu
qemu 2367 1 86 08:29 ? 00:00:03 /usr/bin/qemu-system-x86_64 -machine
accel=kvm -name regular-guest -S -machine pc-i440fx-1.4,accel=kvm,usb=off -m 6144 -smp
4,sockets=4,cores=1,threads=1 -uuid 4ed9ac0b-7f72-dfcf-68b3-e6fe2ac588b2 -nographic
-no-user-config -nodefaults -chardev
socket,id=charmonitor,path=/var/lib/libvirt/qemu/regular-guest.monitor,server,nowait -mon
chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device
piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive
file=/home/test/vmimages/regular-guest.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none
-device
virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1
-netdev tap,fd=23,id=hostnet0,vhost=on,vhostfd=24 -device
virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:80:c1:34,bus=pci.0,addr=0x3 -chardev
pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device
usb-tablet,id=input0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
===================================
2/ Level-1 guest qemu-kvm command-line *with* nesting:
===================================
$ ps -ef | grep -i qemu
qemu 4962 1 21 15:41 ? 00:00:41 /usr/bin/qemu-system-x86_64 -machine
accel=kvm -name regular-guest -S -machine pc-i440fx-1.4,accel=kvm,usb=off -cpu
Haswell,+vmx -m 6144 -smp 4,sockets=4,cores=1,threads=1 -uuid
4ed9ac0b-7f72-dfcf-68b3-e6fe2ac588b2 -nographic -no-user-config -nodefaults -chardev
socket,id=charmonitor,path=/var/lib/libvirt/qemu/regular-guest.monitor,server,nowait -mon
chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device
piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive
file=/home/test/vmimages/regular-guest.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none
-device
virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1
-netdev tap,fd=23,id=hostnet0,vhost=on,vhostfd=24 -device
virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:80:c1:34,bus=pci.0,addr=0x3 -chardev
pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device
usb-tablet,id=input0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
===================================
The setup details are here -
https://raw.github.com/kashyapc/nvmx-haswell/master/SETUP-nVMX.rst
Rich, if you have further thoughts, please let me know.
Thanks.
--
/kashyap