=> Next, is to try: L1 w/o nesting, VMCS shadowing disabled. ----------------------------------------------------------------------
# On L0 $ vi /etc/modprobe.d/dist.conf - s/options kvm-intel nested=y/options kvm-intel nested=n/
# For the test purpose, remove the XML fragment (which adds VMX # extensions) for L1 CPU $ virsh edit regular-guest - Remove the below fragment, save it: <cpu mode='custom' match='exact'> <model fallback='allow'>Haswell</model> <feature policy='require' name='vmx'/> </cpu>
# Reboot $ init 6
# Check for the nested parameter on L0 $ cat /sys/module/kvm_intel/parameters/nested N
# Remove any old appliances $ rm -rf /var/tmp/.guestfs-0
# Run: $ for i in {1..10}; do time guestfish -a /dev/null run; done | tee \ guestfish-timings-L1-no-nested.txt ----------------------------------------------------------------------
Results for the above case -- no nesting, guestfish run in 1st level guest: ---------------------------------------------------------------------- $ for i in {1..10}; do time guestfish -a /dev/null run; done | tee guestfish-timings-L1-no-nesting.txt real 1m10.899s user 0m13.558s sys 0m8.167s real 0m29.899s user 0m0.066s sys 0m0.522s real 0m25.310s user 0m0.056s sys 0m0.516s real 0m35.267s user 0m0.053s sys 0m0.515s real 0m24.872s user 0m0.051s sys 0m0.545s real 0m24.827s user 0m0.044s sys 0m0.538s real 0m41.053s user 0m0.060s sys 0m0.531s real 0m29.107s user 0m0.049s sys 0m0.534s real 0m30.196s user 0m0.051s sys 0m0.532s real 0m52.815s user 0m0.052s sys 0m0.520s ---------------------------------------------------------------------- To summarize: L0 (bare-metal): - guestfish run: ~ 3 seconds L1 (guest): - w/o nesting, guestfish run: ~ 52 seconds - w/ nesting, guestfish run: ~ 8 seconds (Almost 6 1/2 times speed improvement) For Reference: -------------- 1/ Level-1 guest qemu-kvm command-line, indicating that I *did* remove <cpu> fragment gracefully, i.e. no nesting. =================================== $ ps -ef | grep -i qemu qemu 2367 1 86 08:29 ? 00:00:03 /usr/bin/qemu-system-x86_64 -machine accel=kvm -name regular-guest -S -machine pc-i440fx-1.4,accel=kvm,usb=off -m 6144 -smp 4,sockets=4,cores=1,threads=1 -uuid 4ed9ac0b-7f72-dfcf-68b3-e6fe2ac588b2 -nographic -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/regular-guest.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/home/test/vmimages/regular-guest.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=23,id=hostnet0,vhost=on,vhostfd=24 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:80:c1:34,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 =================================== 2/ Level-1 guest qemu-kvm command-line *with* nesting: =================================== $ ps -ef | grep -i qemu qemu 4962 1 21 15:41 ? 00:00:41 /usr/bin/qemu-system-x86_64 -machine accel=kvm -name regular-guest -S -machine pc-i440fx-1.4,accel=kvm,usb=off -cpu Haswell,+vmx -m 6144 -smp 4,sockets=4,cores=1,threads=1 -uuid 4ed9ac0b-7f72-dfcf-68b3-e6fe2ac588b2 -nographic -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/regular-guest.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/home/test/vmimages/regular-guest.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=23,id=hostnet0,vhost=on,vhostfd=24 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:80:c1:34,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 =================================== The setup details are here - https://raw.github.com/kashyapc/nvmx-haswell/master/SETUP-nVMX.rst Rich, if you have further thoughts, please let me know. Thanks. -- /kashyap