Re: [Libguestfs] Cubietruck: QEMU, KVM and Fedora

Hi Kashyap, Thanks for offering to help. The day job caused a pause and it took me an hour to reassert where I was in the process. I usually hang out on freenet as xsited. I am still reading the process for formally reporting the issue as described on the site. Walking through this has shown me some significant difference from previously working almost exclusively with qemu-system-x86_64. It's different. As you asserted before -cpu host is required on the cubietruck to allow kvm to be utilized due to the lack of emulation support. In the meantime, I was waiting to get my hands on a cortex-a15 for the purpose of sussing out an effort involved for a debian 8 host and a openwrt guest. Because the qemu-system-arm supports vexpress-a15 there is better emulation support, but compiling the guest kernel to match the target host cpu offers a better chance that kvm acceleration is utilize? In other word -cpu host is always better?

qemu-system-arm -kernel zImage -dtb cortex_a15x4.dtb -m 512 -M vexpress-a15 -cpu cortex-a15 Is my fuzz understanding of this part correct? Below are the logs I collected since last weekend.

root@cubietruck:~# /usr/bin/qemu-system-arm --version QEMU emulator version 2.5.0 (Debian 1:2.5+dfsg-5), Copyright (c) 2003-2008 Fabrice Bellard root@cubietruck:~# /usr/bin/libguestfs-test-tool -V libguestfs-test-tool 1.30.6 root@cubietruck:~# /usr/bin/libguestfs-test-tool ************************************************************ * IMPORTANT NOTICE * * When reporting bugs, include the COMPLETE, UNEDITED * output below in your bug report. * ************************************************************ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin SELinux: sh: 1: getenforce: not found guestfs_get_append: (null) guestfs_get_autosync: 1 guestfs_get_backend: direct guestfs_get_backend_settings: [] guestfs_get_cachedir: /var/tmp guestfs_get_direct: 0 guestfs_get_hv: /usr/bin/qemu-system-arm guestfs_get_memsize: 500 guestfs_get_network: 0 guestfs_get_path: /usr/lib/arm-linux-gnueabihf/guestfs guestfs_get_pgroup: 0 guestfs_get_program: libguestfs-test-tool guestfs_get_recovery_proc: 1 guestfs_get_selinux: 0 guestfs_get_smp: 1 guestfs_get_tmpdir: /tmp guestfs_get_trace: 0 guestfs_get_verbose: 1 host_cpu: arm Launching appliance, timeout set to 600 seconds. libguestfs: launch: program=libguestfs-test-tool libguestfs: launch: version=1.30.6 libguestfs: launch: backend registered: unix libguestfs: launch: backend registered: uml libguestfs: launch: backend registered: libvirt libguestfs: launch: backend registered: direct libguestfs: launch: backend=direct libguestfs: launch: tmpdir=/tmp/libguestfssaEUV0 libguestfs: launch: umask=0022 libguestfs: launch: euid=0 libguestfs: [00005ms] begin building supermin appliance libguestfs: [00006ms] run supermin libguestfs: command: run: /usr/bin/supermin libguestfs: command: run: \ --build libguestfs: command: run: \ --verbose libguestfs: command: run: \ --if-newer libguestfs: command: run: \ --lock /var/tmp/.guestfs-0/lock libguestfs: command: run: \ --copy-kernel libguestfs: command: run: \ -f ext2 libguestfs: command: run: \ --host-cpu arm libguestfs: command: run: \ --dtb vexpress*a15-tc1.dtb libguestfs: command: run: \ /usr/lib/arm-linux-gnueabihf/guestfs/supermin.d libguestfs: command: run: \ -o /var/tmp/.guestfs-0/appliance.d supermin: version: 5.1.14 supermin: package handler: debian/dpkg supermin: acquiring lock on /var/tmp/.guestfs-0/lock supermin: if-newer: output does not need rebuilding libguestfs: [00075ms] finished building supermin appliance libguestfs: [00076ms] begin testing qemu features libguestfs: command: run: /usr/bin/qemu-system-arm libguestfs: command: run: \ -display none libguestfs: command: run: \ -help libguestfs: command: run: /usr/bin/qemu-system-arm libguestfs: command: run: \ -display none libguestfs: command: run: \ -version libguestfs: qemu version 2.5 libguestfs: command: run: /usr/bin/qemu-system-arm libguestfs: command: run: \ -display none libguestfs: command: run: \ -M virt libguestfs: command: run: \ -machine accel=kvm:tcg libguestfs: command: run: \ -device ? libguestfs: [00573ms] finished testing qemu features [00633ms] /usr/bin/qemu-system-arm \ -global virtio-blk-device.scsi=off \ -nodefconfig \ -enable-fips \ -nodefaults \ -display none \ -M virt \ -cpu host \ -machine accel=kvm:tcg \ -m 500 \ -no-reboot \ -rtc driftfix=slew \ -global kvm-pit.lost_tick_policy=discard \ -kernel /var/tmp/.guestfs-0/appliance.d/kernel \ -dtb /var/tmp/.guestfs-0/appliance.d/dtb \ -initrd /var/tmp/.guestfs-0/appliance.d/initrd \ -device virtio-scsi-device,id=scsi \ -drive file=/tmp/libguestfssaEUV0/scratch.1,cache=unsafe,format=raw,id=hd0,if=none \ -device scsi-hd,drive=hd0 \ -drive file=/var/tmp/.guestfs-0/appliance.d/root,snapshot=on,id=appliance,cache=unsafe,if=none \ -device scsi-hd,drive=appliance \ -device virtio-serial-device \ -serial stdio \ -chardev socket,path=/tmp/libguestfssaEUV0/guestfsd.sock,id=channel0 \ -device virtserialport,chardev=channel0,name=org.libguestfs.channel.0 \ -append 'panic=1 mem=500M console=ttyAMA0 udevtimeout=6000 udev.event-timeout=6000 no_timer_check acpi=off printk.time=1 cgroup_disable=memory root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=xterm-256color' qemu-system-arm: Warning: global kvm-pit.lost_tick_policy has invalid class name Alarm clock root@cubietruck:~# virt-host-validate QEMU: Checking for hardware virtualization : WARN (Only emulated CPUs are available, performance will be significantly limited) QEMU: Checking if device /dev/vhost-net exists : WARN (Load the 'vhost_net' module to improve performance of virtio networking) QEMU: Checking if device /dev/net/tun exists : PASS QEMU: Checking for cgroup 'memory' controller support : WARN (Enable CONFIG_MEMCG in kernel Kconfig file) QEMU: Checking for cgroup 'cpu' controller support : PASS QEMU: Checking for cgroup 'cpu' controller mount-point : PASS QEMU: Checking for cgroup 'cpuacct' controller support : PASS QEMU: Checking for cgroup 'cpuacct' controller mount-point : PASS QEMU: Checking for cgroup 'devices' controller support : PASS QEMU: Checking for cgroup 'devices' controller mount-point : PASS QEMU: Checking for cgroup 'net_cls' controller support : PASS QEMU: Checking for cgroup 'net_cls' controller mount-point : PASS QEMU: Checking for cgroup 'blkio' controller support : PASS QEMU: Checking for cgroup 'blkio' controller mount-point : PASS QEMU: Checking for device assignment IOMMU support : WARN (Unknown if this platform has IOMMU support) LXC: Checking for Linux >= 2.6.26 : PASS LXC: Checking for namespace ipc : PASS LXC: Checking for namespace mnt : PASS LXC: Checking for namespace pid : PASS LXC: Checking for namespace uts : PASS LXC: Checking for namespace net : PASS LXC: Checking for namespace user : PASS LXC: Checking for cgroup 'memory' controller support : FAIL (Enable CONFIG_MEMCG in kernel Kconfig file) LXC: Checking for cgroup 'cpu' controller support : PASS LXC: Checking for cgroup 'cpu' controller mount-point : PASS LXC: Checking for cgroup 'cpuacct' controller support : PASS LXC: Checking for cgroup 'cpuacct' controller mount-point : PASS LXC: Checking for cgroup 'devices' controller support : PASS LXC: Checking for cgroup 'devices' controller mount-point : PASS LXC: Checking for cgroup 'net_cls' controller support : PASS LXC: Checking for cgroup 'net_cls' controller mount-point : PASS LXC: Checking for cgroup 'freezer' controller support : PASS LXC: Checking for cgroup 'freezer' controller mount-point : PASS root@cubietruck:~# Kashyap Chamarthy February 16, 2016 at 1:08 PM On Tue, Feb 16, 2016 at 8:15 PM, Thomas Kee <xsited(a)yahoo.com&gt; wrote: Thanks for the responds and the tool. I am getting an Alarm timeout when running libguestfs-test-tool and it tries to launch qemu-system-arm -cpu host for the first time. Not sure if it is bug report worthy yet. At this point, it's better if you hop over to #libguestfs channel on Freenode, I'm there (but it's late at the moment here in western Europe). I'm there during day time. And, there are other friendly developers that can help out. But let me look at virt-host-validate. Good luck! -t Kashyap Chamarthy February 16, 2016 at 1:33 AM On Tue, Feb 16, 2016 at 1:15 AM, Thomas Kee <xsited(a)yahoo.com&gt; wrote: Oh boy. 10 second later, I remember /dev/kvm. Too much xen lately. :-) Yeah. A related handy tool is `virt-host-validate` , which will perform a bunch of KVM-related checks: --------------------- $ virt-host-validate QEMU: Checking for hardware virtualization : PASS QEMU: Checking if device /dev/kvm exists : PASS QEMU: Checking if device /dev/kvm is accessible : PASS QEMU: Checking if device /dev/vhost-net exists : PASS QEMU: Checking if device /dev/net/tun exists : PASS LXC: Checking for Linux >= 2.6.26 : PASS --------------------- Regards, Kashyap thx Thomas Kee February 15, 2016 at 3:34 PM Hi, Thanks for the write up. Very helpful. Just a couple of doubts: How are you checking the kernel is built with KVM module enabled? With more current QEMU -cpu [ -machine ] still is valid ? Thanks, Thomas root@cubietruck:~# dmesg | grep CPU [ 0.000000] CPU: ARMv7 Processor [410fc074] revision 4 (ARMv7), cr=30c5387d [ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache [ 0.000000] PERCPU: Embedded 12 pages/cpu @ee5c0000 s19916 r8192 d21044 u49152 [ 0.000000] RCU restricting CPUs from NR_CPUS=4 to nr_cpu_ids=2. [ 0.004932] CPU: Testing write buffer coherency: ok [ 0.046656] CPU0: thread -1, cpu 0, socket 0, mpidr 80000000 [ 0.052951] CPU1: thread -1, cpu 1, socket 0, mpidr 80000001 [ 0.053072] Brought up 2 CPUs [ 0.053106] CPU: All CPU(s) started in HYP mode. [ 0.053111] CPU: Virtualization extensions available. [ 1.572315] ledtrig-cpu: registered to indicate activity on CPUs [ 1.575670] ThumbEE CPU extension supported. root@cubietruck:~# dmesg | grep SMP [ 0.000000] Linux version 4.3.0-1-armmp-lpae (debian-kernel(a)lists.debian.org) (gcc version 4.9.3 (Debian 4.9.3-11) ) #1 SMP Debian 4.3.5-1 (2016-02-06) [ 0.000000] Forcing write-allocate cache policy for SMP [ 0.053098] SMP: Total of 2 processors activated (96.00 BogoMIPS). root@cubietruck:~# dmesg | grep kvm [ 1.499947] kvm [1]: interrupt-controller@1c84000 IRQ51 [ 1.500434] kvm [1]: timer IRQ19 [ 1.500462] kvm [1]: Hyp mode initialized successfully root@cubietruck:~# uname -a Linux cubietruck 4.3.0-1-armmp-lpae #1 SMP Debian 4.3.5-1 (2016-02-06) armv7l GNU/Linux root@cubietruck:~# qemu-system-arm -machine help akita Sharp SL-C1000 (Akita) PDA (PXA270) borzoi Sharp SL-C3100 (Borzoi) PDA (PXA270) canon-a1100 Canon PowerShot A1100 IS cheetah Palm Tungsten|E aka. Cheetah PDA (OMAP310) collie Sharp SL-5500 (Collie) PDA (SA-1110) connex Gumstix Connex (PXA255) cubieboard cubietech cubieboard highbank Calxeda Highbank (ECX-1000) imx25-pdk ARM i.MX25 PDK board (ARM926) integratorcp ARM Integrator/CP (ARM926EJ-S) kzm ARM KZM Emulation Baseboard (ARM1136) lm3s6965evb Stellaris LM3S6965EVB lm3s811evb Stellaris LM3S811EVB mainstone Mainstone II (PXA27x) midway Calxeda Midway (ECX-2000) musicpal Marvell 88w8618 / MusicPal (ARM926EJ-S) n800 Nokia N800 tablet aka. RX-34 (OMAP2420) n810 Nokia N810 tablet aka. RX-44 (OMAP2420) netduino2 Netduino 2 Machine none empty machine nuri Samsung NURI board (Exynos4210) realview-eb ARM RealView Emulation Baseboard (ARM926EJ-S) realview-eb-mpcore ARM RealView Emulation Baseboard (ARM11MPCore) realview-pb-a8 ARM RealView Platform Baseboard for Cortex-A8 realview-pbx-a9 ARM RealView Platform Baseboard Explore for Cortex-A9 smdkc210 Samsung SMDKC210 board (Exynos4210) spitz Sharp SL-C3000 (Spitz) PDA (PXA270) sx1 Siemens SX1 (OMAP310) V2 sx1-v1 Siemens SX1 (OMAP310) V1 terrier Sharp SL-C3200 (Terrier) PDA (PXA270) tosa Sharp SL-6000 (Tosa) PDA (PXA255) verdex Gumstix Verdex (PXA270) versatileab ARM Versatile/AB (ARM926EJ-S) versatilepb ARM Versatile/PB (ARM926EJ-S) vexpress-a15 ARM Versatile Express for Cortex-A15 vexpress-a9 ARM Versatile Express for Cortex-A9 virt ARM Virtual Machine xilinx-zynq-a9 Xilinx Zynq Platform Baseboard for Cortex-A9 z2 Zipit Z2 (PXA27x) -- Thomas Kee February 15, 2016 at 4:15 PM Oh boy. 10 second later, I remember /dev/kvm. Too much xen lately. thx Thomas Kee February 15, 2016 at 3:34 PM Hi, Thanks for the write up. Very helpful. Just a couple of doubts: How are you checking the kernel is built with KVM module enabled? With more current QEMU -cpu [ -machine ] still is valid ? Thanks, Thomas root@cubietruck:~# dmesg | grep CPU [ 0.000000] CPU: ARMv7 Processor [410fc074] revision 4 (ARMv7), cr=30c5387d [ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache [ 0.000000] PERCPU: Embedded 12 pages/cpu @ee5c0000 s19916 r8192 d21044 u49152 [ 0.000000] RCU restricting CPUs from NR_CPUS=4 to nr_cpu_ids=2. [ 0.004932] CPU: Testing write buffer coherency: ok [ 0.046656] CPU0: thread -1, cpu 0, socket 0, mpidr 80000000 [ 0.052951] CPU1: thread -1, cpu 1, socket 0, mpidr 80000001 [ 0.053072] Brought up 2 CPUs [ 0.053106] CPU: All CPU(s) started in HYP mode. [ 0.053111] CPU: Virtualization extensions available. [ 1.572315] ledtrig-cpu: registered to indicate activity on CPUs [ 1.575670] ThumbEE CPU extension supported. root@cubietruck:~# dmesg | grep SMP [ 0.000000] Linux version 4.3.0-1-armmp-lpae (debian-kernel(a)lists.debian.org) (gcc version 4.9.3 (Debian 4.9.3-11) ) #1 SMP Debian 4.3.5-1 (2016-02-06) [ 0.000000] Forcing write-allocate cache policy for SMP [ 0.053098] SMP: Total of 2 processors activated (96.00 BogoMIPS). root@cubietruck:~# dmesg | grep kvm [ 1.499947] kvm [1]: interrupt-controller@1c84000 IRQ51 [ 1.500434] kvm [1]: timer IRQ19 [ 1.500462] kvm [1]: Hyp mode initialized successfully root@cubietruck:~# uname -a Linux cubietruck 4.3.0-1-armmp-lpae #1 SMP Debian 4.3.5-1 (2016-02-06) armv7l GNU/Linux root@cubietruck:~# qemu-system-arm -machine help akita Sharp SL-C1000 (Akita) PDA (PXA270) borzoi Sharp SL-C3100 (Borzoi) PDA (PXA270) canon-a1100 Canon PowerShot A1100 IS cheetah Palm Tungsten|E aka. Cheetah PDA (OMAP310) collie Sharp SL-5500 (Collie) PDA (SA-1110) connex Gumstix Connex (PXA255) cubieboard cubietech cubieboard highbank Calxeda Highbank (ECX-1000) imx25-pdk ARM i.MX25 PDK board (ARM926) integratorcp ARM Integrator/CP (ARM926EJ-S) kzm ARM KZM Emulation Baseboard (ARM1136) lm3s6965evb Stellaris LM3S6965EVB lm3s811evb Stellaris LM3S811EVB mainstone Mainstone II (PXA27x) midway Calxeda Midway (ECX-2000) musicpal Marvell 88w8618 / MusicPal (ARM926EJ-S) n800 Nokia N800 tablet aka. RX-34 (OMAP2420) n810 Nokia N810 tablet aka. RX-44 (OMAP2420) netduino2 Netduino 2 Machine none empty machine nuri Samsung NURI board (Exynos4210) realview-eb ARM RealView Emulation Baseboard (ARM926EJ-S) realview-eb-mpcore ARM RealView Emulation Baseboard (ARM11MPCore) realview-pb-a8 ARM RealView Platform Baseboard for Cortex-A8 realview-pbx-a9 ARM RealView Platform Baseboard Explore for Cortex-A9 smdkc210 Samsung SMDKC210 board (Exynos4210) spitz Sharp SL-C3000 (Spitz) PDA (PXA270) sx1 Siemens SX1 (OMAP310) V2 sx1-v1 Siemens SX1 (OMAP310) V1 terrier Sharp SL-C3200 (Terrier) PDA (PXA270) tosa Sharp SL-6000 (Tosa) PDA (PXA255) verdex Gumstix Verdex (PXA270) versatileab ARM Versatile/AB (ARM926EJ-S) versatilepb ARM Versatile/PB (ARM926EJ-S) vexpress-a15 ARM Versatile Express for Cortex-A15 vexpress-a9 ARM Versatile Express for Cortex-A9 virt ARM Virtual Machine xilinx-zynq-a9 Xilinx Zynq Platform Baseboard for Cortex-A9 z2 Zipit Z2 (PXA27x) --