Hello Experts, I need a help from you . I am using CentOS 6.4 and using guestfish to modify an ESX image. I am adding a disk, executing "run" command and mounting it. the "run" command is taking around 50 seconds . Is there any way to minimize it. If we can not minimize it then Can we create and keep the guestfish virtual shell open so that we can just add a disk , mount and modify the image in few seconds. Need your expert advice. Many Thanks in advance. Thanks & Best Regards, Priyanka.
On Fri, Sep 12, 2014 at 10:40:14PM +0530, Priyanka Ranjan wrote:
Hello Experts,
I need a help from you . I am using CentOS 6.4 and using guestfish to modify an ESX image.
I am adding a disk, executing "run" command and mounting it. the "run" command is taking around 50 seconds . Is there any way to minimize it.
So first of all, measure the performance. Run the simple baseline tests here: http://libguestfs.org/guestfs-performance.1.html#baseline-measurements If, after several runs, this stays at ~ 50 seconds, then it's likely because you are running this inside a virtual machine, on Amazon EC2, or on a machine which has virtualization disabled (eg in the BIOS settings). You can fix that by using baremetal (or maybe nested virt). You will find other tips in the guestfs-performance(1) man page that I linked to above.
If we can not minimize it then Can we create and keep the guestfish virtual shell open so that we can just add a disk , mount and modify the image in few seconds.
Yes, but unfortunately not on CentOS 6. On RHEL / CentOS 7 we support hotplugging in both libguestfs and guestfish, and that lets you do exactly what you describe above. http://libguestfs.org/guestfs.3.html#hotplugging Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
Thanks a lot Rechard for your inputs. On Fri, Sep 12, 2014 at 11:23 PM, Richard W.M. Jones <rjones@redhat.com> wrote:
On Fri, Sep 12, 2014 at 10:40:14PM +0530, Priyanka Ranjan wrote:
Hello Experts,
I need a help from you . I am using CentOS 6.4 and using guestfish to modify an ESX image.
I am adding a disk, executing "run" command and mounting it. the "run" command is taking around 50 seconds . Is there any way to minimize it.
So first of all, measure the performance. Run the simple baseline tests here:
http://libguestfs.org/guestfs-performance.1.html#baseline-measurements
If, after several runs, this stays at ~ 50 seconds, then it's likely because you are running this inside a virtual machine, on Amazon EC2, or on a machine which has virtualization disabled (eg in the BIOS settings). You can fix that by using baremetal (or maybe nested virt).
You will find other tips in the guestfs-performance(1) man page that I linked to above.
If we can not minimize it then Can we create and keep the guestfish virtual shell open so that we can just add a disk , mount and modify the image in few seconds.
Yes, but unfortunately not on CentOS 6. On RHEL / CentOS 7 we support hotplugging in both libguestfs and guestfish, and that lets you do exactly what you describe above.
http://libguestfs.org/guestfs.3.html#hotplugging
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
Hello Richard, I thought to move to CentOS 6.5 and capture the performance data to start with. unfortunately I cant move to CentOS 7. But now I am facing very strange issue the "libguestfs launch" is failing. I tried to debug but could not fix this issue. I am attaching all logs below. I am just trying to execute a command "time guestfish -a /dev/null run" mentioned in the performance page. I found that " sudo yum install libguestfs-tools" installed "libguestfs-tools-1.20.x" on CentOS 6.5. I have captured output of all the commands which I have executed. Am I missing something here? Thanks a lot for all your help in advance, [root@PriyankaVM ~]# cat /etc/*issue* CentOS release 6.5 (Final) Kernel \r on an \m [root@PriyankaVM ~]# sudo yum install libguestfs-tools [root@PriyankaVM ~]# rpm -qa|grep guest libguestfs-tools-1.20.11-2.el6.x86_64 libguestfs-tools-c-1.20.11-2.el6.x86_64 vmware-tools-plugins-guestInfo-9.4.0-1.el6.x86_64 libguestfs-1.20.11-2.el6.x86_64 vmware-tools-guestlib-9.4.0-1.el6.x86_64 [root@PriyankaVM ~]# time guestfish -a /dev/null run libguestfs: error: /usr/libexec/qemu-kvm exited with error status 1. To see full error messages you may need to enable debugging. See http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs real 0m26.704s user 0m10.964s sys 0m6.028s [root@PriyankaVM ~]# export LIBGUESTFS_DEBUG=1 [root@PriyankaVM ~]# export LIBGUESTFS_TRACE=1 [root@PriyankaVM ~]# libguestfs-test-tool ************************************************************ * IMPORTANT NOTICE * * When reporting bugs, include the COMPLETE, UNEDITED * output below in your bug report. * ************************************************************ libguestfs: trace: set_verbose true libguestfs: trace: set_verbose = 0 libguestfs: trace: set_verbose true libguestfs: trace: set_verbose = 0 LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1 PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin SELinux: Enforcing libguestfs: trace: add_drive "/tmp/libguestfs-test-tool-sda-k3It4V" "format:raw" libguestfs: trace: add_drive = 0 libguestfs: trace: version libguestfs: trace: version = <struct guestfs_version *> library version: 1.20.11rhel=6,release=2.el6 libguestfs: trace: get_append libguestfs: trace: get_append = "NULL" guestfs_get_append: (null) libguestfs: trace: get_attach_method libguestfs: trace: get_attach_method = "appliance" guestfs_get_attach_method: appliance libguestfs: trace: get_autosync libguestfs: trace: get_autosync = 1 guestfs_get_autosync: 1 libguestfs: trace: get_cachedir libguestfs: trace: get_cachedir = "/var/tmp" guestfs_get_cachedir: /var/tmp libguestfs: trace: get_direct libguestfs: trace: get_direct = 0 guestfs_get_direct: 0 libguestfs: trace: get_memsize libguestfs: trace: get_memsize = 500 guestfs_get_memsize: 500 libguestfs: trace: get_network libguestfs: trace: get_network = 0 guestfs_get_network: 0 libguestfs: trace: get_path libguestfs: trace: get_path = "/usr/lib64/guestfs" guestfs_get_path: /usr/lib64/guestfs libguestfs: trace: get_pgroup libguestfs: trace: get_pgroup = 0 guestfs_get_pgroup: 0 libguestfs: trace: get_qemu libguestfs: trace: get_qemu = "/usr/libexec/qemu-kvm" guestfs_get_qemu: /usr/libexec/qemu-kvm libguestfs: trace: get_recovery_proc libguestfs: trace: get_recovery_proc = 1 guestfs_get_recovery_proc: 1 libguestfs: trace: get_selinux libguestfs: trace: get_selinux = 0 guestfs_get_selinux: 0 libguestfs: trace: get_smp libguestfs: trace: get_smp = 1 guestfs_get_smp: 1 libguestfs: trace: get_tmpdir libguestfs: trace: get_tmpdir = "/tmp" guestfs_get_tmpdir: /tmp libguestfs: trace: get_trace libguestfs: trace: get_trace = 1 guestfs_get_trace: 1 libguestfs: trace: get_verbose libguestfs: trace: get_verbose = 1 guestfs_get_verbose: 1 host_cpu: x86_64 Launching appliance, timeout set to 600 seconds. libguestfs: trace: launch libguestfs: trace: get_tmpdir libguestfs: trace: get_tmpdir = "/tmp" libguestfs: launch: attach-method=appliance libguestfs: launch: tmpdir=/tmp/libguestfsUpuW0B libguestfs: launch: umask=0022 libguestfs: launch: euid=0 libguestfs: command: run: febootstrap-supermin-helper libguestfs: command: run: \ --verbose libguestfs: command: run: \ -f checksum libguestfs: command: run: \ /usr/lib64/guestfs/supermin.d libguestfs: command: run: \ x86_64 supermin helper [00000ms] whitelist = (not specified), host_cpu = x86_64, kernel = (null), initrd = (null), appliance = (null) supermin helper [00000ms] inputs[0] = /usr/lib64/guestfs/supermin.d checking modpath /lib/modules/2.6.32-431.11.2.el6.x86_64 is a directory picked vmlinuz-2.6.32-431.11.2.el6.x86_64 because modpath /lib/modules/2.6.32-431.11.2.el6.x86_64 exists supermin helper [00002ms] finished creating kernel supermin helper [00002ms] visiting /usr/lib64/guestfs/supermin.d supermin helper [00003ms] visiting /usr/lib64/guestfs/supermin.d/base.img supermin helper [00003ms] visiting /usr/lib64/guestfs/supermin.d/daemon.img supermin helper [00003ms] visiting /usr/lib64/guestfs/supermin.d/hostfiles supermin helper [00111ms] visiting /usr/lib64/guestfs/supermin.d/init.img supermin helper [00113ms] visiting /usr/lib64/guestfs/supermin.d/udev-rules.img supermin helper [00113ms] adding kernel modules supermin helper [00278ms] finished creating appliance libguestfs: checksum of existing appliance: b81afcfba02c9dc60949034595a88ebf29731923a9ef273af22835a3f4798d2e libguestfs: trace: get_cachedir libguestfs: trace: get_cachedir = "/var/tmp" libguestfs: [00293ms] begin testing qemu features libguestfs: command: run: /usr/libexec/qemu-kvm libguestfs: command: run: \ -nographic libguestfs: command: run: \ -help libguestfs: command: run: /usr/libexec/qemu-kvm libguestfs: command: run: \ -nographic libguestfs: command: run: \ -version libguestfs: qemu version 0.12 libguestfs: command: run: /usr/libexec/qemu-kvm libguestfs: command: run: \ -nographic libguestfs: command: run: \ -machine accel=kvm:tcg libguestfs: command: run: \ -device ? libguestfs: error: /usr/libexec/qemu-kvm exited with error status 1, see debug messages above libguestfs: trace: launch = -1 (error) libguestfs-test-tool: failed to launch appliance libguestfs: trace: close libguestfs: closing guestfs handle 0x1eacd70 (state 0) libguestfs: command: run: rm libguestfs: command: run: \ -rf /tmp/libguestfsUpuW0B [root@PriyankaVM ~]# time guestfish -a /dev/null run libguestfs: trace: set_verbose true libguestfs: trace: set_verbose = 0 libguestfs: create: flags = 0, handle = 0x221f2b0 libguestfs: trace: set_pgroup true libguestfs: trace: set_pgroup = 0 libguestfs: trace: add_drive "/dev/null" libguestfs: trace: get_tmpdir libguestfs: trace: get_tmpdir = "/tmp" libguestfs: trace: add_drive = 0 libguestfs: trace: launch libguestfs: launch: attach-method=appliance libguestfs: launch: tmpdir=/tmp/libguestfsANZENV libguestfs: launch: umask=0022 libguestfs: launch: euid=0 libguestfs: command: run: febootstrap-supermin-helper libguestfs: command: run: \ --verbose libguestfs: command: run: \ -f checksum libguestfs: command: run: \ /usr/lib64/guestfs/supermin.d libguestfs: command: run: \ x86_64 supermin helper [00000ms] whitelist = (not specified), host_cpu = x86_64, kernel = (null), initrd = (null), appliance = (null) supermin helper [00000ms] inputs[0] = /usr/lib64/guestfs/supermin.d checking modpath /lib/modules/2.6.32-431.11.2.el6.x86_64 is a directory picked vmlinuz-2.6.32-431.11.2.el6.x86_64 because modpath /lib/modules/2.6.32-431.11.2.el6.x86_64 exists supermin helper [00002ms] finished creating kernel supermin helper [00002ms] visiting /usr/lib64/guestfs/supermin.d supermin helper [00002ms] visiting /usr/lib64/guestfs/supermin.d/base.img supermin helper [00002ms] visiting /usr/lib64/guestfs/supermin.d/daemon.img supermin helper [00002ms] visiting /usr/lib64/guestfs/supermin.d/hostfiles supermin helper [00107ms] visiting /usr/lib64/guestfs/supermin.d/init.img supermin helper [00114ms] visiting /usr/lib64/guestfs/supermin.d/udev-rules.img supermin helper [00114ms] adding kernel modules supermin helper [00192ms] finished creating appliance libguestfs: checksum of existing appliance: b81afcfba02c9dc60949034595a88ebf29731923a9ef273af22835a3f4798d2e libguestfs: trace: get_cachedir libguestfs: trace: get_cachedir = "/var/tmp" libguestfs: [00198ms] begin testing qemu features libguestfs: command: run: /usr/libexec/qemu-kvm libguestfs: command: run: \ -nographic libguestfs: command: run: \ -help libguestfs: command: run: /usr/libexec/qemu-kvm libguestfs: command: run: \ -nographic libguestfs: command: run: \ -version libguestfs: qemu version 0.12 libguestfs: command: run: /usr/libexec/qemu-kvm libguestfs: command: run: \ -nographic libguestfs: command: run: \ -machine accel=kvm:tcg libguestfs: command: run: \ -device ? libguestfs: error: /usr/libexec/qemu-kvm exited with error status 1, see debug messages above libguestfs: trace: launch = -1 (error) libguestfs: trace: close libguestfs: closing guestfs handle 0x221f2b0 (state 0) libguestfs: command: run: rm libguestfs: command: run: \ -rf /tmp/libguestfsANZENV real 0m0.341s user 0m0.086s sys 0m0.222s On Sat, Sep 13, 2014 at 7:47 PM, Priyanka Ranjan <priyanka3rdfeb@gmail.com> wrote:
Thanks a lot Rechard for your inputs.
On Fri, Sep 12, 2014 at 11:23 PM, Richard W.M. Jones <rjones@redhat.com> wrote:
On Fri, Sep 12, 2014 at 10:40:14PM +0530, Priyanka Ranjan wrote:
Hello Experts,
I need a help from you . I am using CentOS 6.4 and using guestfish to modify an ESX image.
I am adding a disk, executing "run" command and mounting it. the "run" command is taking around 50 seconds . Is there any way to minimize it.
So first of all, measure the performance. Run the simple baseline tests here:
http://libguestfs.org/guestfs-performance.1.html#baseline-measurements
If, after several runs, this stays at ~ 50 seconds, then it's likely because you are running this inside a virtual machine, on Amazon EC2, or on a machine which has virtualization disabled (eg in the BIOS settings). You can fix that by using baremetal (or maybe nested virt).
You will find other tips in the guestfs-performance(1) man page that I linked to above.
If we can not minimize it then Can we create and keep the guestfish virtual shell open so that we can just add a disk , mount and modify the image in few seconds.
Yes, but unfortunately not on CentOS 6. On RHEL / CentOS 7 we support hotplugging in both libguestfs and guestfish, and that lets you do exactly what you describe above.
http://libguestfs.org/guestfs.3.html#hotplugging
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
On Wed, Sep 17, 2014 at 07:59:03PM +0530, Priyanka Ranjan wrote:
libguestfs: command: run: /usr/libexec/qemu-kvm libguestfs: command: run: \ -nographic libguestfs: command: run: \ -machine accel=kvm:tcg libguestfs: command: run: \ -device ? libguestfs: error: /usr/libexec/qemu-kvm exited with error status 1, see debug messages above libguestfs: trace: launch = -1 (error)
It looks as if qemu is broken on your machine. What's the output of: /usr/libexec/qemu-kvm -nographic -machine accel=kvm:tcg -device \? Note that you don't need to run libguestfs tools as root. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW
Many Thanks Richard for your reply. # /usr/libexec/qemu-kvm -nographic -machine accel=kvm:tcg -device \? open /dev/kvm: No such file or directory failed to initialize KVM: Operation not permitted Back to tcg accelerator. Could not allocate dynamic translator buffer I tried executing "setsebool -P virt_use_execmem=on" also after looking at "http://libguestfs.org/guestfs-faq.1.html" but that also did not help. Please guide me how to fix this issue and proceed further. Thanks again for your help. On Wed, Sep 17, 2014 at 8:14 PM, Richard W.M. Jones <rjones@redhat.com> wrote:
On Wed, Sep 17, 2014 at 07:59:03PM +0530, Priyanka Ranjan wrote:
libguestfs: command: run: /usr/libexec/qemu-kvm libguestfs: command: run: \ -nographic libguestfs: command: run: \ -machine accel=kvm:tcg libguestfs: command: run: \ -device ? libguestfs: error: /usr/libexec/qemu-kvm exited with error status 1, see debug messages above libguestfs: trace: launch = -1 (error)
It looks as if qemu is broken on your machine. What's the output of:
/usr/libexec/qemu-kvm -nographic -machine accel=kvm:tcg -device \?
Note that you don't need to run libguestfs tools as root.
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW
On Wed, Sep 17, 2014 at 08:52:57PM +0530, Priyanka Ranjan wrote:
Many Thanks Richard for your reply.
# /usr/libexec/qemu-kvm -nographic -machine accel=kvm:tcg -device \?
open /dev/kvm: No such file or directory failed to initialize KVM: Operation not permitted Back to tcg accelerator. Could not allocate dynamic translator buffer
I tried executing "setsebool -P virt_use_execmem=on" also after looking at "http://libguestfs.org/guestfs-faq.1.html" but that also did not help. Please guide me how to fix this issue and proceed further.
That command is the way you're supposed to fix this problem. If it doesn't work then I don't know -- perhaps ask on a CentOS or qemu list, or open a bug against CentOS? Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW
Thanks Richard, I have asked CentOS and qemu mailing list about this issue and waiting for their reply. It is really strange that on CentOS 6.4 VM, atleast qemu was wokring but in higher version "6.5" it is not working. On Wed, Sep 17, 2014 at 8:57 PM, Richard W.M. Jones <rjones@redhat.com> wrote:
On Wed, Sep 17, 2014 at 08:52:57PM +0530, Priyanka Ranjan wrote:
Many Thanks Richard for your reply.
# /usr/libexec/qemu-kvm -nographic -machine accel=kvm:tcg -device \?
open /dev/kvm: No such file or directory failed to initialize KVM: Operation not permitted Back to tcg accelerator. Could not allocate dynamic translator buffer
I tried executing "setsebool -P virt_use_execmem=on" also after looking at "http://libguestfs.org/guestfs-faq.1.html" but that also did not help. Please guide me how to fix this issue and proceed further.
That command is the way you're supposed to fix this problem. If it doesn't work then I don't know -- perhaps ask on a CentOS or qemu list, or open a bug against CentOS?
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW
Hello Richard, One more question, you mentioned that "hot plugging" is supported from *CentOS 7 *and gave me a link. That link says "In libguestfs ≥ 1.20, you may add drives and remove after calling "guestfs_launch" <http://libguestfs.org/guestfs.3.html#guestfs_launch>. There are some restrictions, see below. This is called*hotplugging*." When I executed "sudo yum install libguestfs-tools" on CentOS 6.5, it installed libguestfs 1.20.11. So if we fix the current qemu issue then can we use "hotplugging" in CentOS 6.5? Do you see some other issues? Do you recommend CentOS 7 only for the "hotplugging"? Many Thanks for all your help. On Thu, Sep 18, 2014 at 5:51 AM, Priyanka Ranjan <priyanka3rdfeb@gmail.com> wrote:
Thanks Richard, I have asked CentOS and qemu mailing list about this issue and waiting for their reply.
It is really strange that on CentOS 6.4 VM, atleast qemu was wokring but in higher version "6.5" it is not working.
On Wed, Sep 17, 2014 at 8:57 PM, Richard W.M. Jones <rjones@redhat.com> wrote:
On Wed, Sep 17, 2014 at 08:52:57PM +0530, Priyanka Ranjan wrote:
Many Thanks Richard for your reply.
# /usr/libexec/qemu-kvm -nographic -machine accel=kvm:tcg -device \?
open /dev/kvm: No such file or directory failed to initialize KVM: Operation not permitted Back to tcg accelerator. Could not allocate dynamic translator buffer
I tried executing "setsebool -P virt_use_execmem=on" also after looking at "http://libguestfs.org/guestfs-faq.1.html" but that also did not help. Please guide me how to fix this issue and proceed further.
That command is the way you're supposed to fix this problem. If it doesn't work then I don't know -- perhaps ask on a CentOS or qemu list, or open a bug against CentOS?
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW
On Thu, Sep 18, 2014 at 06:53:09AM +0530, Priyanka Ranjan wrote:
Hello Richard, One more question, you mentioned that "hot plugging" is supported from *CentOS 7 *and gave me a link. That link says
"In libguestfs ≥ 1.20, you may add drives and remove after calling "guestfs_launch" <http://libguestfs.org/guestfs.3.html#guestfs_launch>. There are some restrictions, see below. This is called*hotplugging*."
When I executed "sudo yum install libguestfs-tools" on CentOS 6.5, it installed libguestfs 1.20.11. So if we fix the current qemu issue then can we use "hotplugging" in CentOS 6.5? Do you see some other issues? Do you recommend CentOS 7 only for the "hotplugging"?
Hotplugging requires libvirt, and that only works in CentOS >= 7. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
ok , Thanks Richard. On Thu, Sep 18, 2014 at 1:14 PM, Richard W.M. Jones <rjones@redhat.com> wrote:
On Thu, Sep 18, 2014 at 06:53:09AM +0530, Priyanka Ranjan wrote:
Hello Richard, One more question, you mentioned that "hot plugging" is supported from *CentOS 7 *and gave me a link. That link says
"In libguestfs ≥ 1.20, you may add drives and remove after calling "guestfs_launch" <http://libguestfs.org/guestfs.3.html#guestfs_launch>. There are some restrictions, see below. This is called*hotplugging*."
When I executed "sudo yum install libguestfs-tools" on CentOS 6.5, it installed libguestfs 1.20.11. So if we fix the current qemu issue then can we use "hotplugging" in CentOS 6.5? Do you see some other issues? Do you recommend CentOS 7 only for the "hotplugging"?
Hotplugging requires libvirt, and that only works in CentOS >= 7.
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
participants (2)
-
Priyanka Ranjan -
Richard W.M. Jones