What are the advantages and disadvantages of running with or without libvirt?
I didn't see what are the main differences in http://libguestfs.org/guestfs.3.html#backend Specifically, I'm interested in what is faster (direct sounds faster to me), and if there are any major restrictions (networking?) Here's an example command we are running (sorry, Python'ish, but you'll get it): ['virt-sysprep', '--connect', 'qemu:///system', '-a', u'/zram/3.6/images/lago_basic_suite_3_6_engine_root.qcow2', '--selinux-relabel', '--hostname', 'lago_basic_suite_3_6_engine', '--root-password', 'password:123456', '--mkdir', '/root/.ssh', '--chmod', '0700:/root/.ssh', '--upload', '/zram/3.6/id_rsa.pub:/root/.ssh/authorized_keys', '--run-command', 'chown root.root /root/.ssh/authorized_keys', '--mkdir', '/etc/iscsi', '--chmod', '0755:/etc/iscsi', '--write', '/etc/iscsi/initiatorname.iscsi:InitiatorName=iqn.2014-07.org.lago:lago_basic_suite_3_6_engine', '--mkdir', '/etc/selinux', '--chmod', '0755:/etc/selinux', '--write', '/etc/selinux/config:SELINUX=enforcing\nSELINUXTYPE=targeted\n', '--mkdir', '/etc/sysconfig/network-scripts', '--chmod', '0755:/etc/sysconfig/network-scripts', '--write', '/etc/sysconfig/network-scripts/ifcfg-eth0:HWADDR="54:52:c0:a8:c8:03"\nBOOTPROTO="dhcp"\nTYPE="Ethernet"\nONBOOT="yes"\nNAME="eth0"'] TIA, Y.
There's two parts to this question. On Tue, Jan 12, 2016 at 05:26:10PM +0200, Yaniv Kaul wrote:
I didn't see what are the main differences in http://libguestfs.org/guestfs.3.html#backend
The basic concept of the backend is how do we run the libguestfs appliance (http://libguestfs.org/guestfs-internals.1.html#architecture). There are two ways we could run the appliance: either we run qemu directly as a forked subprocess of the program that is linked to libguestfs.so (LIBGUESTFS_BACKEND=direct); or we could run qemu via libvirt (LIBGUESTFS_BACKEND=libvirt). The two code paths are largely equivalent but also quite different in their implementation: https://github.com/libguestfs/libguestfs/blob/master/src/launch-direct.c https://github.com/libguestfs/libguestfs/blob/master/src/launch-libvirt.c
Specifically, I'm interested in what is faster (direct sounds faster to me), and if there are any major restrictions (networking?)
You would think that direct is faster, but I benchmarked this a while back and there's essentially no measurable difference. There are however big differences, and restrictions. Off the top of my head: - libvirt implements sVirt (http://selinuxproject.org/page/SVirt) so it's considerably more secure for examining untrusted disk images - libvirt allows hotplugging of drives, ie. you can call guestfs_add_drive_opts after launching the handle - there are some differences in how networking is done, although they are rather obscure (and networking is not enabled by default anyway, so it would not affect you unless the main program called guestfs_set_network (g, 1) before guestfs_launch) - libvirt is the supported method to run qemu in RHEL - libvirt is way more complex and fails much more frequently :-( It's unfortunately because of the last point that if libvirt fails to start the qemu appliance, we print an error message telling you to try with LIBGUESTFS_BACKEND=direct. Libvirt has different namespaces where you can run guests, eg. for Xen you have to use the xen:/// namespace, for qemu you have a choice (qemu:///session or qemu:///system). The libguestfs appliance mostly runs in the qemu:///session (non-root per-user) namespace, except when you run libguestfs as root when it has to run in the qemu:///system namespace, but that's just a bug in libvirt - https://bugzilla.redhat.com/show_bug.cgi?id=890291 . BUT! I said there were two parts to this question, and this is the second part ... There are several other places where libguestfs calls libvirt, and they are not related to how libguestfs runs its appliance, and that seems to be what you're hitting here:
Here's an example command we are running (sorry, Python'ish, but you'll get it):
['virt-sysprep', '--connect', 'qemu:///system', '-a', u'/zram/3.6/images/lago_basic_suite_3_6_engine_root.qcow2',
The virt-sysprep --connect option isn't anything to do with how libguestfs runs its appliance, and it's not even significant in the virt-sysprep command you've shown here. The virt-sysprep --connect option affects how virt-sysprep (and other tools) process the '-d' option. If you write: virt-sysprep -d some_libvirt_domain --mkdir /foo then virt-sysprep asks libvirt for the XML of `some_libvirt_domain' (basically equivalent to running `virsh dumpxml some_libvirt_domain') and it then parses the libvirt XML to find the disks of `some_libvirt_domain' so it knows where to create the /foo directory. https://github.com/libguestfs/libguestfs/blob/master/src/libvirt-domain.c#L4... Because libvirt has different namespaces for libvirt domain names, we allow you to select the namespace using the -c/--connect option, so you could for example do: virt-sysprep -c xen:/// -d some_xen_domain which is equivalent to doing `virsh -c xen:/// dumpxml some_xen_domain' and parsing that XML. This has nothing to do with how we run the libguestfs appliance. It's complicated ... and often we get confused too. HTH, Rich.
'--selinux-relabel', '--hostname', 'lago_basic_suite_3_6_engine', '--root-password', 'password:123456', '--mkdir', '/root/.ssh', '--chmod', '0700:/root/.ssh', '--upload', '/zram/3.6/id_rsa.pub:/root/.ssh/authorized_keys', '--run-command', 'chown root.root /root/.ssh/authorized_keys', '--mkdir', '/etc/iscsi', '--chmod', '0755:/etc/iscsi', '--write', '/etc/iscsi/initiatorname.iscsi:InitiatorName=iqn.2014-07.org.lago:lago_basic_suite_3_6_engine', '--mkdir', '/etc/selinux', '--chmod', '0755:/etc/selinux', '--write', '/etc/selinux/config:SELINUX=enforcing\nSELINUXTYPE=targeted\n', '--mkdir', '/etc/sysconfig/network-scripts', '--chmod', '0755:/etc/sysconfig/network-scripts', '--write', '/etc/sysconfig/network-scripts/ifcfg-eth0:HWADDR="54:52:c0:a8:c8:03"\nBOOTPROTO="dhcp"\nTYPE="Ethernet"\nONBOOT="yes"\nNAME="eth0"']
TIA, Y.
_______________________________________________ Libguestfs mailing list Libguestfs@redhat.com https://www.redhat.com/mailman/listinfo/libguestfs
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
Sorry for the spam, but I had to say it, great explanation :), thanks a lot! On 01/12 20:01, Richard W.M. Jones wrote:
There's two parts to this question.
On Tue, Jan 12, 2016 at 05:26:10PM +0200, Yaniv Kaul wrote:
I didn't see what are the main differences in http://libguestfs.org/guestfs.3.html#backend
The basic concept of the backend is how do we run the libguestfs appliance (http://libguestfs.org/guestfs-internals.1.html#architecture).
There are two ways we could run the appliance: either we run qemu directly as a forked subprocess of the program that is linked to libguestfs.so (LIBGUESTFS_BACKEND=direct); or we could run qemu via libvirt (LIBGUESTFS_BACKEND=libvirt). The two code paths are largely equivalent but also quite different in their implementation:
https://github.com/libguestfs/libguestfs/blob/master/src/launch-direct.c https://github.com/libguestfs/libguestfs/blob/master/src/launch-libvirt.c
Specifically, I'm interested in what is faster (direct sounds faster to me), and if there are any major restrictions (networking?)
You would think that direct is faster, but I benchmarked this a while back and there's essentially no measurable difference.
There are however big differences, and restrictions. Off the top of my head:
- libvirt implements sVirt (http://selinuxproject.org/page/SVirt) so it's considerably more secure for examining untrusted disk images
- libvirt allows hotplugging of drives, ie. you can call guestfs_add_drive_opts after launching the handle
- there are some differences in how networking is done, although they are rather obscure (and networking is not enabled by default anyway, so it would not affect you unless the main program called guestfs_set_network (g, 1) before guestfs_launch)
- libvirt is the supported method to run qemu in RHEL
- libvirt is way more complex and fails much more frequently :-(
It's unfortunately because of the last point that if libvirt fails to start the qemu appliance, we print an error message telling you to try with LIBGUESTFS_BACKEND=direct.
Libvirt has different namespaces where you can run guests, eg. for Xen you have to use the xen:/// namespace, for qemu you have a choice (qemu:///session or qemu:///system).
The libguestfs appliance mostly runs in the qemu:///session (non-root per-user) namespace, except when you run libguestfs as root when it has to run in the qemu:///system namespace, but that's just a bug in libvirt - https://bugzilla.redhat.com/show_bug.cgi?id=890291 .
BUT! I said there were two parts to this question, and this is the second part ...
There are several other places where libguestfs calls libvirt, and they are not related to how libguestfs runs its appliance, and that seems to be what you're hitting here:
Here's an example command we are running (sorry, Python'ish, but you'll get it):
['virt-sysprep', '--connect', 'qemu:///system', '-a', u'/zram/3.6/images/lago_basic_suite_3_6_engine_root.qcow2',
The virt-sysprep --connect option isn't anything to do with how libguestfs runs its appliance, and it's not even significant in the virt-sysprep command you've shown here.
The virt-sysprep --connect option affects how virt-sysprep (and other tools) process the '-d' option.
If you write:
virt-sysprep -d some_libvirt_domain --mkdir /foo
then virt-sysprep asks libvirt for the XML of `some_libvirt_domain' (basically equivalent to running `virsh dumpxml some_libvirt_domain') and it then parses the libvirt XML to find the disks of `some_libvirt_domain' so it knows where to create the /foo directory.
https://github.com/libguestfs/libguestfs/blob/master/src/libvirt-domain.c#L4...
Because libvirt has different namespaces for libvirt domain names, we allow you to select the namespace using the -c/--connect option, so you could for example do:
virt-sysprep -c xen:/// -d some_xen_domain
which is equivalent to doing `virsh -c xen:/// dumpxml some_xen_domain' and parsing that XML.
This has nothing to do with how we run the libguestfs appliance.
It's complicated ... and often we get confused too.
HTH,
Rich.
'--selinux-relabel', '--hostname', 'lago_basic_suite_3_6_engine', '--root-password', 'password:123456', '--mkdir', '/root/.ssh', '--chmod', '0700:/root/.ssh', '--upload', '/zram/3.6/id_rsa.pub:/root/.ssh/authorized_keys', '--run-command', 'chown root.root /root/.ssh/authorized_keys', '--mkdir', '/etc/iscsi', '--chmod', '0755:/etc/iscsi', '--write', '/etc/iscsi/initiatorname.iscsi:InitiatorName=iqn.2014-07.org.lago:lago_basic_suite_3_6_engine', '--mkdir', '/etc/selinux', '--chmod', '0755:/etc/selinux', '--write', '/etc/selinux/config:SELINUX=enforcing\nSELINUXTYPE=targeted\n', '--mkdir', '/etc/sysconfig/network-scripts', '--chmod', '0755:/etc/sysconfig/network-scripts', '--write', '/etc/sysconfig/network-scripts/ifcfg-eth0:HWADDR="54:52:c0:a8:c8:03"\nBOOTPROTO="dhcp"\nTYPE="Ethernet"\nONBOOT="yes"\nNAME="eth0"']
TIA, Y.
_______________________________________________ Libguestfs mailing list Libguestfs@redhat.com https://www.redhat.com/mailman/listinfo/libguestfs
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
_______________________________________________ Libguestfs mailing list Libguestfs@redhat.com https://www.redhat.com/mailman/listinfo/libguestfs
-- David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Tel.: +420 532 294 605 Email: dcaro@redhat.com IRC: dcaro|dcaroest@{freenode|oftc|redhat} Web: www.redhat.com RHT Global #: 82-62605
On Tue, Jan 12, 2016 at 10:01 PM, Richard W.M. Jones <rjones@redhat.com> wrote:
- libvirt is way more complex and fails much more frequently :-(
It's unfortunately because of the last point that if libvirt fails to start the qemu appliance, we print an error message telling you to try with LIBGUESTFS_BACKEND=direct.
That was one of the motivations. You were hit by it too apparently ( https://bugzilla.redhat.com/show_bug.cgi?id=1271183 ). I removed the connect line and set the environment variable and all looks good. I indeed don't see any much different performance difference. Thanks! Y.
On Tue, Jan 12, 2016 at 10:41:35PM +0200, Yaniv Kaul wrote:
On Tue, Jan 12, 2016 at 10:01 PM, Richard W.M. Jones <rjones@redhat.com> wrote:
- libvirt is way more complex and fails much more frequently :-(
It's unfortunately because of the last point that if libvirt fails to start the qemu appliance, we print an error message telling you to try with LIBGUESTFS_BACKEND=direct.
That was one of the motivations. You were hit by it too apparently ( https://bugzilla.redhat.com/show_bug.cgi?id=1271183 ).
That's one of the bugs. Coincidentally I conducted a test last night to quantify the incidence of libvirt errors, and I'm going to post the results shortly. Rich.
I removed the connect line and set the environment variable and all looks good. I indeed don't see any much different performance difference.
Thanks! Y.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
participants (3)
-
David Caro -
Richard W.M. Jones -
Yaniv Kaul