One possible plan for remote support
John, It's probably worth setting out from what the problem is / what we are trying to solve here. At the moment if you run a command like: virt-inspector -d Guest then libguestfs asks libvirt [-d option] for the XML corresponding to the libvirt guest called 'Guest'. This will contain disk sections like this: <disk type='block' device='disk'> <driver name='qemu' type='raw' cache='none'/> <source dev='/dev/vg_data/guest'/> <target dev='vda' bus='virtio'/> </disk> libguestfs then directly opens the local disk [/dev/vg_data/guest in the example above] and analyzes it. libvirt can also run remotely, eg: virsh -c qemu://remote/system list --all If you ran a command like: virt-inpector -c qemu://remote/system -d Guest then libguestfs connects to libvirtd on the remote server gets the XML, parses out the disks, tries to open them, and usually that's going to fail: files and devices can't be opened by name remotely. Well, unless you've got cluster LVM, but we'll assume that you don't. The idea of remote support is to make this work as seemlessly as possible. Now [since libguestfs 1.19.xx] that we have a libvirt backend to libguestfs [src/launch-libvirt.c], it could be made to work. What we do is we start the libguestfs appliance remotely. Obviously since it is now running remotely, it has access to the guest's disks directly. Now we have two problems: (a) How do we start the libguestfs appliance remotely? (b) How do we redirect the library <-> appliance serial connection back to the library (which is running on a different machine)? Ideally this would be forwarded over the libvirt connection. [http://libguestfs.org/guestfs.3.html#architecture] Both require some development work in libvirt & libguestfs. The background to problem (a) is that currently we run a helper program to create the appliance. This runs locally [src/appliance.c]. It needs to be changed to run remotely (to be more precise: the existing local code needs to be left in place, since it is very robust and required for the non-libvirt case, but we need additional code so it can be done remotely). libvirt already has a mechanism which can be used to run a process on the remote server which is responsible for creating the kernel/initrd, however it has some limitations and it currently only works for Xen: http://libvirt.org/formatdomain.html#elementsOSBootloader Problem (b) is really a shortcoming of libvirt. If you define a virtio-serial socket in libvirt [http://libvirt.org/formatdomain.html#elementCharChannel] then it's a bit stupid that these only work locally. It should be possible (Dan assures me "easy") to make these be forwarded over the libvirt secure connection. Then there's fixing up the libguestfs side of things so this all works in the remote case, ie. so that the correct XML is generated for (a), and that we are able to use the remoted serial socket created in (b). Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming blog: http://rwmj.wordpress.com Fedora now supports 80 OCaml packages (the OPEN alternative to F#) http://cocan.org/getting_started_with_ocaml_on_red_hat_and_fedora
"Richard W.M. Jones" <rjones@redhat.com> writes:
Problem (b) is really a shortcoming of libvirt. If you define a virtio-serial socket in libvirt [http://libvirt.org/formatdomain.html#elementCharChannel] then it's a bit stupid that these only work locally. It should be possible (Dan assures me "easy") to make these be forwarded over the libvirt secure connection.
Focusing just on this bit for now. Dan, do you have any thoughts on how to go about this? I've been digging through the libvirt code trying to understand the pieces. I see there's an existing virDomainOpenConsole API that knows (for the qemu driver) how to properly forward a console/serial/parallel device to a remote client, limited to ptys. It looks like we could pretty easily make this work with channels + pty. Also I see that presently libguestfs is using a unix socket and not pty. I'm guessing this is for performace reasons, but I'm not familiar enough with the underlying performance of each to know for sure. In any case, we'd also need to either (1) make guestfs use a pty source instead of a unix socket (easy), or (2) make virDomainOpenConsole work with unix sockets (not as easy). Thoughts? John
On Mon, Nov 12, 2012 at 05:02:08PM -0500, John Eckersberg wrote:
"Richard W.M. Jones" <rjones@redhat.com> writes:
Problem (b) is really a shortcoming of libvirt. If you define a virtio-serial socket in libvirt [http://libvirt.org/formatdomain.html#elementCharChannel] then it's a bit stupid that these only work locally. It should be possible (Dan assures me "easy") to make these be forwarded over the libvirt secure connection.
Focusing just on this bit for now.
Dan, do you have any thoughts on how to go about this? I've been digging through the libvirt code trying to understand the pieces. I see there's an existing virDomainOpenConsole API that knows (for the qemu driver) how to properly forward a console/serial/parallel device to a remote client, limited to ptys. It looks like we could pretty easily make this work with channels + pty. Also I see that presently libguestfs is using a unix socket and not pty. I'm guessing this is for performace reasons, but I'm not familiar enough with the underlying performance of each to know for sure. In any case, we'd also need to either (1) make guestfs use a pty source instead of a unix socket (easy), or (2) make virDomainOpenConsole work with unix sockets (not as easy).
I'm pretty sure a pty is never going to work. We need an 8-bit clean channel, and conversely don't need any features of a pty. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming blog: http://rwmj.wordpress.com Fedora now supports 80 OCaml packages (the OPEN alternative to F#) http://cocan.org/getting_started_with_ocaml_on_red_hat_and_fedora
On Mon, Nov 12, 2012 at 05:02:08PM -0500, John Eckersberg wrote:
"Richard W.M. Jones" <rjones@redhat.com> writes:
Problem (b) is really a shortcoming of libvirt. If you define a virtio-serial socket in libvirt [http://libvirt.org/formatdomain.html#elementCharChannel] then it's a bit stupid that these only work locally. It should be possible (Dan assures me "easy") to make these be forwarded over the libvirt secure connection.
Focusing just on this bit for now.
Dan, do you have any thoughts on how to go about this? I've been digging through the libvirt code trying to understand the pieces. I see there's an existing virDomainOpenConsole API that knows (for the qemu driver) how to properly forward a console/serial/parallel device to a remote client, limited to ptys. It looks like we could pretty easily make this work with channels + pty. Also I see that presently libguestfs is using a unix socket and not pty. I'm guessing this is for performace reasons, but I'm not familiar enough with the underlying performance of each to know for sure. In any case, we'd also need to either (1) make guestfs use a pty source instead of a unix socket (easy), or (2) make virDomainOpenConsole work with unix sockets (not as easy).
IME, ptys are a total PITA to work with, and would go for unix sockets every time. I don't think it would be very hard to make virDomainOpenConsole work with UNIX sockets actually. Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
participants (3)
-
Daniel P. Berrange -
John Eckersberg -
Richard W.M. Jones