Re: [Libguestfs] parallelization
On Tue, Feb 26, 2013 at 02:56:53PM +0000, Richard W.M. Jones wrote:
On Tue, Feb 26, 2013 at 09:36:30AM -0500, Colin Walters wrote:
> So Richard recently blogged about running many libguestfs mounts in
> parallel.
>
> I actually want parallelization in the other way; I use guestmount to
> access a disk image, and my program is multithreaded. It does
> read/write and most especially calls to link() and fdatasync() from
> threads.
>
> However, this all ends up being serialized, because the guestfsd
> protocol is synchronous, and the daemon side is single-threaded.
> Related to this, the qemu instance is always booted with only one
> vCPU.
>
> Have there been any thoughts on fixing this?
Yes. However it requires changes to the virtio-serial protocol or
perhaps its entire replacement with VMware's vsock protocol (currently
being discussed in upstream qemu). Without this, it's pretty clunky
to open more than one connection from the library to the appliance.
That's if you need to open multiple connections though. Sounds like
in Colin's case he just has one multi-threaded app, in which case
all apps could just share one connection & have the RPC messages
multi-plexed as we do with libvirt. Of course still needs the daemon
to be multithreaded to get any benefit from that.
It would also need guestfsd to become multi-threaded. That's not
so
much of a problem (although it's certainly a bit of work), but solving
the protocol problem would have to be done first.
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|