running libguestfs as a service?
Hello, First off, many thanks for libguestfs... very useful! I'm trying to run libguestfs as an RPC service by using a python daemon with the python libguestfs bindings. With this service I'm "centrally" editing VM images, and often running many GuestFS() on the same image file in a short amount of time to edit the image's contents. The problem I'm running across is that each GuestFS() instance called from within the python daemon doesn't terminate itself (until a timeout much later?). I believe this sometimes causes my image edits to clobber each other and corrupt the image now and then. Each image edit usually amounts to: g = guestfs.GuestFS() g.add_drive(image_file) g.launch() g.mount("/dev/vda1", "/") g.write(file_name, file_contents) g.sync() g.umount_all() g.close() del g As I understand things, there is a reference count that should close "g" once the reference count = 0. However, I can't seem to make that happen inside of my python daemon which continues to run. Any clues on how I might accomplish this or what I'm doing wrong? Thanks
On Fri, Apr 29, 2011 at 01:38:19AM -0400, T Johnson wrote:
Hello,
First off, many thanks for libguestfs... very useful!
I'm trying to run libguestfs as an RPC service by using a python daemon with the python libguestfs bindings. With this service I'm "centrally" editing VM images, and often running many GuestFS() on the same image file in a short amount of time to edit the image's contents.
The problem I'm running across is that each GuestFS() instance called from within the python daemon doesn't terminate itself (until a timeout much later?). I believe this sometimes causes my image edits to clobber each other and corrupt the image now and then. Each image edit usually amounts to:
g = guestfs.GuestFS() g.add_drive(image_file) g.launch() g.mount("/dev/vda1", "/") g.write(file_name, file_contents) g.sync() g.umount_all() g.close() del g
As I understand things, there is a reference count that should close "g" once the reference count = 0. However, I can't seem to make that happen inside of my python daemon which continues to run.
First thing is to find out if 'del g' really closes the handle or not. It's my understanding (of Python bindings) that 'del g' *should* close the handle immediately, assuming you haven't hidden a reference to 'g' somewhere else in the program. Unfortunately we left out a trace call in guestfs_close so instead of using ordinary call tracing you have to do this using debugging: g.set_verbose (1) print >> sys.stderr, "I'm about to call del g ..." del g print >> sys.stderr, "I've called del g." This should produce a sequence of messages like: I'm about to call del g ... closing guestfs handle 0x1234 (state 2) # ie. guestfs_close called I've called del g. So we could get one of two results here: either guestfs_close is being called, or guestfs_close is not being called. If it's *not* being called then it's something to do with reference counting (or you're tucking a reference away somewhere else). I don't know a lot about how Python refcounting is supposed to work, but it's possible there is a mistake in the bindings, eg. not incrementing or decrementing a reference somewhere. If guestfs_close *is* being called then it's a more serious problem in the underlying library. guestfs_close is supposed to kill the qemu subprocess by sending it a SIGTERM, so really it should exit almost immediately, and we also waitpid for the process so really guestfs_close should not be returning before qemu has really exited. If this isn't happening, then getting 'strace' output would be the most useful thing. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://et.redhat.com/~rjones/virt-df/
On Fri, Apr 29, 2011 at 09:00:04AM +0100, Richard W.M. Jones wrote:
On Fri, Apr 29, 2011 at 01:38:19AM -0400, T Johnson wrote:
g.sync() g.umount_all() g.close()
I've just noticed you're calling g.close() there, so include that call in the bit between the print statements as well. Also since the code for the Python bindings has changed quite a bit recently, it would be useful to know which version you are running. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://et.redhat.com/~rjones/virt-df/
On Fri, Apr 29, 2011 at 5:01 AM, Richard W.M. Jones <rjones@redhat.com> wrote:
On Fri, Apr 29, 2011 at 09:00:04AM +0100, Richard W.M. Jones wrote:
On Fri, Apr 29, 2011 at 01:38:19AM -0400, T Johnson wrote:
g.sync() g.umount_all() g.close()
I've just noticed you're calling g.close() there, so include that call in the bit between the print statements as well.
Also since the code for the Python bindings has changed quite a bit recently, it would be useful to know which version you are running.
Hi Rich, Thanks for responding. I'm by no means a python expert so I could very well be doing something wrong, but I've been playing a bit with what you've suggested and noted a few interesting things. 1) As soon as I create 'g', something seems to take over stderr and stdout so I can no longer print debugging output like you suggested. 2) (I assume) because of #1, g.set_verbose(1) results in no output to stdout/stderr either, so it hasn't been helpful so far. 3) However, I then decided to try g.set_trace(1) and that does provide output. In the example I provided before I get this (editing config settings in /etc/sysconfig/network): libguestfs: trace: add_drive "/mnt/images/test_server.img" libguestfs: trace: add_drive = 0 libguestfs: trace: launch libguestfs: trace: launch = 0 libguestfs: trace: mount "/dev/vda1" "/" libguestfs: trace: mount = 0 libguestfs: trace: write "/etc/sysconfig/network" "NETWORKING=yes\x0aNETWORKING_IPV6=yes\x0aHOSTNAME=test9\x0aGATEWAY=10.0.0.1\x0a" libguestfs: trace: write = 0 I can't seem to get any other ouput.. but it is very interesting that once I kill the process I see the remainder of the calls failing from trace output: libguestfs: trace: umount_all libguestfs: trace: umount_all = -1 (error) libguestfs: trace: sync libguestfs: trace: sync = -1 (error) libguestfs: trace: umount_all libguestfs: trace: umount_all = -1 (error) libguestfs: trace: sync libguestfs: trace: sync = -1 (error) So, I guess that is what's failing.. but I haven't been able to determine why yet. This is something I've been running into on much older versions of libguestfs all the way up to now which I'm trying your recent "1.7.17-17.el6" RPMs for rhel6. Thanks, TJ
participants (2)
-
Richard W.M. Jones -
T Johnson