guestfsd crashes when the handle is closed
Hello, I launched guestfsd in my guest with this command : guestfsd* -rv*. When i close the handle the deamon crashes. This is the output i got (Deamon side) : guestfsd: main_loop : new request, len 0x28 fsync /dev/sda /dev/sr0 : No medium found guestfsd : main_loop : proc 282 (internal_autosynx) took 0.19 seconds *read: unexpected end of file on fd 3 * I got the same error with both ubuntu12.04 and fedora 20 guests. I know that in ubuntu the guestfsd version is quite but fedora 20 has 1.26 libguestfs version i think. Any help would be match appreciated ! Thanks ! *Note : *BTW, although i think it's guest side problem, this are details about the host : Fedora 20 , guestfish 1.26.1fedora=20,release=2.fc20,libvirt --
On Wed, May 14, 2014 at 06:07:37PM +0200, Safa Rekik wrote:
Hello,
I launched guestfsd in my guest with this command : guestfsd* -rv*. When i close the handle the deamon crashes.
This is the output i got (Deamon side) : guestfsd: main_loop : new request, len 0x28 fsync /dev/sda /dev/sr0 : No medium found guestfsd : main_loop : proc 282 (internal_autosynx) took 0.19 seconds *read: unexpected end of file on fd 3 *
I got the same error with both ubuntu12.04 and fedora 20 guests. I know that in ubuntu the guestfsd version is quite but fedora 20 has 1.26 libguestfs version i think.
Can you attach a debugger to it and find out where it crashes or exits. Something like: # gdb /usr/sbin/guestfsd `pidof guestfsd` (gdb) break exit (gdb) cont ... when it exits ... (gdb) bt As discussed on IRC, guestfsd is single-threaded at the moment, but one of the jobs for 1.28 is to add support for multiple parallel connections. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v
Hello, So in ubuntu1204 guest This is what i got : *(gdb)* bt full * #0* 0x005a9fc0 in exit () from /lib/i386-linux-gnu/libc.so.6 No symbol table info available. * #1* 0x0805f7dd in ?? () No symbol table info available. * #2* 0x0804aedf in ?? () No symbol table info available. *#3* 0x005904d3 in __libc_start_main () from /lib/i386-linux-gnu/libc.so.6 No symbol table info available. *#4* 0x0804b025 in ?? () No symbol table info available. i also tried to print information about each frame, i found unknown addresses in frame 4 and 3, here are the details : ======================================== --frame 4 : * (gdb) info frame* Stack level 4, frame at 0x0: eip = 0x804b025; saved eip 0x804b025 Outermost frame: outermost caller of frame at 0xbfd55210 Arglist at* unknown address.* Locals at* unknown address*, Previous frame's sp in esp *--*frame 3: * (gdb) info frame * Stack level 3, frame at 0xbfd55210: eip = 0x5904d3 in __libc_start_main; saved eip 0x804b025 called by frame at 0x0, caller of frame at 0xbfd551a0 Arglist at *unknown address*. Locals at *unknown address*, Previous frame's sp is 0xbfd55210 Saved registers: ebx at 0xbfd551fc, ebp at 0xbfd55208, esi at 0xbfd55200, edi at 0xbfd55204, eip at 0xbfd5520c ========================================= For fedora 20 guest, this is the back trace output : ===================================== *(gdb) bt full* #0 0xb73e2040 in exit () from /lib/libc.so.6 No symbol table info available. #1 0xb77263a8 in main_loop () No symbol table info available. #2 0xb76fe190 in main () No symbol table info available. *(gdb) frame 0* #0 0xb73e2040 in exit () from /lib/libc.so.6 (gdb) info frame Stack level 0, frame at 0xbfb022a0: eip = 0xb73e2040 in exit; saved eip 0xb77263a8 called by frame at 0xbfb02350 Arglist at 0xbfb02298, args: Locals at 0xbfb02298, Previous frame's sp is 0xbfb022a0 Saved registers: eip at 0xbfb0229c *(gdb) frame 1* #1 0xb77263a8 in main_loop () (gdb) info frame Stack level 1, frame at 0xbfb02350: eip = 0xb77263a8 in main_loop; saved eip 0xb76fe190 called by frame at 0xbfb025f0, caller of frame at 0xbfb022a0 Arglist at 0xbfb022ac, args: Locals at 0xbfb022ac, Previous frame's sp is 0xbfb02350 Saved registers: ebx at 0xbfb0233c, ebp at 0xbfb02348, esi at 0xbfb02340, edi at 0xbfb02344, eip at 0xbfb0234c *(gdb) frame 2* #2 0xb76fe190 in main () (gdb) info frame Stack level 2, frame at 0xbfb025f0: eip = 0xb76fe190 in main; saved eip 0xb73c9b83 caller of frame at 0xbfb02350 Arglist at 0xbfb025e8, args: Locals at 0xbfb025e8, Previous frame's sp is 0xbfb025f0 Saved registers: ebx at 0xbfb025dc, ebp at 0xbfb025e8, esi at 0xbfb025e0, edi at 0xbfb025e4, eip at 0xbfb025ec ============================================= As for parallel connections, we are looking forward for 1.28 :-). Meanwhile we are trying to test things with a single handle. Thanks ! 2014-05-15 0:16 GMT+02:00 Richard W.M. Jones <rjones@redhat.com>:
On Wed, May 14, 2014 at 06:07:37PM +0200, Safa Rekik wrote:
Hello,
I launched guestfsd in my guest with this command : guestfsd* -rv*. When i close the handle the deamon crashes.
This is the output i got (Deamon side) : guestfsd: main_loop : new request, len 0x28 fsync /dev/sda /dev/sr0 : No medium found guestfsd : main_loop : proc 282 (internal_autosynx) took 0.19 seconds *read: unexpected end of file on fd 3 *
I got the same error with both ubuntu12.04 and fedora 20 guests. I know that in ubuntu the guestfsd version is quite but fedora 20 has 1.26 libguestfs version i think.
Can you attach a debugger to it and find out where it crashes or exits. Something like:
# gdb /usr/sbin/guestfsd `pidof guestfsd` (gdb) break exit (gdb) cont ... when it exits ... (gdb) bt
As discussed on IRC, guestfsd is single-threaded at the moment, but one of the jobs for 1.28 is to add support for multiple parallel connections.
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v
-- Safa REKIK
On Thu, May 15, 2014 at 01:13:47PM +0200, Safa Rekik wrote:
For fedora 20 guest, this is the back trace output : ===================================== *(gdb) bt full* #0 0xb73e2040 in exit () from /lib/libc.so.6 No symbol table info available. #1 0xb77263a8 in main_loop () No symbol table info available. #2 0xb76fe190 in main () No symbol table info available.
You should install the 'libguestfs-debuginfo' package which will produce detailed line numbers. It's not possible to tell what is happening from this output. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/
Hello, This is the ouotput for fedora guest. ========================================== *#0 0xb73b7040 in exit () from /lib/libc.so.6* No symbol table info available. *#1 0xb76fb3a8 in main_loop (_sock=_sock@entry=3) at proto.c:112* xdr = {x_op = XDR_DECODE, x_ops = 0xb753db00 <xdrmem_ops>, x_public = 0x0, x_private = 0xbba929d8 ".0", x_base = 0xbba929b0 "", x_handy = 0} buf = <optimized out> lenbuf = "\000\000\000(" len = 40 hdr = {prog = 536933877, vers = 4, proc = GUESTFS_PROC_INTERNAL_AUTOSYNC, direction = GUESTFS_DIRECTION_CALL, serial = 1192960, progress_hint = 0, optargs_bitmask = 0, status = GUESTFS_STATUS_OK} *#2 0xb76d3190 in main (argc=2, argv=0xbfedcf34) at guestfsd.c:342* options = <synthetic pointer> long_options = {{name = 0xb77448b2 "help", has_arg = 0, flag = 0x0, val = 63}, {name = 0xb7745fb9 "verbose", has_arg = 0, flag = 0x0, val = 118}, {name = 0x0, has_arg = 0, flag = 0x0, val = 0}} c = <optimized out> statbuf = {st_dev = 64769, __pad1 = 0, __st_ino = 2, st_mode = 16877, st_nlink = 18, st_uid = 0, st_gid = 0, st_rdev = 0, __pad2 = 0, st_size = 4096, st_blksize = 4096, st_blocks = 8, st_atim = { tv_sec = 1400158599, tv_nsec = 226617083}, st_mtim = { tv_sec = 1400154384, tv_nsec = 581000000}, st_ctim = { tv_sec = 1400154389, tv_nsec = 390000000}, st_ino = 2} sa = {__sigaction_handler = {sa_handler = 0x1, sa_sigaction = 0x1}, sa_mask = {__val = {0 <repeats 32 times>}}, sa_flags = 0, sa_restorer = 0x0} channel = <optimized out> p = <optimized out> sock = 3 lenbuf = "\365\365", <incomplete sequence \365> xdr = {x_op = XDR_ENCODE, x_ops = 0xb753db00 <xdrmem_ops>, x_public = 0xb76acc46 <_dl_lookup_symbol_x+214> "\203", <incomplete sequence \370>, x_private = 0xbfedcd80 "_IMAGE=/vmlinuz-3.11.10-301.fc20.i686 root=/dev/mapper/fedora-root ro rd.lvm.lv=fedora/swap vconsole.font=latarcyrheb-sun16 rd.lvm.lv=fedora/root rhgb quiet LANG=fr_FR.UTF-8\n", x_base = 0xbfedcd7c "\365\365_\365_IMAGE=/vmlinuz-3.11.10-301.fc20.i686 root=/dev/mapper/fedora-root ro rd.lvm.lv=fedora/swap vconsole.font=latarcyrheb-sun16 rd.lvm.lv=fedora/root rhgb quiet LANG=fr_FR.UTF-8\n", x_handy = 0} len = 4126498805 =============================================== 2014-05-15 13:20 GMT+02:00 Richard W.M. Jones <rjones@redhat.com>:
On Thu, May 15, 2014 at 01:13:47PM +0200, Safa Rekik wrote:
For fedora 20 guest, this is the back trace output : ===================================== *(gdb) bt full* #0 0xb73e2040 in exit () from /lib/libc.so.6 No symbol table info available. #1 0xb77263a8 in main_loop () No symbol table info available. #2 0xb76fe190 in main () No symbol table info available.
You should install the 'libguestfs-debuginfo' package which will produce detailed line numbers. It's not possible to tell what is happening from this output.
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/
-- Safa REKIK
On Thu, May 15, 2014 at 03:23:53PM +0200, Safa Rekik wrote:
Hello,
This is the ouotput for fedora guest. ========================================== *#0 0xb73b7040 in exit () from /lib/libc.so.6* No symbol table info available.
*#1 0xb76fb3a8 in main_loop (_sock=_sock@entry=3) at proto.c:112*
Definitely not what I was expecting: if (len > GUESTFS_MESSAGE_MAX) { fprintf (stderr, "guestfsd: incoming message is too long (%u bytes)\n", len); exit (EXIT_FAILURE); } Are you using two radically different versions of libguestfs on the client and server? (It would have to be really different versions, since it is a long time since we broke backwards compatibility) Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v
Hello, In the client side i have guestfish 1.26.1. For the server, guestfsd was installed with libguestfs-live-service 1.26.1 too. So i don't think this is the problem. BTW, isn't ther a libguestfs-debuginfo packages for debian/ubuntu. Because this is what i am looking for actualy. In fedora , i the daemon works fine when i use it via the libguestfs-live-service .(launch it with *systemctl start guestfsd.service*. ). Actually , i am just testing the guestfsd 1.26 behaviour in fedora to see whether it would be worth compiling it in ubuntu/debian (i think there is no libguestfs 1.26 for debian/ubuntu yet) . 2014-05-15 15:48 GMT+02:00 Richard W.M. Jones <rjones@redhat.com>:
On Thu, May 15, 2014 at 03:23:53PM +0200, Safa Rekik wrote:
Hello,
This is the ouotput for fedora guest. ========================================== *#0 0xb73b7040 in exit () from /lib/libc.so.6* No symbol table info available.
*#1 0xb76fb3a8 in main_loop (_sock=_sock@entry=3) at proto.c:112*
Definitely not what I was expecting:
if (len > GUESTFS_MESSAGE_MAX) { fprintf (stderr, "guestfsd: incoming message is too long (%u bytes)\n", len); exit (EXIT_FAILURE); }
Are you using two radically different versions of libguestfs on the client and server? (It would have to be really different versions, since it is a long time since we broke backwards compatibility)
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v
-- Safa REKIK
On Thu, May 15, 2014 at 04:47:36PM +0200, Safa Rekik wrote:
Hello,
In the client side i have guestfish 1.26.1. For the server, guestfsd was installed with libguestfs-live-service 1.26.1 too. So i don't think this is the problem.
BTW, isn't ther a libguestfs-debuginfo packages for debian/ubuntu. Because this is what i am looking for actualy. In fedora , i the daemon works fine when i use it via the libguestfs-live-service .(launch it with *systemctl start guestfsd.service*. ). Actually , i am just testing the guestfsd 1.26 behaviour in fedora to see whether it would be worth compiling it in ubuntu/debian (i think there is no libguestfs 1.26 for debian/ubuntu yet) .
Right right ... I think what happens is that systemd restarts guestfsd. Basically this is all going to change / be fixed. guestfsd is only required to run once when used in the libguestfs appliance. However when it is changed to handle parallel connections, it will allow reconnection properly. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v
participants (2)
-
Richard W.M. Jones -
Safa Rekik