On Fri, May 31, 2013 at 4:17 PM, Richard W.M. Jones <rjones@redhat.com> wrote:

On Fri, May 31, 2013 at 04:03:32PM +0800, Qiu Yu wrote:
> Actually I'm looking into an issue with OpenStack / Libvirt manipulating
> LVM as an image backend. When the logical volume is created, udev rules
> will set the ownership to root:disk. After libvirt actually starting an
> instance, device node ownership will change to qemu:qemu by libvirt. Then,
> if you stop an instance, the device node ownership will change back to
> root:root. Seems there's some inconsistency here.

This is actually a well-known bug in libvirt.

https://bugzilla.redhat.com/show_bug.cgi?id=796072
(marked as a duplicate of a private bug 547546 for some reason --
I will try to get this made non-private)

You can turn off libvirt's ownership management (set dynamic_ownership
= 0 in /etc/libvirt/qemu.conf) however this will cause other problems.
I don't think there is a proper solution to this yet.

Rich.

Richard,

Really thanks for the advice. But seems I'm hitting another permission issue regarding libguestfs and LVM image.

Following code snippet is stripped from OpenStack data injection logic. Running as non-root user (stack) will report "guestfs_launch failed" error, and the debug message shows it failed to open the LVM device node.

I've already set up the group permission ('disk' and 'fuse') for the 'stack' user. Could you shed some lights of what else should I look into? Thanks!

$ cat test.py

import guestfs

g = guestfs.GuestFS()

g.add_drive_opts("/dev/xenvg/123", format="qcow2")

g.launch()

$ id

uid=501(stack) gid=504(stack) groups=504(stack),6(disk),498(fuse)

$ ll /dev/mapper/xenvg-123

lrwxrwxrwx 1 root root 7 May 31 17:16 /dev/mapper/xenvg-123 -> ../dm-2

$ ll /dev/dm-2

brw-rw---- 1 root disk 253, 2 May 31 18:06 /dev/dm-2

$ python test.py

libguestfs: trace: add_drive_opts "/dev/xenvg/123" "format:qcow2"

libguestfs: trace: add_drive_opts = 0

libguestfs: trace: launch

libguestfs: trace: launch = -1 (error)

Traceback (most recent call last):

File "test.py", line 4, in <module>

g.launch()

File "/usr/lib/python2.6/site-packages/guestfs.py", line 244, in launch

return libguestfsmod.launch (self._o)

RuntimeError: guestfs_launch failed, see earlier error messages

libguestfs: trace: close

$ export LIBGUESTFS_DEBUG=1

$ python test.py

libguestfs: new guestfs handle 0x26a8250

libguestfs: trace: add_drive_opts "/dev/xenvg/123" "format:qcow2"

libguestfs: trace: add_drive_opts = 0

libguestfs: trace: launch

libguestfs: [00000ms] febootstrap-supermin-helper --verbose -f checksum '/usr/lib64/guestfs/supermin.d' x86_64

supermin helper [00000ms] whitelist = (not specified), host_cpu = x86_64, kernel = (null), initrd = (null), appliance = (null)

supermin helper [00000ms] inputs[0] = /usr/lib64/guestfs/supermin.d

checking modpath /lib/modules/2.6.32-358.6.2.ns8.el6.x86_64 is a directory

picked vmlinuz-2.6.32-358.6.2.ns8.el6.x86_64 because modpath /lib/modules/2.6.32-358.6.2.ns8.el6.x86_64 exists

supermin helper [00000ms] finished creating kernel

supermin helper [00000ms] visiting /usr/lib64/guestfs/supermin.d

supermin helper [00000ms] visiting /usr/lib64/guestfs/supermin.d/base.img

supermin helper [00000ms] visiting /usr/lib64/guestfs/supermin.d/daemon.img

supermin helper [00000ms] visiting /usr/lib64/guestfs/supermin.d/hostfiles

supermin helper [00015ms] visiting /usr/lib64/guestfs/supermin.d/init.img

supermin helper [00015ms] adding kernel modules

supermin helper [00080ms] finished creating appliance

libguestfs: [00083ms] begin testing qemu features

libguestfs: [00097ms] finished testing qemu features

libguestfs: accept_from_daemon: 0x26a8250 g->state = 1

[00098ms] /usr/libexec/qemu-kvm \

-global virtio-blk-pci.scsi=off \

-drive file=/dev/xenvg/123,cache=off,format=qcow2,if=virtio \

-nodefconfig \

-enable-kvm \

-nodefaults \

-nographic \

-m 500 \

-no-reboot \

-device virtio-serial \

-serial stdio \

-chardev socket,path=/tmp/libguestfssaaw6T/guestfsd.sock,id=channel0 \

-device virtserialport,chardev=channel0,name=org.libguestfs.channel.0 \

-kernel /var/tmp/.guestfs-501/kernel.30285 \

-initrd /var/tmp/.guestfs-501/initrd.30285 \

-append 'panic=1 console=ttyS0 udevtimeout=300 no_timer_check acpi=off printk.time=1 cgroup_disable=memory selinux=0 guestfs_verbose=1 TERM=screen-bce ' \

-drive file=/var/tmp/.guestfs-501/root.30285,snapshot=on,if=virtio,cache=unsafeqemu-kvm: -drive file=/dev/xenvg/123,cache=off,format=qcow2,if=virtio: could not open disk image /dev/xenvg/123: Invalid argument

libguestfs: child_cleanup: 0x26a8250: child process died

libguestfs: trace: launch = -1 (error)

Traceback (most recent call last):

File "test.py", line 4, in <module>

g.launch()

File "/usr/lib/python2.6/site-packages/guestfs.py", line 244, in launch

return libguestfsmod.launch (self._o)

RuntimeError: guestfs_launch failed, see earlier error messages

libguestfs: trace: close

libguestfs: closing guestfs handle 0x26a8250 (state 0)

Qiu Yu