[libguestfs] Problem using virt-sysprep with RBD images
Hello, I'm having a problem when trying to use virt-sysprep against vms that have rbd disk images. When I run virt-sysprep I get the following error:
root@kitt:~/libguestfs-1.22.4# virt-sysprep -d server-clone-test --firstboot firstboot.sh Examining the guest ... Fatal error: exception Guestfs.Error("rbd: image name must begin with a '/'")
My Disk definition is as follows:
<disk type='network' device='disk'> <driver name='qemu'/> <source protocol='rbd' name='libvirt-pool/server-clone-test'> <host name='192.168.0.40' port='6789'/> <host name='192.168.0.35' port='6789'/> <host name='192.168.0.2' port='6789'/> </source> <target dev='hdb' bus='ide'/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk>
I can resolve this error by changing the name of the libvirt-pool:
<source protocol='rbd' name='/libvirt-pool/server-clone-test'>
root@kitt:~/libguestfs-1.22.4# virsh start server-clone-test error: Failed to start domain server-clone-test error: internal error: process exited while connecting to monitor: W: kvm binary is deprecated, please use qemu-system-x86_64 instead char device redirected to /dev/pts/5 (label charserial0) qemu-system-x86_64: -drive file=rbd:/libvirt-pool/server-clone-test:auth_supported=none:mon_host=192.168.0.40\:6789\;192.168.0.35\:6789\;192.168.0.2\:6789,if=none,id=drive-ide0-0-1:
But this causes an error when attempting to boot the vm: pool name too short
qemu-system-x86_64: -drive file=rbd:/libvirt-pool/server-clone-test:auth_supported=none:mon_host=192.168.0.40\:6789\;192.168.0.35\:6789\;192.168.0.2\:6789,if=none,id=drive-ide0-0-1: could not open disk image rbd:/libvirt-pool/server-clone-test:auth_supported=none:mon_host=192.168.0.40\:6789\;192.168.0.35\:6789\;192.168.0.2\:6789: Invalid argument
I'm currently using libguestfs-1.22.4 and I did run into a known bug [1] that I was able to patch, but I don't think that's related to the issues with disk naming. Is there a way to give virt-sysprep a rbd with the -a flag, or is there a better way to sysprep vms? I would like to avoid having to edit the vm config if possible. Thanks, Jon A [1] https://bugs.launchpad.net/ubuntu/+source/libguestfs/+bug/1243190
On Mon, Nov 04, 2013 at 06:35:42PM -0700, Jon wrote:
Hello,
I'm having a problem when trying to use virt-sysprep against vms that have rbd disk images. When I run virt-sysprep I get the following error:
root@kitt:~/libguestfs-1.22.4# virt-sysprep -d server-clone-test --firstboot firstboot.sh Examining the guest ... Fatal error: exception Guestfs.Error("rbd: image name must begin with a '/'")
My Disk definition is as follows:
<disk type='network' device='disk'> <driver name='qemu'/> <source protocol='rbd' name='libvirt-pool/server-clone-test'> <host name='192.168.0.40' port='6789'/> <host name='192.168.0.35' port='6789'/> <host name='192.168.0.2' port='6789'/> </source> <target dev='hdb' bus='ide'/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk>
I can resolve this error by changing the name of the libvirt-pool:
<source protocol='rbd' name='/libvirt-pool/server-clone-test'>
But this causes an error when attempting to boot the vm:
root@kitt:~/libguestfs-1.22.4# virsh start server-clone-test error: Failed to start domain server-clone-test error: internal error: process exited while connecting to monitor: W: kvm binary is deprecated, please use qemu-system-x86_64 instead char device redirected to /dev/pts/5 (label charserial0) qemu-system-x86_64: -drive file=rbd:/libvirt-pool/server-clone-test:auth_supported=none:mon_host=192.168.0.40\:6789\;192.168.0.35\:6789\;192.168.0.2\:6789,if=none,id=drive-ide0-0-1: pool name too short qemu-system-x86_64: -drive file=rbd:/libvirt-pool/server-clone-test:auth_supported=none:mon_host=192.168.0.40\:6789\;192.168.0.35\:6789\;192.168.0.2\:6789,if=none,id=drive-ide0-0-1: could not open disk image rbd:/libvirt-pool/server-clone-test:auth_supported=none:mon_host=192.168.0.40\:6789\;192.168.0.35\:6789\;192.168.0.2\:6789: Invalid argument
I'm currently using libguestfs-1.22.4 and I did run into a known bug [1] that I was able to patch, but I don't think that's related to the issues with disk naming.
Is there a way to give virt-sysprep a rbd with the -a flag, or is there a better way to sysprep vms? I would like to avoid having to edit the vm config if possible.
This is a bug basically. I have filed a bug report here: https://bugzilla.redhat.com/show_bug.cgi?id=1026688 It would be helpful to try with a newer version of libguestfs. Ubuntu has just pushed the 1.24.0 package. I don't know if it'll help or not, but it would be useful to know if the behaviour changes. https://launchpad.net/ubuntu/+source/libguestfs/1:1.24.0-0ubuntu1/+build/518... Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v
Hello Richard, Thanks for the reply. Indeed this behaviour exists in 1.25.6. Grepping through the source [1], there are a number of files in "./po/*.po[t]?" that contain this message, but I think it's ./src/drives.c where the fail condition is actually detected / set. On line 300 there is an if statement that checks the first character is a slash: if (exportname[0] != '/') { error (g, _("rbd: image name must begin with a '/'")); return NULL; } I tried to comment this out and recompile guestfs, but I get an error: make[2]: Entering directory `/root/libguestfs-1.25.6/builder' ocamlfind ocamlopt -g -warn-error CDEFLMPSUVYZX -package str,unix -I ../src/.libs -I ../ocaml -I ../mllib -package gettext-stub -c builder.ml -o builder.cmx File "builder.ml", line 53, characters 4-14: Error: Unbound value List.iteri make[2]: *** [builder.cmx] Error 2 make[2]: Leaving directory `/root/libguestfs-1.25.6/builder' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/root/libguestfs-1.25.6' make: *** [all] Error 2 Can you provide some assistance resolving this compile error please? Is there perhaps a better place to pull the source from? Thanks, Jon A [1] http://libguestfs.org/download/1.25-development/libguestfs-1.25.6.tar.gz On Tue, Nov 5, 2013 at 2:01 AM, Richard W.M. Jones <rjones@redhat.com>wrote:
On Mon, Nov 04, 2013 at 06:35:42PM -0700, Jon wrote:
Hello,
I'm having a problem when trying to use virt-sysprep against vms that have rbd disk images. When I run virt-sysprep I get the following error:
root@kitt:~/libguestfs-1.22.4# virt-sysprep -d server-clone-test --firstboot firstboot.sh Examining the guest ... Fatal error: exception Guestfs.Error("rbd: image name must begin with a '/'")
My Disk definition is as follows:
<disk type='network' device='disk'> <driver name='qemu'/> <source protocol='rbd' name='libvirt-pool/server-clone-test'> <host name='192.168.0.40' port='6789'/> <host name='192.168.0.35' port='6789'/> <host name='192.168.0.2' port='6789'/> </source> <target dev='hdb' bus='ide'/> <address type='drive' controller='0' bus='0' target='0'
unit='1'/>
</disk>
I can resolve this error by changing the name of the libvirt-pool:
<source protocol='rbd' name='/libvirt-pool/server-clone-test'>
But this causes an error when attempting to boot the vm:
root@kitt:~/libguestfs-1.22.4# virsh start server-clone-test error: Failed to start domain server-clone-test error: internal error: process exited while connecting to monitor: W: kvm binary is deprecated, please use qemu-system-x86_64 instead char device redirected to /dev/pts/5 (label charserial0) qemu-system-x86_64: -drive
file=rbd:/libvirt-pool/server-clone-test:auth_supported=none:mon_host=192.168.0.40\:6789\;192.168.0.35\:6789\;192.168.0.2\:6789,if=none,id=drive-ide0-0-1:
pool name too short
qemu-system-x86_64: -drive
file=rbd:/libvirt-pool/server-clone-test:auth_supported=none:mon_host=192.168.0.40\:6789\;192.168.0.35\:6789\;192.168.0.2\:6789,if=none,id=drive-ide0-0-1:
could not open disk image
rbd:/libvirt-pool/server-clone-test:auth_supported=none:mon_host=192.168.0.40\:6789\;192.168.0.35\:6789\;192.168.0.2\:6789:
Invalid argument
I'm currently using libguestfs-1.22.4 and I did run into a known bug [1] that I was able to patch, but I don't think that's related to the issues with disk naming.
Is there a way to give virt-sysprep a rbd with the -a flag, or is there a better way to sysprep vms? I would like to avoid having to edit the vm config if possible.
This is a bug basically. I have filed a bug report here:
https://bugzilla.redhat.com/show_bug.cgi?id=1026688
It would be helpful to try with a newer version of libguestfs. Ubuntu has just pushed the 1.24.0 package. I don't know if it'll help or not, but it would be useful to know if the behaviour changes.
https://launchpad.net/ubuntu/+source/libguestfs/1:1.24.0-0ubuntu1/+build/518...
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v
On Wed, Nov 06, 2013 at 11:17:14AM -0700, Jon wrote:
I tried to comment this out and recompile guestfs, but I get an error:
make[2]: Entering directory `/root/libguestfs-1.25.6/builder' ocamlfind ocamlopt -g -warn-error CDEFLMPSUVYZX -package str,unix -I ../src/.libs -I ../ocaml -I ../mllib -package gettext-stub -c builder.ml -o builder.cmx File "builder.ml", line 53, characters 4-14: Error: Unbound value List.iteri make[2]: *** [builder.cmx] Error 2 make[2]: Leaving directory `/root/libguestfs-1.25.6/builder' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/root/libguestfs-1.25.6' make: *** [all] Error 2
Hmm, this is actually another bug :-( The attached patch should fix this one and allow you to compile libguestfs with the older version of OCaml that didn't have List.iteri. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top
Hello Richard, Haha, ok, here's a good one: I commented that if statement out at line 300, applied your patch (I see you updated the github of this code, perhaps that's the best place to grab the code from), and when I run virt-sysprep, I get the following parameter for my disk drive:
qemu-system-x86_64: -drive file=rbd:ibvirt-pool/server-clone-test:mon_host=192.168.0.40\:6789\;192.168.0.35\:6789\;192.168.0.2\:6789:auth_supported=none,cache=writeback,id=hd0,if=none: error opening pool ibvirt-pool
Since "ibvirt-pool" doesn't exist. :) So this says to me that the function "create_drive_non_file" is consuming that first slash somewhere. Perhaps "safe_strdup"? I'm not sure where "safe_strdup" is to confirm, but it looks like other functions use the "safe_stardup" function. With that in mind, I didn't want to have to reinvent create_drive_non_file, and without a better understanding of how all the code works together, I came up with a little hack which basically prepends the slash if there isn't one. if (exportname[0] != '/') { // char name[(sizeof exportname) + 1]; char name[256]; snprintf(name, sizeof name, "/%s", exportname); exportname = name; } My concern is long rbd-pool/rbd names, hence the commented out line, but that causes the name of the disk to be:
qemu-system-x86_64: -drive file=rbd:libvirt:mon_host=192.168.0.40\:6789\;192.168.0.35\:6789\;192.168.0.2\:6789:auth_supported=none,cache=writeback,id=hd0,if=none: could not open disk image rbd:libvirt:mon_host=192.168.0.40\:6789\;192.168.0.35\:6789\;192.168.0.2\:6789:auth_supported=none: Invalid argument
For some reason, the commented out line in the if statement above is truncating the end of the name. Admittedly, my C skills are a bit rusty, so I'm probably overlooking something obvious. Any thoughts on this approach? Thanks, Jon A On Wed, Nov 6, 2013 at 12:22 PM, Richard W.M. Jones <rjones@redhat.com>wrote:
On Wed, Nov 06, 2013 at 11:17:14AM -0700, Jon wrote:
I tried to comment this out and recompile guestfs, but I get an error:
make[2]: Entering directory `/root/libguestfs-1.25.6/builder' ocamlfind ocamlopt -g -warn-error CDEFLMPSUVYZX -package str,unix -I ../src/.libs -I ../ocaml -I ../mllib -package gettext-stub -c builder.ml-o builder.cmx File "builder.ml", line 53, characters 4-14: Error: Unbound value List.iteri make[2]: *** [builder.cmx] Error 2 make[2]: Leaving directory `/root/libguestfs-1.25.6/builder' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/root/libguestfs-1.25.6' make: *** [all] Error 2
Hmm, this is actually another bug :-( The attached patch should fix this one and allow you to compile libguestfs with the older version of OCaml that didn't have List.iteri.
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top
On Wed, Nov 06, 2013 at 02:25:54PM -0700, Jon wrote:
Hello Richard,
Haha, ok, here's a good one: I commented that if statement out at line 300, applied your patch (I see you updated the github of this code, perhaps that's the best place to grab the code from), and when I run virt-sysprep, I get the following parameter for my disk drive:
qemu-system-x86_64: -drive file=rbd:ibvirt-pool/server-clone-test:mon_host=192.168.0.40\:6789\;192.168.0.35\:6789\;192.168.0.2\:6789:auth_supported=none,cache=writeback,id=hd0,if=none: error opening pool ibvirt-pool
Since "ibvirt-pool" doesn't exist. :)
So this says to me that the function "create_drive_non_file" is consuming that first slash somewhere.
Unfortunately this code is rather intricate (and intricately broken), but the problem is likely to be here: src/drives.c: /* Skip the mandatory leading '/' character on exportname. */ return safe_asprintf (g, "rbd:%s:mon_host=%s%s%s%s", &src->u.exportname[1], mon_host, username ? username : "", auth, secret ? secret : ""); The problem that this code is trying (and failing) to deal with is that when you use the URL on the guestfish command line, the URL is something like "rbd://example.com/foo/bar" and the path part of this has to start with a "/" character. There is simply no way to specify a URL without this. However this assumption should *not* have made its way into the libguestfs library, since other paths into this code, such as from libvirt XML, have no such restrictions. So it's currently somewhat broken. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming blog: http://rwmj.wordpress.com Fedora now supports 80 OCaml packages (the OPEN alternative to F#)
On Wed, Nov 06, 2013 at 09:37:01PM +0000, Richard W.M. Jones wrote:
when you use the URL on the guestfish command line, the URL is something like "rbd://example.com/foo/bar" and the path part of this has to start with a "/" character. There is simply no way to specify a URL without this.
(I mean, a URL that has a path that doesn't start with a "/".) Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming blog: http://rwmj.wordpress.com Fedora now supports 80 OCaml packages (the OPEN alternative to F#)
Hello Rich, Interesting. Thanks for the explanation. When you specify an rbd on the command line for virt-sysprep, do you expect the path to include the monitor address? e.g.:
virt-sysprep -a rbd://host-name/pool-name/device-name
If I understand correctly, libvirt is able to understand the ceph configuration, so when I create a device with qemu-img I only specify the protocol and pool/device. e.g.:
qemu-img create rbd:pool-name/device-name 5G
(there is some voodoo that I don't understand, I've got a whole thread on trying to get qemu-img to create format 2 rbds by default... but that's for another thread) Would it be possible to specify rbds like this instead? Or is the scope bigger than I'm understanding and that would cause issues with other disk types specified for the --add parameter. It seems like --add can take either a URI or a physical disk path. Thanks for all your help helping me to understand. Best Regards, Jon A On Wed, Nov 6, 2013 at 2:37 PM, Richard W.M. Jones <rjones@redhat.com>wrote:
On Wed, Nov 06, 2013 at 02:25:54PM -0700, Jon wrote:
Hello Richard,
Haha, ok, here's a good one: I commented that if statement out at line 300, applied your patch (I see you updated the github of this code, perhaps that's the best place to grab the code from), and when I run virt-sysprep, I get the following parameter for my disk drive:
qemu-system-x86_64: -drive
file=rbd:ibvirt-pool/server-clone-test:mon_host=192.168.0.40\:6789\;192.168.0.35\:6789\;192.168.0.2\:6789:auth_supported=none,cache=writeback,id=hd0,if=none:
error opening pool ibvirt-pool
Since "ibvirt-pool" doesn't exist. :)
So this says to me that the function "create_drive_non_file" is consuming that first slash somewhere.
Unfortunately this code is rather intricate (and intricately broken), but the problem is likely to be here:
src/drives.c: /* Skip the mandatory leading '/' character on exportname. */ return safe_asprintf (g, "rbd:%s:mon_host=%s%s%s%s", &src->u.exportname[1], mon_host, username ? username : "", auth, secret ? secret : "");
The problem that this code is trying (and failing) to deal with is that when you use the URL on the guestfish command line, the URL is something like "rbd://example.com/foo/bar" and the path part of this has to start with a "/" character. There is simply no way to specify a URL without this.
However this assumption should *not* have made its way into the libguestfs library, since other paths into this code, such as from libvirt XML, have no such restrictions.
So it's currently somewhat broken.
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming blog: http://rwmj.wordpress.com Fedora now supports 80 OCaml packages (the OPEN alternative to F#)
On Wed, Nov 06, 2013 at 02:52:06PM -0700, Jon wrote:
Hello Rich,
Interesting. Thanks for the explanation.
When you specify an rbd on the command line for virt-sysprep, do you expect the path to include the monitor address?
e.g.:
virt-sysprep -a rbd://host-name/pool-name/device-name
Note that these URIs are not Ceph URIs. They are something that libguestfs virt tools invent, implemented in fish/uri.{c,h} and described here: http://libguestfs.org/guestfish.1.html#adding-remote-storage Below that layer is the interface to libguestfs-as-a-library, described here: http://libguestfs.org/guestfs.3.html#remote-storage So if I understand your question correctly, then I guess the answer is "no", although I'm not sure what a "monitor address" is. Because libguestfs virt tools URIs are nothing like and unrelated to Ceph URIs.
If I understand correctly, libvirt is able to understand the ceph configuration, so when I create a device with qemu-img I only specify the protocol and pool/device.
e.g.:
qemu-img create rbd:pool-name/device-name 5G
(there is some voodoo that I don't understand, I've got a whole thread on trying to get qemu-img to create format 2 rbds by default... but that's for another thread)
Would it be possible to specify rbds like this instead? Or is the scope bigger than I'm understanding and that would cause issues with other disk types specified for the --add parameter. It seems like --add can take either a URI or a physical disk path.
I don't know, but see above. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top
participants (2)
-
Jon -
Richard W.M. Jones