Virtualbox vdi Input Format and man pages
Hi, I am new to v2v/libguestfs. I need to convert a 30 GB virtual machine running Windows7 64 bit (a guest on a Linux system) from Virtualbox vdi format to qcow2 (or raw/img--another debate in itself) so I can use libvirt/qemu/kvm to run it and completely migrate away from Virtualbox. The vdi machine is a mission critical production environment and I cannot afford to mess it up, etc. Will keep original vdi intact until have successfully converted and tested. I have done some research and come upon v2v which seems to be just what the doctor ordered except that, in "local" man pages of version 1.36.4 which I have installed, there is NO mention of vdi compatibility as input format. However, on these online man pages <https://linux.die.net/man/1/virt-v2v> I see a section called "Local VirtualBox guests" which seems to indicate that vdi is indeed accepted as input format. Can anyone provide feedback as to the above? v2v seems to be a tool I could use to more easily deal with this conversion/migration than just using qemu-img convert and having to then deal with the other bits and pieces such as virtio drivers, etc. Thanks for any helpful feedback and pointers.
On Sat, Aug 19, 2017 at 08:38:31AM -0600, stef204 wrote:
Hi,
I am new to v2v/libguestfs.
I need to convert a 30 GB virtual machine running Windows7 64 bit (a guest on a Linux system) from Virtualbox vdi format to qcow2 (or raw/img--another debate in itself) so I can use libvirt/qemu/kvm to run it and completely migrate away from Virtualbox.
The vdi machine is a mission critical production environment and I cannot afford to mess it up, etc. Will keep original vdi intact until have successfully converted and tested.
This is very good advice.
I have done some research and come upon v2v which seems to be just what the doctor ordered except that, in "local" man pages of version 1.36.4 which I have installed, there is NO mention of vdi compatibility as input format.
However, on these online man pages <https://linux.die.net/man/1/virt-v2v> I see a section called "Local VirtualBox guests" which seems to indicate that vdi is indeed accepted as input format.
Unfortunately the link documents the old version of virt-v2v (which we normally call "0.9"). The current version is a derivative of that, of sorts, but massively rewritten internally.
Can anyone provide feedback as to the above?
v2v seems to be a tool I could use to more easily deal with this conversion/migration than just using qemu-img convert and having to then deal with the other bits and pieces such as virtio drivers, etc.
It's good that you mention ‘qemu-img convert’, because the significant difference between virt-v2v & plain qemu-img conversion is that virt-v2v will try to install virtio drivers. Installing virtio drivers in a Windows guest post-conversion is a PITA which is why virt-v2v may be better if you require virtio. So ... to the topic: We don't specifically test conversion from VirtualBox or .vdi. But it ought to work. You will need the virtio drivers from: https://fedoraproject.org/wiki/Windows_Virtio_Drivers You can either install that package so it appears under /usr/share/virtio-win, or you can use the .iso file from that package and set the VIRTIO_WIN environment variable to point to it: http://libguestfs.org/virt-v2v.1.html#environment-variables Do you have metadata from VirtualBox (number of CPUs, RAM, etc) in some format? If not, then you should probably use the ‘-i libvirtxml’ input method. You will have to write or modify the metadata, as documented here: http://libguestfs.org/virt-v2v.1.html#minimal-xml-for--i-libvirtxml-option I believe that virt-v2v should be able to transparently convert the .vdi source file. You might or might not need to use the ‘-if vdi‘ option. It's safe to run virt-v2v on the guest disk image (or a copy, if you want to be extra cautious), just to see if it can be converted. If it fails or gives warnings or errors, then please post the full output of ‘virt-v2v -v -x ....’ Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW
Rich, Thanks a lot for your detailed feedback, very helpful. Will follow your suggestions and post again here as needed. 19.08.2017, 10:36, "Richard W.M. Jones" <rjones@redhat.com>:
It's good that you mention ‘qemu-img convert’, because the significant difference between virt-v2v & plain qemu-img conversion is that virt-v2v will try to install virtio drivers. Installing virtio drivers in a Windows guest post-conversion is a PITA which is why virt-v2v may be better if you require virtio.
So ... to the topic: We don't specifically test conversion from VirtualBox or .vdi. But it ought to work.
You will need the virtio drivers from: https://fedoraproject.org/wiki/Windows_Virtio_Drivers You can either install that package so it appears under /usr/share/virtio-win, or you can use the .iso file from that package and set the VIRTIO_WIN environment variable to point to it:
http://libguestfs.org/virt-v2v.1.html#environment-variables
Do you have metadata from VirtualBox (number of CPUs, RAM, etc) in some format? If not, then you should probably use the ‘-i libvirtxml’ input method. You will have to write or modify the metadata, as documented here:
http://libguestfs.org/virt-v2v.1.html#minimal-xml-for--i-libvirtxml-option
I believe that virt-v2v should be able to transparently convert the .vdi source file. You might or might not need to use the ‘-if vdi‘ option.
It's safe to run virt-v2v on the guest disk image (or a copy, if you want to be extra cautious), just to see if it can be converted. If it fails or gives warnings or errors, then please post the full output of ‘virt-v2v -v -x ....’
Rich.
Only able to get to this now, but need additional help, if at all possible. See comments inline below. Thanks. 19.08.2017, 10:36, "Richard W.M. Jones" <rjones@redhat.com>:
On Sat, Aug 19, 2017 at 08:38:31AM -0600, stef204 wrote:
Hi,
I am new to v2v/libguestfs.
I need to convert a 30 GB virtual machine running Windows7 64 bit (a guest on a Linux system) from Virtualbox vdi format to qcow2 (or raw/img--another debate in itself) so I can use libvirt/qemu/kvm to run it and completely migrate away from Virtualbox.
You will need the virtio drivers from: https://fedoraproject.org/wiki/Windows_Virtio_Drivers You can either install that package so it appears under /usr/share/virtio-win, or you can use the .iso file from that package and set the VIRTIO_WIN environment variable to point to it:
The above is done (easy).
Do you have metadata from VirtualBox (number of CPUs, RAM, etc) in some format? If not, then you should probably use the ‘-i libvirtxml’ input method. You will have to write or modify the metadata, as documented here:
http://libguestfs.org/virt-v2v.1.html#minimal-xml-for--i-libvirtxml-option
Please bear with me but I really don't have enough info as to HOW to modify the xml template., beyond the obvious stuff, MAC address, name of VM, etc. Might there bit a more detailed xml template in context of Windows to kvm/qemu I could use?
I believe that virt-v2v should be able to transparently convert the .vdi source file. You might or might not need to use the ‘-if vdi‘ option.
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
On Sun, Oct 08, 2017 at 07:03:50AM -0600, stef204 wrote:
http://libguestfs.org/virt-v2v.1.html#minimal-xml-for--i-libvirtxml-option
Please bear with me but I really don't have enough info as to HOW to modify the xml template., beyond the obvious stuff, MAC address, name of VM, etc.
Might there bit a more detailed xml template in context of Windows to kvm/qemu I could use?
I guess we should probably link to the real documentation: https://libvirt.org/formatdomain.html However virt-v2v has its own parser for libvirt XML and only parses a (very) small subset of these fields. Generally you only need to take that template and change a few fields, name, memory size, number of vCPUs, and make sure there is one <disk> section per disk and one <interface> section per virtual network adaptor. That'll cover 99% of use cases. Use ‘virt-v2v ... -o null’ to do a test conversion. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top
08.10.2017, 08:09, "Richard W.M. Jones" <rjones@redhat.com>:
I guess we should probably link to the real documentation:
https://libvirt.org/formatdomain.html
However virt-v2v has its own parser for libvirt XML and only parses a (very) small subset of these fields. Generally you only need to take that template and change a few fields, name, memory size, number of vCPUs, and make sure there is one <disk> section per disk and one <interface> section per virtual network adaptor. That'll cover 99% of use cases.
Use ‘virt-v2v ... -o null’ to do a test conversion.
Rich.
Thanks so much Rich. Will move forward and post here again.
On Sun, Oct 08, 2017 at 08:47:34AM -0600, stef204 wrote:
08.10.2017, 08:09, "Richard W.M. Jones" <rjones@redhat.com>:
I guess we should probably link to the real documentation:
https://libvirt.org/formatdomain.html
However virt-v2v has its own parser for libvirt XML and only parses a (very) small subset of these fields. Generally you only need to take that template and change a few fields, name, memory size, number of vCPUs, and make sure there is one <disk> section per disk and one <interface> section per virtual network adaptor. That'll cover 99% of use cases.
BTW here is the actual parser, which is the source of all truth on this subject: https://github.com/libguestfs/libguestfs/blob/master/v2v/parse_libvirt_xml.m... https://github.com/libguestfs/libguestfs/blob/master/v2v/parse_libvirt_xml.m... Of course it's quite complex but it should be possible to grep through that file to answer simple questions such as "does virt-v2v look at <some field> in the XML or not?". Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top
08.10.2017, 09:02, "Richard W.M. Jones" <rjones@redhat.com>:
BTW here is the actual parser, which is the source of all truth on this subject:
https://github.com/libguestfs/libguestfs/blob/master/v2v/parse_libvirt_xml.m... https://github.com/libguestfs/libguestfs/blob/master/v2v/parse_libvirt_xml.m...
Of course it's quite complex but it should be possible to grep through that file to answer simple questions such as "does virt-v2v look at <some field> in the XML or not?".
Thanks.
08.10.2017, 08:09, "Richard W.M. Jones" <rjones@redhat.com>:
However virt-v2v has its own parser for libvirt XML and only parses a (very) small subset of these fields. Generally you only need to take that template and change a few fields, name, memory size, number of vCPUs, and make sure there is one <disk> section per disk and one <interface> section per virtual network adaptor. That'll cover 99% of use cases.
Use ‘virt-v2v ... -o null’ to do a test conversion.
I am undoubtedly doing something wrong but cannot spot it. Here is the xml file <https://bpaste.net/show/bd4df241ee37> Error message: % virt-v2v -i libvirtxml Win7.xml -o null [ 0.0] Opening the source -i libvirtxml Win7.xml [ 0.0] Creating an overlay to protect the source from being modified [ 0.1] Initializing the target -o null [ 0.1] Opening the overlay [ 4.3] Inspecting the overlay virt-v2v: error: inspection could not detect the source guest (or physical machine). Assuming that you are running virt-v2v/virt-p2v on a source which is supported (and not, for example, a blank disk), then this should not happen. No root device found in this operating system image. If reporting bugs, run virt-v2v with debugging enabled and include the complete output: virt-v2v -v -x [...] I'm stuck here. I imagine the error is probably obvious but... not to me.
running with -v -x I see this just above the failure: guestfsd: main_loop: new request, len 0x3c commandrvf: stdout=n stderr=y flags=0x0 commandrvf: mount -o ro /dev/sda /sysroot/ mount: wrong fs type, bad option, bad superblock on /dev/sda, missing codepage or helper program, or other error In some cases useful info is found in syslog - try dmesg | tail or so. guestfsd: error: /dev/sda on / (options: 'ro'): mount: wrong fs type, bad option, bad superblock on /dev/sda, missing codepage or helper program, or other error In some cases useful info is found in syslog - try dmesg | tail or so. guestfsd: main_loop: proc 73 (mount_ro) took 0.08 seconds libguestfs: trace: v2v: mount_ro = -1 (error) libguestfs: trace: v2v: inspect_get_roots libguestfs: trace: v2v: inspect_get_roots = [] libguestfs: trace: v2v: inspect_os = [] 08.10.2017, 10:13, "stef204" <stef204@yandex.com>:
08.10.2017, 08:09, "Richard W.M. Jones" <rjones@redhat.com>:
However virt-v2v has its own parser for libvirt XML and only parses a (very) small subset of these fields. Generally you only need to take that template and change a few fields, name, memory size, number of vCPUs, and make sure there is one <disk> section per disk and one <interface> section per virtual network adaptor. That'll cover 99% of use cases.
Use ‘virt-v2v ... -o null’ to do a test conversion.
I am undoubtedly doing something wrong but cannot spot it.
Here is the xml file <https://bpaste.net/show/bd4df241ee37>
Error message:
% virt-v2v -i libvirtxml Win7.xml -o null [ 0.0] Opening the source -i libvirtxml Win7.xml [ 0.0] Creating an overlay to protect the source from being modified [ 0.1] Initializing the target -o null [ 0.1] Opening the overlay [ 4.3] Inspecting the overlay virt-v2v: error: inspection could not detect the source guest (or physical machine).
Assuming that you are running virt-v2v/virt-p2v on a source which is supported (and not, for example, a blank disk), then this should not happen.
No root device found in this operating system image.
If reporting bugs, run virt-v2v with debugging enabled and include the complete output:
virt-v2v -v -x [...]
I'm stuck here. I imagine the error is probably obvious but... not to me.
On Sun, Oct 08, 2017 at 10:28:39AM -0600, stef204 wrote:
running with -v -x I see this just above the failure:
guestfsd: main_loop: new request, len 0x3c commandrvf: stdout=n stderr=y flags=0x0 commandrvf: mount -o ro /dev/sda /sysroot/ mount: wrong fs type, bad option, bad superblock on /dev/sda, missing codepage or helper program, or other error
In some cases useful info is found in syslog - try dmesg | tail or so. guestfsd: error: /dev/sda on / (options: 'ro'): mount: wrong fs type, bad option, bad superblock on /dev/sda, missing codepage or helper program, or other error
In some cases useful info is found in syslog - try dmesg | tail or so. guestfsd: main_loop: proc 73 (mount_ro) took 0.08 seconds libguestfs: trace: v2v: mount_ro = -1 (error) libguestfs: trace: v2v: inspect_get_roots libguestfs: trace: v2v: inspect_get_roots = [] libguestfs: trace: v2v: inspect_os = []
I need to see the entire log. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/
[stef204 sent me the full log since it contains sensitive information] The log says that virt-v2v cannot see anything at all on the 34.1 GB disk (as if the disk is blank). However I think the actual problem is that you've given the wrong disk type in the XML: <disk type='file' device='disk'> <driver name='qemu' type='raw'/> ^^^ <source file='xxx.vdi'/> <target dev='hda' bus='ide'/> </disk> ‘.vdi’ is probably not raw format but something else (most likely 'vdi') According to ‘qemu-img --help’, qemu supports the vdi format, hopefully at least enough to be able to read it which is all that virt-v2v needs. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW
08.10.2017, 13:50, "Richard W.M. Jones" <rjones@redhat.com>:
[stef204 sent me the full log since it contains sensitive information]
The log says that virt-v2v cannot see anything at all on the 34.1 GB disk (as if the disk is blank). However I think the actual problem is that you've given the wrong disk type in the XML:
<disk type='file' device='disk'> <driver name='qemu' type='raw'/> ^^^ <source file='xxx.vdi'/> <target dev='hda' bus='ide'/> </disk>
‘.vdi’ is probably not raw format but something else (most likely 'vdi')
According to ‘qemu-img --help’, qemu supports the vdi format, hopefully at least enough to be able to read it which is all that virt-v2v needs.
OK, so to fix, just change this line--like this? <driver name='qemu' type='vdi'/>
On Sun, Oct 08, 2017 at 02:16:43PM -0600, stef204 wrote:
08.10.2017, 13:50, "Richard W.M. Jones" <rjones@redhat.com>:
[stef204 sent me the full log since it contains sensitive information]
The log says that virt-v2v cannot see anything at all on the 34.1 GB disk (as if the disk is blank). However I think the actual problem is that you've given the wrong disk type in the XML:
<disk type='file' device='disk'> <driver name='qemu' type='raw'/> ^^^ <source file='xxx.vdi'/> <target dev='hda' bus='ide'/> </disk>
‘.vdi’ is probably not raw format but something else (most likely 'vdi')
According to ‘qemu-img --help’, qemu supports the vdi format, hopefully at least enough to be able to read it which is all that virt-v2v needs.
OK, so to fix, just change this line--like this?
<driver name='qemu' type='vdi'/>
I think so, yes. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/
08.10.2017, 13:50, "Richard W.M. Jones" <rjones@redhat.com>:
[stef204 sent me the full log since it contains sensitive information]
The log says that virt-v2v cannot see anything at all on the 34.1 GB disk (as if the disk is blank). However I think the actual problem is that you've given the wrong disk type in the XML:
<disk type='file' device='disk'> <driver name='qemu' type='raw'/> ^^^ <source file='xxx.vdi'/> <target dev='hda' bus='ide'/> </disk>
‘.vdi’ is probably not raw format but something else (most likely 'vdi')
Running it currently with the following change in the xml file seems to be doing the job [it returns "libguestfs: closing guestfs handle 0x55f167f97db0 (state 0)"]: <driver name='qemu' type='vdi'/> I have an important question: While we were going back and forth here and I was waiting for your feedback (which I should repeat, is most appreciated) I tried this command, from reading the man pages: % virt-v2v -i disk Win7-convert.vdi -of qcow2 -on Win7.qcow2 -o local -os /mnt/partition2 The -on option probably was not needed as I ended up with a name like Win7.qcow2-sda when I just wanted Win7.qcow2 However, it completed without failure. I then created a new VM using libvirt/Virt-Manager and it booted just fine. Once booted, I went on to uninstall the virtualbox-guest-additions and install this: <https://www.spice-space.org/download/windows/spice-guest-tools/spice-guest-tools-latest.exe> And all seems to work properly. I have a network connection and clipboard is shared, etc. This was just a test from a copy or the original vdi file I had made using dd. So I am going to possibly redo it "properly" now for a "final" conversion. The question is: What am I getting extra by using the -i libvirtxml option as opposed to the more plain: % virt-v2v -i disk Win7-convert.vdi -of qcow2 -o local -os /mnt/partition2
On Sun, Oct 08, 2017 at 02:41:30PM -0600, stef204 wrote:
The question is: What am I getting extra by using the -i libvirtxml option as opposed to the more plain:
% virt-v2v -i disk Win7-convert.vdi -of qcow2 -o local -os /mnt/partition2
Guests consist of the disk images, plus copious metadata. When you give just the disk image to virt-v2v (‘-i disk disk.img’), it has to make many guesses about the metadata. See this file for details: https://github.com/libguestfs/libguestfs/blob/master/v2v/input_disk.ml Sometimes these guesses will be suboptimal, and by supplying the correct metadata you can get a more correct conversion (‘-i libvirtxml’ or even better getting the true metadata from VirtualBox). Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top
08.10.2017, 16:00, "Richard W.M. Jones" <rjones@redhat.com>:
Guests consist of the disk images, plus copious metadata.
When you give just the disk image to virt-v2v (‘-i disk disk.img’), it has to make many guesses about the metadata. See this file for details: https://github.com/libguestfs/libguestfs/blob/master/v2v/input_disk.ml
Sometimes these guesses will be suboptimal, and by supplying the correct metadata you can get a more correct conversion (‘-i libvirtxml’ or even better getting the true metadata from VirtualBox).
OK, thanks a lot for your helpful feedback, most appreciated.
participants (2)
-
Richard W.M. Jones -
stef204