[PATCH] Don't use libvirt for volume information when converting with libvirtxml
Unless you're lucky, the target volumes won't be in a storage pool, so libvirt will return an error. For libvirtxml we know that the storage is local, so we can inspect it directly. We use qemu-img to determine format and file size. --- lib/Sys/VirtV2V/Connection/LibVirtXMLSource.pm | 48 ++++++++++++++++++++--- 1 files changed, 41 insertions(+), 7 deletions(-) diff --git a/lib/Sys/VirtV2V/Connection/LibVirtXMLSource.pm b/lib/Sys/VirtV2V/Connection/LibVirtXMLSource.pm index ffb9309..4bb5eb7 100644 --- a/lib/Sys/VirtV2V/Connection/LibVirtXMLSource.pm +++ b/lib/Sys/VirtV2V/Connection/LibVirtXMLSource.pm @@ -20,13 +20,14 @@ package Sys::VirtV2V::Connection::LibVirtXMLSource; use strict; use warnings; +use File::stat; use Sys::Virt; use XML::DOM; use XML::DOM::XPath; use Sys::VirtV2V::Connection::Source; use Sys::VirtV2V::Transfer::Local; -use Sys::VirtV2V::Util qw(user_message parse_libvirt_volinfo); +use Sys::VirtV2V::Util qw(user_message); use Locale::TextDomain 'virt-v2v'; @@ -115,12 +116,45 @@ sub get_volume my $self = shift; my ($path) = @_; - # Use a libvirt session connection to inspect local volumes - my $vmm = Sys::Virt->new(uri => 'qemu:///session'); - my $vol = $vmm->get_storage_volume_by_path($path); - - my ($name, $format, $size, $usage, $is_sparse, $is_block) = - parse_libvirt_volinfo($vol, $path); + # Use the output of qemu-img to inspect the path + open(my $qemuimg, '-|', 'env', 'LANG=C', 'qemu-img', 'info', $path) + or die("Unable to execute qemu-img: $!"); + + # qemu-img outputs data similar to: + # image: /var/lib/libvirt/images/p2v.img + # file format: raw + # virtual size: 8.0G (8589934592 bytes) + # disk size: 8.0G + my %info; + while(<$qemuimg>) { + next unless /^([^:]+):\s+(.*?)\s*$/; + $info{$1} = $2; + } + + my (undef, undef, $name) = File::Spec->splitpath($path); + my $format = $info{'file format'}; + + my $vsize = $info{'virtual size'}; + $vsize =~ /\s+\((\d+)\s+bytes\)$/ + or die("qemu-img returned unexpected virtual size: $vsize"); + my $size = $1; + + # For $usage, $is_sparse and $is_block, we need to know if it's a block + # device + # N.B. qemu-img's 'disk size' output isn't useful here + my ($usage, $is_sparse, $is_block); + if (-b $path) { + $is_block = 1; + $usage = $size; + $is_sparse = 0; + } else { + $is_block = 0; + my $st = stat($path); + $usage = $st->blocks * 512; + $is_sparse = $usage < $size ? 1 : 0; + } + + die("size ($size) < usage ($usage)") if $size < $usage; my $transfer = new Sys::VirtV2V::Transfer::Local($path, $format, $is_sparse); -- 1.7.3.5
On Wed, Jan 26, 2011 at 02:34:49PM +0000, Matthew Booth wrote:
Unless you're lucky, the target volumes won't be in a storage pool, so libvirt will return an error. For libvirtxml we know that the storage is local, so we can inspect it directly. We use qemu-img to determine format and file size. [...] + # Use the output of qemu-img to inspect the path + open(my $qemuimg, '-|', 'env', 'LANG=C', 'qemu-img', 'info', $path) + or die("Unable to execute qemu-img: $!");
You need to make sure qemu-img is an explicit Requires in the spec file, ie: Requires: /usr/bin/qemu-img and it should be mentioned in the README file too for other distros. Also other distros name this differently, eg: 'kvm-qemu-img' and maybe other variations.
+ # For $usage, $is_sparse and $is_block, we need to know if it's a block + # device + # N.B. qemu-img's 'disk size' output isn't useful here + my ($usage, $is_sparse, $is_block); + if (-b $path) { + $is_block = 1; + $usage = $size; + $is_sparse = 0; + } else { + $is_block = 0; + my $st = stat($path); + $usage = $st->blocks * 512; + $is_sparse = $usage < $size ? 1 : 0; + } + + die("size ($size) < usage ($usage)") if $size < $usage;
Agreed that qemu-img gets it wrong: # qemu-img info /dev/vg_pin/CentOS5x64 image: /dev/vg_pin/CentOS5x64 file format: raw virtual size: 10G (10737418240 bytes) disk size: 0 *but* I'm not sure your code is correct. I don't know if $st->blocks is always valid, and you can't assume 512-sized blocks. I suggest calling 'blockdev --getsize64', with the caveats above about dependencies and README file. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into Xen guests. http://et.redhat.com/~rjones/virt-p2v
On 26/01/11 15:00, Richard W.M. Jones wrote:
On Wed, Jan 26, 2011 at 02:34:49PM +0000, Matthew Booth wrote:
Unless you're lucky, the target volumes won't be in a storage pool, so libvirt will return an error. For libvirtxml we know that the storage is local, so we can inspect it directly. We use qemu-img to determine format and file size. [...] + # Use the output of qemu-img to inspect the path + open(my $qemuimg, '-|', 'env', 'LANG=C', 'qemu-img', 'info', $path) + or die("Unable to execute qemu-img: $!");
You need to make sure qemu-img is an explicit Requires in the spec file, ie:
Requires: /usr/bin/qemu-img
Good point, I'll make sure it goes in.
and it should be mentioned in the README file too for other distros. Also other distros name this differently, eg: 'kvm-qemu-img' and maybe other variations.
Not sure how I'd handle this for the moment. I think I'll file this under 'fix when it happens'.
+ # For $usage, $is_sparse and $is_block, we need to know if it's a block + # device + # N.B. qemu-img's 'disk size' output isn't useful here + my ($usage, $is_sparse, $is_block); + if (-b $path) { + $is_block = 1; + $usage = $size; + $is_sparse = 0; + } else { + $is_block = 0; + my $st = stat($path); + $usage = $st->blocks * 512; + $is_sparse = $usage< $size ? 1 : 0; + } + + die("size ($size)< usage ($usage)") if $size< $usage;
Agreed that qemu-img gets it wrong:
# qemu-img info /dev/vg_pin/CentOS5x64 image: /dev/vg_pin/CentOS5x64 file format: raw virtual size: 10G (10737418240 bytes) disk size: 0
*but* I'm not sure your code is correct. I don't know if $st->blocks is always valid, and you can't assume 512-sized blocks.
I suggest calling 'blockdev --getsize64', with the caveats above about dependencies and README file.
I observed that qemu-img *does* get the correct 'virtual size' for both block devices and regular files. I don't want to use more external tools than required, so I'd prefer not to call blockdev. Why wouldn't you trust qemu-img here? Incidentally, its 'disk size' value is correct for regular files, but it isn't output in bytes. This is the principal reason I ignore it. I could update the comment to reflect that. As for $st->blocks, any idea when it wouldn't be valid? As it happens, you can assume that this is in units of 512 (see man 2 stat). Although block sizes have moved on, I guess this value has been grandfathered for compatibility. Matt -- Matthew Booth, RHCA, RHCSS Red Hat Engineering, Virtualisation Team GPG ID: D33C3490 GPG FPR: 3733 612D 2D05 5458 8A8A 1600 3441 EA19 D33C 3490
On Wed, Jan 26, 2011 at 03:16:50PM +0000, Matthew Booth wrote:
I observed that qemu-img *does* get the correct 'virtual size' for both block devices and regular files. I don't want to use more external tools than required, so I'd prefer not to call blockdev. Why wouldn't you trust qemu-img here? [...]
Actually I've looked at the code (qemu/block/*.c), yes I would trust it. It uses 'lseek (fd, 0, SEEK_END)' for raw format block devices and files. It uses the metadata for qcow2.
As for $st->blocks, any idea when it wouldn't be valid? As it happens, you can assume that this is in units of 512 (see man 2 stat). Although block sizes have moved on, I guess this value has been grandfathered for compatibility.
OK it does appear to say that. Still I wouldn't be wholly trusting of this, particularly when you mix in 4K sectors and other operating systems. Anyway qemu-img seems to be doing the right thing, so go with that, unless you really need the on-disk allocated size for some reason. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
On Wed, Jan 26, 2011 at 03:00:32PM +0000, Richard W.M. Jones wrote:
On Wed, Jan 26, 2011 at 02:34:49PM +0000, Matthew Booth wrote:
Unless you're lucky, the target volumes won't be in a storage pool, so libvirt will return an error. For libvirtxml we know that the storage is local, so we can inspect it directly. We use qemu-img to determine format and file size. [...] + # Use the output of qemu-img to inspect the path + open(my $qemuimg, '-|', 'env', 'LANG=C', 'qemu-img', 'info', $path) + or die("Unable to execute qemu-img: $!");
You need to make sure qemu-img is an explicit Requires in the spec file, ie:
Requires: /usr/bin/qemu-img
and it should be mentioned in the README file too for other distros. Also other distros name this differently, eg: 'kvm-qemu-img' and maybe other variations.
+ # For $usage, $is_sparse and $is_block, we need to know if it's a block + # device + # N.B. qemu-img's 'disk size' output isn't useful here + my ($usage, $is_sparse, $is_block); + if (-b $path) { + $is_block = 1; + $usage = $size; + $is_sparse = 0; + } else { + $is_block = 0; + my $st = stat($path); + $usage = $st->blocks * 512; + $is_sparse = $usage < $size ? 1 : 0; + } + + die("size ($size) < usage ($usage)") if $size < $usage;
Agreed that qemu-img gets it wrong:
# qemu-img info /dev/vg_pin/CentOS5x64 image: /dev/vg_pin/CentOS5x64 file format: raw virtual size: 10G (10737418240 bytes) disk size: 0
*but* I'm not sure your code is correct. I don't know if $st->blocks is always valid, and you can't assume 512-sized blocks.
For files 'st_blocks' always refers to number of 512 byte blocks, regardless of whether the underlying FS/storage is actually using 512 byte blocks for storing the data. So hardcoding 512 is fine
I suggest calling 'blockdev --getsize64', with the caveats above about dependencies and README file.
No need for calling blockdev actually, the quick way to determine the size of a block device is to just seek to the end of the device and then query the file pointer position, eg open DEV, "/dev/sda"; seek DEV, 0, SEEK_END; $size = tell DEV; close DEV; Regards, Daniel
participants (3)
-
Daniel P. Berrange -
Matthew Booth -
Richard W.M. Jones