April 2010 - Libguestfs - libguestfs List Archives

How many fs types do we need to test
by Pengzhen Cao 30 Apr '10

30 Apr '10

Hi Richard, How many fs types does libguestfs support now? And how many do you suggest we test? We are now covered ext2/3/4/fat32/ntfs. Regards, Pengzhen Cao

3 3

[PATCH] GuestOS: Fix augeas grub configuration
by Matthew Booth 29 Apr '10

29 Apr '10

Augeas now configures the Grub lens for /boot/grub/menu.lst by default. The code which was checking this was broken in the case where there was more than 1 included path. We now check if /boot/grub/menu.lst is included. If it is, we don't reload. If it isn't, we add it, but don't remove any other config first. This fixes RHBZ#586822 --- lib/Sys/VirtV2V/GuestOS/RedHat.pm | 25 +++++++++++++++++-------- 1 files changed, 17 insertions(+), 8 deletions(-) diff --git a/lib/Sys/VirtV2V/GuestOS/RedHat.pm b/lib/Sys/VirtV2V/GuestOS/RedHat.pm index 6dc4c95..a4680c5 100644 --- a/lib/Sys/VirtV2V/GuestOS/RedHat.pm +++ b/lib/Sys/VirtV2V/GuestOS/RedHat.pm @@ -87,7 +87,7 @@ sub new bless($self, $class); $self->_init_selinux(); - $self->_init_augeas_modprobe(); + $self->_init_augeas(); return $self; } @@ -107,7 +107,7 @@ sub _init_selinux $g->touch('/.autorelabel'); } -sub _init_augeas_modprobe +sub _init_augeas { my $self = shift; @@ -158,15 +158,24 @@ sub _init_augeas_modprobe } } - # Remove all includes for the Grub lens, and add only - # /boot/grub/menu.lst + # Check if /boot/grub/menu.lst is included by the Grub lens + my $found = 0; foreach my $incl ($g->aug_match("/augeas/load/Grub/incl")) { - $g->aug_rm($incl); + if ($g->aug_get($incl) eq '/boot/grub/menu.lst') { + $found = 1; + last; + } + } + + # If it wasn't there, add it + unless ($found) { + $g->aug_set("/augeas/load/Grub/incl[last()+1]", + "/boot/grub/menu.lst"); + + # Make augeas pick up the new configuration + $g->aug_load(); } - $g->aug_set("/augeas/load/Grub/incl[last()+1]", "/boot/grub/menu.lst"); - # Make augeas pick up the new configuration - $g->aug_load(); }; # The augeas calls will die() on any error. -- 1.6.6.1

2 1

[PATCH 1/2] SSH: Check for complete transfer of disk images
by Matthew Booth 29 Apr '10

29 Apr '10

--- lib/Sys/VirtV2V/Transfer/SSH.pm | 7 +++++++ 1 files changed, 7 insertions(+), 0 deletions(-) diff --git a/lib/Sys/VirtV2V/Transfer/SSH.pm b/lib/Sys/VirtV2V/Transfer/SSH.pm index 3b08716..ac5a384 100644 --- a/lib/Sys/VirtV2V/Transfer/SSH.pm +++ b/lib/Sys/VirtV2V/Transfer/SSH.pm @@ -82,6 +82,7 @@ sub transfer my $vol = $target->create_volume($name, $size); $vol->open(); + my $written = 0; for (;;) { my $buffer; # Transfer in 8k chunks @@ -93,10 +94,16 @@ sub transfer last if ($in == 0); $vol->write($buffer); + $written += length($buffer); } $vol->close(); + die(user_message(__x("Didn't receive full volume. Received {received} ". + "of {total} bytes.", + received => $written, + total => $size))) unless ($written == $size); + waitpid($pid, 0) == $pid or die("error reaping child: $!"); # If the child returned an error, check for anything on its stderr if ($? != 0) { -- 1.6.6.1

2 2

[PATCH] GuestOS: Fix error when checking for rpm which isn't installed
by Matthew Booth 29 Apr '10

29 Apr '10

--- lib/Sys/VirtV2V/GuestOS/RedHat.pm | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/lib/Sys/VirtV2V/GuestOS/RedHat.pm b/lib/Sys/VirtV2V/GuestOS/RedHat.pm index a973c19..6dc4c95 100644 --- a/lib/Sys/VirtV2V/GuestOS/RedHat.pm +++ b/lib/Sys/VirtV2V/GuestOS/RedHat.pm @@ -686,7 +686,7 @@ sub _get_installed error => $error))); } - my @installed; + my @installed = (); foreach my $installed (@output) { $installed =~ /^(\S+)\s+(\S+)\s+(\S+)$/ or die("Unexpected return from rpm command: $installed"); @@ -779,7 +779,8 @@ sub _get_deppaths my ($name, undef, undef, undef, $arch) = $self->_get_nevra($path); - if ($self->_get_installed($name, $arch) > 0) { + my @installed = $self->_get_installed($name, $arch); + if (@installed > 0) { $required{$path} = 1; foreach my $deppath ($self->_get_deppaths('i386', @$deps)) { -- 1.6.6.1

2 1

Fix 2 issues in ESX transfer
by Matthew Booth 28 Apr '10

28 Apr '10

We were seeing 100% failure rates transferring 10G disk images from ESX on a particular setup. We also weren't spotting the transfer failure, and dying with a strange error from libguestfs. These 2 patches fix the error check which should have made it obvious what was failing, and the underlying error.

2 3

[PATCH hivex] regedit: Add implicit nul-termination when importing strings.
by Richard W.M. Jones 28 Apr '10

28 Apr '10

-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://et.redhat.com/~rjones/libguestfs/ See what it can do: http://et.redhat.com/~rjones/libguestfs/recipes.html

2 2

vmdk support by libguestfs in RHEL and upstream
by Yufang Zhang 28 Apr '10

28 Apr '10

Hi all, As we are doing plan for testing libguestfs, I have a question about vmdk support for libguestfs. In Fedora 12, we can successfully manage a vmdk image via guestfish(add -> run -> mkfs -> mount -> ...). But in RHEL6, for qemu-kvm can not boot with a vmdk image, libguestfs can not manage vmdk either. So is it true that vmdk is only supported by upstream libguestfs but not the version in RHEL6? Thanks. --Yufang

3 4

[PATCH] Fix virt-v2v exit codes
by Matthew Booth 26 Apr '10

26 Apr '10

If something die()s, the error status can get lost in cleanup when various child processes exit and overwrite $?. Specifically, the RHEV target uses a child process for NFS access, and libguestfs launches a qemu child process. This change ensures that DESTROY and END blocks which perform operations on child processes explicitly preserve $?. --- lib/Sys/VirtV2V/Target/RHEV.pm | 10 ++++++++++ v2v/virt-v2v.pl | 7 +++++++ 2 files changed, 17 insertions(+), 0 deletions(-) diff --git a/lib/Sys/VirtV2V/Target/RHEV.pm b/lib/Sys/VirtV2V/Target/RHEV.pm index f14c54f..445893a 100644 --- a/lib/Sys/VirtV2V/Target/RHEV.pm +++ b/lib/Sys/VirtV2V/Target/RHEV.pm @@ -143,11 +143,16 @@ sub DESTROY { my $self = shift; + my $retval = $?; + # Make certain the child process dies with the object if (defined($self->{pid})) { kill(9, $self->{pid}); waitpid($self->{pid}, WNOHANG); + $retval ||= $?; } + + $? = $retval; } package Sys::VirtV2V::Target::RHEV::Vol; @@ -441,6 +446,8 @@ sub DESTROY { my $self = shift; + my $retval = $?; + my $eh = Sys::VirtV2V::ExecHelper->run('umount', $self->{mountdir}); if ($eh->status() != 0) { print STDERR user_message(__x("Failed to unmount {path}. Command ". @@ -449,6 +456,7 @@ sub DESTROY path => $self->{domain_path}, status => $eh->status(), output => $eh->output())); + $retval ||= $eh->status(); } rmdir($self->{mountdir}) @@ -456,6 +464,8 @@ sub DESTROY "{dir}: {error}", dir => $self->{mountdir}, error => $!)); + + $? = $retval; } =item create_volume(name, size) diff --git a/v2v/virt-v2v.pl b/v2v/virt-v2v.pl index 15d0d5b..7951303 100755 --- a/v2v/virt-v2v.pl +++ b/v2v/virt-v2v.pl @@ -350,6 +350,9 @@ exit(0); # We should always attempt to shut down the guest gracefully END { close_guest_handle(); + + # die() sets $? to 255, which is untidy. + $? = $? == 255 ? 1 : $?; } ############################################################################### @@ -364,10 +367,14 @@ sub close_guest_handle $g->umount_all(); $g->sync(); + my $retval = $? + # Note that this undef is what actually causes the underlying handle to # be closed. This is required to allow the RHEV target's temporary mount # directory to be unmounted and deleted prior to exit. $g = undef; + + $? ||= $retval; } } -- 1.6.6.1

3 5

guestfish 1.3.6 segfaults when you use the -a option ...
by Richard W.M. Jones 24 Apr '10

24 Apr '10

Version 1.3.7 corrects this, so don't use 1.3.6. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://et.redhat.com/~rjones/virt-df/

2 2

[PATCH] Config: Change config to lookup dependencies by name
by Matthew Booth 23 Apr '10

23 Apr '10

Conversion would fail if it was necessary to install a package, and multiple architectures of that package were already installed. This was happening specifically with device-mapper on RHEL 5 conversions. Unfortunately the flat dependency list in the config file didn't really allow this to be fixed. The best that could be done is to specify both i386 and x86_64 dependencies, but would mean attempted installation of the i386 version regardless of whether it was already installed. This would fail if the guest didn't have i386 dependencies installed, for example glibc. By making the dependency tree more explicit, we can not only more easily test for both i386 and x86_64 packages, but also more effectively prune what needs to be installed. This fixes RHBZ#583008 --- lib/Sys/VirtV2V/Config.pm | 3 +- lib/Sys/VirtV2V/GuestOS/RedHat.pm | 134 +++++++++++++++++++++++++++++-------- v2v/virt-v2v.conf | 48 ++++++++++--- 3 files changed, 144 insertions(+), 41 deletions(-) diff --git a/lib/Sys/VirtV2V/Config.pm b/lib/Sys/VirtV2V/Config.pm index d90c869..117b55b 100644 --- a/lib/Sys/VirtV2V/Config.pm +++ b/lib/Sys/VirtV2V/Config.pm @@ -107,8 +107,7 @@ sub get_transfer_iso # config file # We use a hash here to avoid duplicates my %path_args; - foreach my $path ($dom->findnodes('/virt-v2v/app/path/text() | '. - '/virt-v2v/app/dep/text()')) { + foreach my $path ($dom->findnodes('/virt-v2v/app/path/text()')) { $path = $path->getData(); # Get the absolute path if iso-root was defined diff --git a/lib/Sys/VirtV2V/GuestOS/RedHat.pm b/lib/Sys/VirtV2V/GuestOS/RedHat.pm index b0ef775..a973c19 100644 --- a/lib/Sys/VirtV2V/GuestOS/RedHat.pm +++ b/lib/Sys/VirtV2V/GuestOS/RedHat.pm @@ -462,7 +462,7 @@ sub add_kernel } } - my ($app, $deps); + my ($app, $depnames); eval { my $desc = $self->{desc}; @@ -475,7 +475,8 @@ sub add_kernel search => $search))); } - ($app, $deps) = $config->match_app($desc, $kernel_pkg, $kernel_arch); + ($app, $depnames) = + $config->match_app($desc, $kernel_pkg, $kernel_arch); }; # Return undef if we didn't find a kernel if ($@) { @@ -483,14 +484,12 @@ sub add_kernel return undef; } - return undef if($self->_is_installed($app)); + return undef if ($self->_newer_installed($app)); - my @install; - # Install any kernel dependencies which aren't already installed - foreach my $dep (@$deps) { - push(@install, $dep) unless($self->_is_installed($dep)); - } - $self->_install_rpms(1, @install); + my $user_arch = $kernel_arch eq 'i686' ? 'i386' : $kernel_arch; + + # Install any required kernel dependencies + $self->_install_rpms(1, $self->_get_deppaths($user_arch, @$depnames)); # Inspect the rpm to work out what kernel version it contains my $version; @@ -621,22 +620,17 @@ sub add_application my ($app, $deps) = $config->match_app($self->{desc}, $label, $user_arch); # Nothing to do if it's already installed - return if($self->_is_installed($app)); + return if ($self->_newer_installed($app)); my @install = ($app); # Add any dependencies which aren't already installed to the install set - foreach my $dep (@$deps) { - push(@install, $dep) unless ($self->_is_installed($dep)); - } + push(@install, $self->_get_deppaths($user_arch, @$deps)); $self->_install_rpms(1, @install); } - -# Return 1 if the requested rpm, or a newer version, is installed -# Return 0 otherwise -sub _is_installed +sub _get_nevra { my $self = shift; my ($rpm) = @_; @@ -657,8 +651,15 @@ sub _is_installed # Ensure epoch is always numeric $epoch = 0 if('(none)' eq $epoch); - # Search installed rpms matching <name>.<arch> - my $found = 0; + return ($name, $epoch, $version, $release, $arch); +} + +sub _get_installed +{ + my $self = shift; + my ($name, $arch) = @_; + + my $g = $self->{g}; my $rpmcmd = ['rpm', '-q', '--qf', '%{EPOCH} %{VERSION} %{RELEASE}\n', "$name.$arch"]; @@ -678,32 +679,60 @@ sub _is_installed # a real error. my $error = $g->sh("LANG=C '".join("' '", @$rpmcmd)."' 2>&1 ||:"); - return 0 if ($error =~ /not installed/); + return () if ($error =~ /not installed/); die(user_message(__x("Error running {command}: {error}", command => join(' ', @$rpmcmd), error => $error))); } + my @installed; foreach my $installed (@output) { $installed =~ /^(\S+)\s+(\S+)\s+(\S+)$/ or die("Unexpected return from rpm command: $installed"); - my ($iepoch, $iversion, $irelease) = ($1, $2, $3); + my ($epoch, $version, $release) = ($1, $2, $3); # Ensure iepoch is always numeric - $iepoch = 0 if('(none)' eq $iepoch); + $epoch = 0 if('(none)' eq $epoch); + + push(@installed, [$epoch, $version, $release]); + } + + return @installed; +} + + +# Return 1 if the requested rpm, or a newer version, is installed +# Return 0 otherwise +sub _newer_installed +{ + my $self = shift; + my ($rpm) = @_; + + my $g = $self->{g}; + + my ($name, $epoch, $version, $release, $arch) = $self->_get_nevra($rpm); + + my @installed = $self->_get_installed($name, $arch); + + # Search installed rpms matching <name>.<arch> + my $found = 0; + foreach my $pkg (@installed) { + my $iepoch = $pkg->[0]; + my $iversion = $pkg->[1]; + my $irelease = $pkg->[2]; # Skip if installed epoch less than requested version - next if($iepoch < $epoch); + next if ($iepoch < $epoch); - if($iepoch eq $epoch) { + if ($iepoch eq $epoch) { # Skip if installed version less than requested version - next if(_rpmvercmp($iversion, $version) < 0); + next if (_rpmvercmp($iversion, $version) < 0); # Skip if install version == requested version, but release less # than requested release - if($iversion eq $version) { - next if(_rpmvercmp($irelease,$release) < 0); + if ($iversion eq $version) { + next if (_rpmvercmp($irelease, $release) < 0); } } @@ -713,6 +742,57 @@ sub _is_installed return $found; } +# Return a list of dependency paths which need to be installed for the given +# apps +sub _get_deppaths +{ + my $self = shift; + my ($arch, @apps) = @_; + + my $desc = $self->{desc}; + my $config = $self->{config}; + + my %required; + foreach my $app (@apps) { + my ($path, $deps) = $config->match_app($desc, $app, $arch); + + if (!$self->_newer_installed($path)) { + $required{$path} = 1; + + foreach my $deppath ($self->_get_deppaths($arch, @$deps)) { + $required{$deppath} = 1; + } + } + + # For x86_64, also check if there is any i386 version installed. If + # there is, check if it needs to be upgraded. + if ($arch eq 'x86_64') { + $path = undef; + $deps = undef; + + # It's not an error if no i386 package is available + eval { + ($path, $deps) = $config->match_app($desc, $app, 'i386'); + }; + + if (defined($path) && !$self->_newer_installed($path)) { + my ($name, undef, undef, undef, $arch) = + $self->_get_nevra($path); + + if ($self->_get_installed($name, $arch) > 0) { + $required{$path} = 1; + + foreach my $deppath ($self->_get_deppaths('i386', @$deps)) { + $required{$deppath} = 1; + } + } + } + } + } + + return keys(%required); +} + # An implementation of rpmvercmp. Compares two rpm version/release numbers and # returns -1/0/1 as appropriate. # Note that this is intended to have the exact same behaviour as the real diff --git a/v2v/virt-v2v.conf b/v2v/virt-v2v.conf index 6b76f79..0ea3c9f 100644 --- a/v2v/virt-v2v.conf +++ b/v2v/virt-v2v.conf @@ -7,24 +7,48 @@ 5 to support VirtIO --> <app os='rhel' major='5' arch='i686' name='kernel'> <path>rhel/5/kernel-2.6.18-128.el5.i686.rpm</path> - <dep>rhel/5/ecryptfs-utils-56-8.el5.i386.rpm</dep> - <dep>rhel/5/lvm2-2.02.40-6.el5.i386.rpm</dep> - <dep>rhel/5/device-mapper-1.02.28-2.el5.i386.rpm</dep> - <dep>rhel/5/device-mapper-event-1.02.28-2.el5.i386.rpm</dep> + <dep>ecryptfs-utils</dep> + <dep>lvm2</dep> </app> <app os='rhel' major='5' arch='i686' name='kernel-PAE'> <path>rhel/5/kernel-PAE-2.6.18-128.el5.i686.rpm</path> - <dep>rhel/5/ecryptfs-utils-56-8.el5.i386.rpm</dep> - <dep>rhel/5/lvm2-2.02.40-6.el5.i386.rpm</dep> - <dep>rhel/5/device-mapper-1.02.28-2.el5.i386.rpm</dep> - <dep>rhel/5/device-mapper-event-1.02.28-2.el5.i386.rpm</dep> + <dep>ecryptfs-utils</dep> + <dep>lvm2</dep> </app> <app os='rhel' major='5' arch='x86_64' name='kernel'> <path>rhel/5/kernel-2.6.18-128.el5.x86_64.rpm</path> - <dep>rhel/5/ecryptfs-utils-56-8.el5.x86_64.rpm</dep> - <dep>rhel/5/lvm2-2.02.40-6.el5.x86_64.rpm</dep> - <dep>rhel/5/device-mapper-1.02.28-2.el5.x86_64.rpm</dep> - <dep>rhel/5/device-mapper-event-1.02.28-2.el5.x86_64.rpm</dep> + <dep>ecryptfs-utils</dep> + <dep>lvm2</dep> + </app> + +  + <app os='rhel' major='5' arch='x86_64' name='ecryptfs-utils'> + <path>rhel/5/ecryptfs-utils-56-8.el5.x86_64.rpm</path> + </app> + <app os='rhel' major='5' arch='i386' name='ecryptfs-utils'> + <path>rhel/5/ecryptfs-utils-56-8.el5.i386.rpm</path> + </app> + <app os='rhel' major='5' arch='x86_64' name='lvm2'> + <path>rhel/5/lvm2-2.02.40-6.el5.x86_64.rpm</path> + <dep>device-mapper</dep> + <dep>device-mapper-event</dep> + </app> + <app os='rhel' major='5' arch='i386' name='lvm2'> + <path>rhel/5/lvm2-2.02.40-6.el5.i386.rpm</path> + <dep>device-mapper</dep> + <dep>device-mapper-event</dep> + </app> + <app os='rhel' major='5' arch='x86_64' name='device-mapper'> + <path>rhel/5/device-mapper-1.02.28-2.el5.x86_64.rpm</path> + </app> + <app os='rhel' major='5' arch='i386' name='device-mapper'> + <path>rhel/5/device-mapper-1.02.28-2.el5.i386.rpm</path> + </app> + <app os='rhel' major='5' arch='x86_64' name='device-mapper-event'> + <path>rhel/5/device-mapper-event-1.02.28-2.el5.x86_64.rpm</path> + </app> + <app os='rhel' major='5' arch='i386' name='device-mapper-event'> + <path>rhel/5/device-mapper-event-1.02.28-2.el5.i386.rpm</path> </app> <!-- RHEL 4 -- 1.6.6.1

1 2