October 2015 - Libguestfs - libguestfs List Archives

Plan for libguestfs 1.30
by Richard W.M. Jones 14 Dec '15

14 Dec '15

It has been a ludicrously long time (over 8 months) since the last stable release of libguestfs. So I'd like to plan a new 1.30 release soon. Please follow-up if there are features / blockers / bugs that need to be addressed for 1.30. As usual, bugs which have "1.30" (without quotes) in the Devel Whiteboard field in Bugzilla are nominated as blockers for the release. You can see a list of those here [currently empty]: https://bugzilla.redhat.com/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&… For a list of all bugs, go to: https://bugzilla.redhat.com/buglist.cgi?component=libguestfs&product=Virtua… ---------------------------------------------------------------------- It's also worth talking about what I'd like to do *after* 1.30 is out. This includes: - threading support https://www.redhat.com/archives/libguestfs/2015-June/thread.html#00118 - external tests https://www.redhat.com/archives/libguestfs/2014-October/thread.html#00042 While thread safety is oft talked about, it's external tests which I consider to be the more important patch series, since it will allow us to build and test libguestfs more easily and more comprehensively for Fedora and RHEL. Unfortunately it's the most difficult and invasive change of the two. You may have your own ideas, if so follow up here. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top

3 6

SELinux relabel API
by Richard W.M. Jones 10 Dec '15

10 Dec '15

[ I realized that we were discussing adding this feature, in various private email, IRC, and this long bugzilla thread: https://bugzilla.redhat.com/show_bug.cgi?id=1060423 That's not how we should do things. Let's discuss it on the mailing list. ] One thing that virt-customize/virt-sysprep/virt-builder have to do is relabel SELinux guests. What we do at the moment is run: if load_policy && fixfiles restore; then rm -f /.autorelabel else touch /.autorelabel echo '%s: SELinux relabelling failed, will relabel at boot instead.' fi while chrooted into the guest (using the 'guestfs_sh' API). This has a number of problems: - It has to load the policy using 'load_policy', but this doesn't work sometimes: * RHEL 5 load_policy takes a parameter. * Doesn't work if appliance kernel is significantly different from guest kernel version, because the binary policy format changes irregularly and is not backwards compatible. * Requires the appliance [host] kernel to be compiled with LSM/SELinux support. - Touching /.autorelabel is often broken, eg. it's broken in Fedora 20 because of systemd (RHBZ#1049656). - /etc/resolv.conf will not be relabelled if guestfs network is on, because of resolv.conf shenanigans in libguestfs.git/daemon/command.c - It requires running guest code, which we'd like to avoid. What would be nice would be to have an API to just do this relabelling. Libguestfs could change this API as required to handle different guests. Dan Walsh helpfully pointed out to us that we've been doing it wrong all along :-) A much better way to relabel is to run: setfiles /etc/selinux/targeted/contexts/files/file_contexts DIR where 'file_contexts' is a file which contains the default labels for files (a set of regexps), and 'DIR' is the directory at which relabelling starts. Note that 'setfiles' would be the libguestfs appliance binary, so no guest binary needs to be run. A simple API could just look like this: guestfs_selinux_relabel (g); which would always use the 'targeted' policy from the guest, and always start relabelling at the root. This would work fine for virt-builder. For Colin's requirements for Project Atomic, I suspect he will want to be able to set the file_contexts file and the root directory, but I'll leave him to describe what would be useful. A couple of notes: - I'd like to avoid baking in assumptions from the 'setfiles' command as far as possible. libguestfs APIs last for many years and some have caused us many years of regret (but that's our job) :-/ - Is it a good idea to tie this into inspection in some way -- for example, inspection could provide us with the path to the current or default SELinux policy. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top

5 10

Re: [Libguestfs] Fwd: [PATCH] v2v: virtio-win: include *.dll too
by Vadim Rozenfeld 03 Dec '15

03 Dec '15

On Tue, 2015-10-27 at 11:00 -0400, Amnon Ilan wrote: > Hi Jeff, > > There were some questions on the libguestfs mailing list regarding virtio-win iso an packaging (see below). > Could you answer it on the list? (just send it to libguestfs(a)redhat.com , no need to register, keep us CCed) > The people asking are from Virtuozzo, they are not customers or official partners, just good contributors upstream. > Note that this mailing list is public. > > Thanks, > Amnon > > > > ----- Forwarded Message ----- > > From: "Richard W.M. Jones" <rjones(a)redhat.com> > > To: "Roman Kagan" <rkagan(a)virtuozzo.com>, libguestfs(a)redhat.com, "Denis Lunev" <den(a)virtuozzo.com> > > Cc: "Amnon Ilan" <ailan(a)redhat.com> > > Sent: Tuesday, October 27, 2015 4:11:10 PM > > Subject: Re: [PATCH] v2v: virtio-win: include *.dll too > > > > On Tue, Oct 27, 2015 at 04:45:00PM +0300, Roman Kagan wrote: > > > On Tue, Oct 27, 2015 at 12:02:40PM +0000, Richard W.M. Jones wrote: > > > > On Tue, Oct 27, 2015 at 02:08:42PM +0300, Roman Kagan wrote: > > > > > On Tue, Oct 27, 2015 at 09:12:41AM +0000, Richard W.M. Jones wrote: > > > > > > On Mon, Oct 26, 2015 at 09:00:03PM +0300, Roman Kagan wrote: > > > > > > > Windows QXL drivers include also qxldd.dll which used to get > > > > > > > filtered > > > > > > > out and not copied over into the guest. As a result QXL driver > > > > > > > failed > > > > > > > to install due to a missing file. > > > > > > > (* Skip files without specific extensions. *) > > > > > > > - let extensions = ["cat"; "inf"; "pdb"; "sys"] in > > > > > > > + let extensions = ["cat"; "dll"; "inf"; "pdb"; "sys"] in > > > > > > > > > > > > Actually no, this part is (may be) wrong. > > > > > > > > > > > > This causes WdfCoInstaller*.dll from the ISO to be copied in. It's > > > > > > not clear to me whether these files are needed - I suspect not. > > > > > Having WdfCoInstaller* is absolutely necessary for successful installation any of our WDF based drivers (serial, balloon, and rng). > > > > > Yes it is, see the corresponding *.inf where it's mentioned in the > > > > > corresponding CopyFiles directive (and yes, I just verified that with > > > > > that file missing the balloon driver installation fails with 'file not > > > > > found' error). > > > > > > > > Maybe we should be looking at the CopyFiles directive? (As you may be > > > > able to guess, I know next to nothing about how Windows drivers work). > > > > > > So do I, and parsing the *.inf files is the last thing I'd like to do > > > here; I'm not sure it can be made reliable (I mean, not the parsing, but > > > making any sense out of the parsed data). > > > > > > However I'm now facing a more serious problem: when I actually went > > > ahead and looked into the virio-win iso packaged in the rhel virtio-win > > > rpm, I found out that its contents differed significantly from what was > > > packaged in that very same rpm as a driver directory hierarchy. > > > > > > Namely, > > > > > > - there are no qxl drivers on the iso, but they are present in the rpm > > > > > > - there are no balloon, qemupciserial, viorng, vioserial in the rpm, > > > but they are present on the iso > > > > > > - qemupciserial is an inf-only 'driver' which matches all windows > > > versions; it's just one copy at the sub-toplevel directory, so it > > > won't pass the virtio_iso_path_matches_guest_os() criteria > > > > > > - there are a lot of duplicates between files for different windows > > > flavors It's due to historical reasons mostly. The best way would be having a set of separate distribution images packaged on per-platform base. Vadim. > > > > > > To sum up, the packaging and naming policy of the virtio-win rpm and the > > > virtio-win iso therein are different and neither is clear. Hardcoding > > > the policy in v2v without actually knowing it appears risky at best. > > > > > > We need guidance here from someone who knows how that virtio-win stuff > > > is packaged and how different it is across distros. Any idea whom to > > > contact on the matter? > > > > Amnon -- CC'd -- I guess? > > > > I'm aware that there are at least some differences in paths, and the > > current virt-v2v code should be able to cope for the two drivers that > > we really care about - viostor and virtio-net. Of course it can't > > install drivers that don't exist on either the ISO or the rpm. > > > > Rich. > > > > > > Anyhow, can you fix up the test at least. > > > > > > Sure, but I'd like first to figure out how to fix up the code ;) > > > > > > Roman. > > > > -- > > Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones > > Read my programming and virtualization blog: http://rwmj.wordpress.com > > virt-df lists disk usage of guests without needing to install any > > software inside the virtual machine. Supports Linux and Windows. > > http://people.redhat.com/~rjones/virt-df/ > >

8 41

P2V conversion failed with "/run/lvm/lvmetad.socket: connect failed: No such file or directory"
by Tejas Gadaria 05 Nov '15

05 Nov '15

Hi, We are trying to do P2V conversion with virt-p2v. we have conversion server (virt-p2v) and physical server (virt-p2v) server configured as per below documentation. http://libguestfs.org/virt-p2v.1.html#kernel-command-line-configuration After "Start Conversion" from GUI interface, we are conversion fails with "/run/lvm/lvmetad.socket: connect failed: No such file or directory" Error. Logs are attached for more details. Also FYI, we are using BL460c Gen8 server. Any help will be appreciated. Thanks, Tejas

5 19

[PATCH 0/7] v2v: Miscellaneous refactorings.
by Richard W.M. Jones 29 Oct '15

29 Oct '15

Just refactoring. Rich.

1 7

[nbdkit] can't import mmap
by Jason Pepas 29 Oct '15

29 Oct '15

Hi, I'm new to nbdkit and am trying to run a simple "single file block device using mmap" setup, using this python script: import mmap def config(key, value): print ("ignored parameter %s=%s" % (key, value)) def open(readonly): print ("open: readonly=%d" % readonly) f = open("disk","r+") m = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_WRITE) return (f,m) def get_size(h): size = 1024*1024 def pread(h, count, offset): (f,m) = h bytes = bytearray(m[offset:offset+count]) return bytes def pwrite(h, buf, offset): (f,m) = h m[offset:offset+len(buf)] = bytearray(buf) When I run this: /usr/sbin/nbdkit -f -v python script=single_file.py I get this error: nbdkit: debug: registering /usr/lib/x86_64-linux-gnu/nbdkit/plugins/nbdkit-python-plugin.so nbdkit: debug: registered /usr/lib/x86_64-linux-gnu/nbdkit/plugins/nbdkit-python-plugin.so (name python) nbdkit: debug: /usr/lib/x86_64-linux-gnu/nbdkit/plugins/nbdkit-python-plugin.so: load nbdkit: debug: /usr/lib/x86_64-linux-gnu/nbdkit/plugins/nbdkit-python-plugin.so: config key=script, value=single_file.py Traceback (most recent call last): File "single_file.py", line 1, in <module> import mmap ImportError: /usr/lib/python2.7/lib-dynload/mmap.x86_64-linux-gnu.so: undefined symbol: PyExc_SystemError nbdkit: error: single_file.py: error running this script Any ideas? Thanks, Jason Pepas

2 2

[PATCH 00/16] Refactoring of configure.ac and guestfs.pod
by Richard W.M. Jones 29 Oct '15

29 Oct '15

Two (not related to each other) refactorings: Patches 1-12 split configure.ac into smaller files using the m4_include mechanism. Patches 13-15 split out parts of guestfs.pod (ie. guestfs(3)) into three new manual pages: guestfs-hacking(3) - how to extend and contribute to libguestfs guestfs-internals(3) - architecture and internals guestfs-security(3) - security and CVEs Patch 16 is a minor tidy up of guestfs.pod. Rich.

1 16

vcenter convert OpenStack
by yang peng 29 Oct '15

29 Oct '15

HI: Dear all. I met a problem about libguest-tools, hope to get your help. even: OS: Fedora 22 rpm version: libguestfs-tools 1.30.3 vrit-v2v 1.30.3 problome: I convert a suse_guest from vcenter to the kvm. export LIBGUESTFS_BACKEND=direct suse_guest: file (suse.vmdk1 suse.my suse.ovf) step1. zip suse-disk1.vmdk.gz suse.my suse.ovf step2. virt-v2v -i ova suse-disk1.vmdk.gz -of qcow2 -o local -os /var/tmp step3: generate suse-sda and suse.xml in /var/tmp/ step4: use suse-sda disk lanch an instance command: /usr/libexec/qemu-kvm -m 4096 -drive file=/var/tmp/suse-sda -net nic,model=virtio -net user -nographic -vnc :10 -usbdevice tablet setep5: I found it the process of virtual chance done in VNC， The VG00 cannot be found， on the other hand,redhat 4 5 6 7 is ok to convert. I am sorry ，I am a non-native English speakers. Thank you very much ,Looking forward to your reply mailing.

2 1

[PATCH] configure: Move language binding detection to separate files.
by Richard W.M. Jones 27 Oct '15

27 Oct '15

This commit starts to split our massive, monolithic configure.ac file into smaller files, using the m4_include mechanism to combine them. I don't know if we should really do this, so I'm open to comments about it. However: - Our configure.ac script is 1800+ lines long, and that's pretty long. - configure.ac lacks structure; splitting it up might improve that. - From what I read, m4_include is a simple include device, so this shouldn't break anything (https://autotools.io/autoconf/macros.html) Rich.

1 1

[PATCHv2] Added xfs support to vfs_minimum_size.
by Maxim Perevedentsev 27 Oct '15

27 Oct '15

--- daemon/daemon.h | 1 + daemon/fs-min-size.c | 7 +++++++ daemon/xfs.c | 16 ++++++++++++++++ generator/actions.ml | 6 +++++- 4 files changed, 29 insertions(+), 1 deletion(-) diff --git a/daemon/daemon.h b/daemon/daemon.h index 4a969dd..1f0cd30 100644 --- a/daemon/daemon.h +++ b/daemon/daemon.h @@ -269,6 +269,7 @@ extern int copy_xattrs (const char *src, const char *dest); extern int xfs_set_uuid (const char *device, const char *uuid); extern int xfs_set_uuid_random (const char *device); extern int xfs_set_label (const char *device, const char *label); +extern int64_t xfs_minimum_size (const char *path); /*-- debug-bmap.c --*/ extern char *debug_bmap (const char *subcmd, size_t argc, char *const *const argv); diff --git a/daemon/fs-min-size.c b/daemon/fs-min-size.c index ca71c4d..ba0f739 100644 --- a/daemon/fs-min-size.c +++ b/daemon/fs-min-size.c @@ -73,6 +73,13 @@ do_vfs_minimum_size (const mountable_t *mountable) r = btrfs_minimum_size (path); } + else if (STREQ (vfs_type, "xfs")) { + CLEANUP_FREE char *path = get_mount_point (mountable->device); + if (path == NULL) + return -1; + r = xfs_minimum_size (path); + } + else NOT_SUPPORTED (-1, "don't know how to get minimum size of '%s' filesystems", vfs_type); diff --git a/daemon/xfs.c b/daemon/xfs.c index f748902..abc2736 100644 --- a/daemon/xfs.c +++ b/daemon/xfs.c @@ -660,3 +660,19 @@ do_xfs_repair (const char *device, return r; } + +int64_t +xfs_minimum_size (const char *path) +{ + CLEANUP_FREE guestfs_int_xfsinfo *info = do_xfs_info (path); + + if (info == NULL) + return -1; + + // XFS does not support shrinking. + if (INT64_MAX / info->xfs_blocksize < info->xfs_datablocks) { + reply_with_error ("filesystem size too big: overflow"); + return -1; + } + return info->xfs_blocksize * info->xfs_datablocks; +} diff --git a/generator/actions.ml b/generator/actions.ml index 8832410..d8af08d 100644 --- a/generator/actions.ml +++ b/generator/actions.ml @@ -12765,6 +12765,10 @@ To read the UUID on a filesystem, call C<guestfs_vfs_uuid>." }; [["mkfs"; "btrfs"; "/dev/sda1"; ""; "NOARG"; ""; ""; "NOARG"]; ["mount"; "/dev/sda1"; "/"]; ["vfs_minimum_size"; "/dev/sda1"]]), []; + InitPartition, Always, TestRun ( + [["mkfs"; "xfs"; "/dev/sda1"; ""; "NOARG"; ""; ""; "NOARG"]; + ["mount"; "/dev/sda1"; "/"]; + ["vfs_minimum_size"; "/dev/sda1"]]), []; ]; shortdesc = "get minimum filesystem size"; longdesc = "\ @@ -12774,7 +12778,7 @@ This is the minimum possible size for filesystem shrinking. If getting minimum size of specified filesystem is not supported, this will fail and set errno as ENOTSUP. -See also L<ntfsresize(8)>, L<resize2fs(8)>, L<btrfs(8)>." }; +See also L<ntfsresize(8)>, L<resize2fs(8)>, L<btrfs(8)>, L<xfs_info(8)>." }; ] -- 1.8.3.1

2 1