May 2014 - Libguestfs - libguestfs List Archives

SELinux relabel API
by Richard W.M. Jones 10 Dec '15

10 Dec '15

[ I realized that we were discussing adding this feature, in various private email, IRC, and this long bugzilla thread: https://bugzilla.redhat.com/show_bug.cgi?id=1060423 That's not how we should do things. Let's discuss it on the mailing list. ] One thing that virt-customize/virt-sysprep/virt-builder have to do is relabel SELinux guests. What we do at the moment is run: if load_policy && fixfiles restore; then rm -f /.autorelabel else touch /.autorelabel echo '%s: SELinux relabelling failed, will relabel at boot instead.' fi while chrooted into the guest (using the 'guestfs_sh' API). This has a number of problems: - It has to load the policy using 'load_policy', but this doesn't work sometimes: * RHEL 5 load_policy takes a parameter. * Doesn't work if appliance kernel is significantly different from guest kernel version, because the binary policy format changes irregularly and is not backwards compatible. * Requires the appliance [host] kernel to be compiled with LSM/SELinux support. - Touching /.autorelabel is often broken, eg. it's broken in Fedora 20 because of systemd (RHBZ#1049656). - /etc/resolv.conf will not be relabelled if guestfs network is on, because of resolv.conf shenanigans in libguestfs.git/daemon/command.c - It requires running guest code, which we'd like to avoid. What would be nice would be to have an API to just do this relabelling. Libguestfs could change this API as required to handle different guests. Dan Walsh helpfully pointed out to us that we've been doing it wrong all along :-) A much better way to relabel is to run: setfiles /etc/selinux/targeted/contexts/files/file_contexts DIR where 'file_contexts' is a file which contains the default labels for files (a set of regexps), and 'DIR' is the directory at which relabelling starts. Note that 'setfiles' would be the libguestfs appliance binary, so no guest binary needs to be run. A simple API could just look like this: guestfs_selinux_relabel (g); which would always use the 'targeted' policy from the guest, and always start relabelling at the root. This would work fine for virt-builder. For Colin's requirements for Project Atomic, I suspect he will want to be able to set the file_contexts file and the root directory, but I'll leave him to describe what would be useful. A couple of notes: - I'd like to avoid baking in assumptions from the 'setfiles' command as far as possible. libguestfs APIs last for many years and some have caused us many years of regret (but that's our job) :-/ - Is it a good idea to tie this into inspection in some way -- for example, inspection could provide us with the path to the current or default SELinux policy. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top

5 10

Re: [Libguestfs] [PATCH 2/2] Use setfiles from the appliance for the SELinux relabel (RHBZ#1089100).
by Richard W.M. Jones 07 Aug '14

07 Aug '14

[Including libguestfs mailing list this time] On Tue, May 27, 2014 at 06:05:15AM -0700, Colin Walters wrote: > On Tue, May 27, 2014, at 02:04 AM, Richard W.M. Jones wrote: > > > > I'm still anxious for Colin to tell us if this API is suitable for his > > needs. > > I'd like an API that allows me to only relabel *unlabeled* files. > The use case here is: I have an existing disk image with an OS, > I want to inject e.g. a systemd service into it. If I do this offline > from libguestfs, the injected /usr/libexec/mydaemon and > /usr/lib/systemd/system/mydaemon.service > won't be labeled, but everything else will be. > > > I'm guessing that OStree does not have /etc/selinux/config, > > Right, it's in the "deployment root" of > /ostree/deploy/$osname/deploy/$checksum/etc/selinux/config Got it: ><fs> ll /ostree/deploy/project-atomic-controller/deploy/afc1794b4b42df77edf1988897b167573b99e299fa39a15b07b235a0e7387d02.0/etc/selinux/targeted/contexts/files/file_contexts -rw-r--r--. 1 root root 352240 Apr 14 20:14 /sysroot/ostree/deploy/project-atomic-controller/deploy/afc1794b4b42df77edf1988897b167573b99e299fa39a15b07b235a0e7387d02.0/etc/selinux/targeted/contexts/files/file_contexts > To figure that out you'd want to use the OSTree APIs; and then it > introduces > further questions around *which* deployments you want to relabel. All? > Only > one (the default?). > > What I do currently in my scripts is only relabel the default, and that > would > be the best default for an API. > > But a totally valid thing to do with OSTree is - say you're running > RHEL7, > and you want to check whether the latest Fedora kernel fixes an issue > you're seeing. You can use ostree to dynamically parallel install > Fedora content in a new deployment root, try it with near-total > safety[1], > and then if it doesn't work, just delete it and free up the space. So I think an API which looks like this ... required params: None optional params: path => Either a directory to be relabelled recursively, or a single file (defaults to "/"). root => Inspection root of guest. Optional, only makes sense when 'contexts' param is *omitted*. contexts => The `file_contexts' file. Defaults to /etc/selinux/$selinux_type/contexts/files/file_contexts OSTree would probably want to pass: /ostree/deploy/$osname/deploy/$checksum/etc/selinux/targeted/contexts/files/file_contexts Inspection could be updated to parse /etc/selinux/config in order to get the default SELinux policy and pass it back through an API such as `inspect-get-selinux-type'. If 'contexts' is omitted, 'root' must be supplied, and it causes an internal call to guestfs_inspect_get_selinux_type (g, root) in order to get the default policy. What do you think? Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW

2 3

Adventures in building libguestfs on non-x86 architectures for Debian
by Hilko Bengen 06 Jul '14

06 Jul '14

Hi, things are progressing slowly, but I feel that eventually I'll be getting there: supermin can be built on all Linux-based architectures that are part of Debian/unstable[1] -- (except for sparc, but I'll ignore that for now.) libguestfs on the other hand currently fails launching the appliance (necessary for running tests) on everything but x86 and mips. Apparently, qemu-system-arm 2.0 as currently available through Debian/unstable doesn't want to tell us anything about available devices unless we specify a machine type. The patch below works around that issue, but I haven't been able to run qemu-system-arm with a supermin appliance on Debian's armhf porterbox so far: 100% CPU usage, no output from the kernel. For armel, there's no kernel with device tree blobs, so I think that I'll need to use linux-image-versatile there or ignore armel altogether. powerpc: qemu-system-ppc does not know about the "pseries" machine, I have figured out that I need to use qemu-system-ppc64 for that. So far, I haven't been able to manually boot a Supermin appliance using qemu-system-ppc, either, though. s390x does not seem to know about virtio-blk-pci. Cheers, -Hilko [1] https://buildd.debian.org/status/package.php?p=supermin [2] https://buildd.debian.org/status/package.php?p=libguestfs diff --git a/src/launch-direct.c b/src/launch-direct.c index 1460c56..2332368 100644 --- a/src/launch-direct.c +++ b/src/launch-direct.c @@ -1020,6 +1020,10 @@ test_qemu (guestfs_h *g, struct backend_direct_data *data) guestfs___cmd_add_arg (cmd3, g->hv); guestfs___cmd_add_arg (cmd3, "-display"); guestfs___cmd_add_arg (cmd3, "none"); +#ifdef MACHINE_TYPE + guestfs___cmd_add_arg (cmd3, "-M"); + guestfs___cmd_add_arg (cmd3, MACHINE_TYPE); +#endif guestfs___cmd_add_arg (cmd3, "-machine"); guestfs___cmd_add_arg (cmd3, "accel=kvm:tcg"); guestfs___cmd_add_arg (cmd3, "-device");

2 3

[PATCH] library: per-handle locking support
by Maros Zatko 06 Jun '14

06 Jun '14

Introduces private pool of muteces (per_handle_locks) guarded by read-write mutex (locks_lock) using guestfs__per_handle_lock_lock(g) and guestfs__per_handle_lock_unlock(g), where g is current valid guestfs_h handle. Above two are used in generator/c.ml at begin and end of each non-deamon call. New lock can be created using guestfs__per_handle_lock_add(g) and destroyed with guestfs__per_handle_lock_remove(g). These two are called automatically too. First one is called by guestfs_create and second by guestfs_destroy. --- generator/c.ml | 10 ++++ src/Makefile.am | 1 + src/handle.c | 4 ++ src/locking.c | 162 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ src/locking.h | 45 ++++++++++++++++ 5 files changed, 222 insertions(+) create mode 100644 src/locking.c create mode 100644 src/locking.h diff --git a/generator/c.ml b/generator/c.ml index ee276dc..e67053e 100644 --- a/generator/c.ml +++ b/generator/c.ml @@ -788,6 +788,9 @@ and generate_internal_actions_h () = pr "#define GUESTFS_INTERNAL_ACTIONS_H_\n"; pr "\n"; + pr "#include \"locking.h\"\n"; + pr "\n"; + List.iter ( fun { c_name = c_name; style = style } -> generate_prototype ~single_line:true ~newline:true ~handle:"g" @@ -1569,6 +1572,9 @@ and generate_client_actions hash () = handle_null_optargs optargs c_name; + pr " guestfs___per_handle_lock_lock (g);\n"; + pr "\n"; + pr " int trace_flag = g->trace;\n"; pr " struct trace_buffer trace_buffer;\n"; (match ret with @@ -1617,6 +1623,10 @@ and generate_client_actions hash () = trace_return name style "r"; ); pr "\n"; + + pr " guestfs___per_handle_lock_unlock (g);\n"; + pr "\n"; + pr " return r;\n"; pr "}\n"; pr "\n" diff --git a/src/Makefile.am b/src/Makefile.am index 3d06203..f1f42d0 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -118,6 +118,7 @@ libguestfs_la_SOURCES = \ libvirt-auth.c \ libvirt-domain.c \ listfs.c \ + locking.c \ lpj.c \ match.c \ osinfo.c \ diff --git a/src/handle.c b/src/handle.c index 719bbcd..8397157 100644 --- a/src/handle.c +++ b/src/handle.c @@ -84,6 +84,8 @@ guestfs_create_flags (unsigned flags, ...) g = calloc (1, sizeof (*g)); if (!g) return NULL; + guestfs___per_handle_lock_add (g); + g->state = CONFIG; g->conn = NULL; @@ -305,6 +307,8 @@ guestfs_close (guestfs_h *g) return; } + guestfs___per_handle_lock_remove (g); + /* Remove the handle from the handles list. */ if (g->close_on_exit) { gl_lock_lock (handles_lock); diff --git a/src/locking.c b/src/locking.c new file mode 100644 index 0000000..2a35fce --- /dev/null +++ b/src/locking.c @@ -0,0 +1,162 @@ +/* libguestfs + * Copyright (C) 2009-2014 Red Hat Inc. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include <config.h> + +#include <stdio.h> +#include <stdlib.h> +#include <unistd.h> +#include <string.h> +#include <errno.h> + +#include "glthread/lock.h" +#include "ignore-value.h" + +#include "guestfs.h" +#include "guestfs-internal.h" +#include "guestfs-internal-actions.h" +#include "guestfs_protocol.h" + +#include "locking.h" + +static gl_lock_t guestfs___lookup_lock_or_abort (guestfs_h *g, int *pos); +static void guestfs___per_handle_lock_is_not_held (guestfs_h *g); +static void ___shrink_lock_arrays (void); + + +gl_lock_define (static, *per_handle_locks) +gl_rwlock_define_initialized (static, locks_lock) + +/* items in handles array bijectively corresponds to items in per_handle_locks + */ +static guestfs_h ** handles; +static size_t n_handles; + + +static gl_lock_t +guestfs___lookup_lock_or_abort (guestfs_h *g, int *pos) +{ + // this function should be called with locks_lock already held + for (size_t i = 0; i < n_handles; i++) { + if (handles[i] == g) { + if (pos != NULL) { + *pos = i; + } + return per_handle_locks[i]; + } + } + // handle not found. aborting + error (g, _("guestfs lock handle not found")); + abort (); +} + +static void +guestfs___per_handle_lock_is_not_held (guestfs_h *g) +{ + // this function should be called with locks_lock already held + + int i; + gl_lock_t l = guestfs___lookup_lock_or_abort (g, &i); + + gl_lock_lock (l); + gl_lock_unlock (l); +} + +void +guestfs___per_handle_lock_add (guestfs_h *g) +{ + gl_rwlock_wrlock (locks_lock); + + n_handles++; + per_handle_locks = realloc (per_handle_locks, + sizeof (*per_handle_locks) * n_handles); + handles = realloc (handles, sizeof (*handles) * n_handles); + + gl_lock_init (per_handle_locks[n_handles - 1]); + handles[n_handles - 1] = g; + + gl_rwlock_unlock (locks_lock); +} + +static void +___shrink_lock_arrays (void) +{ + // locks_lock should be held + + int i; + guestfs___lookup_lock_or_abort (NULL, &i); + + memmove (handles + i, handles + i + 1, (n_handles - i - 1) + * sizeof (*handles)); + memmove (per_handle_locks + i, per_handle_locks + i + 1, + (n_handles - i - 1) * sizeof (*per_handle_locks)); + if (handles == NULL || per_handle_locks == NULL) { + abort (); + } +} + +void +guestfs___per_handle_lock_remove (guestfs_h *g) +{ + // locks_lock prevents per_handle_locks manipulation within this module + // take write lock, since we're going to update guarded per_handle_locks + gl_rwlock_wrlock (locks_lock); + + // ensure we don't get into data race when removing lock + guestfs___per_handle_lock_is_not_held (g); + + int i; + gl_lock_t l = guestfs___lookup_lock_or_abort (g, &i); + gl_lock_destroy (l); + handles[i] = NULL; + // fill in hole we've created + ___shrink_lock_arrays (); + + // and realloc + n_handles--; + if (n_handles > 0) { + per_handle_locks = realloc (per_handle_locks, + sizeof (*per_handle_locks) * n_handles); + handles = realloc (handles, sizeof (*handles) * n_handles); + } + + gl_rwlock_unlock (locks_lock); +} + +void +guestfs___per_handle_lock_lock (guestfs_h *g) +{ + gl_rwlock_rdlock (locks_lock); + + gl_lock_t l = guestfs___lookup_lock_or_abort(g, NULL); + gl_lock_lock(l); + + gl_rwlock_unlock (locks_lock); +} + +void +guestfs___per_handle_lock_unlock (guestfs_h *g) +{ + gl_rwlock_rdlock (locks_lock); + + gl_lock_t l = guestfs___lookup_lock_or_abort(g, NULL); + gl_lock_unlock(l); + + gl_rwlock_unlock (locks_lock); +} + diff --git a/src/locking.h b/src/locking.h new file mode 100644 index 0000000..fc7bc46 --- /dev/null +++ b/src/locking.h @@ -0,0 +1,45 @@ +/* libguestfs + * Copyright (C) 2009-2014 Red Hat Inc. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef _LOCKING_H +#define _LOCKING_H + +#ifdef __cplusplus +extern "C" { +#endif + +#include <config.h> + +#include "glthread/lock.h" + +#include "guestfs.h" +#include "guestfs-internal.h" + +extern void guestfs___per_handle_lock_add (guestfs_h *g); + +extern void guestfs___per_handle_lock_remove (guestfs_h *g); + +extern void guestfs___per_handle_lock_lock (guestfs_h *g); + +extern void guestfs___per_handle_lock_unlock (guestfs_h *g); + +#ifdef __cplusplus +} +#endif + +#endif /* _LOCKING_H */ -- 1.9.3

2 3

libguestfs error
by abhishek jain 03 Jun '14

03 Jun '14

Hi I'm getting following error while compiling libguestfs-1.26.1... CCLD guestfish guestfish-progress.o: In function `progress_bar_init': /home/ubuntu/libguestfs-1.26.1/fish/progress.c:123: undefined reference to `tgetent' guestfish-progress.o: In function `progress_bar_set': /home/ubuntu/libguestfs-1.26.1/fish/progress.c:282: undefined reference to `tgetnum' /home/ubuntu/libguestfs-1.26.1/fish/progress.c:297: undefined reference to `UP' guestfish-progress.o: In function `fprintf': /usr/include/powerpc-linux-gnu/bits/stdio2.h:97: undefined reference to `UP' collect2: error: ld returned 1 exit status make[3]: *** [guestfish] Error 1 make[3]: Leaving directory `/home/ubuntu/libguestfs-1.26.1/fish' make[2]: *** [all] Error 2 make[2]: Leaving directory `/home/ubuntu/libguestfs-1.26.1/fish' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/home/ubuntu/libguestfs-1.26.1' make: *** [all] Error 2 I was able to compile it yesterday but when i formatted my hard disk and tried to install it again,it started showing the above error. Please help regarding this. Thanks Abhishek Jain

3 20

use guestfish to mount live virtual machines
by sjy 30 May '14

30 May '14

Hello, I've tried to use guestfish to lively mount virtual machine image. Here are some problems I met. 1. The name parameter in below command must be org.libguestfs.channel.0. or else the guest would produce error 'failed to connect to virtio-serial' and '/dev/virtio-ports/org.libguestfs.channel.0: No such file or directory' even though there did exists a file 'com.test.my' in 'dev/virtio-ports' directory. sudo kvm -drive file=ubuntu-1204-vm.img -m 512 -vga std -k en-us -vnc :1 -net nic -net tap,ifname=tap1,script=./qemu-ifup,downscript=no -daemonize -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/test.agent,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.test.my 2. I leaved the first problem by altering the device name to default org.libguestfs.channel.0, now I can visit the live guest by guestfish. However, any write or edit operation like vi or write don't work even I set --rw option with guestfish. It seems like guestfish just write the results to a tmp file in /tmp/guestfish**/ directory rather than the guest image file. I don't know if it needs a writeback operation to make the changes happen. Besides, read operation like cat can only read files produced before the mount operation, any real time modification by the guest can't be found with guestfish --live. Any answers will be appreciated.

2 1

[PATCH] build: Enable parallel tests.
by Richard W.M. Jones 30 May '14

30 May '14

This patch enables parallel tests. This has a small performance benefit. Unfortunately several tests still overwrite common filenames (eg. 'test.out') which means that if you apply this patch then tests will fail randomly. To fix this requires every test to be examined manually to ensure that it does not use a common filename, eg. changing 'test.out' to '<testname>.out'. Rich.

1 2

[PATCH] generator: fix daemon functions with optional params but no mandatory params
by Pino Toscano 29 May '14

29 May '14

The struct guestfs_$function_args args; declaration was not emitted in that case, leading to build failure. --- generator/c.ml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/generator/c.ml b/generator/c.ml index ee276dc..1fa05ab 100644 --- a/generator/c.ml +++ b/generator/c.ml @@ -1658,9 +1658,9 @@ and generate_client_actions hash () = let args_passed_to_daemon = List.filter (function FileIn _ | FileOut _ -> false | _ -> true) args in - (match args_passed_to_daemon with - | [] -> () - | _ -> pr " struct guestfs_%s_args args;\n" name + (match args_passed_to_daemon, optargs with + | [], [] -> () + | _, _ -> pr " struct guestfs_%s_args args;\n" name ); pr " guestfs_message_header hdr;\n"; -- 1.9.3

2 1

Re: [Libguestfs] virt-resize does not work
by Richard W.M. Jones 27 May '14

27 May '14

Please keep all replies on the list. On Tue, May 27, 2014 at 11:22:17AM +0200, Info || Vertixo B.V. wrote: > Hi, > > Ah, ok. > Well, i shutdown the VM, follow the instructions and it says that the disk is > successfully resized. > I start the machine and do an df -h and everything is just the same as before, > nothing got bigger. Are you booting the new disk or the old disk? What is the actual df -h output? Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top

2 2

[PATCH] ruby: add :nodoc: comment for internal methods
by Pino Toscano 27 May '14

27 May '14

This way they are ignored by rdoc. --- generator/ruby.ml | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/generator/ruby.ml b/generator/ruby.ml index 1111993..88762ca 100644 --- a/generator/ruby.ml +++ b/generator/ruby.ml @@ -522,6 +522,15 @@ get_all_event_callbacks (guestfs_h *g, size_t *len_rtn) * +guestfs_%s+[http://libguestfs.org/guestfs.3.html#guestfs_%s]). */ " f.name args ret f.shortdesc doc f.name f.name + ) else ( + pr "\ +/* + * call-seq: + * g.%s + * + * :nodoc: + */ +" f.name ); (* Generate the function. Prototype is completely different -- 1.9.3

2 2