Plan for libguestfs 1.30
by Richard W.M. Jones
It has been a ludicrously long time (over 8 months) since the last
stable release of libguestfs. So I'd like to plan a new 1.30 release
soon.
Please follow-up if there are features / blockers / bugs that need to
be addressed for 1.30.
As usual, bugs which have "1.30" (without quotes) in the Devel
Whiteboard field in Bugzilla are nominated as blockers for the
release. You can see a list of those here [currently empty]:
https://bugzilla.redhat.com/buglist.cgi?bug_status=NEW&bug_status=ASSIGNE...
For a list of all bugs, go to:
https://bugzilla.redhat.com/buglist.cgi?component=libguestfs&product=Virt...
----------------------------------------------------------------------
It's also worth talking about what I'd like to do *after* 1.30 is out.
This includes:
- threading support
https://www.redhat.com/archives/libguestfs/2015-June/thread.html#00118
- external tests
https://www.redhat.com/archives/libguestfs/2014-October/thread.html#00042
While thread safety is oft talked about, it's external tests which I
consider to be the more important patch series, since it will allow us
to build and test libguestfs more easily and more comprehensively for
Fedora and RHEL. Unfortunately it's the most difficult and invasive
change of the two.
You may have your own ideas, if so follow up here.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
8 years, 4 months
SELinux relabel API
by Richard W.M. Jones
[
I realized that we were discussing adding this feature, in various
private email, IRC, and this long bugzilla thread:
https://bugzilla.redhat.com/show_bug.cgi?id=1060423
That's not how we should do things. Let's discuss it on the
mailing list.
]
One thing that virt-customize/virt-sysprep/virt-builder have to do is
relabel SELinux guests.
What we do at the moment is run:
if load_policy && fixfiles restore; then
rm -f /.autorelabel
else
touch /.autorelabel
echo '%s: SELinux relabelling failed, will relabel at boot instead.'
fi
while chrooted into the guest (using the 'guestfs_sh' API).
This has a number of problems:
- It has to load the policy using 'load_policy', but this doesn't
work sometimes:
* RHEL 5 load_policy takes a parameter.
* Doesn't work if appliance kernel is significantly different from
guest kernel version, because the binary policy format changes
irregularly and is not backwards compatible.
* Requires the appliance [host] kernel to be compiled with
LSM/SELinux support.
- Touching /.autorelabel is often broken, eg. it's broken in Fedora 20
because of systemd (RHBZ#1049656).
- /etc/resolv.conf will not be relabelled if guestfs network is on,
because of resolv.conf shenanigans in libguestfs.git/daemon/command.c
- It requires running guest code, which we'd like to avoid.
What would be nice would be to have an API to just do this
relabelling. Libguestfs could change this API as required to handle
different guests.
Dan Walsh helpfully pointed out to us that we've been doing it wrong
all along :-) A much better way to relabel is to run:
setfiles /etc/selinux/targeted/contexts/files/file_contexts DIR
where 'file_contexts' is a file which contains the default labels for
files (a set of regexps), and 'DIR' is the directory at which
relabelling starts. Note that 'setfiles' would be the libguestfs
appliance binary, so no guest binary needs to be run.
A simple API could just look like this:
guestfs_selinux_relabel (g);
which would always use the 'targeted' policy from the guest, and
always start relabelling at the root. This would work fine for
virt-builder.
For Colin's requirements for Project Atomic, I suspect he will want to
be able to set the file_contexts file and the root directory, but I'll
leave him to describe what would be useful.
A couple of notes:
- I'd like to avoid baking in assumptions from the 'setfiles' command
as far as possible. libguestfs APIs last for many years and some
have caused us many years of regret (but that's our job) :-/
- Is it a good idea to tie this into inspection in some way -- for
example, inspection could provide us with the path to the current or
default SELinux policy.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
8 years, 5 months
[PATCH] v2v: add --in-place mode
by Roman Kagan
In this mode, converting of the VM configuration, setting up the
rollback path for error cases, transforming the VM storage and so on is
taken care of by a third-party toolset, and virt-v2v is only supposed to
tune up the guest OS directly inside the source VM, to enable it to boot
and run under the input hypervisor.
Signed-off-by: Roman Kagan <rkagan(a)virtuozzo.com>
---
tests/guests/guests.xml.in | 16 +++++++
v2v/Makefile.am | 1 +
v2v/cmdline.ml | 7 +++-
v2v/test-v2v-in-place.sh | 81 +++++++++++++++++++++++++++++++++++
v2v/v2v.ml | 102 +++++++++++++++++++++++++++------------------
v2v/virt-v2v.pod | 17 ++++++++
6 files changed, 183 insertions(+), 41 deletions(-)
create mode 100755 v2v/test-v2v-in-place.sh
diff --git a/tests/guests/guests.xml.in b/tests/guests/guests.xml.in
index 8f7ac81..6f08b80 100644
--- a/tests/guests/guests.xml.in
+++ b/tests/guests/guests.xml.in
@@ -279,4 +279,20 @@
</devices>
</domain>
+ <domain type='test'>
+ <name>windows-overlay</name>
+ <memory>1048576</memory>
+ <os>
+ <type>hvm</type>
+ <boot dev='hd'/>
+ </os>
+ <devices>
+ <disk type='file' device='disk'>
+ <driver name='qemu' type='qcow2'/>
+ <source file='@abs_builddir(a)/windows-overlay.qcow2'/>
+ <target dev='vda' bus='virtio'/>
+ </disk>
+ </devices>
+ </domain>
+
</node>
diff --git a/v2v/Makefile.am b/v2v/Makefile.am
index 06da002..dae063c 100644
--- a/v2v/Makefile.am
+++ b/v2v/Makefile.am
@@ -232,6 +232,7 @@ TESTS += \
test-v2v-cdrom.sh \
test-v2v-i-ova.sh \
test-v2v-i-disk.sh \
+ test-v2v-in-place.sh \
test-v2v-machine-readable.sh \
test-v2v-networks-and-bridges.sh \
test-v2v-no-copy.sh \
diff --git a/v2v/cmdline.ml b/v2v/cmdline.ml
index eaf57dc..2a3224a 100644
--- a/v2v/cmdline.ml
+++ b/v2v/cmdline.ml
@@ -37,6 +37,7 @@ let parse_cmdline () =
let output_format = ref "" in
let output_name = ref "" in
let output_storage = ref "" in
+ let in_place = ref false in
let password_file = ref "" in
let print_source = ref false in
let qemu_boot = ref false in
@@ -160,6 +161,7 @@ let parse_cmdline () =
"-of", Arg.Set_string output_format, "raw|qcow2 " ^ s_"Set output format";
"-on", Arg.Set_string output_name, "name " ^ s_"Rename guest when converting";
"-os", Arg.Set_string output_storage, "storage " ^ s_"Set output storage location";
+ "--in-place", Arg.Set in_place, " " ^ s_"Only tune the guest in the input VM";
"--password-file", Arg.Set_string password_file, "file " ^ s_"Use password from file";
"--print-source", Arg.Set print_source, " " ^ s_"Print source and stop";
"--qemu-boot", Arg.Set qemu_boot, " " ^ s_"Boot in qemu (-o qemu only)";
@@ -226,6 +228,7 @@ read the man page virt-v2v(1).
let output_mode = !output_mode in
let output_name = match !output_name with "" -> None | s -> Some s in
let output_storage = !output_storage in
+ let in_place = !in_place in
let password_file = match !password_file with "" -> None | s -> Some s in
let print_source = !print_source in
let qemu_boot = !qemu_boot in
@@ -305,6 +308,8 @@ read the man page virt-v2v(1).
Input_ova.input_ova filename in
(* Parse the output mode. *)
+ if output_mode <> `Not_set && in_place then
+ error (f_"-o and --in-place cannot be used at the same time");
let output =
match output_mode with
| `Glance ->
@@ -386,5 +391,5 @@ read the man page virt-v2v(1).
input, output,
debug_gc, debug_overlays, do_copy, network_map, no_trim,
- output_alloc, output_format, output_name,
+ output_alloc, output_format, output_name, in_place,
print_source, root_choice
diff --git a/v2v/test-v2v-in-place.sh b/v2v/test-v2v-in-place.sh
new file mode 100755
index 0000000..c19707e
--- /dev/null
+++ b/v2v/test-v2v-in-place.sh
@@ -0,0 +1,81 @@
+#!/bin/bash -
+# libguestfs virt-v2v test script
+# Copyright (C) 2014 Red Hat Inc.
+# Copyright (C) 2015 Parallels IP Holdings GmbH.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+# Test --in-place.
+
+unset CDPATH
+export LANG=C
+set -e
+
+if [ -n "$SKIP_TEST_V2V_IN_PLACE_SH" ]; then
+ echo "$0: test skipped because environment variable is set"
+ exit 77
+fi
+
+if [ "$(guestfish get-backend)" = "uml" ]; then
+ echo "$0: test skipped because UML backend does not support network"
+ exit 77
+fi
+
+# You shouldn't be running the tests as root anyway, but in this case
+# it's especially bad because we don't want to start creating guests
+# or storage pools in the system namespace.
+if [ "$(id -u)" -eq 0 ]; then
+ echo "$0: test skipped because you're running tests as root. Don't do that!"
+ exit 77
+fi
+
+guests_dir="$(cd $(dirname $0)/../tests/guests && pwd)"
+libvirt_uri="test://$guests_dir/guests.xml"
+
+f="$guests_dir/windows.img"
+if ! test -f $f || ! test -s $f; then
+ echo "$0: test skipped because phony Windows image was not created"
+ exit 77
+fi
+
+virt_tools_data_dir=${VIRT_TOOLS_DATA_DIR:-/usr/share/virt-tools}
+if ! test -r $virt_tools_data_dir/rhsrvany.exe; then
+ echo "$0: test skipped because rhsrvany.exe is not installed"
+ exit 77
+fi
+
+fo="$guests_dir/windows-overlay.qcow2"
+rm -f $fo
+qemu-img create -f qcow2 -b $f -o compat=1.1,backing_fmt=raw $fo
+md5="$(md5sum $f)"
+
+$VG virt-v2v --debug-gc \
+ -i libvirt -ic "$libvirt_uri" windows-overlay \
+ --in-place
+
+# Test some aspects of the target disk image.
+guestfish --ro -a $fo -i <<EOF
+ is-dir "/Program Files/Red Hat/Firstboot"
+ is-file "/Program Files/Red Hat/Firstboot/firstboot.bat"
+ is-dir "/Program Files/Red Hat/Firstboot/scripts"
+ is-dir "/Windows/Drivers/VirtIO"
+EOF
+
+
+# Test the base image remained untouched
+test "$md5" = "$(md5sum $f)"
+
+# Clean up.
+rm -r $fo
diff --git a/v2v/v2v.ml b/v2v/v2v.ml
index 242f129..b744056 100644
--- a/v2v/v2v.ml
+++ b/v2v/v2v.ml
@@ -47,7 +47,8 @@ let rec main () =
(* Handle the command line. *)
let input, output,
debug_gc, debug_overlays, do_copy, network_map, no_trim,
- output_alloc, output_format, output_name, print_source, root_choice =
+ output_alloc, output_format, output_name, in_place, print_source,
+ root_choice =
Cmdline.parse_cmdline () in
(* Print the version, easier than asking users to tell us. *)
@@ -117,52 +118,70 @@ let rec main () =
) nics in
{ source with s_nics = nics } in
- (* Create a qcow2 v3 overlay to protect the source image(s). There
- * is a specific reason to use the newer qcow2 variant: Because the
- * L2 table can store zero clusters efficiently, and because
- * discarded blocks are stored as zero clusters, this should allow us
- * to fstrim/blkdiscard and avoid copying significant parts of the
- * data over the wire.
- *)
- message (f_"Creating an overlay to protect the source from being modified");
let overlay_dir = (new Guestfs.guestfs ())#get_cachedir () in
- let overlays =
- List.map (
- fun ({ s_qemu_uri = qemu_uri; s_format = format } as source) ->
- let overlay_file =
- Filename.temp_file ~temp_dir:overlay_dir "v2vovl" ".qcow2" in
- unlink_on_exit overlay_file;
-
- let options =
- "compat=1.1" ^
- (match format with None -> ""
- | Some fmt -> ",backing_fmt=" ^ fmt) in
- let cmd =
- sprintf "qemu-img create -q -f qcow2 -b %s -o %s %s"
- (quote qemu_uri) (quote options) overlay_file in
- if verbose () then printf "%s\n%!" cmd;
- if Sys.command cmd <> 0 then
- error (f_"qemu-img command failed, see earlier errors");
-
- (* Sanity check created overlay (see below). *)
- if not ((new G.guestfs ())#disk_has_backing_file overlay_file) then
- error (f_"internal error: qemu-img did not create overlay with backing file");
-
- overlay_file, source
- ) source.s_disks in
+ let overlays = (
+ if not in_place then (
+ (* Create a qcow2 v3 overlay to protect the source image(s). There
+ * is a specific reason to use the newer qcow2 variant: Because the
+ * L2 table can store zero clusters efficiently, and because
+ * discarded blocks are stored as zero clusters, this should allow us
+ * to fstrim/blkdiscard and avoid copying significant parts of the
+ * data over the wire.
+ *)
+ message (f_"Creating an overlay to protect the source from being modified");
+ List.map (
+ fun ({ s_qemu_uri = qemu_uri; s_format = format } as source) ->
+ let overlay_file =
+ Filename.temp_file ~temp_dir:overlay_dir "v2vovl" ".qcow2" in
+ unlink_on_exit overlay_file;
+
+ let options =
+ "compat=1.1" ^
+ (match format with None -> ""
+ | Some fmt -> ",backing_fmt=" ^ fmt) in
+ let cmd =
+ sprintf "qemu-img create -q -f qcow2 -b %s -o %s %s"
+ (quote qemu_uri) (quote options) overlay_file in
+ if verbose () then printf "%s\n%!" cmd;
+ if Sys.command cmd <> 0 then
+ error (f_"qemu-img command failed, see earlier errors");
+
+ (* Sanity check created overlay (see below). *)
+ if not ((new G.guestfs ())#disk_has_backing_file overlay_file) then
+ error (f_"internal error: qemu-img did not create overlay with backing file");
+
+ overlay_file, source
+ ) source.s_disks
+ ) else []
+ ) in
(* Open the guestfs handle. *)
- message (f_"Opening the overlay");
+ if in_place then
+ message (f_"Opening the guest disks")
+ else
+ message (f_"Opening the overlay");
let g = new G.guestfs () in
if trace () then g#set_trace true;
if verbose () then g#set_verbose true;
g#set_network true;
- List.iter (
- fun (overlay_file, _) ->
- g#add_drive_opts overlay_file
- ~format:"qcow2" ~cachemode:"unsafe" ~discard:"besteffort"
- ~copyonread:true
- ) overlays;
+ if in_place then (
+ List.iter (
+ fun ({s_qemu_uri = qemu_uri; s_format = format}) ->
+ match format with
+ | None ->
+ g#add_drive_opts qemu_uri ~cachemode:"unsafe" ~discard:"besteffort"
+ | Some fmt ->
+ g#add_drive_opts qemu_uri ~format:fmt ~cachemode:"unsafe"
+ ~discard:"besteffort"
+ ) source.s_disks
+ ) else (
+ List.iter (
+ fun (overlay_file, _) ->
+ g#add_drive_opts overlay_file
+ ~format:"qcow2" ~cachemode:"unsafe" ~discard:"besteffort"
+ ~copyonread:true
+ ) overlays
+ );
g#launch ();
@@ -294,6 +313,9 @@ let rec main () =
g#shutdown ();
g#close ();
+ if in_place then
+ exit 0;
+
(* Does the guest require UEFI on the target? *)
message (f_"Checking if the guest needs BIOS or UEFI to boot");
let target_firmware =
diff --git a/v2v/virt-v2v.pod b/v2v/virt-v2v.pod
index ef2dee1..eda1cf7 100644
--- a/v2v/virt-v2v.pod
+++ b/v2v/virt-v2v.pod
@@ -9,6 +9,8 @@ virt-v2v - Convert a guest to use KVM
virt-v2v -ic vpx://vcenter.example.com/Datacenter/esxi vmware_guest \
-o rhev -os rhev.nfs:/export_domain --network rhevm
+ virt-v2v -ic qemu:///system qemu_guest --in-place
+
virt-v2v -i libvirtxml guest-domain.xml -o local -os /var/tmp
virt-v2v -i disk disk.img -o local -os /var/tmp
@@ -75,6 +77,9 @@ booting the guest directly in qemu (mainly for testing).
I<-o rhev> is used to write to a RHEV-M / oVirt target. I<-o vdsm>
is only used when virt-v2v runs under VDSM control.
+I<--in-place> instructs virt-v2v to customize the guest OS in the input
+virtual machine, instead of creating a new VM in the target hypervisor.
+
=head1 EXAMPLES
=head2 Convert from VMware vCenter server to local libvirt
@@ -518,6 +523,18 @@ C<root>.
You will get an error if virt-v2v is unable to mount/write to the
Export Storage Domain.
+=item B<--in-place>
+
+Do not create an output virtual machine in the target hypervisor.
+Instead, adjust the guest OS in the source VM to run in the input
+hypervisor.
+
+This mode is meant for integration with other toolsets, which take the
+responsibility of converting the VM configuration, providing for
+rollback in case of errors, transforming the storage, etc.
+
+Conflicts with all I<-o *> options.
+
=item B<--password-file> file
Instead of asking for password(s) interactively, pass the password
--
2.4.3
8 years, 6 months
[PATCH 0/4] v2v: simplify driver copying from virtio-win iso
by Roman Kagan
Libguestfs supports passing an ISO image as a source of virtio windows
drivers to v2v.
That support, however, looks too heavy-weight: in order to access those
drivers, a separate guestfs handle is created (and thus a new emulator
process is started), which runs until v2v completes.
This series attempts to make it simpler and lighter-weight, by making
the relevant code more local, and by hot-adding the image into the main
guestfs handle.
Roman Kagan (4):
v2v: drop useless forced gc
v2v: consolidate virtio-win file copying
v2v: copy virtio drivers without guestfs handle leak
v2v: reuse main guestfs for virtio win drivers iso
v2v/convert_windows.ml | 184 ++++++++++++++++++++--------------------
v2v/utils.ml | 224 ++++++++++++++++---------------------------------
v2v/v2v.ml | 8 --
3 files changed, 163 insertions(+), 253 deletions(-)
--
2.4.3
8 years, 7 months
[PATCH] Add opensuse.gpg to the dist tarball
by Cédric Bosdonnat
Adding missing opensuse.gpg to EXTRA_DIST.
---
builder/Makefile.am | 1 +
1 file changed, 1 insertion(+)
diff --git a/builder/Makefile.am b/builder/Makefile.am
index 3111ef2..0d5ce3b 100644
--- a/builder/Makefile.am
+++ b/builder/Makefile.am
@@ -22,6 +22,7 @@ AM_YFLAGS = -d
EXTRA_DIST = \
$(SOURCES_MLI) $(SOURCES_ML) $(SOURCES_C) \
libguestfs.gpg \
+ opensuse.gpg \
test-index \
test-simplestreams/streams/v1/index.json \
test-simplestreams/streams/v1/net.cirros-cloud_released_download.json \
--
2.1.4
8 years, 7 months
[PATCH 1/2] ocaml: Use ocamlfind to run ocamldoc.
by Richard W.M. Jones
Using 'ocamlfind ocamldoc' is much faster than running 'ocamldoc'
directly, because ocamlfind will run the native code program
'ocamldoc.opt' if it is available.
This change approximately halves the time taken to compile the ocaml
bindings.
---
ocaml/Makefile.am | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/ocaml/Makefile.am b/ocaml/Makefile.am
index a535b43..b9828e1 100644
--- a/ocaml/Makefile.am
+++ b/ocaml/Makefile.am
@@ -98,7 +98,7 @@ if HAVE_OCAMLDOC
noinst_DATA += html/index.html
html/index.html: $(srcdir)/guestfs.mli $(srcdir)/guestfs.ml
- -$(OCAMLDOC) -d html -html $^
+ -$(OCAMLFIND) ocamldoc -d html -html $^
endif
--
2.5.0
8 years, 7 months
[PATCH 1/4] lib: actions: Remove some unused header files.
by Richard W.M. Jones
---
generator/c.ml | 2 --
1 file changed, 2 deletions(-)
diff --git a/generator/c.ml b/generator/c.ml
index 055b683..963cf21 100644
--- a/generator/c.ml
+++ b/generator/c.ml
@@ -1213,9 +1213,7 @@ and generate_client_actions hash () =
#include <stdlib.h>
#include <stdint.h>
#include <inttypes.h>
-#include <sys/types.h>
#include <sys/stat.h>
-#include <string.h>
#include \"guestfs.h\"
#include \"guestfs-internal.h\"
--
2.5.0
8 years, 7 months
[PATCH 1/2] ocaml: Only build the tests when running 'make check'.
by Richard W.M. Jones
Make the tests 'check_DATA' so they only get built when running the
tests. This saves a couple of seconds on the build time.
---
ocaml/Makefile.am | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/ocaml/Makefile.am b/ocaml/Makefile.am
index e781363..a535b43 100644
--- a/ocaml/Makefile.am
+++ b/ocaml/Makefile.am
@@ -142,9 +142,9 @@ endif
TESTS = run-bindtests $(test_progs_all)
-noinst_DATA += bindtests.bc $(test_progs_all)
+check_DATA = bindtests.bc $(test_progs_all)
if HAVE_OCAMLOPT
-noinst_DATA += bindtests.opt
+check_DATA += bindtests.opt
endif
%.bc: %.cmo mlguestfs.cma
@@ -204,6 +204,6 @@ install-data-hook:
rm $(DESTDIR)$(OCAMLLIB)/guestfs/bindtests.*
rm $(DESTDIR)$(OCAMLLIB)/guestfs/libguestfsocaml.a
-CLEANFILES += $(noinst_DATA)
+CLEANFILES += $(noinst_DATA) $(check_DATA)
endif
--
2.5.0
8 years, 7 months
[PATCH v2] copy-in: error out early if the localpath does not exist
by Pino Toscano
---
src/copy-in-out.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/src/copy-in-out.c b/src/copy-in-out.c
index d5e7fb0..89344d8 100644
--- a/src/copy-in-out.c
+++ b/src/copy-in-out.c
@@ -46,6 +46,12 @@ guestfs_impl_copy_in (guestfs_h *g,
size_t buf_len = strlen (localpath) + 1;
char buf[buf_len];
const char *dirname, *basename;
+ struct stat statbuf;
+
+ if (stat (localpath, &statbuf) == -1) {
+ error (g, _("source '%s' does not exist (or cannot be read)"), localpath);
+ return -1;
+ }
int remote_is_dir = guestfs_is_dir (g, remotedir);
if (remote_is_dir == -1)
--
2.1.0
8 years, 7 months
