[PATCH v3 0/3] Install QEMU-GA from oVirt guest tools ISO on Linux
by Tomáš Golembiovský
changes in v3:
- fix call to install_local
changes in v2:
- moved copy_drivers above copy_files
- renamed copy_files to copy_from_virtio_win
- renamed install to install_local
- use rpm instead of yum
This installs packages with QEMU Guest Agent when converting Linux machine. The
packages should be available on guest tools ISO. The patches work "as-is" but
probably deserve some more attention:
- it is "abusing" Winows_virtio code but renaming/refactoring everything to
remove "windows" from the name and use "guest tools" seems like a lot of
unnecesary work
- support for Debian is missing
I don't know how to install the package only when all it's dependencies are
already installed. dpkg cannot be used to check that (simulate the install).
And attempting to install the package will leave it half-installed (dpkg
cannot roll-back).
Tomáš Golembiovský (3):
v2v: refactor copy_drivers() in Windows_virtio
v2v: linux: install packages
v2v: linux: install QEMU-GA (RHBZ#1619665)
v2v/convert_linux.ml | 2 ++
v2v/linux.ml | 14 ++++++++
v2v/linux.mli | 3 ++
v2v/windows_virtio.ml | 78 +++++++++++++++++++++++++++++++++---------
v2v/windows_virtio.mli | 4 +++
5 files changed, 84 insertions(+), 17 deletions(-)
--
2.19.0
5 years, 6 months
Further nbdkit patches for Haiku
by Richard W.M. Jones
François, thanks for the patches so far.
This morning I pushed:
https://github.com/libguestfs/nbdkit/commit/e26fb6cea3fdba13b5472daf6767e...
build: add missing #include "byte-swapping.h"
https://github.com/libguestfs/nbdkit/commit/b962272a561bfed087316d24b7d51...
build: more fallbacks for mkostemp
The iconv patch makes iconv a requirement. I don't know how many
systems actually lack iconv -- probably not very many -- but even so I
don't want nbdkit to require a library just to compile a single
plugin, as a feature of nbdkit is that the minimal set of
requirements. So I have modified your patch (see attachment) to make
the iconv dependency optional. Can you let me know if this works on
Haiku or if I broke anything?
The final patch was:
https://github.com/mmuman/nbdkit/commit/50ee315525258b0defae06f6e29da841d...
build: avoid hardcoding /var/tmp by default, use P_tmpdir
Unfortunately this won't work well on some Linux systems which use the
/tmp-on-tmpdir misfeature. On these systems, /tmp has only a small,
limited amount of space and is backed by memory, and we must use
/var/tmp for large files. I don't know how to fix this. Perhaps a
simple:
AS_CASE([$host_os],
[haiku*], [LARGE_TMPDIR=/tmp]
[LARGE_TMPDIR=/var/tmp]
)
AC_DEFINE_UNQUOTED([LARGE_TMPDIR],["$LARGE_TMPDIR"],
[Temporary directory for large files])
would be enough for now?
Finally about the list of failing tests:
https://pastebin.com/1PFG85DS
Mostly these are because of the old qemu-io binary. The qemu-io -f
option was added 4 years ago :-/ so it could be time to upgrade.
This error:
./test-shebang-perl.sh: PID file was not created
rect ELF header
FAIL test-shebang-perl.sh (exit status: 1)
(I think the error message is "incorrect ELF header") could be
something to do with the way we use shebangs (#!). IIRC this also
fails on FreeBSD so probably nbdkit is doing something which is not
correct POSIX.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
5 years, 6 months
[PATCH] tests: Fix shebang tests on FreeBSD.
by Richard W.M. Jones
On FreeBSD we cannot go through the shell script (./nbdkit). Instead
replace the path with a direct link to both the nbdkit binary and
locally compiled plugin.
Note this does not go through valgrinding when you run
‘make check-valgrind’ so that needs to be fixed at some point in the
future.
Thanks: François Revol and Eric Blake.
See also:
https://www.redhat.com/archives/libguestfs/2018-November/msg00126.html
---
tests/shebang.pl | 2 +-
tests/shebang.py | 2 +-
tests/shebang.rb | 2 +-
3 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/tests/shebang.pl b/tests/shebang.pl
index 17c7821..e6f1630 100755
--- a/tests/shebang.pl
+++ b/tests/shebang.pl
@@ -1,4 +1,4 @@
-#!../nbdkit perl
+#!../src/nbdkit ../plugins/perl/.libs/nbdkit-perl-plugin.so
use strict;
diff --git a/tests/shebang.py b/tests/shebang.py
index 6f33623..e836255 100755
--- a/tests/shebang.py
+++ b/tests/shebang.py
@@ -1,4 +1,4 @@
-#!../nbdkit python
+#!../src/nbdkit ../plugins/python/.libs/nbdkit-python-plugin.so
disk = bytearray(1024 * 1024)
diff --git a/tests/shebang.rb b/tests/shebang.rb
index 34588d1..ccf4519 100755
--- a/tests/shebang.rb
+++ b/tests/shebang.rb
@@ -1,4 +1,4 @@
-#!../nbdkit ruby
+#!../src/nbdkit ../plugins/ruby/.libs/nbdkit-ruby-plugin.so
include Nbdkit
--
2.19.0.rc0
5 years, 6 months
collectd leaks SIGCHLD == SIG_IGN into plugins
by Richard W.M. Jones
Peter Dimitrov and myself were debugging a very peculiar bug when
libguestfs is run as a plugin from collectd:
https://www.redhat.com/archives/libguestfs/2018-November/thread.html#00023
The long story short is that collectd leaks SIGCHLD == SIG_IGN setting
into plugins:
https://www.redhat.com/archives/libguestfs/2018-November/msg00095.html
This means that any plugin that does the usual pattern of:
pid = fork ();
...
if (waitpid (pid, NULL, 0) == -1) {
perror ("waitpid");
exit (EXIT_FAILURE);
}
will fail, because the forked subprocess is automatically reaped
before waitpid is called, resulting in the wait failing with errno ==
ECHILD.
It is possible to work around this by adding:
signal (SIGCHLD, SIG_DFL);
to the plugin. However I believe this is a bug in collectd, and it
should sanitize signals (and maybe other things) before running
plugins.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
5 years, 6 months
ANNOUNCE: nbdkit 1.8 - an NBD server toolkit with stable plugin API and permissive license
by Richard W.M. Jones
I’m pleased to announce the next stable release of nbdkit.
NBD — Network Block Device — is a protocol for accessing Block Devices
(hard disks and disk-like things) over a Network. nbdkit is a toolkit
for creating NBD servers.
The key features are:
* Multithreaded NBD server written in C with good performance.
* Minimal dependencies for the basic server.
* Liberal license (BSD) allows nbdkit to be linked to proprietary
libraries or included in proprietary code.
* Well-documented, simple plugin API with a stable ABI guarantee.
Lets you export “unconventional” block devices easily.
* You can write plugins in C, Lua, Perl, Python, OCaml, Ruby,
[new!] shell script or Tcl.
* Filters can be stacked in front of plugins to transform the output.
Clone source: https://github.com/libguestfs/nbdkit
Download tarball: http://download.libguestfs.org/nbdkit/1.8-stable/
(other downloads: http://download.libguestfs.org/nbdkit/ )
New in this release:
- New plugin: ‘sh’ for running shell scripts and other external programs.
(https://rwmj.wordpress.com/2018/09/09/write-nbdkit-plugins-in-shell-script/)
- New plugin: ‘partitioning’ which creates a partitioned disk from a
list of partitions in files.
- New plugin: ‘iso’ which creates an ISO (CD-ROM) image on the fly
from a directory of files.
(https://rwmj.wordpress.com/2018/10/18/new-in-nbdkit-create-an-iso-image-o...)
- New plugin: ‘floppy’ which creates a virtual FAT-formatted floppy
disk from a directory of files.
(https://rwmj.wordpress.com/2018/10/29/new-in-nbdkit-create-a-virtual-flop...)
- Magic config key allows simpler invocation of many plugins, eg:
nbdkit file file=foo
becomes:
nbdkit file foo
- data: Add disk2data.pl script to generate data parameters.
- data: Use ‘<file’ to inline a file.
- vddk: Use ‘-D vddk.diskinfo=1’ to enable extra debug information.
- tests: Use a helper function to refactor and simplify most tests.
- partition filter: Can handle > 128 GPT partitions.
- nbdkit can be compiled on the Haiku operating system
(François Revol).
- New nbdkit_parse_bool API for parsing boolean parameters (Eric Blake).
- log: Add logappend=[true|false] (Eric Blake).
- ocaml: New set_error and debug calls.
- Analyze code under AddressSanitizer and ThreadSanitizer and fix bugs
(Eric Blake).
- Analyze code under Coverity and fix bugs.
- Tests now run more quickly.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
5 years, 6 months
Plan for nbdkit 1.8
by Richard W.M. Jones
I would really like to release a new stable nbdkit (1.8) soon.
The reasons are not only because of the many great new features (which
I summarise below) but also a few practical reasons:
- Dropping mandatory [key=].. (ie. "magic config key") for many plugin
parameters makes nbdkit much easier to use and easier to describe to
potential users.
- Demonstrated utility of log filter and error filter for testing and
visualizing block devices.
(https://rwmj.wordpress.com/2018/11/06/nbd-graphical-viewer-raid-5-edition/)
- Lots of attention to limits / large virtual sizes (eg
https://rwmj.wordpress.com/2018/09/05/nbdkit-for-loopback-pt-5-8-exabyte-...)
I hope to give a FOSDEM talk next year and it would be nice to have a
stable set of features available in most distros. If we release 1.8
soon then it could become the baseline next February.
----
The release notes would look something like this [preliminary!]:
- Magic config key allows simpler invocation of many plugins, eg:
nbdkit file file=foo
becomes:
nbdkit file foo
- New plugin: 'sh' for running shell scripts and other external programs.
(https://rwmj.wordpress.com/2018/09/09/write-nbdkit-plugins-in-shell-script/)
- New plugin: 'partitioning' which creates a partitioned disk from a
list of partitions in files.
- New plugin: 'iso' which creates an ISO (CD-ROM) image on the fly
from a directory of files.
(https://rwmj.wordpress.com/2018/10/18/new-in-nbdkit-create-an-iso-image-o...)
- New plugin: 'floppy' which creates a virtual FAT-formatted floppy
disk from a directory of files.
(https://rwmj.wordpress.com/2018/10/29/new-in-nbdkit-create-a-virtual-flop...)
- data: Add disk2data.pl script to generate data parameters.
- data: Use <file to inline a file.
- vddk: Use '-D vddk.diskinfo=1' to enable extra debug information.
- tests: Use a helper function to refactor and simplify most tests.
- partition filter: Can handle > 128 GPT partitions.
- nbdkit can be compiled on the Haiku operating system
(François Revol).
----
Anyhow, if there's something you'd like to get into nbdkit 1.8 and/or
you strongly object to any of the above, please follow up.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
5 years, 6 months
guestfs_launch() fails when C application is started as a systemd service
by Peter Dimitrov
Hello,
I have a simple C program that uses libguestfs to extract info about disk
usage from a libvirt domain. It works when ran manually as root, but fails
when started as a systemd service.
I'm attaching the service file, source code and verbose logs from both the
successful manual run and from the service journal.
SELinix is disabled.
Error messages:
libguestfs: set_socket_create_context: getcon failed: (none): Invalid
argument [you can ignore this message if you are not using SELinux + sVirt]
libguestfs: clear_socket_create_context: setsockcreatecon failed: NULL:
Invalid argument [you can ignore this message if you are not using SELinux
+ sVirt]
libguestfs: error: chown: /tmp/libguestfsvMMaec/guestfsd.sock: Operation
not permitted
libguestfs: clear_socket_create_context: setsockcreatecon failed: NULL:
Invalid argument [you can ignore this message if you are not using SELinux
+ sVirt]
libguestfs: trace: launch = -1 (error)
failed to launch domain: Invalid argument
(Note on the service file: CapabilityBoundingSet is for future development
- it seems to make no difference)
Best Regards,
Peter
5 years, 6 months
[nbdkit PATCH v2 0/5] log appends
by Eric Blake
v2 turned out to be much more involved, as I ended up fixing
several things along the way that I noticed while debugging a
feature addition.
Eric Blake (5):
maint: Improve ./nbdkit option parsing
main: Saner newline printing during --help
utils: Add nbdkit_parse_bool
main: Use new bool parser for --tls
log: Allow user option of appending to log
docs/nbdkit-plugin.pod | 11 +++++++++++
filters/log/nbdkit-log-filter.pod | 5 +++--
include/nbdkit-common.h | 1 +
src/filters.c | 13 +++++++-----
src/main.c | 16 ++++++---------
src/plugins.c | 13 +++++++-----
src/utils.c | 25 +++++++++++++++++++++++
filters/log/log.c | 12 +++++++++--
tests/test-log.sh | 6 +++++-
nbdkit.in | 33 +++++++++++++++++++------------
10 files changed, 97 insertions(+), 38 deletions(-)
--
2.17.2
5 years, 6 months
[nbdkit PATCH] log: Allow user option of appending to log
by Eric Blake
Always truncating a log can wipe out useful history. Add an
option to change this.
Signed-off-by: Eric Blake <eblake(a)redhat.com>
---
Hmm - logappend=false enables appending. Maybe we need a common
utility function for parsing bool values (parse 0 and case-insensitive
"off" or "false" as off, all others as true, or even report values
that can't be parsed reasonably)
filters/log/nbdkit-log-filter.pod | 5 +++--
filters/log/log.c | 8 +++++++-
tests/test-log.sh | 6 +++++-
3 files changed, 15 insertions(+), 4 deletions(-)
diff --git a/filters/log/nbdkit-log-filter.pod b/filters/log/nbdkit-log-filter.pod
index 0903329..88b9f9b 100644
--- a/filters/log/nbdkit-log-filter.pod
+++ b/filters/log/nbdkit-log-filter.pod
@@ -4,7 +4,7 @@ nbdkit-log-filter - nbdkit log filter
=head1 SYNOPSIS
- nbdkit --filter=log plugin logfile=FILE [plugin-args...]
+ nbdkit --filter=log plugin logfile=FILE [logappend=VAL] [plugin-args...]
=head1 DESCRIPTION
@@ -22,7 +22,8 @@ work even when nbdkit is run as a daemon.
The nbdkit-log-filter requires a single parameter C<logfile> which
specifies the path of the file to use for logging. If the file
-already exists, it will be truncated.
+already exists, it will be truncated unless the C<logappend> parameter
+was specified with any non-empty value other than "0".
=head1 EXAMPLES
diff --git a/filters/log/log.c b/filters/log/log.c
index 2079084..e6ed3b4 100644
--- a/filters/log/log.c
+++ b/filters/log/log.c
@@ -43,6 +43,7 @@
#include <pthread.h>
#include <sys/time.h>
#include <assert.h>
+#include <stdbool.h>
#include <nbdkit-filter.h>
@@ -51,6 +52,7 @@
static uint64_t connections;
static char *logfilename;
static FILE *logfile;
+static bool append;
static pthread_mutex_t lock = PTHREAD_MUTEX_INITIALIZER;
static void
@@ -75,6 +77,10 @@ log_config (nbdkit_next_config *next, void *nxdata,
}
return 0;
}
+ if (strcmp (key, "logappend") == 0) {
+ append = value[0] && (value[0] != '0' || value[1]);
+ return 0;
+ }
return next (nxdata, key, value);
}
@@ -86,7 +92,7 @@ log_config_complete (nbdkit_next_config_complete *next, void *nxdata)
nbdkit_error ("missing logfile= parameter for the log filter");
return -1;
}
- logfile = fopen (logfilename, "w");
+ logfile = fopen (logfilename, append ? "a" : "w");
if (!logfile) {
nbdkit_error ("fopen: %m");
return -1;
diff --git a/tests/test-log.sh b/tests/test-log.sh
index f0eacb7..b30d20e 100755
--- a/tests/test-log.sh
+++ b/tests/test-log.sh
@@ -45,7 +45,9 @@ if ! qemu-io -f raw -c 'w 1M 2M' log.img; then
fi
# Run nbdkit with logging enabled to file.
-start_nbdkit -P log.pid -U log.sock --filter=log file log.img logfile=log.log
+echo '# My log' > log.log
+start_nbdkit -P log.pid -U log.sock --filter=log file log.img \
+ logfile=log.log logappend=1
# For easier debugging, dump the final log files before removing them
# on exit.
@@ -61,6 +63,8 @@ cleanup_fn cleanup
qemu-io -f raw -c 'w -P 11 1M 2M' 'nbd+unix://?socket=log.sock'
qemu-io -r -f raw -c 'r -P 11 2M 1M' 'nbd+unix://?socket=log.sock'
+# The log should have been appended, preserving our marker.
+grep '# My log' log.log
# The log should show a write on connection 1, and read on connection 2.
grep 'connection=1 Write id=1 offset=0x100000 count=0x200000 ' log.log
grep 'connection=2 Read id=1 offset=0x200000 count=0x100000 ' log.log
--
2.17.2
5 years, 6 months
[nbdkit PATCH] test-layers: Avoid thread leak
by Eric Blake
Running 'make check' under a ThreadSanitizer build complains that
test-layers leaks a pthread; while harmless, it is also an easy
fix, by detaching the thread to ensure it gets cleaned up.
Signed-off-by: Eric Blake <eblake(a)redhat.com>
---
tests/test-layers.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/tests/test-layers.c b/tests/test-layers.c
index 00e6031..7b58699 100644
--- a/tests/test-layers.c
+++ b/tests/test-layers.c
@@ -152,6 +152,12 @@ main (int argc, char *argv[])
perror ("pthread_create");
exit (EXIT_FAILURE);
}
+ err = pthread_detach(&thread);
+ if (err) {
+ errno = err;
+ perror ("pthread_detach");
+ exit (EXIT_FAILURE);
+ }
/* Note for the purposes of this test we're not very careful about
* checking for errors (except for the bare minimum) or handling the
--
2.17.2
5 years, 6 months