[PATCH nbdkit 0/7] server: Allow datapath debug messages to be suppressed.
by Richard W.M. Jones
The immediate reason for this patch is to reduce the amount of
debugging in virt-v2v with using the virt-v2v -v option (because this
implies running nbdkit in verbose mode too). Most of the messages are
datapath ones about pread/pwrite requests, and in fact as we've added
more filters on top of nbdkit these messages have got more and more
verbose. However they are not particularly interesting so by using
-D nbdkit.backend.datapath=0 we could suppress all these messages,
reducing the size of the log files by over 2/3rds.
Rich.
4 years, 5 months
[PATCH 0/4] Various small build fixes
by Pino Toscano
*** BLURB HERE? ***
Pino Toscano (4):
build: stop shipping files generated by configure
docs: fix out-of-source documentation build
generator: do not generate mlv2v files when not needed
perl: fix path to Build.PL
docs/Makefile.am | 4 ++--
generator/main.ml | 11 +++++++----
perl/Makefile.am | 2 +-
python/Makefile.am | 3 ++-
tests/daemon/Makefile.am | 1 -
5 files changed, 12 insertions(+), 9 deletions(-)
--
2.23.0
4 years, 5 months
[common PATCH] options: ship key-option.pod
by Pino Toscano
Make sure that key-option.pod is packed in release tarballs.
Followup of commit 22d796efc69de44d0ff1aad6796ac4b31b8b3dc7.
---
options/Makefile.am | 3 +++
1 file changed, 3 insertions(+)
diff --git a/options/Makefile.am b/options/Makefile.am
index 2f10d53..28940f1 100644
--- a/options/Makefile.am
+++ b/options/Makefile.am
@@ -17,6 +17,9 @@
include $(top_srcdir)/subdir-rules.mk
+EXTRA_DIST = \
+ key-option.pod
+
# liboptions.la contains guestfish code which is used in other
# C tools for options parsing and a few other things
noinst_LTLIBRARIES = liboptions.la
--
2.23.0
4 years, 5 months
Re: [Libguestfs] Fwd: libguestfs incorrectly detects host CPU architecture
by Richard W.M. Jones
On Thu, Dec 12, 2019 at 11:01:02AM +0100, Vincent Danjean wrote:
> [resend to the good (cloned) bug, sorry for the message in the original bug,
> it was a mistake]
>
> Hi,
>
> On Thu, 5 Dec 2019 11:12:56 +0000 "Richard W.M. Jones" <rjones(a)redhat.com> wrote:
> > I believe this is a new bug and nothing much to do with:
> > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=775761
>
> So I just cloned the bug into #946594 in order to manage the bug reported
> by Pierre Neyron.
>
> > However about this new bug, what is supposed to happen is that
> > common/mlstdutils/guestfs_config.ml is generated by ./configure with
> > the correct @host_cpu@ substituted. If that's not happening then it's
> > a build issue on Debian of some kind.
>
> I closely look at the build system with him. It occurs that :
> - the Debian package use an 'out-of-source' build (ie the build is
> *not* done into the source directories)
> - Debian applies some patches to archive this
> - but the current rules about guestfs_config.ml especially target
> the source directory (whereas ./configure generates it in the
> build directory, as for all other configure generated files)
>
> Easy test :
> - remove common/mlstdutils/guestfs_config.ml in the source directory
> - build the Debian package (dpkg-buildpackage -us -uc)
> => the build fails with:
> ocamlfind ocamlopt -g -annot -safe-string -warn-error CDEFLMPSUVYZX+52-3 -ccopt '-g -O2 -fdebug-prefix-map=/home/polaris/danjean/libguestfs/libguestfs=. -fstack-protector-strong -Wformat -Werror=format-security -fno-strict-overflow -Wno-strict-overflow' -package str,unix -I . -c ../../../../common/mlstdutils/std_utils.ml -o std_utils.cmx
> File "_none_", line 1:
> Warning 58: no cmx file was found in path for module Guestfs_config, and its interface was not compiled with -opaque
>
> => the guestfs_config.ml generated in the build directory is not
> taken into account.
> => the Debian package is currently using the provided (x86-64)
> guestfs_config.ml in the source directory and ignore the generated
> one (in the build directory) on all architectures
> => the Debian package is correctly built only on x86-64...
>
> The root of the problem is in subdir-rules.mk at the root of the
> package. I found a fix the seems to work:
> =============
> --- subdir-rules.mk 2019-12-11 15:45:01.274572831 +0100
> +++ subdir-rules.mk.fixed 2019-12-11 15:44:23.738419530 +0100
> @@ -79,12 +79,12 @@
> guestfs_am_v_jar_ = $(guestfs_am_v_jar_@AM_DEFAULT_V@)
> guestfs_am_v_jar_0 = @echo " JAR " $@;
>
> -%.cmi: $(srcdir)/%.mli
> +%.cmi: %.mli
> $(guestfs_am_v_ocamlcmi)$(OCAMLFIND) ocamlc $(OCAMLFLAGS) $(OCAMLPACKAGES) -c $< -o $@
> -%.cmo: $(srcdir)/%.ml
> +%.cmo: %.ml
> $(guestfs_am_v_ocamlc)$(OCAMLFIND) ocamlc $(OCAMLFLAGS) $(OCAMLPACKAGES) -c $< -o $@
> if HAVE_OCAMLOPT
> -%.cmx: $(srcdir)/%.ml
> +%.cmx: %.ml
> $(guestfs_am_v_ocamlopt)$(OCAMLFIND) ocamlopt $(OCAMLFLAGS) $(OCAMLPACKAGES) -c $< -o $@
> endif
> =============
>
> The idea is to let "make" to check itself in the build directory and then
> in the source directory by not forcing a path and using the VPATH feature
> (as for all internal automake rules)
Pino, Hillu, what do you think?
Rich.
> Looking at the ChangeLog, I saw that the build rules about cmi/cmo/cmx/...
> seems tricky, so I would prefer that someone that know ocaml builds checks
> my patch.
> In any case, the Debian build is broken.
> As upstream does not seem to support out-of-source build, this problem should
> not appear for it. So the fix can go into a debian patch (even if I think that
> my patch is a no-op for a 'in-source' build)
>
> Regards,
> Vincent
>
>
> > Rich.
> >
> > --
> > Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
> > Read my programming and virtualization blog: http://rwmj.wordpress.com
> > Fedora Windows cross-compiler. Compile Windows programs, test, and
> > build Windows installers. Over 100 libraries supported.
> > http://fedoraproject.org/wiki/MinGW
> >
> >
> >
>
> --
> Vincent Danjean GPG key ID 0xD17897FA vdanjean(a)debian.org
> GPG key fingerprint: 621E 3509 654D D77C 43F5 CA4A F6AE F2AF D178 97FA
> Unofficial pkgs: http://moais.imag.fr/membres/vincent.danjean/deb.html
> APT repo: deb http://people.debian.org/~vdanjean/debian unstable main
>
> --
> Vincent Danjean GPG key ID 0xD17897FA vdanjean(a)debian.org
> GPG key fingerprint: 621E 3509 654D D77C 43F5 CA4A F6AE F2AF D178 97FA
> Unofficial pkgs: http://moais.imag.fr/membres/vincent.danjean/deb.html
> APT repo: deb http://people.debian.org/~vdanjean/debian unstable main
>
> --
> Vincent Danjean GPG key ID 0xD17897FA vdanjean(a)debian.org
> GPG key fingerprint: 621E 3509 654D D77C 43F5 CA4A F6AE F2AF D178 97FA
> Unofficial pkgs: http://moais.imag.fr/membres/vincent.danjean/deb.html
> APT repo: deb http://people.debian.org/~vdanjean/debian unstable main
>
> --
> Vincent Danjean GPG key ID 0xD17897FA vdanjean(a)debian.org
> GPG key fingerprint: 621E 3509 654D D77C 43F5 CA4A F6AE F2AF D178 97FA
> Unofficial pkgs: http://moais.imag.fr/membres/vincent.danjean/deb.html
> APT repo: deb http://people.debian.org/~vdanjean/debian unstable main
>
> --
> Vincent Danjean GPG key ID 0xD17897FA vdanjean(a)debian.org
> GPG key fingerprint: 621E 3509 654D D77C 43F5 CA4A F6AE F2AF D178 97FA
> Unofficial pkgs: http://moais.imag.fr/membres/vincent.danjean/deb.html
> APT repo: deb http://people.debian.org/~vdanjean/debian unstable main
>
> --
> Vincent Danjean GPG key ID 0xD17897FA vdanjean(a)debian.org
> GPG key fingerprint: 621E 3509 654D D77C 43F5 CA4A F6AE F2AF D178 97FA
> Unofficial pkgs: http://moais.imag.fr/membres/vincent.danjean/deb.html
> APT repo: deb http://people.debian.org/~vdanjean/debian unstable main
>
> --
> Vincent Danjean GPG key ID 0xD17897FA vdanjean(a)debian.org
> GPG key fingerprint: 621E 3509 654D D77C 43F5 CA4A F6AE F2AF D178 97FA
> Unofficial pkgs: http://moais.imag.fr/membres/vincent.danjean/deb.html
> APT repo: deb http://people.debian.org/~vdanjean/debian unstable main
>
> --
> Vincent Danjean GPG key ID 0xD17897FA vdanjean(a)debian.org
> GPG key fingerprint: 621E 3509 654D D77C 43F5 CA4A F6AE F2AF D178 97FA
> Unofficial pkgs: http://moais.imag.fr/membres/vincent.danjean/deb.html
> APT repo: deb http://people.debian.org/~vdanjean/debian unstable main
>
> --
> Vincent Danjean GPG key ID 0xD17897FA vdanjean(a)debian.org
> GPG key fingerprint: 621E 3509 654D D77C 43F5 CA4A F6AE F2AF D178 97FA
> Unofficial pkgs: http://moais.imag.fr/membres/vincent.danjean/deb.html
> APT repo: deb http://people.debian.org/~vdanjean/debian unstable main
>
> --
> Vincent Danjean GPG key ID 0xD17897FA vdanjean(a)debian.org
> GPG key fingerprint: 621E 3509 654D D77C 43F5 CA4A F6AE F2AF D178 97FA
> Unofficial pkgs: http://moais.imag.fr/membres/vincent.danjean/deb.html
> APT repo: deb http://people.debian.org/~vdanjean/debian unstable main
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines. Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
4 years, 5 months
[PATCH 1/2] podcheck: __INCLUDE:file.pod__ and __VERBATIM:file.txt__ in POD files.
by Pino Toscano
Make sure the pod checker script can deal with the newer additions of
podwrapper.pl.
Followup of commit 46e59e9535c2fcd1c188464b5249a249f22af1a0.
---
podcheck.pl | 36 ++++++++++++++++++++++++++++++++++++
1 file changed, 36 insertions(+)
diff --git a/podcheck.pl b/podcheck.pl
index 527a2e47d..795fe0e9b 100755
--- a/podcheck.pl
+++ b/podcheck.pl
@@ -83,6 +83,15 @@ used where the POD includes patterns which podwrapper would substitute.
=cut
+my @paths;
+
+=item B<--path DIR>
+
+This works like the L<podwrapper.pl(1)> I<--path> option and should be
+used where the POD includes patterns which podwrapper would substitute.
+
+=cut
+
# Clean up the program name.
my $progname = $0;
$progname =~ s{.*/}{};
@@ -91,6 +100,7 @@ $progname =~ s{.*/}{};
GetOptions ("help|?" => \$help,
"ignore=s" => \$ignore,
"insert=s" => \@inserts,
+ "path=s" => \@paths,
"verbatim=s" => \@verbatims,
) or pod2usage (2);
pod2usage (1) if $help;
@@ -117,6 +127,10 @@ foreach (@inserts) {
if $content eq $oldcontent;
}
+# Perform INCLUDE directives.
+$content =~ s{__INCLUDE:([-a-z0-9_]+\.pod)__}
+ {read_whole_file ("$1", use_path => 1)}ge;
+
# Perform @verbatims.
foreach (@verbatims) {
my @a = split /:/, $_, 2;
@@ -128,6 +142,10 @@ foreach (@verbatims) {
if $content eq $oldcontent;
}
+# Perform VERBATIM directives.
+$content =~ s{__VERBATIM:([-a-z0-9_]+\.txt)__}
+ {read_verbatim_file ("$1", use_path => 1)}ge;
+
# Run the tool with --long-options and --short-options.
my @tool_options = ();
open PIPE, "$tool --long-options |"
@@ -235,11 +253,27 @@ printf "$progname: $tool: checked $tool_options_checked tool options, $pod_optio
exit 0;
+sub find_file
+{
+ my $input = shift;
+ my $use_path = shift;
+ local $_;
+
+ my @search_path = (".");
+ push (@search_path, @paths) if $use_path;
+ foreach (@search_path) {
+ return "$_/$input" if -f "$_/$input";
+ }
+ die "$progname: $input: cannot find input file on path"
+}
+
sub read_whole_file
{
my $input = shift;
+ my %options = @_;
local $/ = undef;
+ $input = find_file ($input, $options{use_path});
open FILE, $input or die "$progname: $input: $!";
$_ = <FILE>;
close FILE;
@@ -249,8 +283,10 @@ sub read_whole_file
sub read_verbatim_file
{
my $input = shift;
+ my %options = @_;
my $r = "";
+ $input = find_file ($input, $options{use_path});
open FILE, $input or die "$progname: $input: $!";
while (<FILE>) {
$r .= " $_";
--
2.23.0
4 years, 5 months
[v2v PATCH] v2v: -o libvirt: write CentOS 8 osinfo ID
by Pino Toscano
CentOS 8 is represented with a simpler osinfo ID without a minor
version.
---
v2v/create_libvirt_xml.ml | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/v2v/create_libvirt_xml.ml b/v2v/create_libvirt_xml.ml
index dbc24315..002ce2b8 100644
--- a/v2v/create_libvirt_xml.ml
+++ b/v2v/create_libvirt_xml.ml
@@ -43,9 +43,14 @@ let get_osinfo_id = function
i_major_version = major; i_minor_version = minor } when major < 7 ->
Some (sprintf "http://centos.org/centos/%d.%d" major minor)
- | { i_type = "linux"; i_distro = "centos"; i_major_version = major } ->
+ | { i_type = "linux"; i_distro = "centos"; i_major_version = major }
+ when major = 7 ->
Some (sprintf "http://centos.org/centos/%d.0" major)
+ | { i_type = "linux"; i_distro = "centos"; i_major_version = major }
+ when major >= 8 ->
+ Some (sprintf "http://centos.org/centos/%d" major)
+
| { i_type = "linux"; i_distro = "sles";
i_major_version = major; i_minor_version = 0;
i_product_name = product } when String.find product "Desktop" >= 0 ->
--
2.23.0
4 years, 5 months
[PATCH] inspect: correct osinfo ID for CentOS >= 8
by Pino Toscano
CentOS 8 is represented with a simpler "centos8" osinfo ID.
---
lib/inspect-osinfo.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/lib/inspect-osinfo.c b/lib/inspect-osinfo.c
index 90b7ffd86..ba07e4564 100644
--- a/lib/inspect-osinfo.c
+++ b/lib/inspect-osinfo.c
@@ -40,7 +40,9 @@ guestfs_impl_inspect_get_osinfo (guestfs_h *g, const char *root)
if (STREQ (type, "linux")) {
if (STREQ (distro, "centos")) {
- if (major >= 7)
+ if (major >= 8)
+ return safe_asprintf (g, "%s%d", distro, major);
+ else if (major == 7)
return safe_asprintf (g, "%s%d.0", distro, major);
else if (major == 6)
return safe_asprintf (g, "%s%d.%d", distro, major, minor);
--
2.23.0
4 years, 5 months
[PATCH] rhv-upload: Fix waiting for transfer
by Nir Soffer
We were not considering failures while initializing the transfer. In
this case the transfer phase can change to PAUSED_SYSTEM or
FINISHED_FAILURE, and transfer_url will be None, which failed the
upload with a misleading error:
RuntimeError: direct upload to host not supported, requires
ovirt-engine >= 4.2 and only works when virt-v2v is run within the
oVirt/RHV environment, eg. on an oVirt node
Change the wait loop to consider all cases:
- Transfer failed and was removed
- Transfer failed and will be removed soon
- Transfer paused by the system (cancel required)
- Unexpected transfer phase (cancel required)
- Timeout waiting for TRANSFERRING state (cancel required)
Reported-by: Xiaodai Wang
---
I could easy simulate the case when the system paused the transfer by
injecting an error in vdsm, failing transfer initialization.
The import fail with:
nbdkit: python[1]: error: /home/nsoffer/src/virt-v2v/tmp/rhvupload.1DgXyh/rhv-upload-plugin.py: open: error: Traceback (most recent call last):
File "/home/nsoffer/src/virt-v2v/tmp/rhvupload.1DgXyh/rhv-upload-plugin.py", line 109, in open
transfer = create_transfer(connection, disk, host)
File "/home/nsoffer/src/virt-v2v/tmp/rhvupload.1DgXyh/rhv-upload-plugin.py", line 539, in create_transfer
"transfer %s was paused by system" % transfer.id)
RuntimeError: transfer 32b97384-ac8b-40d5-b423-26d31faabe32 was paused by system
I could not simulate the other cases. This probaly requires injecting
errors in engine.
v2v/rhv-upload-plugin.py | 40 +++++++++++++++++++++++++++++++---------
1 file changed, 31 insertions(+), 9 deletions(-)
diff --git a/v2v/rhv-upload-plugin.py b/v2v/rhv-upload-plugin.py
index 75e4f404..3942ec72 100644
--- a/v2v/rhv-upload-plugin.py
+++ b/v2v/rhv-upload-plugin.py
@@ -513,21 +513,43 @@ def create_transfer(connection, disk, host):
# Get a reference to the created transfer service.
transfer_service = transfers_service.image_transfer_service(transfer.id)
- # After adding a new transfer for the disk, the transfer's status
- # will be INITIALIZING. Wait until the init phase is over. The
- # actual transfer can start when its status is "Transferring".
+ # Wait until transfer's phase change from INITIALIZING to TRANSFERRING. On
+ # errors transfer's phase can change to PAUSED_SYSTEM or FINISHED_FAILURE.
+ # If the transfer was paused, we need to cancel it to remove the disk,
+ # otherwise the system will remove the disk and transfer shortly after.
+
endt = time.time() + timeout
while True:
- transfer = transfer_service.get()
- if transfer.phase != types.ImageTransferPhase.INITIALIZING:
+ time.sleep(1)
+ try:
+ transfer = transfer_service.get()
+ except sdk.NotFoundError:
+ # The system has removed the disk and the transfer.
+ raise RuntimeError("transfer %s was removed" % transfer.id)
+
+ if transfer.phase == types.ImageTransferPhase.FINISHED_FAILURE:
+ # The system will remove the disk and the transfer soon.
+ raise RuntimeError(
+ "transfer %s has failed" % transfer.id)
+
+ if transfer.phase == types.ImageTransferPhase.PAUSED_SYSTEM:
+ transfer_service.cancel()
+ raise RuntimeError(
+ "transfer %s was paused by system" % transfer.id)
+
+ if transfer.phase == types.ImageTransferPhase.TRANSFERRING:
break
- if time.time() > endt:
+
+ if transfer.phase != types.ImageTransferPhase.INITIALIZING:
transfer_service.cancel()
raise RuntimeError(
- "timed out waiting for transfer %s status != INITIALIZING"
- % transfer.id)
+ "unexpected transfer %s phase %s"
+ % (transfer.id, transfer.phase))
- time.sleep(1)
+ if time.time() > endt:
+ transfer_service.cancel()
+ raise RuntimeError(
+ "timed out waiting for transfer %s" % transfer.id)
return transfer
--
2.21.0
4 years, 5 months
OCaml 4.09.0 rebuild complete in Rawhide (was: Re: OCaml 4.09.0 will be added to Fedora 32 via a side tag)
by Richard W.M. Jones
This is now complete and soon the new packages will be merged into
Fedora Rawhide:
https://bodhi.fedoraproject.org/updates/FEDORA-2019-2e0b2d6395
A few non-critical packages failed to build, and I will look at these
later unless someone gets around to it before me. The failures are
listed at the end.
Notable changes:
- Release notes: https://ocaml.org/releases/4.09.0.html
- ocaml-camlp4 (grammar extensions) has finally been deprecated. All
dependent packages were retired already in Fedora 31, and
ocaml-camlp4 itself will soon be retired. Use ocaml-camlp5 or PPX
extension points instead.
- ocamlopt -p (native profiling with gprof) has been removed
upstream. Their argument is that there are better tools (eg perf)
and they work fine with OCaml code.
https://github.com/ocaml/ocaml/issues/2314
- ocaml-x11 (various demo-level APIs for accessing X11 directly) was
moved out of the OCaml distribution upstream. While we might
consider packaging this separately, my advise is to use Gtk etc
instead.
- caml_named_value returns a const pointer. While this is correct,
it also broke a couple of our packages and will require upstream
fixes.
- RISC-V support should be in better shape.
Rich.
ocaml-tplib
https://koji.fedoraproject.org/koji/buildinfo?buildID=1420345
ocaml-p3l
https://koji.fedoraproject.org/koji/buildinfo?buildID=1420339
libguestfs
https://koji.fedoraproject.org/koji/buildinfo?buildID=1420315
ocaml-camlimages
https://koji.fedoraproject.org/koji/buildinfo?buildID=1420324
ocaml-augeas
https://koji.fedoraproject.org/koji/buildinfo?buildID=1420320
why3
https://koji.fedoraproject.org/koji/buildinfo?buildID=1420306
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
4 years, 5 months