[PATCH v2v] v2v: -o openstack: Allow guests to be converted to UEFI (RHBZ#1872094).
by Richard W.M. Jones
Since this output method was written the code has always been capable
of adding the hw_firmware_type = uefi image property, and this
property has been supported since at least OpenStack 12 which is years
old.
Interestingly now all of the output modes support both BIOS and UEFI.
---
v2v/output_openstack.ml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/v2v/output_openstack.ml b/v2v/output_openstack.ml
index 179b0edf63..fdc04b02f8 100644
--- a/v2v/output_openstack.ml
+++ b/v2v/output_openstack.ml
@@ -390,7 +390,7 @@ object
| None -> ""
| Some op -> " -op " ^ op)
- method supported_firmware = [ TargetBIOS ]
+ method supported_firmware = [ TargetBIOS; TargetUEFI ]
(* List of Cinder volume IDs. *)
val mutable volume_ids = []
--
2.28.0.rc2
3 years, 8 months
Running libguestfs-test-tool problem
by Zachary Taylor
Here is the output from running libguestfs-test-tool:
PATH=/home/zmt1002/.local/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/var/lib/snapd/snap/bin
SELinux: sh: getenforce: command not found
guestfs_get_append: (null)
guestfs_get_autosync: 1
guestfs_get_backend: direct
guestfs_get_backend_settings: []
guestfs_get_cachedir: /var/tmp
guestfs_get_hv: /usr/bin/qemu-system-x86_64
guestfs_get_memsize: 768
guestfs_get_network: 0
guestfs_get_path: /usr/lib/guestfs
guestfs_get_pgroup: 0
guestfs_get_program: libguestfs-test-tool
guestfs_get_recovery_proc: 1
guestfs_get_smp: 1
guestfs_get_sockdir: /tmp
guestfs_get_tmpdir: /tmp
guestfs_get_trace: 0
guestfs_get_verbose: 1
host_cpu: x86_64
Launching appliance, timeout set to 600 seconds.
libguestfs: launch: program=libguestfs-test-tool
libguestfs: launch: version=1.42.0
libguestfs: launch: backend registered: unix
libguestfs: launch: backend registered: uml
libguestfs: launch: backend registered: libvirt
libguestfs: launch: backend registered: direct
libguestfs: launch: backend=direct
libguestfs: launch: tmpdir=/tmp/libguestfsR3UAGS
libguestfs: launch: umask=0022
libguestfs: launch: euid=0
libguestfs: begin building supermin appliance
libguestfs: run supermin
libguestfs: command: run: /usr/bin/supermin
libguestfs: command: run: \ --build
libguestfs: command: run: \ --verbose
libguestfs: command: run: \ --if-newer
libguestfs: command: run: \ --lock /var/tmp/.guestfs-0/lock
libguestfs: command: run: \ --copy-kernel
libguestfs: command: run: \ -f ext2
libguestfs: command: run: \ --host-cpu x86_64
libguestfs: command: run: \ /usr/lib/guestfs/supermin.d
libguestfs: command: run: \ -o /var/tmp/.guestfs-0/appliance.d
supermin: version: 5.2.0
supermin: package handler: arch/pacman
supermin: acquiring lock on /var/tmp/.guestfs-0/lock
supermin: build: /usr/lib/guestfs/supermin.d
supermin: reading the supermin appliance
supermin: build: visiting /usr/lib/guestfs/supermin.d/base.tar.gz type
gzip base image (tar)
supermin: build: visiting /usr/lib/guestfs/supermin.d/daemon.tar.gz type
gzip base image (tar)
supermin: build: visiting /usr/lib/guestfs/supermin.d/excludefiles type
uncompressed excludefiles
supermin: build: visiting /usr/lib/guestfs/supermin.d/hostfiles type
uncompressed hostfiles
supermin: build: visiting /usr/lib/guestfs/supermin.d/init.tar.gz type
gzip base image (tar)
supermin: build: visiting /usr/lib/guestfs/supermin.d/packages type
uncompressed packages
supermin: build: visiting /usr/lib/guestfs/supermin.d/udev-rules.tar.gz
type gzip base image (tar)
supermin: mapping package names to installed packages
supermin: failure: failed to parse epoch:version-release field 2.04-11.1
libguestfs: error: /usr/bin/supermin exited with error status 1, see
debug messages above
libguestfs: closing guestfs handle 0x564e8a687240 (state 0)
libguestfs: command: run: rm
libguestfs: command: run: \ -rf /tmp/libguestfsR3UAGS
3 years, 8 months
Translations using Weblate
by Pino Toscano
Hi,
with the changes done last week in libguestfs and virt-v2v, we are now
using Weblate as our translation infrastructure; in particular, we are
using the Fedora instance:
https://translate.fedoraproject.org/projects/libguestfs/
I set up all the various components required, i.e. messages and docs
for libguestfs and virt-v2v. The docs components are "linked" to the
components of the messages, as they are in the same repository. This
means that
- updating the repository of the component with the messages will
trigger the update of the docs component automatically
- there is only one pull request for the repository with all the changes
in the different components
The recent changes I did also imply some workflow changes:
1) the translation catalogs (i.e. the .pot files) are no more generated
automatically during the build. This proved challenging, as updating
the catalog required complete dependencies for all the sources,
which may not be built depending on the available dependencies
2) we will need to regenerate the translation catalogs manually, at
least for now; considering we are on github, a possible way to
automate this is to use a GitHub Action [1], similar to the one we
set up in virt-manager [2]
3) similar to (1), .po files are no more updated from the template
during the build; Weblate handles them now, and I enabled the Weblate
addon to run msgmerge every time the template changes
As I mentioned above, now Weblate "owns" changes to the translations:
this means we don't have to change them manually anymore, and issues
must be fixed directly in Weblate (which will be then send upstream).
Weblate has a lot of checks for problems in messages and even style
nits, so hopefully this should greatly reduce the issues in
translations.
An important configuration bit needed is to notify Weblate every time
the are commits pushed in the repositories; this is done by using GitHub
webhooks, which must be configured per repository as described in the
Weblate documentation [3].
The preferred way to get translation updates on GitHub/Gitlab is via
PRs; I know that we generally don't use them, however I believe we
can accept them from Weblate, as it would simplify things a bit.
There are not often changes in translations, so IMHO there will not
be a big traffic of PRs. Also, I configured our components to push
translations changes to upstream after 24h of no activity, and to batch
commits by author, so there will be less commits sent as PR.
The default configuration in GitHub for PR merging is with git merges,
which can be changed: in the "Settings" tab of a repository, select
"Options", then uncheck "Allow merge commits". Weblate takes care of
rebasing each PR according to the upstream changes, so a "fast forward"
merge will usually work.
I think I wrote down all the important bits I wanted to mention.
Of course there will be more things to mention in the future that I
cannot think about at the moment, so please do ask for anything
related.
[1] https://github.com/features/actions
[2] https://github.com/virt-manager/virt-manager/blob/master/.github/workflow...
[3] https://docs.weblate.org/en/latest/admin/continuous.html#automatically-re...
--
Pino Toscano
3 years, 9 months
[PATCH nbdkit 0/13] Port to Windows without using a separate library.
by Richard W.M. Jones
Also available here:
https://github.com/rwmjones/nbdkit/tree/2020-windows-mingw-nolib
After a lot of work I have made the port to Windows work without using
a separate library. Instead, on Windows only, we build an "import
library" (library of stubs) which resolves references to nbdkit_*
functions in the main program and fixes up the plugin, basically the
first technique outlined in this email:
https://www.redhat.com/archives/libguestfs/2020-August/msg00268.html
Most tests fail, unsurprisingly. The vast majority fail because we
lack support for Unix domain sockets. There is some hope that we
could add this:
https://devblogs.microsoft.com/commandline/af_unix-comes-to-windows/
Although this could fix many tests, it's probably not going to be very
useful for actual Windows users. Another large cause of test failures
is lack of --run option which might also be fixed with a lot of
effort.
As before it's entirely possible to develop on, compile and test this
series using only the tools provided in Fedora or RHEL. You need a
few mingw64-* packages and wine. (Note you don't need to run wine
explicitly, there should be a binfmt handler installed which allows
you to run nbdkit.exe directly). Apart from having to use
./nbdkit.exe to run the wrapper, it should all work the same way, eg:
./nbdkit.exe -f -v memory 1G
./nbdkit.exe -f -v file disk.img
starts a RAM disk or exports a file.
This patch series should contain fixes for everything picked up on in
previous reviews.
Rich.
3 years, 9 months
[libnbd PATCH 0/2] NBD_OPT_INFO support
by Eric Blake
This replaces 13/13 of my v2 series; and now that it has pretty good
testsuite coverage and demonstrable performance improvement to
nbdinfo, I'm going ahead and pushing this now. We may still want to
add further nbd_opt_* commands for other fine-grained tuning of
negotiation, but for now, I think things have stabilized on this end,
and I can return to polishing .list_exports on the nbdkit end.
Eric Blake (2):
api: add nbd_opt_info, nbd_aio_opt_info
info: Use nbd_opt_info for fewer handles during --list
docs/libnbd.pod | 4 +
info/nbdinfo.pod | 2 +
generator/API.ml | 116 ++++++---
generator/states-newstyle-opt-go.c | 26 +-
generator/states-newstyle.c | 1 +
lib/flags.c | 12 +
lib/opt.c | 38 +++
python/t/230-opt-info.py | 82 +++++++
ocaml/tests/Makefile.am | 3 +
ocaml/tests/test_230_opt_info.ml | 108 +++++++++
tests/Makefile.am | 11 +
tests/newstyle-limited.c | 8 +
tests/opt-info.c | 196 +++++++++++++++
tests/opt-info.sh | 45 ++++
.gitignore | 1 +
TODO | 13 -
.../libnbd/libnbd_230_opt_info_test.go | 225 ++++++++++++++++++
info/nbdinfo.c | 56 +++--
18 files changed, 872 insertions(+), 75 deletions(-)
create mode 100644 python/t/230-opt-info.py
create mode 100644 ocaml/tests/test_230_opt_info.ml
create mode 100644 tests/opt-info.c
create mode 100755 tests/opt-info.sh
create mode 100644 golang/src/libguestfs.org/libnbd/libnbd_230_opt_info_test.go
--
2.28.0
3 years, 9 months
[PATCH nbdkit 0/9] Port to Windows.
by Richard W.M. Jones
Also available here:
https://github.com/rwmjones/nbdkit/tree/2020-windows-mingw
This is the port to Windows using native Windows APIs (not MSYS or
Cygwin).
This patch series is at the point where it basically now works. I can
run the server with the memory plugin, and access it remotely using
guestfish, creating filesystems and so on without any apparent
problems.
Nevertheless there are many missing features (see TODO). It would be
really nice to have a native Windows file plugin. It's hard to
imagine an NBD server being taken seriously that cannot serve files!
The major impediment to getting this upstream is probably the
requirement for libnbdkit.so (first patch). Although this is optional
and doesn't break backwards compatibility it's a rather large change
especially as I hope we're coming towards the end of the 1.22
development cycle. So this is more like 1.24 material.
It would also be nice if we were automatically doing CI on this code
so it doesn't regress. While investigating this topic I saw that
gnutls is using gitlab CI to cross-compile and test with mingw-w64:
https://gitlab.com/gnutls/gnutls/-/blob/master/.gitlab-ci.yml
Rich.
3 years, 9 months
[nbdkit PATCH] sh: Prefer dd bs=1 over iflag=count_bytes
by Eric Blake
While iflag=count_bytes combined with bs > 1 allows for more efficient
operation, it is a feature of GNU dd, and not present on other
implementations such as BSD. Sticking to just POSIX features makes
things more portable.
Signed-off-by: Eric Blake <eblake(a)redhat.com>
---
docs/nbdkit-loop.pod | 2 +-
docs/nbdkit.pod | 4 ++--
plugins/eval/nbdkit-eval-plugin.pod | 6 +++---
plugins/sh/nbdkit-sh-plugin.pod | 2 +-
plugins/sh/assemble.sh | 2 +-
plugins/sh/example.sh | 7 +++----
tests/test-cacheextents.sh | 5 ++---
tests/test-eval-file.sh | 5 ++---
tests/test-eval.sh | 3 +--
tests/test-export-name.sh | 5 ++---
tests/test-parallel-sh.sh | 5 ++---
tests/test-readahead-test-plugin.sh | 2 +-
tests/test-readahead.sh | 1 -
tests/test-retry-reopen-fail.sh | 3 +--
tests/test-retry-size.sh | 3 +--
tests/test-retry.sh | 3 +--
tests/test-shell.sh | 5 ++---
tests/test-single-sh.sh | 3 +--
tests/test-tls-fallback.sh | 2 +-
19 files changed, 28 insertions(+), 40 deletions(-)
diff --git a/docs/nbdkit-loop.pod b/docs/nbdkit-loop.pod
index 055b5750..b21c2212 100644
--- a/docs/nbdkit-loop.pod
+++ b/docs/nbdkit-loop.pod
@@ -120,7 +120,7 @@ creates a disk which contains a bad sector:
echo EIO Bad block >&2
exit 1
else
- dd if=/dev/zero count=$3 iflag=count_bytes
+ dd bs=1 if=/dev/zero count=$3
fi ;;
*) exit 2 ;;
esac
diff --git a/docs/nbdkit.pod b/docs/nbdkit.pod
index 98f069f7..67653bc0 100644
--- a/docs/nbdkit.pod
+++ b/docs/nbdkit.pod
@@ -138,7 +138,7 @@ L<nbdkit-sh-plugin(3)>:
nbdkit sh - <<'EOF'
case "$1" in
get_size) echo 1M ;;
- pread) dd if=/dev/zero count=$3 iflag=count_bytes ;;
+ pread) dd bs=1 if=/dev/zero count=$3 ;;
*) exit 2 ;;
esac
EOF
@@ -149,7 +149,7 @@ The same example as above can be written entirely on the command line
using L<nbdkit-eval-plugin(1)>:
nbdkit eval get_size='echo 1M' \
- pread='dd if=/dev/zero count=$3 iflag=count_bytes'
+ pread='dd bs=1 if=/dev/zero count=$3'
=back
diff --git a/plugins/eval/nbdkit-eval-plugin.pod b/plugins/eval/nbdkit-eval-plugin.pod
index 7126c6de..d5918e28 100644
--- a/plugins/eval/nbdkit-eval-plugin.pod
+++ b/plugins/eval/nbdkit-eval-plugin.pod
@@ -51,7 +51,7 @@ all other methods are identical).
Create a 64M read-only disk of zeroes:
nbdkit eval get_size=' echo 64M ' \
- pread=' dd if=/dev/zero count=$3 iflag=count_bytes '
+ pread=' dd bs=1 if=/dev/zero count=$3 '
The following command is the eval plugin equivalent of
L<nbdkit-file-plugin(1)> (except not as fast and missing many
@@ -60,8 +60,8 @@ features):
nbdkit eval \
config='ln -sf "$(realpath "$3")" $tmpdir/file' \
get_size='stat -Lc %s $tmpdir/file' \
- pread='dd if=$tmpdir/file skip=$4 count=$3 iflag=count_bytes,skip_bytes' \
- pwrite='dd of=$tmpdir/file seek=$4 conv=notrunc oflag=seek_bytes' \
+ pread='dd bs=1 if=$tmpdir/file skip=$4 count=$3' \
+ pwrite='dd bs=1 of=$tmpdir/file seek=$4 conv=notrunc' \
file=disk.img
=head1 PARAMETERS
diff --git a/plugins/sh/nbdkit-sh-plugin.pod b/plugins/sh/nbdkit-sh-plugin.pod
index 07d90b57..70741139 100644
--- a/plugins/sh/nbdkit-sh-plugin.pod
+++ b/plugins/sh/nbdkit-sh-plugin.pod
@@ -42,7 +42,7 @@ as the name of the script, like this:
nbdkit sh - <<'EOF'
case "$1" in
get_size) echo 1M ;;
- pread) dd if=/dev/zero count=$3 iflag=count_bytes ;;
+ pread) dd bs=1 if=/dev/zero count=$3 ;;
*) exit 2 ;;
esac
EOF
diff --git a/plugins/sh/assemble.sh b/plugins/sh/assemble.sh
index 5d994857..45b9b698 100755
--- a/plugins/sh/assemble.sh
+++ b/plugins/sh/assemble.sh
@@ -46,7 +46,7 @@ case "$1" in
echo 512
;;
pread)
- dd if=$b skip=$4 count=$3 iflag=count_bytes,skip_bytes
+ dd bs=1 if=$b skip=$4 count=$3
;;
can_write)
# Default is yes to make it writable (but writes will fail).
diff --git a/plugins/sh/example.sh b/plugins/sh/example.sh
index 4f547db0..03249b27 100755
--- a/plugins/sh/example.sh
+++ b/plugins/sh/example.sh
@@ -117,12 +117,12 @@ case "$1" in
pread)
# Read the requested part of the disk and write to stdout.
- dd iflag=skip_bytes,count_bytes skip=$4 count=$3 if=$f || exit 1
+ dd bs=1 skip=$4 count=$3 if=$f || exit 1
;;
pwrite)
# Copy data from stdin and write it to the disk.
- dd oflag=seek_bytes conv=notrunc seek=$4 of=$f || exit 1
+ dd bs=1 conv=notrunc seek=$4 of=$f || exit 1
;;
can_write)
@@ -162,8 +162,7 @@ case "$1" in
# cache)
# Implement an efficient prefetch, if desired.
# It is intentionally omitted from this example.
- # dd iflag=skip_bytes,count_bytes skip=$4 count=$3 \
- # if=$f of=/dev/null || exit 1
+ # dd bs=1 skip=$4 count=$3 if=$f of=/dev/null || exit 1
# ;;
can_cache)
diff --git a/tests/test-cacheextents.sh b/tests/test-cacheextents.sh
index 9580348c..48de9365 100755
--- a/tests/test-cacheextents.sh
+++ b/tests/test-cacheextents.sh
@@ -1,6 +1,6 @@
#!/usr/bin/env bash
# nbdkit
-# Copyright (C) 2019 Red Hat Inc.
+# Copyright (C) 2019-2020 Red Hat Inc.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
@@ -37,7 +37,6 @@ set -e
requires grep --version
requires qemu-img --version
requires qemu-io --version
-requires dd iflag=count_bytes </dev/null
sock="$(mktemp -u)"
sockurl="nbd+unix:///?socket=$sock"
@@ -71,7 +70,7 @@ case "$1" in
echo ${i}M $block_size $((i%4))
done
;;
- pread) dd if=/dev/zero count=$3 iflag=count_bytes ;;
+ pread) dd bs=1 if=/dev/zero count=$3 ;;
can_write) ;;
pwrite) dd of=/dev/null ;;
can_trim) ;;
diff --git a/tests/test-eval-file.sh b/tests/test-eval-file.sh
index 7d6f79d4..d291803f 100755
--- a/tests/test-eval-file.sh
+++ b/tests/test-eval-file.sh
@@ -39,7 +39,6 @@ set -x
requires guestfish --version
requires test -f disk
-requires dd iflag=count_bytes </dev/null
files="eval-file.img"
rm -f $files
@@ -50,8 +49,8 @@ cp disk eval-file.img
nbdkit -fv -U - eval \
config='ln -sf "$(realpath "$3")" $tmpdir/file' \
get_size='stat -Lc %s $tmpdir/file' \
- pread='dd if=$tmpdir/file skip=$4 count=$3 iflag=count_bytes,skip_bytes' \
- pwrite='dd of=$tmpdir/file seek=$4 conv=notrunc oflag=seek_bytes' \
+ pread='dd bs=1 if=$tmpdir/file skip=$4 count=$3' \
+ pwrite='dd bs=1 of=$tmpdir/file seek=$4 conv=notrunc' \
file=eval-file.img \
--run '
guestfish \
diff --git a/tests/test-eval.sh b/tests/test-eval.sh
index 07a9ea1e..212f2836 100755
--- a/tests/test-eval.sh
+++ b/tests/test-eval.sh
@@ -35,7 +35,6 @@ set -e
set -x
requires qemu-img --version
-requires dd iflag=count_bytes </dev/null
files="eval.out eval.missing"
rm -f $files
@@ -47,7 +46,7 @@ cleanup_fn rm -f $files
# nbdkit, and nbdkit shuts down before the .close callback is called.
nbdkit -U - eval \
get_size='echo 64M' \
- pread='dd if=/dev/zero count=$3 iflag=count_bytes' \
+ pread='dd bs=1 if=/dev/zero count=$3' \
missing='echo "in missing: $@" >> eval.missing; exit 2' \
unload='' \
--run 'qemu-img info $nbd; sleep 10' > eval.out
diff --git a/tests/test-export-name.sh b/tests/test-export-name.sh
index 5ff0d088..40f2ce18 100755
--- a/tests/test-export-name.sh
+++ b/tests/test-export-name.sh
@@ -1,6 +1,6 @@
#!/usr/bin/env bash
# nbdkit
-# Copyright (C) 2019 Red Hat Inc.
+# Copyright (C) 2019-2020 Red Hat Inc.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
@@ -35,7 +35,6 @@ set -e
set -x
requires nbdsh --version
-requires dd iflag=count_bytes </dev/null
sock=`mktemp -u`
files="$sock exportname.pid"
@@ -59,7 +58,7 @@ case "$1" in
echo ${#2}
;;
pread)
- printf %s "$2" | dd skip=$4 count=$3 iflag=skip_bytes,count_bytes
+ printf %s "$2" | dd bs=1 skip=$4 count=$3
;;
*) exit 2 ;;
esac
diff --git a/tests/test-parallel-sh.sh b/tests/test-parallel-sh.sh
index 8d8b9b19..4a26f0a8 100755
--- a/tests/test-parallel-sh.sh
+++ b/tests/test-parallel-sh.sh
@@ -36,7 +36,6 @@ source ./functions.sh
requires test -f file-data
requires qemu-io --version
requires timeout --version
-requires dd iflag=count_bytes </dev/null
nbdkit --dump-plugin sh | grep -q ^thread_model=parallel ||
{ echo "nbdkit lacks support for parallel requests"; exit 77; }
@@ -87,8 +86,8 @@ fi
case \$1 in
thread_model) echo parallel ;;
get_size) stat -L -c %s \$f || exit 1 ;;
- pread) dd iflag=skip_bytes,count_bytes skip=\$4 count=\$3 if=\$f || exit 1 ;;
- pwrite) dd oflag=seek_bytes conv=notrunc seek=\$4 of=\$f || exit 1 ;;
+ pread) dd bs=1 skip=\$4 count=\$3 if=\$f || exit 1 ;;
+ pwrite) dd bs=1 conv=notrunc seek=\$4 of=\$f || exit 1 ;;
can_write) ;;
*) exit 2 ;;
esac
diff --git a/tests/test-readahead-test-plugin.sh b/tests/test-readahead-test-plugin.sh
index e8c57c5c..c2af913c 100755
--- a/tests/test-readahead-test-plugin.sh
+++ b/tests/test-readahead-test-plugin.sh
@@ -40,7 +40,7 @@ case "$1" in
;;
pread)
sleep 5
- dd if=/dev/zero count=$3 iflag=count_bytes
+ dd bs=1 if=/dev/zero count=$3
;;
*)
exit 2
diff --git a/tests/test-readahead.sh b/tests/test-readahead.sh
index b1aabd88..329bcd68 100755
--- a/tests/test-readahead.sh
+++ b/tests/test-readahead.sh
@@ -40,7 +40,6 @@ set -x
requires python3 --version
requires python3 -c 'import nbd'
-requires dd iflag=count_bytes </dev/null
files="readahead.img"
rm -f $files
diff --git a/tests/test-retry-reopen-fail.sh b/tests/test-retry-reopen-fail.sh
index 55704f33..60e579e2 100755
--- a/tests/test-retry-reopen-fail.sh
+++ b/tests/test-retry-reopen-fail.sh
@@ -40,7 +40,6 @@ set -x
fail=0
requires qemu-io --version
-requires dd iflag=count_bytes </dev/null
files="retry-reopen-fail-count retry-reopen-fail-open-count
retry-reopen-fail-status"
@@ -96,7 +95,7 @@ case "$1" in
echo "EIO pread failed" >&2
exit 1
else
- dd if=/dev/zero count=$3 iflag=count_bytes
+ dd bs=1 if=/dev/zero count=$3
fi
;;
diff --git a/tests/test-retry-size.sh b/tests/test-retry-size.sh
index ac3daf69..7ddc20ca 100755
--- a/tests/test-retry-size.sh
+++ b/tests/test-retry-size.sh
@@ -37,7 +37,6 @@ set -x
fail=0
requires qemu-io --version
-requires dd iflag=count_bytes </dev/null
files="retry-size-open-count retry-size-fail"
rm -f $files
@@ -81,7 +80,7 @@ case "$1" in
touch retry-size-fail
fi ;;
esac
- dd if=/dev/zero count=$3 iflag=count_bytes
+ dd bs=1 if=/dev/zero count=$3
;;
*) exit 2 ;;
esac
diff --git a/tests/test-retry.sh b/tests/test-retry.sh
index 173bd20a..fd2513f9 100755
--- a/tests/test-retry.sh
+++ b/tests/test-retry.sh
@@ -35,7 +35,6 @@ set -e
set -x
requires qemu-img --version
-requires dd iflag=count_bytes </dev/null
files="retry.img retry-count retry-open-count"
rm -f $files
@@ -65,7 +64,7 @@ case "$1" in
echo "EIO pread failed" >&2
exit 1
else
- dd if=/dev/zero count=$3 iflag=count_bytes
+ dd bs=1 if=/dev/zero count=$3
fi
;;
diff --git a/tests/test-shell.sh b/tests/test-shell.sh
index 028b8b9c..9bf00204 100755
--- a/tests/test-shell.sh
+++ b/tests/test-shell.sh
@@ -41,8 +41,7 @@ case "$1" in
;;
pread)
- dd iflag=skip_bytes,count_bytes skip=$4 count=$3 if=$f || \
- exit 1
+ dd bs=1 skip=$4 count=$3 if=$f || exit 1
;;
pwrite)
@@ -51,7 +50,7 @@ case "$1" in
*) echo "garbage flags: '$5'" >&2
exit 1;
esac
- dd oflag=seek_bytes conv=notrunc seek=$4 of=$f || exit 1
+ dd bs=1 conv=notrunc seek=$4 of=$f || exit 1
;;
can_write)
;;
diff --git a/tests/test-single-sh.sh b/tests/test-single-sh.sh
index 63f38dff..0342e712 100755
--- a/tests/test-single-sh.sh
+++ b/tests/test-single-sh.sh
@@ -35,7 +35,6 @@ set -e
set -x
requires nbdsh --version
-requires dd iflag=count_bytes </dev/null
files="single-sh.script single-sh.log"
rm -f $files
@@ -60,7 +59,7 @@ fi
cat >single-sh.script <<\EOF
case $1 in
get_size) echo 1m ;;
- pread) dd if=/dev/zero count=$3 iflag=count_bytes ;;
+ pread) dd bs=1 if=/dev/zero count=$3 ;;
*) exit 2 ;;
esac
EOF
diff --git a/tests/test-tls-fallback.sh b/tests/test-tls-fallback.sh
index 4b48c1cb..ededae39 100755
--- a/tests/test-tls-fallback.sh
+++ b/tests/test-tls-fallback.sh
@@ -71,7 +71,7 @@ case $1 in
fi
echo $3 ;;
get_size) echo "$2" | wc -c ;;
- pread) echo "$2" | dd skip=$4 count=$3 iflag=skip_bytes,count_bytes ;;
+ pread) echo "$2" | dd bs=1 skip=$4 count=$3 ;;
can_write | can_trim) exit 0 ;;
*) exit 2 ;;
esac
--
2.28.0
3 years, 9 months
[libnbd PATCH v3 0/2] Implementing NBD_OPT_LIST
by Eric Blake
This is a subset of my v2 posting, but limited to just the
NBD_OPT_LIST handling. The biggest change since v2 is the addition of
added unit testing in all four language bindings (C, python, ocaml,
golang). The tests require nbdkit built from git on PATH, and may not
be entirely idiomatic, but I at least validated that they catch issues
(for example, adding an exit statement near the end of the test
reliably caused failure).
I'm also working on similar unit testing for opt_go/opt_list; I'm
leaning more towards having opt_go have a completion handler, if only
because opt_list has to have one (with opt_go, you can tell by whether
the state is negotiating vs. ready whether it passed or failed, but
with info, the state is always negotiating either way). That should
get me to the point of figuring out where my work-in-progress posted
in patches 12-13 of v2 was going wrong.
Eric Blake (2):
api: Add nbd_opt_list
api: Add nbd_aio_opt_list
lib/internal.h | 28 +--
generator/API.ml | 152 +++++++---------
generator/state_machine.ml | 4 +-
generator/states-newstyle-opt-list.c | 82 ++++-----
generator/states-newstyle-opt-starttls.c | 8 +-
generator/states-newstyle.c | 3 +
generator/states.c | 18 +-
lib/handle.c | 78 +-------
lib/opt.c | 74 ++++++++
python/t/220-opt-list.py | 64 +++++++
ocaml/tests/Makefile.am | 3 +
ocaml/tests/test_220_opt_list.ml | 78 ++++++++
tests/Makefile.am | 11 ++
tests/meta-base-allocation.sh | 4 +-
tests/newstyle-limited.c | 17 +-
tests/opt-list.c | 171 ++++++++++++++++++
tests/opt-list.sh | 44 +++++
examples/list-exports.c | 85 ++++++---
interop/list-exports.c | 77 ++++----
.gitignore | 1 +
.../libnbd/libnbd_220_opt_list_test.go | 110 +++++++++++
.../libnbd/libnbd_460_block_status_test.go | 6 +-
info/nbdinfo.c | 106 +++++++----
23 files changed, 881 insertions(+), 343 deletions(-)
create mode 100644 python/t/220-opt-list.py
create mode 100644 ocaml/tests/test_220_opt_list.ml
create mode 100644 tests/opt-list.c
create mode 100755 tests/opt-list.sh
create mode 100644 golang/src/libguestfs.org/libnbd/libnbd_220_opt_list_test.go
--
2.28.0
3 years, 9 months
[nbdkit] Windows errno handling
by Richard W.M. Jones
The Windows port of nbdkit
(https://github.com/rwmjones/nbdkit/tree/2020-windows-mingw) now works
to some extent. However errno handling doesn't work. The way that
Winsock handles errors is incompatible with the way we expect to work
errno in several ways. The long story is here:
https://docs.microsoft.com/en-us/windows/win32/winsock/error-codes-errno-...
https://docs.microsoft.com/en-us/windows/win32/winsock/windows-sockets-er...
The shorter story is:
- Windows functions like send, recv, etc. do not set errno.
- In fact it's not even possible for the concept of a shared global
variable to exist with Windows DLLs, or at least not without a
bunch of work:
(https://docs.microsoft.com/en-us/previous-versions/visualstudio/visual-st...)
- To get and set the thread-local error from the last function
we must call WSAGetLastError and WSASetLastError.
- Error codes are integers with superficial similarity to errno codes
but different symbols and values (eg. WSAEINTR == 10004 is similar
to EINTR).
- I'm not sure if this is just a mingw thing or a Windows thing, but
mingw defines an <errno.h> header which has its own values for
errno like EINTR, and for some reason perror(3) cannot print many
of those values. Still looking into this one ...
nbdkit does approximately 5 different things with errno:
(1a) It reads them by reading the thread-local variable errno.
(1b) It reads and compares them, eg. errno == EINTR.
(2) It writes to them to set particular errno values, eg. errno = EIO.
(3) It saves them around functions such as nbdkit_error(), which
is (1) + (2) but might be a distinct operation eg. using cleanups.
(4) It prints them with perror and %m.
(5) It handles plugins which claim errno_is_preserved.
My first thought was we could define a macro which does:
#ifdef WIN32
#define errno (translate_to_errno (WSAGetLastError ()))
#endif
which reads the last error and translates WSA* to E* codes. This
would solve (1) and is not very invasive for existing code.
We'd have to then need to wrap all assignments to errno
in a macro like:
#ifndef WIN32
#define set_errno(v) (errno = (v))
#else
#define set_errno(v) (WSASetLastError (translate_from_errno (v)))
#endif
This is very invasive for existing code. There are ~60 places in the
existing code which seem to assign to errno, but some of these either
set errno = 0 or are preserving errno (item (3) above), and so we'd
probably want to handle those a bit differently.
Printing of Winsock codes through perror() or %m actually works
(surprisingly). However it does _not_ seem to work if we try to
translate the codes to errno E* values. I need to look at exactly
what's going on here.
Number (5) is actually fairly easy to deal with because there's only
one place where we handle the errno returned by plugins
(server/plugins.c:get_error). I think we'd probably want
errno_is_preserved to mean "WSAGetLastError" or "GetLastError"
contains something of interest.
Thoughts?
Also I really need to look at how some other portable libraries like
curl and gnutls are handling this. Maybe they've already come up with
something.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
3 years, 9 months
[libnbd PATCH v2 00/13] Adding nbd_set_opt_mode to improve nbdinfo
by Eric Blake
Well, I'm not quite done (I still want to get nbdinfo to work on a
single nbd connection for all cases when reading the heads of the
file is not required), but I'm happy with patches 1-11, and 12-13
show where I'm headed for getting NBD_OPT_INFO to work. Posting
now to see if some of the earlier patches are ready to commit while
I continue working on the latter half.
Eric Blake (13):
api: Add nbd_set_full_info and friends
info: Improve info without --list
api: Add nbd_is_state_negotiating for new state
api: Permit several existing API in Negotiating state
api: Add nbd_set_opt_mode
api: Add nbd_opt_abort and nbd_aio_opt_abort
api: Add nbd_opt_go and nbd_aio_opt_go
examples: Update list-exports to demonstrate nbd_opt_go
info: Simplify by using nbd_opt_go
api: Add nbd_opt_list
api: Add nbd_aio_opt_list
wip: api: Give aio_opt_go a completion callback
wip: api: add nbd_opt_info, nbd_aio_opt_info
docs/libnbd.pod | 42 +-
lib/internal.h | 45 +-
lib/nbd-protocol.h | 10 +-
generator/API.ml | 478 +++++++++++++-----
generator/API.mli | 1 +
generator/C.ml | 4 +-
generator/state_machine.ml | 37 +-
generator/states-magic.c | 6 +-
generator/states-newstyle-opt-go.c | 88 +++-
generator/states-newstyle-opt-list.c | 83 ++-
.../states-newstyle-opt-set-meta-context.c | 4 +-
generator/states-newstyle-opt-starttls.c | 13 +-
.../states-newstyle-opt-structured-reply.c | 8 +-
generator/states-newstyle.c | 80 ++-
generator/states.c | 18 +-
lib/connect.c | 5 +-
lib/flags.c | 28 +-
lib/handle.c | 76 +--
lib/is-state.c | 14 +-
lib/opt.c | 221 ++++++++
lib/Makefile.am | 3 +-
tests/Makefile.am | 14 +
tests/errors.c | 11 +-
tests/newstyle-limited.c | 58 ++-
tests/oldstyle.c | 29 +-
tests/opt-abort.c | 95 ++++
tests/opt-info.c | 107 ++++
examples/list-exports.c | 139 +++--
examples/server-flags.c | 23 +
interop/list-exports.c | 77 +--
.gitignore | 2 +
info/Makefile.am | 7 +-
info/info-description.sh | 56 ++
info/nbdinfo.c | 167 +++---
34 files changed, 1613 insertions(+), 436 deletions(-)
create mode 100644 lib/opt.c
create mode 100644 tests/opt-abort.c
create mode 100644 tests/opt-info.c
create mode 100755 info/info-description.sh
--
2.28.0
3 years, 9 months