[nbdkit PATCH 0/9] common: catch up with libnbd
by Laszlo Ersek
This is the counterpart series for
[libnbd PATCH 0/6] common: catch up with nbdkit
http://mid.mail-archive.com/20230301114027.336230-1-lersek@redhat.com
Contrarily to my prognosis in that cover letter, this series does
ultimately extend up to libnbd v1.15.11 in one go. As a result, once
this series is merged, the common subdirs between libnbd @ d169661119f6
and nbdkit @ end-of-this-series will be in sync as much as reasonable.
Laszlo
Eric Blake (1):
build: Silence some cppcheck warnings [partial port]
Laszlo Ersek (6):
use space consistently in function and function-like macro invocations
common/include: add TYPE_IS_POINTER() macro
common/include: extract STATIC_ASSERT() macro
force semicolon after DEFINE_VECTOR_TYPE() macro invocations
vector: introduce DEFINE_POINTER_VECTOR_TYPE()
convert string_vector_(iter(free) + reset()) to string_vector_empty()
Nir Soffer (1):
common/utils/vector.h: Remove stale reference to `size`
Richard W.M. Jones (1):
common/include: Add a function to compute log2(unsigned long)
Makefile.am | 1 +
common/allocators/allocator-internal.h | 2 +-
common/allocators/allocator.c | 2 +-
common/allocators/allocator.h | 18 +-
common/allocators/malloc.c | 4 +-
common/allocators/sparse.c | 4 +-
common/allocators/zstd.c | 4 +-
common/bitmap/bitmap.c | 2 +-
common/bitmap/bitmap.h | 16 +-
common/gpt/efi-crc32.h | 2 +-
common/include/Makefile.am | 1 +
common/include/array-size.h | 2 +-
common/include/byte-swapping.h | 24 +--
common/include/checked-overflow.h | 42 ++---
common/include/compiler-macros.h | 33 +++-
common/include/ispowerof2.h | 13 ++
common/include/iszero.h | 2 +-
common/include/minmax.h | 4 +-
common/include/nextnonzero.h | 2 +-
common/include/random.h | 4 +-
common/include/static-assert.h | 48 +++++
common/include/test-array-size.c | 71 +++-----
common/include/test-ispowerof2.c | 10 +
common/include/test-random.c | 192 ++++++++++----------
common/protocol/nbd-protocol.h | 12 +-
common/regions/regions.c | 2 +-
common/regions/regions.h | 12 +-
common/replacements/open_memstream.c | 4 +-
common/replacements/strndup.c | 2 +-
common/replacements/strndup.h | 2 +-
common/utils/cleanup.h | 24 +--
common/utils/const-string-vector.h | 4 +-
common/utils/environ.c | 3 +-
common/utils/exit-with-parent.c | 6 +-
common/utils/nbdkit-string.h | 4 +-
common/utils/string-vector.h | 11 +-
common/utils/test-vector.c | 7 +-
common/utils/utils.c | 2 +-
common/utils/utils.h | 2 +-
common/utils/vector.h | 63 +++++--
configure.ac | 3 +
filters/blocksize-policy/policy.c | 2 +-
filters/blocksize/blocksize.c | 2 +-
filters/cache/blk.h | 12 +-
filters/cache/cache.c | 2 +-
filters/cache/reclaim.c | 2 +-
filters/checkwrite/checkwrite.c | 2 +-
filters/cow/blk.h | 10 +-
filters/cow/cow.c | 2 +-
filters/ddrescue/ddrescue.c | 4 +-
filters/delay/delay.c | 4 +-
filters/error/error.c | 2 +-
filters/exitlast/exitlast.c | 2 +-
filters/exitwhen/exitwhen.c | 8 +-
filters/exportname/exportname.c | 2 +-
filters/ext2/ext2.c | 2 +-
filters/ext2/io.h | 4 +-
filters/extentlist/extentlist.c | 4 +-
filters/fua/fua.c | 2 +-
filters/gzip/gzip.c | 2 +-
filters/ip/ip.c | 12 +-
filters/limit/limit.c | 2 +-
filters/log/log.c | 2 +-
filters/log/log.h | 8 +-
filters/luks/luks-encryption.c | 4 +-
filters/luks/luks.c | 2 +-
filters/multi-conn/multi-conn.c | 6 +-
filters/nocache/nocache.c | 2 +-
filters/noextents/noextents.c | 2 +-
filters/nofilter/nofilter.c | 2 +-
filters/noparallel/noparallel.c | 2 +-
filters/nozero/nozero.c | 2 +-
filters/offset/offset.c | 2 +-
filters/partition/partition-gpt.c | 4 +-
filters/partition/partition.c | 2 +-
filters/pause/pause.c | 2 +-
filters/protect/protect.c | 4 +-
filters/rate/rate.c | 4 +-
filters/readahead/readahead.c | 2 +-
filters/readahead/readahead.h | 2 +-
filters/retry-request/retry-request.c | 2 +-
filters/retry/retry.c | 2 +-
filters/scan/scan.c | 2 +-
filters/scan/scan.h | 2 +-
filters/swab/swab.c | 2 +-
filters/tar/tar.c | 2 +-
filters/tls-fallback/tls-fallback.c | 2 +-
filters/truncate/truncate.c | 2 +-
filters/xz/blkcache.h | 8 +-
filters/xz/xz.c | 2 +-
include/nbdkit-common.h | 16 +-
plugins/blkio/blkio.c | 4 +-
plugins/cc/cc.c | 4 +-
plugins/cdi/cdi.c | 2 +-
plugins/curl/curl.c | 6 +-
plugins/curl/curldefs.h | 2 +-
plugins/curl/pool.c | 2 +-
plugins/data/data.c | 10 +-
plugins/data/format.c | 6 +-
plugins/eval/eval.c | 4 +-
plugins/example1/example1.c | 2 +-
plugins/example2/example2.c | 2 +-
plugins/example2/winexample2.c | 2 +-
plugins/example3/example3.c | 2 +-
plugins/file/file.c | 8 +-
plugins/file/winfile.c | 2 +-
plugins/floppy/floppy.c | 2 +-
plugins/floppy/virtual-floppy.c | 8 +-
plugins/floppy/virtual-floppy.h | 30 +--
plugins/full/full.c | 2 +-
plugins/guestfs/guestfs-plugin.c | 2 +-
plugins/info/info.c | 4 +-
plugins/iso/iso.c | 5 +-
plugins/libvirt/libvirt-plugin.c | 2 +-
plugins/linuxdisk/linuxdisk.c | 2 +-
plugins/linuxdisk/virtual-disk.h | 6 +-
plugins/lua/lua.c | 2 +-
plugins/memory/memory.c | 2 +-
plugins/null/null.c | 2 +-
plugins/ocaml/callbacks.h | 68 +++----
plugins/ocaml/plugin.c | 2 +-
plugins/ocaml/plugin.h | 2 +-
plugins/ondemand/ondemand.c | 2 +-
plugins/partitioning/partitioning.c | 2 +-
plugins/partitioning/virtual-disk.h | 8 +-
plugins/pattern/pattern.c | 2 +-
plugins/perl/perl.c | 32 ++--
plugins/python/modfunctions.c | 4 +-
plugins/python/plugin.h | 4 +-
plugins/random/random.c | 2 +-
plugins/ruby/ruby.c | 4 +-
plugins/sh/call.h | 6 +-
plugins/sh/sh.c | 2 +-
plugins/sparse-random/sparse-random.c | 4 +-
plugins/split/split.c | 5 +-
plugins/ssh/ssh.c | 2 +-
plugins/tcl/tcl.c | 2 +-
plugins/tmpdisk/tmpdisk.c | 2 +-
plugins/vddk/stats.c | 10 +-
plugins/vddk/vddk.c | 16 +-
plugins/vddk/vddk.h | 10 +-
plugins/zero/zero.c | 2 +-
server/connections.c | 2 +-
server/crypto.c | 2 +-
server/exports.c | 2 +-
server/extents.c | 6 +-
server/internal.h | 98 +++++-----
server/main.c | 4 +-
server/protocol-handshake-newstyle.c | 2 +-
server/public.c | 9 +-
server/socket-activation.c | 2 +-
server/sockets.c | 2 +-
server/test-public.c | 2 +-
server/usergroup.c | 2 +-
tests/cc-shebang.c | 2 +-
tests/dummy-vddk.c | 4 +-
tests/test-ansi-c-plugin.c | 2 +-
tests/test-disconnect-plugin.c | 2 +-
tests/test-flush-plugin.c | 2 +-
tests/test-layers-filter.c | 2 +-
tests/test-layers-plugin.c | 2 +-
tests/test-layers.c | 6 +-
tests/test-read-password-plugin.c | 2 +-
tests/test-shutdown-plugin.c | 2 +-
tests/test-stdio-plugin.c | 2 +-
tests/test.h | 4 +-
tests/web-server.h | 2 +-
wrapper.c | 6 +-
168 files changed, 725 insertions(+), 614 deletions(-)
create mode 100644 common/include/static-assert.h
base-commit: 648a7909b69ed9be576e379097da09e9507fb868
2 years
[V2V PATCH 0/5] Bring support for virtio-scsi back to Windows
by Andrey Drobyshev
Since commits b28cd1dc ("Remove requested_guestcaps / rcaps"), f0afc439
("Remove guestcaps_block_type Virtio_SCSI") support for installing
virtio-scsi driver is missing in virt-v2v. AFAIU plans and demands for
bringing this feature back have been out there for a while. E.g. I've
found a corresponding issue which is still open [1].
The code in b28cd1dc, f0afc439 was removed due to removing the old in-place
support. However, having the new in-place support present and bringing
this same code (partially) back with several additions and improvements,
I'm able to successfully convert and boot a Win guest with a virtio-scsi
disk controller. So please consider the following implementation of
this feature.
[1] https://github.com/libguestfs/virt-v2v/issues/12
v2v:
Andrey Drobyshev (2):
Revert "Remove guestcaps_block_type Virtio_SCSI"
convert_windows: add Inject_virtio_win.Virtio_SCSI as a possible block
type
convert/convert.ml | 2 +-
convert/convert_linux.ml | 9 +++++++--
convert/convert_windows.ml | 1 +
convert/target_bus_assignment.ml | 1 +
lib/create_ovf.ml | 1 +
lib/types.ml | 3 ++-
lib/types.mli | 2 +-
output/openstack_image_properties.ml | 7 +++++++
9 files changed, 22 insertions(+), 6 deletions(-)
common:
Andrey Drobyshev (2):
inject_virtio_win: add Virtio_SCSI to block_type
inject_virtio_win: make virtio-scsi the default block driver
Roman Kagan (1):
inject_virtio_win: match only vendor/device
mlcustomize/inject_virtio_win.ml | 25 ++++++++++++++++---------
mlcustomize/inject_virtio_win.mli | 2 +-
2 files changed, 17 insertions(+), 10 deletions(-)
--
2.31.1
2 years, 1 month
Checksums and other verification
by Richard W.M. Jones
https://github.com/kubevirt/containerized-data-importer/issues/1520
Hi Eric,
We had a question from the Kubevirt team related to the above issue.
The question is roughly if it's possible to calculate the checksum of
an image as an nbdkit filter and/or in the qemu block layer.
Supplemental #1: could qemu-img convert calculate a checksum as it goes
along?
Supplemental #2: could we detect various sorts of common errors, such
a webserver that is incorrectly configured and serves up an error page
containing "<html>"; or something which is supposed to be a disk image
but does not "look like" (in some ill-defined sense) a disk image,
eg. it has no partition table.
I'm not sure if qemu has any existing features covering the above (and
I know for sure that nbdkit doesn't).
One issue is that calculating a checksum involves a linear scan of the
image, although we can at least skip holes.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
nbdkit - Flexible, fast NBD server with plugins
https://gitlab.com/nbdkit/nbdkit
2 years, 1 month
[nbdkit PATCH 0/5] ci: Get to green status on FreeBSD and MacOS
by Eric Blake
I took the easy route of crippling what I couldn't get working, on the
grounds that partial coverage is better than none now that we have
Cirrus CI checking commits on additional platforms.
This series got me to a green checkmark:
https://gitlab.com/ebblake/nbdkit/-/pipelines/793156983
but depends on an as-yet uncommitted patch in libvirt-ci:
https://gitlab.com/libvirt/libvirt-ci/-/merge_requests/360
Eric Blake (5):
ci: Expose more env vars needed by build.sh
ci: Another attempt at MacOS
rust: Skip CI builds on MacOS
golang: Skip CI builds on MacOS and newer FreeBSD
perl: Skip CI builds on newer FreeBSD
.gitlab-ci.yml | 3 +++
ci/cirrus/build.yml | 3 +++
ci/cirrus/macos-12.vars | 4 ++--
ci/gitlab.yml | 7 +++++++
ci/gitlab/build-templates.yml | 31 ++++++++++++++++++++++++++++++-
ci/gitlab/builds.yml | 11 ++++++++---
ci/manifest.yml | 20 +++++++++++++++-----
7 files changed, 68 insertions(+), 11 deletions(-)
--
2.39.2
2 years, 1 month
[libnbd PATCH 0/6] common: catch up with nbdkit
by Laszlo Ersek
If we compare the "common" subdirectory between nbdkit @ 6b4178d0fdfe
("ci: Temporarily disable perl in MacOS", 2023-02-27) and libnbd @
d05cd8f384a7 ("Version 1.15.11.", 2023-02-28), we find differences. We
can categorize these differences along two (orthogonal) axes:
- Intentional or unintentional.
Intentional differences are for example when one of the libnbd and
nbdkit projects has either a customization or a feature that the other
project doesn't need or want. (Customizations can be small, such as a
divergence inside a header file. Feature granularity is larger: a
feature usually corresponds to a full header file.)
Unintentional differences come from missing ports, or incomplete
ports.
- Libnbd not having a change from nbdkit (whether intentionally or not),
vs. nbdkit not having a change from libnbd (ditto).
I've analyzed each of the differences found. This patch set eliminates
those differences where libnbd unintentionally lags behind nbdkit.
Later I plan to post the counterpart series for nbdkit (eliminating the
differences where nbdkit unintentionally lags behind libnbd). Note that
the counterpart series for nbdkit will not target libnbd @ d05cd8f384a7
(that is, libnbd 1.15.11) at once. Instead, it will target libnbd
v1.15.9 at first. Then I plan to port my patches between libnbd v1.15.9
and v1.15.11 to nbdkit separately -- that's going to be more work.
So this series, and the counterpart for nbdkit, will eliminate past
debt, in order to form a common baseline for porting my recent patches
from libnbd to nbdkit.
The larger LISTEN_FDNAMES series will be resumed even later.
Laszlo
Eric Blake (1):
tests: Avoid odd test behavior under NDEBUG [partial port]
Richard W.M. Jones (5):
common/include: Add unit tests [byte-swapping, isaligned, iszero]
common/include: Add further unit tests [minmax]
common: include: Define bswap_16, bswap_32 and bswap_64 [finish port]
vector: Add vector_duplicate function
common/include: Fix MIN and MAX macros so they can be nested [finish
port]
.gitignore | 4 +
common/include/Makefile.am | 20 +++
common/include/test-byte-swapping.c | 108 ++++++++++++
common/include/test-isaligned.c | 62 +++++++
common/include/test-iszero.c | 66 ++++++++
common/include/test-minmax.c | 172 ++++++++++++++++++++
common/utils/vector.h | 17 ++
configure.ac | 3 +
8 files changed, 452 insertions(+)
create mode 100644 common/include/test-byte-swapping.c
create mode 100644 common/include/test-isaligned.c
create mode 100644 common/include/test-iszero.c
create mode 100644 common/include/test-minmax.c
base-commit: d05cd8f384a7278187d7e87c4dc2a346e879773b
2 years, 1 month