[PATCH v2] launch: add support for autodetection of appliance image format
by Pavel Butsykin
This feature allows you to use different image formats for the fixed
appliance. The raw format is used by default.
Signed-off-by: Pavel Butsykin <pbutsykin(a)virtuozzo.com>
---
lib/launch-direct.c | 2 ++
lib/launch-libvirt.c | 19 ++++++++++++-------
m4/guestfs_appliance.m4 | 11 +++++++++++
3 files changed, 25 insertions(+), 7 deletions(-)
diff --git a/lib/launch-direct.c b/lib/launch-direct.c
index 0be662e25..b9b54857a 100644
--- a/lib/launch-direct.c
+++ b/lib/launch-direct.c
@@ -592,7 +592,9 @@ launch_direct (guestfs_h *g, void *datav, const char *arg)
append_list ("id=appliance");
append_list ("cache=unsafe");
append_list ("if=none");
+#ifndef APPLIANCE_FMT_AUTO
append_list ("format=raw");
+#endif
} end_list ();
start_list ("-device") {
append_list ("scsi-hd");
diff --git a/lib/launch-libvirt.c b/lib/launch-libvirt.c
index 4adb2cfb3..030ea6911 100644
--- a/lib/launch-libvirt.c
+++ b/lib/launch-libvirt.c
@@ -212,9 +212,10 @@ get_source_format_or_autodetect (guestfs_h *g, struct drive *drv)
/**
* Create a qcow2 format overlay, with the given C<backing_drive>
- * (file). The C<format> parameter, which must be non-NULL, is the
- * backing file format. This is used to create the appliance overlay,
- * and also for read-only drives.
+ * (file). The C<format> parameter is the backing file format.
+ * The C<format> parameter can be NULL, in this case the backing
+ * format will be determined automatically. This is used to create
+ * the appliance overlay, and also for read-only drives.
*/
static char *
make_qcow2_overlay (guestfs_h *g, const char *backing_drive,
@@ -223,8 +224,6 @@ make_qcow2_overlay (guestfs_h *g, const char *backing_drive,
char *overlay;
struct guestfs_disk_create_argv optargs;
- assert (format != NULL);
-
if (guestfs_int_lazy_make_tmpdir (g) == -1)
return NULL;
@@ -232,8 +231,10 @@ make_qcow2_overlay (guestfs_h *g, const char *backing_drive,
optargs.bitmask = GUESTFS_DISK_CREATE_BACKINGFILE_BITMASK;
optargs.backingfile = backing_drive;
- optargs.bitmask |= GUESTFS_DISK_CREATE_BACKINGFORMAT_BITMASK;
- optargs.backingformat = format;
+ if (format) {
+ optargs.bitmask |= GUESTFS_DISK_CREATE_BACKINGFORMAT_BITMASK;
+ optargs.backingformat = format;
+ }
if (guestfs_disk_create_argv (g, overlay, "qcow2", -1, &optargs) == -1) {
free (overlay);
@@ -461,7 +462,11 @@ launch_libvirt (guestfs_h *g, void *datav, const char *libvirt_uri)
/* Note that appliance can be NULL if using the old-style appliance. */
if (appliance) {
+#ifdef APPLIANCE_FMT_AUTO
+ params.appliance_overlay = make_qcow2_overlay (g, appliance, NULL);
+#else
params.appliance_overlay = make_qcow2_overlay (g, appliance, "raw");
+#endif
if (!params.appliance_overlay)
goto cleanup;
}
diff --git a/m4/guestfs_appliance.m4 b/m4/guestfs_appliance.m4
index 81c43879f..4e1ec8135 100644
--- a/m4/guestfs_appliance.m4
+++ b/m4/guestfs_appliance.m4
@@ -139,3 +139,14 @@ AC_SUBST([GUESTFS_DEFAULT_PATH])
AC_DEFINE_UNQUOTED([GUESTFS_DEFAULT_PATH], ["$GUESTFS_DEFAULT_PATH"],
[Define guestfs default path.])
+
+AC_ARG_ENABLE([appliance-fmt-auto],
+ [AS_HELP_STRING([--enable-appliance-fmt-auto],
+ [enable autodetection of appliance image format @<:@default=no@:>@])],
+ [ENABLE_APPLIANCE_FMT_AUTO="$enableval"],
+ [ENABLE_APPLIANCE_FMT_AUTO=no])
+
+if test "x$ENABLE_APPLIANCE_FMT_AUTO" = "xyes"; then
+ AC_DEFINE([APPLIANCE_FMT_AUTO], [1],
+ [Define to 1 if enabled autodetection of appliance image format.])
+fi
--
2.13.0
4 years, 3 months
1.39 proposal: Let's split up the libguestfs git repo and tarballs
by Richard W.M. Jones
My contention is that the libguestfs git repository is too large and
unwieldy. There are too many separate, unrelated projects and as a
result of that the source has too many dependencies and takes too long
to build and test.
The project divides (sort of) naturally into layers -- the library,
the bindings, the various virt tools -- and could be split along those
lines into separate projects which can then be released and evolve at
their own pace.
My suggested split would be something like this:
* libguestfs: The library, daemon and appliance. That would include
the following directories in a single project:
appliance
bash
contrib
daemon
docs
examples
gnulib
lib
logo
test-tool
tmp
utils
website
* 1 project for each language binding:
csharp
erlang
gobject
golang
haskell
java
lua
ocaml
php
perl
python
ruby
* virt-customize and related tools, we'd probably call this subproject
"virt-builder". It would include virt-builder, virt-customize and
virt-sysprep, since they share a lot of common code.
* 1 project for each of the following items:
small tools written in C
(virt-cat, virt-filesystems, virt-log, virt-ls, virt-tail,
virt-diff, virt-edit, virt-format, guestmount, virt-inspector,
virt-make-fs, virt-rescue)
guestfish
virt-alignment-scan and virt-df
virt-dib
virt-get-kernel
virt-resize
virt-sparsify
virt-v2v and virt-p2v
virt-win-reg
* I'd be inclined to drop the legacy Perl tools virt-tar,
virt-list-filesystems, virt-list-partitions unless someone
especially wished to step forward to maintain them.
* common code and generator: Off to the side we'd somehow need to
package up the common code and the generator for use by all of the
above projects. It wouldn't be a separate project for downstream
packagers, but instead the code would be included (ie. duplicated)
in tarballs and upstream available as a side git repo that you'd
need to include when building (git submodule?). This is somewhat
unspecified.
M4, PO, and tests would be split between the projects as appropriate.
My proposal would be to do this incrementally, rather than all at
once, moving the easier things out first.
Thoughts?
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines. Supports shell scripting,
bindings from many languages. http://libguestfs.org
4 years, 5 months
cross-project patches: Add NBD Fast Zero support
by Eric Blake
This is a cover letter to a series of patches being proposed in tandem
to four different projects:
- nbd: Document a new NBD_CMD_FLAG_FAST_ZERO command flag
- qemu: Implement the flag for both clients and server
- libnbd: Implement the flag for clients
- nbdkit: Implement the flag for servers, including the nbd passthrough
client
If you want to test the patches together, I've pushed a 'fast-zero'
branch to each of:
https://repo.or.cz/nbd/ericb.git/shortlog/refs/heads/fast-zero
https://repo.or.cz/qemu/ericb.git/shortlog/refs/heads/fast-zero
https://repo.or.cz/libnbd/ericb.git/shortlog/refs/heads/fast-zero
https://repo.or.cz/nbdkit/ericb.git/shortlog/refs/heads/fast-zero
I've run several tests to demonstrate why this is useful, as well as
prove that because I have multiple interoperable projects, it is worth
including in the NBD standard. The original proposal was here:
https://lists.debian.org/nbd/2019/03/msg00004.html
where I stated:
> I will not push this without both:
> - a positive review (for example, we may decide that burning another
> NBD_FLAG_* is undesirable, and that we should instead have some sort
> of NBD_OPT_ handshake for determining when the server supports
> NBD_CMF_FLAG_FAST_ZERO)
> - a reference client and server implementation (probably both via qemu,
> since it was qemu that raised the problem in the first place)
Consensus on that thread seemed to be that a new NBD_FLAG was okay; and
this thread solves the second bullet of having reference implementations.
Here's what I did for testing full-path interoperability:
nbdkit memory -> qemu-nbd -> nbdkit nbd -> nbdsh
$ nbdkit -p 10810 --filter=nozero --filter=delay memory 1m delay-write=3
zeromode=emulate
$ qemu-nbd -p 10811 -f raw nbd://localhost:10810
$ nbdkit -p 10812 nbd nbd://localhost:10811
$ time nbdsh --connect nbd://localhost:10812 -c 'buf = h.zero(512, 0)'
# takes more than 3 seconds, but succeeds
$ time nbdsh --connect nbd://localhost:10812 -c 'buf = h.zero(512, 0,
nbd.CMD_FLAG_FAST_ZERO)'
# takes less than 1 second to fail with ENOTSUP
And here's some demonstrations on why the feature matters, starting with
this qemu thread as justification:
https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg06389.html
First, I had to create a scenario where falling back to writes is
noticeably slower than performing a zero operation, and where
pre-zeroing also shows an effect. My choice: let's test 'qemu-img
convert' on an image that is half-sparse (every other megabyte is a
hole) to an in-memory nbd destination. Then I use a series of nbdkit
filters to force the destination to behave in various manners:
log logfile=>(sed ...|uniq -c) (track how many normal/fast zero
requests the client makes)
nozero $params (fine-tune how zero requests behave - the parameters
zeromode and fastzeromode are the real drivers of my various tests)
blocksize maxdata=256k (allows large zero requests, but forces large
writes into smaller chunks, to magnify the effects of write delays and
allow testing to provide obvious results with a smaller image)
delay delay-write=20ms delay-zero=5ms (also to magnify the effects on a
smaller image, with writes penalized more than zeroing)
stats statsfile=/dev/stderr (to track overall time and a decent summary
of how much I/O occurred).
noextents (forces the entire image to report that it is allocated,
which eliminates any testing variability based on whether qemu-img uses
that to bypass a zeroing operation [1])
So here's my one-time setup, followed by repetitions of the nbdkit
command with different parameters to the nozero filter to explore
different behaviors.
$ qemu-img create -f qcow2 src 100m
$ for i in `seq 0 2 99`; do qemu-io -f qcow2 -c "w ${i}m 1m" src; done
$ nbdkit -U - --filter=log --filter=nozero --filter=blocksize \
--filter=delay --filter=stats --filter=noextents memory 100m \
logfile=>(sed -n '/Zero.*\.\./ s/.*\(fast=.\).*/\1/p' |sort|uniq -c) \
statsfile=/dev/stderr delay-write=20ms delay-zero=5s maxdata=256k \
--run 'qemu-img convert -n -f qcow2 -O raw src $nbd' $params
Establish a baseline: when qemu-img does not see write zero support at
all (such as when talking to /dev/nbd0, because the kernel NBD
implementation still does not support write zeroes), qemu is forced to
write the entire disk, including the holes, but doesn't waste any time
pre-zeroing or checking block status for whether the disk is zero (the
default of the nozero filter is to turn off write zero advertisement):
params=
elapsed time: 8.54488 s
write: 400 ops, 104857600 bytes, 9.81712e+07 bits/s
Next, let's emulate what qemu 3.1 was like, with a blind pre-zeroing
pass of the entire image without regards to whether that pass is fast or
slow. For this test, it was easier to use modern qemu and merely ignore
the fast zero bit in nbdkit, but the numbers should be similar when
actually using older qemu. If qemu guessed right that pre-zeroing is
fast, we see:
params='zeromode=plugin fastzeromode=ignore'
elapsed time: 4.30183 s
write: 200 ops, 52428800 bytes, 9.75005e+07 bits/s
zero: 4 ops, 104857600 bytes, 1.95001e+08 bits/s
4 fast=1
which is definite win - instead of having to write the half of the image
that was zero on the source, the fast pre-zeroing pass already cleared
it (qemu-img currently breaks write zeroes into 32M chunks [1], and thus
requires 4 zero requests to pre-zero the image). But if qemu guesses wrong:
params='zeromode=emulate fastzeromode=ignore'
elapsed time: 12.5065 s
write: 600 ops, 157286400 bytes, 1.00611e+08 bits/s
4 fast=1
Ouch - that is actually slower than the case when zeroing is not used at
all, because the zeroes turned into writes result in performing double
the I/O over the data portions of the file (once during the pre-zero
pass, then again during the data). The qemu 3.1 behavior is very
bi-polar in nature, and we don't like that.
So qemu 4.0 introduced BDRV_REQ_NO_FALLBACK, which qemu uses during the
pre-zero request to fail quickly if pre-zeroing is not viable. At the
time, NBD did not have a way to support fast zero requests, so qemu
blindly assumes that pre-zeroing is not viable over NBD:
params='zeromode=emulate fastzeromode=none'
elapsed time: 8.32433 s
write: 400 ops, 104857600 bytes, 1.00772e+08 bits/s
50 fast=0
When zeroing is slow, our time actually beats the baseline by about 0.2
seconds (although zeroing still turned into writes, the use of zero
requests results in less network traffic; you also see that there are 50
zero requests, one per hole, rather than 4 requests for pre-zeroing the
image). So we've avoided the pre-zeroing penalty. However:
params='zeromode=plugin fastzeromode=none'
elapsed time: 4.53951 s
write: 200 ops, 52428800 bytes, 9.23955e+07 bits/s
zero: 50 ops, 52428800 bytes, 9.23955e+07 bits/s
50 fast=0
when zeroing is fast, we're still 0.2 seconds slower than the
pre-zeroing behavior (zeroing runs fast, but one request per hole is
still more transactions than pre-zeroing used to use). The qemu 4.0
decision thus regained the worst degradation seen in 3.1 when zeroing is
slow, but at a penalty to the case when zeroing is fast.
Since guessing is never as nice as knowing, let's repeat the test, but
now exploiting the new NBD fast zero:
params='zeromode=emulate'
elapsed time: 8.41174 s
write: 400 ops, 104857600 bytes, 9.9725e+07 bits/s
50 fast=0
1 fast=1
Good: when zeroes are not fast, qemu-img's initial fast-zero request
immediately fails, and then it switches back to writing the entire image
using regular zeroing for the holes; performance is comparable to the
baseline and to the qemu 4.0 behavior.
params='zeromode=plugin'
elapsed time: 4.31356 s
write: 200 ops, 52428800 bytes, 9.72354e+07 bits/s
zero: 4 ops, 104857600 bytes, 1.94471e+08 bits/s
4 fast=1
Good: when zeroes are fast, qemu-img is able to use pre-zeroing on the
entire image, resulting in fewer zero transactions overall, getting us
back to the qemu 3.1 maximum performance (and better than the 4.0 behavior).
I hope you enjoyed reading this far, and agree with my interpretation of
the numbers about why this feature is useful!
[1] Orthogonal to these patches are other ideas I have for improving the
NBD protocol in its effects to qemu-img convert, which will result in
later cross-project patches:
- NBD should have a way to advertise (probably via NBD_INFO_ during
NBD_OPT_GO) if the initial image is known to begin life with all zeroes
(if that is the case, qemu-img can skip the extents calls and
pre-zeroing pass altogether)
- improving support to allow NBD to pass larger zero requests (qemu is
currently capping zero requests at 32m based on NBD_INFO_BLOCK_SIZE, but
could easily go up to ~4G with proper info advertisement of maximum zero
request sizing, or if we introduce 64-bit commands to the NBD protocol)
Given that NBD extensions need not be present in every server, each
orthogonal improvement should be tested in isolation to show that it
helps, even though qemu-img will probably use all of the extensions at
once when the server supports all of them.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
4 years, 8 months
[p2v PATCH 0/3] Small build fixes
by Pino Toscano
This series for virt-p2v removes the usage of GTK deprecated stuff,
and checks for a required Perl module used during build.
Tested on:
- RHEL 6 (GTK 2)
- RHEL 7 (GTK 3)
- Fedora 30 (GTK 2 & 3)
- Fedora Rawhide/32 (GTK 2 & 3)
Pino Toscano (3):
Copy GtkAttachOptions from GTK >= 3.4
build: disable deprecated GTK stuff
build: require List::MoreUtils
Makefile.am | 1 +
gui-gtk3-compat.h | 9 +++++++++
m4/p2v-progs.m4 | 8 ++++++++
3 files changed, 18 insertions(+)
--
2.21.0
4 years, 8 months
[nbdkit PATCH v2 0/2] caching .can_write
by Eric Blake
This is a subset of the last half of the larger 9-patch series. The
uncontroversial first half of that series is pushed, but here, I tried
to reduce the size of the patches by splitting out some of the more
complex changes, so that the rest of the changes remaining in the
series are more mechanical. In turn, it forced me to write timing
tests, which let me spot another spot where we are wasting time, so
patch 2 is completely new.
As we discussed on IRC, we're early enough in the 1.16 release cycle
that I'll probably just push these when I'm happy with them, whether
or not a review turns up something (we can always do followup
patches). But forcing me to reread the patches for legibility has
been a good exercise, because I really like how it forced me to spend
a good hour on the commit message of patch 1 explaining things, and
how patch 2 then fell out in 10 minutes of hacking to fix the problem
identified in passing in the patch 1 commit message.
Eric Blake (2):
server: Cache per-connection can_write
server: Remember .open(readonly) status
server/internal.h | 5 ++-
server/backend.c | 43 ++++++++++++++++++++-
server/connections.c | 5 ++-
server/filters.c | 6 +--
server/plugins.c | 4 +-
server/protocol-handshake.c | 74 ++++++++++++++++++-------------------
6 files changed, 88 insertions(+), 49 deletions(-)
--
2.21.0
4 years, 8 months
[nbdkit PATCH v2] filters: Stronger version match requirements
by Eric Blake
We documented our intent of only allowing a filter to run with the
same version of nbdkit it was compiled against, but up to now, were
not actually enforcing that - we had only been insisting on the looser
notion of a matching ._api_version, which doesn't help when we've
forgotten to bump that macro when making incompatible API/ABI changes
(see commit 6934d4c1). However, we can't use .version (it was
documented as optional, so even though all our in-tree filters set it,
a theoretical out-of-tree filter could use NULL; and it has changed
offsets in the ABI during previous API changes, such as commit
ee61d232). Thus, we have to bump the API version one final time; but
now, our API guaarantees enough stable ABI to check the version string
as part of our API sanity checking, so we are set to avoid future
maintenance snafus.
All in-tree filters are affected now that .version is no longer
available as an optional member (although it so happens that we are
setting the new mandatory ._version to the same value that they all
used). And thanks to the previous patch adding nbdkit-version.h, even
a theoretical out-of-tree filter will get the correct version string
without having to maintain it by hand.
Note that the offset of ._version differs between 32- and 64-bit
platform ABIs; but on that front, we are safe: Rich and I tested (at
least on Linux where ELF executables encode ABI) that attempts to load
a 32-bit .so from 64-bit nbdkit, or vice versa, gracefully fail
dlopen() for being an incompatible ABI even before we get far enough
to worry about inspecting ._api_version or ._version.
Signed-off-by: Eric Blake <eblake(a)redhat.com>
---
This is what I ended up committing for filter API/ABI sanity checking
(a lot nicer than playing ABI games with the unused bytes in
._api_version).
docs/nbdkit-filter.pod | 7 -------
server/filters.c | 20 +++++++++++++++-----
include/nbdkit-filter.h | 18 ++++++++++--------
filters/blocksize/blocksize.c | 1 -
filters/cache/cache.c | 1 -
filters/cacheextents/cacheextents.c | 1 -
filters/cow/cow.c | 1 -
filters/delay/delay.c | 1 -
filters/error/error.c | 1 -
filters/fua/fua.c | 1 -
filters/log/log.c | 1 -
filters/nocache/nocache.c | 1 -
filters/noextents/noextents.c | 1 -
filters/noparallel/noparallel.c | 1 -
filters/nozero/nozero.c | 1 -
filters/offset/offset.c | 1 -
filters/partition/partition.c | 1 -
filters/rate/rate.c | 1 -
filters/readahead/readahead.c | 1 -
filters/stats/stats.c | 1 -
filters/truncate/truncate.c | 1 -
filters/xz/xz.c | 1 -
tests/test-layers-filter.c | 1 -
tests/test-cxx-filter.cpp | 1 -
24 files changed, 25 insertions(+), 41 deletions(-)
diff --git a/docs/nbdkit-filter.pod b/docs/nbdkit-filter.pod
index 0cd3e26a..4bf87295 100644
--- a/docs/nbdkit-filter.pod
+++ b/docs/nbdkit-filter.pod
@@ -178,13 +178,6 @@ methods.
This field (a string) is required, and B<must> contain only ASCII
alphanumeric characters and be unique amongst all filters.
-=head2 C<.version>
-
- const char *version;
-
-Filters may optionally set a version string which is displayed in help
-and debugging output.
-
=head2 C<.longname>
const char *longname;
diff --git a/server/filters.c b/server/filters.c
index c67676a3..0e10816f 100644
--- a/server/filters.c
+++ b/server/filters.c
@@ -102,7 +102,7 @@ filter_version (struct backend *b)
{
struct backend_filter *f = container_of (b, struct backend_filter, backend);
- return f->filter.version;
+ return f->filter._version;
}
static void
@@ -730,14 +730,24 @@ filter_register (struct backend *next, size_t index, const char *filename,
}
/* We do not provide API or ABI guarantees for filters, other than
- * the ABI position of _api_version that will let us diagnose
- * mismatch when the API changes.
+ * the ABI position and API contents of _api_version and _version to
+ * diagnose mismatch from the current nbdkit version.
*/
if (filter->_api_version != NBDKIT_FILTER_API_VERSION) {
fprintf (stderr,
"%s: %s: filter is incompatible with this version of nbdkit "
- "(_api_version = %d)\n",
- program_name, filename, filter->_api_version);
+ "(_api_version = %d, need %d)\n",
+ program_name, filename, filter->_api_version,
+ NBDKIT_FILTER_API_VERSION);
+ exit (EXIT_FAILURE);
+ }
+ if (filter->_version == NULL ||
+ strcmp (filter->_version, PACKAGE_VERSION) != 0) {
+ fprintf (stderr,
+ "%s: %s: filter is incompatible with this version of nbdkit "
+ "(_version = %s, need %s)\n",
+ program_name, filename, filter->_version ?: "<null>",
+ PACKAGE_VERSION);
exit (EXIT_FAILURE);
}
diff --git a/include/nbdkit-filter.h b/include/nbdkit-filter.h
index 4f8f19ab..8423644e 100644
--- a/include/nbdkit-filter.h
+++ b/include/nbdkit-filter.h
@@ -43,7 +43,7 @@
extern "C" {
#endif
-#define NBDKIT_FILTER_API_VERSION 5 /* Corresponding to v1.14 */
+#define NBDKIT_FILTER_API_VERSION 6 /* Corresponding to v1.16+ */
struct nbdkit_extent {
uint64_t offset;
@@ -93,20 +93,21 @@ struct nbdkit_next_ops {
};
struct nbdkit_filter {
- /* Do not set this field directly; use NBDKIT_REGISTER_FILTER.
- * It exists so that we can diagnose filters compiled against
- * one version of the header with a runtime compiled against a
- * different version.
+ /* Do not set these fields directly; use NBDKIT_REGISTER_FILTER.
+ * They exist so that we can diagnose filters compiled against one
+ * version of the header with a runtime compiled against a different
+ * version. As of API version 6, _version is also part of the
+ * guaranteed ABI, so we no longer have to remember to bump API
+ * versions regardless of other API/ABI changes later in the struct.
*/
int _api_version;
+ const char *_version;
/* Because there is no ABI guarantee, new fields may be added where
- * logically appropriate, as long as we correctly bump
- * NBDKIT_FILTER_API_VERSION once per stable release.
+ * logically appropriate.
*/
const char *name;
const char *longname;
- const char *version;
const char *description;
void (*load) (void);
@@ -179,6 +180,7 @@ struct nbdkit_filter {
filter_init (void) \
{ \
(filter)._api_version = NBDKIT_FILTER_API_VERSION; \
+ (filter)._version = NBDKIT_VERSION_STRING; \
return &(filter); \
}
diff --git a/filters/blocksize/blocksize.c b/filters/blocksize/blocksize.c
index 0978887f..058c236b 100644
--- a/filters/blocksize/blocksize.c
+++ b/filters/blocksize/blocksize.c
@@ -399,7 +399,6 @@ blocksize_cache (struct nbdkit_next_ops *next_ops, void *nxdata,
static struct nbdkit_filter filter = {
.name = "blocksize",
.longname = "nbdkit blocksize filter",
- .version = PACKAGE_VERSION,
.config = blocksize_config,
.config_complete = blocksize_config_complete,
.config_help = blocksize_config_help,
diff --git a/filters/cache/cache.c b/filters/cache/cache.c
index b5dbccd2..fb852df9 100644
--- a/filters/cache/cache.c
+++ b/filters/cache/cache.c
@@ -615,7 +615,6 @@ cache_cache (struct nbdkit_next_ops *next_ops, void *nxdata,
static struct nbdkit_filter filter = {
.name = "cache",
.longname = "nbdkit caching filter",
- .version = PACKAGE_VERSION,
.load = cache_load,
.unload = cache_unload,
.config = cache_config,
diff --git a/filters/cacheextents/cacheextents.c b/filters/cacheextents/cacheextents.c
index b3828f41..e89fe580 100644
--- a/filters/cacheextents/cacheextents.c
+++ b/filters/cacheextents/cacheextents.c
@@ -192,7 +192,6 @@ cacheextents_zero (struct nbdkit_next_ops *next_ops, void *nxdata,
static struct nbdkit_filter filter = {
.name = "cacheextents",
.longname = "nbdkit cacheextents filter",
- .version = PACKAGE_VERSION,
.unload = cacheextents_unload,
.pwrite = cacheextents_pwrite,
.trim = cacheextents_trim,
diff --git a/filters/cow/cow.c b/filters/cow/cow.c
index 9d91d432..491d0009 100644
--- a/filters/cow/cow.c
+++ b/filters/cow/cow.c
@@ -482,7 +482,6 @@ cow_cache (struct nbdkit_next_ops *next_ops, void *nxdata,
static struct nbdkit_filter filter = {
.name = "cow",
.longname = "nbdkit copy-on-write (COW) filter",
- .version = PACKAGE_VERSION,
.load = cow_load,
.unload = cow_unload,
.open = cow_open,
diff --git a/filters/delay/delay.c b/filters/delay/delay.c
index c92a12d7..6b13d459 100644
--- a/filters/delay/delay.c
+++ b/filters/delay/delay.c
@@ -266,7 +266,6 @@ delay_cache (struct nbdkit_next_ops *next_ops, void *nxdata,
static struct nbdkit_filter filter = {
.name = "delay",
.longname = "nbdkit delay filter",
- .version = PACKAGE_VERSION,
.config = delay_config,
.config_help = delay_config_help,
.pread = delay_pread,
diff --git a/filters/error/error.c b/filters/error/error.c
index 968f2837..2d39b251 100644
--- a/filters/error/error.c
+++ b/filters/error/error.c
@@ -375,7 +375,6 @@ error_cache (struct nbdkit_next_ops *next_ops, void *nxdata,
static struct nbdkit_filter filter = {
.name = "error",
.longname = "nbdkit error filter",
- .version = PACKAGE_VERSION,
.load = error_load,
.unload = error_unload,
.config = error_config,
diff --git a/filters/fua/fua.c b/filters/fua/fua.c
index 9d0e561e..83f7a1a7 100644
--- a/filters/fua/fua.c
+++ b/filters/fua/fua.c
@@ -233,7 +233,6 @@ fua_zero (struct nbdkit_next_ops *next_ops, void *nxdata,
static struct nbdkit_filter filter = {
.name = "fua",
.longname = "nbdkit fua filter",
- .version = PACKAGE_VERSION,
.config = fua_config,
.config_help = fua_config_help,
.prepare = fua_prepare,
diff --git a/filters/log/log.c b/filters/log/log.c
index 7cf741e1..6d37d583 100644
--- a/filters/log/log.c
+++ b/filters/log/log.c
@@ -435,7 +435,6 @@ log_cache (struct nbdkit_next_ops *next_ops, void *nxdata,
static struct nbdkit_filter filter = {
.name = "log",
.longname = "nbdkit log filter",
- .version = PACKAGE_VERSION,
.config = log_config,
.config_complete = log_config_complete,
.config_help = log_config_help,
diff --git a/filters/nocache/nocache.c b/filters/nocache/nocache.c
index a3f11984..7ddb84bc 100644
--- a/filters/nocache/nocache.c
+++ b/filters/nocache/nocache.c
@@ -101,7 +101,6 @@ nocache_cache (struct nbdkit_next_ops *next_ops, void *nxdata,
static struct nbdkit_filter filter = {
.name = "nocache",
.longname = "nbdkit nocache filter",
- .version = PACKAGE_VERSION,
.config = nocache_config,
.config_help = nocache_config_help,
.can_cache = nocache_can_cache,
diff --git a/filters/noextents/noextents.c b/filters/noextents/noextents.c
index e39723cd..e6ac33b2 100644
--- a/filters/noextents/noextents.c
+++ b/filters/noextents/noextents.c
@@ -44,7 +44,6 @@ noextents_can_extents (struct nbdkit_next_ops *next_ops, void *nxdata,
static struct nbdkit_filter filter = {
.name = "noextents",
.longname = "nbdkit noextents filter",
- .version = PACKAGE_VERSION,
.can_extents = noextents_can_extents,
};
diff --git a/filters/noparallel/noparallel.c b/filters/noparallel/noparallel.c
index 057485fa..f9006c06 100644
--- a/filters/noparallel/noparallel.c
+++ b/filters/noparallel/noparallel.c
@@ -76,7 +76,6 @@ noparallel_thread_model (void)
static struct nbdkit_filter filter = {
.name = "noparallel",
.longname = "nbdkit noparallel filter",
- .version = PACKAGE_VERSION,
.config = noparallel_config,
.config_help = noparallel_config_help,
.thread_model = noparallel_thread_model,
diff --git a/filters/nozero/nozero.c b/filters/nozero/nozero.c
index 964cce9f..9187ec33 100644
--- a/filters/nozero/nozero.c
+++ b/filters/nozero/nozero.c
@@ -139,7 +139,6 @@ nozero_zero (struct nbdkit_next_ops *next_ops, void *nxdata,
static struct nbdkit_filter filter = {
.name = "nozero",
.longname = "nbdkit nozero filter",
- .version = PACKAGE_VERSION,
.config = nozero_config,
.config_help = nozero_config_help,
.prepare = nozero_prepare,
diff --git a/filters/offset/offset.c b/filters/offset/offset.c
index faae0328..7dc94b1d 100644
--- a/filters/offset/offset.c
+++ b/filters/offset/offset.c
@@ -175,7 +175,6 @@ offset_cache (struct nbdkit_next_ops *next_ops, void *nxdata,
static struct nbdkit_filter filter = {
.name = "offset",
.longname = "nbdkit offset filter",
- .version = PACKAGE_VERSION,
.config = offset_config,
.config_help = offset_config_help,
.get_size = offset_get_size,
diff --git a/filters/partition/partition.c b/filters/partition/partition.c
index b1b1945d..39132ce8 100644
--- a/filters/partition/partition.c
+++ b/filters/partition/partition.c
@@ -265,7 +265,6 @@ partition_cache (struct nbdkit_next_ops *next_ops, void *nxdata,
static struct nbdkit_filter filter = {
.name = "partition",
.longname = "nbdkit partition filter",
- .version = PACKAGE_VERSION,
.config = partition_config,
.config_complete = partition_config_complete,
.config_help = partition_config_help,
diff --git a/filters/rate/rate.c b/filters/rate/rate.c
index dca5e9fc..2b105f91 100644
--- a/filters/rate/rate.c
+++ b/filters/rate/rate.c
@@ -310,7 +310,6 @@ rate_pwrite (struct nbdkit_next_ops *next_ops, void *nxdata,
static struct nbdkit_filter filter = {
.name = "rate",
.longname = "nbdkit rate filter",
- .version = PACKAGE_VERSION,
.unload = rate_unload,
.config = rate_config,
.config_complete = rate_config_complete,
diff --git a/filters/readahead/readahead.c b/filters/readahead/readahead.c
index b6c18096..3ec73e45 100644
--- a/filters/readahead/readahead.c
+++ b/filters/readahead/readahead.c
@@ -247,7 +247,6 @@ readahead_zero (struct nbdkit_next_ops *next_ops, void *nxdata,
static struct nbdkit_filter filter = {
.name = "readahead",
.longname = "nbdkit readahead filter",
- .version = PACKAGE_VERSION,
.unload = readahead_unload,
.prepare = readahead_prepare,
.get_size = readahead_get_size,
diff --git a/filters/stats/stats.c b/filters/stats/stats.c
index d8ff02bc..98282e26 100644
--- a/filters/stats/stats.c
+++ b/filters/stats/stats.c
@@ -290,7 +290,6 @@ stats_cache (struct nbdkit_next_ops *next_ops, void *nxdata,
static struct nbdkit_filter filter = {
.name = "stats",
.longname = "nbdkit stats filter",
- .version = PACKAGE_VERSION,
.unload = stats_unload,
.config = stats_config,
.config_complete = stats_config_complete,
diff --git a/filters/truncate/truncate.c b/filters/truncate/truncate.c
index 93d8f074..ae1f8624 100644
--- a/filters/truncate/truncate.c
+++ b/filters/truncate/truncate.c
@@ -385,7 +385,6 @@ truncate_cache (struct nbdkit_next_ops *next_ops, void *nxdata,
static struct nbdkit_filter filter = {
.name = "truncate",
.longname = "nbdkit truncate filter",
- .version = PACKAGE_VERSION,
.config = truncate_config,
.config_help = truncate_config_help,
.open = truncate_open,
diff --git a/filters/xz/xz.c b/filters/xz/xz.c
index 51ac919f..78351859 100644
--- a/filters/xz/xz.c
+++ b/filters/xz/xz.c
@@ -254,7 +254,6 @@ static int xz_thread_model (void)
static struct nbdkit_filter filter = {
.name = "xz",
.longname = "nbdkit XZ filter",
- .version = PACKAGE_VERSION,
.config = xz_config,
.config_help = xz_config_help,
.thread_model = xz_thread_model,
diff --git a/tests/test-layers-filter.c b/tests/test-layers-filter.c
index bd063bdf..465454c0 100644
--- a/tests/test-layers-filter.c
+++ b/tests/test-layers-filter.c
@@ -261,7 +261,6 @@ test_layers_filter_cache (struct nbdkit_next_ops *next_ops, void *nxdata,
static struct nbdkit_filter filter = {
.name = "testlayers" layer,
- .version = PACKAGE_VERSION,
.load = test_layers_filter_load,
.unload = test_layers_filter_unload,
.config = test_layers_filter_config,
diff --git a/tests/test-cxx-filter.cpp b/tests/test-cxx-filter.cpp
index ba5ff29e..57299a15 100644
--- a/tests/test-cxx-filter.cpp
+++ b/tests/test-cxx-filter.cpp
@@ -49,7 +49,6 @@ namespace {
nbdkit_filter create_filter() {
nbdkit_filter filter = nbdkit_filter ();
filter.name = "cxxfilter";
- filter.version = PACKAGE_VERSION;
return filter;
}
}
--
2.21.0
4 years, 8 months
[PATCH libnbd] Add bindings for Rust language
by Richard W.M. Jones
Still not working, but I took the latest patch and:
- rebased it against libnbd 1.0
- fixed it so it handles new args and cbargs
The generator now runs without warnings.
This patch doesn't handle optargs at all. In C these are converted to
non-optional parameter. Rust doesn't (AFAIK) have optional or
labelled arguments unfortunately.
Rich.
4 years, 8 months
[nbdkit PATCH 0/2] RFC: tighter filter versions
by Eric Blake
This is not intended for v1.14. In fact, we may decide that the
second patch is too gross, although the first one still seems like a
useful improvement in isolation.
I will also point out that all our filters are in-tree, and set the
user-controlled field .version to the current release string. We
could replace the second patch with a simpler one that just checks
._api_version as an int (as before), but adds the additional
requirement that .version match PACKAGE_VERSION from config.h. Then
we don't have to hack around ABI changes of the first field member,
but are slightly more protected if we forget to bump ._api_version in
the future.
Eric Blake (2):
include: Expose nbdkit version information to public
filters: Stronger version match requirements
configure.ac | 15 ++++++++++++-
server/filters.c | 32 +++++++++++++++++++++-----
include/nbdkit-common.h | 2 ++
include/nbdkit-filter.h | 10 ++++-----
.gitignore | 1 +
include/Makefile.am | 1 +
include/nbdkit-version.h.in | 45 +++++++++++++++++++++++++++++++++++++
7 files changed, 93 insertions(+), 13 deletions(-)
create mode 100644 include/nbdkit-version.h.in
--
2.21.0
4 years, 8 months
[nbdkit PATCH 0/9] can_FOO caching, more filter validation
by Eric Blake
It's easy to use the sh script to demonstrate that nbdkit is
inefficiently calling into .get_size, .can_fua, and friends more than
necessary. We've also commented on the list in the past that it would
be nice to ensure that when filters call into next_ops, they are not
violating constraints (as we've have to fix several bugs in the past
where we did not have such checking to protect ourselves). This
series fixes both (well, we may still have buggy filters, but if so,
now we'll get earlier assertion failures instead of random plugin
behavior).
Another benefit - this gives us a single point for adding any code for
performing NBD_INFO_BLOCKSIZE support, where we'll have to hoist what
the blocksize filter is currently doing into something that nbdkit
does from any one backend calling into another with different sizing
constraints. However, that is not done in this series.
My pending work on fast zeroes was easy enough to rebase on top of
this (in fact, I wrote this because I noticed that .can_zero was being
called more than necessary in implementing .can_fast_zero).
Eric Blake (9):
server: Fewer dereferences in filter
server: Consolidate common backend tasks into new backend.c
server: Create new backend_* functions for central handling
server: Rework storage of per-backend handle
server: Cache per-connection size
server: Cache per-connection can_FOO flags
filters: Change semantics of .can_zero
server: Move fallbacks from protocol.c to backend.c
server: Move command validation from protocol.c to backend.c
docs/nbdkit-filter.pod | 62 +++-
docs/nbdkit-plugin.pod | 6 +-
server/internal.h | 108 +++++-
server/Makefile.am | 3 +-
server/backend.c | 515 +++++++++++++++++++++++++++
server/connections.c | 51 +--
server/filters.c | 379 ++++++--------------
server/main.c | 2 +-
server/plugins.c | 192 +++-------
server/protocol-handshake-newstyle.c | 20 +-
server/protocol-handshake-oldstyle.c | 3 +-
server/protocol-handshake.c | 97 +++--
server/protocol.c | 201 +++--------
include/nbdkit-filter.h | 4 +
filters/blocksize/blocksize.c | 2 -
filters/cache/cache.c | 1 -
filters/cow/cow.c | 16 +-
filters/nozero/nozero.c | 9 +-
tests/test-layers.c | 50 +--
19 files changed, 963 insertions(+), 758 deletions(-)
create mode 100644 server/backend.c
--
2.21.0
4 years, 8 months
