[nbdkit PATCH] dir-locals: Set c-basic-offset
by Martin Kletzander
Signed-off-by: Martin Kletzander <mkletzan(a)redhat.com>
---
.dir-locals.el | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/.dir-locals.el b/.dir-locals.el
index c9c3affaba28..e420fdbd9f1a 100644
--- a/.dir-locals.el
+++ b/.dir-locals.el
@@ -1 +1,2 @@
-((c-mode . ((indent-tabs-mode . nil))))
+((c-mode . ((indent-tabs-mode . nil)
+ (c-basic-offset . 2))))
--
2.21.0
5 years, 7 months
[nbdkit PATCH] nbd: Give some examples
by Eric Blake
The docs are a lot more useful with a graphic showing how to wire
together nbdkit as a bridge from old-to-new. The converse, bridging
new-to-old, is best deferred until I add support for the nbd plugin
connecting to a TCP socket.
It is also worth mentioning use of nbdkit filters (after all, qemu-nbd
4.0 was able to deprecate its --partition option by pointing to
'nbdkit --filter=partition nbd ...' as a viable replacement).
Signed-off-by: Eric Blake <eblake(a)redhat.com>
---
I'm pushing this one now, but as it is docs, I'm at least posting it
in case you have suggestions for improvements. I've got a counterpart
patch coming up that documents using the nbd plugin to a TCP
connection to provide TLS support from a modern NBD server to an
ancient oldstyle-only client, once I actually add TCP support.
plugins/nbd/nbdkit-nbd-plugin.pod | 38 +++++++++++++++++++++++++++----
1 file changed, 34 insertions(+), 4 deletions(-)
diff --git a/plugins/nbd/nbdkit-nbd-plugin.pod b/plugins/nbd/nbdkit-nbd-plugin.pod
index cd0db09..5add56f 100644
--- a/plugins/nbd/nbdkit-nbd-plugin.pod
+++ b/plugins/nbd/nbdkit-nbd-plugin.pod
@@ -14,8 +14,10 @@ It provides an NBD server that forwards all traffic as a client to
another existing NBD server. A primary usage of this setup is to
alter the set of features available to the ultimate end client,
without having to change the original server (for example, to convert
-between oldstyle and newstyle, or to add TLS support where the original
-server lacks it).
+between oldstyle and newstyle, or to add TLS support where the
+original server lacks it). Use of this plugin along with nbdkit
+filters (adding I<--filter> to the nbdkit command line) makes it
+possible to apply any nbdkit filter to any other NBD server.
For now, this is limited to connecting to another NBD server over a
named Unix socket without TLS, although it is feasible that future
@@ -40,10 +42,38 @@ empty string).
=back
+=head1 EXAMPLES
+
+Expose the contents of an export served by an old style server over a
+Unix socket to TCP network clients that only want to consume encrypted
+data. Use I<--exit-with-parent> to clean up nbdkit at the same time
+that the old server exits.
+
+ ( sock=`mktemp -u` &&
+ nbdkit --exit-with-parent --tls=require nbd socket=$sock &
+ exec /path/to/oldserver --socket=$sock )
+
+ ┌────────────┐ ┌────────┐ ┌────────────┐
+ │ new client │ ────────▶│ nbdkit │ ────────▶│ old server │
+ └────────────┘ TCP └────────┘ Unix └────────────┘
+
+Combine nbdkit's partition filter with qemu-nbd's ability to visit
+qcow2 files (nbdkit does not have a native qcow2 plugin), performing
+the same task as the deprecated C<qemu-nbd -P 1 -f qcow2
+/path/to/image.qcow2> command:
+
+ ( sock=`mktemp -u` &&
+ nbdkit --exit-with-parent --filter=partition nbd socket=$sock partition=1 &
+ exec qemu-nbd -k $sock -f qcow2 /path/to/image.qcow2 )
+
=head1 SEE ALSO
L<nbdkit(1)>,
-L<nbdkit-plugin(3)>.
+L<nbdkit-captive(1)>,
+L<nbdkit-filter(1)>,
+L<nbdkit-tls(1)>,
+L<nbdkit-plugin(3)>,
+L<qemu-nbd(1)>.
=head1 AUTHORS
@@ -51,4 +81,4 @@ Eric Blake
=head1 COPYRIGHT
-Copyright (C) 2017 Red Hat Inc.
+Copyright (C) 2017, 2019 Red Hat Inc.
--
2.20.1
5 years, 7 months
[PATCH] v2v: Allow output modes to rewrite disk copying
by Martin Kletzander
All the current output modes use the default, It's just that I have a patch that
uses this, so there might be someone in the future who wants to use this and if
not, then at least you can tell me if this is wrong or not.
Signed-off-by: Martin Kletzander <mkletzan(a)redhat.com>
---
v2v/types.ml | 15 +++++++++++++++
v2v/types.mli | 8 +++++++-
v2v/v2v.ml | 17 +++--------------
3 files changed, 25 insertions(+), 15 deletions(-)
diff --git a/v2v/types.ml b/v2v/types.ml
index 77f879200a26..2780f05fdfbf 100644
--- a/v2v/types.ml
+++ b/v2v/types.ml
@@ -521,6 +521,21 @@ class virtual output = object
method override_output_format (_ : overlay) = (None : string option)
method virtual prepare_targets : source -> (string * overlay) list -> target_buses -> guestcaps -> inspect -> target_firmware -> target_file list
method disk_create = (open_guestfs ())#disk_create
+ method disk_copy target compressed =
+ let filename =
+ match target.target_file with
+ | TargetFile filename -> qemu_input_filename filename
+ | TargetURI uri -> uri in
+ let cmd =
+ [ "qemu-img"; "convert" ] @
+ (if not (quiet ()) then [ "-p" ] else []) @
+ [ "-n"; "-f"; "qcow2"; "-O"; target.target_format ] @
+ (if compressed then [ "-c" ] else []) @
+ [ target.target_overlay.ov_overlay_file; filename ] in
+ message (f_"Copying disk to %s (%s)") filename target.target_format;
+ if run_command cmd <> 0 then
+ error (f_"qemu-img command failed, see earlier errors");
+
method virtual create_metadata : source -> target list -> target_buses -> guestcaps -> inspect -> target_firmware -> unit
method keep_serial_console = true
method install_rhev_apt = false
diff --git a/v2v/types.mli b/v2v/types.mli
index be9406100785..a3b89bbcc9b3 100644
--- a/v2v/types.mli
+++ b/v2v/types.mli
@@ -438,7 +438,9 @@ end
│ this by defining output#disk_create.
▼
copying Guest data is copied to the target disks
- │ by running ‘qemu-img convert’.
+ (output#disk_copy) by running ‘qemu-img convert’. In rare
+ │ case output modules can affect the
+ │ behaviour of this in output#disk_copy
│
▼
output#create_metadata VM should be created from the metadata
@@ -485,6 +487,10 @@ class virtual output : object
(** Called in order to create disks on the target. The method has the
same signature as Guestfs#disk_create. Normally you should {b not}
define this since the default method calls Guestfs#disk_create. *)
+ method disk_copy : target -> bool -> unit
+ (** Called in order to copy disks on the target. Normally you should
+ {b not} define this unless you handle the copy yourself in a very
+ special way. *)
method virtual create_metadata : source -> target list -> target_buses -> guestcaps -> inspect -> target_firmware -> unit
(** Called after conversion and copying to create metadata and
do any finalization. *)
diff --git a/v2v/v2v.ml b/v2v/v2v.ml
index 277d8f2c7a3e..1bd2225f7334 100644
--- a/v2v/v2v.ml
+++ b/v2v/v2v.ml
@@ -687,10 +687,10 @@ and copy_targets cmdline targets input output =
fun i t ->
(match t.target_file with
| TargetFile s ->
- message (f_"Copying disk %d/%d to %s (%s)")
+ message (f_"Preparing disk %d/%d in %s (%s)")
(i+1) nr_disks s t.target_format;
| TargetURI s ->
- message (f_"Copying disk %d/%d to qemu URI %s (%s)")
+ message (f_"Preparing disk %d/%d on qemu URI %s (%s)")
(i+1) nr_disks s t.target_format
);
debug "%s" (string_of_overlay t.target_overlay);
@@ -744,19 +744,8 @@ and copy_targets cmdline targets input output =
()
);
- let cmd =
- let filename =
- match t.target_file with
- | TargetFile filename -> qemu_input_filename filename
- | TargetURI uri -> uri in
- [ "qemu-img"; "convert" ] @
- (if not (quiet ()) then [ "-p" ] else []) @
- [ "-n"; "-f"; "qcow2"; "-O"; t.target_format ] @
- (if cmdline.compressed then [ "-c" ] else []) @
- [ overlay_file; filename ] in
let start_time = gettimeofday () in
- if run_command cmd <> 0 then
- error (f_"qemu-img command failed, see earlier errors");
+ output#disk_copy t cmdline.compressed;
let end_time = gettimeofday () in
(* Calculate the actual size on the target. *)
--
2.21.0
5 years, 7 months
[nbdkit PATCH 0/4] Fix truncate handling of real_size
by Eric Blake
While working on adding assertions to pthread_mutex_lock calls, I
noticed that the truncate filter's use of mutex didn't really protect
us, and isn't really necessary. Cleaning that up also spotted a couple
of other potential cleanups.
Eric Blake (4):
filters: Drop useless .open callbacks
truncate: Fix corruption when plugin changes per-connection size
truncate: Test for safe multi-connect size handling
doc: More details on (lack of) dynamic sizing
docs/nbdkit-plugin.pod | 9 ++
filters/cache/cache.c | 10 ---
filters/error/error.c | 10 ---
filters/truncate/truncate.c | 162 +++++++++++++++++++-----------------
tests/Makefile.am | 2 +
tests/test-truncate4.sh | 85 +++++++++++++++++++
6 files changed, 183 insertions(+), 95 deletions(-)
create mode 100755 tests/test-truncate4.sh
--
2.20.1
5 years, 7 months
[PATCH nbdkit v5 FINAL 00/19] Implement extents.
by Richard W.M. Jones
This has already been pushed upstream. I am simply posting these here
so we have a reference in the mailing list in case we find bugs later
(as I'm sure we will - it's a complex patch series).
Great thanks to Eric Blake for tireless review on this one. It also
seems to have identified a few minor bugs in qemu along the way.
Rich.
5 years, 7 months
[nbdkit PATCH v2 0/5] structured replies/.extents for nbd plugin
by Eric Blake
Updated based on other changes that have happened in the meantime:
- rely more on cleanup.h (throughout)
- split structured read for easier review (patch 2 and 3 were combined in v1)
- rely on nbdkit not leaking a server's partial answer (patch 3)
- add tests (patch 5)
- other bug fixes I found while testing it
- drop EOVERFLOW patch for now; it will be separate once upstream
NBD protocol specification is clarified
Eric Blake (5):
nbd: Implement NBD_OPT_GO client request
nbd: Refactor receive loop to prepare for structured replies
nbd: Implement structured reads from server
nbd: Implement .extents
nbd: Test .extents
plugins/nbd/nbd.c | 534 +++++++++++++++++++++++++++++++++++---
tests/Makefile.am | 3 +
tests/test-nbd-extents.sh | 113 ++++++++
3 files changed, 608 insertions(+), 42 deletions(-)
create mode 100755 tests/test-nbd-extents.sh
--
2.20.1
5 years, 8 months
[nbdkit PATCH 0/4] More mutex sanity checking
by Eric Blake
I do have a question about whether patch 2 is right, or whether I've
exposed a bigger problem in the truncate (and possibly other) filter,
but the rest seem fairly straightforward.
Eric Blake (4):
server: Check for pthread lock failures
truncate: Factor out reading real_size under mutex
plugins: Check for mutex failures
filters: Check for mutex failures
filters/cache/cache.c | 23 +++++++++----------
filters/cow/cow.c | 19 +++++++---------
filters/error/error.c | 7 +++---
filters/log/log.c | 3 +--
filters/rate/rate.c | 10 ++++-----
filters/readahead/readahead.c | 3 +--
filters/stats/stats.c | 18 +++++----------
filters/truncate/truncate.c | 41 +++++++++++-----------------------
plugins/file/file.c | 3 +--
server/connections.c | 20 ++++++++++-------
server/locks.c | 42 ++++++++++++++++++++++-------------
filters/error/Makefile.am | 5 ++++-
12 files changed, 92 insertions(+), 102 deletions(-)
--
2.20.1
5 years, 8 months
[nbdkit PATCH] noextents: Document use case with tmpfs
by Eric Blake
tmpfs has a known bug of O(n^2) behavior with lseek(SEEK_HOLE); this
is one situation where the noextents filter can come in handy to avoid
the performance penalty of exposing accurate extents.
Signed-off-by: Eric Blake <eblake(a)redhat.com>
---
filters/noextents/nbdkit-noextents-filter.pod | 7 ++++++-
plugins/file/nbdkit-file-plugin.pod | 9 ++++++++-
2 files changed, 14 insertions(+), 2 deletions(-)
diff --git a/filters/noextents/nbdkit-noextents-filter.pod b/filters/noextents/nbdkit-noextents-filter.pod
index aa2e9e2..46f6bdb 100644
--- a/filters/noextents/nbdkit-noextents-filter.pod
+++ b/filters/noextents/nbdkit-noextents-filter.pod
@@ -13,6 +13,10 @@ client to detect sparse regions of the underlying disk.
C<nbdkit-noextents-filter> disables this so that the plugin appears to
be fully allocated.
+This filter can be useful when combined with L<nbdkit-file-plugin(1)>
+serving a file from a file system known to have poor C<lseek(2)>
+performance (C<tmpfs> is known to be one such system).
+
=head1 PARAMETERS
There are no parameters specific to nbdkit-noextents-filter. Any
@@ -23,7 +27,8 @@ plugin in the normal way.
L<nbdkit(1)>,
L<nbdkit-filter(3)>,
-L<nbdkit-nozero-filter(1)>.
+L<nbdkit-nozero-filter(1)>,
+L<nbdkit-file-plugin(1)>.
=head1 AUTHORS
diff --git a/plugins/file/nbdkit-file-plugin.pod b/plugins/file/nbdkit-file-plugin.pod
index d34a638..9241418 100644
--- a/plugins/file/nbdkit-file-plugin.pod
+++ b/plugins/file/nbdkit-file-plugin.pod
@@ -15,6 +15,12 @@ It serves the named C<FILENAME> over NBD. Local block devices
To concatenate multiple files, use L<nbdkit-split-plugin(1)>.
+If you want to expose a file that resides on a file system known to
+have poor C<lseek(2)> performance when searching for holes (C<tmpfs>
+is known to be one such file system), you can use
+L<nbdkit-noextents-filter(1)> to avoid the penalty of probing for
+holes.
+
=head1 PARAMETERS
=over 4
@@ -88,7 +94,8 @@ or block device efficiently or not.
L<nbdkit(1)>,
L<nbdkit-plugin(3)>,
L<nbdkit-split-plugin(1)>,
-L<nbdkit-partitioning-plugin(1)>.
+L<nbdkit-partitioning-plugin(1)>,
+L<nbdkit-noextents-filter(1)>.
=head1 AUTHORS
--
2.20.1
5 years, 8 months
[PATCH 0/3] Few minor changes for bindings
by Pino Toscano
*** BLURB HERE ***
Pino Toscano (3):
python: modernize inspect_vm example
perl: silence usage of add_cdrom in test
python: silence usage of add_cdrom in test
perl/t/060-handle-properties.t | 7 +++++--
python/examples/inspect_vm.py | 26 ++++++++++++--------------
python/t/test050HandleProperties.py | 5 ++++-
3 files changed, 21 insertions(+), 17 deletions(-)
--
2.20.1
5 years, 8 months
