Supporting sparse disks in nbdkit
by Richard W.M. Jones
I've posted a couple of patches towards the ultimate goal of
implementing NBD_CMD_BLOCK_STATUS / base:allocation in nbdkit. Before
I can do the final patch I think we need to discuss how this would be
exposed to plugins since at the end of the day they need to implement
the feature.
Background reading:
- preparatory patches:
https://www.redhat.com/archives/libguestfs/2019-March/msg00013.html
https://www.redhat.com/archives/libguestfs/2019-March/msg00016.html
- NBD protocol, see in particular NBD_CMD_BLOCK_STATUS and
NBD_REPLY_TYPE_BLOCK_STATUS:
https://github.com/NetworkBlockDevice/nbd/blob/master/doc/proto.md
I think we shouldn't modify the pread() callback. If we decide to
implement Structured Replies properly at some point in the future we
might need to do that, but it's not necessary now.
We could introduce a new call ‘extents’ to return the list of extents.
I believe it would look like this:
struct nbdkit_extent {
uint64_t offset;
uint32_t length; // XXX is 32 bit right here?
uint32_t flag; // hole, zero, data ... more in future?
};
int extents (void *handle, uint32_t count, uint64_t offset,
uint32_t flags /* always 0? */,
size_t *nr_extents, struct nbdkit_extent *extents);
The function is meant to scan [offset, offset+count-1] and return a
list of all extents overlapping this range, and their status (hole,
zero, data).
To make writing plugins easier we could say that extents don't need to
be returned in order, and may include extents which don't actually
overlap the requested range. Also missing regions would mean "hole"
(makes writing the VDDK plugin easier), and adjacent extents of the
same type would be coalesced automatically. But it's an error if
returned extents overlap each other.
nbdkit would need to do some massaging on this to get it into the
right format for NBD_CMD_BLOCK_STATUS. (I'm very confused about what
NBD_CMD_FLAG_REQ_ONE is supposed to do.)
We will also need a corresponding ‘can_extents’, which is analogous to
‘can_write’ etc and is what would control the output of
NBD_OPT_{SET,LIST}_META_CONTEXT.
For nbdkit-file-filter:
- Fairly simple implementation using SEEK_HOLE/SEEK_DATA.
- Not sure how we detect zeroes without reading the file.
For nbdkit-memory-plugin:
- Pretty simple implementation, which can even detect non-hole zeroes.
For VDDK:
- VixDiskLib_QueryAllocatedBlocks can return allocated blocks, but
doesn't return holes separately (they are assumed from what is
omitted from the list). No support for detecting zeroes that I can
see.
Some existing filters would have to be modified to correctly adjust
‘extents’ offsets:
- nbdkit-offset-filter
- nbdkit-partition-filter
- nbdkit-truncate-filter (? maybe not)
- nbdkit-xz-filter is complicated: XZ files support sparseness so in
theory we should try to return this data
Your thoughts on this appreciated,
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
5 years, 2 months
[PATCH nbdkit] Minimal implementation of NBD Structured Replies.
by Richard W.M. Jones
This is about the simplest implementation of NBD Structured Replies
(SR) that we can do right now.
It accepts NBD_OPT_STRUCTURED_REPLIES when negotiated by the client,
but only sends back the simplest possible SR when required to by
NBD_CMD_READ. The rest of the time it will send back simple replies
as before. We do not modify the plugin API so plugins are unable to
send complex SRs.
Also we do not handle human-readable error messages yet because that
would require changes in how we handle nbdkit_error().
Also we do not understand NBD_CMD_FLAG_DF, but that seems to be OK
because (a) we don't advertize the feature and (b) we only send back a
single chunk anyway.
---
docs/nbdkit-protocol.pod | 6 +-
server/protocol.h | 59 +++++++++---
plugins/nbd/nbd.c | 4 +-
server/connections.c | 189 ++++++++++++++++++++++++++++++++-------
tests/test-layers.c | 2 +-
5 files changed, 212 insertions(+), 48 deletions(-)
diff --git a/docs/nbdkit-protocol.pod b/docs/nbdkit-protocol.pod
index c44db6a..68438fa 100644
--- a/docs/nbdkit-protocol.pod
+++ b/docs/nbdkit-protocol.pod
@@ -122,7 +122,11 @@ Supported in nbdkit E<ge> 1.9.9.
=item Structured Replies
-I<Not supported>.
+Supported in nbdkit E<ge> 1.11.8.
+
+However we don’t expose the capability to send structured replies to
+plugins yet, nor do we send human-readable error messages using this
+facility.
=item Block Status
diff --git a/server/protocol.h b/server/protocol.h
index 003fc45..0aadd46 100644
--- a/server/protocol.h
+++ b/server/protocol.h
@@ -1,5 +1,5 @@
/* nbdkit
- * Copyright (C) 2013-2018 Red Hat Inc.
+ * Copyright (C) 2013-2019 Red Hat Inc.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@@ -98,12 +98,13 @@ extern const char *name_of_nbd_flag (int);
/* NBD options (new style handshake only). */
extern const char *name_of_nbd_opt (int);
-#define NBD_OPT_EXPORT_NAME 1
-#define NBD_OPT_ABORT 2
-#define NBD_OPT_LIST 3
-#define NBD_OPT_STARTTLS 5
-#define NBD_OPT_INFO 6
-#define NBD_OPT_GO 7
+#define NBD_OPT_EXPORT_NAME 1
+#define NBD_OPT_ABORT 2
+#define NBD_OPT_LIST 3
+#define NBD_OPT_STARTTLS 5
+#define NBD_OPT_INFO 6
+#define NBD_OPT_GO 7
+#define NBD_OPT_STRUCTURED_REPLY 8
extern const char *name_of_nbd_rep (int);
#define NBD_REP_ACK 1
@@ -144,15 +145,49 @@ struct request {
uint32_t count; /* Request length. */
} __attribute__((packed));
-/* Reply (server -> client). */
-struct reply {
- uint32_t magic; /* NBD_REPLY_MAGIC. */
+/* Simple reply (server -> client). */
+struct simple_reply {
+ uint32_t magic; /* NBD_SIMPLE_REPLY_MAGIC. */
uint32_t error; /* NBD_SUCCESS or one of NBD_E*. */
uint64_t handle; /* Opaque handle. */
} __attribute__((packed));
-#define NBD_REQUEST_MAGIC 0x25609513
-#define NBD_REPLY_MAGIC 0x67446698
+/* Structured reply (server -> client). */
+struct structured_reply {
+ uint32_t magic; /* NBD_STRUCTURED_REPLY_MAGIC. */
+ uint16_t flags; /* NBD_REPLY_FLAG_* */
+ uint16_t type; /* NBD_REPLY_TYPE_* */
+ uint64_t handle; /* Opaque handle. */
+ uint32_t length; /* Length of payload which follows. */
+} __attribute__((packed));
+
+struct structured_reply_offset_data {
+ uint64_t offset; /* offset */
+ /* Followed by data. */
+} __attribute__((packed));
+
+struct structured_reply_error {
+ uint32_t error; /* NBD_E* error number */
+ uint16_t len; /* Length of human readable error. */
+ /* Followed by human readable error string. */
+} __attribute__((packed));
+
+#define NBD_REQUEST_MAGIC 0x25609513
+#define NBD_SIMPLE_REPLY_MAGIC 0x67446698
+#define NBD_STRUCTURED_REPLY_MAGIC 0x668e33ef
+
+/* Structured reply flags. */
+extern const char *name_of_nbd_reply_flag (int);
+#define NBD_REPLY_FLAG_DONE (1<<0)
+
+/* Structured reply types. */
+extern const char *name_of_nbd_reply_type (int);
+#define NBD_REPLY_TYPE_NONE 0
+#define NBD_REPLY_TYPE_OFFSET_DATA 1
+#define NBD_REPLY_TYPE_OFFSET_HOLE 2
+#define NBD_REPLY_TYPE_BLOCK_STATUS 3
+#define NBD_REPLY_TYPE_ERROR 32769
+#define NBD_REPLY_TYPE_ERROR_OFFSET 32770
/* NBD commands. */
extern const char *name_of_nbd_cmd (int);
diff --git a/plugins/nbd/nbd.c b/plugins/nbd/nbd.c
index 674f4a4..2f494cd 100644
--- a/plugins/nbd/nbd.c
+++ b/plugins/nbd/nbd.c
@@ -345,7 +345,7 @@ nbd_request (struct handle *h, uint16_t flags, uint16_t type, uint64_t offset,
static int
nbd_reply_raw (struct handle *h, int *fd)
{
- struct reply rep;
+ struct simple_reply rep;
struct transaction *trans;
void *buf;
uint32_t count;
@@ -353,7 +353,7 @@ nbd_reply_raw (struct handle *h, int *fd)
*fd = -1;
if (read_full (h->fd, &rep, sizeof rep) < 0)
return nbd_mark_dead (h);
- if (be32toh (rep.magic) != NBD_REPLY_MAGIC)
+ if (be32toh (rep.magic) != NBD_SIMPLE_REPLY_MAGIC)
return nbd_mark_dead (h);
nbdkit_debug ("received reply for cookie %#" PRIx64 ", status %s",
rep.handle, name_of_nbd_error(be32toh (rep.error)));
diff --git a/server/connections.c b/server/connections.c
index 51d4912..aeb27f8 100644
--- a/server/connections.c
+++ b/server/connections.c
@@ -86,6 +86,7 @@ struct connection {
bool can_fua;
bool can_multi_conn;
bool using_tls;
+ bool structured_replies;
int sockin, sockout;
connection_recv_function recv;
@@ -905,6 +906,26 @@ _negotiate_handshake_newstyle_options (struct connection *conn)
break;
+ case NBD_OPT_STRUCTURED_REPLY:
+ if (optlen != 0) {
+ if (send_newstyle_option_reply (conn, option, NBD_REP_ERR_INVALID)
+ == -1)
+ return -1;
+ if (conn_recv_full (conn, data, optlen,
+ "read: %s: %m", name_of_nbd_opt (option)) == -1)
+ return -1;
+ continue;
+ }
+
+ debug ("newstyle negotiation: %s: client requested structured replies",
+ name_of_nbd_opt (option));
+
+ if (send_newstyle_option_reply (conn, option, NBD_REP_ACK) == -1)
+ return -1;
+
+ conn->structured_replies = true;
+ break;
+
default:
/* Unknown option. */
if (send_newstyle_option_reply (conn, option, NBD_REP_ERR_UNSUP) == -1)
@@ -1224,12 +1245,123 @@ nbd_errno (int error)
}
}
+static int
+send_simple_reply (struct connection *conn,
+ uint64_t handle, uint16_t cmd,
+ const char *buf, uint32_t count,
+ uint32_t error)
+{
+ ACQUIRE_LOCK_FOR_CURRENT_SCOPE (&conn->write_lock);
+ struct simple_reply reply;
+ int r;
+
+ reply.magic = htobe32 (NBD_SIMPLE_REPLY_MAGIC);
+ reply.handle = handle;
+ reply.error = htobe32 (nbd_errno (error));
+
+ r = conn->send (conn, &reply, sizeof reply);
+ if (r == -1) {
+ nbdkit_error ("write reply: %s: %m", name_of_nbd_cmd (cmd));
+ return set_status (conn, -1);
+ }
+
+ /* Send the read data buffer. */
+ if (cmd == NBD_CMD_READ && !error) {
+ r = conn->send (conn, buf, count);
+ if (r == -1) {
+ nbdkit_error ("write data: %s: %m", name_of_nbd_cmd (cmd));
+ return set_status (conn, -1);
+ }
+ }
+
+ return 1; /* command processed ok */
+}
+
+static int
+send_structured_reply_read (struct connection *conn,
+ uint64_t handle, uint16_t cmd,
+ const char *buf, uint32_t count, uint64_t offset)
+{
+ /* Once we are really using structured replies and sending data back
+ * in chunks, we'll be able to grab the write lock for each chunk,
+ * allowing other threads to interleave replies. As we're not doing
+ * that yet we acquire the lock for the whole function.
+ */
+ ACQUIRE_LOCK_FOR_CURRENT_SCOPE (&conn->write_lock);
+ struct structured_reply reply;
+ struct structured_reply_offset_data offset_data;
+ int r;
+
+ assert (cmd == NBD_CMD_READ);
+
+ reply.magic = htobe32 (NBD_STRUCTURED_REPLY_MAGIC);
+ reply.handle = handle;
+ reply.flags = htobe16 (NBD_REPLY_FLAG_DONE);
+ reply.type = htobe16 (NBD_REPLY_TYPE_OFFSET_DATA);
+ reply.length = htobe32 (sizeof offset_data + count);
+
+ r = conn->send (conn, &reply, sizeof reply);
+ if (r == -1) {
+ nbdkit_error ("write reply: %s: %m", name_of_nbd_cmd (cmd));
+ return set_status (conn, -1);
+ }
+
+ /* Send the offset + read data buffer. */
+ offset_data.offset = htobe64 (offset);
+ r = conn->send (conn, &offset_data, sizeof offset_data);
+ if (r == -1) {
+ nbdkit_error ("write data: %s: %m", name_of_nbd_cmd (cmd));
+ return set_status (conn, -1);
+ }
+
+ r = conn->send (conn, buf, count);
+ if (r == -1) {
+ nbdkit_error ("write data: %s: %m", name_of_nbd_cmd (cmd));
+ return set_status (conn, -1);
+ }
+
+ return 1; /* command processed ok */
+}
+
+static int
+send_structured_reply_error (struct connection *conn,
+ uint64_t handle, uint16_t cmd, uint32_t error)
+{
+ ACQUIRE_LOCK_FOR_CURRENT_SCOPE (&conn->write_lock);
+ struct structured_reply reply;
+ struct structured_reply_error error_data;
+ int r;
+
+ reply.magic = htobe32 (NBD_STRUCTURED_REPLY_MAGIC);
+ reply.handle = handle;
+ reply.flags = htobe16 (NBD_REPLY_FLAG_DONE);
+ reply.type = htobe16 (NBD_REPLY_TYPE_ERROR);
+ reply.length = htobe32 (sizeof error_data + 0 /* no human readable error */);
+
+ r = conn->send (conn, &reply, sizeof reply);
+ if (r == -1) {
+ nbdkit_error ("write error reply: %m");
+ return set_status (conn, -1);
+ }
+
+ /* Send the error. */
+ error_data.error = htobe32 (error);
+ error_data.len = htobe16 (0);
+ r = conn->send (conn, &error_data, sizeof error_data);
+ if (r == -1) {
+ nbdkit_error ("write data: %s: %m", name_of_nbd_cmd (cmd));
+ return set_status (conn, -1);
+ }
+ /* No human readable error message at the moment. */
+
+ return 1; /* command processed ok */
+}
+
static int
recv_request_send_reply (struct connection *conn)
{
int r;
struct request request;
- struct reply reply;
uint16_t cmd, flags;
uint32_t magic, count, error = 0;
uint64_t offset;
@@ -1317,40 +1449,33 @@ recv_request_send_reply (struct connection *conn)
/* Send the reply packet. */
send_reply:
- {
- ACQUIRE_LOCK_FOR_CURRENT_SCOPE (&conn->write_lock);
- if (get_status (conn) < 0)
- return -1;
- reply.magic = htobe32 (NBD_REPLY_MAGIC);
- reply.handle = request.handle;
- reply.error = htobe32 (nbd_errno (error));
+ if (get_status (conn) < 0)
+ return -1;
- if (error != 0) {
- /* Since we're about to send only the limited NBD_E* errno to the
- * client, don't lose the information about what really happened
- * on the server side. Make sure there is a way for the operator
- * to retrieve the real error.
- */
- debug ("sending error reply: %s", strerror (error));
- }
-
- r = conn->send (conn, &reply, sizeof reply);
- if (r == -1) {
- nbdkit_error ("write reply: %s: %m", name_of_nbd_cmd (cmd));
- return set_status (conn, -1);
- }
-
- /* Send the read data buffer. */
- if (cmd == NBD_CMD_READ && !error) {
- r = conn->send (conn, buf, count);
- if (r == -1) {
- nbdkit_error ("write data: %s: %m", name_of_nbd_cmd (cmd));
- return set_status (conn, -1);
- }
- }
+ if (error != 0) {
+ /* Since we're about to send only the limited NBD_E* errno to the
+ * client, don't lose the information about what really happened
+ * on the server side. Make sure there is a way for the operator
+ * to retrieve the real error.
+ */
+ debug ("sending error reply: %s", strerror (error));
}
- return 1; /* command processed ok */
+ /* Currently we prefer to send simple replies for everything except
+ * where we have to (ie. NBD_CMD_READ when structured_replies have
+ * been negotiated). However this prevents us from sending
+ * human-readable error messages to the client, so we should
+ * reconsider this in future.
+ */
+ if (conn->structured_replies && cmd == NBD_CMD_READ) {
+ if (!error)
+ return send_structured_reply_read (conn, request.handle, cmd,
+ buf, count, offset);
+ else
+ return send_structured_reply_error (conn, request.handle, cmd, error);
+ }
+ else
+ return send_simple_reply (conn, request.handle, cmd, buf, count, error);
}
/* Write buffer to conn->sockout and either succeed completely
diff --git a/tests/test-layers.c b/tests/test-layers.c
index c8d15d2..6149c02 100644
--- a/tests/test-layers.c
+++ b/tests/test-layers.c
@@ -92,7 +92,7 @@ main (int argc, char *argv[])
struct new_handshake_finish handshake_finish;
uint16_t eflags;
struct request request;
- struct reply reply;
+ struct simple_reply reply;
char data[512];
#ifndef HAVE_EXIT_WITH_PARENT
--
2.20.1
5 years, 2 months
[PATCH nbdkit] server: Implement minimal implementation of set/list metadata contexts.
by Richard W.M. Jones
None are supported at present, so this always returns an empty list.
---
docs/nbdkit-protocol.pod | 4 ++
server/protocol.h | 2 +
server/connections.c | 84 ++++++++++++++++++++++++++++++++++++++++
3 files changed, 90 insertions(+)
diff --git a/docs/nbdkit-protocol.pod b/docs/nbdkit-protocol.pod
index 68438fa..4c65e00 100644
--- a/docs/nbdkit-protocol.pod
+++ b/docs/nbdkit-protocol.pod
@@ -128,6 +128,10 @@ However we don’t expose the capability to send structured replies to
plugins yet, nor do we send human-readable error messages using this
facility.
+=item Metadata Querying
+
+Supported in nbdkit E<ge> 1.11.8.
+
=item Block Status
I<Not supported>.
diff --git a/server/protocol.h b/server/protocol.h
index 0aadd46..b03555e 100644
--- a/server/protocol.h
+++ b/server/protocol.h
@@ -105,6 +105,8 @@ extern const char *name_of_nbd_opt (int);
#define NBD_OPT_INFO 6
#define NBD_OPT_GO 7
#define NBD_OPT_STRUCTURED_REPLY 8
+#define NBD_OPT_LIST_META_CONTEXT 9
+#define NBD_OPT_SET_META_CONTEXT 10
extern const char *name_of_nbd_rep (int);
#define NBD_REP_ACK 1
diff --git a/server/connections.c b/server/connections.c
index aeb27f8..7e32f00 100644
--- a/server/connections.c
+++ b/server/connections.c
@@ -926,6 +926,90 @@ _negotiate_handshake_newstyle_options (struct connection *conn)
conn->structured_replies = true;
break;
+ case NBD_OPT_LIST_META_CONTEXT:
+ case NBD_OPT_SET_META_CONTEXT:
+ {
+ uint32_t opt_index;
+ uint32_t exportnamelen;
+ uint32_t nr_queries;
+ uint32_t querylen;
+ const char *what;
+
+ optname = name_of_nbd_opt (option);
+ if (conn_recv_full (conn, data, optlen, "read: %s: %m", optname) == -1)
+ return -1;
+
+ if (!conn->structured_replies) {
+ if (send_newstyle_option_reply (conn, option, NBD_REP_ERR_INVALID)
+ == -1)
+ return -1;
+ continue;
+ }
+
+ /* Minimum length of the option payload is the 32 bit export
+ * name plus a zero length export name plus 32 bit number of
+ * queries followed by no queries.
+ */
+ what = "optlen < 8";
+ if (optlen < 8) {
+ opt_meta_invalid_option_len:
+ debug ("newstyle negotiation: %s: invalid option length: %s",
+ optname, what);
+
+ if (send_newstyle_option_reply (conn, option, NBD_REP_ERR_INVALID)
+ == -1)
+ return -1;
+ continue;
+ }
+
+ /* Discard the export name. */
+ memcpy (&exportnamelen, &data[0], 4);
+ exportnamelen = be32toh (exportnamelen);
+ opt_index = 4 + exportnamelen;
+
+ /* Read the number of queries. */
+ what = "reading number of queries";
+ if (opt_index+4 > optlen)
+ goto opt_meta_invalid_option_len;
+ memcpy (&nr_queries, &data[opt_index], 4);
+ nr_queries = be32toh (nr_queries);
+ opt_index += 4;
+
+ /* nr_queries == 0 means return all meta contexts. */
+ if (nr_queries == 0) {
+ /* Nothing is supported now. */
+ if (send_newstyle_option_reply (conn, option, NBD_REP_ACK) == -1)
+ return -1;
+ }
+ else {
+ /* Read and answer each query. */
+ while (nr_queries > 0) {
+ what = "reading query string length";
+ if (opt_index+4 > optlen)
+ goto opt_meta_invalid_option_len;
+ memcpy (&querylen, &data[opt_index], 4);
+ querylen = be32toh (querylen);
+ opt_index += 4;
+ what = "reading query string";
+ if (opt_index + querylen > optlen)
+ goto opt_meta_invalid_option_len;
+
+ debug ("newstyle negotiation: %s: %s %.*s",
+ optname,
+ option == NBD_OPT_LIST_META_CONTEXT ? "query" : "set",
+ (int) querylen, &data[opt_index]);
+
+ /* Ignore query - nothing is supported. */
+
+ opt_index += querylen;
+ nr_queries--;
+ }
+ if (send_newstyle_option_reply (conn, option, NBD_REP_ACK) == -1)
+ return -1;
+ }
+ }
+ break;
+
default:
/* Unknown option. */
if (send_newstyle_option_reply (conn, option, NBD_REP_ERR_UNSUP) == -1)
--
2.20.1
5 years, 2 months
[PATCH nbdkit] Add new filter for rate-limiting connections.
by Richard W.M. Jones
For virt-v2v we have been discussing how to limit network bandwidth.
The initial discussion has been around how to use cgroups to do this
limiting, and that is still probably what we will go with in the end.
However this patch gives us another possibility for certain virt-v2v
inputs, especially VDDK. We could apply a filter on top of the nbdkit
plugin which limits the rate at which it copies data. For example, to
limit the rate to 1 Mbps (megabit per second) we could now do:
nbdkit --filter=rate vddk [etc] rate=1M
The filter is implemented using a simple Token Bucket
(https://en.wikipedia.org/wiki/Token_bucket) and is quite simple while
at the same time using the fully parallel thread model.
Rich.
5 years, 2 months
Support for NBD block status in nbdkit & qemu NBD client
by Richard W.M. Jones
Hi Eric:
Here's the problem: We have a VMware VDDK file or snapshot. The
virtual size of the snapshot is generally much larger than what's
allocated. Copying the snapshot using:
nbdkit vddk file=foo.vmdk --run 'qemu-img convert $nbd out'
is therefore much slower than it could be. (In the real world version
of this we're actually copying the source over the network, but the
command line for that is too complicated to show in a simple example).
VDDK has an API for querying allocated blocks, see
‘VixDiskLib_QueryAllocatedBlocks’ in
https://vdc-download.vmware.com/vmwb-repository/dcr-public/8f96698a-0e7b-...
It's awkward to use but Martin (CC'd) has played with it and it does
seem to work.
The question is if we add Structured Replies + Block Status support to
nbdkit & implement it for the VDDK plugin, will ‘qemu-img convert’
above be able to use that to optimize the copy? My brief reading of
the qemu sources suggests that something has been implemented here so
it likely ought to work?
I guess the other question is if there are any limitations around
alignment that we should be aware of. It seems as if VDDK uses a
larger cluster size for sparseness than its sector size (as does
qcow2).
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
5 years, 2 months
[PATCH nbdkit] Add ssh plugin using libssh.
by Richard W.M. Jones
This adds a simple plugin using libssh (not libssh2). The intended
use for this is with virt-v2v when sourcing guests from VMware over
SSH.
We've had several years of problems getting our libssh-based driver
into qemu. By putting it into nbdkit instead we can bypass that.
However this also lets us combine ssh access with filters, in
particular the recently written ‘rate’ filter.
Rich.
5 years, 2 months
[supermin PATCH] rebuild the output it when SUPERMIN_KERNEL or SUPERMIN_MODULES are defined
by Masayoshi Mizuma
SUPERMIN_KERNEL and SUPERMIN_MODULES don't work to guestfish.
Since guestfish sets --if-newer parameter to supermin, so the environment
variables are not used under the following conditions.
- the output directory exists and,
- the dates of both input files and package database are
older than the output
To solve that, rebuild the output it when SUPERMIN_KERNEL or
SUPERMIN_MODULES are defined even if --if-newer is set.
Signed-off-by: Masayoshi Mizuma <msys.mizuma(a)gmail.com>
---
src/supermin.ml | 18 +++++++++++-------
1 file changed, 11 insertions(+), 7 deletions(-)
diff --git a/src/supermin.ml b/src/supermin.ml
index 7c7135b3..b997643 100644
--- a/src/supermin.ml
+++ b/src/supermin.ml
@@ -236,13 +236,17 @@ appliance automatically.
*)
if if_newer then (
try
- let odate = (lstat outputdir).st_mtime in
- let idates = List.map (fun d -> (lstat d).st_mtime) inputs in
- let pdate = (get_package_handler ()).ph_get_package_database_mtime () in
- if List.for_all (fun idate -> idate < odate) (pdate :: idates) then (
- if debug >= 1 then
- printf "supermin: if-newer: output does not need rebuilding\n%!";
- exit 0
+ let kernel = try getenv "SUPERMIN_KERNEL" with Not_found -> "" in
+ let modules = try getenv "SUPERMIN_MODULES" with Not_found -> "" in
+ if kernel = "" && modules = "" then (
+ let odate = (lstat outputdir).st_mtime in
+ let idates = List.map (fun d -> (lstat d).st_mtime) inputs in
+ let pdate = (get_package_handler ()).ph_get_package_database_mtime () in
+ if List.for_all (fun idate -> idate < odate) (pdate :: idates) then (
+ if debug >= 1 then
+ printf "supermin: if-newer: output does not need rebuilding\n%!";
+ exit 0
+ )
)
with
Unix_error _ -> () (* just continue *)
--
2.20.1
5 years, 2 months