[PATCH EXPERIMENTAL nbdkit 0/2] Port to Windows using mingw.
by Richard W.M. Jones
The patches following do indeed allow you to compile nbdkit.exe, but
it does not actually work yet. I'm posting this experimental series
more as a work in progress and to get feedback.
Note this does not require Windows itself to build or test. You can
cross-compile it using mingw64-* packages on Fedora or Debian, and
test it [spoiler alert: it fails] using Wine.
Rich.
3 years, 9 months
[v2v PATCH] libvirt: read password file outside libvirt auth callback
by Pino Toscano
This way errors that occur while reading the password file are properly
propagated, instead of being reported as errors of the libvirt
authentication callback.
Reported by: Ming Xie.
---
v2v/libvirt_utils.ml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/v2v/libvirt_utils.ml b/v2v/libvirt_utils.ml
index 4d0b8639..1a24b049 100644
--- a/v2v/libvirt_utils.ml
+++ b/v2v/libvirt_utils.ml
@@ -24,8 +24,8 @@ open Common_gettext.Gettext
module. *)
let auth_for_password_file ?password_file () =
+ let password = Option.map read_first_line_from_file password_file in
let auth_fn creds =
- let password = Option.map read_first_line_from_file password_file in
List.map (
function
| { Libvirt.Connect.typ = Libvirt.Connect.CredentialPassphrase } -> password
--
2.26.2
3 years, 9 months
[PATCH nbdkit] New ondemand plugin.
by Richard W.M. Jones
This creates filesystems on demand. A client simply connects with a
desired export name and a new export is created. The export is
persistent (until deleted by the server admin), and clients may
disconnect and reconnect. In some respects this is similar to the
nbdkit-tmpdisk-plugin, or nbdkit-file-plugin with the dir= option.
---
plugins/ondemand/nbdkit-ondemand-plugin.pod | 190 ++++++
plugins/tmpdisk/nbdkit-tmpdisk-plugin.pod | 3 +-
configure.ac | 2 +
plugins/ondemand/Makefile.am | 81 +++
tests/Makefile.am | 10 +
plugins/ondemand/ondemand.c | 634 ++++++++++++++++++++
plugins/ondemand/default-command.sh.in | 57 ++
tests/test-ondemand-list.sh | 66 ++
tests/test-ondemand.sh | 74 +++
.gitignore | 1 +
TODO | 7 +
11 files changed, 1124 insertions(+), 1 deletion(-)
diff --git a/plugins/ondemand/nbdkit-ondemand-plugin.pod b/plugins/ondemand/nbdkit-ondemand-plugin.pod
new file mode 100644
index 00000000..be46b1c2
--- /dev/null
+++ b/plugins/ondemand/nbdkit-ondemand-plugin.pod
@@ -0,0 +1,190 @@
+=head1 NAME
+
+nbdkit-ondemand-plugin - create filesystems on demand
+
+=head1 SYNOPSIS
+
+ nbdkit ondemand dir=EXPORTSDIR [size=]SIZE
+ { [type=ext4|xfs|vfat|...] [label=LABEL]
+ | command=COMMAND [VAR=VALUE ...] }
+
+=head1 DESCRIPTION
+
+This is a plugin for L<nbdkit(1)> which creates persistent filesystems
+on demand. Clients may simply connect to the server, requesting a
+particular export name, and a new filesystem is created if it does not
+exist already. Clients can also disconnect and reconnect with the
+same export name and the same filesystem will still be available.
+Filesystems are stored in a directory on the server, so they also
+persist over nbdkit and server restarts.
+
+Each filesystem is locked while it is in use by a client, preventing
+two clients from accessing the same filesystem (which would cause
+corruption).
+
+Similar plugins include L<nbdkit-file-plugin(1)> which can serve a
+predefined set of exports (clients cannot create more),
+L<nbdkit-tmpdisk-plugin(1)> which creates a fresh temporary filesystem
+for each client, and L<nbdkit-linuxdisk-plugin(1)> which exports a
+single filesystem from a local directory on the server.
+
+When a new export name is requested by a client, a sparse file of the
+same name is created in C<dir=EXPORTSDIR> on the server. The file
+will be formatted with L<mkfs(8)>. The size of the file is currently
+fixed by the C<size=SIZE> parameter, but we intend to make this
+client-configurable in future. The filesystem type and label may also
+be specified, otherwise C<ext4> and no label is used.
+
+Export names must be E<le> C<NAME_MAX> (usually 255) bytes in length
+and must not contain certain characters including C<.>, C</> and C<:>.
+There may be other limitations added in future. Client requests which
+do not obey these restrictions are rejected. As a special case,
+export name C<""> is mapped to the file name F<default>.
+
+=head2 Security considerations
+
+You should B<only> use this in an environment where you trust all your
+clients, since clients can use this plugin to consume arbitrary
+amounts of disk space by creating unlimited exports. It is therefore
+best to take steps to limit where clients can connect from using
+L<nbdkit-ip-filter(1)>, firewalls, or TLS client certificates.
+
+=head2 The command parameter
+
+Instead of running mkfs you can run an arbitrary command (a shell
+script fragment) to create the disk.
+
+The other parameters to the plugin are turned into shell variables
+passed to the command. For example C<type> becomes the shell variable
+C<$type>, etc. Any parameters you want can be passed to the plugin
+and will be turned into shell variables (not only C<type> and
+C<label>) making this a very flexible method to create filesystems and
+disks of all kinds.
+
+Two special variables are also passed to the shell script fragment:
+
+=over 4
+
+=item C<$disk>
+
+The absolute path of the disk file. This is partially controlled by
+the client so you should quote it carefully. This file is not
+pre-created, the command must create it for example using:
+
+ truncate -s $size "$disk"
+
+=item C<$size>
+
+The virtual size in bytes. This is the C<size> parameter, converted
+to bytes. Note the final size served to the client is whatever disk
+size C<command> creates.
+
+=back
+
+=head1 EXAMPLE
+
+Run the server like this:
+
+ mkdir /var/tmp/exports
+ nbdkit ondemand dir=/var/tmp/exports 1G
+
+Clients can connect and create 1G ext4 filesystems on demand using
+commands such as these (note the different export names):
+
+ nbd-client -b 512 server /dev/nbd0 -N export1
+ mount /dev/nbd0 /mnt
+
+ guestfish --format=raw -a nbd://localhost/export2 -m /dev/sda
+
+ qemu-img info nbd:localhost:10809:exportname=export2
+
+On the server you would see two filesystems created:
+
+ $ ls -l /var/tmp/exports
+ -rw-rw-r--. 1 rjones rjones 1073741824 Aug 13 21:40 export1
+ -rw-rw-r--. 1 rjones rjones 1073741824 Aug 13 21:40 export2
+
+The plugin does not clean these up. If they are no longer needed then
+the server admin should delete them (or use a tmp cleaner).
+
+=head1 PARAMETERS
+
+=over 4
+
+=item B<command='>COMMANDB<'>
+
+Instead of running L<mkfs(8)> to create the initial filesystem, run
+C<COMMAND> (a shell script fragment which usually must be quoted to
+protect it from the shell). See L</The command parameter> and
+L</EXAMPLES> sections above.
+
+=item B<dir=>EXPORTSDIR
+
+The directory where filesystems are saved. When first using this
+plugin you should point this to an empty directory. When clients
+connect, filesystems are created here.
+
+This parameter is required.
+
+=item B<label=>LABEL
+
+Select the filesystem label. The default is not set.
+
+=item [B<size=>]SIZE
+
+Specify the virtual size of all of the filesystems.
+
+If using C<command>, this is only a suggested size. The actual size
+of the resulting disk will be the size of the disk created by
+C<command>.
+
+This parameter is required.
+
+C<size=> is a magic config key and may be omitted in most cases.
+See L<nbdkit(1)/Magic parameters>.
+
+=item B<type=>FS
+
+Select the filesystem type. The default is C<ext4>. Most
+non-networked, non-cluster filesystem types supported by the
+L<mkfs(8)> command can be used here.
+
+=back
+
+=head1 FILES
+
+=over 4
+
+=item F<$plugindir/nbdkit-ondemand-plugin.so>
+
+The plugin.
+
+Use C<nbdkit --dump-config> to find the location of C<$plugindir>.
+
+=back
+
+=head1 VERSION
+
+C<nbdkit-ondemand-plugin> first appeared in nbdkit 1.22.
+
+=head1 SEE ALSO
+
+L<nbdkit(1)>,
+L<nbdkit-plugin(3)>,
+L<nbdkit-file-plugin(1)>,
+L<nbdkit-ip-filter(1)>,
+L<nbdkit-limit-filter(1)>,
+L<nbdkit-linuxdisk-plugin(1)>,
+L<nbdkit-memory-plugin(1)>,
+L<nbdkit-tmpdisk-plugin(1)>,
+L<nbdkit-tls(1)>,
+L<mkfs(8)>,
+L<mke2fs(8)>.
+
+=head1 AUTHORS
+
+Richard W.M. Jones
+
+=head1 COPYRIGHT
+
+Copyright (C) 2018-2020 Red Hat Inc.
diff --git a/plugins/tmpdisk/nbdkit-tmpdisk-plugin.pod b/plugins/tmpdisk/nbdkit-tmpdisk-plugin.pod
index be56d68a..9cecad34 100644
--- a/plugins/tmpdisk/nbdkit-tmpdisk-plugin.pod
+++ b/plugins/tmpdisk/nbdkit-tmpdisk-plugin.pod
@@ -13,7 +13,8 @@ nbdkit-tmpdisk-plugin - create a fresh temporary filesystem for each client
This L<nbdkit(1)> plugin is used for creating temporary filesystems
for thin clients. Each time a client connects it will see a fresh,
empty filesystem for its exclusive use. B<The filesystem is deleted>
-when the client disconnects.
+when the client disconnects. If you want a persistent filesystem, use
+L<nbdkit-ondemand-plugin(1)> instead.
When a new client connects, a blank, sparse file of the required size
is created in C<$TMPDIR> (or F</var/tmp>). L<mkfs(8)> is then run on
diff --git a/configure.ac b/configure.ac
index fee6fe9a..5d74251a 100644
--- a/configure.ac
+++ b/configure.ac
@@ -81,6 +81,7 @@ non_lang_plugins="\
memory \
nbd \
null \
+ ondemand \
partitioning \
pattern \
random \
@@ -1119,6 +1120,7 @@ AC_CONFIG_FILES([Makefile
plugins/nbd/Makefile
plugins/null/Makefile
plugins/ocaml/Makefile
+ plugins/ondemand/Makefile
plugins/partitioning/Makefile
plugins/pattern/Makefile
plugins/perl/Makefile
diff --git a/plugins/ondemand/Makefile.am b/plugins/ondemand/Makefile.am
new file mode 100644
index 00000000..a1bc00d6
--- /dev/null
+++ b/plugins/ondemand/Makefile.am
@@ -0,0 +1,81 @@
+# nbdkit
+# Copyright (C) 2017-2020 Red Hat Inc.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are
+# met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+#
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+#
+# * Neither the name of Red Hat nor the names of its contributors may be
+# used to endorse or promote products derived from this software without
+# specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND
+# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
+# USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+# SUCH DAMAGE.
+
+include $(top_srcdir)/common-rules.mk
+
+EXTRA_DIST = \
+ default-command.sh.in \
+ nbdkit-ondemand-plugin.pod \
+ $(NULL)
+
+# The default command we use (if we don't use command=) comes from a
+# shell script which is turned into a C source file.
+BUILT_SOURCES = default-command.c
+
+default-command.c: default-command.sh.in Makefile
+ rm -f $@ $@-t
+ echo 'const char *command =' > $@-t
+ $(SED) -e '/^#/d' -e 's/"/\\"/g' -e 's/\(.*\)/"\1\\n"/' < $< >> $@-t
+ echo ';' >> $@-t
+ mv $@-t $@
+
+plugin_LTLIBRARIES = nbdkit-ondemand-plugin.la
+
+nbdkit_ondemand_plugin_la_SOURCES = \
+ default-command.c \
+ ondemand.c \
+ $(top_srcdir)/include/nbdkit-plugin.h \
+ $(NULL)
+
+nbdkit_ondemand_plugin_la_CPPFLAGS = \
+ -I$(top_srcdir)/include \
+ -I$(top_srcdir)/common/utils \
+ $(NULL)
+nbdkit_ondemand_plugin_la_CFLAGS = $(WARNINGS_CFLAGS)
+nbdkit_ondemand_plugin_la_LDFLAGS = \
+ -module -avoid-version -shared $(SHARED_LDFLAGS) \
+ -Wl,--version-script=$(top_srcdir)/plugins/plugins.syms \
+ $(NULL)
+nbdkit_ondemand_plugin_la_LIBADD = \
+ $(top_builddir)/common/utils/libutils.la \
+ $(NULL)
+
+if HAVE_POD
+
+man_MANS = nbdkit-ondemand-plugin.1
+CLEANFILES += $(man_MANS)
+
+nbdkit-ondemand-plugin.1: nbdkit-ondemand-plugin.pod
+ $(PODWRAPPER) --section=1 --man $@ \
+ --html $(top_builddir)/html/$@.html \
+ $<
+
+endif HAVE_POD
diff --git a/tests/Makefile.am b/tests/Makefile.am
index ca1e76d1..96c8b900 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -757,6 +757,16 @@ test_null_SOURCES = test-null.c
test_null_CFLAGS = $(WARNINGS_CFLAGS) $(LIBNBD_CFLAGS)
test_null_LDADD = $(LIBNBD_LIBS)
+# ondemand plugin test.
+TESTS += \
+ test-ondemand.sh \
+ test-ondemand-list.sh \
+ $(NULL)
+EXTRA_DIST += \
+ test-ondemand.sh \
+ test-ondemand-list.sh \
+ $(NULL)
+
# partitioning plugin test.
TESTS += \
test-partitioning1.sh \
diff --git a/plugins/ondemand/ondemand.c b/plugins/ondemand/ondemand.c
new file mode 100644
index 00000000..f85d630d
--- /dev/null
+++ b/plugins/ondemand/ondemand.c
@@ -0,0 +1,634 @@
+/* nbdkit
+ * Copyright (C) 2017-2020 Red Hat Inc.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are
+ * met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ *
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * * Neither the name of Red Hat nor the names of its contributors may be
+ * used to endorse or promote products derived from this software without
+ * specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+ * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
+ * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+ * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+ * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#include <config.h>
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdbool.h>
+#include <stdint.h>
+#include <inttypes.h>
+#include <string.h>
+#include <unistd.h>
+#include <fcntl.h>
+#include <dirent.h>
+#include <errno.h>
+#include <assert.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <sys/wait.h>
+
+#include <pthread.h>
+
+#define NBDKIT_API_VERSION 2
+#include <nbdkit-plugin.h>
+
+#include "cleanup.h"
+#include "utils.h"
+
+static char *dir; /* dir parameter */
+static DIR *exportsdir; /* opened exports dir */
+static int64_t requested_size = -1; /* size parameter on the command line */
+
+/* Shell variables. */
+static struct var {
+ struct var *next;
+ const char *key, *value;
+} *vars, *last_var;
+
+/* This comes from default-command.c which is generated from
+ * default-command.sh.in.
+ */
+extern const char *command;
+
+static void
+ondemand_unload (void)
+{
+ struct var *v, *v_next;
+
+ for (v = vars; v != NULL; v = v_next) {
+ v_next = v->next;
+ free (v);
+ }
+
+ closedir (exportsdir);
+ free (dir);
+}
+
+static int
+ondemand_config (const char *key, const char *value)
+{
+ if (strcmp (key, "command") == 0) {
+ command = value;
+ }
+ else if (strcmp (key, "size") == 0) {
+ requested_size = nbdkit_parse_size (value);
+ if (requested_size == -1)
+ return -1;
+ }
+ else if (strcmp (key, "dir") == 0) {
+ dir = nbdkit_realpath (value);
+ if (dir == NULL)
+ return -1;
+ }
+
+ /* This parameter cannot be set on the command line since it is used
+ * to pass the disk name to the command.
+ */
+ else if (strcmp (key, "disk") == 0) {
+ nbdkit_error ("'disk' parameter cannot be set on the command line");
+ return -1;
+ }
+
+ /* Any other parameter will be forwarded to a shell variable. */
+ else {
+ struct var *new_var;
+
+ new_var = malloc (sizeof *new_var);
+ if (new_var == NULL) {
+ perror ("malloc");
+ exit (EXIT_FAILURE);
+ }
+
+ new_var->next = NULL;
+ new_var->key = key;
+ new_var->value = value;
+
+ /* Append it to the linked list. */
+ if (vars == NULL) {
+ assert (last_var == NULL);
+ vars = last_var = new_var;
+ }
+ else {
+ assert (last_var != NULL);
+ last_var->next = new_var;
+ last_var = new_var;
+ }
+ }
+
+ return 0;
+}
+
+static int
+ondemand_config_complete (void)
+{
+ if (dir == NULL || requested_size == -1) {
+ nbdkit_error ("dir and size parameters are required");
+ return -1;
+ }
+
+ return 0;
+}
+
+static int
+ondemand_get_ready (void)
+{
+ exportsdir = opendir (dir);
+ if (exportsdir == NULL) {
+ nbdkit_error ("opendir: %s: %m", dir);
+ return -1;
+ }
+
+ return 0;
+}
+
+#define ondemand_config_help \
+ "dir=<EXPORTSDIR> (required) Directory containing filesystems.\n" \
+ "size=<SIZE> (required) Virtual filesystem size.\n" \
+ "label=<LABEL> The filesystem label.\n" \
+ "type=ext4|... The filesystem type.\n" \
+ "command=<COMMAND> Alternate command instead of mkfs."
+
+/* Because we rewind the exportsdir handle, we need a lock to protect
+ * list_exports from being called in parallel.
+ */
+static pthread_mutex_t exports_lock = PTHREAD_MUTEX_INITIALIZER;
+
+static int
+ondemand_list_exports (int readonly, int default_only,
+ struct nbdkit_exports *exports)
+{
+ ACQUIRE_LOCK_FOR_CURRENT_SCOPE (&exports_lock);
+ struct dirent *d;
+
+ /* First entry should be the default export. XXX Should we check if
+ * the "default" file was created? I don't think we need to.
+ */
+ if (nbdkit_add_export (exports, "", NULL) == -1)
+ return -1;
+ if (default_only) return 0;
+
+ /* Read the rest of the exports. */
+ rewinddir (exportsdir);
+
+ /* XXX Output is not sorted. Does it matter? */
+ while (errno = 0, (d = readdir (exportsdir)) != NULL) {
+ /* Skip all dot files. "." anywhere in the export name is
+ * rejected by the plugin, so commands can use dot files to "hide"
+ * files in the export dir (eg. if needing to keep state).
+ */
+ if (d->d_name[0] == '.')
+ continue;
+
+ /* Skip the "default" filename which refers to the "" export. */
+ if (strcmp (d->d_name, "default") == 0)
+ continue;
+
+ if (nbdkit_add_export (exports, d->d_name, NULL) == -1)
+ return -1;
+ }
+
+ /* Did readdir fail? */
+ if (errno != 0) {
+ nbdkit_error ("readdir: %s: %m", dir);
+ return -1;
+ }
+
+ return 0;
+}
+
+struct handle {
+ int fd;
+ int64_t size;
+ const char *exportname;
+ bool can_punch_hole;
+};
+
+/* Since clients that want multi-conn should all pass the same export
+ * name, this is safe.
+ */
+static int
+ondemand_can_multi_conn (void *handle)
+{
+ return 1;
+}
+
+static int
+ondemand_can_trim (void *handle)
+{
+#ifdef FALLOC_FL_PUNCH_HOLE
+ return 1;
+#else
+ return 0;
+#endif
+}
+
+static int
+ondemand_can_fua (void *handle)
+{
+ return NBDKIT_FUA_NATIVE;
+}
+
+/* This creates and runs the full "mkfs" (or whatever) command. */
+static int
+run_command (const char *disk)
+{
+ FILE *fp;
+ CLEANUP_FREE char *cmd = NULL;
+ size_t len = 0;
+ int r;
+ struct var *v;
+
+ fp = open_memstream (&cmd, &len);
+ if (fp == NULL) {
+ nbdkit_error ("open_memstream: %m");
+ return -1;
+ }
+
+ /* Avoid stdin/stdout leaking (because of nbdkit -s). */
+ fprintf (fp, "exec </dev/null >/dev/null\n");
+
+ /* Set the standard shell variables. */
+ fprintf (fp, "disk=");
+ shell_quote (disk, fp);
+ putc ('\n', fp);
+ fprintf (fp, "size=%" PRIi64 "\n", requested_size);
+ putc ('\n', fp);
+
+ /* The other parameters/shell variables. */
+ for (v = vars; v != NULL; v = v->next) {
+ /* Keys probably can never contain shell-unsafe chars (because of
+ * nbdkit's own restrictions), but quoting it makes it safe.
+ */
+ shell_quote (v->key, fp);
+ putc ('=', fp);
+ shell_quote (v->value, fp);
+ putc ('\n', fp);
+ }
+ putc ('\n', fp);
+
+ /* The command. */
+ fprintf (fp, "%s", command);
+
+ if (fclose (fp) == EOF) {
+ nbdkit_error ("memstream failed");
+ return -1;
+ }
+
+ r = system (cmd);
+ if (r == -1) {
+ nbdkit_error ("failed to execute command: %m");
+ return -1;
+ }
+ if (WIFEXITED (r) && WEXITSTATUS (r) != 0) {
+ nbdkit_error ("command exited with code %d", WEXITSTATUS (r));
+ return -1;
+ }
+ else if (WIFSIGNALED (r)) {
+ nbdkit_error ("command killed by signal %d", WTERMSIG (r));
+ return -1;
+ }
+ else if (WIFSTOPPED (r)) {
+ nbdkit_error ("command stopped by signal %d", WSTOPSIG (r));
+ return -1;
+ }
+
+ return 0;
+}
+
+/* For block devices, stat->st_size is not the true size. */
+static int64_t
+block_device_size (int fd)
+{
+ off_t size;
+
+ size = lseek (fd, 0, SEEK_END);
+ if (size == -1) {
+ nbdkit_error ("lseek: %m");
+ return -1;
+ }
+
+ return size;
+}
+
+static void *
+ondemand_open (int readonly)
+{
+ struct handle *h;
+ CLEANUP_FREE char *disk = NULL;
+ int flags, err;
+ struct stat statbuf;
+ struct flock lock;
+
+ h = malloc (sizeof *h);
+ if (h == NULL) {
+ nbdkit_error ("malloc: %m");
+ goto error;
+ }
+ h->fd = -1;
+ h->size = -1;
+ h->can_punch_hole = true;
+
+ /* This is safe since we're only storing it in the handle, so only
+ * for the lifetime of this connection.
+ */
+ h->exportname = nbdkit_export_name ();
+ if (!h->exportname) {
+ nbdkit_error ("internal error: expected nbdkit_export_name () != NULL");
+ goto error;
+ }
+ if (strcmp (h->exportname, "") == 0)
+ h->exportname = "default";
+
+ /* Verify that the export name is valid. */
+ if (strlen (h->exportname) > NAME_MAX ||
+ strchr (h->exportname, '.') ||
+ strchr (h->exportname, '/') ||
+ strchr (h->exportname, ':')) {
+ nbdkit_error ("invalid exportname ‘%s’ rejected", h->exportname);
+ goto error;
+ }
+
+ /* Try to open the filesystem. */
+ if (readonly)
+ flags = O_RDONLY | O_CLOEXEC;
+ else
+ flags = O_RDWR | O_CLOEXEC;
+ h->fd = openat (dirfd (exportsdir), h->exportname, flags);
+ if (h->fd == -1) {
+ if (errno != ENOENT) {
+ nbdkit_error ("open: %s/%s: %m", dir, h->exportname);
+ goto error;
+ }
+
+ /* Create the filesystem. */
+ if (asprintf (&disk, "%s/%s", dir, h->exportname) == -1) {
+ nbdkit_error ("asprintf: %m");
+ goto error;
+ }
+
+ /* Now run the mkfs command. */
+ if (run_command (disk) == -1)
+ goto error;
+
+ h->fd = openat (dirfd (exportsdir), h->exportname, flags);
+ if (h->fd == -1) {
+ nbdkit_error ("open: %s/%s: %m", dir, h->exportname);
+ goto error;
+ }
+ }
+
+ /* Lock the file to prevent filesystem corruption. It's safe for
+ * all clients to be reading. If a client wants to write it must
+ * have exclusive access.
+ *
+ * This uses a currently Linux-specific extension. It requires
+ * Linux >= 3.15 (released in 2014, later backported to RHEL 7).
+ * There is no sensible way to do this in pure POSIX.
+ */
+#ifdef F_OFD_SETLK
+ memset (&lock, 0, sizeof lock);
+ if (readonly)
+ lock.l_type = F_RDLCK;
+ else
+ lock.l_type = F_WRLCK;
+ lock.l_whence = SEEK_SET;
+ lock.l_start = 0;
+ lock.l_len = 0;
+ if (fcntl (h->fd, F_OFD_SETLK, &lock) == -1) {
+ if (errno == EACCES || errno == EAGAIN) {
+ nbdkit_error ("%s: filesystem is locked by another client",
+ h->exportname);
+ /* XXX Would be nice if NBD protocol supported some kind of "is
+ * locked" indication. If it did we could use it here.
+ */
+ errno = EINVAL;
+ goto error;
+ }
+ else {
+ nbdkit_error ("fcntl: %s/%s: %m", dir, h->exportname);
+ goto error;
+ }
+ }
+#endif
+
+ /* Find the size of the disk. */
+ if (fstat (h->fd, &statbuf) == -1) {
+ nbdkit_error ("fstat: %s: %m", disk);
+ goto error;
+ }
+
+ /* The command could set $disk to a regular file or a block device
+ * (or a symlink to either), so we must check that here.
+ */
+ if (S_ISBLK (statbuf.st_mode)) {
+ h->size = block_device_size (h->fd);
+ if (h->size == -1)
+ goto error;
+ }
+ else /* Regular file. */
+ h->size = statbuf.st_size;
+ nbdkit_debug ("ondemand: requested_size = %" PRIi64 ", size = %" PRIi64,
+ requested_size, h->size);
+
+ /* Return the handle. */
+ return h;
+
+ error:
+ err = errno;
+ if (h) {
+ if (h->fd >= 0)
+ close (h->fd);
+ free (h);
+ }
+ errno = err;
+ return NULL;
+}
+
+static void
+ondemand_close (void *handle)
+{
+ struct handle *h = handle;
+
+ close (h->fd);
+ free (h);
+}
+
+static int64_t
+ondemand_get_size (void *handle)
+{
+ struct handle *h = handle;
+
+ return h->size;
+}
+
+/* Read data from the file. */
+static int
+ondemand_pread (void *handle, void *buf,
+ uint32_t count, uint64_t offset,
+ uint32_t flags)
+{
+ struct handle *h = handle;
+
+ while (count > 0) {
+ ssize_t r = pread (h->fd, buf, count, offset);
+ if (r == -1) {
+ nbdkit_error ("pread: %m");
+ return -1;
+ }
+ if (r == 0) {
+ nbdkit_error ("pread: unexpected end of file");
+ return -1;
+ }
+ buf += r;
+ count -= r;
+ offset += r;
+ }
+
+ return 0;
+}
+
+/* Flush the file to disk. */
+static int
+ondemand_flush (void *handle, uint32_t flags)
+{
+ struct handle *h = handle;
+
+ if (fdatasync (h->fd) == -1) {
+ nbdkit_error ("fdatasync: %m");
+ return -1;
+ }
+
+ return 0;
+}
+
+/* Write data to the file. */
+static int
+ondemand_pwrite (void *handle, const void *buf,
+ uint32_t count, uint64_t offset,
+ uint32_t flags)
+{
+ struct handle *h = handle;
+
+ while (count > 0) {
+ ssize_t r = pwrite (h->fd, buf, count, offset);
+ if (r == -1) {
+ nbdkit_error ("pwrite: %m");
+ return -1;
+ }
+ buf += r;
+ count -= r;
+ offset += r;
+ }
+
+ if ((flags & NBDKIT_FLAG_FUA) && ondemand_flush (handle, 0) == -1)
+ return -1;
+
+ return 0;
+}
+
+#if defined (FALLOC_FL_PUNCH_HOLE)
+static int
+do_fallocate (int fd, int mode, off_t offset, off_t len)
+{
+ int r = fallocate (fd, mode, offset, len);
+ if (r == -1 && errno == ENODEV) {
+ /* kernel 3.10 fails with ENODEV for block device. Kernel >= 4.9 fails
+ * with EOPNOTSUPP in this case. Normalize errno to simplify callers.
+ */
+ errno = EOPNOTSUPP;
+ }
+ return r;
+}
+
+static bool
+is_enotsup (int err)
+{
+ return err == ENOTSUP || err == EOPNOTSUPP;
+}
+#endif
+
+/* Punch a hole in the file. */
+static int
+ondemand_trim (void *handle, uint32_t count, uint64_t offset, uint32_t flags)
+{
+#ifdef FALLOC_FL_PUNCH_HOLE
+ struct handle *h = handle;
+ int r;
+
+ if (h->can_punch_hole) {
+ r = do_fallocate (h->fd, FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE,
+ offset, count);
+ if (r == -1) {
+ /* Trim is advisory; we don't care if it fails for anything other
+ * than EIO or EPERM.
+ */
+ if (errno == EPERM || errno == EIO) {
+ nbdkit_error ("fallocate: %m");
+ return -1;
+ }
+
+ if (is_enotsup (EOPNOTSUPP))
+ h->can_punch_hole = false;
+
+ nbdkit_debug ("ignoring failed fallocate during trim: %m");
+ }
+ }
+#endif
+
+ if ((flags & NBDKIT_FLAG_FUA) && ondemand_flush (handle, 0) == -1)
+ return -1;
+
+ return 0;
+}
+
+#define THREAD_MODEL NBDKIT_THREAD_MODEL_PARALLEL
+
+static struct nbdkit_plugin plugin = {
+ .name = "ondemand",
+ .version = PACKAGE_VERSION,
+
+ .unload = ondemand_unload,
+ .config = ondemand_config,
+ .config_complete = ondemand_config_complete,
+ .config_help = ondemand_config_help,
+ .magic_config_key = "size",
+ .get_ready = ondemand_get_ready,
+
+ .list_exports = ondemand_list_exports,
+
+ .can_multi_conn = ondemand_can_multi_conn,
+ .can_trim = ondemand_can_trim,
+ .can_fua = ondemand_can_fua,
+ .get_size = ondemand_get_size,
+
+ .open = ondemand_open,
+ .close = ondemand_close,
+ .pread = ondemand_pread,
+ .pwrite = ondemand_pwrite,
+ .flush = ondemand_flush,
+ .trim = ondemand_trim,
+
+ .errno_is_preserved = 1,
+};
+
+NBDKIT_REGISTER_PLUGIN(plugin)
diff --git a/plugins/ondemand/default-command.sh.in b/plugins/ondemand/default-command.sh.in
new file mode 100644
index 00000000..d60402b5
--- /dev/null
+++ b/plugins/ondemand/default-command.sh.in
@@ -0,0 +1,57 @@
+# nbdkit
+# -*- mode: shell-script -*-
+# Copyright (C) 2017-2020 Red Hat Inc.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are
+# met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+#
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+#
+# * Neither the name of Red Hat nor the names of its contributors may be
+# used to endorse or promote products derived from this software without
+# specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND
+# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
+# USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+# SUCH DAMAGE.
+
+# If not set, default to ext4.
+type="${type:-ext4}"
+
+labelopt='-L'
+
+case "$type" in
+ ext?)
+ extra='-F' ;;
+ *fat|msdos)
+ extra='-I' ;;
+ ntfs)
+ extra='-Q -F'
+ labelopt='-n' ;;
+ xfs)
+ extra='-f' ;;
+esac
+
+# Create the output disk.
+truncate -s $size "$disk"
+
+if [ "x$label" = "x" ]; then
+ mkfs -t "$type" $extra "$disk"
+else
+ mkfs -t "$type" $extra $labelopt "$label" "$disk"
+fi
diff --git a/tests/test-ondemand-list.sh b/tests/test-ondemand-list.sh
new file mode 100755
index 00000000..5e848e8d
--- /dev/null
+++ b/tests/test-ondemand-list.sh
@@ -0,0 +1,66 @@
+#!/usr/bin/env bash
+# nbdkit
+# Copyright (C) 2020 Red Hat Inc.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are
+# met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+#
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+#
+# * Neither the name of Red Hat nor the names of its contributors may be
+# used to endorse or promote products derived from this software without
+# specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND
+# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
+# USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+# SUCH DAMAGE.
+
+source ./functions.sh
+set -e
+set -x
+
+requires qemu-nbd --version
+
+dir=`mktemp -d`
+cleanup_fn rm -rf $dir
+
+out=test-ondemand-list.out
+rm -f $out
+cleanup_fn rm -f $out
+
+# Put some files into the exports directory to pretend that we're
+# restarting nbdkit after a previous run.
+touch $dir/default
+touch $dir/export1
+touch $dir/export2
+touch $dir/export3
+
+export LANG=C
+nbdkit -U - ondemand dir=$dir size=1M \
+ --run 'qemu-nbd -k $unixsocket -L' > $out
+cat $out
+
+# We should have 4 exports, since "default" file is the same as the
+# default export.
+grep "exports available: 4" $out
+
+# Check the 4 exports are present.
+grep "export: ''" $out
+grep "export: 'export1'" $out
+grep "export: 'export2'" $out
+grep "export: 'export3'" $out
diff --git a/tests/test-ondemand.sh b/tests/test-ondemand.sh
new file mode 100755
index 00000000..758c2565
--- /dev/null
+++ b/tests/test-ondemand.sh
@@ -0,0 +1,74 @@
+#!/usr/bin/env bash
+# nbdkit
+# Copyright (C) 2020 Red Hat Inc.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are
+# met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+#
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+#
+# * Neither the name of Red Hat nor the names of its contributors may be
+# used to endorse or promote products derived from this software without
+# specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND
+# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
+# USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+# SUCH DAMAGE.
+
+source ./functions.sh
+set -e
+set -x
+
+requires guestfish --version
+requires qemu-img --version
+
+dir=`mktemp -d`
+cleanup_fn rm -rf $dir
+
+sock=`mktemp -u`
+files="ondemand.pid $sock"
+rm -f $files
+cleanup_fn rm -f $files
+
+# Run nbdkit.
+start_nbdkit -P ondemand.pid -U $sock --log=stderr ondemand dir=$dir size=100M
+
+# Simply querying an export will create the filesystem.
+qemu-img info nbd:unix:$sock
+qemu-img info nbd:unix:$sock:exportname=test
+
+test -f $dir/default
+test -f $dir/test
+
+# These should fail because the exportname is invalid.
+if qemu-img info nbd:unix:$sock:exportname=/bad ||
+ qemu-img info nbd:unix:$sock:exportname=.bad ||
+ qemu-img info nbd:unix:$sock:exportname=bad. ||
+ qemu-img info nbd:unix:$sock:exportname=bad:bad
+then
+ echo "$0: expected failure trying to create bad exportname"
+ exit 1
+fi
+
+# Check the filesystem is persistent.
+guestfish --format=raw -a "nbd://?socket=$sock" -m /dev/sda <<EOF
+ write /test.txt "hello"
+EOF
+guestfish --ro --format=raw -a "nbd://?socket=$sock" -m /dev/sda <<EOF
+ cat /test.txt
+EOF
diff --git a/.gitignore b/.gitignore
index 255a97a5..2c463909 100644
--- a/.gitignore
+++ b/.gitignore
@@ -77,6 +77,7 @@ plugins/*/*.3
/plugins/golang/examples/*/nbdkit-*-plugin.h
/plugins/golang/examples/*/nbdkit-*-plugin.so
/plugins/ocaml/nbdkit-ocamlexample-plugin.so
+/plugins/ondemand/default-command.c
/plugins/rust/Cargo.lock
/plugins/rust/target
/plugins/tmpdisk/default-command.c
diff --git a/TODO b/TODO
index c329382d..d5802974 100644
--- a/TODO
+++ b/TODO
@@ -154,6 +154,13 @@ nbdkit-torrent-plugin:
* The C++ could be a lot more natural. At the moment it's a kind of
“C with C++ extensions”.
+nbdkit-ondemand-plugin:
+
+* Implement more callbacks, eg. .zero
+
+* Allow client to select size up to a limit, eg. by sending export
+ names like ‘export:4G’.
+
Suggestions for language plugins
--------------------------------
--
2.27.0
3 years, 9 months
[PATCH] Increase HIVEX_MAX_VALUES
by Matt Coleman
Hello,
As we discussed briefly on IRC about a month ago, HKLM\SYSTEM\MountedDevices can contain greater than 55000 (HIVEX_MAX_VALUES) values if VMWare’s snapshot functionality is frequently used. This is due to an unintended interaction between Windows and VMWare’s snapshot functionality.
VMware has a knowledge base article regarding this issue, although it does not directly mention MountedDevices: https://kb.vmware.com/s/article/2006849
The attached patch increases HIVEX_MAX_VALUES in order to enable access to those hives with hivex.
--
Matt Coleman
Senior Software Engineer
Datto, Inc.
www.datto.com
3 years, 9 months
[v2v PATCH 00/14] Adaptations to Weblate
by Pino Toscano
We are migrating to Weblate (the Fedora instance, in particular) for
translations instead of Zanata. Adapt our tooling a bit to the different
workflow:
- Weblate takes care of updating the po files whenever a new translation
catalog is available, so stop doing that on our own: this meant also
tweaking the po4a usage for POD documentations, resulting in simpler
rules (IMHO)
- ensure that the references to sources files in translation catalogs
are relative to the top-level directory, so Weblate (and users too,
actually) can properly locate the sources
- extract also C sources
- regenerate the catalogs
- drop references to Zanata
- update the translations to do edits before handing them over to
Weblate
- remove empty translations (v2v is translated less than libguestfs)
Pino Toscano (14):
po: turn language list into LINGUAS file
po-docs: turn language list into LINGUAS file
po: remove rules for pot/po update
podfiles: add missing documentation
po-docs: add missing dependencies for virt-v2v.1
po-docs: split pot and po handling
po: fix references to OCaml sources
po-docs: fix references to sources
Remove references to Zanata
po: extract again messages from C sources
po/po-docs: update catalog templates
po: update translations from catalog templates
po: remove empty translations
po-docs: update translations from catalog templates
.gitignore | 1 -
Makefile.am | 18 +-
m4/guestfs-progs.m4 | 5 +-
po-docs/LINGUAS | 12 +
po-docs/Makefile.am | 62 +-
po-docs/cs.po | 3960 +-
po-docs/de.po | 9299 +--
po-docs/en_GB.po | 62274 +------------------
po-docs/es.po | 5760 +-
po-docs/eu.po | 3993 +-
po-docs/fr.po | 30007 +---------
po-docs/ja.po | 37459 +-----------
po-docs/language.mk | 31 +-
po-docs/nl.po | 3815 +-
po-docs/podfiles | 31 +-
po-docs/pt_BR.po | 3790 +-
po-docs/tg.po | 4009 +-
po-docs/uk.po | 111660 +----------------------------------
po-docs/virt-v2v-docs.pot | 3783 +-
po-docs/zh_CN.po | 3939 +-
po/LINGUAS | 15 +
po/Makefile.am | 32 +-
po/POTFILES | 24 +
po/cs.po | 1294 +-
po/de.po | 2003 +-
po/en_GB.po | 2684 -
po/es.po | 2911 +-
po/eu.po | 2541 -
po/fr.po | 3562 +-
po/gu.po | 2096 +-
po/hi.po | 1944 +-
po/ja.po | 3746 +-
po/kn.po | 1928 +-
po/ml.po | 1910 +-
po/mr.po | 2602 +-
po/nl.po | 3279 +-
po/or.po | 1911 +-
po/pa.po | 2610 +-
po/pl.po | 3101 +-
po/ru.po | 2541 -
po/ta.po | 3292 --
po/te.po | 3288 --
po/tg.po | 2529 -
po/uk.po | 6020 +-
po/virt-v2v.pot | 2014 +-
po/zh_CN.po | 2581 -
subdir-rules.mk | 3 +
zanata-pull.sh | 30 -
zanata.xml | 7 -
49 files changed, 29826 insertions(+), 316580 deletions(-)
create mode 100644 po-docs/LINGUAS
create mode 100644 po/LINGUAS
create mode 100644 po/POTFILES
delete mode 100644 po/en_GB.po
delete mode 100644 po/eu.po
delete mode 100644 po/ru.po
delete mode 100644 po/ta.po
delete mode 100644 po/te.po
delete mode 100644 po/tg.po
delete mode 100644 po/zh_CN.po
delete mode 100755 zanata-pull.sh
delete mode 100644 zanata.xml
--
2.26.2
3 years, 9 months
[PATCH] po-docs: turn language list into LINGUAS file
by Pino Toscano
Use a LINGUAS file with the list of available translations instead of
defining them in a make variable. This way Weblate will be able to
update the list using an available addon, and we do not need to list
those not built.
Accordingly, rename the variable with built languages to
'linguas_translated'.
Signed-off-by: Pino Toscano <ptoscano(a)redhat.com>
---
po-docs/LINGUAS | 12 ++++++++++++
po-docs/Makefile.am | 12 +++++-------
2 files changed, 17 insertions(+), 7 deletions(-)
create mode 100644 po-docs/LINGUAS
diff --git a/po-docs/LINGUAS b/po-docs/LINGUAS
new file mode 100644
index 000000000..85b7927a3
--- /dev/null
+++ b/po-docs/LINGUAS
@@ -0,0 +1,12 @@
+cs
+de
+en_GB
+es
+eu
+fr
+ja
+nl
+pt_BR
+tg
+uk
+zh_CN
diff --git a/po-docs/Makefile.am b/po-docs/Makefile.am
index 028fbbcea..ed21cb152 100644
--- a/po-docs/Makefile.am
+++ b/po-docs/Makefile.am
@@ -21,25 +21,23 @@ include $(top_srcdir)/subdir-rules.mk
# into the po/ directory and the translations into the usual
# libguestfs.pot file.
-# Languages that we translate.
+# Languages.
# Don't use LINGUAS (uppercase) as Gentoo defines it (RHBZ#804464).
-linguas = ja uk
+linguas := $(shell cat $(srcdir)/LINGUAS)
-# Languages that we have PO files for, but we don't translate
-# because there is insufficient coverage.
-linguas_not_translated = cs de en_GB es eu fr nl pt_BR tg zh_CN
+# Languages that we translate, as they have enough coverage.
+linguas_translated = ja uk
EXTRA_DIST = \
libguestfs-docs.pot \
$(linguas:%=%.po) \
- $(linguas_not_translated:%=%.po) \
podfiles
# Build the final man pages from the translated POD files. Each
# language directory contains a Makefile.am that we need to keep up to
# date (note each $lang/Makefile.am should be identical).
# XXX Is there a better way?
-SUBDIRS = $(linguas)
+SUBDIRS = $(linguas_translated)
libguestfs-docs.pot:
cd $(top_srcdir) && $(PO4A_GETTEXTIZE) \
--
2.26.2
3 years, 9 months
[PATCH 0/9] Adaptations to Weblate
by Pino Toscano
We are migrating to Weblate (the Fedora instance, in particular) for
translations instead of Zanata. Adapt our tooling a bit to the different
workflow:
- Weblate takes care of updating the po files whenever a new translation
catalog is available, so stop doing that on our own: this meant also
tweaking the po4a usage for POD documentations, resulting in simpler
rules (IMHO)
- ensure that the references to sources files in translation catalogs
are relative to the top-level directory, so Weblate (and users too,
actually) can properly locate the sources
- regenerate the catalogs
- drop references to Zanata
Pino Toscano (9):
po: turn language list into LINGUAS file
po: remove rules for pot/po update
po-docs: add missing dependencies for guestfish.1
podfiles: add missing documentation
po-docs: split pot and po handling
po: fix references to OCaml sources
po-docs: fix references to sources
po/po-docs: update catalog templates
Remove references to Zanata
.gitignore | 1 -
Makefile.am | 4 +-
docs/guestfs-hacking.pod | 14 -
m4/guestfs-progs.m4 | 5 +-
po-docs/Makefile.am | 50 +-
po-docs/language.mk | 31 +-
po-docs/libguestfs-docs.pot | 37211 +++++++++++++++++-----------------
po-docs/podfiles | 181 +-
po/LINGUAS | 22 +
po/Makefile.am | 24 +-
po/libguestfs.pot | 1597 +-
subdir-rules.mk | 3 +
zanata-pull.sh | 30 -
zanata.xml | 8 -
14 files changed, 19456 insertions(+), 19725 deletions(-)
create mode 100644 po/LINGUAS
delete mode 100755 zanata-pull.sh
delete mode 100644 zanata.xml
--
2.26.2
3 years, 9 months
[PATCH v3] appliance: extract UUID from QCOW2 disk image
by Andrey Shinkevich
For the appliance of the QCOW2 format, the function get_root_uuid()
fails to get the UUID of the disk image.
In this case, let us read the first 256k bytes of the disk image with
the 'qemu-img dd' command. Then pass the read block to the 'file'
command.
Suggested-by: Denis V. Lunev <den(a)openvz.org>
Signed-off-by: Andrey Shinkevich <andrey.shinkevich(a)virtuozzo.com>
---
v3:
01: The code refactoring was made based on reviewers comments.
The e-mail thread Message-Id:
<1597246570-901723-1-git-send-email-andrey.shinkevich(a)virtuozzo.com>
lib/appliance-kcmdline.c | 68 +++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 67 insertions(+), 1 deletion(-)
diff --git a/lib/appliance-kcmdline.c b/lib/appliance-kcmdline.c
index fbeb4f4..e088fc0 100644
--- a/lib/appliance-kcmdline.c
+++ b/lib/appliance-kcmdline.c
@@ -71,7 +71,7 @@ read_uuid (guestfs_h *g, void *retv, const char *line, size_t len)
* The L<file(1)> command does the hard work.
*/
static char *
-get_root_uuid (guestfs_h *g, const char *appliance)
+get_root_uuid_with_file (guestfs_h *g, const char *appliance)
{
CLEANUP_CMD_CLOSE struct command *cmd = guestfs_int_new_command (g);
char *ret = NULL;
@@ -96,6 +96,72 @@ get_root_uuid (guestfs_h *g, const char *appliance)
}
/**
+ * Read the first 256k bytes of the in_file with L<qemu-img(1)> command
+ * and write them into the out_file. That may be useful to get UUID of
+ * the QCOW2 disk image with further L<file(1)> command.
+ * The function returns zero if successful, otherwise -1.
+ */
+static int
+run_qemu_img_dd (guestfs_h *g, const char *in_file, char *out_file)
+{
+ CLEANUP_CMD_CLOSE struct command *cmd = guestfs_int_new_command (g);
+ int r;
+
+ guestfs_int_cmd_add_arg (cmd, "qemu-img");
+ guestfs_int_cmd_add_arg (cmd, "dd");
+ guestfs_int_cmd_add_arg_format (cmd, "if=%s", in_file);
+ guestfs_int_cmd_add_arg_format (cmd, "of=%s", out_file);
+ guestfs_int_cmd_add_arg (cmd, "bs=256k");
+ guestfs_int_cmd_add_arg (cmd, "count=1");
+
+ r = guestfs_int_cmd_run (cmd);
+ if (r == -1) {
+ error (g, "Failed to run qemu-img");
+ return -1;
+ }
+ if (!WIFEXITED (r) || WEXITSTATUS (r) != 0) {
+ guestfs_int_external_command_failed (g, r, "qemu-img dd", NULL);
+ return -1;
+ }
+
+ return 0;
+}
+
+/**
+ * Get the UUID from the appliance disk image.
+ */
+static char *
+get_root_uuid (guestfs_h *g, const char *appliance)
+{
+ char *uuid = NULL;
+ int ret;
+ CLEANUP_UNLINK_FREE char *tmpfile = NULL;
+
+ uuid = get_root_uuid_with_file (g, appliance);
+ if (uuid) {
+ return uuid;
+ }
+
+ tmpfile = guestfs_int_make_temp_path (g, "root", "raw");
+ if (!tmpfile) {
+ error (g, "get_root_uuid: failed to make temporary file");
+ return NULL;
+ }
+
+ ret = run_qemu_img_dd (g, appliance, tmpfile);
+ if (ret < 0) {
+ return NULL;
+ }
+
+ uuid = get_root_uuid_with_file (g, tmpfile);
+ if (!uuid) {
+ error (g, "Failed to get the appliance UUID");
+ }
+
+ return uuid;
+}
+
+/**
* Construct the Linux command line passed to the appliance. This is
* used by the C<direct> and C<libvirt> backends, and is simply
* located in this file because it's a convenient place for this
--
1.8.3.1
3 years, 9 months
[PATCH v2] appliance: extract UUID from QCOW2 disk image
by Andrey Shinkevich
For the appliance of the QCOW2 format, get the UUID of the disk by
reading the first 256k bytes with 'qemu-img dd' command. Then pass the
read block to the 'file' command. In case of failure, run the 'file'
command again directly.
Suggested-by: Denis V. Lunev <den(a)openvz.org>
Signed-off-by: Andrey Shinkevich <andrey.shinkevich(a)virtuozzo.com>
---
v2:
01: The order of the function calls to direct <file> command on the appliance
and the indirect one (through the temporary file) has been swapped.
lib/appliance-kcmdline.c | 70 +++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 69 insertions(+), 1 deletion(-)
diff --git a/lib/appliance-kcmdline.c b/lib/appliance-kcmdline.c
index fbeb4f4..c78524b 100644
--- a/lib/appliance-kcmdline.c
+++ b/lib/appliance-kcmdline.c
@@ -71,7 +71,7 @@ read_uuid (guestfs_h *g, void *retv, const char *line, size_t len)
* The L<file(1)> command does the hard work.
*/
static char *
-get_root_uuid (guestfs_h *g, const char *appliance)
+do_get_root_uuid (guestfs_h *g, const char *appliance)
{
CLEANUP_CMD_CLOSE struct command *cmd = guestfs_int_new_command (g);
char *ret = NULL;
@@ -96,6 +96,74 @@ get_root_uuid (guestfs_h *g, const char *appliance)
}
/**
+ * Read the first 256k bytes of the in_file with L<qemu-img(1)> command
+ * and write them into the out_file. That may be useful to get UUID of
+ * the QCOW2 disk image with further L<file(1)> command.
+ * The function returns zero if successful, otherwise -1.
+ */
+static int
+run_qemu_img_dd (guestfs_h *g, const char *in_file, char *out_file)
+{
+ CLEANUP_CMD_CLOSE struct command *cmd = guestfs_int_new_command (g);
+ int r;
+
+ guestfs_int_cmd_add_arg (cmd, "qemu-img");
+ guestfs_int_cmd_add_arg (cmd, "dd");
+ guestfs_int_cmd_add_arg_format (cmd, "if=%s", in_file);
+ guestfs_int_cmd_add_arg_format (cmd, "of=%s", out_file);
+ guestfs_int_cmd_add_arg (cmd, "bs=256k");
+ guestfs_int_cmd_add_arg (cmd, "count=1");
+
+ r = guestfs_int_cmd_run (cmd);
+ if (r == -1) {
+ error (g, "Failed to run qemu-img");
+ return -1;
+ }
+ if (!WIFEXITED (r) || WEXITSTATUS (r) != 0) {
+ guestfs_int_external_command_failed (g, r, "qemu-img dd", NULL);
+ return -1;
+ }
+
+ return 0;
+}
+
+/**
+ * Get the UUID from the appliance disk image.
+ */
+static char *
+get_root_uuid (guestfs_h *g, const char *appliance)
+{
+ char *uuid = NULL;
+ int ret;
+ char tmp_file[] = "/tmp/libguestfsXXXXXX";
+
+ uuid = do_get_root_uuid (g, appliance);
+ if (uuid) {
+ return uuid;
+ }
+
+ if (!mktemp (tmp_file)) {
+ error (g, "get_root_uuid: mktemp failed");
+ return NULL;
+ }
+
+ ret = run_qemu_img_dd (g, appliance, tmp_file);
+ if (ret == 0) {
+ uuid = do_get_root_uuid (g, tmp_file);
+ if (uuid) {
+ goto out;
+ }
+ }
+
+ /* We get here in case of failure to extract UUID from the temporary file. */
+ error (g, "Failed to get the appliance UUID");
+
+out:
+ unlink (tmp_file);
+ return uuid;
+}
+
+/**
* Construct the Linux command line passed to the appliance. This is
* used by the C<direct> and C<libvirt> backends, and is simply
* located in this file because it's a convenient place for this
--
1.8.3.1
3 years, 9 months
[PATCH] appliance: extract UUID from QCOW2 disk image
by Andrey Shinkevich
For the appliance of the QCOW2 format, get the UUID of the disk by
reading the first 256k bytes with 'qemu-img dd' command. Then pass the
read block to the 'file' command. In case of failure, run the 'file'
command again directly.
Suggested-by: Denis V. Lunev <den(a)openvz.org>
Signed-off-by: Andrey Shinkevich <andrey.shinkevich(a)virtuozzo.com>
---
lib/appliance-kcmdline.c | 70 +++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 69 insertions(+), 1 deletion(-)
diff --git a/lib/appliance-kcmdline.c b/lib/appliance-kcmdline.c
index 211cc46..3bb8bcd 100644
--- a/lib/appliance-kcmdline.c
+++ b/lib/appliance-kcmdline.c
@@ -71,7 +71,7 @@ read_uuid (guestfs_h *g, void *retv, const char *line, size_t len)
* The L<file(1)> command does the hard work.
*/
static char *
-get_root_uuid (guestfs_h *g, const char *appliance)
+do_get_root_uuid (guestfs_h *g, const char *appliance)
{
CLEANUP_CMD_CLOSE struct command *cmd = guestfs_int_new_command (g);
char *ret = NULL;
@@ -96,6 +96,74 @@ get_root_uuid (guestfs_h *g, const char *appliance)
}
/**
+ * Read the first 256k bytes of the in_file with L<qemu-img(1)> command
+ * and write them into the out_file. That may be useful to get UUID of
+ * the QCOW2 disk image with further L<file(1)> command.
+ * The function returns zero if successful, otherwise -1.
+ */
+static int
+run_qemu_img_dd (guestfs_h *g, const char *in_file, char *out_file)
+{
+ CLEANUP_CMD_CLOSE struct command *cmd = guestfs_int_new_command (g);
+ int r;
+
+ guestfs_int_cmd_add_arg (cmd, "qemu-img");
+ guestfs_int_cmd_add_arg (cmd, "dd");
+ guestfs_int_cmd_add_arg_format (cmd, "if=%s", in_file);
+ guestfs_int_cmd_add_arg_format (cmd, "of=%s", out_file);
+ guestfs_int_cmd_add_arg (cmd, "bs=256k");
+ guestfs_int_cmd_add_arg (cmd, "count=1");
+
+ r = guestfs_int_cmd_run (cmd);
+ if (r == -1) {
+ error (g, "Failed to run qemu-img");
+ return -1;
+ }
+ if (!WIFEXITED (r) || WEXITSTATUS (r) != 0) {
+ guestfs_int_external_command_failed (g, r, "qemu-img dd", NULL);
+ return -1;
+ }
+
+ return 0;
+}
+
+/**
+ * Get the UUID from the appliance disk image.
+ */
+static char *
+get_root_uuid (guestfs_h *g, const char *appliance)
+{
+ char *UUID = NULL;
+ int ret;
+ char tmp_file[] = "/tmp/libguestfsXXXXXX";
+
+ if (!mktemp (tmp_file)) {
+ error (g, "get_root_uuid: mktemp failed");
+ return NULL;
+ }
+
+ ret = run_qemu_img_dd (g, appliance, tmp_file);
+ if (ret == 0) {
+ UUID = do_get_root_uuid (g, tmp_file);
+ if (UUID) {
+ goto out;
+ }
+ }
+
+ /* We get here in case of failure to extract a UUID from the temporary file.
+ * Let us try to get the UUID from the appliance directly.
+ */
+ UUID = do_get_root_uuid (g, appliance);
+ if (!UUID) {
+ error (g, "Failed to get the appliance UUID");
+ }
+
+out:
+ unlink (tmp_file);
+ return UUID;
+}
+
+/**
* Construct the Linux command line passed to the appliance. This is
* used by the C<direct> and C<libvirt> backends, and is simply
* located in this file because it's a convenient place for this
--
1.8.3.1
3 years, 9 months
