[PATCH nbdkit] New tmpdisk plugin.
Unfinished (needs tests). This is my attempt to make a "remote tmpfs" plugin as outlined in this prior email: https://www.redhat.com/archives/libguestfs/2020-March/msg00134.html Although it would be possible to construct something a bit like this using existing plugins and filters (perhaps with some new features in those filters) I think it may be nicer to have a dedicated plugin for this, as it makes the documentation simpler to understand and allows us to implement unique features. In particular notice that this plugin only deals with temporary disks so deliberately ignores flush/FUA, which should make it nice and fast. Rich.
This can be used for creating temporary disks to thin clients, as a kind of "remote tmpfs". See also: https://www.redhat.com/archives/libguestfs/2020-March/msg00134.html --- plugins/data/nbdkit-data-plugin.pod | 1 + plugins/file/nbdkit-file-plugin.pod | 1 + plugins/linuxdisk/nbdkit-linuxdisk-plugin.pod | 7 +- plugins/memory/nbdkit-memory-plugin.pod | 3 +- plugins/tmpdisk/nbdkit-tmpdisk-plugin.pod | 157 +++++++ configure.ac | 2 + plugins/tmpdisk/Makefile.am | 66 +++ plugins/tmpdisk/tmpdisk.c | 426 ++++++++++++++++++ 8 files changed, 660 insertions(+), 3 deletions(-) diff --git a/plugins/data/nbdkit-data-plugin.pod b/plugins/data/nbdkit-data-plugin.pod index 057392d3..ef8d1e08 100644 --- a/plugins/data/nbdkit-data-plugin.pod +++ b/plugins/data/nbdkit-data-plugin.pod @@ -269,6 +269,7 @@ L<nbdkit-null-plugin(1)>, L<nbdkit-partitioning-plugin(1)>, L<nbdkit-pattern-plugin(1)>, L<nbdkit-random-plugin(1)>, +L<nbdkit-tmpdisk-plugin(1)>, L<nbdkit-zero-plugin(1)>, L<https://github.com/libguestfs/nbdkit/blob/master/plugins/data/disk2data.pl>, L<https://en.wikipedia.org/wiki/Base64>. diff --git a/plugins/file/nbdkit-file-plugin.pod b/plugins/file/nbdkit-file-plugin.pod index d538b127..0c1cfd57 100644 --- a/plugins/file/nbdkit-file-plugin.pod +++ b/plugins/file/nbdkit-file-plugin.pod @@ -111,6 +111,7 @@ L<nbdkit(1)>, L<nbdkit-plugin(3)>, L<nbdkit-split-plugin(1)>, L<nbdkit-partitioning-plugin(1)>, +L<nbdkit-tmpdisk-plugin(1)>, L<nbdkit-noextents-filter(1)>. =head1 AUTHORS diff --git a/plugins/linuxdisk/nbdkit-linuxdisk-plugin.pod b/plugins/linuxdisk/nbdkit-linuxdisk-plugin.pod index 3cac883c..53f4d89d 100644 --- a/plugins/linuxdisk/nbdkit-linuxdisk-plugin.pod +++ b/plugins/linuxdisk/nbdkit-linuxdisk-plugin.pod @@ -24,7 +24,9 @@ symbolic links, block special devices etc. To create a FAT-formatted virtual floppy disk, see L<nbdkit-floppy-plugin(1)>. To create a CD/ISO, see -L<nbdkit-iso-plugin(1)>. +L<nbdkit-iso-plugin(1)>. To create an empty filesystem for each +client that connects (like a "remote tmpfs") use +L<nbdkit-tmpdisk-plugin(1)>. =head1 EXAMPLES @@ -184,7 +186,8 @@ L<nbdkit-file-plugin(1)>, L<nbdkit-floppy-plugin(1)>, L<nbdkit-iso-plugin(1)>, L<nbdkit-partition-filter(1)>, -L<nbdkit-partitioning-plugin(1)>. +L<nbdkit-partitioning-plugin(1)>, +L<nbdkit-tmpdisk-plugin(1)>, =head1 AUTHORS diff --git a/plugins/memory/nbdkit-memory-plugin.pod b/plugins/memory/nbdkit-memory-plugin.pod index ccdc017d..bc565c55 100644 --- a/plugins/memory/nbdkit-memory-plugin.pod +++ b/plugins/memory/nbdkit-memory-plugin.pod @@ -103,7 +103,8 @@ L<nbdkit-plugin(3)>, L<nbdkit-loop(1)>, L<nbdkit-data-plugin(1)>, L<nbdkit-file-plugin(1)>, -L<nbdkit-info-plugin(1)>. +L<nbdkit-info-plugin(1)>, +L<nbdkit-tmpdisk-plugin(1)>. =head1 AUTHORS diff --git a/plugins/tmpdisk/nbdkit-tmpdisk-plugin.pod b/plugins/tmpdisk/nbdkit-tmpdisk-plugin.pod new file mode 100644 index 00000000..925a5091 --- /dev/null +++ b/plugins/tmpdisk/nbdkit-tmpdisk-plugin.pod @@ -0,0 +1,157 @@ +=head1 NAME + +nbdkit-tmpdisk-plugin - create a fresh temporary filesystem for each client + +=head1 SYNOPSIS + + nbdkit tmpdisk [size=]SIZE + [type=ext4|xfs|vfat|...] [label=LABEL] + [command=COMMAND] + +=head1 DESCRIPTION + +This L<nbdkit(1)> plugin is used for creating temporary filesystems +for thin clients. Each time a client connects it will see a fresh, +empty filesystem for its exclusive use. B<The filesystem is deleted> +when the client disconnects. + +When a new client connects, a blank, sparse file of the required size +is created in C<$TMPDIR> (or F</var/tmp>). L<mkfs(8)> is then run on +the file to create the empty filesystem, and this filesystem is served +to the client. Unlike L<nbdkit-linuxdisk-plugin(1)> each client of +this plugin sees a different disk. + +The size of the disk is chosen using the C<size> parameter. The +filesystem type is C<ext4> but this can be changed using the C<type> +parameter (controlling the I<-t> option of mkfs). + +Instead of running mkfs you can run an arbitrary C<command> to create +the disk. + +=head2 Example + +One use for this is to create a kind of "remote L<tmpfs(5)>" for thin +clients. On the server you would run (see L<nbdkit-service(1)>): + + nbdkit tmpdisk 16G + +and each thin client would have a file F</etc/rc.d/rc.local> +containing: + + nm-online + modprobe nbd + nbd-client server /dev/nbd0 + mount /dev/nbd0 /var/scratch + +Clients would see a fresh, empty C</var/scratch> directory after boot. + +=head2 Security considerations + +Because each client gets a new disk, the amount of disk space +required on the server can be as much as +S<<< I<number of clients> × I<size parameter> >>>. It is therefore +best to limit the number of clients using L<nbdkit-limit-filter(1)> or +take steps to limit where clients can connect from using +L<nbdkit-ip-filter(1)>, firewalls, or TLS client certificates. + +=head1 PARAMETERS + +=over 4 + +=item B<command='>COMMANDB<'> + +Instead of running L<mkfs(8)> to create the initial filesystem, run +C<COMMAND> (which usually must be quoted to protect it from the +shell). The following shell variables may be used in C<COMMAND>: + +=over 4 + +=item C<$disk> + +The absolute path of the file that the command must initialize. This +file is created by nbdkit before the command runs. + +=item C<$label> + +The filesystem label (from the C<label> parameter). + +=item C<$size> + +The virtual size in bytes, which is the same as the file size. + +=item C<$type> + +The filesystem type (from the C<type> parameter), defaulting to +C<ext4>. (Commands can ignore this if it is not relevant). + +=back + +=item B<label=>LABEL + +Select the filesystem label. The default is not set. + +=item [B<size=>]SIZE + +Specify the virtual size of the disk image. + +This parameter is required. + +C<size=> is a magic config key and may be omitted in most cases. +See L<nbdkit(1)/Magic parameters>. + +=item B<type=>FS + +Select the filesystem type. The default is C<ext4>. Most +non-networked, non-cluster filesystem types supported by the +L<mkfs(8)> command can be used here. + +=back + +=head1 ENVIRONMENT VARIABLES + +=over 4 + +=item C<TMPDIR> + +The temporary disks for this plugin are created in this directory, one +per connected client. If not set this defaults to F</var/tmp>. + +=back + +=head1 FILES + +=over 4 + +=item F<$plugindir/nbdkit-tmpdisk-plugin.so> + +The plugin. + +Use C<nbdkit --dump-config> to find the location of C<$plugindir>. + +=back + +=head1 VERSION + +C<nbdkit-tmpdisk-plugin> first appeared in nbdkit 1.20. + +=head1 SEE ALSO + +L<nbdkit(1)>, +L<nbdkit-plugin(3)>, +L<nbdkit-data-plugin(1)>, +L<nbdkit-file-plugin(1)>, +L<nbdkit-ip-filter(1)>, +L<nbdkit-limit-filter(1)>, +L<nbdkit-linuxdisk-plugin(1)>, +L<nbdkit-memory-plugin(1)>, +L<nbdkit-loop(1)>, +L<nbdkit-tls(1)>, +L<mkfs(8)>. + +=head1 AUTHORS + +Richard W.M. Jones + +=head1 COPYRIGHT + +Copyright (C) 2018-2020 Red Hat Inc. diff --git a/configure.ac b/configure.ac index dd9ca765..7b5e61e3 100644 --- a/configure.ac +++ b/configure.ac @@ -86,6 +86,7 @@ non_lang_plugins="\ ssh \ streaming \ tar \ + tmpdisk \ vddk \ zero \ " @@ -1016,6 +1017,7 @@ AC_CONFIG_FILES([Makefile plugins/streaming/Makefile plugins/tar/Makefile plugins/tcl/Makefile + plugins/tmpdisk/Makefile plugins/vddk/Makefile plugins/zero/Makefile filters/Makefile diff --git a/plugins/tmpdisk/Makefile.am b/plugins/tmpdisk/Makefile.am new file mode 100644 index 00000000..2e487e92 --- /dev/null +++ b/plugins/tmpdisk/Makefile.am @@ -0,0 +1,66 @@ +# nbdkit +# Copyright (C) 2017-2020 Red Hat Inc. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are +# met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# * Neither the name of Red Hat nor the names of its contributors may be +# used to endorse or promote products derived from this software without +# specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, +# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A +# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR +# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF +# USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND +# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT +# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# SUCH DAMAGE. + +include $(top_srcdir)/common-rules.mk + +EXTRA_DIST = nbdkit-tmpdisk-plugin.pod + +plugin_LTLIBRARIES = nbdkit-tmpdisk-plugin.la + +nbdkit_tmpdisk_plugin_la_SOURCES = \ + tmpdisk.c \ + $(top_srcdir)/include/nbdkit-plugin.h \ + $(NULL) + +nbdkit_tmpdisk_plugin_la_CPPFLAGS = \ + -I$(top_srcdir)/include \ + -I$(top_srcdir)/common/utils \ + $(NULL) +nbdkit_tmpdisk_plugin_la_CFLAGS = $(WARNINGS_CFLAGS) +nbdkit_tmpdisk_plugin_la_LDFLAGS = \ + -module -avoid-version -shared \ + -Wl,--version-script=$(top_srcdir)/plugins/plugins.syms \ + $(NULL) +nbdkit_tmpdisk_plugin_la_LIBADD = \ + $(top_builddir)/common/utils/libutils.la \ + $(NULL) + +if HAVE_POD + +man_MANS = nbdkit-tmpdisk-plugin.1 +CLEANFILES += $(man_MANS) + +nbdkit-tmpdisk-plugin.1: nbdkit-tmpdisk-plugin.pod + $(PODWRAPPER) --section=1 --man $@ \ + --html $(top_builddir)/html/$@.html \ + $< + +endif HAVE_POD diff --git a/plugins/tmpdisk/tmpdisk.c b/plugins/tmpdisk/tmpdisk.c new file mode 100644 index 00000000..aa3c25f9 --- /dev/null +++ b/plugins/tmpdisk/tmpdisk.c @@ -0,0 +1,426 @@ +/* nbdkit + * Copyright (C) 2017-2020 Red Hat Inc. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * * Neither the name of Red Hat nor the names of its contributors may be + * used to endorse or promote products derived from this software without + * specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, + * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A + * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR + * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF + * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND + * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, + * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT + * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include <config.h> + +#include <stdio.h> +#include <stdlib.h> +#include <stdbool.h> +#include <stdint.h> +#include <inttypes.h> +#include <string.h> +#include <unistd.h> +#include <fcntl.h> +#include <errno.h> +#include <sys/types.h> +#include <sys/wait.h> + +#define NBDKIT_API_VERSION 2 +#include <nbdkit-plugin.h> + +#include "cleanup.h" +#include "utils.h" + +static int64_t size = -1; + +static const char *command = + "labelopt='-L'\n" + "case \"$type\" in\n" + " ext?)\n" + " extra='-F' ;;\n" + " *fat|msdos)\n" + " extra='-I' ;;\n" + " ntfs)\n" + " extra='-Q -F'\n" + " labelopt='-n' ;;\n" + " xfs)\n" + " extra='-f' ;;\n" + "esac\n" + "if [ \"x$label\" = \"x\" ]; then\n" + " mkfs -t \"$type\" $extra $disk >/dev/null\n" + "else\n" + " mkfs -t \"$type\" $extra $labelopt \"$label\" $disk >/dev/null\n" + "fi\n"; +static const char *label = NULL; +static const char *type = "ext4"; + +static int +tmpdisk_config (const char *key, const char *value) +{ + if (strcmp (key, "command") == 0) { + command = value; + } + else if (strcmp (key, "label") == 0) { + if (strcmp (value, "") == 0) + label = NULL; + else + label = value; + } + else if (strcmp (key, "size") == 0) { + size = nbdkit_parse_size (value); + if (size == -1) + return -1; + } + else if (strcmp (key, "type") == 0) { + type = value; + } + else { + nbdkit_error ("unknown parameter '%s'", key); + return -1; + } + + return 0; +} + +static int +tmpdisk_config_complete (void) +{ + if (size == -1) { + nbdkit_error ("size parameter is required"); + return -1; + } + + return 0; +} + +#define tmpdisk_config_help \ + "size=<SIZE> (required) Virtual filesystem size.\n" \ + "label=<LABEL> The filesystem label.\n" \ + "type=ext4|... The filesystem type.\n" \ + "command=<COMMAND> Alternate command instead of mkfs." + +struct handle { + int fd; + bool can_punch_hole; +}; + +/* Absolutely unsafe! Although this is simply returning the default + * value, provide this callback to make it clear. + */ +static int +tmpdisk_can_multi_conn (void *handle) +{ + return 0; +} + +static int +tmpdisk_can_trim (void *handle) +{ +#ifdef FALLOC_FL_PUNCH_HOLE + return 1; +#else + return 0; +#endif +} + +/* Pretend we have native FUA support, but actually because all disks + * are temporarily we will deliberately ignore flush/FUA operations. + */ +static int +tmpdisk_can_fua (void *handle) +{ + return NBDKIT_FUA_NATIVE; +} + +static int64_t +tmpdisk_get_size (void *handle) +{ + return size; +} + +/* This creates and runs the full "mkfs" (or whatever) command. */ +static int +run_command (const char *disk) +{ + FILE *fp; + CLEANUP_FREE char *cmd = NULL; + size_t len = 0; + int r; + + fp = open_memstream (&cmd, &len); + if (fp == NULL) { + nbdkit_error ("open_memstream: %m"); + return -1; + } + + /* Set the shell variables. */ + fprintf (fp, "disk="); + shell_quote (disk, fp); + putc ('\n', fp); + if (label) { + fprintf (fp, "label="); + shell_quote (label, fp); + putc ('\n', fp); + } + fprintf (fp, "size=%" PRIi64 "\n", size); + fprintf (fp, "type="); + shell_quote (type, fp); + putc ('\n', fp); + putc ('\n', fp); + fprintf (fp, "%s", command); + + if (fclose (fp) == EOF) { + nbdkit_error ("memstream failed"); + return -1; + } + + r = system (cmd); + if (r == -1) { + nbdkit_error ("failed to execute command: %m"); + return -1; + } + if (WIFEXITED (r) && WEXITSTATUS (r) != 0) { + nbdkit_error ("command exited with code %d", WEXITSTATUS (r)); + return -1; + } + else if (WIFSIGNALED (r)) { + nbdkit_error ("command killed by signal %d", WTERMSIG (r)); + return -1; + } + else if (WIFSTOPPED (r)) { + nbdkit_error ("command stopped by signal %d", WSTOPSIG (r)); + return -1; + } + + return 0; +} + +static void * +tmpdisk_open (int readonly) +{ + struct handle *h; + CLEANUP_FREE char *disk = NULL; + const char *tmpdir; + + tmpdir = getenv ("TMPDIR"); + if (!tmpdir) + tmpdir = "/var/tmp"; + + h = malloc (sizeof *h); + if (h == NULL) { + nbdkit_error ("malloc: %m"); + goto error; + } + h->fd = -1; + h->can_punch_hole = true; + + /* Create the new disk image for this connection. */ + if (asprintf (&disk, "%s/tmpdiskXXXXXX", tmpdir) == -1) { + nbdkit_error ("asprintf: %m"); + goto error; + } + + h->fd = mkstemp (disk); + if (h->fd == -1) { + nbdkit_error ("mkstemp: %m"); + goto error; + } + + /* Truncate the disk to a sparse file of the right size. */ + if (ftruncate (h->fd, size) == -1) { + nbdkit_error ("ftruncate: %s: %m", disk); + goto error; + } + + /* Now run the mkfs command. */ + if (run_command (disk) == -1) + goto error; + + /* We don't need the disk to appear in the filesystem since we hold + * a file descriptor open and access it through that, so unlink the + * disk. This also ensures it is always cleaned up. + */ + unlink (disk); + + /* Return the handle. */ + return h; + + error: + if (h) { + if (h->fd >= 0) { + close (h->fd); + unlink (disk); + } + free (h); + } + return NULL; +} + +static void +tmpdisk_close (void *handle) +{ + struct handle *h = handle; + + close (h->fd); + free (h); +} + +/* Read data from the file. */ +static int +tmpdisk_pread (void *handle, void *buf, + uint32_t count, uint64_t offset, + uint32_t flags) +{ + struct handle *h = handle; + + while (count > 0) { + ssize_t r = pread (h->fd, buf, count, offset); + if (r == -1) { + nbdkit_error ("pread: %m"); + return -1; + } + if (r == 0) { + nbdkit_error ("pread: unexpected end of file"); + return -1; + } + buf += r; + count -= r; + offset += r; + } + + return 0; +} + +/* Write data to the file. */ +static int +tmpdisk_pwrite (void *handle, const void *buf, + uint32_t count, uint64_t offset, + uint32_t flags) +{ + struct handle *h = handle; + + while (count > 0) { + ssize_t r = pwrite (h->fd, buf, count, offset); + if (r == -1) { + nbdkit_error ("pwrite: %m"); + return -1; + } + buf += r; + count -= r; + offset += r; + } + + /* Deliberately ignore FUA if present in flags. */ + + return 0; +} + +/* This plugin deliberately provides a null flush operation, because + * all of the disks created are temporary. + */ +static int +tmpdisk_flush (void *handle, uint32_t flags) +{ + return 0; +} + +#if defined (FALLOC_FL_PUNCH_HOLE) +static int +do_fallocate (int fd, int mode, off_t offset, off_t len) +{ + int r = fallocate (fd, mode, offset, len); + if (r == -1 && errno == ENODEV) { + /* kernel 3.10 fails with ENODEV for block device. Kernel >= 4.9 fails + * with EOPNOTSUPP in this case. Normalize errno to simplify callers. + */ + errno = EOPNOTSUPP; + } + return r; +} + +static bool +is_enotsup (int err) +{ + return err == ENOTSUP || err == EOPNOTSUPP; +} +#endif + +/* Punch a hole in the file. */ +static int +tmpdisk_trim (void *handle, uint32_t count, uint64_t offset, uint32_t flags) +{ +#ifdef FALLOC_FL_PUNCH_HOLE + struct handle *h = handle; + int r; + + if (h->can_punch_hole) { + r = do_fallocate (h->fd, FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE, + offset, count); + if (r == -1) { + /* Trim is advisory; we don't care if it fails for anything other + * than EIO or EPERM. + */ + if (errno == EPERM || errno == EIO) { + nbdkit_error ("fallocate: %m"); + return -1; + } + + if (is_enotsup (EOPNOTSUPP)) + h->can_punch_hole = false; + + nbdkit_debug ("ignoring failed fallocate during trim: %m"); + } + } +#endif + + /* Deliberately ignore FUA if present in flags. */ + + return 0; +} + +#define THREAD_MODEL NBDKIT_THREAD_MODEL_PARALLEL + +static struct nbdkit_plugin plugin = { + .name = "tmpdisk", + .version = PACKAGE_VERSION, + + .config = tmpdisk_config, + .config_complete = tmpdisk_config_complete, + .config_help = tmpdisk_config_help, + .magic_config_key = "size", + + .can_multi_conn = tmpdisk_can_multi_conn, + .can_trim = tmpdisk_can_trim, + .can_fua = tmpdisk_can_fua, + .get_size = tmpdisk_get_size, + + .open = tmpdisk_open, + .close = tmpdisk_close, + .pread = tmpdisk_pread, + .pwrite = tmpdisk_pwrite, + .flush = tmpdisk_flush, + .trim = tmpdisk_trim, + + .errno_is_preserved = 1, +}; + +NBDKIT_REGISTER_PLUGIN(plugin) -- 2.25.0
participants (1)
-
Richard W.M. Jones