[Libguestfs] [PATCH nbdkit v3 4/4] Add linuxdisk plugin.

From: "Richard W.M. Jones" <rjones(a)redhat.com&gt; This plugin allows you to create a complete ext2, ext3 or ext4 filesystem in a GPT partitioned disk image. This can be attached as a disk to a Linux virtual machine. It is implemented using e2fsprogs mke2fs ‘-d’ option thus allowing the implementation to be very small and simple, with all the hard work done by mke2fs. Although there is some overlap with nbdkit-iso-plugin and nbdkit-floppy-plugin, the implementations and use cases of all three plugins are sufficiently different that it seems to make sense to add another plugin rather than attempting to extend one of the existing plugins. Largely to avoid user error this plugin is read-only. This is a major difference from the floppy plugin: that plugin allows files to be modified (but not resized or created) and writes those changes through to the backing filesystem. While this plugin could easily be made writable, this would cause almost certain disk corruption when someone connected two clients at the same time. In any case it doesn't make much sense for it to be writable by default since the expectation that writes would somehow modify the original directory on the host filesystem cannot be satisfied by this or any reasonable implementation. Users can add the cow filter on top if they really want writes and know what they are doing: instructions plus disclaimer about this are included in the man page. Eventually we could replace the supermin libext2fs code with this plugin, but there are some missing features that would need to be implemented first. --- README | 4 + TODO | 11 +- configure.ac | 2 + plugins/floppy/nbdkit-floppy-plugin.pod | 4 + plugins/iso/nbdkit-iso-plugin.pod | 6 +- plugins/linuxdisk/Makefile.am | 73 ++++++ plugins/linuxdisk/filesystem.c | 267 +++++++++++++++++++++ plugins/linuxdisk/linuxdisk.c | 229 ++++++++++++++++++ plugins/linuxdisk/nbdkit-linuxdisk-plugin.pod | 180 ++++++++++++++ plugins/linuxdisk/partition-gpt.c | 211 ++++++++++++++++ plugins/linuxdisk/virtual-disk.c | 161 +++++++++++++ plugins/linuxdisk/virtual-disk.h | 92 +++++++ .../partitioning/nbdkit-partitioning-plugin.pod | 5 +- tests/Makefile.am | 9 + tests/test-linuxdisk-copy-out.sh | 76 ++++++ tests/test-linuxdisk.sh | 93 +++++++ 16 files changed, 1412 insertions(+), 11 deletions(-) create mode 100644 plugins/linuxdisk/Makefile.am create mode 100644 plugins/linuxdisk/filesystem.c create mode 100644 plugins/linuxdisk/linuxdisk.c create mode 100644 plugins/linuxdisk/nbdkit-linuxdisk-plugin.pod create mode 100644 plugins/linuxdisk/partition-gpt.c create mode 100644 plugins/linuxdisk/virtual-disk.c create mode 100644 plugins/linuxdisk/virtual-disk.h create mode 100755 tests/test-linuxdisk-copy-out.sh create mode 100755 tests/test-linuxdisk.sh diff --git a/README b/README index e394a1f..9c4d844 100644 --- a/README +++ b/README @@ -102,6 +102,10 @@ For the ext2 plugin: - com_err +For the linuxdisk plugin: + + - mke2fs >= 1.42.10 (from e2fsprogs) + For the Perl, example4 and tar plugins: - perl interpreter diff --git a/TODO b/TODO index 94de9c9..59590a1 100644 --- a/TODO +++ b/TODO @@ -85,13 +85,6 @@ directed to qemu-nbd for these use cases. https://lists.gnu.org/archive/html/qemu-devel/2017-11/msg02971.html is a partial solution but it needs cleaning up. -* Create ext2 filesystems - - Similar to nbdkit-floppy-plugin which creates FAT32 filesystems, we - could also create ext2 filesystems. This is relatively - straightforward using libext2fs (part of e2fsprogs). In fact we - already use this library for the unrelated nbdkit-ext2-plugin. - nbdkit-nbd-plugin could use enhancements: * FUA passthrough, rather than extra FLUSH calls. For this, .can_fua @@ -114,6 +107,10 @@ nbdkit-floppy-plugin: * Add multiple dir merging. +nbdkit-linuxdisk-plugin: + +* Add multiple dir merging (in e2fsprogs mke2fs). + Suggestions for filters ----------------------- diff --git a/configure.ac b/configure.ac index 229e38c..4487b5e 100644 --- a/configure.ac +++ b/configure.ac @@ -793,6 +793,7 @@ non_lang_plugins="\ gzip \ iso \ libvirt \ + linuxdisk \ memory \ nbd \ null \ @@ -855,6 +856,7 @@ AC_CONFIG_FILES([Makefile plugins/gzip/Makefile plugins/iso/Makefile plugins/libvirt/Makefile + plugins/linuxdisk/Makefile plugins/lua/Makefile plugins/memory/Makefile plugins/nbd/Makefile diff --git a/plugins/floppy/nbdkit-floppy-plugin.pod b/plugins/floppy/nbdkit-floppy-plugin.pod index ae14946..9087b86 100644 --- a/plugins/floppy/nbdkit-floppy-plugin.pod +++ b/plugins/floppy/nbdkit-floppy-plugin.pod @@ -18,6 +18,9 @@ The virtual floppy disk will have a single partition (using an MBR partition table). In that partition will be a virtual FAT32 filesystem containing the files. Long filenames are supported. +To create a CD/ISO, see L<nbdkit-iso-plugin(1)>. To create a Linux +compatible virtual disk, see L<nbdkit-linuxdisk-plugin(1)>. + =head1 EXAMPLE Create a virtual floppy disk: @@ -74,6 +77,7 @@ important, and it simplifies the implementation greatly. L<nbdkit(1)>, L<nbdkit-plugin(3)>, L<nbdkit-file-plugin(1)>, +L<nbdkit-linuxdisk-plugin(1)>, L<nbdkit-iso-plugin(1)>. =head1 AUTHORS diff --git a/plugins/iso/nbdkit-iso-plugin.pod b/plugins/iso/nbdkit-iso-plugin.pod index 4d9cf41..90e26f0 100644 --- a/plugins/iso/nbdkit-iso-plugin.pod +++ b/plugins/iso/nbdkit-iso-plugin.pod @@ -17,8 +17,9 @@ read-only over the NBD protocol. This plugin uses L<genisoimage(1)> or L<mkisofs(1)> to create the ISO content. -To create a virtual floppy disk instead of a CD, see -L<nbdkit-floppy-plugin(1)>. +To create a FAT-formatted virtual floppy disk instead of a CD, see +L<nbdkit-floppy-plugin(1)>. To create a Linux compatible virtual +disk, see L<nbdkit-linuxdisk-plugin(1)>. =head1 EXAMPLE @@ -96,6 +97,7 @@ L<nbdkit(1)>, L<nbdkit-plugin(3)>, L<nbdkit-file-plugin(1)>, L<nbdkit-floppy-plugin(1)>, +L<nbdkit-linuxdisk-plugin(1)>, L<genisoimage(1)>, L<mkisofs(1)>. diff --git a/plugins/linuxdisk/Makefile.am b/plugins/linuxdisk/Makefile.am new file mode 100644 index 0000000..1e94567 --- /dev/null +++ b/plugins/linuxdisk/Makefile.am @@ -0,0 +1,73 @@ +# nbdkit +# Copyright (C) 2019 Red Hat Inc. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are +# met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# * Neither the name of Red Hat nor the names of its contributors may be +# used to endorse or promote products derived from this software without +# specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, +# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A +# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR +# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF +# USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND +# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT +# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# SUCH DAMAGE. + +include $(top_srcdir)/common-rules.mk + +EXTRA_DIST = nbdkit-linuxdisk-plugin.pod + +plugin_LTLIBRARIES = nbdkit-linuxdisk-plugin.la + +nbdkit_linuxdisk_plugin_la_SOURCES = \ + filesystem.c \ + linuxdisk.c \ + partition-gpt.c \ + virtual-disk.c \ + virtual-disk.h \ + $(top_srcdir)/include/nbdkit-plugin.h + +nbdkit_linuxdisk_plugin_la_CPPFLAGS = \ + -I$(top_srcdir)/common/gpt \ + -I$(top_srcdir)/common/include \ + -I$(top_srcdir)/common/regions \ + -I$(top_srcdir)/common/utils \ + -I$(top_srcdir)/include +nbdkit_linuxdisk_plugin_la_CFLAGS = \ + $(WARNINGS_CFLAGS) +nbdkit_linuxdisk_plugin_la_LIBADD = \ + $(top_builddir)/common/gpt/libgpt.la \ + $(top_builddir)/common/regions/libregions.la \ + $(top_builddir)/common/utils/libutils.la +nbdkit_linuxdisk_plugin_la_LDFLAGS = \ + -module -avoid-version -shared \ + -Wl,--version-script=$(top_srcdir)/plugins/plugins.syms + +if HAVE_POD + +man_MANS = nbdkit-linuxdisk-plugin.1 +CLEANFILES += $(man_MANS) + +nbdkit-linuxdisk-plugin.1: nbdkit-linuxdisk-plugin.pod + $(PODWRAPPER) --section=1 --man $@ \ + --html $(top_builddir)/html/$@.html \ + $< + +endif HAVE_POD diff --git a/plugins/linuxdisk/filesystem.c b/plugins/linuxdisk/filesystem.c new file mode 100644 index 0000000..a1d431a --- /dev/null +++ b/plugins/linuxdisk/filesystem.c @@ -0,0 +1,267 @@ +/* nbdkit + * Copyright (C) 2018-2019 Red Hat Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * * Neither the name of Red Hat nor the names of its contributors may be + * used to endorse or promote products derived from this software without + * specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, + * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A + * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR + * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF + * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND + * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, + * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT + * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include <config.h> + +#include <stdio.h> +#include <stdlib.h> +#include <stdbool.h> +#include <stdint.h> +#include <inttypes.h> +#include <string.h> +#include <unistd.h> +#include <fcntl.h> +#include <errno.h> + +#include <nbdkit-plugin.h> + +#include "minmax.h" +#include "rounding.h" +#include "utils.h" + +#include "virtual-disk.h" + +static int64_t estimate_size (void); +static int mke2fs (const char *filename); + +int +create_filesystem (struct virtual_disk *disk) +{ + const char *tmpdir; + char *filename = NULL; + int fd = -1; + + /* Estimate the filesystem size and compute the final virtual size + * of the disk. We only need to do this if the user didn't specify + * the exact size on the command line. + */ + if (size == 0 || size_add_estimate) { + int64_t estimate; + + estimate = estimate_size (); + if (estimate == -1) + goto error; + + nbdkit_debug ("filesystem size estimate: %" PRIi64, estimate); + + /* Add 20% to the estimate to account for the overhead of + * filesystem metadata. Also set a minimum size. Note we are + * only wasting virtual space (since this will be stored sparsely + * under $TMPDIR) so we can be generous here. + */ + estimate = estimate * 6 / 5; + estimate = MAX (estimate, 1024*1024); + + /* For ext3 and ext4, add something for the journal. */ + if (strncmp (type, "ext", 3) == 0 && type[3] > '2') + estimate += 32*1024*1024; + + if (size_add_estimate) + size += estimate; + else + size = estimate; + } + + /* Round the final size up to a whole number of sectors. */ + size = ROUND_UP (size, SECTOR_SIZE); + + nbdkit_debug ("filesystem virtual size: %" PRIi64, size); + + /* Create the filesystem file. */ + tmpdir = getenv ("TMPDIR"); + if (tmpdir == NULL) + tmpdir = LARGE_TMPDIR; + if (asprintf (&filename, "%s/linuxdiskXXXXXX", tmpdir) == -1) { + nbdkit_error ("asprintf: %m"); + goto error; + } + + fd = mkstemp (filename); + if (fd == -1) { + nbdkit_error ("mkstemp: %s: %m", filename); + goto error; + } + if (ftruncate (fd, size) == -1) { + nbdkit_error ("ftruncate: %s: %m", filename); + goto error; + } + + /* Create the filesystem. */ + if (mke2fs (filename) == -1) + goto error; + + unlink (filename); + free (filename); + disk->filesystem_size = size; + disk->fd = fd; + return 0; + + error: + if (fd >= 0) + close (fd); + if (filename) { + unlink (filename); + free (filename); + } + return -1; +} + +/* Use ‘du’ to estimate the size of the filesystem quickly. We use + * the -c option to allow the possibility of supporting multiple + * directories in future. + * + * Typical output from ‘du -cs dir1 dir2’ is: + * + * 12345 dir1 + * 34567 dir2 + * 46912 total + * + * We ignore everything except the first number on the last line. + */ +static int64_t +estimate_size (void) +{ + char *command = NULL, *line = NULL; + size_t len = 0; + FILE *fp; + int64_t ret; + int r; + + /* Create the du command. */ + fp = open_memstream (&command, &len); + if (fp == NULL) { + nbdkit_error ("open_memstream: %m"); + return -1; + } + fprintf (fp, "du -c -k -s "); + shell_quote (dir, fp); + if (fclose (fp) == EOF) { + nbdkit_error ("memstream failed: %m"); + return -1; + } + + /* Run the command. */ + nbdkit_debug ("%s", command); + fp = popen (command, "r"); + free (command); + if (fp == NULL) { + nbdkit_error ("du command failed: %m"); + return -1; + } + + /* Ignore everything up to the last line. */ + len = 0; + while (getline (&line, &len, fp) != -1) + /* empty */; + if (ferror (fp)) { + nbdkit_error ("getline failed: %m"); + free (line); + pclose (fp); + return -1; + } + + r = pclose (fp); + if (r == -1) { + nbdkit_error ("pclose: %m"); + free (line); + return -1; + } + if (WIFEXITED (r) && WEXITSTATUS (r) != 0) { + nbdkit_error ("du command failed with exit code %d", WEXITSTATUS (r)); + return -1; + } + + /* Parse the last line. */ + if (sscanf (line, "%" SCNi64, &ret) != 1 || ret < 0) { + nbdkit_error ("could not parse last line of output: %s", line); + free (line); + return -1; + } + free (line); + + /* Result is in 1K blocks, convert it to bytes. */ + ret *= 1024; + return ret; +} + +static int +mke2fs (const char *filename) +{ + char *command = NULL; + size_t len = 0; + FILE *fp; + int r; + + /* Create the mke2fs command. */ + fp = open_memstream (&command, &len); + if (fp == NULL) { + nbdkit_error ("open_memstream: %m"); + return -1; + } + + fprintf (fp, "mke2fs -F -t %s ", type); + if (label) { + fprintf (fp, "-L "); + shell_quote (label, fp); + fprintf (fp, " "); + } + fprintf (fp, "-d "); + shell_quote (dir, fp); + fprintf (fp, " "); + shell_quote (filename, fp); + + if (fclose (fp) == EOF) { + nbdkit_error ("memstream failed: %m"); + return -1; + } + + /* Run the command. */ + nbdkit_debug ("%s", command); + r = system (command); + free (command); + + if (WIFEXITED (r) && WEXITSTATUS (r) != 0) { + nbdkit_error ("mke2fs command failed with exit code %d", WEXITSTATUS (r)); + return -1; + } + else if (WIFSIGNALED (r)) { + nbdkit_error ("mke2fs command was killed by signal %d", WTERMSIG (r)); + return -1; + } + else if (WIFSTOPPED (r)) { + nbdkit_error ("mke2fs command was stopped by signal %d", WSTOPSIG (r)); + return -1; + } + + return 0; +} diff --git a/plugins/linuxdisk/linuxdisk.c b/plugins/linuxdisk/linuxdisk.c new file mode 100644 index 0000000..8f50dd3 --- /dev/null +++ b/plugins/linuxdisk/linuxdisk.c @@ -0,0 +1,229 @@ +/* nbdkit + * Copyright (C) 2019 Red Hat Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * * Neither the name of Red Hat nor the names of its contributors may be + * used to endorse or promote products derived from this software without + * specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, + * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A + * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR + * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF + * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND + * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, + * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT + * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include <config.h> + +#include <stdio.h> +#include <stdlib.h> +#include <stdbool.h> +#include <inttypes.h> +#include <string.h> +#include <unistd.h> +#include <time.h> + +#define NBDKIT_API_VERSION 2 + +#include <nbdkit-plugin.h> + +#include "random.h" +#include "regions.h" + +#include "virtual-disk.h" + +/* Directory, label, type, size parameters. */ +char *dir; +const char *label; +const char *type = "ext2"; +int64_t size; +bool size_add_estimate; /* if size=+SIZE was used */ + +/* Virtual disk. */ +static struct virtual_disk disk; + +/* Used to create a random GUID for the partition. */ +struct random_state random_state; + +static void +linuxdisk_load (void) +{ + init_virtual_disk (&disk); + xsrandom (time (NULL), &random_state); +} + +static void +linuxdisk_unload (void) +{ + free_virtual_disk (&disk); + free (dir); +} + +static int +linuxdisk_config (const char *key, const char *value) +{ + if (strcmp (key, "dir") == 0) { + if (dir != NULL) { + /* TODO: Support merging of multiple directories, like iso plugin. */ + nbdkit_error ("dir=<DIRECTORY> must only be set once"); + return -1; + } + dir = nbdkit_realpath (value); + if (dir == NULL) + return -1; + } + else if (strcmp (key, "label") == 0) { + label = value; + } + else if (strcmp (key, "type") == 0) { + if (strncmp (value, "ext", 3) != 0) { + nbdkit_error ("type=<TYPE> must be an filesystem type " + "supported by e2fsprogs"); + return -1; + } + type = value; + } + else if (strcmp (key, "size") == 0) { + if (value[0] == '+') { + size_add_estimate = true; + value++; + } + else + size_add_estimate = false; + size = nbdkit_parse_size (value); + if (size == -1) + return -1; + } + else { + nbdkit_error ("unknown parameter '%s'", key); + return -1; + } + + return 0; +} + +static int +linuxdisk_config_complete (void) +{ + if (dir == NULL) { + nbdkit_error ("you must supply the dir=<DIRECTORY> parameter " + "after the plugin name on the command line"); + return -1; + } + + return create_virtual_disk (&disk); +} + +#define linuxdisk_config_help \ + "dir=<DIRECTORY> (required) The directory to serve.\n" \ + "label=<LABEL> The filesystem label.\n" \ + "type=ext2|ext3|ext4 The filesystem type.\n" \ + "size=[+]<SIZE> The virtual filesystem size." + +static void * +linuxdisk_open (int readonly) +{ + return NBDKIT_HANDLE_NOT_NEEDED; +} + +#define THREAD_MODEL NBDKIT_THREAD_MODEL_PARALLEL + +/* Get the file size. */ +static int64_t +linuxdisk_get_size (void *handle) +{ + return virtual_size (&disk.regions); +} + +/* Serves the same data over multiple connections. */ +static int +linuxdisk_can_multi_conn (void *handle) +{ + return 1; +} + +/* Read data from the virtual disk. */ +static int +linuxdisk_pread (void *handle, void *buf, uint32_t count, uint64_t offset, + uint32_t flags) +{ + while (count > 0) { + const struct region *region = find_region (&disk.regions, offset); + size_t len; + ssize_t r; + + /* Length to end of region. */ + len = region->end - offset + 1; + if (len > count) + len = count; + + switch (region->type) { + case region_file: + /* We don't use region->u.i since there is only one backing + * file, and we have that open already (in ‘disk.fd’). + */ + r = pread (disk.fd, buf, len, offset - region->start); + if (r == -1) { + nbdkit_error ("pread: %m"); + return -1; + } + if (r == 0) { + nbdkit_error ("pread: unexpected end of file"); + return -1; + } + len = r; + break; + + case region_data: + memcpy (buf, &region->u.data[offset - region->start], len); + break; + + case region_zero: + memset (buf, 0, len); + break; + } + + count -= len; + buf += len; + offset += len; + } + + return 0; +} + +static struct nbdkit_plugin plugin = { + .name = "linuxdisk", + .longname = "nbdkit Linux virtual disk plugin", + .version = PACKAGE_VERSION, + .load = linuxdisk_load, + .unload = linuxdisk_unload, + .config = linuxdisk_config, + .config_complete = linuxdisk_config_complete, + .config_help = linuxdisk_config_help, + .magic_config_key = "dir", + .open = linuxdisk_open, + .get_size = linuxdisk_get_size, + .can_multi_conn = linuxdisk_can_multi_conn, + .pread = linuxdisk_pread, + .errno_is_preserved = 1, +}; + +NBDKIT_REGISTER_PLUGIN(plugin) diff --git a/plugins/linuxdisk/nbdkit-linuxdisk-plugin.pod b/plugins/linuxdisk/nbdkit-linuxdisk-plugin.pod new file mode 100644 index 0000000..2483837 --- /dev/null +++ b/plugins/linuxdisk/nbdkit-linuxdisk-plugin.pod @@ -0,0 +1,180 @@ +=head1 NAME + +nbdkit-linuxdisk-plugin - create virtual Linux disk from directory + +=head1 SYNOPSIS + + nbdkit linuxdisk [dir=]DIRECTORY + [label=LABEL] [type=ext2|ext3|ext4] + [size=[+]SIZE] + +=head1 DESCRIPTION + +C<nbdkit-linuxdisk-plugin> is a plugin for L<nbdkit(1)> which creates +an ext2-, ext3- or ext4-formatted disk image from a directory on the +fly. The files in the specified directory (and subdirectories) appear +in the virtual disk, which is served read-only over the NBD protocol. + +The virtual disk is partitioned with a single GPT partition containing +the filesystem. + +The virtual disk can be used as a Linux root (or other) filesystem. +Most features of Linux filesystems are supported, such as hard links, +symbolic links, block special devices etc. + +To create a FAT-formatted virtual floppy disk, see +L<nbdkit-floppy-plugin(1)>. To create a CD/ISO, see +L<nbdkit-iso-plugin(1)>. + +=head1 EXAMPLES + +=over 4 + +=item nbdkit linuxdisk /path/to/directory label=ROOTFS + +Create a virtual disk, giving it a filesystem label. Note that +clients will not be able to modify the filesystem, so it is safe to +share it with multiple clients. + +=item nbdkit --filter=cow linuxdisk /path/to/directory + +Add a writable overlay (see L<nbdkit-cow-filter(1)>, allowing the disk +to be written by the client. B<Multiple clients must not be allowed +to connect at the same time> (even if they all mount it read-only) as +this will cause disk corruption. + +=item nbdkit --filter=cow linuxdisk /path/to/directory size=+1G + +The same but specifying that at least 1G of free space should be +available in the filesystem (not including the space taken by the +initial filesystem). + +=item nbdkit --filter=partition linuxdisk /path/to/directory partition=1 + +Instead of serving a partitioned disk image, serve just the "naked" +filesystem (ie. the first partition, see +L<nbdkit-partition-filter(1)>). + +=item nbdkit -U - linuxdisk /path/to/directory +--run 'qemu-img convert $nbd ext2fs.img' + +This serves nothing. Instead it turns a directory into a disk image, +writing it to F<ext2fs.img> (see L<nbdkit-captive(1)>). The resulting +image is a partitioned disk. + +=item Create a minimal virtual appliance + +This creates and boots a minimal L<busybox(1)>-based virtual +appliance. This assumes that your kernel (F</boot/vmlinuz>) contains +the ext2 or ext4 driver compiled in, but most Linux distro kernels +have that. + + mkdir root root/bin root/dev root/proc root/sbin root/sys + mkdir root/usr root/usr/bin root/usr/sbin + sudo mknod root/dev/console c 5 1 + cp /sbin/busybox root/sbin/ + ln root/sbin/busybox root/sbin/init + ln root/sbin/busybox root/bin/ls + ln root/sbin/busybox root/bin/sh + nbdkit -U - linuxdisk root --run ' + qemu-kvm -display none -kernel /boot/vmlinuz -drive file=nbd:unix:$unixsocket,snapshot=on -append "console=ttyS0 root=/dev/sda1 rw" -serial stdio + ' + +You can drop any extra files you need into the F<root/> directory and +they will be copied into the appliance before boot. After booting +type these commands to complete the environmental setup: + + /sbin/busybox --install + mount -t proc proc /proc + mount -t sysfs sys /sys + +=back + +=head1 PARAMETERS + +=over 4 + +=item [B<dir=>]DIRECTORY + +Specify the directory containing files and subdirectories which will +be added to the virtual disk. Files inside this directory will appear +in the root directory of the virtual disk. + +This parameter is required. + +C<dir=> is a magic config key and may be omitted in most cases. +See L<nbdkit(1)/Magic parameters>. + +=item B<label=>LABEL + +The optional label for the filesystem. + +=item B<size=>SIZE + +=item B<size=+>SIZE + +The total (virtual) size of the filesystem. + +If the C<size> parameter is omitted the plugin will try to size the +filesystem with just enough space to contain the files and directories +that are initially loaded, and there will not be much extra space. + +Using C<size=SIZE> specifies the required virtual size of the whole +filesystem (including initial files and extra space). If this is set +too small for the initial filesystem then the plugin will fail to +start. + +Using C<size=+SIZE> specifies the minimum free space required after +the initial filesystem has been loaded. (The actual free space might +be slightly larger). + +=item B<type=ext2> + +=item B<type=ext3> + +=item B<type=ext4> + +Select the filesystem type. The default is C<ext2>. + +=back + +=head1 NOTES + +=head2 Users and groups. + +The original file UIDs and GIDs are recreated as far as possible. +Note that UIDs/GIDs will likely map to different users and groups when +read by a virtual machine or other NBD client machine. + +=head1 ENVIRONMENT VARIABLES + +=over 4 + +=item C<TMPDIR> + +The filesystem image is stored in a temporary file located in +F</var/tmp> by default. You can override this location by setting the +C<TMPDIR> environment variable before starting nbdkit. + +=back + +=head1 SEE ALSO + +L<mke2fs(8)>, +L<nbdkit(1)>, +L<nbdkit-plugin(3)>, +L<nbdkit-captive(1)>, +L<nbdkit-cow-filter(1)>, +L<nbdkit-file-plugin(1)>, +L<nbdkit-floppy-plugin(1)>, +L<nbdkit-iso-plugin(1)>, +L<nbdkit-partition-filter(1)>, +L<nbdkit-partitioning-plugin(1)>. + +=head1 AUTHORS + +Richard W.M. Jones + +=head1 COPYRIGHT + +Copyright (C) 2019 Red Hat Inc. diff --git a/plugins/linuxdisk/partition-gpt.c b/plugins/linuxdisk/partition-gpt.c new file mode 100644 index 0000000..3278229 --- /dev/null +++ b/plugins/linuxdisk/partition-gpt.c @@ -0,0 +1,211 @@ +/* nbdkit + * Copyright (C) 2018-2019 Red Hat Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * * Neither the name of Red Hat nor the names of its contributors may be + * used to endorse or promote products derived from this software without + * specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, + * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A + * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR + * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF + * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND + * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, + * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT + * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include <config.h> + +#include <stdio.h> +#include <stdlib.h> +#include <stdbool.h> +#include <stdint.h> +#include <inttypes.h> +#include <string.h> +#include <unistd.h> + +#include <nbdkit-plugin.h> + +#include "efi-crc32.h" +#include "gpt.h" +#include "isaligned.h" +#include "rounding.h" +#include "regions.h" + +#include "virtual-disk.h" + +#define PARTITION_TYPE_GUID "0FC63DAF-8483-4772-8E79-3D69D8477DE4" + +static void create_gpt_protective_mbr (struct virtual_disk *disk, + unsigned char *out); +static void create_gpt_partition_header (struct virtual_disk *disk, + const void *pt, bool is_primary, + unsigned char *out); +static void create_gpt_partition_table (struct virtual_disk *disk, + unsigned char *out); + +/* Initialize the partition table structures. */ +int +create_partition_table (struct virtual_disk *disk) +{ + create_gpt_protective_mbr (disk, disk->protective_mbr); + + create_gpt_partition_table (disk, disk->pt); + + create_gpt_partition_header (disk, disk->pt, true, disk->primary_header); + create_gpt_partition_header (disk, disk->pt, false, disk->secondary_header); + + return 0; +} + +static void +chs_too_large (unsigned char *out) +{ + const int c = 1023, h = 254, s = 63; + + out[0] = h; + out[1] = (c & 0x300) >> 2 | s; + out[2] = c & 0xff; +} + +static void +create_mbr_partition_table_entry (const struct region *region, + bool bootable, int partition_id, + unsigned char *out) +{ + uint64_t start_sector, nr_sectors; + uint32_t u32; + + assert (IS_ALIGNED (region->start, SECTOR_SIZE)); + + start_sector = region->start / SECTOR_SIZE; + nr_sectors = DIV_ROUND_UP (region->len, SECTOR_SIZE); + + assert (start_sector <= UINT32_MAX); + assert (nr_sectors <= UINT32_MAX); + + out[0] = bootable ? 0x80 : 0; + chs_too_large (&out[1]); + out[4] = partition_id; + chs_too_large (&out[5]); + u32 = htole32 (start_sector); + memcpy (&out[8], &u32, 4); + u32 = htole32 (nr_sectors); + memcpy (&out[12], &u32, 4); +} + +static void +create_gpt_protective_mbr (struct virtual_disk *disk, unsigned char *out) +{ + struct region region; + uint64_t end; + + /* Protective MBR creates an MBR partition with partition ID 0xee + * which covers the whole of the disk, or as much of the disk as + * expressible with MBR. + */ + region.start = 512; + end = virtual_size (&disk->regions) - 1; + if (end > UINT32_MAX * SECTOR_SIZE) + end = UINT32_MAX * SECTOR_SIZE; + region.end = end; + region.len = region.end - region.start + 1; + + create_mbr_partition_table_entry (&region, false, 0xee, &out[0x1be]); + + /* Boot sector signature. */ + out[0x1fe] = 0x55; + out[0x1ff] = 0xaa; +} + +static void +create_gpt_partition_header (struct virtual_disk *disk, + const void *pt, bool is_primary, + unsigned char *out) +{ + uint64_t nr_lbas; + struct gpt_header *header = (struct gpt_header *) out; + + nr_lbas = virtual_size (&disk->regions) / SECTOR_SIZE; + + memset (header, 0, sizeof *header); + memcpy (header->signature, GPT_SIGNATURE, sizeof (header->signature)); + memcpy (header->revision, GPT_REVISION, sizeof (header->revision)); + header->header_size = htole32 (sizeof *header); + if (is_primary) { + header->current_lba = htole64 (1); + header->backup_lba = htole64 (nr_lbas - 1); + } + else { + header->current_lba = htole64 (nr_lbas - 1); + header->backup_lba = htole64 (1); + } + header->first_usable_lba = htole64 (34); + header->last_usable_lba = htole64 (nr_lbas - 34); + if (is_primary) + header->partition_entries_lba = htole64 (2); + else + header->partition_entries_lba = htole64 (nr_lbas - 33); + header->nr_partition_entries = htole32 (GPT_MIN_PARTITIONS); + header->size_partition_entry = htole32 (GPT_PT_ENTRY_SIZE); + header->crc_partitions = + htole32 (efi_crc32 (pt, GPT_PT_ENTRY_SIZE * GPT_MIN_PARTITIONS)); + + /* Must be computed last. */ + header->crc = htole32 (efi_crc32 (header, sizeof *header)); +} + +static void +create_gpt_partition_table_entry (const struct region *region, + bool bootable, + char partition_type_guid[16], + char guid[16], + unsigned char *out) +{ + struct gpt_entry *entry = (struct gpt_entry *) out; + + assert (sizeof (struct gpt_entry) == GPT_PT_ENTRY_SIZE); + + memcpy (entry->partition_type_guid, partition_type_guid, 16); + memcpy (entry->unique_guid, guid, 16); + + entry->first_lba = htole64 (region->start / SECTOR_SIZE); + entry->last_lba = htole64 (region->end / SECTOR_SIZE); + entry->attributes = htole64 (bootable ? 4 : 0); +} + +static void +create_gpt_partition_table (struct virtual_disk *disk, unsigned char *out) +{ + size_t j; + + for (j = 0; j < nr_regions (&disk->regions); ++j) { + const struct region *region = get_region (&disk->regions, j); + + /* Find the (only) partition region, which has type region_file. */ + if (region->type == region_file) { + create_gpt_partition_table_entry (region, true, + PARTITION_TYPE_GUID, + disk->guid, + out); + out += GPT_PT_ENTRY_SIZE; + } + } +} diff --git a/plugins/linuxdisk/virtual-disk.c b/plugins/linuxdisk/virtual-disk.c new file mode 100644 index 0000000..2f24ce7 --- /dev/null +++ b/plugins/linuxdisk/virtual-disk.c @@ -0,0 +1,161 @@ +/* nbdkit + * Copyright (C) 2018-2019 Red Hat Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * * Neither the name of Red Hat nor the names of its contributors may be + * used to endorse or promote products derived from this software without + * specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, + * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A + * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR + * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF + * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND + * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, + * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT + * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include <config.h> + +#include <stdio.h> +#include <stdlib.h> +#include <stdbool.h> +#include <stdint.h> +#include <inttypes.h> +#include <string.h> +#include <unistd.h> + +#include <nbdkit-plugin.h> + +#include "random.h" +#include "regions.h" + +#include "virtual-disk.h" + +static int create_regions (struct virtual_disk *disk); + +void +init_virtual_disk (struct virtual_disk *disk) +{ + memset (disk, 0, sizeof *disk); + disk->fd = -1; + + init_regions (&disk->regions); +} + +int +create_virtual_disk (struct virtual_disk *disk) +{ + size_t i; + + /* Allocate the partition table structures. We can't fill them in + * until we have created the disk layout. + */ + disk->protective_mbr = calloc (1, SECTOR_SIZE); + disk->primary_header = calloc (1, SECTOR_SIZE); + disk->pt = calloc (1, 32*SECTOR_SIZE); + disk->secondary_header = calloc (1, SECTOR_SIZE); + if (disk->protective_mbr == NULL || + disk->primary_header == NULL || + disk->pt == NULL || + disk->secondary_header == NULL) { + nbdkit_error ("calloc: %m"); + return -1; + } + + /* Create the filesystem. This fills in disk->filesystem_size and + * disk->id. + */ + if (create_filesystem (disk) == -1) + return -1; + + /* Create a random GUID used as "Unique partition GUID". However + * this doesn't follow GUID conventions so in theory could make an + * invalid value. + */ + for (i = 0; i < 16; ++i) + disk->guid[i] = xrandom (&random_state) & 0xff; + + /* Create the virtual disk regions. */ + if (create_regions (disk) == -1) + return -1; + + /* Initialize partition table structures. This depends on + * disk->regions so must be done last. + */ + if (create_partition_table (disk) == -1) + return -1; + + return 0; +} + +void +free_virtual_disk (struct virtual_disk *disk) +{ + free_regions (&disk->regions); + free (disk->protective_mbr); + free (disk->primary_header); + free (disk->pt); + free (disk->secondary_header); + if (disk->fd >= 0) + close (disk->fd); +} + +/* Lay out the final disk. */ +static int +create_regions (struct virtual_disk *disk) +{ + /* Protective MBR. */ + if (append_region_len (&disk->regions, "Protective MBR", + SECTOR_SIZE, 0, 0, + region_data, (void *) disk->protective_mbr) == -1) + return -1; + + /* GPT primary partition table header (LBA 1). */ + if (append_region_len (&disk->regions, "GPT primary header", + SECTOR_SIZE, 0, 0, + region_data, (void *) disk->primary_header) == -1) + return -1; + + /* GPT primary PT (LBA 2..33). */ + if (append_region_len (&disk->regions, "GPT primary PT", + 32*SECTOR_SIZE, 0, 0, + region_data, (void *) disk->pt) == -1) + return -1; + + /* Partition containing the filesystem. Align it to 2048 sectors. */ + if (append_region_len (&disk->regions, "Filesystem", + disk->filesystem_size, 2048*SECTOR_SIZE, 0, + region_file, 0 /* unused */) == -1) + return -1; + + /* GPT secondary PT (LBA -33..-2). */ + if (append_region_len (&disk->regions, "GPT secondary PT", + 32*SECTOR_SIZE, SECTOR_SIZE, 0, + region_data, (void *) disk->pt) == -1) + return -1; + + /* GPT secondary PT header (LBA -1). */ + if (append_region_len (&disk->regions, "GPT secondary header", + SECTOR_SIZE, 0, 0, + region_data, (void *) disk->secondary_header) == -1) + return -1; + + return 0; +} diff --git a/plugins/linuxdisk/virtual-disk.h b/plugins/linuxdisk/virtual-disk.h new file mode 100644 index 0000000..2eee177 --- /dev/null +++ b/plugins/linuxdisk/virtual-disk.h @@ -0,0 +1,92 @@ +/* nbdkit + * Copyright (C) 2018-2019 Red Hat Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * * Neither the name of Red Hat nor the names of its contributors may be + * used to endorse or promote products derived from this software without + * specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, + * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A + * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR + * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF + * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND + * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, + * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT + * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#ifndef NBDKIT_VIRTUAL_DISK_H +#define NBDKIT_VIRTUAL_DISK_H + +#include <stdbool.h> +#include <stdint.h> + +#include "regions.h" + +extern char *dir; +extern const char *label; +extern const char *type; +extern int64_t size; +extern bool size_add_estimate; + +extern struct random_state random_state; + +#define SECTOR_SIZE 512 + +struct virtual_disk { + /* Virtual disk layout. */ + struct regions regions; + + /* Disk protective MBR. */ + uint8_t *protective_mbr; + + /* GPT primary partition table header. */ + uint8_t *primary_header; + + /* GPT primary and secondary (backup) PTs. These are the same. */ + uint8_t *pt; + + /* GPT secondary (backup) PT header. */ + uint8_t *secondary_header; + + /* Size of the filesystem in bytes. */ + uint64_t filesystem_size; + + /* Unique partition GUID. */ + char guid[16]; + + /* File descriptor of the temporary file containing the filesystem. */ + int fd; +}; + +/* virtual-disk.c */ +extern void init_virtual_disk (struct virtual_disk *disk) + __attribute__((__nonnull__ (1))); +extern int create_virtual_disk (struct virtual_disk *disk) + __attribute__((__nonnull__ (1))); +extern void free_virtual_disk (struct virtual_disk *disk) + __attribute__((__nonnull__ (1))); + +/* partition-gpt.c */ +extern int create_partition_table (struct virtual_disk *disk); + +/* filesystem.c */ +extern int create_filesystem (struct virtual_disk *disk); + +#endif /* NBDKIT_VIRTUAL_DISK_H */ diff --git a/plugins/partitioning/nbdkit-partitioning-plugin.pod b/plugins/partitioning/nbdkit-partitioning-plugin.pod index f3d6996..d045e4c 100644 --- a/plugins/partitioning/nbdkit-partitioning-plugin.pod +++ b/plugins/partitioning/nbdkit-partitioning-plugin.pod @@ -19,8 +19,8 @@ If you just want to concatenate files together (without adding a partition table) use L<nbdkit-split-plugin(1)>. If you want to select a single partition from an existing disk, use L<nbdkit-partition-filter(1)>. If you want to create a complete disk -with a filesystem, look at L<nbdkit-floppy-plugin(1)> or -L<nbdkit-iso-plugin(1)>. +with a filesystem, look at L<nbdkit-floppy-plugin(1)>, +L<nbdkit-iso-plugin(1)> or L<nbdkit-linuxdisk-plugin(1)>. The plugin supports read/write access. To limit clients to read-only access use the I<-r> flag. @@ -172,6 +172,7 @@ L<nbdkit(1)>, L<nbdkit-file-plugin(1)>, L<nbdkit-floppy-plugin(1)>, L<nbdkit-iso-plugin(1)>, +L<nbdkit-linuxdisk-plugin(1)>, L<nbdkit-partition-filter(1)>, L<nbdkit-split-plugin(1)>, L<nbdkit-plugin(3)>. diff --git a/tests/Makefile.am b/tests/Makefile.am index c75a9de..3992d9b 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -71,6 +71,8 @@ EXTRA_DIST = \ test-ip.sh \ test-iso.sh \ test-layers.sh \ + test-linuxdisk.sh \ + test-linuxdisk-copy-out.sh \ test-log.sh \ test.lua \ test-memory-largest.sh \ @@ -447,6 +449,13 @@ TESTS += test-iso.sh endif HAVE_GUESTFISH endif HAVE_ISO +# linuxdisk plugin test. +if HAVE_GUESTFISH +TESTS += \ + test-linuxdisk.sh \ + test-linuxdisk-copy-out.sh +endif HAVE_GUESTFISH + # memory plugin test. LIBGUESTFS_TESTS += test-memory TESTS += test-memory-largest.sh test-memory-largest-for-qemu.sh diff --git a/tests/test-linuxdisk-copy-out.sh b/tests/test-linuxdisk-copy-out.sh new file mode 100755 index 0000000..55062d0 --- /dev/null +++ b/tests/test-linuxdisk-copy-out.sh @@ -0,0 +1,76 @@ +#!/usr/bin/env bash +# nbdkit +# Copyright (C) 2018-2019 Red Hat Inc. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are +# met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# * Neither the name of Red Hat nor the names of its contributors may be +# used to endorse or promote products derived from this software without +# specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, +# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A +# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR +# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF +# USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND +# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT +# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# SUCH DAMAGE. + +# Test the linuxdisk plugin with captive nbdkit, as described +# in the man page. + +source ./functions.sh +set -e +set -x + +requires qemu-img --version + +files="linuxdisk-copy-out.img + linuxdisk-copy-out.test1 linuxdisk-copy-out.test2 + linuxdisk-copy-out.test3 linuxdisk-copy-out.test4" +rm -f $files +cleanup_fn rm -f $files + +nbdkit -f -v -U - \ + --filter=partition \ + linuxdisk $srcdir/../plugins partition=1 label=ROOT \ + --run 'qemu-img convert $nbd linuxdisk-copy-out.img' + +# Check the disk content. +guestfish --ro -a linuxdisk-copy-out.img -m /dev/sda <<EOF +# Check some known files and directories exist. + ll / + ll /linuxdisk + is-dir /linuxdisk + is-file /linuxdisk/Makefile.am + +# This reads out all the directory entries and all file contents. + tar-out / - | cat >/dev/null + +# Download some files and compare to local copies. + download /linuxdisk/Makefile linuxdisk-copy-out.test1 + download /linuxdisk/Makefile.am linuxdisk-copy-out.test2 + download /linuxdisk/nbdkit-linuxdisk-plugin.pod linuxdisk-copy-out.test3 + download /linuxdisk/filesystem.c linuxdisk-copy-out.test4 +EOF + +# Compare downloaded files to local versions. +cmp linuxdisk-copy-out.test1 $srcdir/../plugins/linuxdisk/Makefile +cmp linuxdisk-copy-out.test2 $srcdir/../plugins/linuxdisk/Makefile.am +cmp linuxdisk-copy-out.test3 $srcdir/../plugins/linuxdisk/nbdkit-linuxdisk-plugin.pod +cmp linuxdisk-copy-out.test4 $srcdir/../plugins/linuxdisk/filesystem.c diff --git a/tests/test-linuxdisk.sh b/tests/test-linuxdisk.sh new file mode 100755 index 0000000..6105c96 --- /dev/null +++ b/tests/test-linuxdisk.sh @@ -0,0 +1,93 @@ +#!/usr/bin/env bash +# nbdkit +# Copyright (C) 2018-2019 Red Hat Inc. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are +# met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# * Neither the name of Red Hat nor the names of its contributors may be +# used to endorse or promote products derived from this software without +# specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, +# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A +# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR +# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF +# USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND +# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT +# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# SUCH DAMAGE. + +# Test the linuxdisk plugin. + +source ./functions.sh +set -e +set -x + +requires mkfifo --version + +d=linuxdisk.d +rm -rf $d +cleanup_fn rm -rf $d + +# Create a test directory with some regular files, subdirectories and +# special files. +mkdir $d +mkfifo $d/fifo +mkdir $d/sub +cp $srcdir/Makefile.am $d/sub/Makefile.am +ln $d/sub/Makefile.am $d/sub/hardlink +ln -s $d/sub/Makefile.am $d/sub/symlink + +# It would be nice to use the Unix domain socket to test that the +# socket gets created, but in fact that won't work because this socket +# isn't created until after the plugin creates the virtual disk. +start_nbdkit -P $d/linuxdisk.pid \ + -U $d/linuxdisk.sock \ + linuxdisk $d + +# Check the disk content. +guestfish --ro --format=raw -a "nbd://?socket=$PWD/$d/linuxdisk.sock" -m /dev/sda1 <<EOF + ll / + ll /sub + +# Check regular files exist. + is-file /sub/Makefile.am + is-file /sub/hardlink +# XXX Test sparse files in future. + +# Check the specials exist. + is-fifo /fifo + is-symlink /sub/symlink +# XXX Test sockets, etc. in future. + +# Check hard linked files. + lstatns /sub/Makefile.am | cat > $d/nlink.1 + lstatns /sub/hardlink | cat > $d/nlink.2 + +# This reads out all the directory entries and all file contents. + tar-out / - | cat >/dev/null + +# Download file and compare to local copy. + download /sub/Makefile.am $d/Makefile.am +EOF + +# Check the two hard linked files have st_nlink == 2. +grep "st_nlink: 2" $d/nlink.1 +grep "st_nlink: 2" $d/nlink.2 + +# Compare downloaded file to local version. +cmp $d/Makefile.am $srcdir/Makefile.am -- 1.8.3.1