[Libguestfs] ANNOUNCE: nbdkit 1.28 and libnbd 1.10 released

nbdkit is a Network Block Device (NBD) server with stable plugin ABI and permissive license. libnbd is an NBD client library. I'm pleased to announce the latest stable releases of both projects: nbdkit 1.28.0 and libnbd 1.10.0. You can download both from the download directories here: https://download.libguestfs.org/nbdkit/ https://download.libguestfs.org/libnbd/ Release notes are online here and attached below: https://libguestfs.org/nbdkit-release-notes-1.28.1.html https://libguestfs.org/libnbd-release-notes-1.10.1.html Rich. ---------------------------------------------------------------------- These are the release notes for nbdkit stable release 1.28. This describes the major changes since 1.26. nbdkit 1.28.0 was released on 23 September 2021. Security CVE-2021-3716 reset structured replies on starttls nbdkit was vulnerable to injected plaintext when upgrading to a secure connection. For the full announcement see https://www.redhat.com/archives/libguestfs/2021-August/msg00083.html (Eric Blake). All past security issues and information about how to report new ones can be found in nbdkit-security(1). Plugins nbdkit-data-plugin(1) has new "le<NN>:" and "be<NN>:" prefixes for little and big endian words. Also the plugin tries much harder to optimize expressions. The test suite has been greatly expanded to catch potential regressions. nbdkit-floppy-plugin(1) now allows you to reserve free space (Nolan Leake). nbdkit-ssh-plugin(1) no longer references SHA1 host keys as part of the effort to remove insecure algorithms (Daniel Berrangé). nbdkit-vddk-plugin(1) reports "can_flush" and "can_fua" based on what the version of VDDK supports. New debug flag -D vddk.stats=1 prints the amount of time spent in VDDK calls on exit which can be useful for profiling performance. Filters nbdkit-cow-filter(1) and nbdkit-cache-filter(1) no longer break up large read requests into page-sized requests. In addition the default block size for both filters is now 64K. Both changes greatly improve performance. nbdkit-cache-filter(1) has a new "cache-on-read=/PATH" parameter allowing callers to enable and disable the cache-on-read feature at runtime. Also there is a new "cache-min-block-size" parameter letting you select the block size at runtime (thanks Martin Kletzander). nbdkit-cow-filter(1) has a new "cow-on-read" parameter which works similarly to the corresponding cache filter feature. The new "cow-block-size" parameter lets you select the block size at runtime. nbdkit-cow-filter(1) has less verbose debugging. To restore the old debug output use -D cow.verbose=1 nbdkit-delay-filter(1) has new "delay-open" and "delay-close" parameters to inject delays when clients connect and disconnect. Delay filter parameters are now parsed more accurately (thanks Ming Xie). Language bindings The OCaml bindings now call "caml_shutdown" when unloading the plugin. This causes "Stdlib.at_exit" handlers to run correctly, closes file descriptors, releases dependent shared libraries and frees memory. Valgrind on OCaml plugins should not show any false positives about leaked memory. OCaml and Python bindings may now use the ".cleanup" method. References to Python 2 in nbdkit-python-plugin(3) have been removed. Server Fix captive nbdkit $uri variable so when TLS is used the URI is constructed with the "nbds:" prefix. Additionally add a new variable $tls which can be used by the subprocess to tell if TLS is enabled. Debug messages are now printed atomically. This means that debug messages are no longer broken up across multiple lines if there are other processes writing to stderr at the same time (which often happened when using captive nbdkit). Enhanced valgrind support (./configure --enable-valgrind) can now be enabled safely and with no performance impact even in production builds (Eric Blake). API Plugins or filters using "nbdkit_nanosleep" now don't hang if the client closes the socket abruptly (thanks Ming Xie). Bug fixes nbdkit-data-plugin(1) and nbdkit-memory-plugin(1) using allocator=malloc no longer crash because of memory corruption in some corner cases (only seen on s390x, but could happen on other architectures). Meanwhile "allocator=zstd" no longer crashes when zeroing unallocated space. Tests Tests now use the new "GLIBC_TUNABLES" feature, replacing "MALLOC_CHECK_" on glibc ≥ 2.34 (thanks Eric Blake, Siddhesh Poyarekar). Build configure.ac now uses spaces consistently, and has been modernized to support the latest autotools (Eric Blake). podwrapper.pl has been unified (almost) with the copy in libnbd. Continue fuzzing using AFL++. Updated the fuzzing documentation. AUTHORS Authors of nbdkit 1.28: Daniel P. Berrangé Eric Blake Martin Kletzander Nolan Leake Richard W.M. Jones ---------------------------------------------------------------------- These are the release notes for libnbd stable release 1.10. This describes the major changes since 1.8. libnbd 1.10.0 was released on 23 September 2021. Security There were no security bugs found in libnbd during this release cycle. If you find a security issue, please read SECURITY in the source (online here: https://gitlab.com/nbdkit/libnbd/blob/master/SECURITY). To find out about previous security issues in libnbd, see libnbd-security(3). New APIs No new APIs were added in 1.10. Enhancements to existing APIs nbd_get_uri(3) no longer returns service names (eg. "nbd://localhost:nbd"). Instead it always returns raw port numbers for portability. nbd_connect_uri(3) now supports "tls-certificates=DIR" query parameter, making it much easier to connect to servers using TLS with X.509 certificates. Also error messages from this API have been improved in the case of some common URI user errors. Python "nbd.aio_connect" implements support for "AF_UNIX" sockets. Fix invalid use of "unsafe.Pointer" in Go bindings. Tools nbdcopy(1) now uses a default request size of 2M (instead of 32M). This default performs better in most cases. nbdinfo(1) has a new --map --totals mode which displays a summary of the map. Also new --can and --is options let you test export properties (eg. --is read-only). nbdinfo(1) --map option uses "data" instead of "allocated" because of ambiguity about what "allocated" means (Eric Blake, Nir Soffer). nbdinfo(1) shows the export size in both bytes and human units (like "1K"). The machine-parsable JSON output has not changed. nbdfuse(1) now supports efficient zeroing. Note this requires Linux kernel ≥ 5.14. nbdsh(1) has new option -n which avoids creating the implicit handle "h". Also new option -v which enables debugging. Also the initial help banner is now context sensitive giving more relevant information depending on how nbdsh was invoked. Tests CI tests were greatly enhanced and many platform-specific fixes were made. To view the latest CI tests and results see: https://gitlab.com/nbdkit/libnbd/-/pipelines (Martin Kletzander). Tests now use the new "GLIBC_TUNABLES" feature, replacing "MALLOC_CHECK_" on glibc ≥ 2.34 (thanks Eric Blake, Siddhesh Poyarekar). Other improvements and bug fixes The nbdcopy(1) progress bar should be displayed more accurately in multithreaded mode. Documentation nbd_connect(3) and nbd_aio_connect(3) documentation has been revised and improved. More consistent option styling is used throughout the documentation. podwrapper.pl has been unified (almost) with the copy in nbdkit. Build configure.ac now uses spaces consistently, and has been modernized to support the latest autotools (Eric Blake). We now warn about large stack frames, and a few places which used large stack frames have been fixed. Continue fuzzing using AFL++. Updated the fuzzing documentation. Fix building from git with --disable-ocaml. As long as only "ocamlc" is installed, the generator should still be built and run (Martin Kletzander). AUTHORS Anson Lo Eric Blake Martin Kletzander Richard W.M. Jones -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org