[Libguestfs] ANNOUNCE: nbdkit 1.30 and libnbd 1.12 released

nbdkit is a Network Block Device (NBD) server with stable plugin ABI and permissive license. libnbd is an NBD client library. I'm pleased to announce the latest stable releases of both projects: nbdkit 1.30.0 and libnbd 1.12.0. You can download both from the download directories here: https://download.libguestfs.org/nbdkit/ https://download.libguestfs.org/libnbd/ Release notes are online here and attached below: https://libguestfs.org/nbdkit-release-notes-1.30.1.html https://libguestfs.org/libnbd-release-notes-1.12.1.html Rich. ------------------------------ These are the release notes for nbdkit stable release 1.30. This describes the major changes since 1.28. nbdkit 1.30.0 was released on 24 February 2022. Security There were no security issues found. All past security issues and information about how to report new ones can be found in nbdkit-security(1). Plugins nbdkit-streaming-plugin has been removed. It was deprecated in 1.26 and scheduled for removal in this release. If you were using this plugin, use nbdcopy(1) instead. nbdkit-vddk-plugin(1) has several changes: • This plugin can now create (as well as reading and writing) local VMDK files. See the new "create=true" option and the various "create-*" options. • Read and write is now implemented using the VDDK Async functions, which improves performance. The full nbdkit parallel thread model is supported. • VDDK ≥ 6.5 (released Nov 2016) is now the minimum required version. • Stats collected when using -D vddk.stats=1 have been extended to include number of calls and bytes transferred. • --dump-plugin output now includes the VDDK major version and information about each VDDK API that is implemented by the library. • A new example scripts/vddk-open.sh is provided to help with automating connections to remote ESXi virtual machines. nbdkit-curl-plugin(1) adds support for choosing TLS/SSL versions and ciphers and TLS 1.3 cipher versions (Przemyslaw Czarnowski). nbdkit-file-plugin(1) now implements "cache=none" for writes so that the Linux page cache is not destroyed when writing huge files. nbdkit-cc-plugin(1) now implements the ".cleanup" callback. Also we document how to use this plugin to implement OCaml plugin scripts. nbdkit-info-plugin(1) --dump-plugin option now prints "info_address=yes" when the platform supports "mode=address". Filters New nbdkit-retry-request-filter(1), which is similar to nbdkit-retry-filter(1) but only retries a single failing request. New nbdkit-protect-filter(1) lets you add write-protection over regions of a plugin. New nbdkit-blocksize-policy-filter(1) lets you adjust or set the block size constraints and policy of underlying plugins. See "API" below. nbdkit-cow-filter(1) now permits the COW block size to be set as small as 4096 bytes. Debug messages in nbdkit-error-filter(1) are now easier to read because they no longer all have an "error:" prefix (Eric Blake). Language bindings For plugins written in OCaml the minimum version of OCaml is now 4.03 (instead of 4.02.2 previously). Various source-level incompatible changes were made to OCaml plugins in this release. Please consult the new documentation and example if writing plugins in OCaml. OCaml plugins now support "magic_config_key". Several fixes to the handling of integers in Python plugins (Nir Soffer). New Python example which simulates request errors (Nir Soffer). Server The server no longer requests the "AI_ADDRCONFIG" hint when opening a server socket. This improves handling of IPv6. In a related change, the -i (--ipaddr) option now works as intended, and new -4 and -6 options have been added to restrict the listening socket to IPv4 or IPv6 addresses (Laszlo Ersek). API There is a new ".block_size" callback for plugins and filters. nbdkit uses this to export block size constraints to clients, specifically the minimum, preferred and maximum block size that clients should use. As well as regular C plugins, nbdkit-cc-plugin(3), nbdkit-eval-plugin(1), nbdkit-ocaml-plugin(3), nbdkit-python-plugin(3) and nbdkit-sh-plugin(3) support block size constraints. nbdkit-nbd-plugin(1) reexports block size constraints from the proxied NBD server. nbdkit-vddk-plugin(1) now advertises a minimum 512 byte block size. nbdkit-blocksize-filter(1), nbdkit-cache-filter(1), nbdkit-cow-filter(1) and nbdkit-swab-filter(1) adjust block size constraints from plugins (Eric Blake). nbdkit-blocksize-filter(1) can also use the block size hints from the plugin instead of needing manual adjustment on the command line. nbdkit-log-filter(1) logs block size constraints (Eric Blake). nbdkit-blocksize-policy-filter(1) can be used to add block size constraints to plugins which don't support it, or adjust constraints, or set the error policy. Bug fixes nbdkit-memory-plugin(1) (and others), using "allocator=malloc,mlock=true" was broken. This was especially evident on non-Linux or on Linux with 64K pages (eg. POWER). It should now work correctly. Tests "./configure --disable-libguestfs-tests" can now be used to disable tests which need libguestfs, without disabling nbdkit-guestfs-plugin(1). We now use mke2fs(8) instead of guestfish(1) to create the ext4 test image. On armv7 this allows the test suite to run much more quickly. The time taken to run the test suite has been reduced significantly. Build Multiple fixes to the Windows (MinGW) build. The test suite should now pass fully if you have the version of Wine with the unofficial "AF_UNIX" patch. The top level bash source directory has been renamed to bash- completion. This prevents problems when setting $PATH to the source directory and trying to run bash. Internals The performance of the internal vector library has been improved greatly and overflow is now handled correctly (Nir Soffer, Laszlo Ersek and Eric Blake). New "nbdkit-string.h" header file which defines a common string type (based on vector). Existing places which defined a string based on vector have been updated to use it. "MIN" and "MAX" macros can be nested (thanks Eric Blake). SEE ALSO nbdkit(1). AUTHORS Authors of nbdkit 1.30: Alan Somers Eric Blake Hilko Bengen Laszlo Ersek Nir Soffer Przemyslaw Czarnowski Richard W.M. Jones ------------------------------ These are the release notes for libnbd stable release 1.12. This describes the major changes since 1.10. libnbd 1.12.0 was released on 24 February 2022. Security CVE-2022-0485 Silent data corruption when using nbdcopy(1). See the full announcement here: https://listman.redhat.com/archives/libguestfs/2022-February/msg00104.html If you find a security issue, please read SECURITY in the source (online here: https://gitlab.com/nbdkit/libnbd/blob/master/SECURITY). To find out about previous security issues in libnbd, see libnbd-security(3). New APIs get_pread_initialize set_pread_initialize Control whether libnbd clears the pread buffer to avoid leaking memory contents if the client does not properly handle errors. These were added as part of the fix for CVE-2022-0485 (Eric Blake). get_request_block_size set_request_block_size Control whether libnbd requests block size constraints from the server during negotiation (Eric Blake). Enhancements to existing APIs Error messages about incorrect URIs in nbd_connect_uri(3) have been improved to make common mistakes clearer for the user. Tools New syntax: "nbdinfo [ CMD ... ]" allowing you to query the properties of an NBD server directly. nbdcopy(1) new --queue-size option to control the maximum size of the internal buffer. nbdcopy(1) now names the source and destination handles to make it easier to understand debug output. Tests Adopt libvirt-ci's ci/gitlab.yml (Martin Kletzander). Fix the OCaml extents example (thanks Laszlo Ersek). Golang benchmarks were added to the test suite (Nir Soffer). The large tests/errors.c test was split into about a dozen separate tests. Also this adds a new test for checking server-side block size constraint error policy which was missing before. Other improvements and bug fixes New OCaml "NBD.with_handle" helper which ensures that "NBD.close" is always called even if the inner code throws an exception. The OCaml bindings now consistently use the correct types for buffer sizes everywhere (Laszlo Ersek). Several improvements and fixes to the golang bindings and tests. The golang bindings have been relicensed to LGPLv2+ like the rest of the library and are now published as a golang module at https://libguestfs.org/libnbd (Nir Soffer). The Python bindings no longer crash if you pass "None" in place of a buffer parameter. In addition some memory leaks were fixed (Eric Blake). Various memory leaks have been fixed when using the optional strict_mode settings (Eric Blake). The performance of the internal vector library has been improved greatly and overflow is now handled correctly (Nir Soffer, Laszlo Ersek and Eric Blake). Add "simple_copy" and "aio_copy" Golang examples (Nir Soffer). Error handling was reviewed and fixed across many of the example programs and documentation (Eric Blake, Nir Soffer). Simplify and optimize handling of the extents callback in Golang (Nir Soffer). Golang AioBuffer was enhanced, making it safer to use, faster and adding documentation (Nir Soffer). Documentation Document the limits on lengths and sizes of parameters to various APIs (Eric Blake). Build You can now build programs that use the OCaml bindings of libnbd against the libnbd build directory instead of requiring libnbd to be installed (Laszlo Ersek). Compatibility with OCaml 4.13. Line locations in the state machine should be more accurate when stepping through with a debugger like gdb. .editorconfig file can be used to help code formatting, see https://editorconfig.org/ (Nir Soffer, Eric Blake). "MIN" and "MAX" macros can be nested (thanks Eric Blake). SEE ALSO libnbd(3). AUTHORS Eric Blake Laszlo Ersek Martin Kletzander Nir Soffer Richard W.M. Jones -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v