I'm pleased to announce the releases of libnbd 1.2 and nbdkit 1.16. These are a high performance Network Block Device (NBD) client library and server. Key features of libnbd: * Synchronous API for ease of use. * Asynchronous API for writing non-blocking, multithreaded clients. You can mix both APIs freely. * High performance. * Minimal dependencies for the basic library. * Well-documented, stable API. * Bindings in several programming languages. * Shell (nbdsh) for command line and scripting. Git: https://github.com/libguestfs/libnbd Download: http://download.libguestfs.org/libnbd/1.2-stable/ Fedora: https://koji.fedoraproject.org/koji/packageinfo?packageID=28807 Key features of nbdkit: * Multithreaded NBD server written in C with good performance. * Minimal dependencies for the basic server. * Liberal license (BSD) allows nbdkit to be linked to proprietary libraries or included in proprietary code. * Well-documented, simple plugin API with a stable ABI guarantee. Lets you export “unconventional” block devices easily. * You can write plugins in C, Lua, Perl, Python, OCaml, Ruby, Rust, shell script or Tcl. * Filters can be stacked in front of plugins to transform the output. Git: https://github.com/libguestfs/nbdkit Download: http://download.libguestfs.org/nbdkit/1.16-stable/ Fedora: https://koji.fedoraproject.org/koji/packageinfo?packageID=16469 *** Release notes for libnbd 1.2 *** These are the release notes for libnbd stable release 1.2. This describes the major changes since 1.0. libnbd 1.2.0 was released on 14th November 2019. Security Two security problems were found during development of libnbd 1.2. Both were backported to the 1.0 stable branch. Upgrading is highly advisable. CVE-2019-14842 protocol downgrade attack when using "LIBNBD_TLS_REQUIRE" See the full announcement and links to mitigation, tests and fixes here: https://www.redhat.com/archives/libguestfs/2019-September/msg00128.html remote code execution vulnerability See the full announcement here: https://www.redhat.com/archives/libguestfs/2019-October/msg00060.html New APIs nbd_can_fast_zero(3) Test support by the server for fast zeroing (Eric Blake). nbd_connect_socket(3) nbd_aio_connect_socket(3) Connect to a local connected socket which you create in your main program using your own chosen method. nbd_connect_systemd_socket_activation(3) nbd_aio_connect_systemd_socket_activation(3) Connect to local processes that support systemd socket activation. nbd_connect_vsock(3) nbd_aio_connect_vsock(3) Used to connect to servers over "AF_VSOCK". nbd_get_handshake_flags(3) nbd_set_handshake_flags(3) nbd_get_request_structured_replies(3) nbd_set_request_structured_replies(3) nbd_get_structured_replies_negotiated(3) Can be used when testing NBD servers to avoid various NBD features (Eric Blake). nbd_get_protocol(3) Get the NBD protocol variant that the server supports. nbd_get_tls_negotiated(3) Did we actually negotiate a TLS connection? nbd_set_uri_allow_local_file(3) nbd_set_uri_allow_tls(3) nbd_set_uri_allow_transports(3) These can be used to filter NBD URIs before calling nbd_connect_uri(3). New features New tool nbdfuse(1) lets you create a loop-mounted file backed by an NBD server without needing root. "AF_VSOCK" is now a supported protocol (thanks Stefan Hajnoczi and Stefano Garzarella). Support for the "FAST_ZERO" flag (Eric Blake). Allow disabling certain protocol features, to make it easier to test servers (Eric Blake). Stack-allocated Variable Length Arrays (VLAs) are now banned throughout the library, making the library easier to consume from threads and other small stack situations. Reproducible builds (Chris Lamb). Support for filtering potentially dangerous or undesirable NBD URI features. Documentation Many improvements to the generated manual pages, including: · Separate "RETURN VALUE" and "ERRORS" sections for each API function. · Example code. · Relevant links can be added to the "SEE ALSO" section. · Link to NBD URI specification where relevant, and improve documentation around what URIs libnbd supports. · Document libnbd version number scheme. · Document limits on export name length, encoding etc. New libnbd-security(3) man page listing past security issues and remediations (Eric Blake). Tools nbdsh(1) has a new --base-allocation option which can be used to request "base:allocation" metadata context. New nbdsh(1) --uri (-u) option to connect to URIs. Tests You can now fuzz libnbd using either American Fuzzy Lop or clang’s libFuzzer. Add unit tests for nbdsh(1) (Eric Blake). Improved interop testing with various NBD servers and features. Other improvements and bug fixes nbd_connect_tcp(3) now tries to return the correct errno(3) from the underlying connect(2) call when that fails. The nbd-protocol.h header file is now shared between libnbd and nbdkit. Better fork-safety in "nbd_connect_*" APIs. The code was analyzed with Coverity and various problems identified and fixed. *** Release notes for nbdkit 1.16 *** These are the release notes for nbdkit stable release 1.16. This describes the major changes since 1.14. nbdkit 1.16.0 was released on 14th November 2019. Security Two security issues were found during development of nbdkit 1.16. Fixes for these were backported to older stable branches. Upgrading to the fixed versions is highly recommended. The new nbdkit-security(1) man page contains an up to date list of past security issues. CVE-2019-14850 denial of service due to premature opening of back-end connection See the full announcement and links to mitigation, tests and fixes here: https://www.redhat.com/archives/libguestfs/2019-September/msg00084.html CVE-2019-14851 assertion failure by issuing commands in the wrong order This CVE was caused by the fix to the previous issue. See the full announcement and links to mitigation, tests and fixes here: https://www.redhat.com/archives/libguestfs/2019-September/msg00272.html New features Add support for fast zeroing. Plugins can expose this using the new ".can_fast_zero" method (Eric Blake). nbdkit-partitioning-plugin(1) allows use of "mbr-id=default" or "type-guid=default" to go back to the default MBR byte or partition type GUID. New --mask-handshake server flag can be used for testing client feature negotiation (Eric Blake). The client export name is passed to nbdkit-captive(1) --run parameter as $exportname (Eric Blake). Captive --run commands which fail (eg. aborting) now cause nbdkit to exit with an error instead of errors being silently ignored (Eric Blake). File descriptors can be passed to password parameters, eg: "password=-3" which means that the password should be read from file descriptor 3. nbdkit can now serve over the "AF_VSOCK" protocol (thanks Stefan Hajnoczi). New --log=null option discards error messages. Plugins Python 2 support has been dropped from nbdkit-python-plugin(3) in line with Python 2 end of life at the beginning of 2020. Python ≥ 3.3 is required by this plugin. If you wish to continue to use Python 2 then you will need to use nbdkit 1.14. New nbdkit-info-plugin(1) which returns various server information back to the client. It can be used for testing server latency amongst other things. nbdkit-data-plugin(1) now allows you to write "BYTE*N" to get repeated bytes (eg. nbdkit data data="0x55*4096"). nbdkit-ssh-plugin(1) new parameter "compression=true|false" to control transport compression. nbdkit-vddk-plugin(1) is no longer compiled on non-x86 platforms since VMware has only ever shipped VDDK on x86. nbdkit-sh-plugin(1) scripts can now see the client exportname and can use the "magic_config_key" feature. Filters New nbdkit-retry-filter(1) which can reopen the plugin transparently on certain types of failures (lots of help from Eric Blake). API Macros "NBDKIT_VERSION_MAJOR", "NBDKIT_VERSION_MINOR", "NBDKIT_VERSION_MICRO" expose the compile-time version of nbdkit to plugins and filters (Eric Blake). Filters (which unlike plugins do not have a public stable API) must now exactly match the version of nbdkit when loaded (Eric Blake). New ".can_fast_zero" method (Eric Blake). New "nbdkit_export_name" server function for reading the export name passed by the client. New "nbdkit_peer_name" server function to return the client address (like getpeername(2)). New server functions for safely parsing integers: "nbdkit_parse_int", "nbdkit_parse_unsigned", "nbdkit_parse_int8_t", "nbdkit_parse_uint8_t", "nbdkit_parse_int16_t", "nbdkit_parse_uint16_t", "nbdkit_parse_int32_t", "nbdkit_parse_uint32_t", "nbdkit_parse_int64_t", "nbdkit_parse_uint64_t". Bug fixes ".trim" with FUA flag set now works (Eric Blake). Documentation The previous release notes have been turned into man pages. Tests Several tests now optionally use nbdsh(1) instead of qemu-io. You can now fuzz nbdkit using either American Fuzzy Lop or clang’s libFuzzer. Several tests have had sleep times increased to make them more stable when run on slow or heavily loaded machines. Internals Reproducible builds (Chris Lamb). Compile code with -Wshadow warning (Eric Blake). The internal backend system has been extensively overhauled. In particular this means that we now validate request ranges as requests are passed between filters and down to the plugin, making it easier to find bugs in filters early (Eric Blake). Plugin size and "can_*" flags are cached more aggressively by the server (Eric Blake). Variable Length Arrays (VLAs) on stack are now banned throughout the code. The nbd-protocol.h header describing the NBD protocol is now shared with libnbd(3). Plugin ".unload" method is now called after all worker threads have exited, avoiding races at server shutdown. Code was audited using Coverity and various problems were fixed. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v _______________________________________________ Libguestfs mailing list Libguestfs(a)redhat.com https://www.redhat.com/mailman/listinfo/libguestfs