5:45 a.m.
I'm pleased to announce the releases of libnbd 1.2 and nbdkit 1.16.
These are a high performance Network Block Device (NBD) client library
and server.
Key features of libnbd:
* Synchronous API for ease of use.
* Asynchronous API for writing non-blocking, multithreaded clients.
You can mix both APIs freely.
* High performance.
* Minimal dependencies for the basic library.
* Well-documented, stable API.
* Bindings in several programming languages.
* Shell (nbdsh) for command line and scripting.
Git: https://github.com/libguestfs/libnbd
Download: http://download.libguestfs.org/libnbd/1.2-stable/
Fedora: https://koji.fedoraproject.org/koji/packageinfo?packageID=28807
Key features of nbdkit:
* Multithreaded NBD server written in C with good performance.
* Minimal dependencies for the basic server.
* Liberal license (BSD) allows nbdkit to be linked to proprietary
libraries or included in proprietary code.
* Well-documented, simple plugin API with a stable ABI guarantee.
Lets you export “unconventional” block devices easily.
* You can write plugins in C, Lua, Perl, Python, OCaml, Ruby, Rust,
shell script or Tcl.
* Filters can be stacked in front of plugins to transform the output.
Git: https://github.com/libguestfs/nbdkit
Download: http://download.libguestfs.org/nbdkit/1.16-stable/
Fedora: https://koji.fedoraproject.org/koji/packageinfo?packageID=16469
*** Release notes for libnbd 1.2 ***
These are the release notes for libnbd stable release 1.2. This
describes the major changes since 1.0.
libnbd 1.2.0 was released on 14th November 2019.
Security
Two security problems were found during development of libnbd 1.2.
Both were backported to the 1.0 stable branch. Upgrading is highly
advisable.
CVE-2019-14842 protocol downgrade attack when using
"LIBNBD_TLS_REQUIRE"
See the full announcement and links to mitigation, tests and fixes
here:
https://www.redhat.com/archives/libguestfs/2019-September/msg00128.html
remote code execution vulnerability
See the full announcement here:
https://www.redhat.com/archives/libguestfs/2019-October/msg00060.html
New APIs
nbd_can_fast_zero(3)
Test support by the server for fast zeroing (Eric Blake).
nbd_connect_socket(3)
nbd_aio_connect_socket(3)
Connect to a local connected socket which you create in your main
program using your own chosen method.
nbd_connect_systemd_socket_activation(3)
nbd_aio_connect_systemd_socket_activation(3)
Connect to local processes that support systemd socket activation.
nbd_connect_vsock(3)
nbd_aio_connect_vsock(3)
Used to connect to servers over "AF_VSOCK".
nbd_get_handshake_flags(3)
nbd_set_handshake_flags(3)
nbd_get_request_structured_replies(3)
nbd_set_request_structured_replies(3)
nbd_get_structured_replies_negotiated(3)
Can be used when testing NBD servers to avoid various NBD features
(Eric Blake).
nbd_get_protocol(3)
Get the NBD protocol variant that the server supports.
nbd_get_tls_negotiated(3)
Did we actually negotiate a TLS connection?
nbd_set_uri_allow_local_file(3)
nbd_set_uri_allow_tls(3)
nbd_set_uri_allow_transports(3)
These can be used to filter NBD URIs before calling
nbd_connect_uri(3).
New features
New tool nbdfuse(1) lets you create a loop-mounted file backed by an
NBD server without needing root.
"AF_VSOCK" is now a supported protocol (thanks Stefan Hajnoczi and
Stefano Garzarella).
Support for the "FAST_ZERO" flag (Eric Blake).
Allow disabling certain protocol features, to make it easier to test
servers (Eric Blake).
Stack-allocated Variable Length Arrays (VLAs) are now banned throughout
the library, making the library easier to consume from threads and
other small stack situations.
Reproducible builds (Chris Lamb).
Support for filtering potentially dangerous or undesirable NBD URI
features.
Documentation
Many improvements to the generated manual pages, including:
· Separate "RETURN VALUE" and "ERRORS" sections for each API
function.
· Example code.
· Relevant links can be added to the "SEE ALSO" section.
· Link to NBD URI specification where relevant, and improve
documentation around what URIs libnbd supports.
· Document libnbd version number scheme.
· Document limits on export name length, encoding etc.
New libnbd-security(3) man page listing past security issues and
remediations (Eric Blake).
Tools
nbdsh(1) has a new --base-allocation option which can be used to
request "base:allocation" metadata context.
New nbdsh(1) --uri (-u) option to connect to URIs.
Tests
You can now fuzz libnbd using either American Fuzzy Lop or clang’s
libFuzzer.
Add unit tests for nbdsh(1) (Eric Blake).
Improved interop testing with various NBD servers and features.
Other improvements and bug fixes
nbd_connect_tcp(3) now tries to return the correct errno(3) from the
underlying connect(2) call when that fails.
The nbd-protocol.h header file is now shared between libnbd and nbdkit.
Better fork-safety in "nbd_connect_*" APIs.
The code was analyzed with Coverity and various problems identified and
fixed.
*** Release notes for nbdkit 1.16 ***
These are the release notes for nbdkit stable release 1.16. This
describes the major changes since 1.14.
nbdkit 1.16.0 was released on 14th November 2019.
Security
Two security issues were found during development of nbdkit 1.16.
Fixes for these were backported to older stable branches. Upgrading to
the fixed versions is highly recommended. The new nbdkit-security(1)
man page contains an up to date list of past security issues.
CVE-2019-14850 denial of service due to premature opening of back-end
connection
See the full announcement and links to mitigation, tests and fixes
here:
https://www.redhat.com/archives/libguestfs/2019-September/msg00084.html
CVE-2019-14851 assertion failure by issuing commands in the wrong order
This CVE was caused by the fix to the previous issue.
See the full announcement and links to mitigation, tests and fixes
here:
https://www.redhat.com/archives/libguestfs/2019-September/msg00272.html
New features
Add support for fast zeroing. Plugins can expose this using the new
".can_fast_zero" method (Eric Blake).
nbdkit-partitioning-plugin(1) allows use of "mbr-id=default" or
"type-guid=default" to go back to the default MBR byte or partition
type GUID.
New --mask-handshake server flag can be used for testing client feature
negotiation (Eric Blake).
The client export name is passed to nbdkit-captive(1) --run parameter
as $exportname (Eric Blake).
Captive --run commands which fail (eg. aborting) now cause nbdkit to
exit with an error instead of errors being silently ignored (Eric
Blake).
File descriptors can be passed to password parameters, eg:
"password=-3" which means that the password should be read from file
descriptor 3.
nbdkit can now serve over the "AF_VSOCK" protocol (thanks Stefan
Hajnoczi).
New --log=null option discards error messages.
Plugins
Python 2 support has been dropped from nbdkit-python-plugin(3) in line
with Python 2 end of life at the beginning of 2020. Python ≥ 3.3 is
required by this plugin. If you wish to continue to use Python 2 then
you will need to use nbdkit 1.14.
New nbdkit-info-plugin(1) which returns various server information back
to the client. It can be used for testing server latency amongst other
things.
nbdkit-data-plugin(1) now allows you to write "BYTE*N" to get repeated
bytes (eg. nbdkit data data="0x55*4096").
nbdkit-ssh-plugin(1) new parameter "compression=true|false" to control
transport compression.
nbdkit-vddk-plugin(1) is no longer compiled on non-x86 platforms since
VMware has only ever shipped VDDK on x86.
nbdkit-sh-plugin(1) scripts can now see the client exportname and can
use the "magic_config_key" feature.
Filters
New nbdkit-retry-filter(1) which can reopen the plugin transparently on
certain types of failures (lots of help from Eric Blake).
API
Macros "NBDKIT_VERSION_MAJOR", "NBDKIT_VERSION_MINOR",
"NBDKIT_VERSION_MICRO" expose the compile-time version of nbdkit to
plugins and filters (Eric Blake).
Filters (which unlike plugins do not have a public stable API) must now
exactly match the version of nbdkit when loaded (Eric Blake).
New ".can_fast_zero" method (Eric Blake).
New "nbdkit_export_name" server function for reading the export name
passed by the client.
New "nbdkit_peer_name" server function to return the client address
(like getpeername(2)).
New server functions for safely parsing integers: "nbdkit_parse_int",
"nbdkit_parse_unsigned", "nbdkit_parse_int8_t",
"nbdkit_parse_uint8_t",
"nbdkit_parse_int16_t", "nbdkit_parse_uint16_t",
"nbdkit_parse_int32_t", "nbdkit_parse_uint32_t",
"nbdkit_parse_int64_t", "nbdkit_parse_uint64_t".
Bug fixes
".trim" with FUA flag set now works (Eric Blake).
Documentation
The previous release notes have been turned into man pages.
Tests
Several tests now optionally use nbdsh(1) instead of qemu-io.
You can now fuzz nbdkit using either American Fuzzy Lop or clang’s
libFuzzer.
Several tests have had sleep times increased to make them more stable
when run on slow or heavily loaded machines.
Internals
Reproducible builds (Chris Lamb).
Compile code with -Wshadow warning (Eric Blake).
The internal backend system has been extensively overhauled. In
particular this means that we now validate request ranges as requests
are passed between filters and down to the plugin, making it easier to
find bugs in filters early (Eric Blake).
Plugin size and "can_*" flags are cached more aggressively by the
server (Eric Blake).
Variable Length Arrays (VLAs) on stack are now banned throughout the
code.
The nbd-protocol.h header describing the NBD protocol is now shared
with libnbd(3).
Plugin ".unload" method is now called after all worker threads have
exited, avoiding races at server shutdown.
Code was audited using Coverity and various problems were fixed.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines. Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v