OK I think I have a way to solve this. However we need to post the writes to another thread, not the reads. The reason is that writes can be spooled up almost indefinitely so never block, whereas we may block on reads if the server doesn't respond fast enough. The actual change is very simple - I'll write a patch up later. The caller calls a new API such as: int nbd_set_concurrent_writer (void *data, void (*writer) (void *data, void *buf, size_t len)); ‘struct socket’ is modified here so that if a concurrent writer callback is set in the handle it calls the writer function instead of send(2): https://github.com/libguestfs/libnbd/blob/9d05f54f4daa892a1e581e1d68bc2dd4da... There also has to be an error notification function that the writer can call if it encounters an error (which are essentially unrecoverable - it just moves the state to DEAD). This pushes the implementation complexity to the caller, but I think that is necessary and justified: - libraries shouldn't be creating threads - the caller has a choice of how to implement this, eg. using one writer thread over all handles, or one thread per handle - the caller can do clever stuff like pinning the writer thread to a physical core or vectoring interrupts, which would be almost impossible if the thread was hidden inside the library By including an example it should be possible to get over some of the complexity. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW