Answering a few questions: * Unclear why it's necessary to rename nbdkit-common.h -> nbdkit-compat.h - It's not a rename, but a new file. nbdkit-compat.h is included for PE DLL compatibility, and is included in nbdkit-common.h * Functions are indirected through struct nbdkit_functions _nbdkit_functions which I guess is necessary because of PE linking (we've had similar problems with PE linking restrictions in other programs). However I wasn't very clear how this actually works, and we cannot break existing Linux binaries, so this needs more thought and must be conditional on Windows only. - There are macro checks for an windows environment. If it's windows, then indirection is used. Otherwise, the original extern definitions are used. This shouldn't cause an ABI change on Linux. * Is there no way to get the socket peer on Windows? - It's not a lack of ways that prevents me from implementing this. The nbdkit port is to MSYS2 (which is a superset of mingw-64), and this is needed because nbdkit uses an extensive set of POSIX features that are not present in mingw-64, e.g. fork). This port requires MSYS2 to run. However, the user might choose to implement the plugins / filters using mingw-64, or just MSVC runtime, and I don't want to bind users to MSYS2. Now this causes a potential ABI incompatibility: MSYS2 socket definitions are provided by sys/socket.h, but mingw-64 / MSVC socket definitions are usually provided by winsock.h or winsock2.h. To prevent possible issues, I removed nbdkit_peer_name. This function is not defined in the include headers if Windows is detected, so it will cause a compile time error if someone attempts to use it. On Sat, Mar 21, 2020 at 1:20 PM Richard W.M. Jones <rjones(a)redhat.com&gt; wrote:

The rebased version is at https://github.com/gyf304/nbdkit/commits/msys2-port It is currently buildable with: autoreconf -i && ./configure host=x86_64-pc-msys && make -j A few changes are added: * nbdkit_peer_name is added back but will always error. * __declspec(dllexport) is added to the headers for PE compatibility. * -D_spawnv=spawnv is now removed. I think I didn't run ./configure properly last time. Here is the list of features that are tested to compile. Optional server features: bash-completion ........................ yes manual pages ........................... yes SELinux ................................ no TLS .................................... no Optional plugins: curl ................................... no example4 ............................... yes ext2 ................................... no floppy ................................. no guestfs ................................ no gzip ................................... no iso .................................... no libvirt ................................ no nbd .................................... no ssh .................................... no tar .................................... yes vddk ................................... yes Languages: lua .................................... no ocaml .................................. no perl ................................... yes python ................................. yes ruby ................................... no rust ................................... no tcl .................................... no Optional filters: ext2 ................................... no xz ..................................... no On Sun, Mar 22, 2020 at 10:28 AM Richard W.M. Jones <rjones(a)redhat.com&gt; wrote:

On Tue, Mar 24, 2020 at 01:43:52PM -0500, Eric Blake wrote: Isn't it because we rely on it, since our plugins need symbols that are undefined at link time such as nbdkit_*? Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v

I think that will break the ABI backward compatibility. On Tue, Mar 24, 2020 at 15:16 Eric Blake <eblake(a)redhat.com&gt; wrote:

On Tue, Mar 24, 2020 at 02:16:32PM -0500, Eric Blake wrote: (For non-mingw platforms) this breaks the source API promises rather seriously, so if I understand your proposal correctly I don't think this is a good idea. It's possibly something we can consider for internal plugins, or for the V3 API. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org

I see. That makes sense.

I think I understand now that libnbdkit.so won't break the ABI for existing plugins. Does it require that plugins for newer nbdkit use -lnbdkit (which would be a source API break) or would it still be possible to compile without this? I guess as long as plugins do not start using -no-undefined then it would still work, so it wouldn't be a source API break. I had a look into how we might implement libnbdkit.so. Some functions are obviously self-contained (eg. nbdkit_parse_*, nbdkit_realpath, nbdkit_debug, nbdkit_error, nbdkit_*extents). Unfortunately some functions depend themselves on internals of the server: * nbdkit_nanosleep, nbdkit_export_name, nbdkit_peer_name call threadlocal_get_conn * nbdkit_set_error calls threadlocal_set_error * nbdkit_shutdown must set the quit global (or call a server function) I guess we can deal with the first ones by moving threadlocal.c into the same library, although it's a bit awkward. The quit flag is still more awkward because you have to move a lot of quit pipe handling code into the library which has knock-on effects all over. Rich. ---------------------------------------------------------------------- server/debug.c:nbdkit_vdebug (const char *fs, va_list args) server/debug.c:nbdkit_debug (const char *fs, ...) server/extents.c:nbdkit_extents_new (uint64_t start, uint64_t end) server/extents.c:nbdkit_extents_free (struct nbdkit_extents *exts) server/extents.c:nbdkit_extents_count (const struct nbdkit_extents *exts) server/extents.c:nbdkit_get_extent (const struct nbdkit_extents *exts, size_t i) server/extents.c:nbdkit_add_extent (struct nbdkit_extents *exts, server/log.c:nbdkit_verror (const char *fs, va_list args) server/log.c:nbdkit_error (const char *fs, ...) server/log.c:nbdkit_vfprintf(FILE *f, const char *fmt, va_list args) server/plugins.c:nbdkit_set_error (int err) server/public.c:nbdkit_absolute_path (const char *path) server/public.c:nbdkit_realpath (const char *path) server/public.c:nbdkit_parse_int (const char *what, const char *str, int *rp) server/public.c:nbdkit_parse_int8_t (const char *what, const char *str, int8_t *rp) server/public.c:nbdkit_parse_int16_t (const char *what, const char *str, int16_t *rp) server/public.c:nbdkit_parse_int32_t (const char *what, const char *str, int32_t *rp) server/public.c:nbdkit_parse_int64_t (const char *what, const char *str, int64_t *rp) server/public.c:nbdkit_parse_unsigned (const char *what, const char *str, unsigned *rp) server/public.c:nbdkit_parse_uint8_t (const char *what, const char *str, uint8_t *rp) server/public.c:nbdkit_parse_uint16_t (const char *what, const char *str, uint16_t *rp) server/public.c:nbdkit_parse_uint32_t (const char *what, const char *str, uint32_t *rp) server/public.c:nbdkit_parse_uint64_t (const char *what, const char *str, uint64_t *rp) server/public.c:nbdkit_parse_size (const char *str) server/public.c:nbdkit_parse_bool (const char *str) server/public.c:nbdkit_read_password (const char *value, char **password) server/public.c:nbdkit_nanosleep (unsigned sec, unsigned nsec) server/public.c:nbdkit_export_name (void) server/public.c:nbdkit_peer_name (struct sockaddr *addr, socklen_t *addrlen) server/quit.c:nbdkit_shutdown (void) -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top

On Wed, Mar 25, 2020 at 07:46:37AM -0500, Eric Blake wrote: Yes this is a good point. Also I suppose this interface between nbdkit <-> libnbdkit.so is *not* ABI so we can change this at will? It would mean that nbdkit & libnbdkit.so must always be shipped together, but that doesn't seem to be a problem. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/