[Libguestfs] Extending nbdkit to support listing exports

Hi Eric, Nir. $SUBJECT - it's complicated! Imagine a scenario where we have extended the file plugin so you can point it at a directory and it will use the client-supplied export name to select a file to serve. Also we have extended the tar filter so that you can map tar file components into export names. You may be able to see where this is going ... Now we point the file plugin at a directory of tar files, and layer the tar filter on top, and then what happens? The client can only supply a single export name, so how can it specify component X of tar file Y? tar filter ---> file plugin tar-exportname=on exportname=on directory=/my-files ---> directory of tar files It gets worse! At the moment filters and plugins can read the export name sent by the client by calling the global function nbdkit_export_name(). I'm not so worried about filters, but plugins must be able to continue to use this global function. In the scenario above both the tar filter and the file plugin would see the same string (if we did nothing), and it would only make sense to one or the other but never to both. Listing exports is also complicated: Should we list only the nearest filter's exports? Or the plugin's exports? Cartesian product of both somehow? That's the background, but how do we solve it? ---------------------------------------------- I'd like to say first of all that I believe export names are already a dumpster fire of potential insecurity, and that we shouldn't have the core server attempt to parse them, so solutions involving splitting the export names into components (like tar-component/file-name) should not be considered because they are too risky, and run into arbitrary limits (what escape character to use? what is the max length?) Also changing the NBD protocol is not under consideration. I think we need to consider these aspects separately: (a) How filters and plugins get the client exportname. (b) How filters and plugins respond when the client issues NBD_OPT_LIST. (c) How the server, filters and plugins respond to NBD_OPT_INFO. (a) Client exportname --------------------- The client sends the export name of the export it wants to access with NBD_OPT_EXPORT_NAME or the newer NBD_OPT_GO. This is an opaque string (not a filename, pathname etc). Currently filters and plugins can fetch this using nbdkit_export_name(). However it cannot be modified by filters. My proposal is that we extend the .open() callback with an extra export_name parameter: void *filter_open (int readonly, const char *export_name); For plugins I have already proposed this for inclusion in API V3. We already do this in nbdkit-sh-plugin. For backwards compatibility with existing plugins we'd make nbdkit_export_name() return the export name passed down by the last filter in the chain, and deprecate this function in V3. For filters the next_open field would take the export_name and this would allow filters to modify the export name. nbdkit-tar-filter would take an explicit tar-export-name=<new> parameter to pass a different export name down to the underlying plugin. If the tar filter was implementing its own export name processing then this parameter would either be required or would default to "". (If the tar filter was not implementing export name functionality it would pass it down unchanged). (b) Listing exports with NBD_OPT_LIST ------------------------------------- I believe we need a new .list_exports() callback for filters and plugins to handle this case. I'm not completely clear on the API, but it could be something as simple as: const char **list_exports (void); returning a NULL-terminated list of strings. Is it conceivable we will need additional fields in future? Plugins which do not implement this would be assumed to return a single export "" (or perhaps the -e parameter??) Filters could ignore or replace exports by optionally implementing this callback. I wouldn't recommend that filters modify export names however. (c) Handling NBD_OPT_INFO ------------------------- I believe our current implementation of this option (which is definitely not being robustly tested!) is something like this: - NBD_OPT_INFO received from client (containing the export name) - open a backend handle - query backend for flags - reply to client - close backend handle - process the next option Assuming this is actually correct, probably we can make sure that it passes the export name to the backend_open, and everything should just work. In fact this would happen as a side effect of fixing (a) above so I don't believe there's anything to do here except to have some test coverage. ---------------------------------------------------------------------- Thoughts welcomed as always, Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v