On 11/22/19 3:07 PM, Richard W.M. Jones wrote:
On Fri, Nov 22, 2019 at 02:55:31PM -0600, Eric Blake wrote:
Unrelated side topic: in your recent addition of eval.sh, you wondered if we should promote it to a full-blown plugin rather than just an example script. But reading 'man nbdkit-sh-plugin', there is no mention of turning an executable script into a full-blown plugin via a shebang, the way that python documents it. [I guess 'man nbdkit' sort of mentions it under Shebang scripts]
I believe it's not possible to do it for sh plugins.
For (eg) python plugins it works like this:
#!/usr/sbin/nbdkit python -> runs nbdkit python <script name> -> <script name> is interpreted as magic script= parameter -> the python plugin works by loading the script using PyRun_SimpleFileEx which interprets the contents of the script as python code
However for shell it doesn't work:
#!/usr/sbin/nbdkit sh -> runs nbdkit sh <script name> -> <script name> is interpreted as magic script= parameter -> the sh plugin works by actually executing the script -> executing the script repeats the steps over from the top, causing an infinite loop
If you can think of a way to make this work it would be a useful feature IMO.
Hmm. Right now, we document that: nbdkit sh file requires 'file' to be executable, and that we call exec*("file", { "file", "op", ... }) and that the interpreter need not be sh (file can start with any #! to run a script that will be executed by another interpreter). In order to get a shebang redirect to work, though, it seems like we must NOT directly re-execute the file, but instead force the file to be parsed by "/bin/sh" (or have some other means of identifying exactly which interpreter to use, outside of the normal #! means), where we would invoke exec*("/bin/sh", {"file", "op", ...}) and thereby skip the kernel's attempt to re-execute under a different interpreter. Perhaps we could get this by teaching the sh plugin to read() the first line of the script; if it starts with #! and contains 'nbdkit sh', then we force exec through "sh" (hmm, is there a way to force exec through bash instead?). Otherwise, we exec through the file-name as-is. Then our sh plugin would support two modes: the existing mode (we exec any interpreter by letting the kernel interpret the #! line, but you can't get shebang forwarding over to nbdkit), and the new mode (we exec a shell directly to bypass kernel shebang interpretation, but now you can write a shell script that gets shebang forwarding over to nbdkit). It's also a bummer that different platforms vary on how #! lines are parsed - you can't portably pass more than a single argument, so there is no portable way to write '#!/path/to/nbdkit sh shell=/bin/bash' as a convenient way to choose which shell to use when exec'ing the script while bypassing the #!, short of having nbdkit reimplement what GNU coreutils recently added as '/bin/env -S' (and that reimplementation would have to be in nbdkit proper, not in the sh plugin, because such a shebang line would result in 'nbdkit' 'sh shell=/bin/bash' 'file' 'args', but there is no '/path/to/nbdkit-plugin-sh shell=/bin/bash.so' plugin to be loaded). If we choose to make the sh plugin smart enough to parse line 1 to see if it is a shebang containing the substring 'nbdkit sh', we could also make it parse line 2 to pick up some magic comment line of 'SHELL=/bin/bash' as the binary to exec. That's a bit more complicated to document, but might serve to let us use the sh plugin for the two separate modes documented above (mode 1 to run an arbitrary executable file which may use shebang to call out its own interpreter, mode 2 to run an arbitrary file with a fixed interpreter allowing that file to start with a shebang to call out to nbdkit). Or maybe we create two different plugins (keep the existing 'sh' plugin with existing semantics, and add a new 'shell' plugin that shares 99% of the code but forces execution with a fixed shell). The other thing to consider is what brought this topic up - a question of whether plugins/sh/eval.sh is worth a conversion into a standalone plugin. With other languages, our choice is between: nbdkit python script ./script (where #! reinvokes nbdkit python script) But we don't have a way to write: nbdkit script where nbdkit recognizes that 'script' is not a .so file, but IS a file that calls out python in its #!, and therefore attempts to load the python.so plugin. The question with the eval.sh plugin is if we can write: nbdkit eval pread=... and have nbdkit figure out that 'eval' is not a plugin but IS a file with a #! calling out 'nbdkit sh', and therefore load the 'sh' plugin with 'script=eval'. In other words, maybe it's time to teach nbdkit general magic on how to handle the plugin argument (if an .so is found then load it; if not, then inspect the file to see which .so to use instead). Or maybe we just write a C form of a plugin named 'eval' that does the same things as what 'sh eval.sh' does, sharing as much code as possible, but where you have no shebang form because you never pass a script= argument to the eval plugin. All interesting thoughts, but I'm not sure I got us any closer to a nice conclusion. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org