On Mon, Oct 09, 2023 at 01:32:07PM +0200, Laszlo Ersek wrote:
On 10/9/23 13:12, Richard W.M. Jones wrote:
On Mon, Oct 09, 2023 at 12:57:14PM +0200, Laszlo Ersek wrote:
On 10/9/23 09:46, Richard W.M. Jones wrote:
Hi Eric, a couple of POSIX questions for you from nbdkit.
The first question is from an AUR comment on nbdkit:
https://aur.archlinux.org/packages/nbdkit#comment-937381
I think there's a bash-ism in the logscript parameter in this test:
https://gitlab.com/nbdkit/nbdkit/-/blame/master/tests/test-log-script-info.s...
I believe it is happening in the $(( .. )) expression. How do we write that so it'll work in a posix shell?
(I'm not Eric, but curious! :) )
Here I think we should just explicitly insist on bash.
Unfortunately we can't do that, not easily anyway.
We use bash for writing the test scripts because it's the sane choice for shell programming, and we declare /bin/bash (or something with 'env') at the top of each of those scripts. This introduces a *build* dependency from nbdkit to /bin/bash. That's all fine.
However when nbdkit is installed in production & runs an external script, it uses system(3) which uses /bin/sh. That might not be bash, and indeed bash might not even be installed on the same machine as nbdkit.
[Note: I think the whole Debian dash-for-/bin/sh is the stupidest idea I ever heard, but (a) it's a thing and (b) there's BSD and Macs where they don't want to use bash for licensing reasons.]
Arguably for the tests we could have some way to cause nbdkit to use /bin/bash instead of /bin/sh when running external scripts, but it's major complexity to implement.
So we gotta use a lowest common denominator for these external scripts, even in our tests. Note only this one test is affected.
[Aside: Windows is a whole separate kettle of fish. Currently the Windows port of nbdkit doesn't support --run for other reasons, but if it did it would probably run some Windows-ish thing (COMMAND.COM? Power Shell?). I'm not sure how Windows behaves for the other external commands we try to run like logscript.]
Shell arrays are a bash-specific feature, and the extent array is deeply ingrained. See especially commit df63b23b6280 ("log: Use strict shell quoting for every parameter displayed in the log file.", 2021-01-04). In particular the assignment
extents=(0x0 0x8000 "hole,zero" 0x8000 0x8000 "")
turns "extents" into an array variable.
In the bug tracker, comment <https://aur.archlinux.org/packages/nbdkit#comment-937375> says, "Thus this is an upstream issue; their scripts are calling sh when they should be calling bash". I think that's correct; for logscript=..., we should require /bin/bash in the manual, and execute the script with /bin/bash explicitly, not just system().
This would create a runtime dependency from nbdkit to /bin/bash which I'd like to avoid.
The runtime dependency is already there in our logscript interface; the
name=(a b c ... z)
syntax is already bash-only, for defining a shell array.
Hmm, the extents are indeed passed as a flattened array, ie. extents=(<offset> <length> "<descr>" [repeats ...]) I don't think I was aware this wasn't in POSIX. However that's a dependency from _nbdkit-log-filter_ to bash, which is slightly different. (Nevertheless it'd be nice if we could remove it ...)
So the question is basically how to best emulate an array in the POSIX shell. Some (rough) options that occur to me:
- Use named variables such as name_0, name_1, name_2, ... and so on. Requires eval tricks, and if the array is large, it creates many variables, which some shells (?) may have issues with.
- Generate a shell function with a huge case statement; like "get_name 0" should print "a", "get_name 1" should print "b", etc. The caller would then do
element=$(get_name $idx)
- write the elements of the array to a text file (one, quoted, element per line), and then use a combination of "tail" and "head" for fetching the right line. Incredibly slow, of course.
... I'm sure stackoverflow has further / better ideas for emulating arrays in the POSIX shell.
And then, because keeping the current (fast, but nonportable) solution would be nice, we should probably add a new argument for the log filter, "compat" or "posix" or some such, which would select the more restricted interface.
Yup. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top