Hi, most shell scripts in the v2v projects start with a shebang like this: #!/bin/bash - I *think* I understand the intent of the single hyphen, but (a) it seems unnecessary, (b) even if we insisted, using the double-hyphen separator "--" is much more idiomatic (even though the shell, per POSIX, is supposed to interpret "-" identically to "--"). Regarding why I think the hyphen is unnecessary: - setuid shell scripts are not a thing on any platform we (should) care about - the script name is not reinterpreted as an option *anyway* Consider: cat >-v <<EOT #!/bin/bash echo hello \$1 EOT chmod +x -- -v PATH=$PWD:$PATH -v world --> hello world That is, when we run the script under the name "-v" (with it being on the PATH) and with the single command line argument "world", we'd naively asume that the shebang translated to: /bin/bash -v world where "-v" came from the script's name, and "world" would be taken as the name of a shell script to execute. Thus, the idea would be to prevent this (i.e., to pass options to the interpreter by renaming or (sym)linking the shell script). But that's not what we actually see; what we see is consistent with the command: /bin/bash -- -v world --> hello world In fact, if I append "sleep 1000" to the script, I can also fetch: $ hexdump -C /proc/10975/cmdline 00000000 62 61 73 68 00 2d 2d 00 2d 76 00 77 6f 72 6c 64 |bash.--.-v.world| 00000010 00 |.| 00000011 So glibc and/or the Linux kernel already inserts the "--" option/operand delimiter! I intend to contribute a shell script to virt-p2v; do I need to use the hyphen in the shebang? (If so, I prefer the double-hyphen.) Thanks! Laszlo
On Thu, Aug 25, 2022 at 03:32:15PM +0200, Laszlo Ersek wrote:
Hi,
most shell scripts in the v2v projects start with a shebang like this:
#!/bin/bash -
I *think* I understand the intent of the single hyphen, but (a) it seems unnecessary, (b) even if we insisted, using the double-hyphen separator "--" is much more idiomatic (even though the shell, per POSIX, is supposed to interpret "-" identically to "--").
Regarding why I think the hyphen is unnecessary:
- setuid shell scripts are not a thing on any platform we (should) care about
- the script name is not reinterpreted as an option *anyway*
Consider:
cat >-v <<EOT #!/bin/bash echo hello \$1 EOT
chmod +x -- -v
PATH=$PWD:$PATH -v world --> hello world
That is, when we run the script under the name "-v" (with it being on the PATH) and with the single command line argument "world", we'd naively asume that the shebang translated to:
/bin/bash -v world
where "-v" came from the script's name, and "world" would be taken as the name of a shell script to execute. Thus, the idea would be to prevent this (i.e., to pass options to the interpreter by renaming or (sym)linking the shell script).
But that's not what we actually see; what we see is consistent with the command:
/bin/bash -- -v world --> hello world
In fact, if I append "sleep 1000" to the script, I can also fetch:
$ hexdump -C /proc/10975/cmdline 00000000 62 61 73 68 00 2d 2d 00 2d 76 00 77 6f 72 6c 64 |bash.--.-v.world| 00000010 00 |.| 00000011
So glibc and/or the Linux kernel already inserts the "--" option/operand delimiter!
I intend to contribute a shell script to virt-p2v; do I need to use the hyphen in the shebang? (If so, I prefer the double-hyphen.)
Eric should be able to give the definitive answer here. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top
On Thu, Aug 25, 2022 at 02:35:47PM +0100, Richard W.M. Jones wrote:
On Thu, Aug 25, 2022 at 03:32:15PM +0200, Laszlo Ersek wrote:
Hi,
most shell scripts in the v2v projects start with a shebang like this:
#!/bin/bash -
I *think* I understand the intent of the single hyphen, but (a) it seems unnecessary, (b) even if we insisted, using the double-hyphen separator "--" is much more idiomatic (even though the shell, per POSIX, is supposed to interpret "-" identically to "--").
Regarding why I think the hyphen is unnecessary:
- setuid shell scripts are not a thing on any platform we (should) care about
- the script name is not reinterpreted as an option *anyway*
Consider:
cat >-v <<EOT #!/bin/bash echo hello \$1 EOT
chmod +x -- -v
PATH=$PWD:$PATH -v world
More believable as: (PATH=$PWD:$PATH; -v world) In your style, you are relying on the shell using the just-modified PATH setting to look up the location of -v; but POSIX allows both a system where the shell does its own lookup (and sticks to the old PATH, only passing the new PATH to the child process of the -v it found on the old PATH), as well as a system where the shell forks off a subprocess, the subprocess modifies environ to set the new PATH, and then relies on execvp() or similar to do the lookup on what is already the new environ. But that's a distraction from your main point, that yes, Linux injects "--" to the interpreter for a shebang, to protect against an executable script whose name begins with '-'.
So glibc and/or the Linux kernel already inserts the "--" option/operand delimiter!
Shebang interpretation is sometimes by the kernel, and sometimes by the shell, and it differs by OS. Some OS's do word-splitting on the rest of the shebang after the first space, some treat the entire rest of the line as a single argument. For a nice summary of some of the warts of shebang lines, see https://unix.stackexchange.com/questions/29608/why-is-it-better-to-use-usr-b... If you are worried about /bin/bash being available on all systems where a v2v script will be installed, you probably want '#!/bin/env bash' to find the first bash in PATH. But once you add your first whitespace, it is no longer portable to use any others, so while '#!/bin/bash -' works, '#!/bin/env bash -' does not. I find that the issue of writing a shebang to use a program that may not always be installed in the same absolute location across all OSs (and therefore where '#!/bin/env name' is useful) is more common than the issue of writing a shebang that wants to pass an explicit '-' or '--' end-of-option marker to protect against the script being installed under a name like '-c ls'. Note that GNU coreutils recently added '/bin/env -S', modeled after a similar feature first in BSD systems, which makes it possible to write '#!/bin/env -S app --with args' parsed in a more similar manner regardless of whether the OS pre-splits the line or lumps it in one argument, but '#!/bin/env -S bash -' is also unlikely to work portably as 'env -S' is not yet in widespread usage.
I intend to contribute a shell script to virt-p2v; do I need to use the hyphen in the shebang? (If so, I prefer the double-hyphen.)
Eric should be able to give the definitive answer here.
POSIX intentionally does not standardize #!, so the definitive answer is that there isn't one ;) But unless we plan on catering to someone sym- or hard-linking an alternative name for your script that begins with '-' (which I find overkill - no one sane does that), I think omitting the '-' in the #! line is the best course of action, because the only thing it does is add a layer of protection on non-Linux machines against someone creating a stupid alternative name for the script, which is unlikely to happen. And omitting the '-' frees us up to insert /bin/env if we are worried about /bin/bash being non-portable. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
On Fri, Aug 26, 2022 at 07:29:34AM -0500, Eric Blake wrote:
On Thu, Aug 25, 2022 at 02:35:47PM +0100, Richard W.M. Jones wrote:
On Thu, Aug 25, 2022 at 03:32:15PM +0200, Laszlo Ersek wrote:
Hi,
most shell scripts in the v2v projects start with a shebang like this:
#!/bin/bash -
I *think* I understand the intent of the single hyphen, but (a) it seems unnecessary, (b) even if we insisted, using the double-hyphen separator "--" is much more idiomatic (even though the shell, per POSIX, is supposed to interpret "-" identically to "--").
Regarding why I think the hyphen is unnecessary:
- setuid shell scripts are not a thing on any platform we (should) care about
- the script name is not reinterpreted as an option *anyway*
Consider:
cat >-v <<EOT #!/bin/bash echo hello \$1 EOT
chmod +x -- -v
PATH=$PWD:$PATH -v world
More believable as:
(PATH=$PWD:$PATH; -v world)
In your style, you are relying on the shell using the just-modified PATH setting to look up the location of -v; but POSIX allows both a system where the shell does its own lookup (and sticks to the old PATH, only passing the new PATH to the child process of the -v it found on the old PATH), as well as a system where the shell forks off a subprocess, the subprocess modifies environ to set the new PATH, and then relies on execvp() or similar to do the lookup on what is already the new environ.
But that's a distraction from your main point, that yes, Linux injects "--" to the interpreter for a shebang, to protect against an executable script whose name begins with '-'.
So glibc and/or the Linux kernel already inserts the "--" option/operand delimiter!
Shebang interpretation is sometimes by the kernel, and sometimes by the shell, and it differs by OS. Some OS's do word-splitting on the rest of the shebang after the first space, some treat the entire rest of the line as a single argument.
For a nice summary of some of the warts of shebang lines, see
https://unix.stackexchange.com/questions/29608/why-is-it-better-to-use-usr-b...
If you are worried about /bin/bash being available on all systems where a v2v script will be installed, you probably want '#!/bin/env bash' to find the first bash in PATH. But once you add your first whitespace, it is no longer portable to use any others, so while '#!/bin/bash -' works, '#!/bin/env bash -' does not. I find that the issue of writing a shebang to use a program that may not always be installed in the same absolute location across all OSs (and therefore where '#!/bin/env name' is useful) is more common than the issue of writing a shebang that wants to pass an explicit '-' or '--' end-of-option marker to protect against the script being installed under a name like '-c ls'.
Fedora packaging guidelines -- rightly or wrongly -- forbid #!/usr/bin/env: https://docs.fedoraproject.org/en-US/packaging-guidelines/#_shebang_lines although apparently RPM rewrites them automatically so perhaps we would not notice.
Note that GNU coreutils recently added '/bin/env -S', modeled after a similar feature first in BSD systems, which makes it possible to write '#!/bin/env -S app --with args' parsed in a more similar manner regardless of whether the OS pre-splits the line or lumps it in one argument, but '#!/bin/env -S bash -' is also unlikely to work portably as 'env -S' is not yet in widespread usage.
I intend to contribute a shell script to virt-p2v; do I need to use the hyphen in the shebang? (If so, I prefer the double-hyphen.)
Eric should be able to give the definitive answer here.
POSIX intentionally does not standardize #!, so the definitive answer is that there isn't one ;) But unless we plan on catering to someone sym- or hard-linking an alternative name for your script that begins with '-' (which I find overkill - no one sane does that), I think omitting the '-' in the #! line is the best course of action, because the only thing it does is add a layer of protection on non-Linux machines against someone creating a stupid alternative name for the script, which is unlikely to happen. And omitting the '-' frees us up to insert /bin/env if we are worried about /bin/bash being non-portable.
virt-p2v is a relatively Linux-specific project. The ISO always runs Linux, and arguably therefore you'd always want to run the scripts that prepare the appliance like virt-p2v-make-<foo> on Linux. For other projects we need to worry about what FreeBSD, OpenBSD and nowadays macOS do. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW
On Fri, Aug 26, 2022 at 01:37:37PM +0100, Richard W.M. Jones wrote:
Shebang interpretation is sometimes by the kernel, and sometimes by the shell, and it differs by OS. Some OS's do word-splitting on the rest of the shebang after the first space, some treat the entire rest of the line as a single argument.
For a nice summary of some of the warts of shebang lines, see
https://unix.stackexchange.com/questions/29608/why-is-it-better-to-use-usr-b...
If you are worried about /bin/bash being available on all systems where a v2v script will be installed, you probably want '#!/bin/env bash' to find the first bash in PATH. But once you add your first whitespace, it is no longer portable to use any others, so while '#!/bin/bash -' works, '#!/bin/env bash -' does not. I find that the issue of writing a shebang to use a program that may not always be installed in the same absolute location across all OSs (and therefore where '#!/bin/env name' is useful) is more common than the issue of writing a shebang that wants to pass an explicit '-' or '--' end-of-option marker to protect against the script being installed under a name like '-c ls'.
Fedora packaging guidelines -- rightly or wrongly -- forbid #!/usr/bin/env:
https://docs.fedoraproject.org/en-US/packaging-guidelines/#_shebang_lines
although apparently RPM rewrites them automatically so perhaps we would not notice.
Yeah, for an executable installed as part of the system, you really DO want it hard-coded to the interpreter also installed by the system (because that's what was integration-tested), and not using env (which is dependent on the user's environment on whether it behaves the same or picks a different interpreter than what the system integration-tested). It's a good policy for the RPM, but less so for scripts that are not installed as part of a single system (that is, scripts in $HOME/bin that is network-mounted across multiple different OSs is very much a good place to use #!/bin/env). -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
participants (3)
-
Eric Blake -
Laszlo Ersek -
Richard W.M. Jones