[nbdkit PATCH] cc: Allow configuration without absolute paths
In https://gitlab.com/nbdkit/nbdkit/-/merge_requests/30, Khem reports that in a cross-compilation environment, nbdkit embeds the absolute name of the cross-compiler into the resulting cc plugin, even though running the plugin should be favoring the bare name 'cc'. This in turn leads to non-reproducible builds. As the goal of cross-compiling nbdkit is to produce a binary that behaves identically regardless of the build environment used, this means we need to give the user control over the defaults for CC and CFLAGS embedded into the cc plugin. However, instead of trying to munge the build environment variable as suggested in that merge request, I found it cleaner to just add additional precious variables to be set at configure time, as in: ./configure CC=/path/to/cross-compiler CC_PLUGIN_CC='ccache gcc' ... Reported-by: Khem Raj Signed-off-by: Eric Blake <eblake@redhat.com> --- gitlab doesn't let me see the right email address to cc; if I can figure that out, I'll tweak the Reported-by line as appropriate before committing... --- plugins/cc/nbdkit-cc-plugin.pod | 9 ++++++--- configure.ac | 11 +++++++++++ plugins/cc/Makefile.am | 4 ++-- 3 files changed, 19 insertions(+), 5 deletions(-) diff --git a/plugins/cc/nbdkit-cc-plugin.pod b/plugins/cc/nbdkit-cc-plugin.pod index 2974890c..2bc3cfb8 100644 --- a/plugins/cc/nbdkit-cc-plugin.pod +++ b/plugins/cc/nbdkit-cc-plugin.pod @@ -45,9 +45,12 @@ To replace the compiler flags: The plugin parameters C<CC>, C<CFLAGS> and C<EXTRA_CFLAGS> (written in uppercase) can be used to control which C compiler and C compiler -flags are used. If not set, the default compiler and flags from when -nbdkit was itself compiled from source are used. To see what those -were you can do: +flags are used. If not set, you can hardcode the defaults for C<CC> +and C<CFLAGS> at the time nbdkit is compiled from source by +configuring with C<CC_PLUGIN_CC=...> and C<CC_PLUGIN_CFLAGS=...>, +otherwise, the configuration for compiling nbdkit itself is used +(C<EXTRA_CFLAGS> can only be set from the command line when starting +the cc plugin). To see what those were you can do: $ nbdkit cc --dump-plugin ... diff --git a/configure.ac b/configure.ac index afc5ddab..e5e261c8 100644 --- a/configure.ac +++ b/configure.ac @@ -820,6 +820,15 @@ AC_ARG_ENABLE([plugins], [disable all bundled plugins and filters])]) AM_CONDITIONAL([HAVE_PLUGINS], [test "x$enable_plugins" != "xno"]) +dnl For the cc plugin, let the user hard-code their preferred compiler setup +dnl Default to the settings used for nbdkit itself +AC_ARG_VAR([CC_PLUGIN_CC], + [Value to use for CC when building the cc plugin, default $CC]) +: "${CC_PLUGIN_CC:=$CC}" +AC_ARG_VAR([CC_PLUGIN_CFLAGS], + [Value to use for CFLAGS when building the cc plugin, default $CFLAGS]) +: "${CC_PLUGIN_CFLAGS:=$CFLAGS}" + dnl Check for Perl, for embedding in the perl plugin. dnl Note that the perl binary is checked above. AC_ARG_ENABLE([perl], @@ -1716,6 +1725,8 @@ feature "tests using libguestfs" \ test "x$HAVE_LIBGUESTFS_TRUE" = "x" && \ test "x$USE_LIBGUESTFS_FOR_TESTS_TRUE" = "x" feature "zlib-ng" test "x$ZLIB_NG_LIBS" != "x" +print cc-plugin-CC "$CC_PLUGIN_CC" +print cc-plugin-CFLAGS "$CC_PLUGIN_CFLAGS" echo echo "If any optional component is configured ‘no’ when you expected ‘yes’" diff --git a/plugins/cc/Makefile.am b/plugins/cc/Makefile.am index 935d125f..088b5ff3 100644 --- a/plugins/cc/Makefile.am +++ b/plugins/cc/Makefile.am @@ -45,8 +45,8 @@ nbdkit_cc_plugin_la_SOURCES = \ $(NULL) nbdkit_cc_plugin_la_CPPFLAGS = \ - -DCC="\"$(CC)\"" \ - -DCFLAGS="\"$(CFLAGS)\"" \ + -DCC="\"$(CC_PLUGIN_CC)\"" \ + -DCFLAGS="\"$(CC_PLUGIN_CFLAGS)\"" \ -I$(top_srcdir)/include \ -I$(top_builddir)/include \ -I$(top_srcdir)/common/include \ -- 2.41.0
On Fri, Aug 18, 2023 at 08:27:45AM -0500, Eric Blake wrote:
In https://gitlab.com/nbdkit/nbdkit/-/merge_requests/30, Khem reports that in a cross-compilation environment, nbdkit embeds the absolute name of the cross-compiler into the resulting cc plugin, even though running the plugin should be favoring the bare name 'cc'. This in turn leads to non-reproducible builds. As the goal of cross-compiling nbdkit is to produce a binary that behaves identically regardless of the build environment used, this means we need to give the user control over the defaults for CC and CFLAGS embedded into the cc plugin.
However, instead of trying to munge the build environment variable as suggested in that merge request, I found it cleaner to just add additional precious variables to be set at configure time, as in:
./configure CC=/path/to/cross-compiler CC_PLUGIN_CC='ccache gcc' ...
Reported-by: Khem Raj Signed-off-by: Eric Blake <eblake@redhat.com> ---
gitlab doesn't let me see the right email address to cc; if I can figure that out, I'll tweak the Reported-by line as appropriate before committing... --- plugins/cc/nbdkit-cc-plugin.pod | 9 ++++++--- configure.ac | 11 +++++++++++ plugins/cc/Makefile.am | 4 ++-- 3 files changed, 19 insertions(+), 5 deletions(-)
diff --git a/plugins/cc/nbdkit-cc-plugin.pod b/plugins/cc/nbdkit-cc-plugin.pod index 2974890c..2bc3cfb8 100644 --- a/plugins/cc/nbdkit-cc-plugin.pod +++ b/plugins/cc/nbdkit-cc-plugin.pod @@ -45,9 +45,12 @@ To replace the compiler flags:
The plugin parameters C<CC>, C<CFLAGS> and C<EXTRA_CFLAGS> (written in uppercase) can be used to control which C compiler and C compiler -flags are used. If not set, the default compiler and flags from when -nbdkit was itself compiled from source are used. To see what those -were you can do: +flags are used. If not set, you can hardcode the defaults for C<CC> +and C<CFLAGS> at the time nbdkit is compiled from source by +configuring with C<CC_PLUGIN_CC=...> and C<CC_PLUGIN_CFLAGS=...>, +otherwise, the configuration for compiling nbdkit itself is used +(C<EXTRA_CFLAGS> can only be set from the command line when starting +the cc plugin). To see what those were you can do:
$ nbdkit cc --dump-plugin ...
Widening the context, ... CC=gcc CFLAGS=-g -O2 -fPIC -shared The C<CFLAGS> parameter overrides the built-in flags completely. The C<EXTRA_CFLAGS> parameter adds extra flags to the built-in flags. Since we already mention EXTRA_CFLAGS below the example, I'm not sure if my addition of a parenthetical about EXTRA_CFLAGS above is worthwhile, or just adds noise.
diff --git a/configure.ac b/configure.ac index afc5ddab..e5e261c8 100644 --- a/configure.ac +++ b/configure.ac @@ -820,6 +820,15 @@ AC_ARG_ENABLE([plugins], [disable all bundled plugins and filters])]) AM_CONDITIONAL([HAVE_PLUGINS], [test "x$enable_plugins" != "xno"])
+dnl For the cc plugin, let the user hard-code their preferred compiler setup +dnl Default to the settings used for nbdkit itself +AC_ARG_VAR([CC_PLUGIN_CC], + [Value to use for CC when building the cc plugin, default $CC])
I'm wondering if there is a better way to word this (it shows up in './configure --help' output). Maybe: [Value to hard-code into the cc plugin's default for CC, instead of $CC]
+: "${CC_PLUGIN_CC:=$CC}" +AC_ARG_VAR([CC_PLUGIN_CFLAGS], + [Value to use for CFLAGS when building the cc plugin, default $CFLAGS]) +: "${CC_PLUGIN_CFLAGS:=$CFLAGS}"
and similar -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org
On Fri, Aug 18, 2023 at 09:10:32AM -0500, Eric Blake wrote:
On Fri, Aug 18, 2023 at 08:27:45AM -0500, Eric Blake wrote:
In https://gitlab.com/nbdkit/nbdkit/-/merge_requests/30, Khem reports that in a cross-compilation environment, nbdkit embeds the absolute name of the cross-compiler into the resulting cc plugin, even though running the plugin should be favoring the bare name 'cc'. This in turn leads to non-reproducible builds. As the goal of cross-compiling nbdkit is to produce a binary that behaves identically regardless of the build environment used, this means we need to give the user control over the defaults for CC and CFLAGS embedded into the cc plugin.
However, instead of trying to munge the build environment variable as suggested in that merge request, I found it cleaner to just add additional precious variables to be set at configure time, as in:
./configure CC=/path/to/cross-compiler CC_PLUGIN_CC='ccache gcc' ...
Reported-by: Khem Raj Signed-off-by: Eric Blake <eblake@redhat.com>
Do we need 'Fixes:' here with the link to the merge request?
---
gitlab doesn't let me see the right email address to cc; if I can figure that out, I'll tweak the Reported-by line as appropriate before committing... --- plugins/cc/nbdkit-cc-plugin.pod | 9 ++++++--- configure.ac | 11 +++++++++++ plugins/cc/Makefile.am | 4 ++-- 3 files changed, 19 insertions(+), 5 deletions(-)
diff --git a/plugins/cc/nbdkit-cc-plugin.pod b/plugins/cc/nbdkit-cc-plugin.pod index 2974890c..2bc3cfb8 100644 --- a/plugins/cc/nbdkit-cc-plugin.pod +++ b/plugins/cc/nbdkit-cc-plugin.pod @@ -45,9 +45,12 @@ To replace the compiler flags:
The plugin parameters C<CC>, C<CFLAGS> and C<EXTRA_CFLAGS> (written in uppercase) can be used to control which C compiler and C compiler -flags are used. If not set, the default compiler and flags from when -nbdkit was itself compiled from source are used. To see what those -were you can do: +flags are used. If not set, you can hardcode the defaults for C<CC> +and C<CFLAGS> at the time nbdkit is compiled from source by +configuring with C<CC_PLUGIN_CC=...> and C<CC_PLUGIN_CFLAGS=...>, +otherwise, the configuration for compiling nbdkit itself is used +(C<EXTRA_CFLAGS> can only be set from the command line when starting +the cc plugin). To see what those were you can do:
$ nbdkit cc --dump-plugin ...
Widening the context,
... CC=gcc CFLAGS=-g -O2 -fPIC -shared
The C<CFLAGS> parameter overrides the built-in flags completely. The C<EXTRA_CFLAGS> parameter adds extra flags to the built-in flags.
Since we already mention EXTRA_CFLAGS below the example, I'm not sure if my addition of a parenthetical about EXTRA_CFLAGS above is worthwhile, or just adds noise.
TBH I would just leave the documentation as it is now. The user of the plugin doesn't care that there was a ./configure flag that the downstream packager could have used.
diff --git a/configure.ac b/configure.ac index afc5ddab..e5e261c8 100644 --- a/configure.ac +++ b/configure.ac @@ -820,6 +820,15 @@ AC_ARG_ENABLE([plugins], [disable all bundled plugins and filters])]) AM_CONDITIONAL([HAVE_PLUGINS], [test "x$enable_plugins" != "xno"])
+dnl For the cc plugin, let the user hard-code their preferred compiler setup +dnl Default to the settings used for nbdkit itself +AC_ARG_VAR([CC_PLUGIN_CC], + [Value to use for CC when building the cc plugin, default $CC])
I'm wondering if there is a better way to word this (it shows up in './configure --help' output). Maybe:
[Value to hard-code into the cc plugin's default for CC, instead of $CC]
This seems better. Does $CC expand here? It might actally be better if it appears as literal "$CC".
+: "${CC_PLUGIN_CC:=$CC}" +AC_ARG_VAR([CC_PLUGIN_CFLAGS], + [Value to use for CFLAGS when building the cc plugin, default $CFLAGS]) +: "${CC_PLUGIN_CFLAGS:=$CFLAGS}"
and similar
Reviewed-by: Richard W.M. Jones <rjones@redhat.com> Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
participants (2)
-
Eric Blake -
Richard W.M. Jones