[PATCH nbdkit v2 0/2] tests: Invoke MALLOC_CHECK_ correctly and only for glibc
This is essentially the same as v1 (https://listman.redhat.com/archives/libguestfs/2021-August/msg00088.html) except: - Fixed a typo in the commit message. - Added a second patch which fixes* the valgrind issue. *Fix or workaround depending on your point of view - see the comment in the patch. Rich.
MALLOC_CHECK_ and MALLOC_PERTURB_ environment variables have been replaced in glibc 2.34 by GLIBC_TUNABLES settings[1][2]. In addition the new glibc requires that you preload a new library called libc_malloc_debug.so.0 to get these features. These features never worked in non-glibc (but the environment variables are ignored). Only define MALLOC_* environment variables when we detect glibc < 2.34, using the ordinary glibc macros[3]. For glibc >= 2.34, use the new library and GLIBC_TUNABLES. Note this relies on LD_PRELOAD warning but otherwise not breaking if a preload library does not exist. [1] https://www.gnu.org/software/libc/manual/html_node/Memory-Allocation-Tunable... [2] https://sourceware.org/pipermail/libc-alpha/2021-August/129718.html [3] https://sourceforge.net/p/predef/wiki/Libraries/ Thanks: Eric Blake --- configure.ac | 24 ++++++++++++++++++++++++ tests/Makefile.am | 23 +++++++++++++++++------ 2 files changed, 41 insertions(+), 6 deletions(-) diff --git a/configure.ac b/configure.ac index 9eb9b174..bdb56b37 100644 --- a/configure.ac +++ b/configure.ac @@ -569,6 +569,30 @@ AS_IF([test "x$is_windows" = "xyes"],[ AC_SEARCH_LIBS([getaddrinfo], [network socket]) +dnl Does this libc have MALLOC_CHECK_ and MALLOC_PERTURB_ (both glibc) +dnl and does the feature require libc_malloc_debug.so (glibc >= 2.34)? +AC_MSG_CHECKING([if this is glibc]) +AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ +#include <limits.h> +#ifndef __GLIBC__ +#error "not glibc" +#endif + ]])], [is_glibc=yes], [is_glibc=no] +) +AC_MSG_RESULT([$is_glibc]) +AM_CONDITIONAL([HAVE_GLIBC], [test "x$is_glibc" = "xyes"]) + +AC_MSG_CHECKING([if this is glibc >= 2.34]) +AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ +#include <limits.h> +#if !defined(__GLIBC__) || __GLIBC__ < 2 || (__GLIBC__ == 2 && __GLIBC_MINOR__ < 34) +#error "not glibc 2.34" +#endif + ]])], [is_glibc_234=yes], [is_glibc_234=no] +) +AC_MSG_RESULT([$is_glibc_234]) +AM_CONDITIONAL([HAVE_GLIBC_234], [test "x$is_glibc_234" = "xyes"]) + dnl Check for SELinux socket labelling (optional). PKG_CHECK_MODULES([LIBSELINUX], [libselinux], [ AC_SUBST([LIBSELINUX_CFLAGS]) diff --git a/tests/Makefile.am b/tests/Makefile.am index a6e27d95..5919187b 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -61,21 +61,32 @@ EXTRA_DIST = README.tests EXTRA_PROGRAMS = # Use the 'direct' backend, and ensure maximum libguestfs debugging. -# # Enable libnbd debugging. -# -# Enable malloc-check as a cheap way to find some use-after-free and -# uninitialized read problems when using glibc, and doesn't affect -# normal operation or other libc. TESTS_ENVIRONMENT = \ SRCDIR=$(srcdir) \ LIBGUESTFS_ATTACH_METHOD=appliance \ LIBGUESTFS_DEBUG=1 \ LIBGUESTFS_TRACE=1 \ LIBNBD_DEBUG=1 \ + $(NULL) + +# Enable malloc-check as a cheap way to find some use-after-free and +# uninitialized read problems when using glibc, and doesn't affect +# normal operation or other libc. +random = $(shell bash -c 'echo $$(( 1 + (RANDOM & 255) ))') +if HAVE_GLIBC_234 +TESTS_ENVIRONMENT += \ + LD_PRELOAD="$${LD_PRELOAD:+"$$LD_PRELOAD:"}$(libdir)/libc_malloc_debug.so.0" \ + GLIBC_TUNABLES=glibc.malloc.check=1:glibc.malloc.perturb=$(random) \ + $(NULL) +else +if HAVE_GLIBC +TESTS_ENVIRONMENT += \ MALLOC_CHECK_=1 \ - MALLOC_PERTURB_=$(shell bash -c 'echo $$(( 1 + (RANDOM & 255) ))') \ + MALLOC_PERTURB_=$(random) \ $(NULL) +endif +endif if !IS_WINDOWS # Ensure we're testing the local copy by running everything through -- 2.32.0
On Sun, Aug 22, 2021 at 01:57:15PM +0100, Richard W.M. Jones wrote:
MALLOC_CHECK_ and MALLOC_PERTURB_ environment variables have been replaced in glibc 2.34 by GLIBC_TUNABLES settings[1][2]. In addition the new glibc requires that you preload a new library called libc_malloc_debug.so.0 to get these features.
Do we actually require the soname .so.0, or can we get by with the more generic version-less .so (and thus continue to work when glibc bumps to .so.1, provided that bump is backwards compatible for our usage)?
These features never worked in non-glibc (but the environment variables are ignored).
Only define MALLOC_* environment variables when we detect glibc < 2.34, using the ordinary glibc macros[3].
For glibc >= 2.34, use the new library and GLIBC_TUNABLES. Note this relies on LD_PRELOAD warning but otherwise not breaking if a preload library does not exist.
[1] https://www.gnu.org/software/libc/manual/html_node/Memory-Allocation-Tunable... [2] https://sourceware.org/pipermail/libc-alpha/2021-August/129718.html [3] https://sourceforge.net/p/predef/wiki/Libraries/
Thanks: Eric Blake --- +# Enable malloc-check as a cheap way to find some use-after-free and +# uninitialized read problems when using glibc, and doesn't affect +# normal operation or other libc. +random = $(shell bash -c 'echo $$(( 1 + (RANDOM & 255) ))') +if HAVE_GLIBC_234 +TESTS_ENVIRONMENT += \ + LD_PRELOAD="$${LD_PRELOAD:+"$$LD_PRELOAD:"}$(libdir)/libc_malloc_debug.so.0" \ + GLIBC_TUNABLES=glibc.malloc.check=1:glibc.malloc.perturb=$(random) \ + $(NULL)
-- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
On Mon, Aug 23, 2021 at 09:36:44AM -0500, Eric Blake wrote:
On Sun, Aug 22, 2021 at 01:57:15PM +0100, Richard W.M. Jones wrote:
MALLOC_CHECK_ and MALLOC_PERTURB_ environment variables have been replaced in glibc 2.34 by GLIBC_TUNABLES settings[1][2]. In addition the new glibc requires that you preload a new library called libc_malloc_debug.so.0 to get these features.
Do we actually require the soname .so.0, or can we get by with the more generic version-less .so (and thus continue to work when glibc bumps to .so.1, provided that bump is backwards compatible for our usage)?
For Fedora the naked .so file is packaged in glibc-devel. You could argue that developers are going to need that file in order to build anything (since it's required to link programs), so I guess it is OK to remove the ".0". Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
--- wrapper.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/wrapper.c b/wrapper.c index ed77a0a1..42a0cd88 100644 --- a/wrapper.c +++ b/wrapper.c @@ -220,6 +220,15 @@ main (int argc, char *argv[]) * https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/... */ unsetenv ("DEBUGINFOD_URLS"); + + /* Temporary(?) workaround for: + * https://sourceware.org/bugzilla/show_bug.cgi?id=28256 + * + * But should we actually use malloc checking etc when we are + * valgrinding? It seems to duplicate work done by valgrind and + * might even hide issues. + */ + unsetenv ("GLIBC_TUNABLES"); } else { s = getenv ("NBDKIT_GDB"); -- 2.32.0
On Sun, Aug 22, 2021 at 01:57:16PM +0100, Richard W.M. Jones wrote:
--- wrapper.c | 9 +++++++++ 1 file changed, 9 insertions(+)
diff --git a/wrapper.c b/wrapper.c index ed77a0a1..42a0cd88 100644 --- a/wrapper.c +++ b/wrapper.c @@ -220,6 +220,15 @@ main (int argc, char *argv[]) * https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/... */ unsetenv ("DEBUGINFOD_URLS"); + + /* Temporary(?) workaround for: + * https://sourceware.org/bugzilla/show_bug.cgi?id=28256 + * + * But should we actually use malloc checking etc when we are + * valgrinding? It seems to duplicate work done by valgrind and + * might even hide issues. + */ + unsetenv ("GLIBC_TUNABLES");
Good point about reducing the cross-interference between glibc and what valgrind wants to prove. I'm fine if we permanently disable glibc's malloc checking when we know we are running under valgrind. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Thanks for the review. This is what I pushed which I think should incorporate all the suggestions you made. I have also checked it works with older glibc now. https://gitlab.com/nbdkit/nbdkit/-/commit/362e0fdcae37db876e13b944102a5c152e... https://gitlab.com/nbdkit/nbdkit/-/commit/ae8c815e54e66ac7b421e0a6929c39d9ce... Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v
There's a problem here ... For background see: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/... (You have to click the "..." in that email) If libc_malloc_debug.so.0 isn't installed then LD_PRELOAD generates lots of warning messages and this actually breaks some tests. If ./configure is used without --libdir then we try to set LD_PRELOAD=/usr/local/lib/libc_malloc_debug.so.0 which causes the warnings/failures. My first idea was to test in ./configure if $libdir/libc_malloc_debug.so.0 exists and only set LD_PRELOAD if so. However that doesn't work because of $!"£% exec_prefix (a.k.a. the most useless GNU feature that no one uses but causes all the trouble). Also if you have glibc.i686 and glibc.x86_64 installed then both /lib64/libc_malloc_debug.so.0 and /lib/libc_malloc_debug.so.0 exist and you must choose the correct one. On other distros the path will be different, eg. the 64 bit one in /lib/libc_malloc_debug.so.0 or /lib/x86_64-linux-gnu/... etc. So I don't know how to solve any of this. It would be nicer if glibc handled the details and automatically loaded the right library if GLIBC_TUNABLES was set. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/
On Tue, Aug 24, 2021 at 2:52 PM Richard W.M. Jones <rjones@redhat.com> wrote:
There's a problem here ...
For background see: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/... (You have to click the "..." in that email)
If libc_malloc_debug.so.0 isn't installed then LD_PRELOAD generates lots of warning messages and this actually breaks some tests.
If ./configure is used without --libdir then we try to set LD_PRELOAD=/usr/local/lib/libc_malloc_debug.so.0 which causes the warnings/failures.
My first idea was to test in ./configure if $libdir/libc_malloc_debug.so.0 exists and only set LD_PRELOAD if so. However that doesn't work because of $!"£% exec_prefix (a.k.a. the most useless GNU feature that no one uses but causes all the trouble).
Also if you have glibc.i686 and glibc.x86_64 installed then both /lib64/libc_malloc_debug.so.0 and /lib/libc_malloc_debug.so.0 exist and you must choose the correct one. On other distros the path will be different, eg. the 64 bit one in /lib/libc_malloc_debug.so.0 or /lib/x86_64-linux-gnu/... etc.
ld.so has a new --list-diagnostics option (since 2.34 AFAICT, backported to at least 2.33) that lists the system directories; maybe that could help? $ /lib64/ld-linux-x86-64.so.2 --list-diagnostics | grep system_dirs path.system_dirs[0x0]="/lib64/" path.system_dirs[0x1]="/usr/lib64/" libc_malloc_debug.so.0 will always be in one of these directories unless a distribution has misconfigured their glibc and not added their multilib paths into system_dirs.
So I don't know how to solve any of this. It would be nicer if glibc handled the details and automatically loaded the right library if GLIBC_TUNABLES was set.
It is not impossible to implement; tunable parsing happens early enough that if any tunable requires a preloaded module then we can make it happen as long as the module is in path.system_dirs. We could fail silently too; it's certainly better than ignoring silently. Do you see any gotchas here Florian? Siddhesh
participants (3)
-
Eric Blake -
Richard W.M. Jones -
Siddhesh Poyarekar