1:58 p.m.
On 2/17/20 12:17 PM, Richard W.M. Jones wrote:
Unfortunately this series doesn't work with the real VDDK library
:-(
I see:
nbdkit: debug: vddk: config_complete
nbdkit: debug: dlopen shim prefix set to
/home/rjones/tmp/vddk-6.7.3/vmware-vix-disklib-distrib/lib64
Then a little bit later during VDDK initialization it fails with:
nbdkit: debug: VixDiskLib: Could not load default plugins from
/home/rjones/tmp/vddk-6.7.3/vmware-vix-disklib-distrib/lib64/libdiskLibPlugin.so: Cannot
open library: libcrypto.so.1.0.2: cannot open shared object file: No such file or
directory.
I was curious if the replacement dlopen() was being called at all, and
it is certainly called from plugins/vddk/vddk.c:
nbdkit: debug: dlopen:
/home/rjones/tmp/vddk-6.7.3/vmware-vix-disklib-distrib/lib64/libvixDiskLib.so.6 flags=2
It's also called from VDDK itself:
nbdkit: debug: dlopen:
/home/rjones/tmp/vddk-6.7.3/vmware-vix-disklib-distrib/lib64/libdiskLibPlugin.so flags=257
That's nice to know...
but it's not being explicitly called for libcrypto.so because
immediately after the above message I see the error:
nbdkit: debug: VixDiskLib: Could not load default plugins from
/home/rjones/tmp/vddk-6.7.3/vmware-vix-disklib-distrib/lib64/libdiskLibPlugin.so: Cannot
open library: libcrypto.so.1.0.2: cannot open shared object file: No such file or
directory.
So I guess libcrypto is actually loaded from a DT_NEEDED entry:
...but that's indeed frustrating.
$ eu-readelf -d
/home/rjones/tmp/vddk-6.7.3/vmware-vix-disklib-distrib/lib64/libdiskLibPlugin.so | grep
NEEDED
NEEDED Shared library: [libpthread.so.0]
NEEDED Shared library: [libstdc++.so.6]
NEEDED Shared library: [libdl.so.2]
NEEDED Shared library: [libcrypto.so.1.0.2]
NEEDED Shared library: [libssl.so.1.0.2]
NEEDED Shared library: [libexpat.so]
NEEDED Shared library: [libz.so.1]
NEEDED Shared library: [libvmacore.so]
NEEDED Shared library: [libvmomi.so]
NEEDED Shared library: [libvim-types.so]
NEEDED Shared library: [librt.so.1]
NEEDED Shared library: [libgcc_s.so.1]
NEEDED Shared library: [libc.so.6]
NEEDED Shared library: [ld-linux-x86-64.so.2]
which I suppose doesn't actually go through dlopen ...
And we were wondering if the dlopen() with an absolute file name would
help the dynamic linker alter its search path for its dependencies.
That may have answered our question, but not the way we wanted. :(
la_objsearch() seems like it should catch these DT_NEEDED entries, but
given the fact that glibc does not yet support DT_AUDIT for
single-process rather than global auditing, we're stuck with having to
set LD_AUDIT in the environment (which is no different than having to
set LD_LIBRARY_PATH in the environment).
Next idea: how hard would it be to re-exec nbdkit? Here's what I'm
envisioning:
nbdkit proper exposes some way to let a plugin query argv[] (necessary,
so that re-exec gets the same arguments). Re-exec is only safe at a
point prior to the end of .config_complete, and must be done in a way
that does not cause infinite recursion on re-exec attempts.
In vddk's .config_complete, if the key reexec= is absent, libdir=... is
present, and dlopen()/init of libvixDiskLib.so fails, then we grab argv
from nbdkit and re-exec 'LD_LIBRARY_PATH=libdir... nbdkit vddk
same_args... reexec=true'. Conversely, if key reexec= is present, then
.config_complete calls setenv() to reset LD_LIBRARY_PATH to its former
state. .config_complete runs prior to the point where nbdkit fork()s
the --run child, so the child will see the original environment rather
than the one rewritten by the re-exec attempt, limiting our temporary
LD_LIBRARY_PATH pollution to just the nbdkit process.
In the meantime, patch 3 (delaying from the dlopen() of libvixDiskLib.so
from .load over to .config_complete) is worthwhile to push now, even if
it doesn't change the status quo (I didn't change it from what you posted).
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org