12:26 p.m.
On 2024-04-28 17:34:42+0000, Richard W.M. Jones wrote:
On Sun, Apr 28, 2024 at 06:22:43PM +0200, Thomas Weißschuh wrote:
> On 2024-04-28 17:09:06+0000, Richard W.M. Jones wrote:
> > On Sun, Apr 28, 2024 at 05:44:53PM +0200, Thomas Weißschuh wrote:
> > > The gcs plugin uses the google cloud module and can't work without
it.
> > > Skip the test if the module is unavailable.
> > >
> > > Signed-off-by: Thomas Weißschuh <thomas(a)t-8ch.de>
> > > ---
> > > tests/test-gcs.sh | 1 +
> > > 1 file changed, 1 insertion(+)
> > >
> > > diff --git a/tests/test-gcs.sh b/tests/test-gcs.sh
> > > index ed823377dc5a..9b165e8159be 100755
> > > --- a/tests/test-gcs.sh
> > > +++ b/tests/test-gcs.sh
> > > @@ -40,6 +40,7 @@ requires $PYTHON --version
> > > requires_nbdcopy
> > > requires_plugin python
> > > skip_if_valgrind "because Python code leaks memory"
> > > +requires $PYTHON -c 'import google.cloud'
> >
> > It shouldn't do (I think?) ... Did you see:
> >
> >
https://gitlab.com/nbdkit/nbdkit/-/commit/aca30111632091bbc91096f19638b9e...
>
> I do have that commit, and I don't have that module installed.
>
> The actual problem is the import of "google.api_core", which itself is
> part of google-api-core which I don't have installed.
>
> That is a very narrow band of platforms this test can run on.
It should run without any google.* modules installed, and also if any
combination of google.* modules are installed. In all cases it ought
to load the mocked module from tests/test-gcs and ignore the installed
google.* modules, or if it can't do that (see my commit message above)
it should skip the test.
For me, that all works.
I missed the tests/test-gcs/google/api_core directory.
However I found last week that there's something seriously weird
about
the google.* modules (not to mention Python module loading in
general), so I can believe the test might not work in some
combination.
Nevertheless, the commit as proposed is definitely wrong. The gcs
test does not (or should not) use any installed google.* module, so if
it does then that's a bug of some kind.
Agree.
I narrowed it down to python-protobuf being installed.
It creates /usr/lib/python3.12/site-packages/google .
I think this should be a namespace package, which is handled specially,
But I'm not sure if the problem is in python-protobuf, the Python import
machinery or the setup by test nbdkit testsuite.
Will investigate.
What the full log (tests/test-gcs.sh.log) when it fails?
FAIL: test-gcs.sh
=================
+ requires_run
+ is_windows
++ nbdkit --dump-config
++ grep '^host_os'
++ cut -d= -f2
+ host_os=linux-gnu
+ [[ linux-gnu =~ mingw|msys ]]
+ requires hexdump --version
+ requires /usr/bin/python3 --version
+ requires_nbdcopy
+ requires nbdcopy --version
+ requires_nbdsh_uri
+ requires nbdsh -c 'exit(not h.supports_uri())'
+ requires_plugin python
+ requires nbdkit python --version
+ skip_if_valgrind 'because Python code leaks memory'
+ '[' '' = 1 ']'
+ requires_not /usr/bin/python3 -c 'import google.cloud.storage'
+ requires test -d ../../tests/test-gcs
+ prepend PYTHONPATH ../../tests/test-gcs
+ eval 'PYTHONPATH=../../tests/test-gcs${PYTHONPATH:+:$PYTHONPATH}'
++ PYTHONPATH=../../tests/test-gcs
+ export PYTHONPATH
+ file=gcs.out
+ rm -f gcs.out
+ cleanup_fn rm -f gcs.out
+ local _hook=_cleanup_hook0
+ declare -ag _cleanup_hook0
+ eval '_cleanup_hook0=("$@")'
++ _cleanup_hook0=("$@")
+ nbdkit -v gcs json-credentials=TEST_JSON_CREDENTIALS bucket=MY_FILES key=MY_KEY --run
'nbdcopy "$uri" gcs.out'
/home/t-8ch/src/nbdkit/build/server/nbdkit -v "--run=nbdcopy \"\$uri\"
gcs.out" -- /home/t-8ch/src/nbdkit/build/plugins/python/.libs/nbdkit-python-plugin.so
/home/t-8ch/src/nbdkit/build/plugins/gcs/nbdkit-gcs-plugin
json-credentials=TEST_JSON_CREDENTIALS bucket=MY_FILES key=MY_KEY
nbdkit: debug: nbdkit 1.39.4
nbdkit: debug: TLS disabled: could not load TLS certificates
nbdkit: debug: service mode: unix socket
nbdkit: debug: NBD URI: nbd+unix://?socket=/tmp/nbdkitQvp5XP/socket
nbdkit: debug: registering plugin
/home/t-8ch/src/nbdkit/build/plugins/python/.libs/nbdkit-python-plugin.so
nbdkit: debug: registered plugin
/home/t-8ch/src/nbdkit/build/plugins/python/.libs/nbdkit-python-plugin.so (name python)
nbdkit: debug: python: load
nbdkit: debug: python: config key=script,
value=/home/t-8ch/src/nbdkit/build/plugins/gcs/nbdkit-gcs-plugin
Traceback (most recent call last):
File "/home/t-8ch/src/nbdkit/build/plugins/gcs/nbdkit-gcs-plugin", line 46, in
<module>
from google.api_core.exceptions import NotFound, GatewayTimeout, \
ModuleNotFoundError: No module named 'google.api_core'
nbdkit: error: /home/t-8ch/src/nbdkit/build/plugins/gcs/nbdkit-gcs-plugin: error running
this script
++ _run_cleanup_hooks
++ local _status=1 _i
++ set +e
++ trap '' INT QUIT TERM EXIT ERR
++ echo ../../tests/test-gcs.sh: run cleanup hooks: exit code 1
../../tests/test-gcs.sh: run cleanup hooks: exit code 1
++ (( _i = 0 ))
++ (( _i < 1 ))
++ eval '"${_cleanup_hook0[@]}"'
+++ rm -f gcs.out
++ (( ++_i ))
++ (( _i < 1 ))
++ exit 1
FAIL test-gcs.sh (exit status: 1)
Thomas