12:53 p.m.
Pipeline #665159249 has failed!
Project: libnbd ( https://gitlab.com/nbdkit/libnbd )
Branch: master ( https://gitlab.com/nbdkit/libnbd/-/commits/master )
Commit: 018d55a8 (
https://gitlab.com/nbdkit/libnbd/-/commit/018d55a83be047824017e23eec6c0ed... )
Commit Message: tests: Skip rather than fail on missing nbdkit ...
Commit Author: Eric Blake ( https://gitlab.com/ebblake )
Pipeline #665159249 ( https://gitlab.com/nbdkit/libnbd/-/pipelines/665159249 ) triggered
by Eric Blake ( https://gitlab.com/ebblake )
had 3 failed jobs.
Job #3163966652 ( https://gitlab.com/nbdkit/libnbd/-/jobs/3163966652/raw )
Stage: builds
Name: i686-debian-11-prebuilt-env
Job #3163966650 ( https://gitlab.com/nbdkit/libnbd/-/jobs/3163966650/raw )
Stage: builds
Name: i686-debian-10-prebuilt-env
Job #3163966643 ( https://gitlab.com/nbdkit/libnbd/-/jobs/3163966643/raw )
Stage: builds
Name: x86_64-opensuse-leap-153-prebuilt-env
--
You're receiving this email because of your account on gitlab.com.
2 p.m.
On Wed, Oct 12, 2022 at 05:53:51PM +0000, GitLab wrote:
Pipeline #665159249 has failed!
Project: libnbd ( https://gitlab.com/nbdkit/libnbd )
Branch: master ( https://gitlab.com/nbdkit/libnbd/-/commits/master )
Commit: 018d55a8 (
https://gitlab.com/nbdkit/libnbd/-/commit/018d55a83be047824017e23eec6c0ed... )
Commit Message: tests: Skip rather than fail on missing nbdkit ...
Commit Author: Eric Blake ( https://gitlab.com/ebblake )
Pipeline #665159249 ( https://gitlab.com/nbdkit/libnbd/-/pipelines/665159249 ) triggered
by Eric Blake ( https://gitlab.com/ebblake )
had 3 failed jobs.
Yay - forcing a container rebuild fixed the alpine-315 problem in the
previous run (where the alpine container was broken from a timeout
during the previous container build). Thanks to danpb on IRC for the
hint (documenting it here if I need to do it again:
<danpb> eblake: assuming you have admin rights, go to
https://gitlab.com/nbdkit/libnbd/-/pipelines
<danpb> eblake: and hit 'Run pipeline' and in the form that appears
<danpb> eblake: select 'master' branch, and then enter a variable
'RUN_CONTAINER_BUILDS' with a value of 1
<danpb> eblake: that will force a recreation of all container images from currently
committed master
)
However, even with that fixed,
Job #3163966652 ( https://gitlab.com/nbdkit/libnbd/-/jobs/3163966652/raw )
Stage: builds
Name: i686-debian-11-prebuilt-env
Job #3163966650 ( https://gitlab.com/nbdkit/libnbd/-/jobs/3163966650/raw )
Stage: builds
Name: i686-debian-10-prebuilt-env
these two are new failures, due to a typo in upstream lcitool; merge
request for that pending at
https://gitlab.com/libvirt/libvirt-ci/-/merge_requests/327, and I'm
pushing a patch with files regenerated with that lcitool fix applied.
Job #3163966643 (
https://gitlab.com/nbdkit/libnbd/-/jobs/3163966643/raw )
Stage: builds
Name: x86_64-opensuse-leap-153-prebuilt-env
This one is still failing because of a bug in gnutls; the log is
reporting:
libnbd: debug: nbd1: nbd_connect_command: transition:
NEWSTYLE.OPT_STARTTLS.RECV_REPLY_PAYLOAD -> NEWSTYLE.OPT_STARTTLS.CHECK_REPLY
free(): invalid pointer
libnbd: debug: nbd1: nbd_connect_command: transition: NEWSTYLE.OPT_STARTTLS.CHECK_REPLY
-> NEWSTYLE.OPT_STARTTLS.TLS_HANDSHAKE_READ
libnbd: debug: nbd1: nbd_connect_command: transition:
NEWSTYLE.OPT_STARTTLS.TLS_HANDSHAKE_READ -> DEAD
libnbd: debug: nbd1: nbd_connect_command: leave: error="nbd_connect_command:
gnutls_handshake: Error in the pull function. (-1/1)"
That libc message about invalid free() is scary; I'm not yet sure
whether it is a bug in opensuse-leap's gnutls package or something
we're doing wrong in libnbd.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
3:49 a.m.
On Wed, Oct 12, 2022 at 02:00:21PM -0500, Eric Blake wrote:
> Job #3163966643 (
https://gitlab.com/nbdkit/libnbd/-/jobs/3163966643/raw )
>
> Stage: builds
> Name: x86_64-opensuse-leap-153-prebuilt-env
This one is still failing because of a bug in gnutls; the log is
reporting:
libnbd: debug: nbd1: nbd_connect_command: transition:
NEWSTYLE.OPT_STARTTLS.RECV_REPLY_PAYLOAD -> NEWSTYLE.OPT_STARTTLS.CHECK_REPLY
free(): invalid pointer
libnbd: debug: nbd1: nbd_connect_command: transition: NEWSTYLE.OPT_STARTTLS.CHECK_REPLY
-> NEWSTYLE.OPT_STARTTLS.TLS_HANDSHAKE_READ
libnbd: debug: nbd1: nbd_connect_command: transition:
NEWSTYLE.OPT_STARTTLS.TLS_HANDSHAKE_READ -> DEAD
libnbd: debug: nbd1: nbd_connect_command: leave: error="nbd_connect_command:
gnutls_handshake: Error in the pull function. (-1/1)"
That libc message about invalid free() is scary; I'm not yet sure
whether it is a bug in opensuse-leap's gnutls package or something
we're doing wrong in libnbd.
I had a look into this. Unfortunately I only have OpenSUSE Tumbleweed
available. It doesn't fail for me in Tumbleweed. (It also doesn't
fail in the CI pipeline for Tumbleweed.)
So I guess this problem is somehow specific to nbdkit or gnutls in
OpenSUSE 15.3.
We can probably ignore this failure, under the assumption it is fixed
upstream.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
4:33 a.m.
On Thu, Oct 13, 2022 at 09:49:09AM +0100, Richard W.M. Jones wrote:
On Wed, Oct 12, 2022 at 02:00:21PM -0500, Eric Blake wrote:
> > Job #3163966643 ( https://gitlab.com/nbdkit/libnbd/-/jobs/3163966643/raw )
> >
> > Stage: builds
> > Name: x86_64-opensuse-leap-153-prebuilt-env
>
> This one is still failing because of a bug in gnutls; the log is
> reporting:
>
> libnbd: debug: nbd1: nbd_connect_command: transition:
NEWSTYLE.OPT_STARTTLS.RECV_REPLY_PAYLOAD -> NEWSTYLE.OPT_STARTTLS.CHECK_REPLY
> free(): invalid pointer
> libnbd: debug: nbd1: nbd_connect_command: transition:
NEWSTYLE.OPT_STARTTLS.CHECK_REPLY -> NEWSTYLE.OPT_STARTTLS.TLS_HANDSHAKE_READ
> libnbd: debug: nbd1: nbd_connect_command: transition:
NEWSTYLE.OPT_STARTTLS.TLS_HANDSHAKE_READ -> DEAD
> libnbd: debug: nbd1: nbd_connect_command: leave: error="nbd_connect_command:
gnutls_handshake: Error in the pull function. (-1/1)"
>
> That libc message about invalid free() is scary; I'm not yet sure
> whether it is a bug in opensuse-leap's gnutls package or something
> we're doing wrong in libnbd.
I had a look into this. Unfortunately I only have OpenSUSE Tumbleweed
available. It doesn't fail for me in Tumbleweed. (It also doesn't
fail in the CI pipeline for Tumbleweed.)
Anyone has access to the CI env. Line 9 of the build log
shows the container env used:
Using docker image sha256:e4a8e52b0bbb712a544a90d21b21010daad8ab3e85a768cfea38571461ec85fc
for registry.gitlab.com/nbdkit/libnbd/ci-opensuse-leap-153:latest with digest
registry.gitlab.com/nbdkit/libnbd/ci-opensuse-leap-153@sha256:11179119130...
...
You just need to launch the same container, clone the git repo and
then run the build commands
IOW, on your local machine do:
$ podman run -it registry.gitlab.com/nbdkit/libnbd/ci-opensuse-leap-153:latestn
# git clone https://gitlab.com/nbdkit/libnbd
# cd libnbd
# autoreconf -if
# ./configure --enable-gcc-warnings --with-gnutls --with-libxml2 --enable-fuse
--enable-ocaml --enable-python --enable-golang
# make -j 20
# cd tests
# ./connect-tls-psk
requires nbdkit --tls-verify-peer -U - null --run 'exit 0'
nbdkit: pattern: error: failed to set TLS session priority to
@NBDKIT,SYSTEM:+ECDHE-PSK:+DHE-PSK:+PSK: The request is invalid.
nbd_connect_command: gnutls_handshake: Error in the push function. (-1/1)
What's interesting here is that this shows the real error
mesage about TLS sessino priority.
If you set MALLOC_CHECK=1, however, then we loose the useful
error message:
# MALLOC_CHECK_=1 MALLOC_PERTURB_=146 ./connect-tls-psk
requires nbdkit --tls-verify-peer -U - null --run 'exit 0'
free(): invalid pointer
nbd_connect_command: gnutls_handshake: Error in the pull function. (-1/1)
which was unfortunate for debuggability.
I confirmed it is nbdkit that is crashing and it appears to be
in gnutls code.
Looking at the image there is no /etc/crypto-policies directory,
and nor is there any 'crypto-policies' package available in the
distro.
So they have mis-built nbdkit in leap 15.3 with TLS priority
string of @NBDKIT,SYSTEM, despite not having support for that
in their distro.
So I guess this problem is somehow specific to nbdkit or gnutls in
OpenSUSE 15.3.
Yep, broken nbdkit, compared by free() crash bug in gnutls
hiding the real error
We can probably ignore this failure, under the assumption it is
fixed
upstream.
In ci/manifest.yml set 'allow-failure: true' for 15.3, and
re-run lcitool manifest.
Or disable gnutls build on 15.3 for CI purposes by passing --without-gnutls
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
4:02 p.m.
Hi Daniel,
Thanks for the detailed report!
On 10/13/22 03:33, Daniel P. Berrangé wrote:
On Thu, Oct 13, 2022 at 09:49:09AM +0100, Richard W.M. Jones wrote:
> On Wed, Oct 12, 2022 at 02:00:21PM -0500, Eric Blake wrote:
>>> Job #3163966643 ( https://gitlab.com/nbdkit/libnbd/-/jobs/3163966643/raw )
>>>
>>> Stage: builds
>>> Name: x86_64-opensuse-leap-153-prebuilt-env
>>
>> This one is still failing because of a bug in gnutls; the log is
>> reporting:
>>
>> libnbd: debug: nbd1: nbd_connect_command: transition:
NEWSTYLE.OPT_STARTTLS.RECV_REPLY_PAYLOAD -> NEWSTYLE.OPT_STARTTLS.CHECK_REPLY
>> free(): invalid pointer
>> libnbd: debug: nbd1: nbd_connect_command: transition:
NEWSTYLE.OPT_STARTTLS.CHECK_REPLY -> NEWSTYLE.OPT_STARTTLS.TLS_HANDSHAKE_READ
>> libnbd: debug: nbd1: nbd_connect_command: transition:
NEWSTYLE.OPT_STARTTLS.TLS_HANDSHAKE_READ -> DEAD
>> libnbd: debug: nbd1: nbd_connect_command: leave: error="nbd_connect_command:
gnutls_handshake: Error in the pull function. (-1/1)"
>>
>> That libc message about invalid free() is scary; I'm not yet sure
>> whether it is a bug in opensuse-leap's gnutls package or something
>> we're doing wrong in libnbd.
>
> I had a look into this. Unfortunately I only have OpenSUSE Tumbleweed
> available. It doesn't fail for me in Tumbleweed. (It also doesn't
> fail in the CI pipeline for Tumbleweed.)
Anyone has access to the CI env. Line 9 of the build log
shows the container env used:
Using docker image
sha256:e4a8e52b0bbb712a544a90d21b21010daad8ab3e85a768cfea38571461ec85fc for
registry.gitlab.com/nbdkit/libnbd/ci-opensuse-leap-153:latest with digest
registry.gitlab.com/nbdkit/libnbd/ci-opensuse-leap-153@sha256:11179119130...
...
You just need to launch the same container, clone the git repo and
then run the build commands
IOW, on your local machine do:
$ podman run -it registry.gitlab.com/nbdkit/libnbd/ci-opensuse-leap-153:latestn
# git clone https://gitlab.com/nbdkit/libnbd
# cd libnbd
# autoreconf -if
# ./configure --enable-gcc-warnings --with-gnutls --with-libxml2 --enable-fuse
--enable-ocaml --enable-python --enable-golang
# make -j 20
# cd tests
# ./connect-tls-psk
requires nbdkit --tls-verify-peer -U - null --run 'exit 0'
nbdkit: pattern: error: failed to set TLS session priority to
@NBDKIT,SYSTEM:+ECDHE-PSK:+DHE-PSK:+PSK: The request is invalid.
nbd_connect_command: gnutls_handshake: Error in the push function. (-1/1)
What's interesting here is that this shows the real error
mesage about TLS sessino priority.
If you set MALLOC_CHECK=1, however, then we loose the useful
error message:
# MALLOC_CHECK_=1 MALLOC_PERTURB_=146 ./connect-tls-psk
requires nbdkit --tls-verify-peer -U - null --run 'exit 0'
free(): invalid pointer
nbd_connect_command: gnutls_handshake: Error in the pull function. (-1/1)
which was unfortunate for debuggability.
I confirmed it is nbdkit that is crashing and it appears to be
in gnutls code.
Looking at the image there is no /etc/crypto-policies directory,
and nor is there any 'crypto-policies' package available in the
distro.
Indeed. Leap 15.4 and newer include the crypto-policies package. Should the
container move to a 15.4 base?
So they have mis-built nbdkit in leap 15.3 with TLS priority
string of @NBDKIT,SYSTEM, despite not having support for that
in their distro.
I'll fix this in our downstream packages. Thanks a lot for bringing it to my
attention.
Regards,
Jim
5:52 p.m.
On Thu, Oct 13, 2022 at 03:02:51PM -0600, Jim Fehlig wrote:
Hi Daniel,
Thanks for the detailed report!
> What's interesting here is that this shows the real error
> mesage about TLS sessino priority.
>
> If you set MALLOC_CHECK=1, however, then we loose the useful
> error message:
>
> # MALLOC_CHECK_=1 MALLOC_PERTURB_=146 ./connect-tls-psk
> requires nbdkit --tls-verify-peer -U - null --run 'exit 0'
> free(): invalid pointer
> nbd_connect_command: gnutls_handshake: Error in the pull function. (-1/1)
>
> which was unfortunate for debuggability.
>
> I confirmed it is nbdkit that is crashing and it appears to be
> in gnutls code.
>
> Looking at the image there is no /etc/crypto-policies directory,
> and nor is there any 'crypto-policies' package available in the
> distro.
Indeed. Leap 15.4 and newer include the crypto-policies package. Should the
container move to a 15.4 base?
Looking further, in addition to the nbdkit bug depending on a priority
string that is not possible in the base 15.3 distro, there is a
definite bug in gnutls 3.6.7 shipped in the distro that was later
fixed by gnutls commit 90142f2d "Use inih to parse configuration
file". Look at the gnutls code base of lib/priority.c prior to that
patch:
static char *system_priority_buf = NULL;
...
char *_gnutls_resolve_priorities(const char* priorities)
...
#ifdef HAVE_FMEMOPEN
/* Always try to refresh the cached data, to
* allow it to be updated without restarting
* all applications
*/
_gnutls_update_system_priorities();
fp = fmemopen(system_priority_buf, system_priority_buf_size,
"r");
#else
fp = fopen(system_priority_file, "r");
#endif
...
fclose(fp);
fp = NULL;
This is very much a case of older gnutls mis-using fmemopen(), such
that an fclose() on a second or third attempt to use
system_priority_buf is indeed freeing an invalid pointer.
> So they have mis-built nbdkit in leap 15.3 with TLS priority
> string of @NBDKIT,SYSTEM, despite not having support for that
> in their distro.
I'll fix this in our downstream packages. Thanks a lot for bringing it to my
attention.
You may also want to point the gnutls maintainers to the need to
backport that patch (or otherwise fix that nastiness).
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
2:17 a.m.
On Thu, Oct 13, 2022 at 03:02:51PM -0600, Jim Fehlig wrote:
Hi Daniel,
Thanks for the detailed report!
On 10/13/22 03:33, Daniel P. Berrangé wrote:
> On Thu, Oct 13, 2022 at 09:49:09AM +0100, Richard W.M. Jones wrote:
> > On Wed, Oct 12, 2022 at 02:00:21PM -0500, Eric Blake wrote:
> > > > Job #3163966643 (
https://gitlab.com/nbdkit/libnbd/-/jobs/3163966643/raw )
> > > >
> > > > Stage: builds
> > > > Name: x86_64-opensuse-leap-153-prebuilt-env
> > >
> > > This one is still failing because of a bug in gnutls; the log is
> > > reporting:
> > >
> > > libnbd: debug: nbd1: nbd_connect_command: transition:
NEWSTYLE.OPT_STARTTLS.RECV_REPLY_PAYLOAD -> NEWSTYLE.OPT_STARTTLS.CHECK_REPLY
> > > free(): invalid pointer
> > > libnbd: debug: nbd1: nbd_connect_command: transition:
NEWSTYLE.OPT_STARTTLS.CHECK_REPLY -> NEWSTYLE.OPT_STARTTLS.TLS_HANDSHAKE_READ
> > > libnbd: debug: nbd1: nbd_connect_command: transition:
NEWSTYLE.OPT_STARTTLS.TLS_HANDSHAKE_READ -> DEAD
> > > libnbd: debug: nbd1: nbd_connect_command: leave:
error="nbd_connect_command: gnutls_handshake: Error in the pull function.
(-1/1)"
> > >
> > > That libc message about invalid free() is scary; I'm not yet sure
> > > whether it is a bug in opensuse-leap's gnutls package or something
> > > we're doing wrong in libnbd.
> >
> > I had a look into this. Unfortunately I only have OpenSUSE Tumbleweed
> > available. It doesn't fail for me in Tumbleweed. (It also doesn't
> > fail in the CI pipeline for Tumbleweed.)
>
> Anyone has access to the CI env. Line 9 of the build log
> shows the container env used:
>
> Using docker image
sha256:e4a8e52b0bbb712a544a90d21b21010daad8ab3e85a768cfea38571461ec85fc for
registry.gitlab.com/nbdkit/libnbd/ci-opensuse-leap-153:latest with digest
registry.gitlab.com/nbdkit/libnbd/ci-opensuse-leap-153@sha256:11179119130...
...
>
> You just need to launch the same container, clone the git repo and
> then run the build commands
>
> IOW, on your local machine do:
>
> $ podman run -it registry.gitlab.com/nbdkit/libnbd/ci-opensuse-leap-153:latestn
> # git clone https://gitlab.com/nbdkit/libnbd
> # cd libnbd
> # autoreconf -if
> # ./configure --enable-gcc-warnings --with-gnutls --with-libxml2 --enable-fuse
--enable-ocaml --enable-python --enable-golang
>
> # make -j 20
> # cd tests
> # ./connect-tls-psk
> requires nbdkit --tls-verify-peer -U - null --run 'exit 0'
> nbdkit: pattern: error: failed to set TLS session priority to
@NBDKIT,SYSTEM:+ECDHE-PSK:+DHE-PSK:+PSK: The request is invalid.
> nbd_connect_command: gnutls_handshake: Error in the push function. (-1/1)
>
> What's interesting here is that this shows the real error
> mesage about TLS sessino priority.
>
> If you set MALLOC_CHECK=1, however, then we loose the useful
> error message:
>
> # MALLOC_CHECK_=1 MALLOC_PERTURB_=146 ./connect-tls-psk
> requires nbdkit --tls-verify-peer -U - null --run 'exit 0'
> free(): invalid pointer
> nbd_connect_command: gnutls_handshake: Error in the pull function. (-1/1)
>
> which was unfortunate for debuggability.
>
> I confirmed it is nbdkit that is crashing and it appears to be
> in gnutls code.
>
> Looking at the image there is no /etc/crypto-policies directory,
> and nor is there any 'crypto-policies' package available in the
> distro.
Indeed. Leap 15.4 and newer include the crypto-policies package. Should the
container move to a 15.4 base?
Yes, we need to add 15.4 to libvirt-ci facts database, given
the relative EOL dates.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
4:22 p.m.
On 10/14/22 01:17, Daniel P. Berrangé wrote:
On Thu, Oct 13, 2022 at 03:02:51PM -0600, Jim Fehlig wrote:
> Hi Daniel,
>
> Thanks for the detailed report!
>
> On 10/13/22 03:33, Daniel P. Berrangé wrote:
>> On Thu, Oct 13, 2022 at 09:49:09AM +0100, Richard W.M. Jones wrote:
>>> On Wed, Oct 12, 2022 at 02:00:21PM -0500, Eric Blake wrote:
>>>>> Job #3163966643 (
https://gitlab.com/nbdkit/libnbd/-/jobs/3163966643/raw )
>>>>>
>>>>> Stage: builds
>>>>> Name: x86_64-opensuse-leap-153-prebuilt-env
>>>>
>>>> This one is still failing because of a bug in gnutls; the log is
>>>> reporting:
>>>>
>>>> libnbd: debug: nbd1: nbd_connect_command: transition:
NEWSTYLE.OPT_STARTTLS.RECV_REPLY_PAYLOAD -> NEWSTYLE.OPT_STARTTLS.CHECK_REPLY
>>>> free(): invalid pointer
>>>> libnbd: debug: nbd1: nbd_connect_command: transition:
NEWSTYLE.OPT_STARTTLS.CHECK_REPLY -> NEWSTYLE.OPT_STARTTLS.TLS_HANDSHAKE_READ
>>>> libnbd: debug: nbd1: nbd_connect_command: transition:
NEWSTYLE.OPT_STARTTLS.TLS_HANDSHAKE_READ -> DEAD
>>>> libnbd: debug: nbd1: nbd_connect_command: leave:
error="nbd_connect_command: gnutls_handshake: Error in the pull function.
(-1/1)"
>>>>
>>>> That libc message about invalid free() is scary; I'm not yet sure
>>>> whether it is a bug in opensuse-leap's gnutls package or something
>>>> we're doing wrong in libnbd.
>>>
>>> I had a look into this. Unfortunately I only have OpenSUSE Tumbleweed
>>> available. It doesn't fail for me in Tumbleweed. (It also doesn't
>>> fail in the CI pipeline for Tumbleweed.)
>>
>> Anyone has access to the CI env. Line 9 of the build log
>> shows the container env used:
>>
>> Using docker image
sha256:e4a8e52b0bbb712a544a90d21b21010daad8ab3e85a768cfea38571461ec85fc for
registry.gitlab.com/nbdkit/libnbd/ci-opensuse-leap-153:latest with digest
registry.gitlab.com/nbdkit/libnbd/ci-opensuse-leap-153@sha256:11179119130...
...
>>
>> You just need to launch the same container, clone the git repo and
>> then run the build commands
>>
>> IOW, on your local machine do:
>>
>> $ podman run -it
registry.gitlab.com/nbdkit/libnbd/ci-opensuse-leap-153:latestn
>> # git clone https://gitlab.com/nbdkit/libnbd
>> # cd libnbd
>> # autoreconf -if
>> # ./configure --enable-gcc-warnings --with-gnutls --with-libxml2
--enable-fuse --enable-ocaml --enable-python --enable-golang
>>
>> # make -j 20
>> # cd tests
>> # ./connect-tls-psk
>> requires nbdkit --tls-verify-peer -U - null --run 'exit 0'
>> nbdkit: pattern: error: failed to set TLS session priority to
@NBDKIT,SYSTEM:+ECDHE-PSK:+DHE-PSK:+PSK: The request is invalid.
>> nbd_connect_command: gnutls_handshake: Error in the push function. (-1/1)
>>
>> What's interesting here is that this shows the real error
>> mesage about TLS sessino priority.
>>
>> If you set MALLOC_CHECK=1, however, then we loose the useful
>> error message:
>>
>> # MALLOC_CHECK_=1 MALLOC_PERTURB_=146 ./connect-tls-psk
>> requires nbdkit --tls-verify-peer -U - null --run 'exit 0'
>> free(): invalid pointer
>> nbd_connect_command: gnutls_handshake: Error in the pull function. (-1/1)
>>
>> which was unfortunate for debuggability.
>>
>> I confirmed it is nbdkit that is crashing and it appears to be
>> in gnutls code.
>>
>> Looking at the image there is no /etc/crypto-policies directory,
>> and nor is there any 'crypto-policies' package available in the
>> distro.
>
> Indeed. Leap 15.4 and newer include the crypto-policies package. Should the
> container move to a 15.4 base?
Yes, we need to add 15.4 to libvirt-ci facts database, given
the relative EOL dates.
I was about to do that today and see you've already taken care of it :-).
Thanks!
Jim
3:10 a.m.
On Mon, Oct 17, 2022 at 03:22:17PM -0600, Jim Fehlig wrote:
On 10/14/22 01:17, Daniel P. Berrangé wrote:
> On Thu, Oct 13, 2022 at 03:02:51PM -0600, Jim Fehlig wrote:
> > Hi Daniel,
> >
> > Thanks for the detailed report!
> >
> > On 10/13/22 03:33, Daniel P. Berrangé wrote:
> > > On Thu, Oct 13, 2022 at 09:49:09AM +0100, Richard W.M. Jones wrote:
> > > > On Wed, Oct 12, 2022 at 02:00:21PM -0500, Eric Blake wrote:
> > > > > > Job #3163966643 (
https://gitlab.com/nbdkit/libnbd/-/jobs/3163966643/raw )
> > > > > >
> > > > > > Stage: builds
> > > > > > Name: x86_64-opensuse-leap-153-prebuilt-env
> > > > >
> > > > > This one is still failing because of a bug in gnutls; the log
is
> > > > > reporting:
> > > > >
> > > > > libnbd: debug: nbd1: nbd_connect_command: transition:
NEWSTYLE.OPT_STARTTLS.RECV_REPLY_PAYLOAD -> NEWSTYLE.OPT_STARTTLS.CHECK_REPLY
> > > > > free(): invalid pointer
> > > > > libnbd: debug: nbd1: nbd_connect_command: transition:
NEWSTYLE.OPT_STARTTLS.CHECK_REPLY -> NEWSTYLE.OPT_STARTTLS.TLS_HANDSHAKE_READ
> > > > > libnbd: debug: nbd1: nbd_connect_command: transition:
NEWSTYLE.OPT_STARTTLS.TLS_HANDSHAKE_READ -> DEAD
> > > > > libnbd: debug: nbd1: nbd_connect_command: leave:
error="nbd_connect_command: gnutls_handshake: Error in the pull function.
(-1/1)"
> > > > >
> > > > > That libc message about invalid free() is scary; I'm not yet
sure
> > > > > whether it is a bug in opensuse-leap's gnutls package or
something
> > > > > we're doing wrong in libnbd.
> > > >
> > > > I had a look into this. Unfortunately I only have OpenSUSE
Tumbleweed
> > > > available. It doesn't fail for me in Tumbleweed. (It also
doesn't
> > > > fail in the CI pipeline for Tumbleweed.)
> > >
> > > Anyone has access to the CI env. Line 9 of the build log
> > > shows the container env used:
> > >
> > > Using docker image
sha256:e4a8e52b0bbb712a544a90d21b21010daad8ab3e85a768cfea38571461ec85fc for
registry.gitlab.com/nbdkit/libnbd/ci-opensuse-leap-153:latest with digest
registry.gitlab.com/nbdkit/libnbd/ci-opensuse-leap-153@sha256:11179119130...
...
> > >
> > > You just need to launch the same container, clone the git repo and
> > > then run the build commands
> > >
> > > IOW, on your local machine do:
> > >
> > > $ podman run -it
registry.gitlab.com/nbdkit/libnbd/ci-opensuse-leap-153:latestn
> > > # git clone https://gitlab.com/nbdkit/libnbd
> > > # cd libnbd
> > > # autoreconf -if
> > > # ./configure --enable-gcc-warnings --with-gnutls --with-libxml2
--enable-fuse --enable-ocaml --enable-python --enable-golang
> > >
> > > # make -j 20
> > > # cd tests
> > > # ./connect-tls-psk
> > > requires nbdkit --tls-verify-peer -U - null --run 'exit 0'
> > > nbdkit: pattern: error: failed to set TLS session priority to
@NBDKIT,SYSTEM:+ECDHE-PSK:+DHE-PSK:+PSK: The request is invalid.
> > > nbd_connect_command: gnutls_handshake: Error in the push function.
(-1/1)
> > >
> > > What's interesting here is that this shows the real error
> > > mesage about TLS sessino priority.
> > >
> > > If you set MALLOC_CHECK=1, however, then we loose the useful
> > > error message:
> > >
> > > # MALLOC_CHECK_=1 MALLOC_PERTURB_=146 ./connect-tls-psk
> > > requires nbdkit --tls-verify-peer -U - null --run 'exit 0'
> > > free(): invalid pointer
> > > nbd_connect_command: gnutls_handshake: Error in the pull function.
(-1/1)
> > >
> > > which was unfortunate for debuggability.
> > >
> > > I confirmed it is nbdkit that is crashing and it appears to be
> > > in gnutls code.
> > >
> > > Looking at the image there is no /etc/crypto-policies directory,
> > > and nor is there any 'crypto-policies' package available in the
> > > distro.
> >
> > Indeed. Leap 15.4 and newer include the crypto-policies package. Should the
> > container move to a 15.4 base?
>
> Yes, we need to add 15.4 to libvirt-ci facts database, given
> the relative EOL dates.
I was about to do that today and see you've already taken care of it :-).
Opps, yes, I meant to reply to this mail to say so. When I looked, it
turned out to be trivial as no package changes were needed, so I just
submitted it.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
2:59 p.m.
On Tue, Oct 18, 2022 at 09:10:24AM +0100, Daniel P. Berrangé wrote:
> > >
> > > Indeed. Leap 15.4 and newer include the crypto-policies package. Should
the
> > > container move to a 15.4 base?
> >
> > Yes, we need to add 15.4 to libvirt-ci facts database, given
> > the relative EOL dates.
>
> I was about to do that today and see you've already taken care of it :-).
Opps, yes, I meant to reply to this mail to say so. When I looked, it
turned out to be trivial as no package changes were needed, so I just
submitted it.
I see the change in the libvirt-ci repo, but not how to apply it to
the libnbd repo. Running
lcitool manifest
updates a few .yml files, but does not do anything that obviously
replaces opensuse-leap-153 with opensuse-leap-154, at least in the
changes I saw. But that could be because I'm unfamiliar with how the
process is supposed to work.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
3:27 a.m.
On Tue, Oct 18, 2022 at 02:59:37PM -0500, Eric Blake wrote:
On Tue, Oct 18, 2022 at 09:10:24AM +0100, Daniel P. Berrangé wrote:
> > > >
> > > > Indeed. Leap 15.4 and newer include the crypto-policies package.
Should the
> > > > container move to a 15.4 base?
> > >
> > > Yes, we need to add 15.4 to libvirt-ci facts database, given
> > > the relative EOL dates.
> >
> > I was about to do that today and see you've already taken care of it :-).
>
> Opps, yes, I meant to reply to this mail to say so. When I looked, it
> turned out to be trivial as no package changes were needed, so I just
> submitted it.
I see the change in the libvirt-ci repo, but not how to apply it to
the libnbd repo. Running
lcitool manifest
updates a few .yml files, but does not do anything that obviously
replaces opensuse-leap-153 with opensuse-leap-154, at least in the
changes I saw. But that could be because I'm unfamiliar with how the
process is supposed to work.
The project local ci/manifest.yml explicitly declares what OS targets
your project wants to perform CI against. So either add 15.4 there
in addition to 15.3, or replace 15.3 with 15.4 as you prefer.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
764
days inactive
771
days old
10 comments
5 participants
participants (5)
-
Daniel P. Berrangé -
Eric Blake -
GitLab -
Jim Fehlig -
Richard W.M. Jones