[libnbd PATCH 00/20] Many CI fixes
Lot of clean-ups for the CI. I also added support for easily configurable way of skipping particular tests on selected distros. However, I only skipped those that do not have and easy way of detecting a broken dependency and such like. What is still missing is: - All the CentOS builds have issues with three tests that seem like a protocol violation, but can very well be a bug in some test dependency. Since this is the closest to RHEL we can get, I think it's worth investigating. - FreeBSDs do not even build and it looks like some libtool wrapper script is fed into a C preprocessor, which fails terribly due to the script comments (but should not be fed there at all, of course. - MacOS has issues, although that one is marked to allow failures and the issue does not seem to be on our side anyway. I won't look into it for a while and see if it sorts itself out. Martin Kletzander (20): fuse: Only support defined fallocate modes interop: Do not test against broken qemu-storage-daemon nbdsh: Check for python-exec2c installations tests: Fix hexdump probing tests: Clean up qemu-nbd detection in tests: Enhance fuse probing tests: Fix port randomisation tests: Require cap_sys_admin where root is required tests: Do not use magic parameter for some nbdkit plugins ci: Change docker driver and archive logs ci: Remove cross-compilation targets ci: Remove debian-10-clang build ci: Update build files ci: Add support for FreeBSD-13.0 ci: Fix run commands in README ci: Some build script fixes ci: Add support for test skipping ci: Rename build_script.sh to just build.sh ci: Skip some broken tests/distro combinations ci: Disable `make dist` on openSUSE tests/Makefile.am | 6 +- tests/functions.sh.in | 58 ++++ interop/interop-qemu-storage-daemon.sh | 5 +- .gitlab-ci.yml | 247 ++---------------- ci/README.rst | 16 +- ci/build.sh | 112 ++++++++ ci/build_script.sh | 56 ---- ci/cirrus/build.yml | 2 +- ci/cirrus/freebsd-12.vars | 5 +- ci/cirrus/freebsd-13.vars | 13 + ci/cirrus/freebsd-current.vars | 5 +- ci/cirrus/macos-11.vars | 6 +- ci/containers/centos-8.Dockerfile | 12 +- ci/containers/centos-stream-8.Dockerfile | 12 +- .../debian-10-cross-aarch64.Dockerfile | 86 ------ .../debian-10-cross-armv6l.Dockerfile | 86 ------ .../debian-10-cross-armv7l.Dockerfile | 86 ------ ci/containers/debian-10-cross-i686.Dockerfile | 86 ------ ci/containers/debian-10-cross-mips.Dockerfile | 86 ------ .../debian-10-cross-mips64el.Dockerfile | 86 ------ .../debian-10-cross-mipsel.Dockerfile | 86 ------ .../debian-10-cross-ppc64le.Dockerfile | 86 ------ .../debian-10-cross-s390x.Dockerfile | 86 ------ ci/containers/debian-10.Dockerfile | 9 +- .../debian-sid-cross-aarch64.Dockerfile | 86 ------ .../debian-sid-cross-armv6l.Dockerfile | 86 ------ .../debian-sid-cross-armv7l.Dockerfile | 86 ------ .../debian-sid-cross-i686.Dockerfile | 86 ------ .../debian-sid-cross-mips64el.Dockerfile | 86 ------ .../debian-sid-cross-mipsel.Dockerfile | 86 ------ .../debian-sid-cross-ppc64le.Dockerfile | 86 ------ .../debian-sid-cross-s390x.Dockerfile | 86 ------ ci/containers/debian-sid.Dockerfile | 9 +- ci/containers/fedora-33.Dockerfile | 14 +- ci/containers/fedora-34.Dockerfile | 14 +- .../fedora-rawhide-cross-mingw32.Dockerfile | 71 ----- .../fedora-rawhide-cross-mingw64.Dockerfile | 71 ----- ci/containers/fedora-rawhide.Dockerfile | 14 +- ci/containers/opensuse-leap-152.Dockerfile | 12 +- ci/containers/opensuse-tumbleweed.Dockerfile | 12 +- ci/containers/refresh | 20 -- ci/containers/ubuntu-1804.Dockerfile | 8 +- ci/containers/ubuntu-2004.Dockerfile | 9 +- ci/skipped_tests | 9 + copy/copy-block-to-nbd.sh | 3 +- copy/copy-file-to-qcow2.sh | 16 +- copy/copy-nbd-to-block.sh | 3 +- copy/copy-nbd-to-file.sh | 2 +- copy/copy-nbd-to-nbd.sh | 2 +- copy/copy-nbd-to-small-block-error.sh | 5 +- copy/copy-tls.sh | 2 +- fuse/operations.c | 8 + fuse/test-errors.sh | 2 +- fuse/test-file-mode.sh | 3 +- fuse/test-nbdkit-command.sh | 2 +- fuse/test-nbdkit-file-null.sh | 2 +- fuse/test-nbdkit.sh | 3 +- fuse/test-parallel.sh | 2 +- fuse/test-pattern.sh | 4 +- fuse/test-qcow2.sh | 3 +- fuse/test-trim.sh | 2 +- fuse/test-unix.sh | 2 +- fuse/test-zero.sh | 2 +- info/Makefile.am | 1 + info/info-description-qemu.sh | 17 +- info/info-list-json-qemu.sh | 17 +- info/info-list-qemu.sh | 17 +- info/info-map-qemu-allocation-depth.sh | 16 +- info/info-map-qemu-dirty-bitmap.sh | 23 +- sh/nbdsh.in | 3 +- 70 files changed, 377 insertions(+), 2074 deletions(-) create mode 100755 ci/build.sh delete mode 100755 ci/build_script.sh create mode 100644 ci/cirrus/freebsd-13.vars delete mode 100644 ci/containers/debian-10-cross-aarch64.Dockerfile delete mode 100644 ci/containers/debian-10-cross-armv6l.Dockerfile delete mode 100644 ci/containers/debian-10-cross-armv7l.Dockerfile delete mode 100644 ci/containers/debian-10-cross-i686.Dockerfile delete mode 100644 ci/containers/debian-10-cross-mips.Dockerfile delete mode 100644 ci/containers/debian-10-cross-mips64el.Dockerfile delete mode 100644 ci/containers/debian-10-cross-mipsel.Dockerfile delete mode 100644 ci/containers/debian-10-cross-ppc64le.Dockerfile delete mode 100644 ci/containers/debian-10-cross-s390x.Dockerfile delete mode 100644 ci/containers/debian-sid-cross-aarch64.Dockerfile delete mode 100644 ci/containers/debian-sid-cross-armv6l.Dockerfile delete mode 100644 ci/containers/debian-sid-cross-armv7l.Dockerfile delete mode 100644 ci/containers/debian-sid-cross-i686.Dockerfile delete mode 100644 ci/containers/debian-sid-cross-mips64el.Dockerfile delete mode 100644 ci/containers/debian-sid-cross-mipsel.Dockerfile delete mode 100644 ci/containers/debian-sid-cross-ppc64le.Dockerfile delete mode 100644 ci/containers/debian-sid-cross-s390x.Dockerfile delete mode 100644 ci/containers/fedora-rawhide-cross-mingw32.Dockerfile delete mode 100644 ci/containers/fedora-rawhide-cross-mingw64.Dockerfile create mode 100644 ci/skipped_tests -- 2.31.1
None of them is defined on FreeBSD, for example, and the only other way to make the code compile would be to define it ourselves. Signed-off-by: Martin Kletzander <mkletzan@redhat.com> --- fuse/operations.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/fuse/operations.c b/fuse/operations.c index de04ba7b6d72..098613f66f33 100644 --- a/fuse/operations.c +++ b/fuse/operations.c @@ -440,6 +440,9 @@ nbdfuse_fallocate (const char *path, int mode, off_t offset, off_t len, if (readonly) return -EACCES; +#if defined(FALLOC_FL_PUNCH_HOLE) || defined(FALLOC_FL_ZERO_RANGE) +# if defined(FALLOC_FL_PUNCH_HOLE) + if (mode & FALLOC_FL_PUNCH_HOLE) { if (!nbd_can_trim (nbd.ptr[0])) return -EOPNOTSUPP; /* Trim not supported. */ @@ -448,6 +451,9 @@ nbdfuse_fallocate (const char *path, int mode, off_t offset, off_t len, return 0; } } +#endif /*# if defined(FALLOC_FL_PUNCH_HOLE) */ + +#if defined(FALLOC_FL_ZERO_RANGE) /* As of FUSE 35 this is not supported by the kernel module and it * always returns EOPNOTSUPP. */ @@ -470,7 +476,9 @@ nbdfuse_fallocate (const char *path, int mode, off_t offset, off_t len, return 0; } } +#endif /* defined(FALLOC_FL_ZERO_RANGE) */ else +#endif /* defined(FALLOC_FL_PUNCH_HOLE) || defined(FALLOC_FL_ZERO_RANGE) */ return -EOPNOTSUPP; } -- 2.31.1
On Tue, Jun 08, 2021 at 09:53:42AM +0200, Martin Kletzander wrote:
None of them is defined on FreeBSD, for example, and the only other way to make the code compile would be to define it ourselves.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com> --- fuse/operations.c | 8 ++++++++ 1 file changed, 8 insertions(+)
diff --git a/fuse/operations.c b/fuse/operations.c index de04ba7b6d72..098613f66f33 100644 --- a/fuse/operations.c +++ b/fuse/operations.c @@ -440,6 +440,9 @@ nbdfuse_fallocate (const char *path, int mode, off_t offset, off_t len, if (readonly) return -EACCES;
+#if defined(FALLOC_FL_PUNCH_HOLE) || defined(FALLOC_FL_ZERO_RANGE) +# if defined(FALLOC_FL_PUNCH_HOLE)
I like this indentation of preprocessor directives...
+ if (mode & FALLOC_FL_PUNCH_HOLE) { if (!nbd_can_trim (nbd.ptr[0])) return -EOPNOTSUPP; /* Trim not supported. */ @@ -448,6 +451,9 @@ nbdfuse_fallocate (const char *path, int mode, off_t offset, off_t len, return 0; } } +#endif /*# if defined(FALLOC_FL_PUNCH_HOLE) */ + +#if defined(FALLOC_FL_ZERO_RANGE)
...but you forgot it here
/* As of FUSE 35 this is not supported by the kernel module and it * always returns EOPNOTSUPP. */ @@ -470,7 +476,9 @@ nbdfuse_fallocate (const char *path, int mode, off_t offset, off_t len, return 0; } } +#endif /* defined(FALLOC_FL_ZERO_RANGE) */
...as well as here.
else +#endif /* defined(FALLOC_FL_PUNCH_HOLE) || defined(FALLOC_FL_ZERO_RANGE) */ return -EOPNOTSUPP; }
To some extent, it would also be possible to write: #ifndef FALLOC_FL_PUNCH_HOLE # define FALLOC_FL_PUNCH_HOLE 0 #endif #ifndef FALLOC_FL_ZERO_RANGE # define FALLOC_FL_ZERO_RANGE 0 #endif static int nbdfuse_fallocate(...) { ... if (mode & FALLOC_FL_PUNCH_HOLE) { ... } else if (mode & FALLOC_FL_ZERO_RANGE) { ... } else return -EOPNOTSUPP; } for fewer in-function #ifdefs, but that is a style choice, not a correctness issue, so I won't insist. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
On Tue, Jun 08, 2021 at 07:44:20AM -0500, Eric Blake wrote:
On Tue, Jun 08, 2021 at 09:53:42AM +0200, Martin Kletzander wrote:
None of them is defined on FreeBSD, for example, and the only other way to make the code compile would be to define it ourselves.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com> --- fuse/operations.c | 8 ++++++++ 1 file changed, 8 insertions(+)
diff --git a/fuse/operations.c b/fuse/operations.c index de04ba7b6d72..098613f66f33 100644 --- a/fuse/operations.c +++ b/fuse/operations.c @@ -440,6 +440,9 @@ nbdfuse_fallocate (const char *path, int mode, off_t offset, off_t len, if (readonly) return -EACCES;
+#if defined(FALLOC_FL_PUNCH_HOLE) || defined(FALLOC_FL_ZERO_RANGE) +# if defined(FALLOC_FL_PUNCH_HOLE)
I like this indentation of preprocessor directives...
+ if (mode & FALLOC_FL_PUNCH_HOLE) { if (!nbd_can_trim (nbd.ptr[0])) return -EOPNOTSUPP; /* Trim not supported. */ @@ -448,6 +451,9 @@ nbdfuse_fallocate (const char *path, int mode, off_t offset, off_t len, return 0; } } +#endif /*# if defined(FALLOC_FL_PUNCH_HOLE) */ + +#if defined(FALLOC_FL_ZERO_RANGE)
...but you forgot it here
/* As of FUSE 35 this is not supported by the kernel module and it * always returns EOPNOTSUPP. */ @@ -470,7 +476,9 @@ nbdfuse_fallocate (const char *path, int mode, off_t offset, off_t len, return 0; } } +#endif /* defined(FALLOC_FL_ZERO_RANGE) */
...as well as here.
else +#endif /* defined(FALLOC_FL_PUNCH_HOLE) || defined(FALLOC_FL_ZERO_RANGE) */ return -EOPNOTSUPP; }
To some extent, it would also be possible to write:
#ifndef FALLOC_FL_PUNCH_HOLE # define FALLOC_FL_PUNCH_HOLE 0 #endif #ifndef FALLOC_FL_ZERO_RANGE # define FALLOC_FL_ZERO_RANGE 0 #endif
static int nbdfuse_fallocate(...) { ...
if (mode & FALLOC_FL_PUNCH_HOLE) { ... } else if (mode & FALLOC_FL_ZERO_RANGE) { ... } else return -EOPNOTSUPP; }
for fewer in-function #ifdefs, but that is a style choice, not a correctness issue, so I won't insist.
I like your version way more, thanks!
-- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
The version 6.0.0 has very often a bug that makes the test fail, but which should be fixed in later versions. Version 5.1.0 has an issue as well. The version check is not the nicest way to check for this, but is enough. Signed-off-by: Martin Kletzander <mkletzan@redhat.com> asdf Signed-off-by: Martin Kletzander <mkletzan@redhat.com> --- interop/interop-qemu-storage-daemon.sh | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/interop/interop-qemu-storage-daemon.sh b/interop/interop-qemu-storage-daemon.sh index 09d1898ef459..dc1efcdf4ca1 100755 --- a/interop/interop-qemu-storage-daemon.sh +++ b/interop/interop-qemu-storage-daemon.sh @@ -23,7 +23,10 @@ set -e set -x requires test "x$QEMU_STORAGE_DAEMON" != "x" -requires $QEMU_STORAGE_DAEMON --version +requires sed --version +qsd_version="$($QEMU_STORAGE_DAEMON --version | sed -n '1s/qemu-storage-daemon version \([0-9.]*\) .*/\1/p')" +requires_not test "$qsd_version" = "6.0.0" +requires_not test "$qsd_version" = "5.1.0" requires nbdsh --version requires qemu-img --version -- 2.31.1
On Tue, Jun 08, 2021 at 09:53:43AM +0200, Martin Kletzander wrote:
The version 6.0.0 has very often a bug that makes the test fail, but which should be fixed in later versions. Version 5.1.0 has an issue as well. The version check is not the nicest way to check for this, but is enough.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
asdf
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Bogus line and double s-o-b is odd.
--- interop/interop-qemu-storage-daemon.sh | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/interop/interop-qemu-storage-daemon.sh b/interop/interop-qemu-storage-daemon.sh index 09d1898ef459..dc1efcdf4ca1 100755 --- a/interop/interop-qemu-storage-daemon.sh +++ b/interop/interop-qemu-storage-daemon.sh @@ -23,7 +23,10 @@ set -e set -x
requires test "x$QEMU_STORAGE_DAEMON" != "x" -requires $QEMU_STORAGE_DAEMON --version +requires sed --version +qsd_version="$($QEMU_STORAGE_DAEMON --version | sed -n '1s/qemu-storage-daemon version \([0-9.]*\) .*/\1/p')"
Long line, you could use backslash-newline to wrap it
+requires_not test "$qsd_version" = "6.0.0" +requires_not test "$qsd_version" = "5.1.0"
Yeah, a bit ugly (especially once downstreams start shipping a variant of 6.0 with the patch backported), but does the job. Could we instead make it a feature check, by attempting to fire up qemu-storage-daemon with socket activation and checking for the known failure mode? -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
On Tue, Jun 08, 2021 at 07:47:38AM -0500, Eric Blake wrote:
On Tue, Jun 08, 2021 at 09:53:43AM +0200, Martin Kletzander wrote:
The version 6.0.0 has very often a bug that makes the test fail, but which should be fixed in later versions. Version 5.1.0 has an issue as well. The version check is not the nicest way to check for this, but is enough.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
asdf
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Bogus line and double s-o-b is odd.
squash leftovers, thanks for noticing
--- interop/interop-qemu-storage-daemon.sh | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/interop/interop-qemu-storage-daemon.sh b/interop/interop-qemu-storage-daemon.sh index 09d1898ef459..dc1efcdf4ca1 100755 --- a/interop/interop-qemu-storage-daemon.sh +++ b/interop/interop-qemu-storage-daemon.sh @@ -23,7 +23,10 @@ set -e set -x
requires test "x$QEMU_STORAGE_DAEMON" != "x" -requires $QEMU_STORAGE_DAEMON --version +requires sed --version +qsd_version="$($QEMU_STORAGE_DAEMON --version | sed -n '1s/qemu-storage-daemon version \([0-9.]*\) .*/\1/p')"
Long line, you could use backslash-newline to wrap it
Yes, there is also one character to be removed, the space after the version group, because on some systems there is some build info appended and on some the line ends after the micro number of the version.
+requires_not test "$qsd_version" = "6.0.0" +requires_not test "$qsd_version" = "5.1.0"
Yeah, a bit ugly (especially once downstreams start shipping a variant of 6.0 with the patch backported), but does the job.
Could we instead make it a feature check, by attempting to fire up qemu-storage-daemon with socket activation and checking for the known failure mode?
I guess so, but Rich would be the one to know whether that is testable in some easy way or not, because the only thing I know is that these two versions are, allegedly, broken.
-- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
On Tue, Jun 08, 2021 at 02:59:28PM +0200, Martin Kletzander wrote:
On Tue, Jun 08, 2021 at 07:47:38AM -0500, Eric Blake wrote:
On Tue, Jun 08, 2021 at 09:53:43AM +0200, Martin Kletzander wrote:
+requires_not test "$qsd_version" = "6.0.0" +requires_not test "$qsd_version" = "5.1.0"
Yeah, a bit ugly (especially once downstreams start shipping a variant of 6.0 with the patch backported), but does the job.
Could we instead make it a feature check, by attempting to fire up qemu-storage-daemon with socket activation and checking for the known failure mode?
I guess so, but Rich would be the one to know whether that is testable in some easy way or not, because the only thing I know is that these two versions are, allegedly, broken.
I couldn't really come up with anything that would be simpler than what Martin has proposed above. q-s-d is not very composable :-( Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top
Platforms that utilise python-exec [0] wrapper scripts (mainly Gentoo) utilise basenames of python scripts and executables to decide which python implementation to use for running that particular command. That makes the nbdsh "hack" in sh/nbdsh.in not work, even if 'exec -a' is supported by sh. Signed-off-by: Martin Kletzander <mkletzan@redhat.com> --- sh/nbdsh.in | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/sh/nbdsh.in b/sh/nbdsh.in index d10f0c1b6b26..333c2d5f8b96 100644 --- a/sh/nbdsh.in +++ b/sh/nbdsh.in @@ -19,7 +19,8 @@ # Test if /bin/sh supports exec -a option (only supported in bash 4.2 # and above, and not part of POSIX). -if /bin/sh -c 'exec -a test true' 2>/dev/null; then +if /bin/sh -c 'exec -a test true' 2>/dev/null && \ + test $(readlink $(which @PYTHON@)) != python-exec2c; then exec -a nbdsh @PYTHON@ -mnbd "$@" else exec @PYTHON@ -mnbd "$@" -- 2.31.1
On Tue, Jun 08, 2021 at 09:53:44AM +0200, Martin Kletzander wrote:
Platforms that utilise python-exec [0] wrapper scripts (mainly Gentoo) utilise
Did you mean "[0]" to reference a URL?
basenames of python scripts and executables to decide which python implementation to use for running that particular command. That makes the nbdsh "hack" in sh/nbdsh.in not work, even if 'exec -a' is supported by sh.
It's annoying that we have to work around Gentoo here by adding a dependency on coreutils. I suppose FreeBSD does have readlink(1). But my feeling is that the Gentoo script is broken if it cannot cope with an unexpected basename. Would a configure-time test be better here? It seems like it would have less chance of breaking on the end-user machine (I know that on Gentoo, "end-user" is a distinction without a difference but it would make nbdsh more robust on non-Gentoo). Also it would mean the package didn't need to depend on coreutils. Rich.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com> --- sh/nbdsh.in | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/sh/nbdsh.in b/sh/nbdsh.in index d10f0c1b6b26..333c2d5f8b96 100644 --- a/sh/nbdsh.in +++ b/sh/nbdsh.in @@ -19,7 +19,8 @@
# Test if /bin/sh supports exec -a option (only supported in bash 4.2 # and above, and not part of POSIX). -if /bin/sh -c 'exec -a test true' 2>/dev/null; then +if /bin/sh -c 'exec -a test true' 2>/dev/null && \ + test $(readlink $(which @PYTHON@)) != python-exec2c; then exec -a nbdsh @PYTHON@ -mnbd "$@" else exec @PYTHON@ -mnbd "$@" -- 2.31.1
_______________________________________________ Libguestfs mailing list Libguestfs@redhat.com https://listman.redhat.com/mailman/listinfo/libguestfs
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
On Tue, Jun 08, 2021 at 10:50:35AM +0100, Richard W.M. Jones wrote:
On Tue, Jun 08, 2021 at 09:53:44AM +0200, Martin Kletzander wrote:
Platforms that utilise python-exec [0] wrapper scripts (mainly Gentoo) utilise
Did you mean "[0]" to reference a URL?
basenames of python scripts and executables to decide which python implementation to use for running that particular command. That makes the nbdsh "hack" in sh/nbdsh.in not work, even if 'exec -a' is supported by sh.
It's annoying that we have to work around Gentoo here by adding a dependency on coreutils. I suppose FreeBSD does have readlink(1).
Yes, coreutils are everywhere and the dependency is even documented in the libnbd readme, so that should not be an issue. Of course the check can be done in various ways, even just the mere presence of the wrapper could be checked with the usual `python-exec2c --version`. Come to think of it, that might actually be nicer.
But my feeling is that the Gentoo script is broken if it cannot cope with an unexpected basename.
Well, it's not "broken", but more of a "non-standard". This is the way they made it work to have any python package installed with any python interpreter version. All the binaries and scripts are symlinks to python-exec2 and that decides whether to run python 3.6 or 3.9 (or historically 2.7 for example). For rolling distro it is a bit more complex to manage it. But that is not really the point, sorry. Anyway, it does almost the same thing as you, except it checks for which python implementation the package is installed and it does that in python (with some bells and whistles).
Would a configure-time test be better here? It seems like it would have less chance of breaking on the end-user machine (I know that on Gentoo, "end-user" is a distinction without a difference but it would make nbdsh more robust on non-Gentoo). Also it would mean the package didn't need to depend on coreutils.
What could actually be the best solution for this if the nbdsh.in is just changed to: #!/usr/bin/env @PYTHON@ import nbdsh nbdsh.shell() or you can also use runpy: https://docs.python.org/3/library/runpy.html We can drop this patch until we figure this out. It is not that important. But having it behave similarly to above would also remove the "non-standardiness" of nbdsh.
Rich.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com> --- sh/nbdsh.in | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/sh/nbdsh.in b/sh/nbdsh.in index d10f0c1b6b26..333c2d5f8b96 100644 --- a/sh/nbdsh.in +++ b/sh/nbdsh.in @@ -19,7 +19,8 @@
# Test if /bin/sh supports exec -a option (only supported in bash 4.2 # and above, and not part of POSIX). -if /bin/sh -c 'exec -a test true' 2>/dev/null; then +if /bin/sh -c 'exec -a test true' 2>/dev/null && \ + test $(readlink $(which @PYTHON@)) != python-exec2c; then exec -a nbdsh @PYTHON@ -mnbd "$@" else exec @PYTHON@ -mnbd "$@" -- 2.31.1
_______________________________________________ Libguestfs mailing list Libguestfs@redhat.com https://listman.redhat.com/mailman/listinfo/libguestfs
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
It does not support `--version` parameter (at least on some platforms). Signed-off-by: Martin Kletzander <mkletzan@redhat.com> --- copy/copy-nbd-to-file.sh | 2 +- copy/copy-nbd-to-nbd.sh | 2 +- copy/copy-tls.sh | 2 +- fuse/test-pattern.sh | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/copy/copy-nbd-to-file.sh b/copy/copy-nbd-to-file.sh index a7d37303e05f..33f215adcdff 100755 --- a/copy/copy-nbd-to-file.sh +++ b/copy/copy-nbd-to-file.sh @@ -22,7 +22,7 @@ set -e set -x requires nbdkit --exit-with-parent --version -requires hexdump --version +requires hexdump -C /dev/null requires stat --version file=copy-nbd-to-file.file diff --git a/copy/copy-nbd-to-nbd.sh b/copy/copy-nbd-to-nbd.sh index 9afddadd621c..60c535256091 100755 --- a/copy/copy-nbd-to-nbd.sh +++ b/copy/copy-nbd-to-nbd.sh @@ -23,7 +23,7 @@ set -x requires nbdkit --exit-with-parent --version requires cmp /dev/null /dev/null -requires hexdump --version +requires hexdump -C /dev/null pidfile1=copy-nbd-to-nbd.pid1 pidfile2=copy-nbd-to-nbd.pid2 diff --git a/copy/copy-tls.sh b/copy/copy-tls.sh index 0352a05f168f..a9183b7854d7 100755 --- a/copy/copy-tls.sh +++ b/copy/copy-tls.sh @@ -29,7 +29,7 @@ requires test "x$PSKTOOL" != "x" requires nbdkit --exit-with-parent --version requires bash -c "nbdkit --dump-config | grep tls=yes" requires cmp /dev/null /dev/null -requires hexdump --version +requires hexdump -C /dev/null pskfile=copy-tls.psk pidfile1=copy-tls.pid1 diff --git a/fuse/test-pattern.sh b/fuse/test-pattern.sh index 44ecaa204bf6..f32f9d093285 100755 --- a/fuse/test-pattern.sh +++ b/fuse/test-pattern.sh @@ -27,7 +27,7 @@ set -x requires fusermount3 --version requires dd --version requires dd iflag=count_bytes,skip_bytes </dev/null -requires hexdump --version +requires hexdump -C /dev/null requires nbdkit --version requires nbdkit pattern --version -- 2.31.1
It was already different in copy/ and info/, so it is nicer to merge them together. Signed-off-by: Martin Kletzander <mkletzan@redhat.com> --- tests/functions.sh.in | 19 +++++++++++++++++++ copy/copy-file-to-qcow2.sh | 16 +++------------- info/Makefile.am | 1 + info/info-description-qemu.sh | 17 +++-------------- info/info-list-json-qemu.sh | 17 +++-------------- info/info-list-qemu.sh | 17 +++-------------- info/info-map-qemu-allocation-depth.sh | 16 +++------------- info/info-map-qemu-dirty-bitmap.sh | 23 +++-------------------- 8 files changed, 38 insertions(+), 88 deletions(-) diff --git a/tests/functions.sh.in b/tests/functions.sh.in index f42c6906d0f8..271d8b357685 100644 --- a/tests/functions.sh.in +++ b/tests/functions.sh.in @@ -158,3 +158,22 @@ pick_unused_port () done echo picked unused port $port } + +# Wait for qemu-nbd to start up. +wait_for_pidfile () +{ + local binary=$1 + local pid=$2 + + for i in {1..60}; do + if test -f $pid; then + break + fi + sleep 1 + done + + if ! test -f $pid; then + echo "$0: $binary did not start up" + exit 1 + fi +} diff --git a/copy/copy-file-to-qcow2.sh b/copy/copy-file-to-qcow2.sh index 57c18b383bb7..dfd870b76413 100755 --- a/copy/copy-file-to-qcow2.sh +++ b/copy/copy-file-to-qcow2.sh @@ -21,7 +21,7 @@ set -e set -x -requires $QEMU_NBD --version +requires $QEMU_NBD --pid-file=test.pid --version requires qemu-img --version requires cmp --version requires dd --version @@ -54,20 +54,10 @@ qemu-img create -f qcow2 $qcow2 $size # Run qemu-nbd as a separate process so that we can copy to and from # the single process in two separate operations. -qemu-nbd -f qcow2 -t --socket=$sock --pid-file=$pidfile $qcow2 & +$QEMU_NBD -f qcow2 -t --socket=$sock --pid-file=$pidfile $qcow2 & cleanup_fn kill $! -# Wait for qemu-nbd to start up. -for i in {1..60}; do - if test -f $pidfile; then - break - fi - sleep 1 -done -if ! test -f $pidfile; then - echo "$0: qemu-nbd did not start up" - exit 1 -fi +wait_for_pidfile qemu-nbd $pid $VG nbdcopy $file "nbd+unix:///?socket=$sock" $VG nbdcopy "nbd+unix:///?socket=$sock" $file2 diff --git a/info/Makefile.am b/info/Makefile.am index c62d51b7b067..5c717c700b35 100644 --- a/info/Makefile.am +++ b/info/Makefile.am @@ -54,6 +54,7 @@ TESTS_ENVIRONMENT = \ MALLOC_CHECK_=1 \ MALLOC_PERTURB_=$(shell bash -c 'echo $$(( 1 + (RANDOM & 255) ))') \ EXPECTED_VERSION=$(VERSION) \ + QEMU_NBD=$(QEMU_NBD) \ $(NULL) LOG_COMPILER = $(top_builddir)/run TESTS = diff --git a/info/info-description-qemu.sh b/info/info-description-qemu.sh index af2bd96b1cf6..e86fbe0eda5d 100755 --- a/info/info-description-qemu.sh +++ b/info/info-description-qemu.sh @@ -21,8 +21,7 @@ set -e set -x -requires qemu-nbd --version -requires bash -c 'qemu-nbd --help | grep pid-file' +requires $QEMU_NBD --pid-file=test.pid --version requires truncate --version img=info-description-qemu.img @@ -33,20 +32,10 @@ cleanup_fn rm -f $img $out $pid $sock rm -f $img $out $pid $sock truncate -s 1M $img -qemu-nbd -f raw -t --socket=$sock --pid-file=$pid -x "hello" -D "world" $img & +$QEMU_NBD -f raw -t --socket=$sock --pid-file=$pid -x "hello" -D "world" $img & cleanup_fn kill $! -# Wait for qemu-nbd to start up. -for i in {1..60}; do - if test -f $pid; then - break - fi - sleep 1 -done -if ! test -f $pid; then - echo "$0: qemu-nbd did not start up" - exit 1 -fi +wait_for_pidfile qemu-nbd $pid $VG nbdinfo "nbd+unix:///hello?socket=$sock" > $out cat $out diff --git a/info/info-list-json-qemu.sh b/info/info-list-json-qemu.sh index 9933c38a2c1d..06e8f4994ec2 100755 --- a/info/info-list-json-qemu.sh +++ b/info/info-list-json-qemu.sh @@ -21,8 +21,7 @@ set -e set -x -requires qemu-nbd --version -requires bash -c 'qemu-nbd --help | grep pid-file' +requires $QEMU_NBD --pid-file=test.pid --version requires truncate --version requires jq --version @@ -34,20 +33,10 @@ cleanup_fn rm -f $img $out $pid $sock rm -f $img $out $pid $sock truncate -s 1M $img -qemu-nbd -t --socket=$sock --pid-file=$pid -x "hello" -D "world" $img & +$QEMU_NBD -t --socket=$sock --pid-file=$pid -x "hello" -D "world" $img & cleanup_fn kill $! -# Wait for qemu-nbd to start up. -for i in {1..60}; do - if test -f $pid; then - break - fi - sleep 1 -done -if ! test -f $pid; then - echo "$0: qemu-nbd did not start up" - exit 1 -fi +wait_for_pidfile qemu-nbd $pid # Test twice, once with an export name not on the list,... $VG nbdinfo "nbd+unix://?socket=$sock" --list --json > $out diff --git a/info/info-list-qemu.sh b/info/info-list-qemu.sh index a6dcd4248b5f..fb6f57bcccf3 100755 --- a/info/info-list-qemu.sh +++ b/info/info-list-qemu.sh @@ -21,8 +21,7 @@ set -e set -x -requires qemu-nbd --version -requires bash -c 'qemu-nbd --help | grep pid-file' +requires $QEMU_NBD --pid-file=test.pid --version requires truncate --version img=info-list-qemu.img @@ -33,20 +32,10 @@ cleanup_fn rm -f $img $out $pid $sock rm -f $img $out $pid $sock truncate -s 1M $img -qemu-nbd -t --socket=$sock --pid-file=$pid -x "hello" -D "world" $img & +$QEMU_NBD -t --socket=$sock --pid-file=$pid -x "hello" -D "world" $img & cleanup_fn kill $! -# Wait for qemu-nbd to start up. -for i in {1..60}; do - if test -f $pid; then - break - fi - sleep 1 -done -if ! test -f $pid; then - echo "$0: qemu-nbd did not start up" - exit 1 -fi +wait_for_pidfile qemu-nbd $pid # Test twice, once with an export name not on the list,... $VG nbdinfo "nbd+unix://?socket=$sock" --list > $out diff --git a/info/info-map-qemu-allocation-depth.sh b/info/info-map-qemu-allocation-depth.sh index a5dab0b83c52..0bbe07c2a73a 100755 --- a/info/info-map-qemu-allocation-depth.sh +++ b/info/info-map-qemu-allocation-depth.sh @@ -26,7 +26,7 @@ set -x requires qemu-img --version requires qemu-io --version -requires qemu-nbd -A --version +requires $QEMU_NBD -A --pid-file=test.pid --version requires tr --version base=info-map-qemu-allocation-depth @@ -59,20 +59,10 @@ pid=$base.pid cleanup_fn rm -f $sock $pid rm -f $sock $pid -qemu-nbd -t --socket=$sock --pid-file=$pid -f qcow2 -A $f4 & +$QEMU_NBD -t --socket=$sock --pid-file=$pid -f qcow2 -A $f4 & cleanup_fn kill $! -# Wait for qemu-nbd to start up. -for i in {1..60}; do - if test -f $pid; then - break - fi - sleep 1 -done -if ! test -f $pid; then - echo "$0: qemu-nbd did not start up" - exit 1 -fi +wait_for_pidfile qemu-nbd $pid $VG nbdinfo --map=qemu:allocation-depth "nbd+unix://?socket=$sock" > $out cat $out diff --git a/info/info-map-qemu-dirty-bitmap.sh b/info/info-map-qemu-dirty-bitmap.sh index 01b62c1305a2..d9a6a5f143fc 100755 --- a/info/info-map-qemu-dirty-bitmap.sh +++ b/info/info-map-qemu-dirty-bitmap.sh @@ -26,17 +26,10 @@ set -x requires qemu-img --version requires qemu-io --version -requires qemu-nbd --version -requires bash -c 'qemu-nbd --help | grep pid-file' +requires $QEMU_NBD -B test --pid-file=test.pid --version requires_qemu requires tr --version -# This test uses the qemu-nbd -B option. -if ! qemu-nbd --help | grep -sq -- -B; then - echo "$0: skipping because qemu-nbd does not support the -B option" - exit 77 -fi - f=info-map-qemu-dirty-bitmap.qcow2 out=info-map-qemu-dirty-bitmap.out cleanup_fn rm -f $f $out @@ -60,20 +53,10 @@ pid=info-map-qemu-dirty-bitmap.pid cleanup_fn rm -f $sock $pid rm -f $sock $pid -qemu-nbd -t --socket=$sock --pid-file=$pid -f qcow2 -B bitmap0 $f & +$QEMU_NBD -t --socket=$sock --pid-file=$pid -f qcow2 -B bitmap0 $f & cleanup_fn kill $! -# Wait for qemu-nbd to start up. -for i in {1..60}; do - if test -f $pid; then - break - fi - sleep 1 -done -if ! test -f $pid; then - echo "$0: qemu-nbd did not start up" - exit 1 -fi +wait_for_pidfile qemu-nbd $pid $VG nbdinfo --map=qemu:dirty-bitmap:bitmap0 "nbd+unix://?socket=$sock" > $out cat $out -- 2.31.1
On Tue, Jun 08, 2021 at 09:53:46AM +0200, Martin Kletzander wrote:
diff --git a/copy/copy-file-to-qcow2.sh b/copy/copy-file-to-qcow2.sh index 57c18b383bb7..dfd870b76413 100755 --- a/copy/copy-file-to-qcow2.sh +++ b/copy/copy-file-to-qcow2.sh @@ -21,7 +21,7 @@ set -e set -x
-requires $QEMU_NBD --version +requires $QEMU_NBD --pid-file=test.pid --version requires qemu-img --version requires cmp --version requires dd --version
I think I fixed this one at the same time that you did :-) My fix probably conflicts on this hunk. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-builder quickly builds VMs from scratch http://libguestfs.org/virt-builder.1.html
On Tue, Jun 08, 2021 at 10:52:35AM +0100, Richard W.M. Jones wrote:
On Tue, Jun 08, 2021 at 09:53:46AM +0200, Martin Kletzander wrote:
diff --git a/copy/copy-file-to-qcow2.sh b/copy/copy-file-to-qcow2.sh index 57c18b383bb7..dfd870b76413 100755 --- a/copy/copy-file-to-qcow2.sh +++ b/copy/copy-file-to-qcow2.sh @@ -21,7 +21,7 @@ set -e set -x
-requires $QEMU_NBD --version +requires $QEMU_NBD --pid-file=test.pid --version requires qemu-img --version requires cmp --version requires dd --version
I think I fixed this one at the same time that you did :-) My fix probably conflicts on this hunk.
Yes. I rebased my branch on top of yours in the meantime and fixed the conflict.
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-builder quickly builds VMs from scratch http://libguestfs.org/virt-builder.1.html
Signed-off-by: Martin Kletzander <mkletzan@redhat.com> --- tests/functions.sh.in | 10 ++++++++++ fuse/test-errors.sh | 2 +- fuse/test-file-mode.sh | 3 +-- fuse/test-nbdkit-command.sh | 2 +- fuse/test-nbdkit-file-null.sh | 2 +- fuse/test-nbdkit.sh | 3 +-- fuse/test-parallel.sh | 2 +- fuse/test-pattern.sh | 2 +- fuse/test-qcow2.sh | 3 +-- fuse/test-trim.sh | 2 +- fuse/test-unix.sh | 2 +- fuse/test-zero.sh | 2 +- 12 files changed, 21 insertions(+), 14 deletions(-) diff --git a/tests/functions.sh.in b/tests/functions.sh.in index 271d8b357685..6f18f06238d5 100644 --- a/tests/functions.sh.in +++ b/tests/functions.sh.in @@ -118,6 +118,16 @@ requires_linux_kernel_version () requires test "$(printf "$kver\n$min" | sort -V | head -n 1)" = "$min" } +# requires_fuse +# +# Check for fusermount3 and accessibility of /dev/fuse. That is not the best +# way to check that it is going to work, but so far it is enough. +requires_fuse () +{ + requires test -r /dev/fuse + requires fusermount3 --version +} + # Tests that run under check-root should use this. requires_root () { diff --git a/fuse/test-errors.sh b/fuse/test-errors.sh index 2a08eec50971..4827a4756fd5 100755 --- a/fuse/test-errors.sh +++ b/fuse/test-errors.sh @@ -23,7 +23,7 @@ set -e set -x -requires fusermount3 --version +requires_fuse requires dd --version requires dd iflag=count_bytes,skip_bytes </dev/null requires nbdkit --version diff --git a/fuse/test-file-mode.sh b/fuse/test-file-mode.sh index 7c684805eeff..19ce030faf4f 100755 --- a/fuse/test-file-mode.sh +++ b/fuse/test-file-mode.sh @@ -23,11 +23,10 @@ set -e set -x -requires fusermount3 --version +requires_fuse requires nbdkit --exit-with-parent --version requires cmp --version requires dd --version -requires test -r /dev/fuse if ! test -r /dev/urandom; then echo "$0: test skipped: /dev/urandom not readable" diff --git a/fuse/test-nbdkit-command.sh b/fuse/test-nbdkit-command.sh index 3fdabb740837..a743d60b5989 100755 --- a/fuse/test-nbdkit-command.sh +++ b/fuse/test-nbdkit-command.sh @@ -23,7 +23,7 @@ set -e set -x -requires fusermount3 --version +requires_fuse requires nbdkit --exit-with-parent --version requires cmp --version requires dd --version diff --git a/fuse/test-nbdkit-file-null.sh b/fuse/test-nbdkit-file-null.sh index f7b8ed29b0f6..6454b1e0c687 100755 --- a/fuse/test-nbdkit-file-null.sh +++ b/fuse/test-nbdkit-file-null.sh @@ -23,7 +23,7 @@ set -e set -x -requires fusermount3 --version +requires_fuse requires nbdkit --exit-with-parent --version pidfile=test-nbdkit-file-null.pid diff --git a/fuse/test-nbdkit.sh b/fuse/test-nbdkit.sh index fd0c9ce72356..047835ae7ec9 100755 --- a/fuse/test-nbdkit.sh +++ b/fuse/test-nbdkit.sh @@ -23,11 +23,10 @@ set -e set -x -requires fusermount3 --version +requires_fuse requires nbdkit --exit-with-parent --version requires cmp --version requires dd --version -requires test -r /dev/fuse if ! test -r /dev/urandom; then echo "$0: test skipped: /dev/urandom not readable" diff --git a/fuse/test-parallel.sh b/fuse/test-parallel.sh index 076c9a2b8482..55bb7c8e8e7d 100755 --- a/fuse/test-parallel.sh +++ b/fuse/test-parallel.sh @@ -23,7 +23,7 @@ set -e set -x -requires fusermount3 --version +requires_fuse requires dd --version requires nbdkit --version requires nbdkit memory --version diff --git a/fuse/test-pattern.sh b/fuse/test-pattern.sh index f32f9d093285..1ff06e3eb095 100755 --- a/fuse/test-pattern.sh +++ b/fuse/test-pattern.sh @@ -24,7 +24,7 @@ set -e set -x -requires fusermount3 --version +requires_fuse requires dd --version requires dd iflag=count_bytes,skip_bytes </dev/null requires hexdump -C /dev/null diff --git a/fuse/test-qcow2.sh b/fuse/test-qcow2.sh index 0c5836ad24f1..0545d1d8a933 100755 --- a/fuse/test-qcow2.sh +++ b/fuse/test-qcow2.sh @@ -24,12 +24,11 @@ set -e set -x -requires fusermount3 --version +requires_fuse requires qemu-nbd --version requires qemu-img --version requires cmp --version requires dd --version -requires test -r /dev/fuse if ! test -r /dev/urandom; then echo "$0: test skipped: /dev/urandom not readable" diff --git a/fuse/test-trim.sh b/fuse/test-trim.sh index 824dad79d16f..c845ba900881 100755 --- a/fuse/test-trim.sh +++ b/fuse/test-trim.sh @@ -23,7 +23,7 @@ set -e set -x -requires fusermount3 --version +requires_fuse requires fallocate --version requires dd --version requires nbdkit --version diff --git a/fuse/test-unix.sh b/fuse/test-unix.sh index 20179bd17694..0a962f1da1f1 100755 --- a/fuse/test-unix.sh +++ b/fuse/test-unix.sh @@ -23,7 +23,7 @@ set -e set -x -requires fusermount3 --version +requires_fuse requires nbdkit --exit-with-parent --version # Difficult to arrange for this test to be run this test under diff --git a/fuse/test-zero.sh b/fuse/test-zero.sh index e5de270d9a8c..bf9d1320a930 100755 --- a/fuse/test-zero.sh +++ b/fuse/test-zero.sh @@ -28,7 +28,7 @@ set -x # https://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse.git/log/?h=for... requires_linux_kernel_version 5.14 -requires fusermount3 --version +requires_fuse requires fallocate --version requires dd --version requires nbdkit --version -- 2.31.1
The shell/environment variable `$RANDOM` is not defined POSIX, so let bash evaluate the whole expression by changing double quotes to single quotes. Signed-off-by: Martin Kletzander <mkletzan@redhat.com> --- tests/Makefile.am | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/Makefile.am b/tests/Makefile.am index 6a3a1f092572..c65bf8be399e 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -608,7 +608,7 @@ TESTS += \ connect-uri-nbd-unix \ $(NULL) -RANDOM1 := $(shell bash -c "echo $$(( 32768 + (RANDOM & 16383) ))") +RANDOM1 := $(shell bash -c 'echo $$(( 32768 + (RANDOM & 16383) ))') connect_uri_nbd_SOURCES = connect-uri.c connect_uri_nbd_CPPFLAGS = \ -I$(top_srcdir)/include \ @@ -640,7 +640,7 @@ TESTS += \ connect-uri-nbds-unix \ $(NULL) -RANDOM2 := $(shell bash -c "echo $$(( 32768 + (RANDOM & 16383) ))") +RANDOM2 := $(shell bash -c 'echo $$(( 32768 + (RANDOM & 16383) ))') connect_uri_nbds_SOURCES = connect-uri.c connect_uri_nbds_CPPFLAGS = \ -I$(top_srcdir)/include \ @@ -671,7 +671,7 @@ TESTS += \ connect-uri-nbds-psk \ $(NULL) -RANDOM3 := $(shell bash -c "echo $$(( 32768 + (RANDOM & 16383) ))") +RANDOM3 := $(shell bash -c 'echo $$(( 32768 + (RANDOM & 16383) ))') connect_uri_nbds_psk_SOURCES = connect-uri.c connect_uri_nbds_psk_CPPFLAGS = \ -I$(top_srcdir)/include \ -- 2.31.1
On Tue, Jun 08, 2021 at 09:53:48AM +0200, Martin Kletzander wrote:
The shell/environment variable `$RANDOM` is not defined POSIX, so let bash evaluate the whole expression by changing double quotes to single quotes.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com> --- tests/Makefile.am | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/tests/Makefile.am b/tests/Makefile.am index 6a3a1f092572..c65bf8be399e 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -608,7 +608,7 @@ TESTS += \ connect-uri-nbd-unix \ $(NULL)
-RANDOM1 := $(shell bash -c "echo $$(( 32768 + (RANDOM & 16383) ))") +RANDOM1 := $(shell bash -c 'echo $$(( 32768 + (RANDOM & 16383) ))')
ACK. We absolutely want the $(()) run by bash, not by the shell invoking bash ;) -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Some tests check for root privileges, but do not check if they have the appropriate capabilities for all the tasks they are going to perform. Not being sure which ones are the proper ones all these currently check for cap_sys_admin, although the code is made so that it can check for any capability that is requested. Signed-off-by: Martin Kletzander <mkletzan@redhat.com> --- tests/functions.sh.in | 29 +++++++++++++++++++++++++++ copy/copy-block-to-nbd.sh | 1 + copy/copy-nbd-to-block.sh | 1 + copy/copy-nbd-to-small-block-error.sh | 1 + 4 files changed, 32 insertions(+) diff --git a/tests/functions.sh.in b/tests/functions.sh.in index 6f18f06238d5..3472f49b84e3 100644 --- a/tests/functions.sh.in +++ b/tests/functions.sh.in @@ -128,6 +128,35 @@ requires_fuse () requires fusermount3 --version } +# requires_caps +# +# Check for linux capabilities. Parameters are in the form of "cap_name", e.g. +# requires_caps cap_net_admin cap_chown +# +# This should be coupled with requires_root as it will not fail when capsh +# utility from libcapng is not installed or the capabilities are not found in +# /proc/<pid>/status (to future-proof this against non-Linux platforms). +requires_caps () +{ + test -r /proc/$$/status || return 0 + type capsh 2>/dev/null >&2 || return 0 + + local cap_eff + local cap_str + + cap_eff="$(sed -n 's/CapEff:\s*\([^0-9a-fA-F]*\)/\1/p' /proc/$$/status)" + test -z "$cap_eff" && return 0 + + cap_str=$(capsh --decode="$cap_eff") + while test "$#" -gt 0; do + if [[ ! "$cap_str" =~ [,=]$1(,|$) ]]; then + echo "$0: test skipped because of missing capability: $1" + exit 77 + fi + shift + done +} + # Tests that run under check-root should use this. requires_root () { diff --git a/copy/copy-block-to-nbd.sh b/copy/copy-block-to-nbd.sh index f29af6eb16b0..00d45427e967 100755 --- a/copy/copy-block-to-nbd.sh +++ b/copy/copy-block-to-nbd.sh @@ -22,6 +22,7 @@ set -e set -x requires_root +requires_caps cap_sys_admin requires nbdkit --exit-with-parent --version requires test -r /sys/module/nbd requires nbd-client --version diff --git a/copy/copy-nbd-to-block.sh b/copy/copy-nbd-to-block.sh index 4924af991586..b3e6402c38a6 100755 --- a/copy/copy-nbd-to-block.sh +++ b/copy/copy-nbd-to-block.sh @@ -22,6 +22,7 @@ set -e set -x requires_root +requires_caps cap_sys_admin requires nbdkit --exit-with-parent --version requires test -r /sys/module/nbd requires nbd-client --version diff --git a/copy/copy-nbd-to-small-block-error.sh b/copy/copy-nbd-to-small-block-error.sh index 5a3953a3b0ad..ef5da9a0df62 100755 --- a/copy/copy-nbd-to-small-block-error.sh +++ b/copy/copy-nbd-to-small-block-error.sh @@ -22,6 +22,7 @@ set -e set -x requires_root +requires_caps cap_sys_admin requires nbdkit --exit-with-parent --version requires test -r /sys/module/nbd requires nbd-client --version -- 2.31.1
The 'memory' and 'pattern' plugins gained support for that some time ago, however on some systems such nbdkit is not in the official repositories. This makes sure that the tests don't fail unnecessarily on such systems. Signed-off-by: Martin Kletzander <mkletzan@redhat.com> --- copy/copy-block-to-nbd.sh | 2 +- copy/copy-nbd-to-block.sh | 2 +- copy/copy-nbd-to-small-block-error.sh | 4 ++-- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/copy/copy-block-to-nbd.sh b/copy/copy-block-to-nbd.sh index 00d45427e967..6894e17dd79e 100755 --- a/copy/copy-block-to-nbd.sh +++ b/copy/copy-block-to-nbd.sh @@ -35,7 +35,7 @@ cleanup_fn rm -f $pidfile $sock cleanup_fn nbd-client -d /dev/nbd0 # Run an nbdkit server to act as the backing for /dev/nbd0. -nbdkit --exit-with-parent -f -v -P $pidfile -U $sock pattern 5M & +nbdkit --exit-with-parent -f -v -P $pidfile -U $sock pattern size=5M & # Wait for the pidfile to appear. for i in {1..60}; do if test -f $pidfile; then diff --git a/copy/copy-nbd-to-block.sh b/copy/copy-nbd-to-block.sh index b3e6402c38a6..9961c3f7bd6b 100755 --- a/copy/copy-nbd-to-block.sh +++ b/copy/copy-nbd-to-block.sh @@ -50,7 +50,7 @@ fi nbd-client -unix $sock /dev/nbd0 -b 512 -$VG nbdcopy -- [ nbdkit --exit-with-parent -v pattern 5M ] /dev/nbd0 +$VG nbdcopy -- [ nbdkit --exit-with-parent -v pattern size=5M ] /dev/nbd0 # Check that /dev/nbd0 is still a block device and we didn't # accidentally overwrite it with a regular file. diff --git a/copy/copy-nbd-to-small-block-error.sh b/copy/copy-nbd-to-small-block-error.sh index ef5da9a0df62..75032e5b0d49 100755 --- a/copy/copy-nbd-to-small-block-error.sh +++ b/copy/copy-nbd-to-small-block-error.sh @@ -36,8 +36,8 @@ sock2=$(mktemp -u /tmp/libnbd-test-copy.XXXXXX) cleanup_fn rm -f $pidfile1 $pidfile2 $sock1 $sock2 cleanup_fn nbd-client -d /dev/nbd0 -nbdkit --exit-with-parent -f -v -P $pidfile1 -U $sock1 pattern 10M & -nbdkit --exit-with-parent -f -v -P $pidfile2 -U $sock2 memory 5M & +nbdkit --exit-with-parent -f -v -P $pidfile1 -U $sock1 pattern size=10M & +nbdkit --exit-with-parent -f -v -P $pidfile2 -U $sock2 memory size=5M & # Wait for the pidfiles to appear. for i in {1..60}; do if test -f $pidfile1 && test -f $pidfile2; then -- 2.31.1
Logs are by default kept for 1 week, and when we output them directly, we sometimes exceed the limit for the job. This also makes the output cleaner. Docker driver overlayfs2 should be faster for docker dind builds. Signed-off-by: Martin Kletzander <mkletzan@redhat.com> --- .gitlab-ci.yml | 30 ++++++++++++++++-------------- ci/build_script.sh | 7 +------ 2 files changed, 17 insertions(+), 20 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 07d81ca779df..74807381b8ee 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,5 +1,6 @@ variables: GIT_DEPTH: 100 + DOCKER_DRIVER: overlay2 stages: - containers @@ -32,9 +33,8 @@ stages: after_script: - docker logout -.native_build_job: +.base_build_job: stage: builds - image: $CI_REGISTRY_IMAGE/$NAME:latest cache: paths: - ccache/ @@ -43,6 +43,20 @@ stages: - *script_variables script: - ci/build_script.sh + artifacts: + paths: + - "config.log" + - "**/test-suite.log" + public: true + when: always + +.native_build_job: + extends: .base_build_job + image: $CI_REGISTRY_IMAGE/$NAME:latest + +.cross_build_job: + extends: .base_build_job + image: $CI_REGISTRY_IMAGE/$NAME-cross-$CROSS:latest # Jobs that we delegate to Cirrus CI because they require an operating # system other than Linux. These jobs will only run if the required @@ -83,18 +97,6 @@ stages: rules: - if: "$CIRRUS_GITHUB_REPO && $CIRRUS_API_TOKEN" -.cross_build_job: - stage: builds - image: $CI_REGISTRY_IMAGE/$NAME-cross-$CROSS:latest - cache: - paths: - - ccache/ - key: "$CI_JOB_NAME" - before_script: - - *script_variables - script: - - ci/build_script.sh - # Native container build jobs x64-centos-8-container: diff --git a/ci/build_script.sh b/ci/build_script.sh index ae236640449f..8e2cb5b986ca 100755 --- a/ci/build_script.sh +++ b/ci/build_script.sh @@ -29,12 +29,7 @@ main() { return 0 fi - if ! $MAKE check - then - find . -name test-suite.log -exec grep -l '^X\?FAIL:' '{}' \+ | \ - xargs cat - exit 1 - fi + $MAKE check if test "$CHECK_VALGRIND" = "force" then -- 2.31.1
Signed-off-by: Martin Kletzander <mkletzan@redhat.com> --- .gitlab-ci.yml | 203 ------------------ .../debian-10-cross-aarch64.Dockerfile | 86 -------- .../debian-10-cross-armv6l.Dockerfile | 86 -------- .../debian-10-cross-armv7l.Dockerfile | 86 -------- ci/containers/debian-10-cross-i686.Dockerfile | 86 -------- ci/containers/debian-10-cross-mips.Dockerfile | 86 -------- .../debian-10-cross-mips64el.Dockerfile | 86 -------- .../debian-10-cross-mipsel.Dockerfile | 86 -------- .../debian-10-cross-ppc64le.Dockerfile | 86 -------- .../debian-10-cross-s390x.Dockerfile | 86 -------- .../debian-sid-cross-aarch64.Dockerfile | 86 -------- .../debian-sid-cross-armv6l.Dockerfile | 86 -------- .../debian-sid-cross-armv7l.Dockerfile | 86 -------- .../debian-sid-cross-i686.Dockerfile | 86 -------- .../debian-sid-cross-mips64el.Dockerfile | 86 -------- .../debian-sid-cross-mipsel.Dockerfile | 86 -------- .../debian-sid-cross-ppc64le.Dockerfile | 86 -------- .../debian-sid-cross-s390x.Dockerfile | 86 -------- .../fedora-rawhide-cross-mingw32.Dockerfile | 71 ------ .../fedora-rawhide-cross-mingw64.Dockerfile | 71 ------ ci/containers/refresh | 20 -- 21 files changed, 1827 deletions(-) delete mode 100644 ci/containers/debian-10-cross-aarch64.Dockerfile delete mode 100644 ci/containers/debian-10-cross-armv6l.Dockerfile delete mode 100644 ci/containers/debian-10-cross-armv7l.Dockerfile delete mode 100644 ci/containers/debian-10-cross-i686.Dockerfile delete mode 100644 ci/containers/debian-10-cross-mips.Dockerfile delete mode 100644 ci/containers/debian-10-cross-mips64el.Dockerfile delete mode 100644 ci/containers/debian-10-cross-mipsel.Dockerfile delete mode 100644 ci/containers/debian-10-cross-ppc64le.Dockerfile delete mode 100644 ci/containers/debian-10-cross-s390x.Dockerfile delete mode 100644 ci/containers/debian-sid-cross-aarch64.Dockerfile delete mode 100644 ci/containers/debian-sid-cross-armv6l.Dockerfile delete mode 100644 ci/containers/debian-sid-cross-armv7l.Dockerfile delete mode 100644 ci/containers/debian-sid-cross-i686.Dockerfile delete mode 100644 ci/containers/debian-sid-cross-mips64el.Dockerfile delete mode 100644 ci/containers/debian-sid-cross-mipsel.Dockerfile delete mode 100644 ci/containers/debian-sid-cross-ppc64le.Dockerfile delete mode 100644 ci/containers/debian-sid-cross-s390x.Dockerfile delete mode 100644 ci/containers/fedora-rawhide-cross-mingw32.Dockerfile delete mode 100644 ci/containers/fedora-rawhide-cross-mingw64.Dockerfile diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 74807381b8ee..cd2167c52ab8 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -54,10 +54,6 @@ stages: extends: .base_build_job image: $CI_REGISTRY_IMAGE/$NAME:latest -.cross_build_job: - extends: .base_build_job - image: $CI_REGISTRY_IMAGE/$NAME-cross-$CROSS:latest - # Jobs that we delegate to Cirrus CI because they require an operating # system other than Linux. These jobs will only run if the required # setup has been performed on the GitLab account (see ci/README.rst). @@ -154,112 +150,6 @@ x64-ubuntu-2004-container: variables: NAME: ubuntu-2004 - -# Cross-build containers build jobs - -aarch64-debian-10-container: - extends: .container_job - allow_failure: true - variables: - NAME: debian-10-cross-aarch64 - -armv6l-debian-10-container: - extends: .container_job - variables: - NAME: debian-10-cross-armv6l - -armv7l-debian-10-container: - extends: .container_job - variables: - NAME: debian-10-cross-armv7l - -i686-debian-10-container: - extends: .container_job - allow_failure: true - variables: - NAME: debian-10-cross-i686 - -mips-debian-10-container: - extends: .container_job - variables: - NAME: debian-10-cross-mips - -mips64el-debian-10-container: - extends: .container_job - allow_failure: true - variables: - NAME: debian-10-cross-mips64el - -mipsel-debian-10-container: - extends: .container_job - variables: - NAME: debian-10-cross-mipsel - -ppc64le-debian-10-container: - extends: .container_job - variables: - NAME: debian-10-cross-ppc64le - -s390x-debian-10-container: - extends: .container_job - allow_failure: true - variables: - NAME: debian-10-cross-s390x - -aarch64-debian-sid-container: - extends: .container_job - variables: - NAME: debian-sid-cross-aarch64 - -armv6l-debian-sid-container: - extends: .container_job - allow_failure: true - variables: - NAME: debian-sid-cross-armv6l - -armv7l-debian-sid-container: - extends: .container_job - allow_failure: true - variables: - NAME: debian-sid-cross-armv7l - -i686-debian-sid-container: - extends: .container_job - variables: - NAME: debian-sid-cross-i686 - -mips64el-debian-sid-container: - extends: .container_job - variables: - NAME: debian-sid-cross-mips64el - -mipsel-debian-sid-container: - extends: .container_job - allow_failure: true - variables: - NAME: debian-sid-cross-mipsel - -ppc64le-debian-sid-container: - extends: .container_job - allow_failure: true - variables: - NAME: debian-sid-cross-ppc64le - -s390x-debian-sid-container: - extends: .container_job - variables: - NAME: debian-sid-cross-s390x - -mingw32-fedora-rawhide-container: - extends: .container_job - variables: - NAME: fedora-rawhide-cross-mingw32 - -mingw64-fedora-rawhide-container: - extends: .container_job - variables: - NAME: fedora-rawhide-cross-mingw64 - # Native architecture build + test jobs x64-debian-10: @@ -386,96 +276,3 @@ x64-macos-11-build: INSTALL_COMMAND: brew install PATH_EXTRA: /usr/local/opt/ccache/libexec:/usr/local/opt/gettext/bin:/usr/local/opt/libpcap/bin:/usr/local/opt/libxslt/bin:/usr/local/opt/rpcgen/bin PKG_CONFIG_PATH: /usr/local/opt/curl/lib/pkgconfig:/usr/local/opt/libpcap/lib/pkgconfig:/usr/local/opt/libxml2/lib/pkgconfig:/usr/local/opt/ncurses/lib/pkgconfig:/usr/local/opt/readline/lib/pkgconfig - - -# Cross compiled build jobs - -aarch64-debian-sid: - extends: .cross_build_job - needs: - - aarch64-debian-sid-container - variables: - NAME: debian-sid - CROSS: aarch64 - -armv6l-debian-10: - extends: .cross_build_job - needs: - - armv6l-debian-10-container - variables: - NAME: debian-10 - CROSS: armv6l - -armv7l-debian-10: - extends: .cross_build_job - needs: - - armv7l-debian-10-container - variables: - NAME: debian-10 - CROSS: armv7l - -i686-debian-sid: - extends: .cross_build_job - needs: - - i686-debian-sid-container - variables: - NAME: debian-sid - CROSS: i686 - -mips-debian-10: - extends: .cross_build_job - needs: - - mips-debian-10-container - variables: - NAME: debian-10 - CROSS: mips - -mips64el-debian-sid: - extends: .cross_build_job - needs: - - mips64el-debian-sid-container - variables: - NAME: debian-sid - CROSS: mips64el - -mipsel-debian-10: - extends: .cross_build_job - needs: - - mipsel-debian-10-container - variables: - NAME: debian-10 - CROSS: mipsel - -ppc64le-debian-10: - extends: .cross_build_job - needs: - - ppc64le-debian-10-container - variables: - NAME: debian-10 - CROSS: ppc64le - -s390x-debian-sid: - extends: .cross_build_job - needs: - - s390x-debian-sid-container - variables: - NAME: debian-sid - CROSS: s390x - -mingw32-fedora-rawhide: - extends: .cross_build_job - allow_failure: true - needs: - - mingw32-fedora-rawhide-container - variables: - NAME: fedora-rawhide - CROSS: mingw32 - -mingw64-fedora-rawhide: - extends: .cross_build_job - allow_failure: true - needs: - - mingw64-fedora-rawhide-container - variables: - NAME: fedora-rawhide - CROSS: mingw64 diff --git a/ci/containers/debian-10-cross-aarch64.Dockerfile b/ci/containers/debian-10-cross-aarch64.Dockerfile deleted file mode 100644 index 5c19904dc45f..000000000000 --- a/ci/containers/debian-10-cross-aarch64.Dockerfile +++ /dev/null @@ -1,86 +0,0 @@ -# THIS FILE WAS AUTO-GENERATED -# -# $ lcitool dockerfile --cross aarch64 debian-10 libnbd -# -# https://gitlab.com/libvirt/libvirt-ci/-/commit/5fd2cdeb8958df86a81b639fd096f... - -FROM docker.io/library/debian:10-slim - -RUN export DEBIAN_FRONTEND=noninteractive && \ - apt-get update && \ - apt-get install -y eatmydata && \ - eatmydata apt-get dist-upgrade -y && \ - eatmydata apt-get install --no-install-recommends -y \ - autoconf \ - automake \ - bash-completion \ - ca-certificates \ - ccache \ - coreutils \ - diffutils \ - flake8 \ - git \ - gnutls-bin \ - golang \ - iproute2 \ - jq \ - libtool-bin \ - locales \ - make \ - nbd-client \ - nbd-server \ - nbdkit \ - ocaml \ - ocaml-findlib \ - perl-base \ - pkgconf \ - python3-dev \ - qemu-kvm \ - qemu-utils \ - sed && \ - eatmydata apt-get autoremove -y && \ - eatmydata apt-get autoclean -y && \ - sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ - dpkg-reconfigure locales && \ - dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt && \ - mkdir -p /usr/libexec/ccache-wrappers && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/aarch64-linux-gnu-c++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/aarch64-linux-gnu-cc && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/aarch64-linux-gnu-g++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/aarch64-linux-gnu-gcc - -RUN export DEBIAN_FRONTEND=noninteractive && \ - dpkg --add-architecture arm64 && \ - eatmydata apt-get update && \ - eatmydata apt-get dist-upgrade -y && \ - eatmydata apt-get install --no-install-recommends -y dpkg-dev && \ - eatmydata apt-get install --no-install-recommends -y \ - g++-aarch64-linux-gnu \ - gcc-aarch64-linux-gnu \ - libc6-dev:arm64 \ - libev-dev:arm64 \ - libfuse3-dev:arm64 \ - libglib2.0-dev:arm64 \ - libgnutls28-dev:arm64 \ - libxml2-dev:arm64 && \ - eatmydata apt-get autoremove -y && \ - eatmydata apt-get autoclean -y && \ - mkdir -p /usr/local/share/meson/cross && \ - echo "[binaries]\n\ -c = '/usr/bin/aarch64-linux-gnu-gcc'\n\ -ar = '/usr/bin/aarch64-linux-gnu-gcc-ar'\n\ -strip = '/usr/bin/aarch64-linux-gnu-strip'\n\ -pkgconfig = '/usr/bin/aarch64-linux-gnu-pkg-config'\n\ -\n\ -[host_machine]\n\ -system = 'linux'\n\ -cpu_family = 'aarch64'\n\ -cpu = 'aarch64'\n\ -endian = 'little'" > /usr/local/share/meson/cross/aarch64-linux-gnu - -ENV LANG "en_US.UTF-8" -ENV MAKE "/usr/bin/make" -ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" - -ENV ABI "aarch64-linux-gnu" -ENV CONFIGURE_OPTS "--host=aarch64-linux-gnu" diff --git a/ci/containers/debian-10-cross-armv6l.Dockerfile b/ci/containers/debian-10-cross-armv6l.Dockerfile deleted file mode 100644 index cfba6456170f..000000000000 --- a/ci/containers/debian-10-cross-armv6l.Dockerfile +++ /dev/null @@ -1,86 +0,0 @@ -# THIS FILE WAS AUTO-GENERATED -# -# $ lcitool dockerfile --cross armv6l debian-10 libnbd -# -# https://gitlab.com/libvirt/libvirt-ci/-/commit/5fd2cdeb8958df86a81b639fd096f... - -FROM docker.io/library/debian:10-slim - -RUN export DEBIAN_FRONTEND=noninteractive && \ - apt-get update && \ - apt-get install -y eatmydata && \ - eatmydata apt-get dist-upgrade -y && \ - eatmydata apt-get install --no-install-recommends -y \ - autoconf \ - automake \ - bash-completion \ - ca-certificates \ - ccache \ - coreutils \ - diffutils \ - flake8 \ - git \ - gnutls-bin \ - golang \ - iproute2 \ - jq \ - libtool-bin \ - locales \ - make \ - nbd-client \ - nbd-server \ - nbdkit \ - ocaml \ - ocaml-findlib \ - perl-base \ - pkgconf \ - python3-dev \ - qemu-kvm \ - qemu-utils \ - sed && \ - eatmydata apt-get autoremove -y && \ - eatmydata apt-get autoclean -y && \ - sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ - dpkg-reconfigure locales && \ - dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt && \ - mkdir -p /usr/libexec/ccache-wrappers && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/arm-linux-gnueabi-c++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/arm-linux-gnueabi-cc && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/arm-linux-gnueabi-g++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/arm-linux-gnueabi-gcc - -RUN export DEBIAN_FRONTEND=noninteractive && \ - dpkg --add-architecture armel && \ - eatmydata apt-get update && \ - eatmydata apt-get dist-upgrade -y && \ - eatmydata apt-get install --no-install-recommends -y dpkg-dev && \ - eatmydata apt-get install --no-install-recommends -y \ - g++-arm-linux-gnueabi \ - gcc-arm-linux-gnueabi \ - libc6-dev:armel \ - libev-dev:armel \ - libfuse3-dev:armel \ - libglib2.0-dev:armel \ - libgnutls28-dev:armel \ - libxml2-dev:armel && \ - eatmydata apt-get autoremove -y && \ - eatmydata apt-get autoclean -y && \ - mkdir -p /usr/local/share/meson/cross && \ - echo "[binaries]\n\ -c = '/usr/bin/arm-linux-gnueabi-gcc'\n\ -ar = '/usr/bin/arm-linux-gnueabi-gcc-ar'\n\ -strip = '/usr/bin/arm-linux-gnueabi-strip'\n\ -pkgconfig = '/usr/bin/arm-linux-gnueabi-pkg-config'\n\ -\n\ -[host_machine]\n\ -system = 'linux'\n\ -cpu_family = 'arm'\n\ -cpu = 'arm'\n\ -endian = 'little'" > /usr/local/share/meson/cross/arm-linux-gnueabi - -ENV LANG "en_US.UTF-8" -ENV MAKE "/usr/bin/make" -ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" - -ENV ABI "arm-linux-gnueabi" -ENV CONFIGURE_OPTS "--host=arm-linux-gnueabi" diff --git a/ci/containers/debian-10-cross-armv7l.Dockerfile b/ci/containers/debian-10-cross-armv7l.Dockerfile deleted file mode 100644 index 693ca11a0d5e..000000000000 --- a/ci/containers/debian-10-cross-armv7l.Dockerfile +++ /dev/null @@ -1,86 +0,0 @@ -# THIS FILE WAS AUTO-GENERATED -# -# $ lcitool dockerfile --cross armv7l debian-10 libnbd -# -# https://gitlab.com/libvirt/libvirt-ci/-/commit/5fd2cdeb8958df86a81b639fd096f... - -FROM docker.io/library/debian:10-slim - -RUN export DEBIAN_FRONTEND=noninteractive && \ - apt-get update && \ - apt-get install -y eatmydata && \ - eatmydata apt-get dist-upgrade -y && \ - eatmydata apt-get install --no-install-recommends -y \ - autoconf \ - automake \ - bash-completion \ - ca-certificates \ - ccache \ - coreutils \ - diffutils \ - flake8 \ - git \ - gnutls-bin \ - golang \ - iproute2 \ - jq \ - libtool-bin \ - locales \ - make \ - nbd-client \ - nbd-server \ - nbdkit \ - ocaml \ - ocaml-findlib \ - perl-base \ - pkgconf \ - python3-dev \ - qemu-kvm \ - qemu-utils \ - sed && \ - eatmydata apt-get autoremove -y && \ - eatmydata apt-get autoclean -y && \ - sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ - dpkg-reconfigure locales && \ - dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt && \ - mkdir -p /usr/libexec/ccache-wrappers && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/arm-linux-gnueabihf-c++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/arm-linux-gnueabihf-cc && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/arm-linux-gnueabihf-g++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/arm-linux-gnueabihf-gcc - -RUN export DEBIAN_FRONTEND=noninteractive && \ - dpkg --add-architecture armhf && \ - eatmydata apt-get update && \ - eatmydata apt-get dist-upgrade -y && \ - eatmydata apt-get install --no-install-recommends -y dpkg-dev && \ - eatmydata apt-get install --no-install-recommends -y \ - g++-arm-linux-gnueabihf \ - gcc-arm-linux-gnueabihf \ - libc6-dev:armhf \ - libev-dev:armhf \ - libfuse3-dev:armhf \ - libglib2.0-dev:armhf \ - libgnutls28-dev:armhf \ - libxml2-dev:armhf && \ - eatmydata apt-get autoremove -y && \ - eatmydata apt-get autoclean -y && \ - mkdir -p /usr/local/share/meson/cross && \ - echo "[binaries]\n\ -c = '/usr/bin/arm-linux-gnueabihf-gcc'\n\ -ar = '/usr/bin/arm-linux-gnueabihf-gcc-ar'\n\ -strip = '/usr/bin/arm-linux-gnueabihf-strip'\n\ -pkgconfig = '/usr/bin/arm-linux-gnueabihf-pkg-config'\n\ -\n\ -[host_machine]\n\ -system = 'linux'\n\ -cpu_family = 'arm'\n\ -cpu = 'armhf'\n\ -endian = 'little'" > /usr/local/share/meson/cross/arm-linux-gnueabihf - -ENV LANG "en_US.UTF-8" -ENV MAKE "/usr/bin/make" -ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" - -ENV ABI "arm-linux-gnueabihf" -ENV CONFIGURE_OPTS "--host=arm-linux-gnueabihf" diff --git a/ci/containers/debian-10-cross-i686.Dockerfile b/ci/containers/debian-10-cross-i686.Dockerfile deleted file mode 100644 index 5e3688f51766..000000000000 --- a/ci/containers/debian-10-cross-i686.Dockerfile +++ /dev/null @@ -1,86 +0,0 @@ -# THIS FILE WAS AUTO-GENERATED -# -# $ lcitool dockerfile --cross i686 debian-10 libnbd -# -# https://gitlab.com/libvirt/libvirt-ci/-/commit/5fd2cdeb8958df86a81b639fd096f... - -FROM docker.io/library/debian:10-slim - -RUN export DEBIAN_FRONTEND=noninteractive && \ - apt-get update && \ - apt-get install -y eatmydata && \ - eatmydata apt-get dist-upgrade -y && \ - eatmydata apt-get install --no-install-recommends -y \ - autoconf \ - automake \ - bash-completion \ - ca-certificates \ - ccache \ - coreutils \ - diffutils \ - flake8 \ - git \ - gnutls-bin \ - golang \ - iproute2 \ - jq \ - libtool-bin \ - locales \ - make \ - nbd-client \ - nbd-server \ - nbdkit \ - ocaml \ - ocaml-findlib \ - perl-base \ - pkgconf \ - python3-dev \ - qemu-kvm \ - qemu-utils \ - sed && \ - eatmydata apt-get autoremove -y && \ - eatmydata apt-get autoclean -y && \ - sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ - dpkg-reconfigure locales && \ - dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt && \ - mkdir -p /usr/libexec/ccache-wrappers && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/i686-linux-gnu-c++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/i686-linux-gnu-cc && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/i686-linux-gnu-g++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/i686-linux-gnu-gcc - -RUN export DEBIAN_FRONTEND=noninteractive && \ - dpkg --add-architecture i386 && \ - eatmydata apt-get update && \ - eatmydata apt-get dist-upgrade -y && \ - eatmydata apt-get install --no-install-recommends -y dpkg-dev && \ - eatmydata apt-get install --no-install-recommends -y \ - g++-i686-linux-gnu \ - gcc-i686-linux-gnu \ - libc6-dev:i386 \ - libev-dev:i386 \ - libfuse3-dev:i386 \ - libglib2.0-dev:i386 \ - libgnutls28-dev:i386 \ - libxml2-dev:i386 && \ - eatmydata apt-get autoremove -y && \ - eatmydata apt-get autoclean -y && \ - mkdir -p /usr/local/share/meson/cross && \ - echo "[binaries]\n\ -c = '/usr/bin/i686-linux-gnu-gcc'\n\ -ar = '/usr/bin/i686-linux-gnu-gcc-ar'\n\ -strip = '/usr/bin/i686-linux-gnu-strip'\n\ -pkgconfig = '/usr/bin/i686-linux-gnu-pkg-config'\n\ -\n\ -[host_machine]\n\ -system = 'linux'\n\ -cpu_family = 'x86'\n\ -cpu = 'i686'\n\ -endian = 'little'" > /usr/local/share/meson/cross/i686-linux-gnu - -ENV LANG "en_US.UTF-8" -ENV MAKE "/usr/bin/make" -ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" - -ENV ABI "i686-linux-gnu" -ENV CONFIGURE_OPTS "--host=i686-linux-gnu" diff --git a/ci/containers/debian-10-cross-mips.Dockerfile b/ci/containers/debian-10-cross-mips.Dockerfile deleted file mode 100644 index a5c3a91ab97d..000000000000 --- a/ci/containers/debian-10-cross-mips.Dockerfile +++ /dev/null @@ -1,86 +0,0 @@ -# THIS FILE WAS AUTO-GENERATED -# -# $ lcitool dockerfile --cross mips debian-10 libnbd -# -# https://gitlab.com/libvirt/libvirt-ci/-/commit/5fd2cdeb8958df86a81b639fd096f... - -FROM docker.io/library/debian:10-slim - -RUN export DEBIAN_FRONTEND=noninteractive && \ - apt-get update && \ - apt-get install -y eatmydata && \ - eatmydata apt-get dist-upgrade -y && \ - eatmydata apt-get install --no-install-recommends -y \ - autoconf \ - automake \ - bash-completion \ - ca-certificates \ - ccache \ - coreutils \ - diffutils \ - flake8 \ - git \ - gnutls-bin \ - golang \ - iproute2 \ - jq \ - libtool-bin \ - locales \ - make \ - nbd-client \ - nbd-server \ - nbdkit \ - ocaml \ - ocaml-findlib \ - perl-base \ - pkgconf \ - python3-dev \ - qemu-kvm \ - qemu-utils \ - sed && \ - eatmydata apt-get autoremove -y && \ - eatmydata apt-get autoclean -y && \ - sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ - dpkg-reconfigure locales && \ - dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt && \ - mkdir -p /usr/libexec/ccache-wrappers && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mips-linux-gnu-c++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mips-linux-gnu-cc && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mips-linux-gnu-g++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mips-linux-gnu-gcc - -RUN export DEBIAN_FRONTEND=noninteractive && \ - dpkg --add-architecture mips && \ - eatmydata apt-get update && \ - eatmydata apt-get dist-upgrade -y && \ - eatmydata apt-get install --no-install-recommends -y dpkg-dev && \ - eatmydata apt-get install --no-install-recommends -y \ - g++-mips-linux-gnu \ - gcc-mips-linux-gnu \ - libc6-dev:mips \ - libev-dev:mips \ - libfuse3-dev:mips \ - libglib2.0-dev:mips \ - libgnutls28-dev:mips \ - libxml2-dev:mips && \ - eatmydata apt-get autoremove -y && \ - eatmydata apt-get autoclean -y && \ - mkdir -p /usr/local/share/meson/cross && \ - echo "[binaries]\n\ -c = '/usr/bin/mips-linux-gnu-gcc'\n\ -ar = '/usr/bin/mips-linux-gnu-gcc-ar'\n\ -strip = '/usr/bin/mips-linux-gnu-strip'\n\ -pkgconfig = '/usr/bin/mips-linux-gnu-pkg-config'\n\ -\n\ -[host_machine]\n\ -system = 'linux'\n\ -cpu_family = 'mips'\n\ -cpu = 'mips'\n\ -endian = 'big'" > /usr/local/share/meson/cross/mips-linux-gnu - -ENV LANG "en_US.UTF-8" -ENV MAKE "/usr/bin/make" -ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" - -ENV ABI "mips-linux-gnu" -ENV CONFIGURE_OPTS "--host=mips-linux-gnu" diff --git a/ci/containers/debian-10-cross-mips64el.Dockerfile b/ci/containers/debian-10-cross-mips64el.Dockerfile deleted file mode 100644 index 3fc65ad6c47f..000000000000 --- a/ci/containers/debian-10-cross-mips64el.Dockerfile +++ /dev/null @@ -1,86 +0,0 @@ -# THIS FILE WAS AUTO-GENERATED -# -# $ lcitool dockerfile --cross mips64el debian-10 libnbd -# -# https://gitlab.com/libvirt/libvirt-ci/-/commit/5fd2cdeb8958df86a81b639fd096f... - -FROM docker.io/library/debian:10-slim - -RUN export DEBIAN_FRONTEND=noninteractive && \ - apt-get update && \ - apt-get install -y eatmydata && \ - eatmydata apt-get dist-upgrade -y && \ - eatmydata apt-get install --no-install-recommends -y \ - autoconf \ - automake \ - bash-completion \ - ca-certificates \ - ccache \ - coreutils \ - diffutils \ - flake8 \ - git \ - gnutls-bin \ - golang \ - iproute2 \ - jq \ - libtool-bin \ - locales \ - make \ - nbd-client \ - nbd-server \ - nbdkit \ - ocaml \ - ocaml-findlib \ - perl-base \ - pkgconf \ - python3-dev \ - qemu-kvm \ - qemu-utils \ - sed && \ - eatmydata apt-get autoremove -y && \ - eatmydata apt-get autoclean -y && \ - sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ - dpkg-reconfigure locales && \ - dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt && \ - mkdir -p /usr/libexec/ccache-wrappers && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mips64el-linux-gnuabi64-c++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mips64el-linux-gnuabi64-cc && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mips64el-linux-gnuabi64-g++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mips64el-linux-gnuabi64-gcc - -RUN export DEBIAN_FRONTEND=noninteractive && \ - dpkg --add-architecture mips64el && \ - eatmydata apt-get update && \ - eatmydata apt-get dist-upgrade -y && \ - eatmydata apt-get install --no-install-recommends -y dpkg-dev && \ - eatmydata apt-get install --no-install-recommends -y \ - g++-mips64el-linux-gnuabi64 \ - gcc-mips64el-linux-gnuabi64 \ - libc6-dev:mips64el \ - libev-dev:mips64el \ - libfuse3-dev:mips64el \ - libglib2.0-dev:mips64el \ - libgnutls28-dev:mips64el \ - libxml2-dev:mips64el && \ - eatmydata apt-get autoremove -y && \ - eatmydata apt-get autoclean -y && \ - mkdir -p /usr/local/share/meson/cross && \ - echo "[binaries]\n\ -c = '/usr/bin/mips64el-linux-gnuabi64-gcc'\n\ -ar = '/usr/bin/mips64el-linux-gnuabi64-gcc-ar'\n\ -strip = '/usr/bin/mips64el-linux-gnuabi64-strip'\n\ -pkgconfig = '/usr/bin/mips64el-linux-gnuabi64-pkg-config'\n\ -\n\ -[host_machine]\n\ -system = 'linux'\n\ -cpu_family = 'mips64'\n\ -cpu = 'mips64el'\n\ -endian = 'little'" > /usr/local/share/meson/cross/mips64el-linux-gnuabi64 - -ENV LANG "en_US.UTF-8" -ENV MAKE "/usr/bin/make" -ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" - -ENV ABI "mips64el-linux-gnuabi64" -ENV CONFIGURE_OPTS "--host=mips64el-linux-gnuabi64" diff --git a/ci/containers/debian-10-cross-mipsel.Dockerfile b/ci/containers/debian-10-cross-mipsel.Dockerfile deleted file mode 100644 index f9a0aff68896..000000000000 --- a/ci/containers/debian-10-cross-mipsel.Dockerfile +++ /dev/null @@ -1,86 +0,0 @@ -# THIS FILE WAS AUTO-GENERATED -# -# $ lcitool dockerfile --cross mipsel debian-10 libnbd -# -# https://gitlab.com/libvirt/libvirt-ci/-/commit/5fd2cdeb8958df86a81b639fd096f... - -FROM docker.io/library/debian:10-slim - -RUN export DEBIAN_FRONTEND=noninteractive && \ - apt-get update && \ - apt-get install -y eatmydata && \ - eatmydata apt-get dist-upgrade -y && \ - eatmydata apt-get install --no-install-recommends -y \ - autoconf \ - automake \ - bash-completion \ - ca-certificates \ - ccache \ - coreutils \ - diffutils \ - flake8 \ - git \ - gnutls-bin \ - golang \ - iproute2 \ - jq \ - libtool-bin \ - locales \ - make \ - nbd-client \ - nbd-server \ - nbdkit \ - ocaml \ - ocaml-findlib \ - perl-base \ - pkgconf \ - python3-dev \ - qemu-kvm \ - qemu-utils \ - sed && \ - eatmydata apt-get autoremove -y && \ - eatmydata apt-get autoclean -y && \ - sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ - dpkg-reconfigure locales && \ - dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt && \ - mkdir -p /usr/libexec/ccache-wrappers && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mipsel-linux-gnu-c++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mipsel-linux-gnu-cc && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mipsel-linux-gnu-g++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mipsel-linux-gnu-gcc - -RUN export DEBIAN_FRONTEND=noninteractive && \ - dpkg --add-architecture mipsel && \ - eatmydata apt-get update && \ - eatmydata apt-get dist-upgrade -y && \ - eatmydata apt-get install --no-install-recommends -y dpkg-dev && \ - eatmydata apt-get install --no-install-recommends -y \ - g++-mipsel-linux-gnu \ - gcc-mipsel-linux-gnu \ - libc6-dev:mipsel \ - libev-dev:mipsel \ - libfuse3-dev:mipsel \ - libglib2.0-dev:mipsel \ - libgnutls28-dev:mipsel \ - libxml2-dev:mipsel && \ - eatmydata apt-get autoremove -y && \ - eatmydata apt-get autoclean -y && \ - mkdir -p /usr/local/share/meson/cross && \ - echo "[binaries]\n\ -c = '/usr/bin/mipsel-linux-gnu-gcc'\n\ -ar = '/usr/bin/mipsel-linux-gnu-gcc-ar'\n\ -strip = '/usr/bin/mipsel-linux-gnu-strip'\n\ -pkgconfig = '/usr/bin/mipsel-linux-gnu-pkg-config'\n\ -\n\ -[host_machine]\n\ -system = 'linux'\n\ -cpu_family = 'mips'\n\ -cpu = 'mipsel'\n\ -endian = 'little'" > /usr/local/share/meson/cross/mipsel-linux-gnu - -ENV LANG "en_US.UTF-8" -ENV MAKE "/usr/bin/make" -ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" - -ENV ABI "mipsel-linux-gnu" -ENV CONFIGURE_OPTS "--host=mipsel-linux-gnu" diff --git a/ci/containers/debian-10-cross-ppc64le.Dockerfile b/ci/containers/debian-10-cross-ppc64le.Dockerfile deleted file mode 100644 index 5cb602985c2a..000000000000 --- a/ci/containers/debian-10-cross-ppc64le.Dockerfile +++ /dev/null @@ -1,86 +0,0 @@ -# THIS FILE WAS AUTO-GENERATED -# -# $ lcitool dockerfile --cross ppc64le debian-10 libnbd -# -# https://gitlab.com/libvirt/libvirt-ci/-/commit/5fd2cdeb8958df86a81b639fd096f... - -FROM docker.io/library/debian:10-slim - -RUN export DEBIAN_FRONTEND=noninteractive && \ - apt-get update && \ - apt-get install -y eatmydata && \ - eatmydata apt-get dist-upgrade -y && \ - eatmydata apt-get install --no-install-recommends -y \ - autoconf \ - automake \ - bash-completion \ - ca-certificates \ - ccache \ - coreutils \ - diffutils \ - flake8 \ - git \ - gnutls-bin \ - golang \ - iproute2 \ - jq \ - libtool-bin \ - locales \ - make \ - nbd-client \ - nbd-server \ - nbdkit \ - ocaml \ - ocaml-findlib \ - perl-base \ - pkgconf \ - python3-dev \ - qemu-kvm \ - qemu-utils \ - sed && \ - eatmydata apt-get autoremove -y && \ - eatmydata apt-get autoclean -y && \ - sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ - dpkg-reconfigure locales && \ - dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt && \ - mkdir -p /usr/libexec/ccache-wrappers && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/powerpc64le-linux-gnu-c++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/powerpc64le-linux-gnu-cc && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/powerpc64le-linux-gnu-g++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/powerpc64le-linux-gnu-gcc - -RUN export DEBIAN_FRONTEND=noninteractive && \ - dpkg --add-architecture ppc64el && \ - eatmydata apt-get update && \ - eatmydata apt-get dist-upgrade -y && \ - eatmydata apt-get install --no-install-recommends -y dpkg-dev && \ - eatmydata apt-get install --no-install-recommends -y \ - g++-powerpc64le-linux-gnu \ - gcc-powerpc64le-linux-gnu \ - libc6-dev:ppc64el \ - libev-dev:ppc64el \ - libfuse3-dev:ppc64el \ - libglib2.0-dev:ppc64el \ - libgnutls28-dev:ppc64el \ - libxml2-dev:ppc64el && \ - eatmydata apt-get autoremove -y && \ - eatmydata apt-get autoclean -y && \ - mkdir -p /usr/local/share/meson/cross && \ - echo "[binaries]\n\ -c = '/usr/bin/powerpc64le-linux-gnu-gcc'\n\ -ar = '/usr/bin/powerpc64le-linux-gnu-gcc-ar'\n\ -strip = '/usr/bin/powerpc64le-linux-gnu-strip'\n\ -pkgconfig = '/usr/bin/powerpc64le-linux-gnu-pkg-config'\n\ -\n\ -[host_machine]\n\ -system = 'linux'\n\ -cpu_family = 'ppc64'\n\ -cpu = 'powerpc64le'\n\ -endian = 'little'" > /usr/local/share/meson/cross/powerpc64le-linux-gnu - -ENV LANG "en_US.UTF-8" -ENV MAKE "/usr/bin/make" -ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" - -ENV ABI "powerpc64le-linux-gnu" -ENV CONFIGURE_OPTS "--host=powerpc64le-linux-gnu" diff --git a/ci/containers/debian-10-cross-s390x.Dockerfile b/ci/containers/debian-10-cross-s390x.Dockerfile deleted file mode 100644 index 8280822e5e62..000000000000 --- a/ci/containers/debian-10-cross-s390x.Dockerfile +++ /dev/null @@ -1,86 +0,0 @@ -# THIS FILE WAS AUTO-GENERATED -# -# $ lcitool dockerfile --cross s390x debian-10 libnbd -# -# https://gitlab.com/libvirt/libvirt-ci/-/commit/5fd2cdeb8958df86a81b639fd096f... - -FROM docker.io/library/debian:10-slim - -RUN export DEBIAN_FRONTEND=noninteractive && \ - apt-get update && \ - apt-get install -y eatmydata && \ - eatmydata apt-get dist-upgrade -y && \ - eatmydata apt-get install --no-install-recommends -y \ - autoconf \ - automake \ - bash-completion \ - ca-certificates \ - ccache \ - coreutils \ - diffutils \ - flake8 \ - git \ - gnutls-bin \ - golang \ - iproute2 \ - jq \ - libtool-bin \ - locales \ - make \ - nbd-client \ - nbd-server \ - nbdkit \ - ocaml \ - ocaml-findlib \ - perl-base \ - pkgconf \ - python3-dev \ - qemu-kvm \ - qemu-utils \ - sed && \ - eatmydata apt-get autoremove -y && \ - eatmydata apt-get autoclean -y && \ - sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ - dpkg-reconfigure locales && \ - dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt && \ - mkdir -p /usr/libexec/ccache-wrappers && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/s390x-linux-gnu-c++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/s390x-linux-gnu-cc && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/s390x-linux-gnu-g++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/s390x-linux-gnu-gcc - -RUN export DEBIAN_FRONTEND=noninteractive && \ - dpkg --add-architecture s390x && \ - eatmydata apt-get update && \ - eatmydata apt-get dist-upgrade -y && \ - eatmydata apt-get install --no-install-recommends -y dpkg-dev && \ - eatmydata apt-get install --no-install-recommends -y \ - g++-s390x-linux-gnu \ - gcc-s390x-linux-gnu \ - libc6-dev:s390x \ - libev-dev:s390x \ - libfuse3-dev:s390x \ - libglib2.0-dev:s390x \ - libgnutls28-dev:s390x \ - libxml2-dev:s390x && \ - eatmydata apt-get autoremove -y && \ - eatmydata apt-get autoclean -y && \ - mkdir -p /usr/local/share/meson/cross && \ - echo "[binaries]\n\ -c = '/usr/bin/s390x-linux-gnu-gcc'\n\ -ar = '/usr/bin/s390x-linux-gnu-gcc-ar'\n\ -strip = '/usr/bin/s390x-linux-gnu-strip'\n\ -pkgconfig = '/usr/bin/s390x-linux-gnu-pkg-config'\n\ -\n\ -[host_machine]\n\ -system = 'linux'\n\ -cpu_family = 's390x'\n\ -cpu = 's390x'\n\ -endian = 'big'" > /usr/local/share/meson/cross/s390x-linux-gnu - -ENV LANG "en_US.UTF-8" -ENV MAKE "/usr/bin/make" -ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" - -ENV ABI "s390x-linux-gnu" -ENV CONFIGURE_OPTS "--host=s390x-linux-gnu" diff --git a/ci/containers/debian-sid-cross-aarch64.Dockerfile b/ci/containers/debian-sid-cross-aarch64.Dockerfile deleted file mode 100644 index c158d3d03ab5..000000000000 --- a/ci/containers/debian-sid-cross-aarch64.Dockerfile +++ /dev/null @@ -1,86 +0,0 @@ -# THIS FILE WAS AUTO-GENERATED -# -# $ lcitool dockerfile --cross aarch64 debian-sid libnbd -# -# https://gitlab.com/libvirt/libvirt-ci/-/commit/5fd2cdeb8958df86a81b639fd096f... - -FROM docker.io/library/debian:sid-slim - -RUN export DEBIAN_FRONTEND=noninteractive && \ - apt-get update && \ - apt-get install -y eatmydata && \ - eatmydata apt-get dist-upgrade -y && \ - eatmydata apt-get install --no-install-recommends -y \ - autoconf \ - automake \ - bash-completion \ - ca-certificates \ - ccache \ - coreutils \ - diffutils \ - flake8 \ - git \ - gnutls-bin \ - golang \ - iproute2 \ - jq \ - libtool-bin \ - locales \ - make \ - nbd-client \ - nbd-server \ - nbdkit \ - ocaml \ - ocaml-findlib \ - perl-base \ - pkgconf \ - python3-dev \ - qemu-kvm \ - qemu-utils \ - sed && \ - eatmydata apt-get autoremove -y && \ - eatmydata apt-get autoclean -y && \ - sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ - dpkg-reconfigure locales && \ - dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt && \ - mkdir -p /usr/libexec/ccache-wrappers && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/aarch64-linux-gnu-c++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/aarch64-linux-gnu-cc && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/aarch64-linux-gnu-g++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/aarch64-linux-gnu-gcc - -RUN export DEBIAN_FRONTEND=noninteractive && \ - dpkg --add-architecture arm64 && \ - eatmydata apt-get update && \ - eatmydata apt-get dist-upgrade -y && \ - eatmydata apt-get install --no-install-recommends -y dpkg-dev && \ - eatmydata apt-get install --no-install-recommends -y \ - g++-aarch64-linux-gnu \ - gcc-aarch64-linux-gnu \ - libc6-dev:arm64 \ - libev-dev:arm64 \ - libfuse3-dev:arm64 \ - libglib2.0-dev:arm64 \ - libgnutls28-dev:arm64 \ - libxml2-dev:arm64 && \ - eatmydata apt-get autoremove -y && \ - eatmydata apt-get autoclean -y && \ - mkdir -p /usr/local/share/meson/cross && \ - echo "[binaries]\n\ -c = '/usr/bin/aarch64-linux-gnu-gcc'\n\ -ar = '/usr/bin/aarch64-linux-gnu-gcc-ar'\n\ -strip = '/usr/bin/aarch64-linux-gnu-strip'\n\ -pkgconfig = '/usr/bin/aarch64-linux-gnu-pkg-config'\n\ -\n\ -[host_machine]\n\ -system = 'linux'\n\ -cpu_family = 'aarch64'\n\ -cpu = 'aarch64'\n\ -endian = 'little'" > /usr/local/share/meson/cross/aarch64-linux-gnu - -ENV LANG "en_US.UTF-8" -ENV MAKE "/usr/bin/make" -ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" - -ENV ABI "aarch64-linux-gnu" -ENV CONFIGURE_OPTS "--host=aarch64-linux-gnu" diff --git a/ci/containers/debian-sid-cross-armv6l.Dockerfile b/ci/containers/debian-sid-cross-armv6l.Dockerfile deleted file mode 100644 index a68338db9c91..000000000000 --- a/ci/containers/debian-sid-cross-armv6l.Dockerfile +++ /dev/null @@ -1,86 +0,0 @@ -# THIS FILE WAS AUTO-GENERATED -# -# $ lcitool dockerfile --cross armv6l debian-sid libnbd -# -# https://gitlab.com/libvirt/libvirt-ci/-/commit/5fd2cdeb8958df86a81b639fd096f... - -FROM docker.io/library/debian:sid-slim - -RUN export DEBIAN_FRONTEND=noninteractive && \ - apt-get update && \ - apt-get install -y eatmydata && \ - eatmydata apt-get dist-upgrade -y && \ - eatmydata apt-get install --no-install-recommends -y \ - autoconf \ - automake \ - bash-completion \ - ca-certificates \ - ccache \ - coreutils \ - diffutils \ - flake8 \ - git \ - gnutls-bin \ - golang \ - iproute2 \ - jq \ - libtool-bin \ - locales \ - make \ - nbd-client \ - nbd-server \ - nbdkit \ - ocaml \ - ocaml-findlib \ - perl-base \ - pkgconf \ - python3-dev \ - qemu-kvm \ - qemu-utils \ - sed && \ - eatmydata apt-get autoremove -y && \ - eatmydata apt-get autoclean -y && \ - sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ - dpkg-reconfigure locales && \ - dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt && \ - mkdir -p /usr/libexec/ccache-wrappers && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/arm-linux-gnueabi-c++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/arm-linux-gnueabi-cc && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/arm-linux-gnueabi-g++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/arm-linux-gnueabi-gcc - -RUN export DEBIAN_FRONTEND=noninteractive && \ - dpkg --add-architecture armel && \ - eatmydata apt-get update && \ - eatmydata apt-get dist-upgrade -y && \ - eatmydata apt-get install --no-install-recommends -y dpkg-dev && \ - eatmydata apt-get install --no-install-recommends -y \ - g++-arm-linux-gnueabi \ - gcc-arm-linux-gnueabi \ - libc6-dev:armel \ - libev-dev:armel \ - libfuse3-dev:armel \ - libglib2.0-dev:armel \ - libgnutls28-dev:armel \ - libxml2-dev:armel && \ - eatmydata apt-get autoremove -y && \ - eatmydata apt-get autoclean -y && \ - mkdir -p /usr/local/share/meson/cross && \ - echo "[binaries]\n\ -c = '/usr/bin/arm-linux-gnueabi-gcc'\n\ -ar = '/usr/bin/arm-linux-gnueabi-gcc-ar'\n\ -strip = '/usr/bin/arm-linux-gnueabi-strip'\n\ -pkgconfig = '/usr/bin/arm-linux-gnueabi-pkg-config'\n\ -\n\ -[host_machine]\n\ -system = 'linux'\n\ -cpu_family = 'arm'\n\ -cpu = 'arm'\n\ -endian = 'little'" > /usr/local/share/meson/cross/arm-linux-gnueabi - -ENV LANG "en_US.UTF-8" -ENV MAKE "/usr/bin/make" -ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" - -ENV ABI "arm-linux-gnueabi" -ENV CONFIGURE_OPTS "--host=arm-linux-gnueabi" diff --git a/ci/containers/debian-sid-cross-armv7l.Dockerfile b/ci/containers/debian-sid-cross-armv7l.Dockerfile deleted file mode 100644 index 1f9085bd7e1e..000000000000 --- a/ci/containers/debian-sid-cross-armv7l.Dockerfile +++ /dev/null @@ -1,86 +0,0 @@ -# THIS FILE WAS AUTO-GENERATED -# -# $ lcitool dockerfile --cross armv7l debian-sid libnbd -# -# https://gitlab.com/libvirt/libvirt-ci/-/commit/5fd2cdeb8958df86a81b639fd096f... - -FROM docker.io/library/debian:sid-slim - -RUN export DEBIAN_FRONTEND=noninteractive && \ - apt-get update && \ - apt-get install -y eatmydata && \ - eatmydata apt-get dist-upgrade -y && \ - eatmydata apt-get install --no-install-recommends -y \ - autoconf \ - automake \ - bash-completion \ - ca-certificates \ - ccache \ - coreutils \ - diffutils \ - flake8 \ - git \ - gnutls-bin \ - golang \ - iproute2 \ - jq \ - libtool-bin \ - locales \ - make \ - nbd-client \ - nbd-server \ - nbdkit \ - ocaml \ - ocaml-findlib \ - perl-base \ - pkgconf \ - python3-dev \ - qemu-kvm \ - qemu-utils \ - sed && \ - eatmydata apt-get autoremove -y && \ - eatmydata apt-get autoclean -y && \ - sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ - dpkg-reconfigure locales && \ - dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt && \ - mkdir -p /usr/libexec/ccache-wrappers && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/arm-linux-gnueabihf-c++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/arm-linux-gnueabihf-cc && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/arm-linux-gnueabihf-g++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/arm-linux-gnueabihf-gcc - -RUN export DEBIAN_FRONTEND=noninteractive && \ - dpkg --add-architecture armhf && \ - eatmydata apt-get update && \ - eatmydata apt-get dist-upgrade -y && \ - eatmydata apt-get install --no-install-recommends -y dpkg-dev && \ - eatmydata apt-get install --no-install-recommends -y \ - g++-arm-linux-gnueabihf \ - gcc-arm-linux-gnueabihf \ - libc6-dev:armhf \ - libev-dev:armhf \ - libfuse3-dev:armhf \ - libglib2.0-dev:armhf \ - libgnutls28-dev:armhf \ - libxml2-dev:armhf && \ - eatmydata apt-get autoremove -y && \ - eatmydata apt-get autoclean -y && \ - mkdir -p /usr/local/share/meson/cross && \ - echo "[binaries]\n\ -c = '/usr/bin/arm-linux-gnueabihf-gcc'\n\ -ar = '/usr/bin/arm-linux-gnueabihf-gcc-ar'\n\ -strip = '/usr/bin/arm-linux-gnueabihf-strip'\n\ -pkgconfig = '/usr/bin/arm-linux-gnueabihf-pkg-config'\n\ -\n\ -[host_machine]\n\ -system = 'linux'\n\ -cpu_family = 'arm'\n\ -cpu = 'armhf'\n\ -endian = 'little'" > /usr/local/share/meson/cross/arm-linux-gnueabihf - -ENV LANG "en_US.UTF-8" -ENV MAKE "/usr/bin/make" -ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" - -ENV ABI "arm-linux-gnueabihf" -ENV CONFIGURE_OPTS "--host=arm-linux-gnueabihf" diff --git a/ci/containers/debian-sid-cross-i686.Dockerfile b/ci/containers/debian-sid-cross-i686.Dockerfile deleted file mode 100644 index 37b28a612adf..000000000000 --- a/ci/containers/debian-sid-cross-i686.Dockerfile +++ /dev/null @@ -1,86 +0,0 @@ -# THIS FILE WAS AUTO-GENERATED -# -# $ lcitool dockerfile --cross i686 debian-sid libnbd -# -# https://gitlab.com/libvirt/libvirt-ci/-/commit/5fd2cdeb8958df86a81b639fd096f... - -FROM docker.io/library/debian:sid-slim - -RUN export DEBIAN_FRONTEND=noninteractive && \ - apt-get update && \ - apt-get install -y eatmydata && \ - eatmydata apt-get dist-upgrade -y && \ - eatmydata apt-get install --no-install-recommends -y \ - autoconf \ - automake \ - bash-completion \ - ca-certificates \ - ccache \ - coreutils \ - diffutils \ - flake8 \ - git \ - gnutls-bin \ - golang \ - iproute2 \ - jq \ - libtool-bin \ - locales \ - make \ - nbd-client \ - nbd-server \ - nbdkit \ - ocaml \ - ocaml-findlib \ - perl-base \ - pkgconf \ - python3-dev \ - qemu-kvm \ - qemu-utils \ - sed && \ - eatmydata apt-get autoremove -y && \ - eatmydata apt-get autoclean -y && \ - sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ - dpkg-reconfigure locales && \ - dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt && \ - mkdir -p /usr/libexec/ccache-wrappers && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/i686-linux-gnu-c++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/i686-linux-gnu-cc && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/i686-linux-gnu-g++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/i686-linux-gnu-gcc - -RUN export DEBIAN_FRONTEND=noninteractive && \ - dpkg --add-architecture i386 && \ - eatmydata apt-get update && \ - eatmydata apt-get dist-upgrade -y && \ - eatmydata apt-get install --no-install-recommends -y dpkg-dev && \ - eatmydata apt-get install --no-install-recommends -y \ - g++-i686-linux-gnu \ - gcc-i686-linux-gnu \ - libc6-dev:i386 \ - libev-dev:i386 \ - libfuse3-dev:i386 \ - libglib2.0-dev:i386 \ - libgnutls28-dev:i386 \ - libxml2-dev:i386 && \ - eatmydata apt-get autoremove -y && \ - eatmydata apt-get autoclean -y && \ - mkdir -p /usr/local/share/meson/cross && \ - echo "[binaries]\n\ -c = '/usr/bin/i686-linux-gnu-gcc'\n\ -ar = '/usr/bin/i686-linux-gnu-gcc-ar'\n\ -strip = '/usr/bin/i686-linux-gnu-strip'\n\ -pkgconfig = '/usr/bin/i686-linux-gnu-pkg-config'\n\ -\n\ -[host_machine]\n\ -system = 'linux'\n\ -cpu_family = 'x86'\n\ -cpu = 'i686'\n\ -endian = 'little'" > /usr/local/share/meson/cross/i686-linux-gnu - -ENV LANG "en_US.UTF-8" -ENV MAKE "/usr/bin/make" -ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" - -ENV ABI "i686-linux-gnu" -ENV CONFIGURE_OPTS "--host=i686-linux-gnu" diff --git a/ci/containers/debian-sid-cross-mips64el.Dockerfile b/ci/containers/debian-sid-cross-mips64el.Dockerfile deleted file mode 100644 index dee0fd013d2f..000000000000 --- a/ci/containers/debian-sid-cross-mips64el.Dockerfile +++ /dev/null @@ -1,86 +0,0 @@ -# THIS FILE WAS AUTO-GENERATED -# -# $ lcitool dockerfile --cross mips64el debian-sid libnbd -# -# https://gitlab.com/libvirt/libvirt-ci/-/commit/5fd2cdeb8958df86a81b639fd096f... - -FROM docker.io/library/debian:sid-slim - -RUN export DEBIAN_FRONTEND=noninteractive && \ - apt-get update && \ - apt-get install -y eatmydata && \ - eatmydata apt-get dist-upgrade -y && \ - eatmydata apt-get install --no-install-recommends -y \ - autoconf \ - automake \ - bash-completion \ - ca-certificates \ - ccache \ - coreutils \ - diffutils \ - flake8 \ - git \ - gnutls-bin \ - golang \ - iproute2 \ - jq \ - libtool-bin \ - locales \ - make \ - nbd-client \ - nbd-server \ - nbdkit \ - ocaml \ - ocaml-findlib \ - perl-base \ - pkgconf \ - python3-dev \ - qemu-kvm \ - qemu-utils \ - sed && \ - eatmydata apt-get autoremove -y && \ - eatmydata apt-get autoclean -y && \ - sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ - dpkg-reconfigure locales && \ - dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt && \ - mkdir -p /usr/libexec/ccache-wrappers && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mips64el-linux-gnuabi64-c++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mips64el-linux-gnuabi64-cc && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mips64el-linux-gnuabi64-g++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mips64el-linux-gnuabi64-gcc - -RUN export DEBIAN_FRONTEND=noninteractive && \ - dpkg --add-architecture mips64el && \ - eatmydata apt-get update && \ - eatmydata apt-get dist-upgrade -y && \ - eatmydata apt-get install --no-install-recommends -y dpkg-dev && \ - eatmydata apt-get install --no-install-recommends -y \ - g++-mips64el-linux-gnuabi64 \ - gcc-mips64el-linux-gnuabi64 \ - libc6-dev:mips64el \ - libev-dev:mips64el \ - libfuse3-dev:mips64el \ - libglib2.0-dev:mips64el \ - libgnutls28-dev:mips64el \ - libxml2-dev:mips64el && \ - eatmydata apt-get autoremove -y && \ - eatmydata apt-get autoclean -y && \ - mkdir -p /usr/local/share/meson/cross && \ - echo "[binaries]\n\ -c = '/usr/bin/mips64el-linux-gnuabi64-gcc'\n\ -ar = '/usr/bin/mips64el-linux-gnuabi64-gcc-ar'\n\ -strip = '/usr/bin/mips64el-linux-gnuabi64-strip'\n\ -pkgconfig = '/usr/bin/mips64el-linux-gnuabi64-pkg-config'\n\ -\n\ -[host_machine]\n\ -system = 'linux'\n\ -cpu_family = 'mips64'\n\ -cpu = 'mips64el'\n\ -endian = 'little'" > /usr/local/share/meson/cross/mips64el-linux-gnuabi64 - -ENV LANG "en_US.UTF-8" -ENV MAKE "/usr/bin/make" -ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" - -ENV ABI "mips64el-linux-gnuabi64" -ENV CONFIGURE_OPTS "--host=mips64el-linux-gnuabi64" diff --git a/ci/containers/debian-sid-cross-mipsel.Dockerfile b/ci/containers/debian-sid-cross-mipsel.Dockerfile deleted file mode 100644 index 67feb7dede2c..000000000000 --- a/ci/containers/debian-sid-cross-mipsel.Dockerfile +++ /dev/null @@ -1,86 +0,0 @@ -# THIS FILE WAS AUTO-GENERATED -# -# $ lcitool dockerfile --cross mipsel debian-sid libnbd -# -# https://gitlab.com/libvirt/libvirt-ci/-/commit/5fd2cdeb8958df86a81b639fd096f... - -FROM docker.io/library/debian:sid-slim - -RUN export DEBIAN_FRONTEND=noninteractive && \ - apt-get update && \ - apt-get install -y eatmydata && \ - eatmydata apt-get dist-upgrade -y && \ - eatmydata apt-get install --no-install-recommends -y \ - autoconf \ - automake \ - bash-completion \ - ca-certificates \ - ccache \ - coreutils \ - diffutils \ - flake8 \ - git \ - gnutls-bin \ - golang \ - iproute2 \ - jq \ - libtool-bin \ - locales \ - make \ - nbd-client \ - nbd-server \ - nbdkit \ - ocaml \ - ocaml-findlib \ - perl-base \ - pkgconf \ - python3-dev \ - qemu-kvm \ - qemu-utils \ - sed && \ - eatmydata apt-get autoremove -y && \ - eatmydata apt-get autoclean -y && \ - sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ - dpkg-reconfigure locales && \ - dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt && \ - mkdir -p /usr/libexec/ccache-wrappers && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mipsel-linux-gnu-c++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mipsel-linux-gnu-cc && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mipsel-linux-gnu-g++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mipsel-linux-gnu-gcc - -RUN export DEBIAN_FRONTEND=noninteractive && \ - dpkg --add-architecture mipsel && \ - eatmydata apt-get update && \ - eatmydata apt-get dist-upgrade -y && \ - eatmydata apt-get install --no-install-recommends -y dpkg-dev && \ - eatmydata apt-get install --no-install-recommends -y \ - g++-mipsel-linux-gnu \ - gcc-mipsel-linux-gnu \ - libc6-dev:mipsel \ - libev-dev:mipsel \ - libfuse3-dev:mipsel \ - libglib2.0-dev:mipsel \ - libgnutls28-dev:mipsel \ - libxml2-dev:mipsel && \ - eatmydata apt-get autoremove -y && \ - eatmydata apt-get autoclean -y && \ - mkdir -p /usr/local/share/meson/cross && \ - echo "[binaries]\n\ -c = '/usr/bin/mipsel-linux-gnu-gcc'\n\ -ar = '/usr/bin/mipsel-linux-gnu-gcc-ar'\n\ -strip = '/usr/bin/mipsel-linux-gnu-strip'\n\ -pkgconfig = '/usr/bin/mipsel-linux-gnu-pkg-config'\n\ -\n\ -[host_machine]\n\ -system = 'linux'\n\ -cpu_family = 'mips'\n\ -cpu = 'mipsel'\n\ -endian = 'little'" > /usr/local/share/meson/cross/mipsel-linux-gnu - -ENV LANG "en_US.UTF-8" -ENV MAKE "/usr/bin/make" -ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" - -ENV ABI "mipsel-linux-gnu" -ENV CONFIGURE_OPTS "--host=mipsel-linux-gnu" diff --git a/ci/containers/debian-sid-cross-ppc64le.Dockerfile b/ci/containers/debian-sid-cross-ppc64le.Dockerfile deleted file mode 100644 index 9914e4c2cfeb..000000000000 --- a/ci/containers/debian-sid-cross-ppc64le.Dockerfile +++ /dev/null @@ -1,86 +0,0 @@ -# THIS FILE WAS AUTO-GENERATED -# -# $ lcitool dockerfile --cross ppc64le debian-sid libnbd -# -# https://gitlab.com/libvirt/libvirt-ci/-/commit/5fd2cdeb8958df86a81b639fd096f... - -FROM docker.io/library/debian:sid-slim - -RUN export DEBIAN_FRONTEND=noninteractive && \ - apt-get update && \ - apt-get install -y eatmydata && \ - eatmydata apt-get dist-upgrade -y && \ - eatmydata apt-get install --no-install-recommends -y \ - autoconf \ - automake \ - bash-completion \ - ca-certificates \ - ccache \ - coreutils \ - diffutils \ - flake8 \ - git \ - gnutls-bin \ - golang \ - iproute2 \ - jq \ - libtool-bin \ - locales \ - make \ - nbd-client \ - nbd-server \ - nbdkit \ - ocaml \ - ocaml-findlib \ - perl-base \ - pkgconf \ - python3-dev \ - qemu-kvm \ - qemu-utils \ - sed && \ - eatmydata apt-get autoremove -y && \ - eatmydata apt-get autoclean -y && \ - sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ - dpkg-reconfigure locales && \ - dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt && \ - mkdir -p /usr/libexec/ccache-wrappers && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/powerpc64le-linux-gnu-c++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/powerpc64le-linux-gnu-cc && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/powerpc64le-linux-gnu-g++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/powerpc64le-linux-gnu-gcc - -RUN export DEBIAN_FRONTEND=noninteractive && \ - dpkg --add-architecture ppc64el && \ - eatmydata apt-get update && \ - eatmydata apt-get dist-upgrade -y && \ - eatmydata apt-get install --no-install-recommends -y dpkg-dev && \ - eatmydata apt-get install --no-install-recommends -y \ - g++-powerpc64le-linux-gnu \ - gcc-powerpc64le-linux-gnu \ - libc6-dev:ppc64el \ - libev-dev:ppc64el \ - libfuse3-dev:ppc64el \ - libglib2.0-dev:ppc64el \ - libgnutls28-dev:ppc64el \ - libxml2-dev:ppc64el && \ - eatmydata apt-get autoremove -y && \ - eatmydata apt-get autoclean -y && \ - mkdir -p /usr/local/share/meson/cross && \ - echo "[binaries]\n\ -c = '/usr/bin/powerpc64le-linux-gnu-gcc'\n\ -ar = '/usr/bin/powerpc64le-linux-gnu-gcc-ar'\n\ -strip = '/usr/bin/powerpc64le-linux-gnu-strip'\n\ -pkgconfig = '/usr/bin/powerpc64le-linux-gnu-pkg-config'\n\ -\n\ -[host_machine]\n\ -system = 'linux'\n\ -cpu_family = 'ppc64'\n\ -cpu = 'powerpc64le'\n\ -endian = 'little'" > /usr/local/share/meson/cross/powerpc64le-linux-gnu - -ENV LANG "en_US.UTF-8" -ENV MAKE "/usr/bin/make" -ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" - -ENV ABI "powerpc64le-linux-gnu" -ENV CONFIGURE_OPTS "--host=powerpc64le-linux-gnu" diff --git a/ci/containers/debian-sid-cross-s390x.Dockerfile b/ci/containers/debian-sid-cross-s390x.Dockerfile deleted file mode 100644 index 84d82fd3ecc9..000000000000 --- a/ci/containers/debian-sid-cross-s390x.Dockerfile +++ /dev/null @@ -1,86 +0,0 @@ -# THIS FILE WAS AUTO-GENERATED -# -# $ lcitool dockerfile --cross s390x debian-sid libnbd -# -# https://gitlab.com/libvirt/libvirt-ci/-/commit/5fd2cdeb8958df86a81b639fd096f... - -FROM docker.io/library/debian:sid-slim - -RUN export DEBIAN_FRONTEND=noninteractive && \ - apt-get update && \ - apt-get install -y eatmydata && \ - eatmydata apt-get dist-upgrade -y && \ - eatmydata apt-get install --no-install-recommends -y \ - autoconf \ - automake \ - bash-completion \ - ca-certificates \ - ccache \ - coreutils \ - diffutils \ - flake8 \ - git \ - gnutls-bin \ - golang \ - iproute2 \ - jq \ - libtool-bin \ - locales \ - make \ - nbd-client \ - nbd-server \ - nbdkit \ - ocaml \ - ocaml-findlib \ - perl-base \ - pkgconf \ - python3-dev \ - qemu-kvm \ - qemu-utils \ - sed && \ - eatmydata apt-get autoremove -y && \ - eatmydata apt-get autoclean -y && \ - sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ - dpkg-reconfigure locales && \ - dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt && \ - mkdir -p /usr/libexec/ccache-wrappers && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/s390x-linux-gnu-c++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/s390x-linux-gnu-cc && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/s390x-linux-gnu-g++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/s390x-linux-gnu-gcc - -RUN export DEBIAN_FRONTEND=noninteractive && \ - dpkg --add-architecture s390x && \ - eatmydata apt-get update && \ - eatmydata apt-get dist-upgrade -y && \ - eatmydata apt-get install --no-install-recommends -y dpkg-dev && \ - eatmydata apt-get install --no-install-recommends -y \ - g++-s390x-linux-gnu \ - gcc-s390x-linux-gnu \ - libc6-dev:s390x \ - libev-dev:s390x \ - libfuse3-dev:s390x \ - libglib2.0-dev:s390x \ - libgnutls28-dev:s390x \ - libxml2-dev:s390x && \ - eatmydata apt-get autoremove -y && \ - eatmydata apt-get autoclean -y && \ - mkdir -p /usr/local/share/meson/cross && \ - echo "[binaries]\n\ -c = '/usr/bin/s390x-linux-gnu-gcc'\n\ -ar = '/usr/bin/s390x-linux-gnu-gcc-ar'\n\ -strip = '/usr/bin/s390x-linux-gnu-strip'\n\ -pkgconfig = '/usr/bin/s390x-linux-gnu-pkg-config'\n\ -\n\ -[host_machine]\n\ -system = 'linux'\n\ -cpu_family = 's390x'\n\ -cpu = 's390x'\n\ -endian = 'big'" > /usr/local/share/meson/cross/s390x-linux-gnu - -ENV LANG "en_US.UTF-8" -ENV MAKE "/usr/bin/make" -ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" - -ENV ABI "s390x-linux-gnu" -ENV CONFIGURE_OPTS "--host=s390x-linux-gnu" diff --git a/ci/containers/fedora-rawhide-cross-mingw32.Dockerfile b/ci/containers/fedora-rawhide-cross-mingw32.Dockerfile deleted file mode 100644 index 08af4c68b0f9..000000000000 --- a/ci/containers/fedora-rawhide-cross-mingw32.Dockerfile +++ /dev/null @@ -1,71 +0,0 @@ -# THIS FILE WAS AUTO-GENERATED -# -# $ lcitool dockerfile --cross mingw32 fedora-rawhide libnbd -# -# https://gitlab.com/libvirt/libvirt-ci/-/commit/5fd2cdeb8958df86a81b639fd096f... - -FROM registry.fedoraproject.org/fedora:rawhide - -RUN dnf update -y --nogpgcheck fedora-gpg-keys && \ - dnf install -y nosync && \ - echo -e '#!/bin/sh\n\ -if test -d /usr/lib64\n\ -then\n\ - export LD_PRELOAD=/usr/lib64/nosync/nosync.so\n\ -else\n\ - export LD_PRELOAD=/usr/lib/nosync/nosync.so\n\ -fi\n\ -exec "$@"' > /usr/bin/nosync && \ - chmod +x /usr/bin/nosync && \ - nosync dnf update -y && \ - nosync dnf install -y \ - autoconf \ - automake \ - bash-completion \ - ca-certificates \ - ccache \ - coreutils \ - diffutils \ - git \ - glibc-langpack-en \ - gnutls \ - golang \ - iproute \ - jq \ - libtool \ - make \ - nbd \ - nbdkit \ - ocaml \ - ocaml-findlib \ - perl-base \ - python3-devel \ - python3-flake8 \ - qemu-img \ - qemu-kvm \ - sed && \ - nosync dnf autoremove -y && \ - nosync dnf clean all -y && \ - rpm -qa | sort > /packages.txt && \ - mkdir -p /usr/libexec/ccache-wrappers && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/i686-w64-mingw32-c++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/i686-w64-mingw32-cc && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/i686-w64-mingw32-g++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/i686-w64-mingw32-gcc - -RUN nosync dnf install -y \ - mingw32-gcc \ - mingw32-gcc-c++ \ - mingw32-glib2 \ - mingw32-gnutls \ - mingw32-headers \ - mingw32-libxml2 \ - mingw32-pkg-config && \ - nosync dnf clean all -y - -ENV LANG "en_US.UTF-8" -ENV MAKE "/usr/bin/make" -ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" - -ENV ABI "i686-w64-mingw32" -ENV CONFIGURE_OPTS "--host=i686-w64-mingw32" diff --git a/ci/containers/fedora-rawhide-cross-mingw64.Dockerfile b/ci/containers/fedora-rawhide-cross-mingw64.Dockerfile deleted file mode 100644 index 67e5305ce1c5..000000000000 --- a/ci/containers/fedora-rawhide-cross-mingw64.Dockerfile +++ /dev/null @@ -1,71 +0,0 @@ -# THIS FILE WAS AUTO-GENERATED -# -# $ lcitool dockerfile --cross mingw64 fedora-rawhide libnbd -# -# https://gitlab.com/libvirt/libvirt-ci/-/commit/5fd2cdeb8958df86a81b639fd096f... - -FROM registry.fedoraproject.org/fedora:rawhide - -RUN dnf update -y --nogpgcheck fedora-gpg-keys && \ - dnf install -y nosync && \ - echo -e '#!/bin/sh\n\ -if test -d /usr/lib64\n\ -then\n\ - export LD_PRELOAD=/usr/lib64/nosync/nosync.so\n\ -else\n\ - export LD_PRELOAD=/usr/lib/nosync/nosync.so\n\ -fi\n\ -exec "$@"' > /usr/bin/nosync && \ - chmod +x /usr/bin/nosync && \ - nosync dnf update -y && \ - nosync dnf install -y \ - autoconf \ - automake \ - bash-completion \ - ca-certificates \ - ccache \ - coreutils \ - diffutils \ - git \ - glibc-langpack-en \ - gnutls \ - golang \ - iproute \ - jq \ - libtool \ - make \ - nbd \ - nbdkit \ - ocaml \ - ocaml-findlib \ - perl-base \ - python3-devel \ - python3-flake8 \ - qemu-img \ - qemu-kvm \ - sed && \ - nosync dnf autoremove -y && \ - nosync dnf clean all -y && \ - rpm -qa | sort > /packages.txt && \ - mkdir -p /usr/libexec/ccache-wrappers && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/x86_64-w64-mingw32-c++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/x86_64-w64-mingw32-cc && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/x86_64-w64-mingw32-g++ && \ - ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/x86_64-w64-mingw32-gcc - -RUN nosync dnf install -y \ - mingw64-gcc \ - mingw64-gcc-c++ \ - mingw64-glib2 \ - mingw64-gnutls \ - mingw64-headers \ - mingw64-libxml2 \ - mingw64-pkg-config && \ - nosync dnf clean all -y - -ENV LANG "en_US.UTF-8" -ENV MAKE "/usr/bin/make" -ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" - -ENV ABI "x86_64-w64-mingw32" -ENV CONFIGURE_OPTS "--host=x86_64-w64-mingw32" diff --git a/ci/containers/refresh b/ci/containers/refresh index 327f642ed2ec..7ce5332b940a 100755 --- a/ci/containers/refresh +++ b/ci/containers/refresh @@ -18,25 +18,5 @@ HOSTS=$($LCITOOL hosts | grep -Ev 'freebsd|macos') for host in $HOSTS do - case "$host" in - fedora-rawhide) - for cross in mingw32 mingw64 - do - $LCITOOL dockerfile $host libnbd --cross $cross > $host-cross-$cross.Dockerfile - done - ;; - debian-*) - for cross in aarch64 armv6l armv7l i686 mips mips64el mipsel ppc64le s390x - do - if test "$host-cross-$cross" = "debian-9-cross-i686" || - test "$host-cross-$cross" = "debian-sid-cross-mips" - then - continue - fi - $LCITOOL dockerfile $host libnbd --cross $cross > $host-cross-$cross.Dockerfile - done - ;; - esac - $LCITOOL dockerfile $host libnbd > $host.Dockerfile done -- 2.31.1
Signed-off-by: Martin Kletzander <mkletzan@redhat.com> --- .gitlab-ci.yml | 8 -------- 1 file changed, 8 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index cd2167c52ab8..ad73daa9c01f 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -159,14 +159,6 @@ x64-debian-10: variables: NAME: debian-10 -x64-debian-10-clang: - extends: .native_build_job - needs: - - x64-debian-10-container - variables: - NAME: debian-10 - CC: clang - x64-debian-sid: extends: .native_build_job needs: -- 2.31.1
The upstream patch for libvirt-ci is still not completed yet, but there are some improvements that need to be pulled in to clean up some of the builds. Signed-off-by: Martin Kletzander <mkletzan@redhat.com> --- ci/cirrus/freebsd-12.vars | 5 ++--- ci/cirrus/freebsd-current.vars | 5 ++--- ci/cirrus/macos-11.vars | 6 +++--- ci/containers/centos-8.Dockerfile | 12 ++++++++---- ci/containers/centos-stream-8.Dockerfile | 12 ++++++++---- ci/containers/debian-10.Dockerfile | 9 ++++++--- ci/containers/debian-sid.Dockerfile | 9 ++++++--- ci/containers/fedora-33.Dockerfile | 14 +++++++++----- ci/containers/fedora-34.Dockerfile | 14 +++++++++----- ci/containers/fedora-rawhide.Dockerfile | 14 +++++++++----- ci/containers/opensuse-leap-152.Dockerfile | 12 ++++++++---- ci/containers/opensuse-tumbleweed.Dockerfile | 12 ++++++++---- ci/containers/ubuntu-1804.Dockerfile | 8 +++++--- ci/containers/ubuntu-2004.Dockerfile | 9 ++++++--- 14 files changed, 89 insertions(+), 52 deletions(-) diff --git a/ci/cirrus/freebsd-12.vars b/ci/cirrus/freebsd-12.vars index 4e7f653a859c..66619a081735 100644 --- a/ci/cirrus/freebsd-12.vars +++ b/ci/cirrus/freebsd-12.vars @@ -2,13 +2,12 @@ # # $ lcitool variables freebsd-12 libnbd # -# https://gitlab.com/libvirt/libvirt-ci/-/commit/ea3fb1a9a5976a2c70b85e4621dce... +# https://gitlab.com/libvirt/libvirt-ci/-/commit/cf6a6438c15ff690d720ee667986e... PACKAGING_COMMAND='pkg' -CC='/usr/bin/clang' CCACHE='/usr/local/bin/ccache' MAKE='/usr/local/bin/gmake' NINJA='/usr/local/bin/ninja' PYTHON='/usr/local/bin/python3' PIP3='/usr/local/bin/pip-3.7' -PKGS='autoconf automake bash-completion ca_root_nss ccache coreutils diffutils fusefs-libs git glib gmake gnutls go gsed libev libtool libxml2 nbdkit ocaml ocaml-findlib perl5 pkgconf python3 qemu' +PKGS='autoconf automake bash-completion ca_root_nss ccache diffutils fusefs-libs3 git glib gmake gnutls go gsed jq libev libtool libxml2 nbd-server nbdkit ocaml ocaml-findlib p5-Pod-Simple perl5 pkgconf py37-flake8 python3 qemu' diff --git a/ci/cirrus/freebsd-current.vars b/ci/cirrus/freebsd-current.vars index dfdf58e8616b..bca65bb0a114 100644 --- a/ci/cirrus/freebsd-current.vars +++ b/ci/cirrus/freebsd-current.vars @@ -2,13 +2,12 @@ # # $ lcitool variables freebsd-current libnbd # -# https://gitlab.com/libvirt/libvirt-ci/-/commit/ea3fb1a9a5976a2c70b85e4621dce... +# https://gitlab.com/libvirt/libvirt-ci/-/commit/cf6a6438c15ff690d720ee667986e... PACKAGING_COMMAND='pkg' -CC='/usr/bin/clang' CCACHE='/usr/local/bin/ccache' MAKE='/usr/local/bin/gmake' NINJA='/usr/local/bin/ninja' PYTHON='/usr/local/bin/python3' PIP3='/usr/local/bin/pip-3.7' -PKGS='autoconf automake bash-completion ca_root_nss ccache coreutils diffutils fusefs-libs git glib gmake gnutls go gsed libev libtool libxml2 nbdkit ocaml ocaml-findlib perl5 pkgconf python3 qemu' +PKGS='autoconf automake bash-completion ca_root_nss ccache diffutils fusefs-libs3 git glib gmake gnutls go gsed jq libev libtool libxml2 nbd-server nbdkit ocaml ocaml-findlib p5-Pod-Simple perl5 pkgconf py37-flake8 python3 qemu' diff --git a/ci/cirrus/macos-11.vars b/ci/cirrus/macos-11.vars index 413a203782eb..90ca6ed28877 100644 --- a/ci/cirrus/macos-11.vars +++ b/ci/cirrus/macos-11.vars @@ -2,13 +2,13 @@ # # $ lcitool variables macos-11 libnbd # -# https://gitlab.com/libvirt/libvirt-ci/-/commit/ea3fb1a9a5976a2c70b85e4621dce... +# https://gitlab.com/libvirt/libvirt-ci/-/commit/cf6a6438c15ff690d720ee667986e... PACKAGING_COMMAND='brew' -CC='/usr/bin/clang' CCACHE='/usr/local/bin/ccache' MAKE='/usr/local/bin/gmake' NINJA='/usr/local/bin/ninja' PYTHON='/usr/local/bin/python3' PIP3='/usr/local/bin/pip3' -PKGS='autoconf automake bash-completion ccache coreutils diffutils git glib gnu-sed gnutls golang libev libtool libxml2 make ocaml ocaml-findlib perl pkg-config python3 qemu' +PKGS='autoconf automake bash-completion ccache cpanminus diffutils flake8 fuse3 git glib gnu-sed gnutls golang jq libev libtool libxml2 make nbd ocaml ocaml-findlib ocamldoc perl pkg-config python3 qemu' +CPAN_PKGS='Pod::Simple' diff --git a/ci/containers/centos-8.Dockerfile b/ci/containers/centos-8.Dockerfile index e5bcd4ee7a9f..fe82a2878570 100644 --- a/ci/containers/centos-8.Dockerfile +++ b/ci/containers/centos-8.Dockerfile @@ -2,7 +2,7 @@ # # $ lcitool dockerfile centos-8 libnbd # -# https://gitlab.com/libvirt/libvirt-ci/-/commit/5fd2cdeb8958df86a81b639fd096f... +# https://gitlab.com/libvirt/libvirt-ci/-/commit/cf6a6438c15ff690d720ee667986e... FROM docker.io/library/centos:8 @@ -18,8 +18,8 @@ RUN dnf update -y && \ ca-certificates \ ccache \ clang \ - coreutils-single \ diffutils \ + fuse3 \ fuse3-devel \ gcc \ gcc-c++ \ @@ -27,8 +27,8 @@ RUN dnf update -y && \ glib2-devel \ glibc-devel \ glibc-langpack-en \ - gnutls \ gnutls-devel \ + gnutls-utils \ golang \ iproute \ jq \ @@ -39,13 +39,17 @@ RUN dnf update -y && \ nbdkit \ ocaml \ ocaml-findlib \ + ocamldoc \ perl \ + perl-Pod-Simple \ + perl-podlators \ pkgconfig \ python3-devel \ python3-flake8 \ qemu-img \ qemu-kvm \ - sed && \ + sed \ + util-linux && \ dnf autoremove -y && \ dnf clean all -y && \ rpm -qa | sort > /packages.txt && \ diff --git a/ci/containers/centos-stream-8.Dockerfile b/ci/containers/centos-stream-8.Dockerfile index e7c18bf3d23f..d00bd32e4f99 100644 --- a/ci/containers/centos-stream-8.Dockerfile +++ b/ci/containers/centos-stream-8.Dockerfile @@ -2,7 +2,7 @@ # # $ lcitool dockerfile centos-stream-8 libnbd # -# https://gitlab.com/libvirt/libvirt-ci/-/commit/5fd2cdeb8958df86a81b639fd096f... +# https://gitlab.com/libvirt/libvirt-ci/-/commit/cf6a6438c15ff690d720ee667986e... FROM quay.io/centos/centos:stream8 @@ -18,8 +18,8 @@ RUN dnf update -y && \ ca-certificates \ ccache \ clang \ - coreutils-single \ diffutils \ + fuse3 \ fuse3-devel \ gcc \ gcc-c++ \ @@ -27,8 +27,8 @@ RUN dnf update -y && \ glib2-devel \ glibc-devel \ glibc-langpack-en \ - gnutls \ gnutls-devel \ + gnutls-utils \ golang \ iproute \ jq \ @@ -39,13 +39,17 @@ RUN dnf update -y && \ nbdkit \ ocaml \ ocaml-findlib \ + ocamldoc \ perl \ + perl-Pod-Simple \ + perl-podlators \ pkgconfig \ python3-devel \ python3-flake8 \ qemu-img \ qemu-kvm \ - sed && \ + sed \ + util-linux && \ dnf autoremove -y && \ dnf clean all -y && \ rpm -qa | sort > /packages.txt && \ diff --git a/ci/containers/debian-10.Dockerfile b/ci/containers/debian-10.Dockerfile index 1d0d5eefa22d..8d9eeb41f714 100644 --- a/ci/containers/debian-10.Dockerfile +++ b/ci/containers/debian-10.Dockerfile @@ -2,7 +2,7 @@ # # $ lcitool dockerfile debian-10 libnbd # -# https://gitlab.com/libvirt/libvirt-ci/-/commit/5fd2cdeb8958df86a81b639fd096f... +# https://gitlab.com/libvirt/libvirt-ci/-/commit/cf6a6438c15ff690d720ee667986e... FROM docker.io/library/debian:10-slim @@ -14,12 +14,13 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ autoconf \ automake \ bash-completion \ + bsdmainutils \ ca-certificates \ ccache \ clang \ - coreutils \ diffutils \ flake8 \ + fuse3 \ g++ \ gcc \ git \ @@ -41,10 +42,12 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ nbdkit \ ocaml \ ocaml-findlib \ + ocaml-nox \ + perl \ perl-base \ pkgconf \ python3-dev \ - qemu-kvm \ + qemu \ qemu-utils \ sed && \ eatmydata apt-get autoremove -y && \ diff --git a/ci/containers/debian-sid.Dockerfile b/ci/containers/debian-sid.Dockerfile index 9e42d34dc56f..f470180e8e6f 100644 --- a/ci/containers/debian-sid.Dockerfile +++ b/ci/containers/debian-sid.Dockerfile @@ -2,7 +2,7 @@ # # $ lcitool dockerfile debian-sid libnbd # -# https://gitlab.com/libvirt/libvirt-ci/-/commit/5fd2cdeb8958df86a81b639fd096f... +# https://gitlab.com/libvirt/libvirt-ci/-/commit/cf6a6438c15ff690d720ee667986e... FROM docker.io/library/debian:sid-slim @@ -14,12 +14,13 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ autoconf \ automake \ bash-completion \ + bsdextrautils \ ca-certificates \ ccache \ clang \ - coreutils \ diffutils \ flake8 \ + fuse3 \ g++ \ gcc \ git \ @@ -41,10 +42,12 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ nbdkit \ ocaml \ ocaml-findlib \ + ocaml-nox \ + perl \ perl-base \ pkgconf \ python3-dev \ - qemu-kvm \ + qemu \ qemu-utils \ sed && \ eatmydata apt-get autoremove -y && \ diff --git a/ci/containers/fedora-33.Dockerfile b/ci/containers/fedora-33.Dockerfile index a3b54c247a22..667819f641d4 100644 --- a/ci/containers/fedora-33.Dockerfile +++ b/ci/containers/fedora-33.Dockerfile @@ -2,7 +2,7 @@ # # $ lcitool dockerfile fedora-33 libnbd # -# https://gitlab.com/libvirt/libvirt-ci/-/commit/5fd2cdeb8958df86a81b639fd096f... +# https://gitlab.com/libvirt/libvirt-ci/-/commit/cf6a6438c15ff690d720ee667986e... FROM registry.fedoraproject.org/fedora:33 @@ -24,8 +24,8 @@ exec "$@"' > /usr/bin/nosync && \ ca-certificates \ ccache \ clang \ - coreutils \ diffutils \ + fuse3 \ fuse3-devel \ gcc \ gcc-c++ \ @@ -33,8 +33,8 @@ exec "$@"' > /usr/bin/nosync && \ glib2-devel \ glibc-devel \ glibc-langpack-en \ - gnutls \ gnutls-devel \ + gnutls-utils \ golang \ iproute \ jq \ @@ -46,13 +46,17 @@ exec "$@"' > /usr/bin/nosync && \ nbdkit \ ocaml \ ocaml-findlib \ + ocamldoc \ + perl-Pod-Simple \ perl-base \ + perl-podlators \ pkgconfig \ python3-devel \ python3-flake8 \ + qemu \ qemu-img \ - qemu-kvm \ - sed && \ + sed \ + util-linux && \ nosync dnf autoremove -y && \ nosync dnf clean all -y && \ rpm -qa | sort > /packages.txt && \ diff --git a/ci/containers/fedora-34.Dockerfile b/ci/containers/fedora-34.Dockerfile index 85f68f107b66..a44bb8b3a3e5 100644 --- a/ci/containers/fedora-34.Dockerfile +++ b/ci/containers/fedora-34.Dockerfile @@ -2,7 +2,7 @@ # # $ lcitool dockerfile fedora-34 libnbd # -# https://gitlab.com/libvirt/libvirt-ci/-/commit/5fd2cdeb8958df86a81b639fd096f... +# https://gitlab.com/libvirt/libvirt-ci/-/commit/cf6a6438c15ff690d720ee667986e... FROM registry.fedoraproject.org/fedora:34 @@ -24,8 +24,8 @@ exec "$@"' > /usr/bin/nosync && \ ca-certificates \ ccache \ clang \ - coreutils \ diffutils \ + fuse3 \ fuse3-devel \ gcc \ gcc-c++ \ @@ -33,8 +33,8 @@ exec "$@"' > /usr/bin/nosync && \ glib2-devel \ glibc-devel \ glibc-langpack-en \ - gnutls \ gnutls-devel \ + gnutls-utils \ golang \ iproute \ jq \ @@ -46,13 +46,17 @@ exec "$@"' > /usr/bin/nosync && \ nbdkit \ ocaml \ ocaml-findlib \ + ocamldoc \ + perl-Pod-Simple \ perl-base \ + perl-podlators \ pkgconfig \ python3-devel \ python3-flake8 \ + qemu \ qemu-img \ - qemu-kvm \ - sed && \ + sed \ + util-linux && \ nosync dnf autoremove -y && \ nosync dnf clean all -y && \ rpm -qa | sort > /packages.txt && \ diff --git a/ci/containers/fedora-rawhide.Dockerfile b/ci/containers/fedora-rawhide.Dockerfile index e50b4837b6ea..17f5345a4e6e 100644 --- a/ci/containers/fedora-rawhide.Dockerfile +++ b/ci/containers/fedora-rawhide.Dockerfile @@ -2,7 +2,7 @@ # # $ lcitool dockerfile fedora-rawhide libnbd # -# https://gitlab.com/libvirt/libvirt-ci/-/commit/5fd2cdeb8958df86a81b639fd096f... +# https://gitlab.com/libvirt/libvirt-ci/-/commit/cf6a6438c15ff690d720ee667986e... FROM registry.fedoraproject.org/fedora:rawhide @@ -25,8 +25,8 @@ exec "$@"' > /usr/bin/nosync && \ ca-certificates \ ccache \ clang \ - coreutils \ diffutils \ + fuse3 \ fuse3-devel \ gcc \ gcc-c++ \ @@ -34,8 +34,8 @@ exec "$@"' > /usr/bin/nosync && \ glib2-devel \ glibc-devel \ glibc-langpack-en \ - gnutls \ gnutls-devel \ + gnutls-utils \ golang \ iproute \ jq \ @@ -47,13 +47,17 @@ exec "$@"' > /usr/bin/nosync && \ nbdkit \ ocaml \ ocaml-findlib \ + ocamldoc \ + perl-Pod-Simple \ perl-base \ + perl-podlators \ pkgconfig \ python3-devel \ python3-flake8 \ + qemu \ qemu-img \ - qemu-kvm \ - sed && \ + sed \ + util-linux && \ nosync dnf autoremove -y && \ nosync dnf clean all -y && \ rpm -qa | sort > /packages.txt && \ diff --git a/ci/containers/opensuse-leap-152.Dockerfile b/ci/containers/opensuse-leap-152.Dockerfile index cfc664a686ca..0e6fc8c44f47 100644 --- a/ci/containers/opensuse-leap-152.Dockerfile +++ b/ci/containers/opensuse-leap-152.Dockerfile @@ -2,7 +2,7 @@ # # $ lcitool dockerfile opensuse-leap-152 libnbd # -# https://gitlab.com/libvirt/libvirt-ci/-/commit/5fd2cdeb8958df86a81b639fd096f... +# https://gitlab.com/libvirt/libvirt-ci/-/commit/cf6a6438c15ff690d720ee667986e... FROM registry.opensuse.org/opensuse/leap:15.2 @@ -14,8 +14,8 @@ RUN zypper update -y && \ ca-certificates \ ccache \ clang \ - coreutils \ diffutils \ + fuse3 \ fuse3-devel \ gcc \ gcc-c++ \ @@ -35,13 +35,17 @@ RUN zypper update -y && \ nbd \ ocaml \ ocaml-findlib \ + ocaml-ocamldoc \ + perl \ + perl-Pod-Simple \ perl-base \ pkgconfig \ python3-devel \ python3-flake8 \ - qemu-kvm \ + qemu \ qemu-tools \ - sed && \ + sed \ + util-linux && \ zypper clean --all && \ rpm -qa | sort > /packages.txt && \ mkdir -p /usr/libexec/ccache-wrappers && \ diff --git a/ci/containers/opensuse-tumbleweed.Dockerfile b/ci/containers/opensuse-tumbleweed.Dockerfile index 0a399aae7ea3..26df75bb2ff1 100644 --- a/ci/containers/opensuse-tumbleweed.Dockerfile +++ b/ci/containers/opensuse-tumbleweed.Dockerfile @@ -2,7 +2,7 @@ # # $ lcitool dockerfile opensuse-tumbleweed libnbd # -# https://gitlab.com/libvirt/libvirt-ci/-/commit/5fd2cdeb8958df86a81b639fd096f... +# https://gitlab.com/libvirt/libvirt-ci/-/commit/cf6a6438c15ff690d720ee667986e... FROM registry.opensuse.org/opensuse/tumbleweed:latest @@ -14,8 +14,8 @@ RUN zypper update -y && \ ca-certificates \ ccache \ clang \ - coreutils \ diffutils \ + fuse3 \ fuse3-devel \ gcc \ gcc-c++ \ @@ -36,13 +36,17 @@ RUN zypper update -y && \ nbdkit \ ocaml \ ocaml-findlib \ + ocaml-ocamldoc \ + perl \ + perl-Pod-Simple \ perl-base \ pkgconfig \ python3-devel \ python3-flake8 \ - qemu-kvm \ + qemu \ qemu-tools \ - sed && \ + sed \ + util-linux && \ zypper clean --all && \ rpm -qa | sort > /packages.txt && \ mkdir -p /usr/libexec/ccache-wrappers && \ diff --git a/ci/containers/ubuntu-1804.Dockerfile b/ci/containers/ubuntu-1804.Dockerfile index 2a4ad5a34715..0426b4392038 100644 --- a/ci/containers/ubuntu-1804.Dockerfile +++ b/ci/containers/ubuntu-1804.Dockerfile @@ -2,7 +2,7 @@ # # $ lcitool dockerfile ubuntu-1804 libnbd # -# https://gitlab.com/libvirt/libvirt-ci/-/commit/5fd2cdeb8958df86a81b639fd096f... +# https://gitlab.com/libvirt/libvirt-ci/-/commit/cf6a6438c15ff690d720ee667986e... FROM docker.io/library/ubuntu:18.04 @@ -14,10 +14,10 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ autoconf \ automake \ bash-completion \ + bsdmainutils \ ca-certificates \ ccache \ clang \ - coreutils \ diffutils \ flake8 \ g++ \ @@ -39,10 +39,12 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ nbd-server \ ocaml \ ocaml-findlib \ + ocaml-nox \ + perl \ perl-base \ pkgconf \ python3-dev \ - qemu-kvm \ + qemu \ qemu-utils \ sed && \ eatmydata apt-get autoremove -y && \ diff --git a/ci/containers/ubuntu-2004.Dockerfile b/ci/containers/ubuntu-2004.Dockerfile index ea1cf9bd7e8a..f23b42ae9241 100644 --- a/ci/containers/ubuntu-2004.Dockerfile +++ b/ci/containers/ubuntu-2004.Dockerfile @@ -2,7 +2,7 @@ # # $ lcitool dockerfile ubuntu-2004 libnbd # -# https://gitlab.com/libvirt/libvirt-ci/-/commit/5fd2cdeb8958df86a81b639fd096f... +# https://gitlab.com/libvirt/libvirt-ci/-/commit/cf6a6438c15ff690d720ee667986e... FROM docker.io/library/ubuntu:20.04 @@ -14,12 +14,13 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ autoconf \ automake \ bash-completion \ + bsdmainutils \ ca-certificates \ ccache \ clang \ - coreutils \ diffutils \ flake8 \ + fuse3 \ g++ \ gcc \ git \ @@ -41,10 +42,12 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ nbdkit \ ocaml \ ocaml-findlib \ + ocaml-nox \ + perl \ perl-base \ pkgconf \ python3-dev \ - qemu-kvm \ + qemu \ qemu-utils \ sed && \ eatmydata apt-get autoremove -y && \ -- 2.31.1
Signed-off-by: Martin Kletzander <mkletzan@redhat.com> --- .gitlab-ci.yml | 10 ++++++++++ ci/cirrus/freebsd-13.vars | 13 +++++++++++++ 2 files changed, 23 insertions(+) create mode 100644 ci/cirrus/freebsd-13.vars diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index ad73daa9c01f..4dd3d55791ca 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -256,6 +256,16 @@ x64-freebsd-12-build: UPDATE_COMMAND: pkg update INSTALL_COMMAND: pkg install -y +x64-freebsd-13-build: + extends: .cirrus_build_job + variables: + NAME: freebsd-13 + CIRRUS_VM_INSTANCE_TYPE: freebsd_instance + CIRRUS_VM_IMAGE_SELECTOR: image_family + CIRRUS_VM_IMAGE_NAME: freebsd-13-0 + UPDATE_COMMAND: pkg update + INSTALL_COMMAND: pkg install -y + x64-macos-11-build: extends: .cirrus_build_job allow_failure: true diff --git a/ci/cirrus/freebsd-13.vars b/ci/cirrus/freebsd-13.vars new file mode 100644 index 000000000000..7df442bad110 --- /dev/null +++ b/ci/cirrus/freebsd-13.vars @@ -0,0 +1,13 @@ +# THIS FILE WAS AUTO-GENERATED +# +# $ lcitool variables freebsd-13 libnbd +# +# https://gitlab.com/libvirt/libvirt-ci/-/commit/cf6a6438c15ff690d720ee667986e... + +PACKAGING_COMMAND='pkg' +CCACHE='/usr/local/bin/ccache' +MAKE='/usr/local/bin/gmake' +NINJA='/usr/local/bin/ninja' +PYTHON='/usr/local/bin/python3' +PIP3='/usr/local/bin/pip-3.7' +PKGS='autoconf automake bash-completion ca_root_nss ccache diffutils fusefs-libs3 git glib gmake gnutls go gsed jq libev libtool libxml2 nbd-server nbdkit ocaml ocaml-findlib p5-Pod-Simple perl5 pkgconf py37-flake8 python3 qemu' -- 2.31.1
Mention how to run the script and keep the same UID, podman and docker use a little bit different arguments for that. While on it also fix the -v parameter for docker which, on some machines, only accepts absolute paths. Signed-off-by: Martin Kletzander <mkletzan@redhat.com> --- ci/README.rst | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/ci/README.rst b/ci/README.rst index 752d8fe46410..9f9a91ed2771 100644 --- a/ci/README.rst +++ b/ci/README.rst @@ -57,13 +57,21 @@ this case from the root of the repository: .. code-block:: shell - podman run -it --rm -v .:/repo:z -w /repo libnbd-fedora-rawhide bash + podman run -it --rm --userns=keep-id -v .:/repo:z -w /repo libnbd-fedora-rawhide bash + +or + +.. code-block:: shell + + docker run -it --rm --user $UID:$UID -v $PWD:/repo:z -w /repo libnbd-fedora-rawhide bash which will bind-mount the current directory (root of the repository in our case) onto /repo inside the container and also use that path as the working directory (just so you do not have to ``cd /repo`` before any commands. This example illustrates running bash, which can be used to debug any issues in the build, -but any command can be specified, for example the build script directly. +but any command can be specified, for example the build script directly. It +also runs the command under a user with the same UID as the user running the +command (even with the same UID). Since the directory is bind-mounted any changes will be visible in your local repository and vice versa. That is useful when you want to, for example, make a -- 2.31.1
Provide a default for `$MAKE` so that the script is more easily usable outside of CI and propagate `$CONFIGURE_OPTS` from lcitool to the configure script. Lastly describe why cross compilation checking is skipped. Even though cross compilation is completely disabled in CI, the more generic of a script we have the easier the adaptation to other projects could be. Signed-off-by: Martin Kletzander <mkletzan@redhat.com> --- ci/build_script.sh | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/ci/build_script.sh b/ci/build_script.sh index 8e2cb5b986ca..cecbbba0bbc0 100755 --- a/ci/build_script.sh +++ b/ci/build_script.sh @@ -3,6 +3,8 @@ set -e main() { + local MAKE="${MAKE-make -j $(getconf _NPROCESSORS_ONLN)}" + autoreconf -if CONFIG_ARGS="\ @@ -20,13 +22,14 @@ main() { CONFIG_ARGS="$CONFIG_ARGS --enable-golang" fi - ./configure $CONFIG_ARGS + ./configure $CONFIGURE_OPTS $CONFIG_ARGS $MAKE if test -n "$CROSS" then - return 0 + echo "Possibly run tests with an emulator in the future" + return 0 fi $MAKE check -- 2.31.1
Some tests cannot be ran successfully and/or cannot properly probe for all their dependency configurations. This allows for skipping of individual test cases based on the OS and its version. Signed-off-by: Martin Kletzander <mkletzan@redhat.com> --- ci/build_script.sh | 60 +++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 59 insertions(+), 1 deletion(-) diff --git a/ci/build_script.sh b/ci/build_script.sh index cecbbba0bbc0..36bd51eb8522 100755 --- a/ci/build_script.sh +++ b/ci/build_script.sh @@ -32,7 +32,65 @@ main() { return 0 fi - $MAKE check + # Add a way to run all the tests, even the skipped ones, with an environment + # variable, so that it can be set fora branch or fork in GitLab. + if test "$SKIPPED_TESTS" != "force" + then + # Skip tests from ci/skipped_tests if this is the right OS version + # The file + local os_id + os_id="$(sh -c '. /etc/os-release; echo "${NAME}-${VERSION_ID}"')" + + echo OS ID: $os_id + + # Skips comments and empty lines + grep '^[^#]' ci/skipped_tests | while read skipper + do + local regex="${skipper%%;*}" + local tests="${skipper#*;}" + + echo SKIPPER: "$skipper" + echo REGEX: "$regex" + echo TESTS: "$tests" + + # Ignore lines not meant for current $os_id + if ! echo "$os_id" | grep -q "$regex"; then echo NOPE; continue; fi + + echo SKIPPING $tests + + for test_case in $tests + do + local test_case_bckup="${test_case}_skipped_test" + if ! git ls-files "$test_case" | grep -q "$test_case" + then + make -C "$(dirname "$test_case")" "$(basename "$test_case")" 2>/dev/null || : + fi + echo Backing up "$test_case" to "${test_case_bckup}" + cp "$test_case" "${test_case_bckup}" + + echo 'echo "Test skipped based on ci/skipped_tests file"'> "$test_case" + echo 'exit 77'>> "$test_case" + chmod +x "$test_case" + done + done + fi + + # We do not want the check to cause complete failure as we need to clean up + # the skipped test cases + local failed=0 + $MAKE check || failed=1 + + find . -name '*_skipped_test' -print | while read test_case_bckup + do + local test_case="${test_case_bckup%_skipped_test}" + + echo Moving "${test_case_bckup}" back to "${test_case}" + rm -f "${test_case}" + mv -f "${test_case_bckup}" "${test_case}" + done + + # Now it is safe to fail + test "$failed" -eq 0 if test "$CHECK_VALGRIND" = "force" then -- 2.31.1
On Tue, Jun 08, 2021 at 09:53:58AM +0200, Martin Kletzander wrote:
Some tests cannot be ran successfully and/or cannot properly probe for all their dependency configurations. This allows for skipping of individual test cases based on the OS and its version.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com> --- ci/build_script.sh | 60 +++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 59 insertions(+), 1 deletion(-)
diff --git a/ci/build_script.sh b/ci/build_script.sh index cecbbba0bbc0..36bd51eb8522 100755 --- a/ci/build_script.sh +++ b/ci/build_script.sh @@ -32,7 +32,65 @@ main() { return 0 fi
- $MAKE check + # Add a way to run all the tests, even the skipped ones, with an environment + # variable, so that it can be set fora branch or fork in GitLab.
for a
+ if test "$SKIPPED_TESTS" != "force" + then + # Skip tests from ci/skipped_tests if this is the right OS version + # The file + local os_id
ci/build_script.sh is currently listed as #!/bin/sh, but local is a bashism. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
On Tue, Jun 08, 2021 at 08:03:57AM -0500, Eric Blake wrote:
On Tue, Jun 08, 2021 at 09:53:58AM +0200, Martin Kletzander wrote:
Some tests cannot be ran successfully and/or cannot properly probe for all their dependency configurations. This allows for skipping of individual test cases based on the OS and its version.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com> --- ci/build_script.sh | 60 +++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 59 insertions(+), 1 deletion(-)
diff --git a/ci/build_script.sh b/ci/build_script.sh index cecbbba0bbc0..36bd51eb8522 100755 --- a/ci/build_script.sh +++ b/ci/build_script.sh @@ -32,7 +32,65 @@ main() { return 0 fi
- $MAKE check + # Add a way to run all the tests, even the skipped ones, with an environment + # variable, so that it can be set fora branch or fork in GitLab.
for a
+ if test "$SKIPPED_TESTS" != "force" + then + # Skip tests from ci/skipped_tests if this is the right OS version + # The file + local os_id
ci/build_script.sh is currently listed as #!/bin/sh, but local is a bashism.
Oh, and it's not as easy as that either as I see: https://stackoverflow.com/questions/18597697/posix-compliant-way-to-scope-va... Anyway, I'll just remove the `local`. I did not go through all the obstacles of making it POSIX-compatible to just throw it away for some variable scoping =) I see I made the same mistake in one previous patch as well, so I'll clean that too. Thanks
-- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
On Tue, Jun 08, 2021 at 03:14:21PM +0200, Martin Kletzander wrote:
On Tue, Jun 08, 2021 at 08:03:57AM -0500, Eric Blake wrote:
On Tue, Jun 08, 2021 at 09:53:58AM +0200, Martin Kletzander wrote:
Some tests cannot be ran successfully and/or cannot properly probe for all their dependency configurations. This allows for skipping of individual test cases based on the OS and its version.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com> --- ci/build_script.sh | 60 +++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 59 insertions(+), 1 deletion(-)
diff --git a/ci/build_script.sh b/ci/build_script.sh index cecbbba0bbc0..36bd51eb8522 100755 --- a/ci/build_script.sh +++ b/ci/build_script.sh @@ -32,7 +32,65 @@ main() { return 0 fi
- $MAKE check + # Add a way to run all the tests, even the skipped ones, with an environment + # variable, so that it can be set fora branch or fork in GitLab.
for a
+ if test "$SKIPPED_TESTS" != "force" + then + # Skip tests from ci/skipped_tests if this is the right OS version + # The file + local os_id
ci/build_script.sh is currently listed as #!/bin/sh, but local is a bashism.
Oh, and it's not as easy as that either as I see:
https://stackoverflow.com/questions/18597697/posix-compliant-way-to-scope-va...
Anyway, I'll just remove the `local`. I did not go through all the obstacles of making it POSIX-compatible to just throw it away for some variable scoping =)
I see I made the same mistake in one previous patch as well, so I'll clean that too.
We're allowed to assume /bin/bash exists in general libnbd tests, although I don't know if that applies here since I'm not totally clear if these CI scripts run in the environment that has all the deps installed or not. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top
On Tue, Jun 08, 2021 at 05:22:06PM +0100, Richard W.M. Jones wrote:
On Tue, Jun 08, 2021 at 03:14:21PM +0200, Martin Kletzander wrote:
On Tue, Jun 08, 2021 at 08:03:57AM -0500, Eric Blake wrote:
On Tue, Jun 08, 2021 at 09:53:58AM +0200, Martin Kletzander wrote:
Some tests cannot be ran successfully and/or cannot properly probe for all their dependency configurations. This allows for skipping of individual test cases based on the OS and its version.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com> --- ci/build_script.sh | 60 +++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 59 insertions(+), 1 deletion(-)
diff --git a/ci/build_script.sh b/ci/build_script.sh index cecbbba0bbc0..36bd51eb8522 100755 --- a/ci/build_script.sh +++ b/ci/build_script.sh @@ -32,7 +32,65 @@ main() { return 0 fi
- $MAKE check + # Add a way to run all the tests, even the skipped ones, with an environment + # variable, so that it can be set fora branch or fork in GitLab.
for a
+ if test "$SKIPPED_TESTS" != "force" + then + # Skip tests from ci/skipped_tests if this is the right OS version + # The file + local os_id
ci/build_script.sh is currently listed as #!/bin/sh, but local is a bashism.
Oh, and it's not as easy as that either as I see:
https://stackoverflow.com/questions/18597697/posix-compliant-way-to-scope-va...
Anyway, I'll just remove the `local`. I did not go through all the obstacles of making it POSIX-compatible to just throw it away for some variable scoping =)
I see I made the same mistake in one previous patch as well, so I'll clean that too.
We're allowed to assume /bin/bash exists in general libnbd tests, although I don't know if that applies here since I'm not totally clear if these CI scripts run in the environment that has all the deps installed or not.
I wanted to keep the build script as portable as reasonably possible. Local variable scoping looks to be available even in the most limited of shells, but there is not need for this here. It was just one thing that I learned because I've seen someone else get bitten by not using it =) This is a linear script that does not even need a function let alone any scopes. Coincidentally it is yet another thing I learned in a very similar fashion. In the meantime I added a fix/workaround for CentOS-es and fixed the FreeBSD weirdness. The only thing missing now is tests on FreeBSD, which fail a lot, and (if there is some spare time) fix up some package name mapping for MacOS. I still have one more thing that bothers me, though. The symlinks in bash/ are set to be generated and packed into the distribution (using the `dist_` prefix for `dist_bashcompdir_DATA`), but they are defined conditionally based on whether bash-completion (and libxml2) are available on the system. This, of course, breaks `make dist` for systems where any of the packages is not available. I think that there are few ways how to get out of it: 1) Remove the `dist_` prefix -- this would generate the files during build time when the availability of the dependencies is more relevant. 2) Remove the conditionals -- creating the files only relies only on `ln` to create the symlink so there is no need for the dependency to be available. 3) Remove the rules and commit the symlinks to the repository -- this would keep the symlinks in git, they would get installed based on the conditionals, but would be always available for packing into the distribution, at least that's how I understand automake. I apologise for hijacking unrelated thread for this, but I would really like your opinion on that. Have a nice day, Martin
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top
On Tue, Jun 08, 2021 at 09:09:32PM +0200, Martin Kletzander wrote:
I still have one more thing that bothers me, though. The symlinks in bash/ are set to be generated and packed into the distribution (using the `dist_` prefix for `dist_bashcompdir_DATA`), but they are defined conditionally based on whether bash-completion (and libxml2) are available on the system. This, of course, breaks `make dist` for systems where any of the packages is not available. I think that there are few ways how to get out of it:
Yeah, it's never a good idea to have the contents of a tarball differ according to what was present on the machine of the person running 'make dist'. Tarballs should be invariant, such that what gets installed should be determined solely by the results of ./configure on the machine unpacking the tarball, not on the machine that created the tarball. So you have indeed uncovered a bug that needs fixing.
1) Remove the `dist_` prefix -- this would generate the files during build time when the availability of the dependencies is more relevant.
That means the files only get installed on platforms where configure says it is useful to install them, rather than being part of the tarball. Seems reasonable.
2) Remove the conditionals -- creating the files only relies only on `ln` to create the symlink so there is no need for the dependency to be available.
That says the files should always be part of the tarball, regardless of whether they will be installed. Also seems reasonable.
3) Remove the rules and commit the symlinks to the repository -- this would keep the symlinks in git, they would get installed based on the conditionals, but would be always available for packing into the distribution, at least that's how I understand automake.
The autotools frown on shipping a tarball with a symlink (because not all systems support symlinks, so extracting such a tarball is not portable). That recommendation still exists today, where the major culprits are Windows and FAT filesystems. Windows actually supports symlinks on NTFS file systems, but not necessarily out-of-the-box, but more importantly, anyone who unpacks the tarball on a flash drive (which are most-often formatted as FAT) suffer from the problem. So just as we shouldn't stick symlinks in the tarball (but instead should generate them during './config.status'), we probably shouldn't need to store symlinks in git. Of all of these options, I'm leaning the most towards 1. But it's not at the top of my priority list at the moment, so I suspect it will be easier for you to propose a patch and me review it than for me to try and write the patch. Thanks for working on CI, by the way! -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
On Wed, Jun 09, 2021 at 11:00:26AM -0500, Eric Blake wrote:
On Tue, Jun 08, 2021 at 09:09:32PM +0200, Martin Kletzander wrote:
I still have one more thing that bothers me, though. The symlinks in bash/ are set to be generated and packed into the distribution (using the `dist_` prefix for `dist_bashcompdir_DATA`), but they are defined conditionally based on whether bash-completion (and libxml2) are available on the system. This, of course, breaks `make dist` for systems where any of the packages is not available. I think that there are few ways how to get out of it:
Yeah, it's never a good idea to have the contents of a tarball differ according to what was present on the machine of the person running 'make dist'. Tarballs should be invariant, such that what gets installed should be determined solely by the results of ./configure on the machine unpacking the tarball, not on the machine that created the tarball. So you have indeed uncovered a bug that needs fixing.
I've been gradually fixing similar issues over time, eg most recently: https://gitlab.com/nbdkit/nbdkit/-/commit/021f44bfed9b222677c90402486bec76c1... So yes, I agree with what Eric said. Rich.
1) Remove the `dist_` prefix -- this would generate the files during build time when the availability of the dependencies is more relevant.
That means the files only get installed on platforms where configure says it is useful to install them, rather than being part of the tarball. Seems reasonable.
2) Remove the conditionals -- creating the files only relies only on `ln` to create the symlink so there is no need for the dependency to be available.
That says the files should always be part of the tarball, regardless of whether they will be installed. Also seems reasonable.
3) Remove the rules and commit the symlinks to the repository -- this would keep the symlinks in git, they would get installed based on the conditionals, but would be always available for packing into the distribution, at least that's how I understand automake.
The autotools frown on shipping a tarball with a symlink (because not all systems support symlinks, so extracting such a tarball is not portable). That recommendation still exists today, where the major culprits are Windows and FAT filesystems. Windows actually supports symlinks on NTFS file systems, but not necessarily out-of-the-box, but more importantly, anyone who unpacks the tarball on a flash drive (which are most-often formatted as FAT) suffer from the problem. So just as we shouldn't stick symlinks in the tarball (but instead should generate them during './config.status'), we probably shouldn't need to store symlinks in git.
Of all of these options, I'm leaning the most towards 1. But it's not at the top of my priority list at the moment, so I suspect it will be easier for you to propose a patch and me review it than for me to try and write the patch.
Thanks for working on CI, by the way!
-- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/
On Wed, Jun 09, 2021 at 06:17:39PM +0100, Richard W.M. Jones wrote:
On Wed, Jun 09, 2021 at 11:00:26AM -0500, Eric Blake wrote:
On Tue, Jun 08, 2021 at 09:09:32PM +0200, Martin Kletzander wrote:
I still have one more thing that bothers me, though. The symlinks in bash/ are set to be generated and packed into the distribution (using the `dist_` prefix for `dist_bashcompdir_DATA`), but they are defined conditionally based on whether bash-completion (and libxml2) are available on the system. This, of course, breaks `make dist` for systems where any of the packages is not available. I think that there are few ways how to get out of it:
Yeah, it's never a good idea to have the contents of a tarball differ according to what was present on the machine of the person running 'make dist'. Tarballs should be invariant, such that what gets installed should be determined solely by the results of ./configure on the machine unpacking the tarball, not on the machine that created the tarball. So you have indeed uncovered a bug that needs fixing.
I've been gradually fixing similar issues over time, eg most recently:
https://gitlab.com/nbdkit/nbdkit/-/commit/021f44bfed9b222677c90402486bec76c1...
So yes, I agree with what Eric said.
Rich.
1) Remove the `dist_` prefix -- this would generate the files during build time when the availability of the dependencies is more relevant.
That means the files only get installed on platforms where configure says it is useful to install them, rather than being part of the tarball. Seems reasonable.
2) Remove the conditionals -- creating the files only relies only on `ln` to create the symlink so there is no need for the dependency to be available.
That says the files should always be part of the tarball, regardless of whether they will be installed. Also seems reasonable.
3) Remove the rules and commit the symlinks to the repository -- this would keep the symlinks in git, they would get installed based on the conditionals, but would be always available for packing into the distribution, at least that's how I understand automake.
The autotools frown on shipping a tarball with a symlink (because not all systems support symlinks, so extracting such a tarball is not portable). That recommendation still exists today, where the major culprits are Windows and FAT filesystems. Windows actually supports symlinks on NTFS file systems, but not necessarily out-of-the-box, but more importantly, anyone who unpacks the tarball on a flash drive (which are most-often formatted as FAT) suffer from the problem. So just as we shouldn't stick symlinks in the tarball (but instead should generate them during './config.status'), we probably shouldn't need to store symlinks in git.
Of all of these options, I'm leaning the most towards 1. But it's not at the top of my priority list at the moment, so I suspect it will be easier for you to propose a patch and me review it than for me to try and write the patch.
I'll replace the "skip make dist on opensuse" with this patch in the next version. Thanks for confirming what I thought.
Thanks for working on CI, by the way!
-- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/
The "script" is kind of implied and a bother to write all the time when testing. Signed-off-by: Martin Kletzander <mkletzan@redhat.com> --- .gitlab-ci.yml | 2 +- ci/README.rst | 4 ++-- ci/{build_script.sh => build.sh} | 0 ci/cirrus/build.yml | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) rename ci/{build_script.sh => build.sh} (100%) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 4dd3d55791ca..2993ca987c4d 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -42,7 +42,7 @@ stages: before_script: - *script_variables script: - - ci/build_script.sh + - ci/build.sh artifacts: paths: - "config.log" diff --git a/ci/README.rst b/ci/README.rst index 9f9a91ed2771..c1c97362e41f 100644 --- a/ci/README.rst +++ b/ci/README.rst @@ -4,8 +4,8 @@ Continuous Integration This document provides describes the continuous integration setup for this repository. -The main script that is executed is ``ci/build_script.sh`` and should contain -all necessary checks to be performed, although some of them might be run +The main script that is executed is ``ci/build.sh`` and should contain all +necessary checks to be performed, although some of them might be run conditionally. lcitool diff --git a/ci/build_script.sh b/ci/build.sh similarity index 100% rename from ci/build_script.sh rename to ci/build.sh diff --git a/ci/cirrus/build.yml b/ci/cirrus/build.yml index 5a5c65e3b640..66e3cdc472cb 100644 --- a/ci/cirrus/build.yml +++ b/ci/cirrus/build.yml @@ -19,4 +19,4 @@ build_task: - git fetch origin "$CI_COMMIT_REF_NAME" - git reset --hard "$CI_COMMIT_SHA" build_script: - - ci/build_script.sh + - ci/build.sh -- 2.31.1
This cleans up most of the mess in GitLab CI. Signed-off-by: Martin Kletzander <mkletzan@redhat.com> --- ci/skipped_tests | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 ci/skipped_tests diff --git a/ci/skipped_tests b/ci/skipped_tests new file mode 100644 index 000000000000..05e2a30e97ad --- /dev/null +++ b/ci/skipped_tests @@ -0,0 +1,9 @@ +# Old nbd-server and built without syslog support, tests deadlock, old qemu-img version +^Ubuntu-18\.04$;interop/interop-nbd-server interop/list-exports-nbd-server interop/structured-read.sh +^openSUSE Leap-15;interop/interop-nbd-server interop/list-exports-nbd-server + +# Similar for Tumbleweed, except tests do not deadlock, only limit to version 2021* for now +^openSUSE Tumbleweed-2021;interop/interop-nbd-server interop/list-exports-nbd-server + +# Debian 10 has weird golang issues (old golang anyway) and old qemu-img +^Debian GNU/Linux-10;golang/run-tests.sh interop/structured-read.sh -- 2.31.1
They do not ship `bash-completion.pc` file, which is required to make the package. It does not completely make sense to me, based on the Makefile, but I do not know where else to check. Signed-off-by: Martin Kletzander <mkletzan@redhat.com> --- .gitlab-ci.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 2993ca987c4d..8365312fece2 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -223,6 +223,7 @@ x64-opensuse-leap-152: - x64-opensuse-leap-152-container variables: NAME: opensuse-leap-152 + DIST: skip x64-opensuse-tumbleweed: extends: .native_build_job @@ -230,6 +231,7 @@ x64-opensuse-tumbleweed: - x64-opensuse-tumbleweed-container variables: NAME: opensuse-tumbleweed + DIST: skip x64-ubuntu-1804: extends: .native_build_job -- 2.31.1
I'm lukewarm about patch 3, but all the rest are fine so ACK on those. More comments below: On Tue, Jun 08, 2021 at 09:53:41AM +0200, Martin Kletzander wrote:
Lot of clean-ups for the CI. I also added support for easily configurable way of skipping particular tests on selected distros. However, I only skipped those that do not have and easy way of detecting a broken dependency and such like. What is still missing is:
- All the CentOS builds have issues with three tests that seem like a protocol violation, but can very well be a bug in some test dependency. Since this is the closest to RHEL we can get, I think it's worth investigating.
I can't access the gitlab pipelines at the moment because there's some problem with gitlab+Firefox, but which version of CentOS was this and what kind of problem was it? FWIW I built and testing libnbd and nbdkit on RHEL 7, RHEL 8 and CentOS 9 Stream yesterday. I fixed a few minor things (see commit history) but basically they were all working.
- FreeBSDs do not even build and it looks like some libtool wrapper script is fed into a C preprocessor, which fails terribly due to the script comments (but should not be fed there at all, of course.
It was building OK for me yesterday (and also on OpenBSD). Do you know what the error was?
- MacOS has issues, although that one is marked to allow failures and the issue does not seem to be on our side anyway. I won't look into it for a while and see if it sorts itself out.
I don't have a way to test/reproduce macOS failures but if I could see the failures I might have some ideas. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v
On Tue, Jun 08, 2021 at 11:01:22AM +0100, Richard W.M. Jones wrote:
I'm lukewarm about patch 3, but all the rest are fine so ACK on those. More comments below:
On Tue, Jun 08, 2021 at 09:53:41AM +0200, Martin Kletzander wrote:
Lot of clean-ups for the CI. I also added support for easily configurable way of skipping particular tests on selected distros. However, I only skipped those that do not have and easy way of detecting a broken dependency and such like. What is still missing is:
- All the CentOS builds have issues with three tests that seem like a protocol violation, but can very well be a bug in some test dependency. Since this is the closest to RHEL we can get, I think it's worth investigating.
I can't access the gitlab pipelines at the moment because there's some problem with gitlab+Firefox, but which version of CentOS was this and
With fastly.com in fact - it seems to be working again now. So the CentOS 8 bug is: nbdkit: file[1]: error: openat: : No such file or directory I'm going to put this down to a bug in nbdkit 1.22.0 that shipped in CentOS 8.4. This will be updated to nbdkit 1.24 in the next release, which shouldn't have this bug. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/
On Tue, Jun 08, 2021 at 11:01:22AM +0100, Richard W.M. Jones wrote:
I'm lukewarm about patch 3, but all the rest are fine so ACK on those. More comments below:
On Tue, Jun 08, 2021 at 09:53:41AM +0200, Martin Kletzander wrote:
Lot of clean-ups for the CI. I also added support for easily configurable way of skipping particular tests on selected distros. However, I only skipped those that do not have and easy way of detecting a broken dependency and such like. What is still missing is:
- All the CentOS builds have issues with three tests that seem like a protocol violation, but can very well be a bug in some test dependency. Since this is the closest to RHEL we can get, I think it's worth investigating.
I can't access the gitlab pipelines at the moment because there's some problem with gitlab+Firefox, but which version of CentOS was this and what kind of problem was it?
That happens to me when I upgrade Firefox and do not restart it. It works for me on FF89. And I looked at the pipelines *a lot* lately =) Anyway, the test-suite.log is available here: https://gitlab.com/nertpinx/libnbd/-/jobs/1326561396/artifacts/file/info/tes... just `curl ... | less` works fine on that link. It mostly complains about: nbdkit: file[1]: error: openat: : No such file or directory ... libnbd: debug: nbd1: nbd_opt_info: leave: error="nbd_opt_info: server replied with error to opt_info request: No such file or directory" couple of times and then ends with: libnbd: debug: nbd1: nbd_shutdown: leave: error="nbd_shutdown: invalid state: CLOSED: the handle must be connected with the server: Invalid argument" libnbd: debug: nbd1: nbd_close: closing handle It happens with info-uri.sh and info-list-uris.sh only. OK, scratch that, I see your reply now, you already looked at it. If that is a bug in nbdkit 1.22, then we can just change the requires of these files to require at least version 1.24.
FWIW I built and testing libnbd and nbdkit on RHEL 7, RHEL 8 and CentOS 9 Stream yesterday. I fixed a few minor things (see commit history) but basically they were all working.
- FreeBSDs do not even build and it looks like some libtool wrapper script is fed into a C preprocessor, which fails terribly due to the script comments (but should not be fed there at all, of course.
It was building OK for me yesterday (and also on OpenBSD). Do you know what the error was?
There has to be some kind of setup that I we do not replicate in the
- MacOS has issues, although that one is marked to allow failures and the issue does not seem to be on our side anyway. I won't look into it for a while and see if it sorts itself out.
I don't have a way to test/reproduce macOS failures but if I could see the failures I might have some ideas.
From my experience it usually just works when it works on FreeBSD, so fixing FreeBSD issues usually helps with that. It turned out I was not properly updating the system before doing anything else. That could also solve the issue above. Since it is an unrelated one-liner I pushed it here: https://gitlab.com/nbdkit/libnbd/-/commit/716d76fdecf6a8f238da5f795a37736fe7... I rebased this series on top of that and will see how it goes while I incorporate Eric's review.
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v
participants (3)
-
Eric Blake -
Martin Kletzander -
Richard W.M. Jones