Race condition in VM setup?
Running libguestfs-related tools as part of tests (libguestfs itself, guestfs-tools, nbdkit, etc.) in Debian's build and autopkgtest infrastructure would lead to flaky tests which I had trouble reproducing. Examples of bug reports relating to this problem: - https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1121473 - https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1123409 - https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1120924 Adding a partition and then creating or mounting a filesystem failed pretty often. I suspected a race condition involving udev creating the device node and I think it is more likely if qemu runs in TCG mode. For the Debian package I have added patches that makes the mkfs and mount functions simply wait up to 15s for the device node or path to appear, see below. This seems to have fixed the problem, at least I haven't seen reports breaking test runs since I added it. Is there a better way to do this? Cheers, -Hilko diff --git a/daemon/mkfs.c b/daemon/mkfs.c index fe2e678..66c937b 100644 --- a/daemon/mkfs.c +++ b/daemon/mkfs.c @@ -52,6 +52,14 @@ do_mkfs (const char *fstype, const char *device, int blocksize, CLEANUP_FREE char *err = NULL; int extfs = 0; + /* Wait up to 15s for device to appear */ + for (int j=0; i < 150; i++) { + if (!access (device, F_OK)) { + break; + } + usleep(100000); + } + if (fstype_is_extfs (fstype)) extfs = 1; diff --git a/daemon/mount.ml b/daemon/mount.ml index 171bea1..2f74555 100644 --- a/daemon/mount.ml +++ b/daemon/mount.ml @@ -30,6 +30,18 @@ let mount_vfs options vfs mountable mountpoint = let args = ref [] in + (* Wait up to 15s for device to appear *) + (match mountable.m_type with + | MountableDevice | MountablePath -> + if String.starts_with "/dev/" mountable.m_device then + let i = ref 0 in + while (not (Sys.file_exists mountable.m_device)) && (!i < 150) do + Unix.sleepf 0.1; + i := !i + 1; + done + | _ -> () + ); + (* -o options *) (match options, mountable.m_type with | "", (MountableDevice | MountablePath) -> ()
On Fri, Feb 13, 2026 at 05:08:57PM +0100, Hilko Bengen wrote:
Running libguestfs-related tools as part of tests (libguestfs itself, guestfs-tools, nbdkit, etc.) in Debian's build and autopkgtest infrastructure would lead to flaky tests which I had trouble reproducing.
Examples of bug reports relating to this problem: - https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1121473 - https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1123409 - https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1120924
Adding a partition and then creating or mounting a filesystem failed pretty often. I suspected a race condition involving udev creating the device node and I think it is more likely if qemu runs in TCG mode.
Yes this is correct. It's this systemd bug: https://github.com/systemd/systemd/issues/40499 There is a systemd fix in (upcoming) v260, see the bug for details. Adding a workaround is possible, but I think we'd prefer to get the systemd fix instead. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com nbdkit - Flexible, fast NBD server with plugins https://gitlab.com/nbdkit/nbdkit
participants (2)
-
Hilko Bengen -
Richard W.M. Jones