On Thu, May 26, 2022 at 09:58:50AM +0100, Richard W.M. Jones wrote:
Is there any way to do this without the literal sleeps? Gitlab CI in particular appears to be very contended (I guess it runs in parallel on huge systems with vast numbers of unrelated containers). I've seen threads being created that are so starved they never run at all even in tests running for many tens of seconds.
IIUC, GitLab uses GCE "spot" VMs for its shared runners https://cloud.google.com/compute/docs/instances/spot TL;DR; these are massively cheaper than normal VMs (60-91% cheaper), but GCE might pre-empt the VM if it needs to reclaim resources for more important VMs on the host. IOW, we have to expect a (usually low level) of non-determinsitic failures and stalls from the CI jobs using shared runners, and be willing to hit the restart job button if a problem occurs. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|