On Tue, Apr 16, 2024 at 10:22:32PM +0100, Richard W.M. Jones wrote: > This has not worked for a long time because of a fundamental bug in > how Ruby's garbage collector interacts with pthread stacks. There has > been a bug open for a long time (over 14 years) which has never been > fixed: > > https://redmine.ruby-lang.org/issues/2294#note-34 > > As that discussion is rather hard to follow, the basic problem is > this: > > When you embed Ruby you must tell the interpreter where "the stack" > is. You do this by calling RUBY_INIT_STACK early on (which we do in > load(), on the main thread). This marks the top of stack. > > From time to time Ruby's garbage collector will scan the stack > starting at the current stack pointer, and going up til it reaches the > previously marked top of stack. It scans this memory looking for > roots etc. > > This strategy fails completely if you have threads. In a thread > (using a different stack entirely) it scans randomly across the heap, > resulting in weird behaviour at best, but more usually straight > crashes. Urgh, horrible. > > We've had a Ruby plugin for a long time, but it broke with Ruby 1.9, > when Ruby changed from some kind of cooperative threading to using > native threads (and therefore separate stacks). > > The tests have been disabled since early 2018 (nbdkit < 1.2). I > re-enabled the tests just now (Ruby 3.3) but it's broken in the same > way as ever. > > The only changes that have happened since the tests were disabled are > mechanical ones / project-wide cleanups. > > It currently crashes in open(). > > It seems highly unlikely that anyone is using this plugin. IIUC, the '--threads' arg is just a per-client limit. So if there are multiple clients, you'll still get many threads using ruby even with "--threads = 1".

If spawning nbdkit with --single though, you'll get a separate process per client, and then with --threads=1, you could can sure that only 1 thread is calling into ruby.

Is there a way a plugin can declare that it is non-threadsafe, and thus allow nbdkit to refuse to run it without --single and --threads=1, being set ?

I guess that will complicate your test suite though unless you can skip all tests which rely on multi-connections and threads.

With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|