On Thu, Feb 3, 2022 at 10:52 PM Eric Blake <eblake(a)redhat.com> wrote:
On Thu, Feb 03, 2022 at 10:42:09PM +0200, Nir Soffer wrote:
> > > I wondered why we are using these numbers. I think using same pattern
> > > like {lang}/tests/{name}_test.{suffix} would be nicer.
> >
> > But we want to run them in order, or at least express that there is an
> > ordering to them.
>
> Tests ordering is an anti-pattern. Tests must not depend on each other
> so you can run any tests in any order.
They are independent, you should be able to run any of the tests in
isolation (modulo however easy or hard it is to convince that language
to run a subset of its testsuite). But our numbering still makes
sense: if tests 1xx and 5xx both fail, we are better off debugging the
failure in 1xx first, because we've hosed the core interaction with
the language bindings, whereas 5xx is more the application usage built
on top of what is assumed to be a working core.
These semantics can be expressed better with names. For example
keep the core tests in core/ directory, application tests in application/,
or use file name prefix:
libnbd_core_handle_test.go
...
libnbd_app_aio_copy_test.go
Nir