[tests] numpydantic.testing - exposing helpers for 3rd-party interface development & combinatoric testing

[sneakers-the-rat]

Currently

we have fixtures for specifying a number of test cases for dtypes and shapes, and then each interface validates against those test cases.

To make that work, we..

• have a ValidationCase model that takes an annotation and a specification for an array to test against it.
• The ValidationCase model can generates a model with an annotation that matches the spec in the validation case
• each interface then has helper methods to generate an array from the spec and then to validate it.

The shape/dtype cases look something like this:

@pytest.fixture(
    scope="module",
    params=[
        ValidationCase(annotation=RGB_UNION, shape=(5, 5), passes=True),
        ValidationCase(annotation=RGB_UNION, shape=(5, 5, 3), passes=True),
        ValidationCase(annotation=RGB_UNION, shape=(5, 5, 3, 6), passes=False),
        ValidationCase(annotation=RGB_UNION, shape=(5, 5, 4, 6), passes=False),
        # ...
    ])
def shape_cases(request) -> ValidationCase:
    return request.param

The numpy interface tests are like this

def numpy_array(case: ValidationCase) -> np.ndarray:
    if issubclass(case.dtype, BaseModel):
        return np.full(shape=case.shape, fill_value=case.dtype(x=1))
    else:
        return np.zeros(shape=case.shape, dtype=case.dtype)

def _test_np_case(case: ValidationCase):
    array = numpy_array(case)
    if case.passes:
        case.model(array=array)
    else:
        with pytest.raises((ValidationError, DtypeError, ShapeError)):
            case.model(array=array)

@pytest.mark.shape
def test_numpy_shape(shape_cases):
    _test_np_case(shape_cases)

@pytest.mark.dtype
def test_numpy_dtype(dtype_cases):
    _test_np_case(dtype_cases)

Problem

this is proving to be a little bit brittle, and we don't test against, eg, the product of the dtype and shape cases even though we should.

we also want to test behaviors that should be true for all generators, all dtypes, and all shapes, but currently we can only run those tests against a fixed model and array pair for each interface.

we also want to make it possible for people to develop their own interfaces, and so it would be nice if we could facilitate that by providing structure for testing them and a set of tests that need to pass in order for an interface to be added to the package. (or not if they don't want to idrc)

So this PR...

• enables linting for the testing module
• Creates a new model InterfaceTestHelper that provides classmethods for generating an array from a ValidationCase and for validating the array against it
• Creates a new numpydantic.testing module that contains
  • the testing helper classes
  • the test cases as tuples and product iterators
• Expands the ValidationCase to take an InterfaceTestHelper which generalizes what is currently repeated for each of the interfaces with a lot of boilerplate.
• Adds a merge method to ValidationCase to allow multiple partial validation cases to be combined.
• Adds combinatoric test fixtures for combinations of shape, dtype, and interface

Also

• bugfix model type validation from json in dask and numpy
• dump dtype with zarr round tripping and use it when restoring

[coveralls]

Pull Request Test Coverage Report for Build 11289913767

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

• For more information on this, see Tracking coverage changes with pull request builds.
• To avoid this issue with future PRs, see these Recommended CI Configurations.
• For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

• 313 of 317 (98.74%) changed or added relevant lines in 10 files are covered.
• 9 unchanged lines in 3 files lost coverage.
• Overall coverage decreased (-0.4%) to 98.351%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
src/numpydantic/interface/numpy.py	7	8	87.5%
src/numpydantic/serialization.py	1	2	50.0%
src/numpydantic/testing/interfaces.py	132	134	98.51%

Files with Coverage Reduction	New Missed Lines	%
src/numpydantic/interface/zarr.py	1	99.03%
src/numpydantic/interface/interface.py	3	98.56%
src/numpydantic/interface/video.py	5	96.24%

Totals
Change from base Build 11063861484:	-0.4%
Covered Lines:	1491
Relevant Lines:	1516

---

💛 - Coveralls