Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue #57 - Introducing DataClasses and Pydantic (Static Type) Validation #99

Merged
merged 7 commits into from
Aug 4, 2021

Conversation

robons
Copy link
Contributor

@robons robons commented Jul 23, 2021

N.B. This PR relies on 3 separate bug fixes to the pydantic library which are currently stored in the forked repo at https://github.com/robons/pydantic/
N.B. - This does not allow us to start using specific validators on a per-attribute basis. This is something we might want to look into in future.

Dataclasses

This PR makes quite a change to our code-base. It enables us to easily validate that all of a class' attributes match the associated static type annotations. To support this, and to make a bit easier, it switches the majority of csvqb's models over to being dataclasses.

Dataclasses give us a simpler way of defining classes and makes our lives easier by implementing common functions you might want on your class including init, repr, str, eq, etc.

Dataclasses also provide an easy way for converting your model to a dictionary and specifying what the default values are for your parameters.

And all you have to do is annotate the model with the @dataclass attribute...

@dataclass
class NewQbDimension(QbDimension):
    label: str
    description: Optional[str] = field(default=None, repr=False)
    code_list: Optional[QbCodeList] = field(default=None, repr=False)
    parent_dimension_uri: Optional[str] = field(default=None, repr=False)
    source_uri: Optional[str] = field(default=None, repr=False)
    range_uri: Optional[str] = field(default=None, repr=False)
    uri_safe_identifier_override: Optional[str] = field(default=None, repr=False)

Note that I haven't had to write the __init__ function, or any other the other functions that the dataclasses will implement for me.

Examples

Example of validation process:

from csvqb.models.cube import *
cube = Cube(CatalogMetadata(None), None, [
    QbColumn("Some Column", NewQbDimension(None)),
    QbColumn(None, ExistingQbAttribute(None))
])
[error.message for error in cube.validate()]
[
    "Cube(metadata=CatalogMetadata(title=None)) - ('metadata', 'title') - none is not an allowed value",
    "Cube(metadata=CatalogMetadata(title=None)) - ('columns', 0, 'component', 'label') - none is not an allowed value",
    "Cube(metadata=CatalogMetadata(title=None)) - ('columns', 1, 'csv_column_title') - none is not an allowed value",
    "Cube(metadata=CatalogMetadata(title=None)) - ('columns', 1, 'component', 'attribute_uri') - none is not an allowed value"
]

Example where user has incorrectly defined columns:

from csvqb.models.cube import *

cube = Cube(CatalogMetadata("Some Title"), None, [
    NewQbUnit("Shouldn't be here")
])
[error.message for error in cube.validate()]
[
    "Cube(metadata=CatalogMetadata(title='Some Title')) - ('columns', 0) - instance of CsvColumn, tuple or dict expected",
    "'NewQbUnit' object has no attribute 'csv_column_title'"
]

It's also possible to directly call the pydantic_validation() method on any model which extends PydanticModel.

This satisfies Issue #57

@robons robons marked this pull request as ready for review August 3, 2021 15:41
@robons robons requested a review from canwaf August 3, 2021 15:41
@robons robons changed the title Issue #57 - Culmination of work on introducing pydantic validation to… Issue #57 - Introducing DataClasses and Pydantic (Static Type) Validation Aug 3, 2021
Copy link
Contributor

@canwaf canwaf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thus concludes my review. Happy merging.

csvqb/csvqb/models/cube/csvqb/columns.py Show resolved Hide resolved
csvqb/csvqb/models/cube/catalog.py Show resolved Hide resolved
csvqb/csvqb/models/validationerror.py Show resolved Hide resolved
csvqb/csvqb/tests/unit/pydantic/test_cube.py Show resolved Hide resolved
@robons robons merged commit dcd4aba into main Aug 4, 2021
@robons robons deleted the robons-57-pydantic branch August 4, 2021 16:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants