Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add test to ensure every table update has corresponding _apply_table_update function #952

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

kevinjqliu
Copy link
Contributor

@kevinjqliu kevinjqliu commented Jul 21, 2024

Pre-req: #822. This test will fail until #822 is merged

While looking at #864 and #950, I noticed that some TableUpdate classes do not have corresponding _apply_table_update function.
I think we should enforce this in tests so that any future TableUpdate have the necessary _apply_table_update to process the update.

Example

This test currently fails since RemoveSnapshotRefUpdate and RemoveSnapshotsUpdate both are missing corresponding _apply_table_update function.

Run:
poetry run pytest tests/table/test_init.py::test_table_update_have_corresponding_dispatch

================================================================== FAILURES ==================================================================
_______________________________________________ test_table_update_have_corresponding_dispatch ________________________________________________

    def test_table_update_have_corresponding_dispatch() -> None:
        from pyiceberg.table import TableUpdate, _apply_table_update
    
        # every TableUpdate class should have corresponding `_apply_table_update` dispatch function
        table_update_class = set(TableUpdate.__origin__.__args__)  # type: ignore
        dispatch_function_class = set(_apply_table_update.registry.keys())
        missing_dispatch_function = table_update_class - dispatch_function_class
>       assert len(missing_dispatch_function) == 0, f"Missing dispatch function for {missing_dispatch_function}"
E       AssertionError: Missing dispatch function for {<class 'pyiceberg.table.RemoveSnapshotRefUpdate'>, <class 'pyiceberg.table.RemoveSnapshotsUpdate'>}
E       assert 2 == 0
E        +  where 2 = len({<class 'pyiceberg.table.RemoveSnapshotRefUpdate'>, <class 'pyiceberg.table.RemoveSnapshotsUpdate'>})

tests/table/test_init.py:1166: AssertionError
========================================================== short test summary info ===========================================================
FAILED tests/table/test_init.py::test_table_update_have_corresponding_dispatch - AssertionError: Missing dispatch function for {<class 'pyiceberg.table.RemoveSnapshotRefUpdate'>, <class 'pyiceberg.table.RemoveSnapshots...

@kevinjqliu kevinjqliu force-pushed the kevinjqliu/enforce-table-update-dispatch branch from be27ba3 to edf0c17 Compare August 7, 2024 18:00
@kevinjqliu kevinjqliu changed the title add test Add test to ensure every table update has corresponding _apply_table_update function Aug 7, 2024
@kevinjqliu kevinjqliu added this to the PyIceberg 0.9.0 release milestone Oct 30, 2024
@kevinjqliu kevinjqliu force-pushed the kevinjqliu/enforce-table-update-dispatch branch from edf0c17 to c5bcea7 Compare November 3, 2024 00:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant