Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] Add an allowlist of DataTypes that ColumnRangeStatistics supports and validation of TableStatistics #1632

Merged
merged 13 commits into from
Nov 27, 2023
Merged

[BUG] Add an allowlist of DataTypes that ColumnRangeStatistics supports and validation of TableStatistics #1632

merged 13 commits into from
Nov 27, 2023

Commits on Nov 23, 2023

  1. Add an allowlist of DataTypes that ColumnRangeStatistics supports

    Jay Chia committed Nov 23, 2023
    Configuration menu
    Copy the full SHA
    b2d3c4c View commit details
    Browse the repository at this point in the history

  2. Perform pruning in TableStats

    Jay Chia committed Nov 23, 2023
    Configuration menu
    Copy the full SHA
    faf036f View commit details
    Browse the repository at this point in the history

  3. Add TODOs

    Jay Chia committed Nov 23, 2023
    Configuration menu
    Copy the full SHA
    26dff4d View commit details
    Browse the repository at this point in the history

  4. Use individual schams instead of unioned schema

    Jay Chia committed Nov 23, 2023
    Configuration menu
    Copy the full SHA
    aaf02cf View commit details
    Browse the repository at this point in the history

  5. Iterate through schema instead of RowGroupMetadata when constructing … 

    …statistics
    Jay Chia committed Nov 23, 2023
    Configuration menu
    Copy the full SHA
    a58b431 View commit details
    Browse the repository at this point in the history

  6. Add validation of MicroPartitions in ::new_* functions

    Jay Chia committed Nov 23, 2023
    Configuration menu
    Copy the full SHA
    c5284ef View commit details
    Browse the repository at this point in the history

  7. Added validations but now failing roundtrip tests

    Jay Chia committed Nov 23, 2023
    Configuration menu
    Copy the full SHA
    1d49106 View commit details
    Browse the repository at this point in the history

  8. Fix 3 failing roundtrip tests by casting the statistics Series when p… 

    …arsing from Parquet
    Jay Chia committed Nov 23, 2023
    Configuration menu
    Copy the full SHA
    2286c38 View commit details
    Browse the repository at this point in the history

  9. Cast TableStatistics to schema and loosen validation against ScanTask… 

    … schema
    Jay Chia committed Nov 23, 2023
    Configuration menu
    Copy the full SHA
    cfdb8ab View commit details
    Browse the repository at this point in the history

  10. rm is_subset function

    Jay Chia committed Nov 23, 2023
    Configuration menu
    Copy the full SHA
    60608ec View commit details
    Browse the repository at this point in the history

  11. rm is_subset function

    Jay Chia committed Nov 23, 2023
    Configuration menu
    Copy the full SHA
    eddba58 View commit details
    Browse the repository at this point in the history

  12. reuse casting logic

    Jay Chia committed Nov 23, 2023
    Configuration menu
    Copy the full SHA
    99463be View commit details
    Browse the repository at this point in the history

Commits on Nov 27, 2023

  1. Refactor casting logic for ColumnRangeStatistics to only allow certai… 

    …n types
    Jay Chia committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    e0507d8 View commit details
    Browse the repository at this point in the history