Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEAT] [New Query Planner] [2/N] Push partition spec into physical plan, remove Coalesce logical op. #1540

Merged
merged 10 commits into from
Oct 30, 2023
Merged

[FEAT] [New Query Planner] [2/N] Push partition spec into physical plan, remove Coalesce logical op. #1540

merged 10 commits into from
Oct 30, 2023

Commits on Oct 27, 2023

  1. Remove Coalesce logical op.

    @clarkzinzow
    clarkzinzow committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    c1a5343 View commit details
    Browse the repository at this point in the history

  2. Push partition spec into physical plan.

    @clarkzinzow
    clarkzinzow committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    6449328 View commit details
    Browse the repository at this point in the history

  3. Add back support for df.num_partitions().

    @clarkzinzow
    clarkzinzow committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    a583401 View commit details
    Browse the repository at this point in the history

  4. Fix optimization tests.

    @clarkzinzow
    clarkzinzow committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    1c0ecc2 View commit details
    Browse the repository at this point in the history

  5. Add test coverage for physical Project op's partition spec translation.

    @clarkzinzow
    clarkzinzow committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    5e248f5 View commit details
    Browse the repository at this point in the history

  6. Add DropRepartitions as logical optimization + planner split.

    @clarkzinzow
    clarkzinzow committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    49a7b79 View commit details
    Browse the repository at this point in the history

  7. Apply suggestions from code review 

    Co-authored-by: Jay Chia <[email protected]>
    @clarkzinzow @jaychia
    clarkzinzow and jaychia authored Oct 27, 2023
    Configuration menu
    Copy the full SHA
    4b79560 View commit details
    Browse the repository at this point in the history

Commits on Oct 30, 2023

  1. Disallow repartitioning with Range scheme at builder stage.

    @clarkzinzow
    clarkzinzow committed Oct 30, 2023
    Configuration menu
    Copy the full SHA
    4de97f4 View commit details
    Browse the repository at this point in the history

  2. Refactor to consolidated + eager drop repartitioning short-circuit.

    @clarkzinzow
    clarkzinzow committed Oct 30, 2023
    Configuration menu
    Copy the full SHA
    06d45c0 View commit details
    Browse the repository at this point in the history

  3. Add DropRepartition comment about logical -> physical optimization.

    @clarkzinzow
    clarkzinzow committed Oct 30, 2023
    Configuration menu
    Copy the full SHA
    824d5b5 View commit details
    Browse the repository at this point in the history