Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Batching support for row-based bounded window functions #9973

Merged
merged 25 commits into from
Dec 13, 2023
Merged

Batching support for row-based bounded window functions #9973

merged 25 commits into from
Dec 13, 2023

Commits on Nov 14, 2023

  1. Initial swipe: Batched, bounded windows.

    @mythrocks
    mythrocks committed Nov 14, 2023
    Configuration menu
    Copy the full SHA
    ae3eec0 View commit details
    Browse the repository at this point in the history

  2. Bounded windows routed to BatchedBoundedWindowExec. 

    GpuBatchedBoundedWindowExec is currently identical to GpuWindowExec,
in that it does no batching yet.
After rerouting, the tests all seem to still pass.
    @mythrocks
    mythrocks committed Nov 14, 2023
    Configuration menu
    Copy the full SHA
    2aa3f5d View commit details
    Browse the repository at this point in the history

Commits on Nov 20, 2023

  1. First swipe at new version. 

    Compiling.
Yet to test.

Signed-off-by: MithunR <[email protected]>
    @mythrocks
    mythrocks committed Nov 20, 2023
    Configuration menu
    Copy the full SHA
    cb45866 View commit details
    Browse the repository at this point in the history

Commits on Nov 22, 2023

  1. Mostly working, but buggy. Losing some rows in the output.

    @mythrocks
    mythrocks committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    c3cb1a4 View commit details
    Browse the repository at this point in the history

Commits on Dec 1, 2023

  1. Fixed up the math. Looks to be working at 150M.

    @mythrocks
    mythrocks committed Dec 1, 2023
    Configuration menu
    Copy the full SHA
    1230fed View commit details
    Browse the repository at this point in the history

Commits on Dec 4, 2023

  1. Minor refactor/cleanup.

    @mythrocks
    mythrocks committed Dec 4, 2023
    Configuration menu
    Copy the full SHA
    159dc62 View commit details
    Browse the repository at this point in the history

  2. Clearing cache on task completion.

    @mythrocks
    mythrocks committed Dec 4, 2023
    Configuration menu
    Copy the full SHA
    392e7c6 View commit details
    Browse the repository at this point in the history

Commits on Dec 5, 2023

  1. Fixed leak from trim().

    @mythrocks
    mythrocks committed Dec 5, 2023
    Configuration menu
    Copy the full SHA
    5dbbdd2 View commit details
    Browse the repository at this point in the history

  2. Document onTaskCompletion.

    @mythrocks
    mythrocks committed Dec 5, 2023
    Configuration menu
    Copy the full SHA
    d00747f View commit details
    Browse the repository at this point in the history

  3. Optimization: Skip window kernel if no output for current batch.

    @mythrocks
    mythrocks committed Dec 5, 2023
    Configuration menu
    Copy the full SHA
    b5bd065 View commit details
    Browse the repository at this point in the history

  4. Removed commented code, prints.

    @mythrocks
    mythrocks committed Dec 5, 2023
    Configuration menu
    Copy the full SHA
    94d9bc4 View commit details
    Browse the repository at this point in the history

  5. Switched to exposing negative minPreceding. 

    Also built safety guards to disable optimization for very large
window extents.
    @mythrocks
    mythrocks committed Dec 5, 2023
    Configuration menu
    Copy the full SHA
    60245fe View commit details
    Browse the repository at this point in the history

Commits on Dec 6, 2023

  1. Removed incorrect error message.

    @mythrocks
    mythrocks committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    682afdc View commit details
    Browse the repository at this point in the history

  2. Tests for varying finite window combinations.

    @mythrocks
    mythrocks committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    4767080 View commit details
    Browse the repository at this point in the history

  3. Tests for unpartitioned cases. 

    Plus, some minor reformatting.
    @mythrocks
    mythrocks committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    fa5ab16 View commit details
    Browse the repository at this point in the history

  4. Fixed leak in concat.

    @mythrocks
    mythrocks committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    3f50224 View commit details
    Browse the repository at this point in the history

  5. Test that large extents fall back to GpuWindowExec. 

    Signed-off-by: MithunR <[email protected]>
    @mythrocks
    mythrocks committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    e08cfa2 View commit details
    Browse the repository at this point in the history

  6. Fix build break with Scala 2.13.

    @mythrocks
    mythrocks committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    e0581c0 View commit details
    Browse the repository at this point in the history

Commits on Dec 7, 2023

  1. Support for negative offsets. 

    This now allows for `LEAD()`, `LAG()`, and regular window functions with
negative values for `preceding`,`following` window bounds.
    @mythrocks
    mythrocks committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    0ba8be6 View commit details
    Browse the repository at this point in the history

  2. Removed erroneous batching. 

    This commit fixes the batching.  The new exec should not have to receive
batched input.
    @mythrocks
    mythrocks committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    b5fda09 View commit details
    Browse the repository at this point in the history

  3. Config changes: 

    1. Renamed config. '.extent' to '.max'.
2. Fixed documentation for said config.
3. Removed TODOs that were already handled.
    @mythrocks
    mythrocks committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    f9de13a View commit details
    Browse the repository at this point in the history

  4. Merge remote-tracking branch 'origin/branch-24.02' into batched-bound… 

    …ed-windows
    @mythrocks
    mythrocks committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    31896e3 View commit details
    Browse the repository at this point in the history

  5. Docs update for batched row window config.

    @mythrocks
    mythrocks committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    4a5f5c5 View commit details
    Browse the repository at this point in the history

Commits on Dec 12, 2023

  1. Merge remote-tracking branch 'origin/branch-24.02' into batched-bound… 

    …ed-windows
    @mythrocks
    mythrocks committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    4246cbb View commit details
    Browse the repository at this point in the history

Commits on Dec 13, 2023

  1. Fixed output column order. This fixes the empty output problem.

    @mythrocks
    mythrocks committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    5d475ca View commit details
    Browse the repository at this point in the history