Skip to content
This repository has been archived by the owner on Aug 2, 2022. It is now read-only.

Support ranking window functions #753

Merged

Conversation

dai-chen
Copy link
Member

@dai-chen dai-chen commented Sep 22, 2020

Issue #, if available: #751

Description of changes: This PR is to add basic support for ranking window functions.

  1. Add window operator and related code changes: AST/logical/physical window operator, visitor and other util code.
  2. Add ranking window functions: ROW_NUMBER, RANK and DENSE_RANK.
  3. Rename PPL_ASC/DESC to DEFAULT_ASC/DESC.

To-Dos: Window function name: for now toString() is used as symbol name. This restricts multiple use of same window function, ex. SELECT RANK() OVER(...), RANK() OVER(...).

Design: In Abstract Syntax Tree (AST), the window function is still on the project item list in Project AST node. After analyzing, it is separated into an individual logical Window operator with window function and its definition along with Sort operator that guarantee order of input data of window operator. Eventually both converts to its correspondent physical operators.

window_function (5) (2)-design

Testing: Add UT and comparison test: https://github.com/dai-chen/sql/blob/window-function-poc/integ-test/src/test/resources/correctness/queries/window.txt

Documentation: (Note that you need to enable the new SQL engine to try out this feature: https://github.com/opendistro-for-elasticsearch/sql/blob/master/docs/user/admin/settings.rst#opendistro-sql-engine-new-enabled)

  1. Ranking functions: https://github.com/opendistro-for-elasticsearch/sql/blob/develop/docs/user/dql/window.rst
  2. Limitations: https://github.com/opendistro-for-elasticsearch/sql/blob/develop/docs/user/limitations/limitations.rst

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@dai-chen dai-chen added BI integration Issues for integration with BI tools SQL labels Sep 22, 2020
@dai-chen dai-chen self-assigned this Sep 22, 2020
@dai-chen dai-chen marked this pull request as ready for review October 5, 2020 16:07
@dai-chen dai-chen requested a review from chloe-zh October 7, 2020 21:06
@dai-chen dai-chen added the enhancement New feature or request label Oct 7, 2020
Copy link
Contributor

@penghuo penghuo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the change.

@dai-chen dai-chen merged commit 7e5f582 into opendistro-for-elasticsearch:develop Oct 9, 2020
@dai-chen dai-chen deleted the window-function-poc branch October 9, 2020 21:41
@dai-chen dai-chen restored the window-function-poc branch October 12, 2020 18:16
@dai-chen dai-chen deleted the window-function-poc branch October 12, 2020 22:35
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
BI integration Issues for integration with BI tools enhancement New feature or request SQL
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants