Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-28067][SQL] Fix incorrect results for decimal aggregate sum by returning null on decimal overflow #27627

Closed
wants to merge 15 commits into from
Closed

[SPARK-28067][SQL] Fix incorrect results for decimal aggregate sum by returning null on decimal overflow #27627

wants to merge 15 commits into from

Commits on Apr 27, 2020

  1. Fix the incorrect results for sum for decimal overflow, support repor… 

    …ting null for it when ansiEnabled is false and throw exception if ansiEnabled is true
    @skambha
    skambha committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    edb32b9 View commit details
    Browse the repository at this point in the history

  2. Add the jira testcase

    @skambha
    skambha committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    8fe9aa4 View commit details
    Browse the repository at this point in the history

  3. Change formatting

    @skambha
    skambha committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    1adc512 View commit details
    Browse the repository at this point in the history

  4. hasoverflow changes

    @skambha
    skambha committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    d90c790 View commit details
    Browse the repository at this point in the history

  5. Fix formatting

    @skambha
    skambha committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    136e6dc View commit details
    Browse the repository at this point in the history

  6. Add a new implementation for sum for dec type to handle overflow, Add… 

    … a new DecimalSum and add a analyzer rule and changes to optimizer rule. Add tests

For now the DecimalSum has the code to handle other types, incase we want to put this logic back into Sum after discussion
    @skambha
    skambha committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    6979e8d View commit details
    Browse the repository at this point in the history

  7. Put back the decimal handling changes into Sum like how we started with

    @skambha
    skambha committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    4119e02 View commit details
    Browse the repository at this point in the history

  8. Use a new flag isEmptyOrNulls to handle the scenario with all nulls a… 

    …s well and also to use it for identifying overflow scenarios
    @skambha
    skambha committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    cc2fec0 View commit details
    Browse the repository at this point in the history

  9. Rebase

    @skambha
    skambha committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    23739c9 View commit details
    Browse the repository at this point in the history

Commits on Apr 28, 2020

  1. rebase

    @skambha
    skambha committed Apr 28, 2020
    Configuration menu
    Copy the full SHA
    fa45378 View commit details
    Browse the repository at this point in the history

Commits on May 15, 2020

  1. Add new test and also make the agg buffer changes only for decimal type

    @skambha
    skambha committed May 15, 2020
    Configuration menu
    Copy the full SHA
    fbd80a6 View commit details
    Browse the repository at this point in the history

Commits on May 21, 2020

  1. simplify

    @cloud-fan
    cloud-fan committed May 21, 2020
    Configuration menu
    Copy the full SHA
    de2d68f View commit details
    Browse the repository at this point in the history

Commits on May 29, 2020

  1. Fix the test failure

    @skambha
    skambha committed May 29, 2020
    Configuration menu
    Copy the full SHA
    8339e28 View commit details
    Browse the repository at this point in the history

  2. Cleanup comments

    @skambha
    skambha committed May 29, 2020
    Configuration menu
    Copy the full SHA
    59a00c4 View commit details
    Browse the repository at this point in the history

Commits on Jun 1, 2020

  1. cleanup code comments

    @skambha
    skambha committed Jun 1, 2020
    Configuration menu
    Copy the full SHA
    7795888 View commit details
    Browse the repository at this point in the history