Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEA] Add DELTA_BINARY_PACKED decoding support to Parquet reader #13637

Merged
merged 85 commits into from
Aug 23, 2023
Merged

[FEA] Add DELTA_BINARY_PACKED decoding support to Parquet reader #13637

merged 85 commits into from
Aug 23, 2023

Commits on Jun 26, 2023

  1. add DELTA_BINARY_PACKED decoder

    @etseidl
    etseidl committed Jun 26, 2023
    Configuration menu
    Copy the full SHA
    5e9cf26 View commit details
    Browse the repository at this point in the history

Commits on Jun 27, 2023

  1. start merging in changes from rapidsai#13622

    @etseidl
    etseidl committed Jun 27, 2023
    Configuration menu
    Copy the full SHA
    9326321 View commit details
    Browse the repository at this point in the history

  2. get reduce working on device

    @etseidl
    etseidl committed Jun 27, 2023
    Configuration menu
    Copy the full SHA
    ee7511d View commit details
    Browse the repository at this point in the history

  3. use functor for transform iterator

    @etseidl
    etseidl committed Jun 27, 2023
    Configuration menu
    Copy the full SHA
    2cafe62 View commit details
    Browse the repository at this point in the history

  4. change filter functors to use kernel_mask

    @etseidl
    etseidl committed Jun 27, 2023
    Configuration menu
    Copy the full SHA
    219ff0b View commit details
    Browse the repository at this point in the history

Commits on Jun 28, 2023

  1. pull in changes from rapidsai#13622

    @etseidl
    etseidl committed Jun 28, 2023
    Configuration menu
    Copy the full SHA
    0e181a8 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'rapidsai:branch-23.08' into feature/delta_binary

    @etseidl
    etseidl authored Jun 28, 2023
    Configuration menu
    Copy the full SHA
    129d9ab View commit details
    Browse the repository at this point in the history

  3. Merge branch 'rapidsai:branch-23.08' into feature/delta_binary

    @etseidl
    etseidl authored Jun 28, 2023
    Configuration menu
    Copy the full SHA
    2b5a25d View commit details
    Browse the repository at this point in the history

  4. use less shared memory for delta binary decoder

    @etseidl
    etseidl committed Jun 28, 2023
    Configuration menu
    Copy the full SHA
    b58e55c View commit details
    Browse the repository at this point in the history

  5. Merge branch 'feature/delta_binary' of github.com:etseidl/cudf into f… 

    …eature/delta_binary
    @etseidl
    etseidl committed Jun 28, 2023
    Configuration menu
    Copy the full SHA
    c63e503 View commit details
    Browse the repository at this point in the history

  6. spelling

    @etseidl
    etseidl committed Jun 28, 2023
    Configuration menu
    Copy the full SHA
    2c5e087 View commit details
    Browse the repository at this point in the history

Commits on Jun 29, 2023

  1. change encoding to unsupported type

    @etseidl
    etseidl committed Jun 29, 2023
    Configuration menu
    Copy the full SHA
    996893e View commit details
    Browse the repository at this point in the history

  2. add python test of delta parser

    @etseidl
    etseidl committed Jun 29, 2023
    Configuration menu
    Copy the full SHA
    f1f74dc View commit details
    Browse the repository at this point in the history

  3. test delta with nulls

    @etseidl
    etseidl committed Jun 29, 2023
    Configuration menu
    Copy the full SHA
    639b8ab View commit details
    Browse the repository at this point in the history

  4. Merge branch 'rapidsai:branch-23.08' into feature/delta_binary

    @etseidl
    etseidl authored Jun 29, 2023
    Configuration menu
    Copy the full SHA
    02cd2be View commit details
    Browse the repository at this point in the history

  5. Merge branch 'rapidsai:branch-23.08' into feature/delta_binary

    @etseidl
    etseidl authored Jun 29, 2023
    Configuration menu
    Copy the full SHA
    9871d66 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'rapidsai:branch-23.08' into feature/delta_binary

    @etseidl
    etseidl authored Jun 29, 2023
    Configuration menu
    Copy the full SHA
    7debd29 View commit details
    Browse the repository at this point in the history

Commits on Jun 30, 2023

  1. add comments to skip_values and decode_batch

    @etseidl
    etseidl committed Jun 30, 2023
    Configuration menu
    Copy the full SHA
    c1bbb84 View commit details
    Browse the repository at this point in the history

  2. revert east volatile changes

    @etseidl
    etseidl committed Jun 30, 2023
    Configuration menu
    Copy the full SHA
    8e66a08 View commit details
    Browse the repository at this point in the history

Commits on Jul 7, 2023

  1. update doc string

    @etseidl
    etseidl committed Jul 7, 2023
    Configuration menu
    Copy the full SHA
    7b09c4f View commit details
    Browse the repository at this point in the history

  2. Merge branch 'feature/delta_binary' of github.com:etseidl/cudf into f… 

    …eature/delta_binary
    @etseidl
    etseidl committed Jul 7, 2023
    Configuration menu
    Copy the full SHA
    5e05872 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'rapidsai:branch-23.08' into feature/delta_binary

    @etseidl
    etseidl authored Jul 7, 2023
    Configuration menu
    Copy the full SHA
    6576aa3 View commit details
    Browse the repository at this point in the history

Commits on Jul 12, 2023

  1. Merge branch 'branch-23.08' into feature/delta_binary

    @etseidl
    etseidl authored Jul 12, 2023
    Configuration menu
    Copy the full SHA
    1d6efbc View commit details
    Browse the repository at this point in the history

Commits on Jul 13, 2023

  1. Merge branch 'branch-23.08' into feature/delta_binary

    @vuule
    vuule authored Jul 13, 2023
    Configuration menu
    Copy the full SHA
    334fb53 View commit details
    Browse the repository at this point in the history

Commits on Jul 17, 2023

  1. Merge branch 'rapidsai:branch-23.08' into feature/delta_binary

    @etseidl
    etseidl authored Jul 17, 2023
    Configuration menu
    Copy the full SHA
    d18dabf View commit details
    Browse the repository at this point in the history

Commits on Jul 18, 2023

  1. Merge branch 'rapidsai:branch-23.08' into feature/delta_binary

    @etseidl
    etseidl authored Jul 18, 2023
    Configuration menu
    Copy the full SHA
    05eb40f View commit details
    Browse the repository at this point in the history

Commits on Jul 19, 2023

  1. Merge branch 'branch-23.08' into feature/delta_binary

    @etseidl
    etseidl authored Jul 19, 2023
    Configuration menu
    Copy the full SHA
    a957ecc View commit details
    Browse the repository at this point in the history

  2. fix for header location

    @etseidl
    etseidl committed Jul 19, 2023
    Configuration menu
    Copy the full SHA
    9b636c7 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'rapidsai:branch-23.08' into feature/delta_binary

    @etseidl
    etseidl authored Jul 19, 2023
    Configuration menu
    Copy the full SHA
    6a53d43 View commit details
    Browse the repository at this point in the history

  4. fix some short-circuit logic

    @etseidl
    etseidl committed Jul 19, 2023
    Configuration menu
    Copy the full SHA
    ceb22ab View commit details
    Browse the repository at this point in the history

Commits on Jul 25, 2023

  1. Merge branch 'branch-23.10' into feature/delta_binary

    @etseidl
    etseidl authored Jul 25, 2023
    Configuration menu
    Copy the full SHA
    c00be0b View commit details
    Browse the repository at this point in the history

Commits on Jul 26, 2023

  1. Merge branch 'branch-23.10' into feature/delta_binary

    @etseidl
    etseidl authored Jul 26, 2023
    Configuration menu
    Copy the full SHA
    e30aa11 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'branch-23.10' into feature/delta_binary

    @etseidl
    etseidl authored Jul 26, 2023
    Configuration menu
    Copy the full SHA
    7ef4be2 View commit details
    Browse the repository at this point in the history

  3. Merge remote-tracking branch 'origin/branch-23.10' into feature/delta… 

    …_binary
    @etseidl
    etseidl committed Jul 26, 2023
    Configuration menu
    Copy the full SHA
    2410e47 View commit details
    Browse the repository at this point in the history

Commits on Jul 28, 2023

  1. Merge branch 'branch-23.10' into feature/delta_binary

    @etseidl
    etseidl authored Jul 28, 2023
    Configuration menu
    Copy the full SHA
    4aa783d View commit details
    Browse the repository at this point in the history

  2. Merge branch 'branch-23.10' into feature/delta_binary

    @etseidl
    etseidl authored Jul 28, 2023
    Configuration menu
    Copy the full SHA
    94afb8d View commit details
    Browse the repository at this point in the history

Commits on Jul 31, 2023

  1. Merge branch 'rapidsai:branch-23.10' into feature/delta_binary

    @etseidl
    etseidl authored Jul 31, 2023
    Configuration menu
    Copy the full SHA
    12b9bab View commit details
    Browse the repository at this point in the history

Commits on Aug 1, 2023

  1. Merge branch 'rapidsai:branch-23.10' into feature/delta_binary

    @etseidl
    etseidl authored Aug 1, 2023
    Configuration menu
    Copy the full SHA
    a6f7957 View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2023

  1. Merge branch 'branch-23.10' into feature/delta_binary

    @etseidl
    etseidl authored Aug 2, 2023
    Configuration menu
    Copy the full SHA
    b7dbf47 View commit details
    Browse the repository at this point in the history

  2. rename function

    @etseidl
    etseidl committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    ea49a23 View commit details
    Browse the repository at this point in the history

  3. clean up kernel_mask_for_page()

    @etseidl
    etseidl committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    3a9f186 View commit details
    Browse the repository at this point in the history

  4. remove TODO

    @etseidl
    etseidl committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    d7671d7 View commit details
    Browse the repository at this point in the history

  5. add some documentation to kernel_mask_bits

    @etseidl
    etseidl committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    7084d89 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'branch-23.10' into feature/delta_binary

    @etseidl
    etseidl authored Aug 2, 2023
    Configuration menu
    Copy the full SHA
    f60bc1c View commit details
    Browse the repository at this point in the history

Commits on Aug 3, 2023

  1. use rand_dataframe() to produce test data

    @etseidl
    etseidl committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    9ebeca9 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'rapidsai:branch-23.10' into feature/delta_binary

    @etseidl
    etseidl authored Aug 3, 2023
    Configuration menu
    Copy the full SHA
    07e73ac View commit details
    Browse the repository at this point in the history

  3. Merge remote-tracking branch 'origin/branch-23.10' into feature/delta… 

    …_binary
    @etseidl
    etseidl committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    410ab67 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'branch-23.10' into feature/delta_binary

    @vuule
    vuule authored Aug 3, 2023
    Configuration menu
    Copy the full SHA
    f779455 View commit details
    Browse the repository at this point in the history

Commits on Aug 4, 2023

  1. formatting

    @etseidl
    etseidl committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    761393f View commit details
    Browse the repository at this point in the history

  2. Merge branch 'rapidsai:branch-23.10' into feature/delta_binary

    @etseidl
    etseidl authored Aug 4, 2023
    Configuration menu
    Copy the full SHA
    ced688d View commit details
    Browse the repository at this point in the history

  3. implement suggestion from review 

    Co-authored-by: Vukasin Milovanovic <[email protected]>
    @etseidl @vuule
    etseidl and vuule authored Aug 4, 2023
    Configuration menu
    Copy the full SHA
    4846032 View commit details
    Browse the repository at this point in the history

  4. more suggestions from review

    @etseidl
    etseidl committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    d615625 View commit details
    Browse the repository at this point in the history

  5. Merge remote-tracking branch 'origin/branch-23.10' into feature/delta… 

    …_binary
    @etseidl
    etseidl committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    c1ce34c View commit details
    Browse the repository at this point in the history

Commits on Aug 5, 2023

  1. restore old unrolled loop for testing

    @etseidl
    etseidl committed Aug 5, 2023
    Configuration menu
    Copy the full SHA
    835e866 View commit details
    Browse the repository at this point in the history

Commits on Aug 7, 2023

  1. add note to revisit bit unpacker with delta_byte_array

    @etseidl
    etseidl committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    6353a4a View commit details
    Browse the repository at this point in the history

  2. Merge branch 'rapidsai:branch-23.10' into feature/delta_binary

    @etseidl
    etseidl authored Aug 7, 2023
    Configuration menu
    Copy the full SHA
    9ffea01 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'branch-23.10' into feature/delta_binary

    @etseidl
    etseidl authored Aug 7, 2023
    Configuration menu
    Copy the full SHA
    0904418 View commit details
    Browse the repository at this point in the history

Commits on Aug 8, 2023

  1. Merge branch 'branch-23.10' into feature/delta_binary

    @etseidl
    etseidl authored Aug 8, 2023
    Configuration menu
    Copy the full SHA
    616d3fc View commit details
    Browse the repository at this point in the history

  2. fix and test int8 and int16 handling

    @etseidl
    etseidl committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    e924a97 View commit details
    Browse the repository at this point in the history

  3. fix for single row files

    @etseidl
    etseidl committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    d0bf0cd View commit details
    Browse the repository at this point in the history

Commits on Aug 9, 2023

  1. Merge branch 'rapidsai:branch-23.10' into feature/delta_binary

    @etseidl
    etseidl authored Aug 9, 2023
    Configuration menu
    Copy the full SHA
    d3b0c09 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'branch-23.10' into feature/delta_binary

    @vuule
    vuule authored Aug 9, 2023
    Configuration menu
    Copy the full SHA
    3c83b89 View commit details
    Browse the repository at this point in the history

  3. clean up some docstrings

    @etseidl
    etseidl committed Aug 9, 2023
    Configuration menu
    Copy the full SHA
    5ac20a0 View commit details
    Browse the repository at this point in the history

Commits on Aug 14, 2023

  1. Merge remote-tracking branch 'origin/branch-23.10' into feature/delta… 

    …_binary
    @etseidl
    etseidl committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    9914e1f View commit details
    Browse the repository at this point in the history

  2. Merge branch 'rapidsai:branch-23.10' into feature/delta_binary

    @etseidl
    etseidl authored Aug 14, 2023
    Configuration menu
    Copy the full SHA
    7d077f7 View commit details
    Browse the repository at this point in the history

Commits on Aug 15, 2023

  1. Merge branch 'rapidsai:branch-23.10' into feature/delta_binary

    @etseidl
    etseidl authored Aug 15, 2023
    Configuration menu
    Copy the full SHA
    62e0493 View commit details
    Browse the repository at this point in the history

Commits on Aug 16, 2023

  1. Merge branch 'branch-23.10' into feature/delta_binary

    @etseidl
    etseidl authored Aug 16, 2023
    Configuration menu
    Copy the full SHA
    5725c61 View commit details
    Browse the repository at this point in the history

Commits on Aug 17, 2023

  1. Merge branch 'branch-23.10' into feature/delta_binary

    @etseidl
    etseidl authored Aug 17, 2023
    Configuration menu
    Copy the full SHA
    0ab8c15 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'rapidsai:branch-23.10' into feature/delta_binary

    @etseidl
    etseidl authored Aug 17, 2023
    Configuration menu
    Copy the full SHA
    b404de7 View commit details
    Browse the repository at this point in the history

  3. Apply suggestions from code review 

    Co-authored-by: nvdbaranec <[email protected]>
    @etseidl @nvdbaranec
    etseidl and nvdbaranec authored Aug 17, 2023
    Configuration menu
    Copy the full SHA
    1d137ec View commit details
    Browse the repository at this point in the history

  4. fix docstring

    @etseidl
    etseidl committed Aug 17, 2023
    Configuration menu
    Copy the full SHA
    6cd3e00 View commit details
    Browse the repository at this point in the history

  5. need to pass num_threads as template param to make constexpr

    @etseidl
    etseidl committed Aug 17, 2023
    Configuration menu
    Copy the full SHA
    a774ac1 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'rapidsai:branch-23.10' into feature/delta_binary

    @etseidl
    etseidl authored Aug 17, 2023
    Configuration menu
    Copy the full SHA
    60b45b3 View commit details
    Browse the repository at this point in the history

Commits on Aug 18, 2023

  1. Merge branch 'rapidsai:branch-23.10' into feature/delta_binary

    @etseidl
    etseidl authored Aug 18, 2023
    Configuration menu
    Copy the full SHA
    fb44e80 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'branch-23.10' into feature/delta_binary

    @etseidl
    etseidl authored Aug 18, 2023
    Configuration menu
    Copy the full SHA
    1d9cc7d View commit details
    Browse the repository at this point in the history

Commits on Aug 21, 2023

  1. more descriptive name and docstring

    @etseidl
    etseidl committed Aug 21, 2023
    Configuration menu
    Copy the full SHA
    6655e86 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/branch-23.10' into feature/delta… 

    …_binary
    @etseidl
    etseidl committed Aug 21, 2023
    Configuration menu
    Copy the full SHA
    9dbc3ff View commit details
    Browse the repository at this point in the history

Commits on Aug 22, 2023

  1. Merge branch 'rapidsai:branch-23.10' into feature/delta_binary

    @etseidl
    etseidl authored Aug 22, 2023
    Configuration menu
    Copy the full SHA
    ee4385c View commit details
    Browse the repository at this point in the history

  2. Apply suggestions from code review 

    Co-authored-by: nvdbaranec <[email protected]>
    @etseidl @nvdbaranec
    etseidl and nvdbaranec authored Aug 22, 2023
    Configuration menu
    Copy the full SHA
    373bdd9 View commit details
    Browse the repository at this point in the history

  3. formatting

    @etseidl
    etseidl committed Aug 22, 2023
    Configuration menu
    Copy the full SHA
    67be449 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'rapidsai:branch-23.10' into feature/delta_binary

    @etseidl
    etseidl authored Aug 22, 2023
    Configuration menu
    Copy the full SHA
    e6068d7 View commit details
    Browse the repository at this point in the history

Commits on Aug 23, 2023

  1. Merge branch 'branch-23.10' into feature/delta_binary

    @etseidl
    etseidl authored Aug 23, 2023
    Configuration menu
    Copy the full SHA
    bbf957e View commit details
    Browse the repository at this point in the history

  2. Merge branch 'rapidsai:branch-23.10' into feature/delta_binary

    @etseidl
    etseidl authored Aug 23, 2023
    Configuration menu
    Copy the full SHA
    ed5dcdb View commit details
    Browse the repository at this point in the history

  3. implement suggestion from review 

    Co-authored-by: GALI PREM SAGAR <[email protected]>
    @etseidl @galipremsagar
    etseidl and galipremsagar authored Aug 23, 2023
    Configuration menu
    Copy the full SHA
    eb4ff08 View commit details
    Browse the repository at this point in the history

  4. Merge remote-tracking branch 'origin/branch-23.10' into feature/delta… 

    …_binary
    @etseidl
    etseidl committed Aug 23, 2023
    Configuration menu
    Copy the full SHA
    9afd0fb View commit details
    Browse the repository at this point in the history