Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[WIP] Add support for DELTA_BINARY_PACKED and DELTA_BYTE_ARRAY encodings to Parquet reader #12948

Closed
wants to merge 164 commits into from
Closed

[WIP] Add support for DELTA_BINARY_PACKED and DELTA_BYTE_ARRAY encodings to Parquet reader #12948

wants to merge 164 commits into from

Commits on Feb 9, 2023

  1. test for unsupported page encodings and throw an error if one is dete… 

    …cted
    @etseidl
    etseidl committed Feb 9, 2023
    Configuration menu
    Copy the full SHA
    ae74d50 View commit details
    Browse the repository at this point in the history

Commits on Feb 10, 2023

  1. add test

    @etseidl
    etseidl committed Feb 10, 2023
    Configuration menu
    Copy the full SHA
    04fcd01 View commit details
    Browse the repository at this point in the history

  2. use sizeof

    @etseidl
    etseidl committed Feb 10, 2023
    Configuration menu
    Copy the full SHA
    30c0ecb View commit details
    Browse the repository at this point in the history

  3. use correct version of clang-format

    @etseidl
    etseidl committed Feb 10, 2023
    Configuration menu
    Copy the full SHA
    03a5960 View commit details
    Browse the repository at this point in the history

  4. replace thrust::any_of with thrust::count_if

    @etseidl
    etseidl committed Feb 10, 2023
    Configuration menu
    Copy the full SHA
    f231b3a View commit details
    Browse the repository at this point in the history

  5. Merge branch 'rapidsai:branch-23.04' into feature/validate_encodings

    @etseidl
    etseidl authored Feb 10, 2023
    Configuration menu
    Copy the full SHA
    5f1ef57 View commit details
    Browse the repository at this point in the history

  6. move test to python

    @etseidl
    etseidl committed Feb 10, 2023
    Configuration menu
    Copy the full SHA
    85c2bf6 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'feature/validate_encodings' of github.com:etseidl/cudf … 

    …into feature/validate_encodings
    @etseidl
    etseidl committed Feb 10, 2023
    Configuration menu
    Copy the full SHA
    21ed189 View commit details
    Browse the repository at this point in the history

  8. Merge branch 'rapidsai:branch-23.04' into feature/validate_encodings

    @etseidl
    etseidl authored Feb 10, 2023
    Configuration menu
    Copy the full SHA
    0fa7f3f View commit details
    Browse the repository at this point in the history

Commits on Feb 13, 2023

  1. update comment about deprecation of BIT_PACKED

    @etseidl
    etseidl committed Feb 13, 2023
    Configuration menu
    Copy the full SHA
    b5573eb View commit details
    Browse the repository at this point in the history

  2. Merge branch 'feature/validate_encodings' of github.com:etseidl/cudf … 

    …into feature/validate_encodings
    @etseidl
    etseidl committed Feb 13, 2023
    Configuration menu
    Copy the full SHA
    f1af811 View commit details
    Browse the repository at this point in the history

  3. add comment per review suggestion

    @etseidl
    etseidl committed Feb 13, 2023
    Configuration menu
    Copy the full SHA
    febeb02 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'rapidsai:branch-23.04' into feature/validate_encodings

    @etseidl
    etseidl authored Feb 13, 2023
    Configuration menu
    Copy the full SHA
    0015e08 View commit details
    Browse the repository at this point in the history

  5. add comment to update is_supported_encoding

    @etseidl
    etseidl committed Feb 13, 2023
    Configuration menu
    Copy the full SHA
    aec40aa View commit details
    Browse the repository at this point in the history

  6. Merge branch 'feature/validate_encodings' of github.com:etseidl/cudf … 

    …into feature/validate_encodings
    @etseidl
    etseidl committed Feb 13, 2023
    Configuration menu
    Copy the full SHA
    e6fef99 View commit details
    Browse the repository at this point in the history

  7. switch check back to host code

    @etseidl
    etseidl committed Feb 13, 2023
    Configuration menu
    Copy the full SHA
    432c9df View commit details
    Browse the repository at this point in the history

Commits on Feb 28, 2023

  1. test for unsupported page encodings and throw an error if one is dete… 

    …cted
    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    ae4f895 View commit details
    Browse the repository at this point in the history

  2. add test

    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    5494030 View commit details
    Browse the repository at this point in the history

  3. use correct version of clang-format

    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    9b3c00b View commit details
    Browse the repository at this point in the history

  4. replace thrust::any_of with thrust::count_if

    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    509c0ec View commit details
    Browse the repository at this point in the history

  5. move test to python

    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    bdc0f3e View commit details
    Browse the repository at this point in the history

  6. update comment about deprecation of BIT_PACKED

    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    268f939 View commit details
    Browse the repository at this point in the history

  7. add comment per review suggestion

    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    f05490a View commit details
    Browse the repository at this point in the history

  8. add comment to update is_supported_encoding

    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    3efbe26 View commit details
    Browse the repository at this point in the history

  9. switch check back to host code

    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    ecc1179 View commit details
    Browse the repository at this point in the history

  10. checkpoint...can decode one page serially

    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    d8f5512 View commit details
    Browse the repository at this point in the history

  11. checkpoint...works multithreaded no, no output

    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    9943162 View commit details
    Browse the repository at this point in the history

  12. checkpoint...move decode to separate function

    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    a6ab85a View commit details
    Browse the repository at this point in the history

  13. clean up some clang-format mess

    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    573fffb View commit details
    Browse the repository at this point in the history

  14. use cub inclusive scan to roll up the deltas

    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    7dafba6 View commit details
    Browse the repository at this point in the history

  15. checkpoint. works for non-nested and no nulls

    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    8a0333a View commit details
    Browse the repository at this point in the history

  16. refactor some for reuse later

    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    49ec0a4 View commit details
    Browse the repository at this point in the history

  17. seems to work for nested with nulls and skip_rows now

    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    73d7d76 View commit details
    Browse the repository at this point in the history

  18. checkpoint...delta byte array initialization works

    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    aca8753 View commit details
    Browse the repository at this point in the history

  19. checkpoint

    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    27ffd38 View commit details
    Browse the repository at this point in the history

  20. refactor DecodeDeltaBinary to use 3 warps

    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    235449c View commit details
    Browse the repository at this point in the history

  21. checkpoint...works except for properly decoding strings

    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    7e2ff53 View commit details
    Browse the repository at this point in the history

  22. fix check for end of data

    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    a0ed800 View commit details
    Browse the repository at this point in the history

  23. fix delta decoding to work on mini-blocks larger than 32. 

    some cleanup as well.
    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    a041646 View commit details
    Browse the repository at this point in the history

  24. get DeltaForThread working up to 64 bits

    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    1e35270 View commit details
    Browse the repository at this point in the history

  25. add looping version of bit unpacking

    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    27c3b03 View commit details
    Browse the repository at this point in the history

  26. fix problem with producers getting ahead of consumers

    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    b64de4f View commit details
    Browse the repository at this point in the history

  27. simplify decode logic to stop caring about first value since it's in … 

    …a rolling buffer.

allow for batch sizes > 32
    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    3033cac View commit details
    Browse the repository at this point in the history

  28. checkpoint...strings without prefixes working

    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    916fd3e View commit details
    Browse the repository at this point in the history

  29. strings work now, but no skip_rows yet

    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    00874e4 View commit details
    Browse the repository at this point in the history

  30. cleanup

    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    d7d4864 View commit details
    Browse the repository at this point in the history

  31. more cleanup

    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    c3b2f9e View commit details
    Browse the repository at this point in the history

  32. checkpoint...kinda working again except skip_rows for nested columns

    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    c656770 View commit details
    Browse the repository at this point in the history

  33. fix to not overwrite page info

    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    cd23b18 View commit details
    Browse the repository at this point in the history

  34. skip_rows working for nested columns now

    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    6dd2aaa View commit details
    Browse the repository at this point in the history

  35. Merge branch 'feature/validate_encodings' into feature/delta_binary

    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    c8a33f3 View commit details
    Browse the repository at this point in the history

  36. rework ComputePageStringSizes to not need an extra vector. instead add 

    a size field to the PageInfo struct.
also skip calling ComputePageStringSizes if none of the pages have
DELTA_BYTE_ARRAY encoding.
    @etseidl
    etseidl committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    6d01a97 View commit details
    Browse the repository at this point in the history

  37. Merge branch 'rapidsai:branch-23.04' into feature/delta_binary

    @etseidl
    etseidl authored Feb 28, 2023
    Configuration menu
    Copy the full SHA
    deba306 View commit details
    Browse the repository at this point in the history

Commits on Mar 1, 2023

  1. Merge branch 'rapidsai:branch-23.04' into feature/delta_binary

    @etseidl
    etseidl authored Mar 1, 2023
    Configuration menu
    Copy the full SHA
    97d5a8c View commit details
    Browse the repository at this point in the history

  2. Merge branch 'rapidsai:branch-23.04' into feature/delta_binary

    @etseidl
    etseidl authored Mar 1, 2023
    Configuration menu
    Copy the full SHA
    b8c72e0 View commit details
    Browse the repository at this point in the history

Commits on Mar 2, 2023

  1. replace test file for unsupported encoding check

    @etseidl
    etseidl committed Mar 2, 2023
    Configuration menu
    Copy the full SHA
    d46ab9c View commit details
    Browse the repository at this point in the history

Commits on Mar 4, 2023

  1. try a more efficient string copy for delta byte array

    @etseidl
    etseidl committed Mar 4, 2023
    Configuration menu
    Copy the full SHA
    1ebb3f9 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/branch-23.04' into feature/delta… 

    …_binary
    @etseidl
    etseidl committed Mar 4, 2023
    Configuration menu
    Copy the full SHA
    f16e077 View commit details
    Browse the repository at this point in the history

Commits on Mar 6, 2023

  1. fix for block ending on last mini-block

    @etseidl
    etseidl committed Mar 6, 2023
    Configuration menu
    Copy the full SHA
    69f825f View commit details
    Browse the repository at this point in the history

  2. skip string computation when not needed because of skip_rows

    @etseidl
    etseidl committed Mar 6, 2023
    Configuration menu
    Copy the full SHA
    208d588 View commit details
    Browse the repository at this point in the history

  3. add struct for DELTA_BYTE_ARRAY decoding, add consts

    @etseidl
    etseidl committed Mar 6, 2023
    Configuration menu
    Copy the full SHA
    7e12df0 View commit details
    Browse the repository at this point in the history

  4. fix for nested data with nulls and skipping

    @etseidl
    etseidl committed Mar 6, 2023
    Configuration menu
    Copy the full SHA
    63fcf72 View commit details
    Browse the repository at this point in the history

  5. don't pass target_pos to string calc functions

    @etseidl
    etseidl committed Mar 6, 2023
    Configuration menu
    Copy the full SHA
    98df483 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'rapidsai:branch-23.04' into feature/delta_binary

    @etseidl
    etseidl authored Mar 6, 2023
    Configuration menu
    Copy the full SHA
    66f8f4e View commit details
    Browse the repository at this point in the history

  7. clean up some ints

    @etseidl
    etseidl committed Mar 6, 2023
    Configuration menu
    Copy the full SHA
    9c7a9de View commit details
    Browse the repository at this point in the history

  8. Merge branch 'rapidsai:branch-23.04' into feature/delta_binary

    @etseidl
    etseidl authored Mar 6, 2023
    Configuration menu
    Copy the full SHA
    b5bf8ea View commit details
    Browse the repository at this point in the history

  9. Merge branch 'rapidsai:branch-23.04' into feature/delta_binary

    @etseidl
    etseidl authored Mar 6, 2023
    Configuration menu
    Copy the full SHA
    5ca9112 View commit details
    Browse the repository at this point in the history

Commits on Mar 7, 2023

  1. remove todo

    @etseidl
    etseidl committed Mar 7, 2023
    Configuration menu
    Copy the full SHA
    132ab25 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'rapidsai:branch-23.04' into feature/delta_binary

    @etseidl
    etseidl authored Mar 7, 2023
    Configuration menu
    Copy the full SHA
    e76ad56 View commit details
    Browse the repository at this point in the history

  3. move find_end lambda to function FindEndOfBlock

    @etseidl
    etseidl committed Mar 7, 2023
    Configuration menu
    Copy the full SHA
    852eab2 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'feature/delta_binary' of github.com:etseidl/cudf into f… 

    …eature/delta_binary
    @etseidl
    etseidl committed Mar 7, 2023
    Configuration menu
    Copy the full SHA
    05a36e9 View commit details
    Browse the repository at this point in the history

  5. add some more comments, get rid of magic 32's

    @etseidl
    etseidl committed Mar 7, 2023
    Configuration menu
    Copy the full SHA
    e5d393a View commit details
    Browse the repository at this point in the history

  6. get rid of old-school cast

    @etseidl
    etseidl committed Mar 7, 2023
    Configuration menu
    Copy the full SHA
    6e6b997 View commit details
    Browse the repository at this point in the history

  7. remove FIXME

    @etseidl
    etseidl committed Mar 7, 2023
    Configuration menu
    Copy the full SHA
    49d8df5 View commit details
    Browse the repository at this point in the history

Commits on Mar 8, 2023

  1. add v2 indicator to page flags

    @etseidl
    etseidl committed Mar 8, 2023
    Configuration menu
    Copy the full SHA
    492fbab View commit details
    Browse the repository at this point in the history

  2. more consts

    @etseidl
    etseidl committed Mar 8, 2023
    Configuration menu
    Copy the full SHA
    6028074 View commit details
    Browse the repository at this point in the history

  3. refactor and rename get_vlq64, add documentation

    @etseidl
    etseidl committed Mar 8, 2023
    Configuration menu
    Copy the full SHA
    83533cd View commit details
    Browse the repository at this point in the history

  4. use loop for bit unpacking

    @etseidl
    etseidl committed Mar 8, 2023
    Configuration menu
    Copy the full SHA
    55959da View commit details
    Browse the repository at this point in the history

Commits on Mar 9, 2023

  1. when dealing with mini-blocks, then need to decrement value_count 

    because first value is in the header. but when using value_count
for extracting values, don't decrement.
    @etseidl
    etseidl committed Mar 9, 2023
    Configuration menu
    Copy the full SHA
    55fd2d4 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'rapidsai:branch-23.04' into feature/delta_binary

    @etseidl
    etseidl authored Mar 9, 2023
    Configuration menu
    Copy the full SHA
    b80fe6e View commit details
    Browse the repository at this point in the history

  3. use num_encoded_values to indicate intent when comparing to value_count

    @etseidl
    etseidl committed Mar 9, 2023
    Configuration menu
    Copy the full SHA
    9c47002 View commit details
    Browse the repository at this point in the history

  4. do not compute string sizes when delta_byte_array encoding is used

    @etseidl
    etseidl committed Mar 9, 2023
    Configuration menu
    Copy the full SHA
    a3bb211 View commit details
    Browse the repository at this point in the history

  5. fix string offset calc to use long ints 

    remove check on data size since it's for all columns. would need a
  separate check for each column.
    @etseidl
    etseidl committed Mar 9, 2023
    Configuration menu
    Copy the full SHA
    fcedc62 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'rapidsai:branch-23.04' into feature/delta_binary

    @etseidl
    etseidl authored Mar 9, 2023
    Configuration menu
    Copy the full SHA
    967f7e2 View commit details
    Browse the repository at this point in the history

Commits on Mar 10, 2023

  1. Merge branch 'rapidsai:branch-23.04' into feature/delta_binary

    @etseidl
    etseidl authored Mar 10, 2023
    Configuration menu
    Copy the full SHA
    3b0c0fd View commit details
    Browse the repository at this point in the history

Commits on Mar 12, 2023

  1. Merge branch 'rapidsai:branch-23.04' into feature/delta_binary

    @etseidl
    etseidl authored Mar 12, 2023
    Configuration menu
    Copy the full SHA
    8daaee3 View commit details
    Browse the repository at this point in the history

Commits on Mar 14, 2023

  1. Merge branch 'rapidsai:branch-23.04' into feature/delta_binary

    @etseidl
    etseidl authored Mar 14, 2023
    Configuration menu
    Copy the full SHA
    67b3b56 View commit details
    Browse the repository at this point in the history

Commits on Mar 15, 2023

  1. Merge branch 'rapidsai:branch-23.04' into feature/delta_binary

    @etseidl
    etseidl authored Mar 15, 2023
    Configuration menu
    Copy the full SHA
    ef9e2eb View commit details
    Browse the repository at this point in the history

  2. use shift instead of masking to unset a bit

    @etseidl
    etseidl committed Mar 15, 2023
    Configuration menu
    Copy the full SHA
    08749b7 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'feature/delta_binary' of github.com:etseidl/cudf into f… 

    …eature/delta_binary
    @etseidl
    etseidl committed Mar 15, 2023
    Configuration menu
    Copy the full SHA
    0a3ac27 View commit details
    Browse the repository at this point in the history

  4. more fix ups for value_count

    @etseidl
    etseidl committed Mar 15, 2023
    Configuration menu
    Copy the full SHA
    ff28cb9 View commit details
    Browse the repository at this point in the history

Commits on Mar 17, 2023

  1. Merge branch 'rapidsai:branch-23.04' into feature/delta_binary

    @etseidl
    etseidl authored Mar 17, 2023
    Configuration menu
    Copy the full SHA
    378743f View commit details
    Browse the repository at this point in the history

Commits on Mar 20, 2023

  1. Merge branch 'rapidsai:branch-23.04' into feature/delta_binary

    @etseidl
    etseidl authored Mar 20, 2023
    Configuration menu
    Copy the full SHA
    1a7f09d View commit details
    Browse the repository at this point in the history

Commits on Mar 21, 2023

  1. clean up some comments

    @etseidl
    etseidl committed Mar 21, 2023
    Configuration menu
    Copy the full SHA
    e1d53a2 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'rapidsai:branch-23.04' into feature/delta_binary

    @etseidl
    etseidl authored Mar 21, 2023
    Configuration menu
    Copy the full SHA
    f8f65d5 View commit details
    Browse the repository at this point in the history

Commits on Mar 22, 2023

  1. implement suggestion from reviewa. only need to perform check of neig… 

    …hbors

once, not every iteration
    @etseidl
    etseidl committed Mar 22, 2023
    Configuration menu
    Copy the full SHA
    c4e31cb View commit details
    Browse the repository at this point in the history

  2. missed a brace

    @etseidl
    etseidl committed Mar 22, 2023
    Configuration menu
    Copy the full SHA
    916afd8 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'rapidsai:branch-23.04' into feature/delta_binary

    @etseidl
    etseidl authored Mar 22, 2023
    Configuration menu
    Copy the full SHA
    32a55be View commit details
    Browse the repository at this point in the history

  4. remove check for prefix_lens[i] == 0 in loop

    @etseidl
    etseidl committed Mar 22, 2023
    Configuration menu
    Copy the full SHA
    338e0ce View commit details
    Browse the repository at this point in the history

  5. simplify StringScan further

    @etseidl
    etseidl committed Mar 22, 2023
    Configuration menu
    Copy the full SHA
    2b41eca View commit details
    Browse the repository at this point in the history

Commits on Mar 23, 2023

  1. Merge branch 'rapidsai:branch-23.04' into feature/delta_binary

    @etseidl
    etseidl authored Mar 23, 2023
    Configuration menu
    Copy the full SHA
    d57b4b6 View commit details
    Browse the repository at this point in the history

  2. split delta decoders into separate kernels to save on shared mem

    @etseidl
    etseidl committed Mar 23, 2023
    Configuration menu
    Copy the full SHA
    23341df View commit details
    Browse the repository at this point in the history

Commits on Mar 24, 2023

  1. Merge branch 'rapidsai:branch-23.04' into feature/delta_binary

    @etseidl
    etseidl authored Mar 24, 2023
    Configuration menu
    Copy the full SHA
    de6a0a3 View commit details
    Browse the repository at this point in the history

Commits on Mar 29, 2023

  1. rename ComputePageStringSizes

    @etseidl
    etseidl committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    04f906a View commit details
    Browse the repository at this point in the history

  2. add some description of the encodings

    @etseidl
    etseidl committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    eaab701 View commit details
    Browse the repository at this point in the history

  3. add note about the bit packing

    @etseidl
    etseidl committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    35a46b6 View commit details
    Browse the repository at this point in the history

  4. rework CalcMiniBlockValues to reduce syncwarp calls

    @etseidl
    etseidl committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    a6cca65 View commit details
    Browse the repository at this point in the history

  5. more changes from review

    @etseidl
    etseidl committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    e42ecd0 View commit details
    Browse the repository at this point in the history

  6. fix WarpReduce bug caught in review

    @etseidl
    etseidl committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    01354ca View commit details
    Browse the repository at this point in the history

  7. Merge branch 'feature/delta_binary' of github.com:etseidl/cudf into f… 

    …eature/delta_binary
    @etseidl
    etseidl committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    f938b61 View commit details
    Browse the repository at this point in the history

  8. more comments

    @etseidl
    etseidl committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    4fe791d View commit details
    Browse the repository at this point in the history

  9. Merge branch 'branch-23.04' into feature/delta_binary

    @etseidl
    etseidl authored Mar 29, 2023
    Configuration menu
    Copy the full SHA
    1ed4f1c View commit details
    Browse the repository at this point in the history

  10. missed a name change

    @etseidl
    etseidl committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    a564f7c View commit details
    Browse the repository at this point in the history

  11. rework string size calc to use single_lane_block_sum_reduce

    @etseidl
    etseidl committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    713fbf0 View commit details
    Browse the repository at this point in the history

  12. move some work done on the host in decode_page_data to device in Comp… 

    …uteDeltaPageStringSizes
    @etseidl
    etseidl committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    fdfb3a4 View commit details
    Browse the repository at this point in the history

  13. add comment

    @etseidl
    etseidl committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    adfb4a5 View commit details
    Browse the repository at this point in the history

  14. fix bug in page string size calc

    @etseidl
    etseidl committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    ab33c0d View commit details
    Browse the repository at this point in the history

  15. fix bug in last transform in ComputeDeltaPageStringSizes

    @etseidl
    etseidl committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    0842190 View commit details
    Browse the repository at this point in the history

  16. fix stupid bug

    @etseidl
    etseidl committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    bdebca0 View commit details
    Browse the repository at this point in the history

Commits on Mar 31, 2023

  1. Merge branch 'rapidsai:branch-23.06' into feature/delta_binary

    @etseidl
    etseidl authored Mar 31, 2023
    Configuration menu
    Copy the full SHA
    5ba6b4d View commit details
    Browse the repository at this point in the history

Commits on Apr 3, 2023

  1. silence (bogus) IDE warnings

    @etseidl
    etseidl committed Apr 3, 2023
    Configuration menu
    Copy the full SHA
    55eb17f View commit details
    Browse the repository at this point in the history

  2. formatting

    @etseidl
    etseidl committed Apr 3, 2023
    Configuration menu
    Copy the full SHA
    44c87c0 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'rapidsai:branch-23.06' into feature/delta_binary

    @etseidl
    etseidl authored Apr 3, 2023
    Configuration menu
    Copy the full SHA
    6f5fb7b View commit details
    Browse the repository at this point in the history

Commits on Apr 4, 2023

  1. add character parallel version of string calc. helps for long strings.

    @etseidl
    etseidl committed Apr 4, 2023
    Configuration menu
    Copy the full SHA
    43526f9 View commit details
    Browse the repository at this point in the history

Commits on Apr 5, 2023

  1. Merge branch 'rapidsai:branch-23.06' into feature/delta_binary

    @etseidl
    etseidl authored Apr 5, 2023
    Configuration menu
    Copy the full SHA
    2f51172 View commit details
    Browse the repository at this point in the history

Commits on Apr 8, 2023

  1. Merge remote-tracking branch 'origin/branch-23.06' into feature/delta… 

    …_binary
    @etseidl
    etseidl committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    d3686f4 View commit details
    Browse the repository at this point in the history

  2. post merge cleanup

    @etseidl
    etseidl committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    606ead3 View commit details
    Browse the repository at this point in the history

Commits on Apr 11, 2023

  1. restore stuff removed in merge

    @etseidl
    etseidl committed Apr 11, 2023
    Configuration menu
    Copy the full SHA
    5a8b268 View commit details
    Browse the repository at this point in the history

Commits on Apr 13, 2023

  1. Merge branch 'rapidsai:branch-23.06' into feature/delta_binary

    @etseidl
    etseidl authored Apr 13, 2023
    Configuration menu
    Copy the full SHA
    a734a14 View commit details
    Browse the repository at this point in the history

Commits on Apr 14, 2023

  1. Merge remote-tracking branch 'origin/branch-23.06' into feature/delta… 

    …_binary
    @etseidl
    etseidl committed Apr 14, 2023
    Configuration menu
    Copy the full SHA
    b1a7f27 View commit details
    Browse the repository at this point in the history

Commits on Apr 24, 2023

  1. Merge branch 'rapidsai:branch-23.06' into feature/delta_binary

    @etseidl
    etseidl authored Apr 24, 2023
    Configuration menu
    Copy the full SHA
    b5c9bd4 View commit details
    Browse the repository at this point in the history

Commits on Apr 26, 2023

  1. Merge branch 'rapidsai:branch-23.06' into feature/delta_binary

    @etseidl
    etseidl authored Apr 26, 2023
    Configuration menu
    Copy the full SHA
    3993d69 View commit details
    Browse the repository at this point in the history

Commits on May 3, 2023

  1. Merge remote-tracking branch 'origin/branch-23.06' into feature/delta… 

    …_binary
    @etseidl
    etseidl committed May 3, 2023
    Configuration menu
    Copy the full SHA
    d893b1d View commit details
    Browse the repository at this point in the history

  2. Merge branch 'feature/delta_binary' of github.com:etseidl/cudf into f… 

    …eature/delta_binary
    @etseidl
    etseidl committed May 3, 2023
    Configuration menu
    Copy the full SHA
    cd850da View commit details
    Browse the repository at this point in the history

Commits on May 19, 2023

  1. DOC

    @raydouglass
    raydouglass committed May 19, 2023
    Configuration menu
    Copy the full SHA
    0b566b9 View commit details
    Browse the repository at this point in the history

Commits on May 23, 2023

  1. Merge pull request rapidsai#13416 from rapidsai/branch-23.06 

    Forward-merge branch-23.06 to branch-23.08
    @GPUtester
    GPUtester authored May 23, 2023
    Configuration menu
    Copy the full SHA
    c823dd3 View commit details
    Browse the repository at this point in the history

  2. Merge pull request rapidsai#13420 from rapidsai/branch-23.06 

    Forward-merge branch-23.06 to branch-23.08
    @GPUtester
    GPUtester authored May 23, 2023
    Configuration menu
    Copy the full SHA
    212b1c0 View commit details
    Browse the repository at this point in the history

  3. Merge pull request rapidsai#13421 from rapidsai/branch-23.06 

    Forward-merge branch-23.06 to branch-23.08
    @GPUtester
    GPUtester authored May 23, 2023
    Configuration menu
    Copy the full SHA
    905c61e View commit details
    Browse the repository at this point in the history

Commits on May 24, 2023

  1. Merge pull request rapidsai#13425 from rapidsai/branch-23.06 

    Forward-merge branch-23.06 to branch-23.08
    @GPUtester
    GPUtester authored May 24, 2023
    Configuration menu
    Copy the full SHA
    097b828 View commit details
    Browse the repository at this point in the history

  2. Merge pull request rapidsai#13427 from rapidsai/branch-23.06 

    Forward-merge branch-23.06 to branch-23.08
    @GPUtester
    GPUtester authored May 24, 2023
    Configuration menu
    Copy the full SHA
    9a0f87c View commit details
    Browse the repository at this point in the history

  3. Inline Cython exception handler (rapidsai#13411) 

    I originally placed the exception handler into a separate C++ header file that could be included by the Cython header because I figured that reading C++ inlined in Cython would be more confusing to devs. Unfortunately, the current approach complicates the build system due to the need to ensure that the directory containing the C++ header is always in the include path, which becomes problematic depending on where the files including the exception handler are (anywhere outside of `_lib` becomes problematic). Inlining is the simplest solution to this problem.

Authors:
  - Vyas Ramasubramani (https://github.com/vyasr)

Approvers:
  - Ashwin Srinath (https://github.com/shwina)
  - Bradley Dice (https://github.com/bdice)

URL: rapidsai#13411
    @vyasr
    vyasr authored May 24, 2023
    Configuration menu
    Copy the full SHA
    063a924 View commit details
    Browse the repository at this point in the history

  4. Merge pull request rapidsai#13430 from rapidsai/branch-23.06 

    Forward-merge branch-23.06 to branch-23.08
    @GPUtester
    GPUtester authored May 24, 2023
    Configuration menu
    Copy the full SHA
    aaf9362 View commit details
    Browse the repository at this point in the history

  5. Merge pull request rapidsai#13432 from rapidsai/branch-23.06 

    Forward-merge branch-23.06 to branch-23.08
    @GPUtester
    GPUtester authored May 24, 2023
    Configuration menu
    Copy the full SHA
    e29c691 View commit details
    Browse the repository at this point in the history

  6. Merge pull request rapidsai#13436 from rapidsai/branch-23.06 

    Forward-merge branch-23.06 to branch-23.08
    @GPUtester
    GPUtester authored May 24, 2023
    Configuration menu
    Copy the full SHA
    fd13c87 View commit details
    Browse the repository at this point in the history

  7. Merge pull request rapidsai#13439 from rapidsai/branch-23.06 

    Forward-merge branch-23.06 to branch-23.08
    @GPUtester
    GPUtester authored May 24, 2023
    Configuration menu
    Copy the full SHA
    0f0ebfd View commit details
    Browse the repository at this point in the history

Commits on May 25, 2023

  1. Merge pull request rapidsai#13441 from rapidsai/branch-23.06 

    Forward-merge branch-23.06 to branch-23.08
    @GPUtester
    GPUtester authored May 25, 2023
    Configuration menu
    Copy the full SHA
    0536a3a View commit details
    Browse the repository at this point in the history

  2. Merge pull request rapidsai#13443 from rapidsai/branch-23.06 

    Forward-merge branch-23.06 to branch-23.08
    @GPUtester
    GPUtester authored May 25, 2023
    Configuration menu
    Copy the full SHA
    5d5d367 View commit details
    Browse the repository at this point in the history

  3. Init JNI version 23.08.0-SNAPSHOT (rapidsai#13401) 

    Bump up JNI version to 23.08.0-SNAPSHOT in branch-23.08

Authors:
  - Peixin (https://github.com/pxLi)

Approvers:
  - Nghia Truong (https://github.com/ttnghia)
  - Jason Lowe (https://github.com/jlowe)

URL: rapidsai#13401
    @pxLi
    pxLi authored May 25, 2023
    Configuration menu
    Copy the full SHA
    a03da13 View commit details
    Browse the repository at this point in the history

  4. Merge pull request rapidsai#13445 from rapidsai/branch-23.06 

    Forward-merge branch-23.06 to branch-23.08
    @GPUtester
    GPUtester authored May 25, 2023
    Configuration menu
    Copy the full SHA
    126fa35 View commit details
    Browse the repository at this point in the history

  5. Merge pull request rapidsai#13446 from rapidsai/branch-23.06 

    Forward-merge branch-23.06 to branch-23.08
    @GPUtester
    GPUtester authored May 25, 2023
    Configuration menu
    Copy the full SHA
    7f97b27 View commit details
    Browse the repository at this point in the history

  6. Merge pull request rapidsai#13447 from rapidsai/branch-23.06 

    Forward-merge branch-23.06 to branch-23.08
    @GPUtester
    GPUtester authored May 25, 2023
    Configuration menu
    Copy the full SHA
    2def7f1 View commit details
    Browse the repository at this point in the history

  7. Merge pull request rapidsai#13448 from rapidsai/branch-23.06 

    Forward-merge branch-23.06 to branch-23.08
    @GPUtester
    GPUtester authored May 25, 2023
    Configuration menu
    Copy the full SHA
    53c685b View commit details
    Browse the repository at this point in the history

  8. Merge pull request rapidsai#13451 from rapidsai/branch-23.06 

    Forward-merge branch-23.06 to branch-23.08
    @GPUtester
    GPUtester authored May 25, 2023
    Configuration menu
    Copy the full SHA
    37f76c8 View commit details
    Browse the repository at this point in the history

  9. Reject functions without bytecode from _can_be_jitted in GroupBy Ap… 

    …ply (rapidsai#13429)

Closes rapidsai#13426

Authors:
  - https://github.com/brandon-b-miller

Approvers:
  - Matthew Roeschke (https://github.com/mroeschke)
  - Bradley Dice (https://github.com/bdice)

URL: rapidsai#13429
    @brandon-b-miller
    brandon-b-miller authored May 25, 2023
    Configuration menu
    Copy the full SHA
    5b3e3ab View commit details
    Browse the repository at this point in the history

Commits on May 26, 2023

  1. Separate io-text and nvtext pytests into different files (rapidsai#13435 

    )

Cleans up source files for nvtext and io-text pytests. The pytests are placed into separate files: `test_io_text.py` for the io-text pytests and `test_nvtext.py` for the nvtext pytests. Also removed the `python/cudf/cudf/tests/text` folder which contained 2 empty `.py` files.

Authors:
  - David Wendt (https://github.com/davidwendt)

Approvers:
  - Lawrence Mitchell (https://github.com/wence-)

URL: rapidsai#13435
    @davidwendt
    davidwendt authored May 26, 2023
    Configuration menu
    Copy the full SHA
    cc317ed View commit details
    Browse the repository at this point in the history

  2. Allow newer scikit-build (rapidsai#13424) 

    This PR attempts to allow using newer versions of scikit-build again.

cf. rapidsai#13188

Authors:
  - Vyas Ramasubramani (https://github.com/vyasr)

Approvers:
  - Bradley Dice (https://github.com/bdice)
  - AJ Schmidt (https://github.com/ajschmidt8)
  - Lawrence Mitchell (https://github.com/wence-)

URL: rapidsai#13424
    @vyasr
    vyasr authored May 26, 2023
    Configuration menu
    Copy the full SHA
    90bb887 View commit details
    Browse the repository at this point in the history

Commits on May 30, 2023

  1. JNI: Remove cleaned objects in memory cleaner (rapidsai#13378) 

    closes rapidsai#13412
Remove weak references of cleaned resources when a resource is cleaned.
The cleaned objects are never leaked, it's safe to remove the weak references. 
This is to reduce the memory usage.

Authors:
  - Chong Gao (https://github.com/res-life)

Approvers:
  - Jason Lowe (https://github.com/jlowe)
  - Robert (Bobby) Evans (https://github.com/revans2)
  - MithunR (https://github.com/mythrocks)

URL: rapidsai#13378
    @res-life
    res-life authored May 30, 2023
    Configuration menu
    Copy the full SHA
    4384c3b View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/branch-23.06' into feature/delta… 

    …_binary
    @etseidl
    etseidl committed May 30, 2023
    Configuration menu
    Copy the full SHA
    6707ab6 View commit details
    Browse the repository at this point in the history

  3. finish merge

    @etseidl
    etseidl committed May 30, 2023
    Configuration menu
    Copy the full SHA
    27f41cc View commit details
    Browse the repository at this point in the history

  4. Merge branch 'rapidsai:branch-23.08' into feature/delta_binary

    @etseidl
    etseidl authored May 30, 2023
    Configuration menu
    Copy the full SHA
    9ff14ea View commit details
    Browse the repository at this point in the history

  5. Merge pull request rapidsai#13471 from rapidsai/branch-23.06 

    Forward-merge branch-23.06 to branch-23.08
    @GPUtester
    GPUtester authored May 30, 2023
    Configuration menu
    Copy the full SHA
    5e12c25 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'rapidsai:branch-23.08' into feature/delta_binary

    @etseidl
    etseidl authored May 30, 2023
    Configuration menu
    Copy the full SHA
    dff6992 View commit details
    Browse the repository at this point in the history

  7. Ensure cccl packages don't clash with upstream version (rapidsai#13235) 

    Depends on: rapidsai/rapids-cmake#393

Once the above PR is merged, this updated logic ensures that cudf places the custom versions of cccl packages in correct places, and can find them once installed.

Authors:
  - Robert Maynard (https://github.com/robertmaynard)
  - Vyas Ramasubramani (https://github.com/vyasr)

Approvers:
  - Bradley Dice (https://github.com/bdice)
  - Vyas Ramasubramani (https://github.com/vyasr)

URL: rapidsai#13235
    @robertmaynard
    robertmaynard authored May 30, 2023
    Configuration menu
    Copy the full SHA
    87a8ede View commit details
    Browse the repository at this point in the history

Commits on May 31, 2023

  1. Merge branch 'rapidsai:branch-23.08' into feature/delta_binary

    @etseidl
    etseidl authored May 31, 2023
    Configuration menu
    Copy the full SHA
    d6c0020 View commit details
    Browse the repository at this point in the history