-
Notifications
You must be signed in to change notification settings - Fork 758
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Clarify GenericColumnReader::read_records #4540
Conversation
425290f
to
f60c307
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
parquet/src/column/reader.rs
Outdated
/// Read up to `num_records` returning the number of complete records, non-null | ||
/// values and levels decoded | ||
/// Read up to `num_records` whole records, returning the number of complete | ||
/// records, non-null values and levels decoded. Records will not be partially read |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What does "partially read" mean? Like does it mean that the a record will not be read unless there is sufficient data for all columns?
If so, maybe
/// records, non-null values and levels decoded. Records will not be partially read | |
/// records, non-null values and levels decoded. Records will not be partially read, | |
/// they are only read as long as there is sufficient data for all columns available. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You only have a single column here, this is referring to record shredding of repeated values, will clarify further...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you
Which issue does this PR close?
Closes #.
Rationale for this change
#4376 (comment)
What changes are included in this PR?
Are there any user-facing changes?