You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When reading parquet from s3, it's important to prefetch some file or row-groups. However, larger prefetch depth might cause more memory usage. And what make it even worse is that, interfaces like to_table and other reading might has different behaviors for prefetch. So we'd better and more document for it.
Component(s)
Documentation, Parquet, Python
The text was updated successfully, but these errors were encountered:
Describe the enhancement requested
Currently, we has the issue below:
When reading parquet from s3, it's important to prefetch some file or row-groups. However, larger prefetch depth might cause more memory usage. And what make it even worse is that, interfaces like
to_table
and other reading might has different behaviors for prefetch. So we'd better and more document for it.Component(s)
Documentation, Parquet, Python
The text was updated successfully, but these errors were encountered: