You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem or challenge?
While working on benchmarking for #7655 and starting to compare to serialize/write performance of other systems, I realized our current default parquet compression codec is Uncompressed. This is surprising as one almost always wants compression when using parquet.
The default is currently inherited up from arrow-rs
Is your feature request related to a problem or challenge?
While working on benchmarking for #7655 and starting to compare to serialize/write performance of other systems, I realized our current default parquet compression codec is Uncompressed. This is surprising as one almost always wants compression when using parquet.
The default is currently inherited up from
arrow-rs
https://github.com/apache/arrow-rs/blob/471f6dd2911d8328ca56efe2f685e08c0a3fb8c8/parquet/src/file/properties.rs#L33-L34
Describe the solution you'd like
Set a sensible default compression such as Zstd level 3.
Describe alternatives you've considered
The default of Uncompressed is coming from the
arrow-rs
crate. We could instead update the default there.Additional context
No response
The text was updated successfully, but these errors were encountered: