You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem or challenge?
For Parquet files, the time_elapsed_opening metric encompasses all of (in execution order):
Reading and parsing the footer/metadata (which is usually negligeable unless the page index is enabled, no metric either way)
Applying the row-level pushdown filter (pushdown_eval_time metric)
Parsing statistics and using them to prune (no metric)
Parsing Bloom Filters and using them to prune (no metric)
Pruning using the page filter (page_filter_eval_time metric)
Selecting columns and projecting (no metric, negligeable time)
Describe the solution you'd like
Add missing metrics:
metadata_load_time for part 1
statistics_eval_time for part 3
bloom_filter_eval_timefor part 4
Rename pushdown_eval_time to row_pushdown_eval_time, because statistics and eval time are also pushdown filters
Describe alternatives you've considered
Split statistics_eval_time and bloom_filter_eval_time each into two metrics, one for load time and one for eval time. But eval time is negligeable for both, so probably not worth it.
Additional context
No response
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem or challenge?
For Parquet files, the
time_elapsed_opening
metric encompasses all of (in execution order):pushdown_eval_time
metric)page_filter_eval_time
metric)Describe the solution you'd like
metadata_load_time
for part 1statistics_eval_time
for part 3bloom_filter_eval_time
for part 4pushdown_eval_time
torow_pushdown_eval_time
, because statistics and eval time are also pushdown filtersDescribe alternatives you've considered
Split
statistics_eval_time
andbloom_filter_eval_time
each into two metrics, one for load time and one for eval time. But eval time is negligeable for both, so probably not worth it.Additional context
No response
The text was updated successfully, but these errors were encountered: