Poor performance compared to duck db on ClickBench

[dudzicp]

Hello,
I opened benchmarks linked in readme and it turns out that on comparable machine, datafusion performs few times worse than e.g. duckdb. What could be the reason for this?

[korowa]

Hi!

I suppose it'll be more fair to compare datafusion with duckdb-parquet -- in this case both systems have to process same data without transforming it to systems internal format.

Currently there are several configuration options, disabled by default, which could significantly improve performance for queries over parquet files if enabled:

1. Enable parquet filter pushdown by default #3463 and Enable parquet page level skipping (page index pruning) by default #4085 - these two could help to speed up queries with filters, by reducing input size prior to actual scanning
2. Enable parquet parallel scans by default #5125 - this one should speed up all parquet scans in general (benchmark is performed over single parquet file and currently, by default, datafusion lacks parallelism in this case)

To sum up -- at this moment, datafusion by default doesnt utilize all features available for parquet format to speed up scanning/filtering.

[korowa]

(And yeah, although, enabling all these settings could give data noticeable boost, enabling them doesn't guarantee outperforming/parity with duckdb -- but it could be a subject for further discussion 🙃 )

[dudzicp]

Thanks, I think it would make sense to update the benchmarks

[alippai]

@korowa this is interesting. So if datafusion uses parquet you can’t avoid decompression and syscalls, but if it uses Arrow then it doesn’t have statistics, bloom filter, page index?

[korowa]

Actually first two settings I mentioned above still could help with reducing amount of decompressing work.

if it uses Arrow then it doesn’t have statistics, bloom filter, page index

Correct me, please, if I'm wrong -- do you mean smth like ADBC? Unfortunately, I'm not aware of any use case of Arrow as data source.

[alippai]

I had the impression datafusion supports .arrow tables / datasets either in-memory or from disk. Likely I was wrong :)

[korowa]

Ah, got it! Not sure about "out of the box" functionality, but it should be possible to achieve via custom_datasource -- actually it even provides some API for pushing down predicates (not sure if it could be useful in case of arrow/feather files)

[dudzicp]

Is there any benchmark tracking so we can observe how performance changes over time?