feat(optimizer): rewrite predicate and accelerate tpch19

[xudong963]

I hereby agree to the terms of the CLA available at: https://databend.rs/dev/policies/cla/

Summary

Rewrite OR predicates, tpch 19 can be optimized to Inner Join

+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| explain                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Project: [revenue]                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|     EvalScalar: [sum(l_extendedprice * 1 - l_discount)]                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
|         Aggregate: group items: [], aggregate functions: [sum(l_extendedprice * 1 - l_discount)]                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
|             EvalScalar: [*(lineitem.l_extendedprice, -(1, lineitem.l_discount))]                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
|                 Filter: [((((((part.p_brand = Brand#52) AND (in(part.p_container, SM CASE, SM BOX, SM PACK, SM PKG))) AND (lineitem.l_quantity >= 4)) AND (lineitem.l_quantity <= 14)) AND (<=(part.p_size, 5))) OR (((((part.p_brand = Brand#11) AND (in(part.p_container, MED BAG, MED BOX, MED PKG, MED PACK))) AND (lineitem.l_quantity >= 18)) AND (lineitem.l_quantity <= 28)) AND (<=(part.p_size, 10)))) OR (((((part.p_brand = Brand#51) AND (in(part.p_container, LG CASE, LG BOX, LG PACK, LG PKG))) AND (lineitem.l_quantity >= 29)) AND (lineitem.l_quantity <= 39)) AND (<=(part.p_size, 15)))] |
|                     HashJoin: INNER, build keys: [part.p_partkey], probe keys: [lineitem.l_partkey], join filters: []                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
|                         Filter: [in(lineitem.l_shipmode, AIR, AIR REG), lineitem.l_shipinstruct = DELIVER IN PERSON]                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
|                             Scan: default.default.lineitem                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
|                         Filter: [>=(part.p_size, 1)]                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
|                             Scan: default.default.part                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
10 rows in set (0.05 sec)
Read 0 rows, 0.00 B in 0.002 sec., 0 rows/sec., 0.00 B/sec.

Changelog

• New Feature

Related Issues

Fixes #6096

[mergify]

Thanks for the contribution!
I have applied any labels matching special text in your PR Changelog.

Please review the labels and make any necessary changes.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Updated
databend	⬜️ Ignored (Inspect)		Jun 29, 2022 at 11:02AM (UTC)

[xudong963]

Tpch 19 can run without OOM with 1G' data @BohuTANG

mysql> source tpch-q19.sql
+--------------------+
| revenue            |
+--------------------+
| 3083843.0577999996 |
+--------------------+
1 row in set (4.34 sec)
Read 6201215 rows, 950.74 MiB in 4.266 sec., 1.45 million rows/sec., 222.84 MiB/sec.

[leiysky]

I think you can add this rule as a RuleNormalizeDisjunctiveFilter.

[xudong963]

I think you can add this rule as a RuleNormalizeDisjunctiveFilter.

Done!