Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

planner: avoid change the type of the input of sum/avg #36372

Merged
merged 12 commits into from
Jul 29, 2022
Merged

planner: avoid change the type of the input of sum/avg #36372

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
5df919a
avoid change the type of the input of sum/avg
windtalker Jul 20, 2022
de4372e
fix tests in explain_test
windtalker Jul 21, 2022
0cdb51f
fix ci
windtalker Jul 21, 2022
d716899
update tests
windtalker Jul 21, 2022
2430d48
fix ci
windtalker Jul 22, 2022
2176290
fix ci
windtalker Jul 22, 2022
b4038ed
fix ci
windtalker Jul 22, 2022
3813b8c
fix ut
windtalker Jul 22, 2022
338684f
Merge branch 'master' into agg_refine
ti-chi-bot Jul 29, 2022
8e1d9c1
Merge branch 'master' into agg_refine
ti-chi-bot Jul 29, 2022
fdded85
Merge branch 'master' into agg_refine
ti-chi-bot Jul 29, 2022
df95a77
Merge branch 'master' into agg_refine
ti-chi-bot Jul 29, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions cmd/explaintest/r/agg_predicate_pushdown.result
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ id estRows task access object operator info
Projection 711.11 root test.t.a, test.t.b, Column#5
└─Selection 711.11 root gt(Column#5, 3)
└─HashAgg 888.89 root group by:Column#16, Column#17, Column#18, funcs:avg(Column#13)->Column#5, funcs:firstrow(Column#14)->test.t.a, funcs:firstrow(Column#15)->test.t.b
└─Projection 1111.11 root cast(test.t.c, decimal(15,4) BINARY)->Column#13, test.t.a, test.t.b, test.t.a, test.t.b, test.t.c
└─Projection 1111.11 root cast(test.t.c, decimal(20,0) BINARY)->Column#13, test.t.a, test.t.b, test.t.a, test.t.b, test.t.c
└─TableReader 1111.11 root data:Selection
└─Selection 1111.11 cop[tikv] gt(test.t.a, 1), gt(test.t.a, 2), gt(test.t.b, 2)
└─TableFullScan 10000.00 cop[tikv] table:t keep order:false, stats:pseudo
Expand All @@ -17,7 +17,7 @@ id estRows task access object operator info
Projection 657.65 root test.t.a, test.t.b, Column#5
└─Selection 657.65 root gt(Column#5, 3)
└─HashAgg 822.06 root group by:Column#16, Column#17, Column#18, funcs:avg(Column#13)->Column#5, funcs:firstrow(Column#14)->test.t.a, funcs:firstrow(Column#15)->test.t.b
└─Projection 1027.57 root cast(test.t.c, decimal(15,4) BINARY)->Column#13, test.t.a, test.t.b, test.t.a, test.t.b, test.t.c
└─Projection 1027.57 root cast(test.t.c, decimal(20,0) BINARY)->Column#13, test.t.a, test.t.b, test.t.a, test.t.b, test.t.c
└─TableReader 1027.57 root data:Selection
└─Selection 1027.57 cop[tikv] gt(test.t.b, 2), or(gt(test.t.a, 1), gt(test.t.b, 2)), or(gt(test.t.a, 2), lt(test.t.b, 1))
└─TableFullScan 10000.00 cop[tikv] table:t keep order:false, stats:pseudo
Expand All @@ -27,7 +27,7 @@ id estRows task access object operator info
Projection 3027.54 root test.t.a, test.t.b, Column#5
└─Selection 3027.54 root or(and(gt(test.t.a, 1), gt(test.t.b, 2)), or(and(gt(test.t.a, 2), lt(test.t.b, 1)), and(gt(test.t.b, 2), gt(Column#5, 3))))
└─HashAgg 3784.43 root group by:Column#16, Column#17, Column#18, funcs:avg(Column#13)->Column#5, funcs:firstrow(Column#14)->test.t.a, funcs:firstrow(Column#15)->test.t.b
└─Projection 4730.53 root cast(test.t.c, decimal(15,4) BINARY)->Column#13, test.t.a, test.t.b, test.t.a, test.t.b, test.t.c
└─Projection 4730.53 root cast(test.t.c, decimal(20,0) BINARY)->Column#13, test.t.a, test.t.b, test.t.a, test.t.b, test.t.c
└─TableReader 4730.53 root data:Selection
└─Selection 4730.53 cop[tikv] or(and(gt(test.t.a, 1), gt(test.t.b, 2)), or(and(gt(test.t.a, 2), lt(test.t.b, 1)), gt(test.t.b, 2)))
└─TableFullScan 10000.00 cop[tikv] table:t keep order:false, stats:pseudo
Expand All @@ -37,7 +37,7 @@ id estRows task access object operator info
Projection 2126.93 root test.t.a, test.t.b, Column#5
└─Selection 2126.93 root or(gt(test.t.a, 1), gt(Column#5, 1))
└─HashAgg 2658.67 root group by:Column#16, Column#17, Column#18, funcs:avg(Column#13)->Column#5, funcs:firstrow(Column#14)->test.t.a, funcs:firstrow(Column#15)->test.t.b
└─Projection 3323.33 root cast(test.t.c, decimal(15,4) BINARY)->Column#13, test.t.a, test.t.b, test.t.a, test.t.b, test.t.c
└─Projection 3323.33 root cast(test.t.c, decimal(20,0) BINARY)->Column#13, test.t.a, test.t.b, test.t.a, test.t.b, test.t.c
└─TableReader 3323.33 root data:Selection
└─Selection 3323.33 cop[tikv] lt(test.t.a, 3)
└─TableFullScan 10000.00 cop[tikv] table:t keep order:false, stats:pseudo
Expand All @@ -47,7 +47,7 @@ id estRows task access object operator info
Projection 6393.60 root test.t.a, test.t.b, Column#5
└─Selection 6393.60 root or(and(gt(test.t.a, 1), gt(Column#5, 1)), lt(test.t.a, 3))
└─HashAgg 7992.00 root group by:Column#16, Column#17, Column#18, funcs:avg(Column#13)->Column#5, funcs:firstrow(Column#14)->test.t.a, funcs:firstrow(Column#15)->test.t.b
└─Projection 9990.00 root cast(test.t.c, decimal(15,4) BINARY)->Column#13, test.t.a, test.t.b, test.t.a, test.t.b, test.t.c
└─Projection 9990.00 root cast(test.t.c, decimal(20,0) BINARY)->Column#13, test.t.a, test.t.b, test.t.a, test.t.b, test.t.c
└─TableReader 9990.00 root data:Selection
└─Selection 9990.00 cop[tikv] or(gt(test.t.a, 1), lt(test.t.a, 3))
└─TableFullScan 10000.00 cop[tikv] table:t keep order:false, stats:pseudo
Expand Down
12 changes: 6 additions & 6 deletions cmd/explaintest/r/explain_easy.result
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -763,7 +763,7 @@ explain format = 'brief' select a, b from (select a, b, avg(b) over (partition b
id estRows task access object operator info
Projection 2666.67 root test.t.a, test.t.b
└─Selection 2666.67 root gt(cast(test.t.a, decimal(20,0) BINARY), Column#5), lt(test.t.b, 10)
└─Window 3333.33 root avg(cast(test.t.b, decimal(15,4) BINARY))->Column#5 over(partition by test.t.a)
└─Window 3333.33 root avg(cast(test.t.b, decimal(20,0) BINARY))->Column#5 over(partition by test.t.a)
└─Sort 3333.33 root test.t.a
└─TableReader 3333.33 root data:Selection
└─Selection 3333.33 cop[tikv] gt(test.t.a, 10)
Expand Down Expand Up @@ -925,7 +925,7 @@ StreamAgg 1.00 root funcs:sum(Column#14)->Column#13
explain format = 'brief' select avg(t1.a) from t t1 join t t2 on t1.a=t2.a;
id estRows task access object operator info
StreamAgg 1.00 root funcs:avg(Column#14)->Column#13
└─Projection 12487.50 root cast(test.t.a, decimal(8,4) BINARY)->Column#14
└─Projection 12487.50 root cast(test.t.a, decimal(20,0) BINARY)->Column#14
└─HashJoin 12487.50 root inner join, equal:[eq(test.t.a, test.t.a)]
├─TableReader(Build) 9990.00 root data:Selection
│ └─Selection 9990.00 cop[tikv] not(isnull(test.t.a))
Expand All @@ -936,7 +936,7 @@ StreamAgg 1.00 root funcs:avg(Column#14)->Column#13
explain format = 'brief' select avg(t1.b) from t t1 join t t2 on t1.b=t2.b;
id estRows task access object operator info
StreamAgg 1.00 root funcs:avg(Column#14)->Column#13
└─Projection 12487.50 root cast(test.t.b, decimal(10,4) BINARY)->Column#14
└─Projection 12487.50 root cast(test.t.b, decimal(20,0) BINARY)->Column#14
└─HashJoin 12487.50 root inner join, equal:[eq(test.t.b, test.t.b)]
├─TableReader(Build) 9990.00 root data:Selection
│ └─Selection 9990.00 cop[tikv] not(isnull(test.t.b))
Expand All @@ -947,7 +947,7 @@ StreamAgg 1.00 root funcs:avg(Column#14)->Column#13
explain format = 'brief' select avg(t1.c) from t t1 join t t2 on t1.c=t2.c;
id estRows task access object operator info
StreamAgg 1.00 root funcs:avg(Column#14)->Column#13
└─Projection 12487.50 root cast(test.t.c, decimal(13,4) BINARY)->Column#14
└─Projection 12487.50 root cast(test.t.c, decimal(20,0) BINARY)->Column#14
└─HashJoin 12487.50 root inner join, equal:[eq(test.t.c, test.t.c)]
├─TableReader(Build) 9990.00 root data:Selection
│ └─Selection 9990.00 cop[tikv] not(isnull(test.t.c))
Expand All @@ -958,7 +958,7 @@ StreamAgg 1.00 root funcs:avg(Column#14)->Column#13
explain format = 'brief' select avg(t1.d) from t t1 join t t2 on t1.d=t2.d;
id estRows task access object operator info
StreamAgg 1.00 root funcs:avg(Column#14)->Column#13
└─Projection 12487.50 root cast(test.t.d, decimal(15,4) BINARY)->Column#14
└─Projection 12487.50 root cast(test.t.d, decimal(20,0) BINARY)->Column#14
└─HashJoin 12487.50 root inner join, equal:[eq(test.t.d, test.t.d)]
├─TableReader(Build) 9990.00 root data:Selection
│ └─Selection 9990.00 cop[tikv] not(isnull(test.t.d))
Expand All @@ -969,7 +969,7 @@ StreamAgg 1.00 root funcs:avg(Column#14)->Column#13
explain format = 'brief' select avg(t1.e) from t t1 join t t2 on t1.e=t2.e;
id estRows task access object operator info
StreamAgg 1.00 root funcs:avg(Column#14)->Column#13
└─Projection 12487.50 root cast(test.t.e, decimal(24,4) BINARY)->Column#14
└─Projection 12487.50 root cast(test.t.e, decimal(20,0) BINARY)->Column#14
└─HashJoin 12487.50 root inner join, equal:[eq(test.t.e, test.t.e)]
├─TableReader(Build) 9990.00 root data:Selection
│ └─Selection 9990.00 cop[tikv] not(isnull(test.t.e))
Expand Down
2 changes: 1 addition & 1 deletion cmd/explaintest/r/index_merge.result
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -536,7 +536,7 @@ Sort_16 1841.86 root test.t1.c1
│ └─TableRowIDScan_24 5542.21 cop[tikv] table:t1 keep order:false, stats:pseudo
└─TopN_29(Probe) 1.00 root test.t2.c1, offset:2, count:1
└─HashAgg_36 2660.44 root group by:Column#21, funcs:avg(Column#19)->Column#9, funcs:firstrow(Column#20)->test.t2.c1
└─Projection_48 3325.55 root cast(test.t2.c1, decimal(15,4) BINARY)->Column#19, test.t2.c1, test.t2.c1
└─Projection_48 3325.55 root cast(test.t2.c1, decimal(20,0) BINARY)->Column#19, test.t2.c1, test.t2.c1
└─IndexMerge_41 3325.55 root
├─Selection_38(Build) 3.32 cop[tikv] eq(test.t1.c1, test.t2.c1)
│ └─IndexRangeScan_37 3323.33 cop[tikv] table:t2, index:c1(c1) range:[-inf,10), keep order:false, stats:pseudo
Expand Down
2 changes: 1 addition & 1 deletion expression/aggregation/agg_to_pb_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ func TestAggFunc2Pb(t *testing.T) {

jsons := []string{
`{"tp":3002,"children":[{"tp":201,"val":"gAAAAAAAAAE=","sig":0,"field_type":{"tp":5,"flag":0,"flen":-1,"decimal":-1,"collate":-63,"charset":"binary"},"has_distinct":false}],"sig":0,"field_type":{"tp":5,"flag":0,"flen":-1,"decimal":-1,"collate":-63,"charset":"binary"},"has_distinct":%v,"aggFuncMode":0}`,
`{"tp":3001,"children":[{"tp":201,"val":"gAAAAAAAAAE=","sig":0,"field_type":{"tp":5,"flag":0,"flen":-1,"decimal":-1,"collate":-63,"charset":"binary"},"has_distinct":false}],"sig":0,"field_type":{"tp":8,"flag":0,"flen":-1,"decimal":-1,"collate":-63,"charset":"binary"},"has_distinct":%v,"aggFuncMode":0}`,
`{"tp":3001,"children":[{"tp":201,"val":"gAAAAAAAAAE=","sig":0,"field_type":{"tp":5,"flag":0,"flen":-1,"decimal":-1,"collate":-63,"charset":"binary"},"has_distinct":false}],"sig":0,"field_type":{"tp":8,"flag":0,"flen":20,"decimal":0,"collate":-63,"charset":"binary"},"has_distinct":%v,"aggFuncMode":0}`,
`{"tp":3003,"children":[{"tp":201,"val":"gAAAAAAAAAE=","sig":0,"field_type":{"tp":5,"flag":0,"flen":-1,"decimal":-1,"collate":-63,"charset":"binary"},"has_distinct":false}],"sig":0,"field_type":{"tp":5,"flag":0,"flen":-1,"decimal":-1,"collate":-63,"charset":"binary"},"has_distinct":%v,"aggFuncMode":0}`,
`{"tp":3007,"val":"AAAAAAAABAA=","children":[{"tp":201,"val":"gAAAAAAAAAE=","sig":0,"field_type":{"tp":5,"flag":0,"flen":-1,"decimal":-1,"collate":-63,"charset":"binary"},"has_distinct":false}],"sig":0,"field_type":{"tp":15,"flag":0,"flen":-1,"decimal":-1,"collate":-46,"charset":"utf8mb4"},"has_distinct":%v,"aggFuncMode":0}`,
`{"tp":3005,"children":[{"tp":201,"val":"gAAAAAAAAAE=","sig":0,"field_type":{"tp":5,"flag":0,"flen":-1,"decimal":-1,"collate":-63,"charset":"binary"},"has_distinct":false}],"sig":0,"field_type":{"tp":5,"flag":0,"flen":-1,"decimal":-1,"collate":-63,"charset":"binary"},"has_distinct":%v,"aggFuncMode":0}`,
Expand Down
22 changes: 0 additions & 22 deletions expression/aggregation/base_func.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -423,28 +423,6 @@ func (a *baseFuncDesc) WrapCastForAggArgs(ctx sessionctx.Context) {
}
tpOld := a.Args[i].GetType().GetType()
a.Args[i] = castFunc(ctx, a.Args[i])
if a.Name != ast.AggFuncAvg && a.Name != ast.AggFuncSum {
continue
}
// After wrapping cast on the argument, flen etc. may not the same
// as the type of the aggregation function. The following part set
// the type of the argument exactly as the type of the aggregation
// function.
// Note: If the `tp` of argument is the same as the `tp` of the
// aggregation function, it will not wrap cast function on it
// internally. The reason of the special handling for `Column` is
// that the `RetType` of `Column` refers to the `infoschema`, so we
// need to set a new variable for it to avoid modifying the
// definition in `infoschema`.
if col, ok := a.Args[i].(*expression.Column); ok {
col.RetType = types.NewFieldType(col.RetType.GetType())
}
// originTp is used when the `tp` of column is TypeFloat32 while
// the type of the aggregation function is TypeFloat64.
originTp := a.Args[i].GetType().GetType()
*(a.Args[i].GetType()) = *(a.RetTp)
a.Args[i].GetType().SetType(originTp)

// refine each mysql integer type to the needed decimal precision for sum
if a.Name == ast.AggFuncSum {
adjustDecimalLenForSumInteger(a.Args[i].GetType(), tpOld)
Expand Down
2 changes: 2 additions & 0 deletions expression/builtin_cast.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1941,7 +1941,9 @@ func WrapWithCastAsDecimal(ctx sessionctx.Context, expr Expression) Expression {
tp.SetDecimalUnderLimit(expr.GetType().GetDecimal())

if expr.GetType().EvalType() == types.ETInt {
// todo set the flen returned by minimalDecimalLenForHoldingInteger
tp.SetFlen(mysql.MaxIntWidth)
tp.SetDecimal(0)
}
if tp.GetFlen() == types.UnspecifiedLength || tp.GetFlen() > mysql.MaxDecimalWidth {
tp.SetFlen(mysql.MaxDecimalWidth)
Expand Down
Loading