tidb reports error if there is a leftsemijoin on the top but refers no columns on probe side on which runs a mpp join #1718

fzhedu · 2021-04-01T09:59:14Z

mysql -h 172.16.5.85 -P53324 -uroot -D agg_sub_datat10000

[fzh@h81 ~]$ tiup cluster display v408
Found cluster newer version:

    The latest version:         v1.4.0
    Local installed version:    v1.3.5
    Update current component:   tiup update cluster
    Update all components:      tiup update --all

Starting component `cluster`: /data2/fzh/.tiup/components/cluster/v1.3.5/tiup-cluster display v408
Cluster type:       tidb
Cluster name:       v408
Cluster version:    nightly
SSH type:           builtin
Dashboard URL:      http://172.16.5.85:53322/dashboard
ID                 Role     Host         Ports                                OS/Arch       Status   Data Dir                           Deploy Dir
--                 ----     ----         -----                                -------       ------   --------                           ----------
172.16.5.85:53322  pd       172.16.5.85  53322/53323                          linux/x86_64  Up|L|UI  /data1/fzh/tidb-data/pd-53322      /data1/fzh/tidb-deploy/pd-53322
172.16.5.85:53324  tidb     172.16.5.85  53324/53325                          linux/x86_64  Up       -                                  /data1/fzh/tidb-deploy/tidb-53324
172.16.5.59:53334  tiflash  172.16.5.59  53334/53335/53337/53338/53339/53336  linux/x86_64  Up       /data2/fzh/tidb-data/tiflash-9000  /data2/fzh/tidb-deploy/tiflash-9000
172.16.5.81:53334  tiflash  172.16.5.81  53334/53335/53337/53338/53339/53336  linux/x86_64  Up       /data2/fzh/tidb-data/tiflash-9000  /data2/fzh/tidb-deploy/tiflash-9000
172.16.5.85:53334  tiflash  172.16.5.85  53334/53335/53337/53338/53339/53336  linux/x86_64  Up       /data2/fzh/tidb-data/tiflash-9000  /data2/fzh/tidb-deploy/tiflash-9000
172.16.5.85:53326  tikv     172.16.5.85  53326/53327                          linux/x86_64  Up       /data1/fzh/tidb-data/tikv-53326    /data1/fzh/tidb-deploy/tikv-53326
Total nodes: 6

mysql> SELECT  ( 'g', 'p' )  IN ( SELECT   SUBQUERY1_t1 . `col_varchar_binary` AS SUBQUERY1_field1 , MAX(  SUBQUERY1_t1 . `col_varchar_binary_not_null_key` ) AS SUBQUERY1_field2 FROM ( `X` AS SUBQUERY1_t1 RIGHT  JOIN `V` AS SUBQUERY1_t2 ON (SUBQUERY1_t2 . `col_decimal_30_10` = SUBQUERY1_t1 . `pk`  ) ) WHERE  ( SUBQUERY1_t1 . `col_int` , SUBQUERY1_t2 . `col_decimal` ) NOT IN ( SELECT   CHILD_SUBQUERY1_t1 . `col_int_key` AS CHILD_SUBQUERY1_field1 , CHILD_SUBQUERY1_t1 . `col_decimal_not_null_key` AS child_subquery1_field2 FROM `D` AS CHILD_SUBQUERY1_t1    ) GROUP BY SUBQUERY1_field1  ) AS field1 FROM ( `FF` AS table1 STRAIGHT_JOIN `B` AS table2 ON (table2 . `col_varchar_binary_key` = table1 . `col_varchar_binary_not_null`  ) ) WHERE (  table1 . `col_decimal_30_10`  IN ( SELECT DISTINCT  SUBQUERY2_t1 . `col_decimal` AS SUBQUERY2_field1 FROM `S` AS SUBQUERY2_t1    ) ) AND ( table1 . `col_int_not_null_key` = table2 . `col_int_key` AND table1 . `col_varchar_binary` < 'r' )   ;
+--------+
| field1 |
+--------+
|   NULL |
|   NULL |
|   NULL |
+--------+
3 rows in set (0.01 sec)

mysql> set @@tidb_isolation_read_engines='tiflash,tidb'; set @@tidb_allow_mpp=1;
Query OK, 0 rows affected (0.00 sec)

Query OK, 0 rows affected (0.00 sec)

mysql> SELECT  ( 'g', 'p' )  IN ( SELECT   SUBQUERY1_t1 . `col_varchar_binary` AS SUBQUERY1_field1 , MAX(  SUBQUERY1_t1 . `col_varchar_binary_not_null_key` ) AS SUBQUERY1_field2 FROM ( `X` AS SUBQUERY1_t1 RIGHT  JOIN `V` AS SUBQUERY1_t2 ON (SUBQUERY1_t2 . `col_decimal_30_10` = SUBQUERY1_t1 . `pk`  ) ) WHERE  ( SUBQUERY1_t1 . `col_int` , SUBQUERY1_t2 . `col_decimal` ) NOT IN ( SELECT   CHILD_SUBQUERY1_t1 . `col_int_key` AS CHILD_SUBQUERY1_field1 , CHILD_SUBQUERY1_t1 . `col_decimal_not_null_key` AS child_subquery1_field2 FROM `D` AS CHILD_SUBQUERY1_t1    ) GROUP BY SUBQUERY1_field1  ) AS field1 FROM ( `FF` AS table1 STRAIGHT_JOIN `B` AS table2 ON (table2 . `col_varchar_binary_key` = table1 . `col_varchar_binary_not_null`  ) ) WHERE (  table1 . `col_decimal_30_10`  IN ( SELECT DISTINCT  SUBQUERY2_t1 . `col_decimal` AS SUBQUERY2_field1 FROM `S` AS SUBQUERY2_t1    ) ) AND ( table1 . `col_int_not_null_key` = table2 . `col_int_key` AND table1 . `col_varchar_binary` < 'r' )   ;
ERROR 1105 (HY000): runtime error: index out of range [0] with length 0
mysql> desc SELECT  ( 'g', 'p' )  IN ( SELECT   SUBQUERY1_t1 . `col_varchar_binary` AS SUBQUERY1_field1 , MAX(  SUBQUERY1_t1 . `col_varchar_binary_not_null_key` ) AS SUBQUERY1_field2 FROM ( `X` AS SUBQUERY1_t1 RIGHT  JOIN `V` AS SUBQUERY1_t2 ON (SUBQUERY1_t2 . `col_decimal_30_10` = SUBQUERY1_t1 . `pk`  ) ) WHERE  ( SUBQUERY1_t1 . `col_int` , SUBQUERY1_t2 . `col_decimal` ) NOT IN ( SELECT   CHILD_SUBQUERY1_t1 . `col_int_key` AS CHILD_SUBQUERY1_field1 , CHILD_SUBQUERY1_t1 . `col_decimal_not_null_key` AS child_subquery1_field2 FROM `D` AS CHILD_SUBQUERY1_t1    ) GROUP BY SUBQUERY1_field1  ) AS field1 FROM ( `FF` AS table1 STRAIGHT_JOIN `B` AS table2 ON (table2 . `col_varchar_binary_key` = table1 . `col_varchar_binary_not_null`  ) ) WHERE (  table1 . `col_decimal_30_10`  IN ( SELECT DISTINCT  SUBQUERY2_t1 . `col_decimal` AS SUBQUERY2_field1 FROM `S` AS SUBQUERY2_t1    ) ) AND ( table1 . `col_int_not_null_key` = table2 . `col_int_key` AND table1 . `col_varchar_binary` < 'r' )   ;
+------------------------------------------------+---------+-------------------+--------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| id                                             | estRows | task              | access object            | operator info                                                                                                                                                                                                                   |
+------------------------------------------------+---------+-------------------+--------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| HashJoin_30                                    | 4.04    | root              |                          | CARTESIAN left outer semi join, other cond:eq("g", agg_sub_datat10000.x.col_varchar_binary), eq("p", Column#297)                                                                                                                |
| ├─HashAgg_94(Build)                            | 8.32    | root              |                          | group by:agg_sub_datat10000.x.col_varchar_binary, funcs:max(agg_sub_datat10000.x.col_varchar_binary_not_null_key)->Column#297, funcs:firstrow(agg_sub_datat10000.x.col_varchar_binary)->agg_sub_datat10000.x.col_varchar_binary |
| │ └─HashJoin_99                                | 63.20   | root              |                          | CARTESIAN anti semi join, other cond:eq(agg_sub_datat10000.v.col_decimal, agg_sub_datat10000.d.col_decimal_not_null_key), eq(agg_sub_datat10000.x.col_int, agg_sub_datat10000.d.col_int_key)                                    |
| │   ├─TableReader_114(Build)                   | 30.00   | root              |                          | data:TableFullScan_113                                                                                                                                                                                                          |
| │   │ └─TableFullScan_113                      | 30.00   | cop[tiflash]      | table:CHILD_SUBQUERY1_t1 | keep order:false, stats:pseudo                                                                                                                                                                                                  |
| │   └─HashJoin_101(Probe)                      | 79.00   | root              |                          | right outer join, equal:[eq(Column#299, agg_sub_datat10000.v.col_decimal_30_10)]                                                                                                                                                |
| │     ├─Projection_108(Build)                  | 13.00   | root              |                          | agg_sub_datat10000.x.col_varchar_binary, agg_sub_datat10000.x.col_int, agg_sub_datat10000.x.col_varchar_binary_not_null_key, cast(agg_sub_datat10000.x.pk, decimal(20,0) BINARY)->Column#299                                    |
| │     │ └─TableReader_110                      | 13.00   | root              |                          | data:TableFullScan_109                                                                                                                                                                                                          |
| │     │   └─TableFullScan_109                  | 13.00   | cop[tiflash]      | table:SUBQUERY1_t1       | keep order:false, stats:pseudo                                                                                                                                                                                                  |
| │     └─TableReader_112(Probe)                 | 79.00   | root              |                          | data:TableFullScan_111                                                                                                                                                                                                          |
| │       └─TableFullScan_111                    | 79.00   | cop[tiflash]      | table:SUBQUERY1_t2       | keep order:false, stats:pseudo                                                                                                                                                                                                  |
| └─TableReader_58(Probe)                        | 4.04    | root              |                          | data:ExchangeSender_57                                                                                                                                                                                                          |
|   └─ExchangeSender_57                          | 4.04    | batchCop[tiflash] |                          | ExchangeType: PassThrough                                                                                                                                                                                                       |
|     └─HashJoin_32                              | 4.04    | batchCop[tiflash] |                          | inner join, equal:[eq(agg_sub_datat10000.s.col_decimal, agg_sub_datat10000.ff.col_decimal_30_10)]                                                                                                                               |
|       ├─ExchangeReceiver_56(Build)             | 4.04    | batchCop[tiflash] |                          |                                                                                                                                                                                                                                 |
|       │ └─ExchangeSender_55                    | 4.04    | batchCop[tiflash] |                          | ExchangeType: Broadcast                                                                                                                                                                                                         |
|       │   └─HashJoin_48                        | 4.04    | batchCop[tiflash] |                          | inner join, equal:[eq(agg_sub_datat10000.ff.col_varchar_binary_not_null, agg_sub_datat10000.b.col_varchar_binary_key) eq(agg_sub_datat10000.ff.col_int_not_null_key, agg_sub_datat10000.b.col_int_key)]                         |
|       │     ├─ExchangeReceiver_52(Build)       | 1.33    | batchCop[tiflash] |                          |                                                                                                                                                                                                                                 |
|       │     │ └─ExchangeSender_51              | 1.33    | batchCop[tiflash] |                          | ExchangeType: Broadcast                                                                                                                                                                                                         |
|       │     │   └─Selection_50                 | 1.33    | batchCop[tiflash] |                          | lt(agg_sub_datat10000.ff.col_varchar_binary, "r"), not(isnull(agg_sub_datat10000.ff.col_decimal_30_10))                                                                                                                         |
|       │     │     └─TableFullScan_49           | 4.00    | batchCop[tiflash] | table:table1             | keep order:false, stats:pseudo                                                                                                                                                                                                  |
|       │     └─Selection_54(Probe)              | 651.00  | batchCop[tiflash] |                          | not(isnull(agg_sub_datat10000.b.col_int_key)), not(isnull(agg_sub_datat10000.b.col_varchar_binary_key))                                                                                                                         |
|       │       └─TableFullScan_53               | 1000.00 | batchCop[tiflash] | table:table2             | keep order:false                                                                                                                                                                                                                |
|       └─Projection_41(Probe)                   | 41.56   | batchCop[tiflash] |                          | agg_sub_datat10000.s.col_decimal                                                                                                                                                                                                |
|         └─HashAgg_35                           | 41.56   | batchCop[tiflash] |                          | group by:agg_sub_datat10000.s.col_decimal, funcs:firstrow(agg_sub_datat10000.s.col_decimal)->agg_sub_datat10000.s.col_decimal                                                                                                   |
|           └─ExchangeReceiver_40                | 51.95   | batchCop[tiflash] |                          |                                                                                                                                                                                                                                 |
|             └─ExchangeSender_39                | 51.95   | batchCop[tiflash] |                          | ExchangeType: HashPartition, Hash Cols: agg_sub_datat10000.s.col_decimal                                                                                                                                                        |
|               └─Selection_38                   | 51.95   | batchCop[tiflash] |                          | not(isnull(agg_sub_datat10000.s.col_decimal))                                                                                                                                                                                   |
|                 └─TableFullScan_37             | 52.00   | batchCop[tiflash] | table:SUBQUERY2_t1       | keep order:false, stats:pseudo                                                                                                                                                                                                  |
+------------------------------------------------+---------+-------------------+--------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

[2021/04/01 17:52:53.532 +08:00] [ERROR] [misc.go:90] ["panic in the recoverable goroutine"] [r={}] ["stack trace"="github.com/pingcap/tidb/util.WithRecovery.func1\n\t/home/jenkins/agent/workspace/tidb_master/go/src/github.com/pingcap/tidb/util/misc.go:92\nruntime.gopanic\n\t/usr/local/go/src/runtime/panic.go:679\nruntime.goPanicIndex\n\t/usr/local/go/src/runtime/panic.go:75\ngithub.com/pingcap/tidb/util/chunk.(*Column).GetString\n\t/home/jenkins/agent/workspace/tidb_master/go/src/github.com/pingcap/tidb/util/chunk/column.go:528\ngithub.com/pingcap/tidb/util/chunk.Row.GetString\n\t/home/jenkins/agent/workspace/tidb_master/go/src/github.com/pingcap/tidb/util/chunk/row.go:72\ngithub.com/pingcap/tidb/expression.(*Column).EvalString\n\t/home/jenkins/agent/workspace/tidb_master/go/src/github.com/pingcap/tidb/expression/column.go:397\ngithub.com/pingcap/tidb/expression.CompareStringWithCollationInfo\n\t/home/jenkins/agent/workspace/tidb_master/go/src/github.com/pingcap/tidb/expression/builtin_compare.go:2657\ngithub.com/pingcap/tidb/expression.(*builtinEQStringSig).evalInt\n\t/home/jenkins/agent/workspace/tidb_master/go/src/github.com/pingcap/tidb/expression/builtin_compare.go:2135\ngithub.com/pingcap/tidb/expression.(*ScalarFunction).EvalInt\n\t/home/jenkins/agent/workspace/tidb_master/go/src/github.com/pingcap/tidb/expression/scalar_function.go:378\ngithub.com/pingcap/tidb/expression.(*ScalarFunction).Eval\n\t/home/jenkins/agent/workspace/tidb_master/go/src/github.com/pingcap/tidb/expression/scalar_function.go:345\ngithub.com/pingcap/tidb/expression.EvalBool\n\t/home/jenkins/agent/workspace/tidb_master/go/src/github.com/pingcap/tidb/expression/expression.go:235\ngithub.com/pingcap/tidb/executor.(*leftOuterSemiJoiner).tryToMatchInners\n\t/home/jenkins/agent/workspace/tidb_master/go/src/github.com/pingcap/tidb/executor/joiner.go:519\ngithub.com/pingcap/tidb/executor.(*HashJoinExec).joinMatchedProbeSideRow2Chunk\n\t/home/jenkins/agent/workspace/tidb_master/go/src/github.com/pingcap/tidb/executor/join.go:527\ngithub.com/pingcap/tidb/executor.(*HashJoinExec).join2Chunk\n\t/home/jenkins/agent/workspace/tidb_master/go/src/github.com/pingcap/tidb/executor/join.go:596\ngithub.com/pingcap/tidb/executor.(*HashJoinExec).runJoinWorker\n\t/home/jenkins/agent/workspace/tidb_master/go/src/github.com/pingcap/tidb/executor/join.go:457\ngithub.com/pingcap/tidb/executor.(*HashJoinExec).fetchAndProbeHashTable.func2\n\t/home/jenkins/agent/workspace/tidb_master/go/src/github.com/pingcap/tidb/executor/join.go:339\ngithub.com/pingcap/tidb/util.WithRecovery\n\t/home/jenkins/agent/workspace/tidb_master/go/src/github.com/pingcap/tidb/util/misc.go:95"]
[2021/04/01 17:52:53.533 +08:00] [INFO] [conn.go:812] ["command dispatched failed"] [conn=7873] [connInfo="id:7873, addr:127.0.0.1:35408 status:10, collation:utf8_general_ci, user:root"] [command=Query] [status="inTxn:0, autocommit:1"] [sql="SELECT  ( 'g', 'p' )  IN ( SELECT   SUBQUERY1_t1 . `col_varchar_binary` AS SUBQUERY1_field1 , MAX(  SUBQUERY1_t1 . `col_varchar_binary_not_null_key` ) AS SUBQUERY1_field2 FROM ( `X` AS SUBQUERY1_t1 RIGHT  JOIN `V` AS SUBQUERY1_t2 ON (SUBQUERY1_t2 . `col_decimal_30_10` = SUBQUERY1_t1 . `pk`  ) ) WHERE  ( SUBQUERY1_t1 . `col_int` , SUBQUERY1_t2 . `col_decimal` ) NOT IN ( SELECT   CHILD_SUBQUERY1_t1 . `col_int_key` AS CHILD_SUBQUERY1_field1 , CHILD_SUBQUERY1_t1 . `col_decimal_not_null_key` AS child_subquery1_field2 FROM `D` AS CHILD_SUBQUERY1_t1    ) GROUP BY SUBQUERY1_field1  ) AS field1 FROM ( `FF` AS table1 STRAIGHT_JOIN `B` AS table2 ON (table2 . `col_varchar_binary_key` = table1 . `col_varchar_binary_not_null`  ) ) WHERE (  table1 . `col_decimal_30_10`  IN ( SELECT DISTINCT  SUBQUERY2_t1 . `col_decimal` AS SUBQUERY2_field1 FROM `S` AS SUBQUERY2_t1    ) ) AND ( table1 . `col_int_not_null_key` = table2 . `col_int_key` AND table1 . `col_varchar_binary` < 'r' )"] [txn_mode=PESSIMISTIC] [err="runtime error: index out of range [0] with length 0\ngithub.com/pingcap/tidb/executor.(*HashJoinExec).handleJoinWorkerPanic\n\t/home/jenkins/agent/workspace/tidb_master/go/src/github.com/pingcap/tidb/executor/join.go:357\ngithub.com/pingcap/tidb/util.WithRecovery.func1\n\t/home/jenkins/agent/workspace/tidb_master/go/src/github.com/pingcap/tidb/util/misc.go:87\nruntime.gopanic\n\t/usr/local/go/src/runtime/panic.go:679\nruntime.goPanicIndex\n\t/usr/local/go/src/runtime/panic.go:75\ngithub.com/pingcap/tidb/util/chunk.(*Column).GetString\n\t/home/jenkins/agent/workspace/tidb_master/go/src/github.com/pingcap/tidb/util/chunk/column.go:528\ngithub.com/pingcap/tidb/util/chunk.Row.GetString\n\t/home/jenkins/agent/workspace/tidb_master/go/src/github.com/pingcap/tidb/util/chunk/row.go:72\ngithub.com/pingcap/tidb/expression.(*Column).EvalString\n\t/home/jenkins/agent/workspace/tidb_master/go/src/github.com/pingcap/tidb/expression/column.go:397\ngithub.com/pingcap/tidb/expression.CompareStringWithCollationInfo\n\t/home/jenkins/agent/workspace/tidb_master/go/src/github.com/pingcap/tidb/expression/builtin_compare.go:2657\ngithub.com/pingcap/tidb/expression.(*builtinEQStringSig).evalInt\n\t/home/jenkins/agent/workspace/tidb_master/go/src/github.com/pingcap/tidb/expression/builtin_compare.go:2135\ngithub.com/pingcap/tidb/expression.(*ScalarFunction).EvalInt\n\t/home/jenkins/agent/workspace/tidb_master/go/src/github.com/pingcap/tidb/expression/scalar_function.go:378\ngithub.com/pingcap/tidb/expression.(*ScalarFunction).Eval\n\t/home/jenkins/agent/workspace/tidb_master/go/src/github.com/pingcap/tidb/expression/scalar_function.go:345\ngithub.com/pingcap/tidb/expression.EvalBool\n\t/home/jenkins/agent/workspace/tidb_master/go/src/github.com/pingcap/tidb/expression/expression.go:235\ngithub.com/pingcap/tidb/executor.(*leftOuterSemiJoiner).tryToMatchInners\n\t/home/jenkins/agent/workspace/tidb_master/go/src/github.com/pingcap/tidb/executor/joiner.go:519\ngithub.com/pingcap/tidb/executor.(*HashJoinExec).joinMatchedProbeSideRow2Chunk\n\t/home/jenkins/agent/workspace/tidb_master/go/src/github.com/pingcap/tidb/executor/join.go:527\ngithub.com/pingcap/tidb/executor.(*HashJoinExec).join2Chunk\n\t/home/jenkins/agent/workspace/tidb_master/go/src/github.com/pingcap/tidb/executor/join.go:596\ngithub.com/pingcap/tidb/executor.(*HashJoinExec).runJoinWorker\n\t/home/jenkins/agent/workspace/tidb_master/go/src/github.com/pingcap/tidb/executor/join.go:457\ngithub.com/pingcap/tidb/executor.(*HashJoinExec).fetchAndProbeHashTable.func2\n\t/home/jenkins/agent/workspace/tidb_master/go/src/github.com/pingcap/tidb/executor/join.go:339\ngithub.com/pingcap/tidb/util.WithRecovery\n\t/home/jenkins/agent/workspace/tidb_master/go/src/github.com/pingcap/tidb/util/misc.go:95\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1357"]

The text was updated successfully, but these errors were encountered:

hanfei1991 · 2021-04-01T13:46:28Z

Here I reproduce this case in a smaller stmt:

mysql> explain SELECT  ('g') IN (select t1. `col_varchar_binary` from `X` as t1 ) FROM ( `FF` AS table1 STRAIGHT_JOIN `B` AS table2 ON (table2 . `col_varchar_binary_key` = table1 . `col_varchar_binary_not_null`  ) ) WHERE (  table1 . `
col_decimal_30_10`  IN ( SELECT DISTINCT  SUBQUERY2_t1 . `col_decimal` AS SUBQUERY2_field1 FROM `S` AS SUBQUERY2_t1    ) ) AND ( table1 . `col_int_not_null_key` = table2 . `col_int_key` AND table1 . `col_varchar_binary` < 'r' )   ;
+------------------------------------------------+---------+-------------------+--------------------+--------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------+
| id                                             | estRows | task              | access object      | operator info
                                                                   |
+------------------------------------------------+---------+-------------------+--------------------+--------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------+
| HashJoin_20                                    | 4.04    | root              |                    | CARTESIAN left outer semi join, other cond:eq("g", agg_sub_datat10000.x.col_varchar_binary)
                                                                   |
| ├─TableReader_85(Build)                        | 13.00   | root              |                    | data:TableFullScan_84
                                                                   |
| │ └─TableFullScan_84                           | 13.00   | cop[tiflash]      | table:t1           | keep order:false, stats:pseudo
                                                                   |
| └─TableReader_48(Probe)                        | 4.04    | root              |                    | data:ExchangeSender_47
                                                                   |
|   └─ExchangeSender_47                          | 4.04    | batchCop[tiflash] |                    | ExchangeType: PassThrough
                                                                   |
|     └─HashJoin_22                              | 4.04    | batchCop[tiflash] |                    | inner join, equal:[eq(agg_sub_datat10000.s.col_decimal, agg_sub_datat10000.ff.col_decimal_30_10)]
                                                                   |
|       ├─ExchangeReceiver_46(Build)             | 4.04    | batchCop[tiflash] |                    |
                                                                   |
|       │ └─ExchangeSender_45                    | 4.04    | batchCop[tiflash] |                    | ExchangeType: Broadcast
                                                                   |
|       │   └─HashJoin_38                        | 4.04    | batchCop[tiflash] |                    | inner join, equal:[eq(agg_sub_datat10000.ff.col_varchar_binary_not_null, agg_sub_datat10000.b.col_varchar_binary_key) eq(agg_sub_data
t10000.ff.col_int_not_null_key, agg_sub_datat10000.b.col_int_key)] |
|       │     ├─ExchangeReceiver_42(Build)       | 1.33    | batchCop[tiflash] |                    |
                                                                   |
|       │     │ └─ExchangeSender_41              | 1.33    | batchCop[tiflash] |                    | ExchangeType: Broadcast
                                                                   |
|       │     │   └─Selection_40                 | 1.33    | batchCop[tiflash] |                    | lt(agg_sub_datat10000.ff.col_varchar_binary, "r"), not(isnull(agg_sub_datat10000.ff.col_decimal_30_10))
                                                                   |
|       │     │     └─TableFullScan_39           | 4.00    | batchCop[tiflash] | table:table1       | keep order:false, stats:pseudo
                                                                   |
|       │     └─Selection_44(Probe)              | 651.00  | batchCop[tiflash] |                    | not(isnull(agg_sub_datat10000.b.col_int_key)), not(isnull(agg_sub_datat10000.b.col_varchar_binary_key))
                                                                   |
|       │       └─TableFullScan_43               | 1000.00 | batchCop[tiflash] | table:table2       | keep order:false
                                                                   |
|       └─Projection_31(Probe)                   | 41.56   | batchCop[tiflash] |                    | agg_sub_datat10000.s.col_decimal
                                                                   |
|         └─HashAgg_25                           | 41.56   | batchCop[tiflash] |                    | group by:agg_sub_datat10000.s.col_decimal, funcs:firstrow(agg_sub_datat10000.s.col_decimal)->agg_sub_datat10000.s.col_decimal
                                                                   |
|           └─ExchangeReceiver_30                | 51.95   | batchCop[tiflash] |                    |
                                                                   |
|             └─ExchangeSender_29                | 51.95   | batchCop[tiflash] |                    | ExchangeType: HashPartition, Hash Cols: agg_sub_datat10000.s.col_decimal
                                                                   |
|               └─Selection_28                   | 51.95   | batchCop[tiflash] |                    | not(isnull(agg_sub_datat10000.s.col_decimal))
                                                                   |
|                 └─TableFullScan_27             | 52.00   | batchCop[tiflash] | table:SUBQUERY2_t1 | keep order:false, stats:pseudo
                                                                   |
+------------------------------------------------+---------+-------------------+--------------------+--------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------+

hanfei1991 · 2021-04-01T13:51:55Z

Note that for hash join 20, it doesn't refer any columns on the left side. Then the column prunner cut all the columns of its left child, which orignally is a projection. After Logical Optimize, the left child of Hash join 20 is project 21 with no columns and the child of project 21 is Hash Join 22 with an empty schema. Like this:

Hash Join 20 -------
         |                          |
Project 21              table scan 84
         |
Hash Join 22

But note that hash join 22 actually needs two columns as equal condition, so the children of hash join 22 has non-empty schema.

hanfei1991 · 2021-04-01T14:02:37Z

But Join doesn't pass it's schema to TiFlash, so TiFlash must assume that the join's schema is strictly equal to the combined schema of its children. The remake of Schema is here: https://github.com/pingcap/tidb/blob/master/planner/core/task.go#L734

Now we have Hash Join 22 with non-empty schema and Project 21 with empty schema.

hanfei1991 · 2021-04-01T14:06:04Z

Then, resolve the index of hash join 20's other condition, we get index 0. But in the optimization in rule_eliminate, we remove all the empty project. The reason is here pingcap/tidb#6318 . These two work-around makes the schema length of the left child of hash join 20 changes after resolving index, and causes this bug.

hanfei1991 · 2021-04-16T10:50:44Z

fixed by pingcap/tidb#24024

fzhedu added the type/bug The issue is confirmed as a bug. label Apr 1, 2021

fzhedu changed the title ~~tidb panic if enable mpp~~ tidb panic if enable mpp in a corner case Apr 1, 2021

zanmato1984 added the severity/major label Apr 1, 2021

hanfei1991 changed the title ~~tidb panic if enable mpp in a corner case~~ tidb reports error if there is a leftsemijoin on the top but refers no columns on the tiflash side on which runs a mpp join Apr 1, 2021

hanfei1991 changed the title ~~tidb reports error if there is a leftsemijoin on the top but refers no columns on the tiflash side on which runs a mpp join~~ tidb reports error if there is a leftsemijoin on the top but refers no columns on probe side on which runs a mpp join Apr 1, 2021

zanmato1984 added this to the v5.0.1 milestone Apr 13, 2021

zanmato1984 added priority/release-blocker This issue blocks a release. Please solve it ASAP. and removed priority/release-blocker This issue blocks a release. Please solve it ASAP. labels Apr 13, 2021

zanmato1984 removed this from the v5.0.1 milestone Apr 13, 2021

guo-shaoge mentioned this issue Apr 15, 2021

executor: (*Column).GetString panic pingcap/tidb#23887

Closed

hanfei1991 closed this as completed Apr 16, 2021

fzhedu added the plan bugs related to plans label Nov 15, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

tidb reports error if there is a leftsemijoin on the top but refers no columns on probe side on which runs a mpp join #1718

tidb reports error if there is a leftsemijoin on the top but refers no columns on probe side on which runs a mpp join #1718

fzhedu commented Apr 1, 2021 •

edited

Loading

hanfei1991 commented Apr 1, 2021

hanfei1991 commented Apr 1, 2021 •

edited

Loading

hanfei1991 commented Apr 1, 2021

hanfei1991 commented Apr 1, 2021

hanfei1991 commented Apr 16, 2021

tidb reports error if there is a leftsemijoin on the top but refers no columns on probe side on which runs a mpp join #1718

tidb reports error if there is a leftsemijoin on the top but refers no columns on probe side on which runs a mpp join #1718

Comments

fzhedu commented Apr 1, 2021 • edited Loading

hanfei1991 commented Apr 1, 2021

hanfei1991 commented Apr 1, 2021 • edited Loading

hanfei1991 commented Apr 1, 2021

hanfei1991 commented Apr 1, 2021

hanfei1991 commented Apr 16, 2021

fzhedu commented Apr 1, 2021 •

edited

Loading

hanfei1991 commented Apr 1, 2021 •

edited

Loading