planner: column sizes under UniStore are not correct

[qw4990]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

A minimal reproducible case for this:

create table tx (a int);
insert into tx values (1);
analyze table tx;
explain select * from tx where a=1; -- use the column a to let it can be loaded into memory
show stats_histograms where table_name="tx";

And the results are different between Unistore and TiKV:

mysql> show stats_histograms where table_name="tx"; -- Unistore
+---------+------------+----------------+-------------+----------+---------------------+----------------+------------+--------------+-------------+
| Db_name | Table_name | Partition_name | Column_name | Is_index | Update_time         | Distinct_count | Null_count | Avg_col_size | Correlation |
+---------+------------+----------------+-------------+----------+---------------------+----------------+------------+--------------+-------------+
| test    | tx         |                | a           |        0 | 2022-01-17 17:25:24 |              1 |          0 |            1 |           1 |
+---------+------------+----------------+-------------+----------+---------------------+----------------+------------+--------------+-------------+

mysql> show stats_histograms where table_name="tx"; -- In TiKV
+---------+------------+----------------+-------------+----------+---------------------+----------------+------------+--------------+-------------+
| Db_name | Table_name | Partition_name | Column_name | Is_index | Update_time         | Distinct_count | Null_count | Avg_col_size | Correlation |
+---------+------------+----------------+-------------+----------+---------------------+----------------+------------+--------------+-------------+
| test    | tx         |                | a           |        0 | 2022-01-17 17:26:31 |              1 |          0 |            9 |           1 |
+---------+------------+----------------+-------------+----------+---------------------+----------------+------------+--------------+-------------+

2. What did you expect to see? (Required)

The column size under UniStore should be 9.

3. What did you see instead (Required)

It's 1.

4. What is your TiDB version? (Required)

+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| tidb_version()                                                                                                                                                                                                                                                                                                                  |
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Release Version: v5.5.0-alpha-122-g8c395318a-dirty
Edition: Community
Git Commit Hash: 8c395318ae1c4b39e7eb601c40f8955e4e5b6ff1
Git Branch: master
UTC Build Time: 2022-01-17 10:32:27
GoVersion: go1.16.3
Race Enabled: false
TiKV Min Version: v3.0.0-60965b006877ca7234adaced7890d7b029ed1306
Check Table Before Drop: false |
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

[github-actions]

Please check whether the issue should be labeled with 'affects-x.y' or 'fixes-x.y.z', and then remove 'needs-more-info' label.

[xuyifangreeneyes]

@djshow832 @qw4990 It seems that merging https://github.com/pingcap/tidb-test/pull/1512 closed the issue. We should reopen it.

[xuyifangreeneyes]

/assign

[xuyifangreeneyes]

There are two problems which cause the inconsistency.

1. In TiKV side, when adding column size to total_size, we need to minus 1 for the flag byte but we miss that. Hence in TiKV avg_col_size is 1 + 8 = 9.
2. In unistore side, we call tablecodec.EncodeValue when scanning table, which encodes integers with dynamic size(1,2,4,8 byte). Hence in unistore avg_col_size is 1.

avg_col_size is used to calculate disk scan cost, network cost and some operator cost(such as join, agg). Integers in disk are dynamic-size(1,2,4,8 byte) while integers in memory and network are fixed-size(8 byte)(for point-get integers in network are also dynamic-size, which is a special case). Since for now we only have one avg_col_size, we decide to regard integers' avg_col_size as 8 to keep it simple.

[xuyifangreeneyes]

/reopen

[ti-chi-bot]

@xuyifangreeneyes: Reopened this issue.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[xuyifangreeneyes]

We only solve half of the problem last time so I reopen it. Though the problem is not very serve, it brings trouble when you want to write some explain tests for both tidb on unistore and tidb on tikv. cc @zimulala