When add partition on partition table with no unique key or primary key. The cdc will stop dml replication about the partition table without errors and warnings.

[erwadba]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. What did you do? If possible, provide a recipe for reproducing the error.
   About cdc

# cdc cli changefeed create --pd=http://xx.xx.xx.xx:2379 --sink-uri="mysql://user:[email protected]:3306/?worker-count=2&max-txn-row=32" --changefeed-id="xxxx" --sort-engine="unified"  --config=changefeed_mysql.toml
# cat changefeed_mysql.toml
case-sensitive = true
enable-old-value = true
force-replicate = true

[filter]
rules = ['xx.*']

reproducing the error step:

1.create partition table with no unique key or primary key.
use xx;
create table a(
	id int,
	LogTime datetime,
	key idx (id)
)PARTITION BY RANGE ( TO_DAYS(`LogTime`) ) (
	PARTITION `p202101` VALUES LESS THAN (738187),
  PARTITION `p202102` VALUES LESS THAN (738215)
);
2.add partition
alter table a add PARTITION (  PARTITION `p20210401` VALUES LESS THAN (738246));

3.insert values on the table. The value will not replicate to the target.
MySQL [xx]> insert into a values(1,'2021-1-1');
Query OK, 1 row affected (0.03 sec)
MySQL [xx]> insert into a values(1,'2021-2-1');
Query OK, 1 row affected (0.01 sec)
MySQL [xx]> 

2. What did you expect to see?
   report some warnings or errors.

3. What did you see instead?
   report nothing.

4. Versions of the cluster

   • Upstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):

     09:14:46 test> SELECT tidb_version()；
     tidb_version(): Release Version: v5.2.0
     Edition: Community
     Git Commit Hash: 05d2210647d6a1503a8d772477e43b14a024f609
     Git Branch: heads/refs/tags/v5.2.0
     UTC Build Time: 2021-08-27 05:57:10
     GoVersion: go1.16.4
     Race Enabled: false
     TiKV Min Version: v3.0.0-60965b006877ca7234adaced7890d7b029ed1306
     Check Table Before Drop: false
     1 row in set (0.00 sec)
     09:14:50 test>
     

   • TiCDC version (execute cdc version):

     Release Version: v5.2.0
     Git Commit Hash: 81c22b1c1b2041e2806160d8c7e1105a70815ff5
     Git Branch: heads/refs/tags/v5.2.0
     UTC Build Time: 2021-08-26 08:37:01
     Go Version: go version go1.16.4 linux/amd64
     Failpoint Build: false
     

[amyangfei]

TiCDC doesn't support to replicate table without valid index (table without primary key or unique key),
ref: https://docs.pingcap.com/tidb/stable/ticdc-overview#restrictions
I found force-replicate is enabled, the above comment is incorrect

[erwadba]

When force-replicate is enabled. Add partition on partition table without valid index. The cdc will stop dml replication about the partition table without errors and warnings.
The problem maybe here L469-L471. Maybe it should change the if code like L510-L512 in schema_storage.go:
https://github.com/pingcap/ticdc/blob/a7bb6c47cfba8996980b8d29bb156ae9225c7a3f/cdc/entry/schema_storage.go#L469-L471

https://github.com/pingcap/ticdc/blob/a7bb6c47cfba8996980b8d29bb156ae9225c7a3f/cdc/entry/schema_storage.go#L510-L512

[asddongmen]

When force-replicate is enabled. Add partition on partition table without valid index. The cdc will stop dml replication about the partition table without errors and warnings.
The problem maybe here L469-L471. Maybe it should change the if code like L510-L512 in schema_storage.go:
https://github.com/pingcap/ticdc/blob/a7bb6c47cfba8996980b8d29bb156ae9225c7a3f/cdc/entry/schema_storage.go#L469-L471

https://github.com/pingcap/ticdc/blob/a7bb6c47cfba8996980b8d29bb156ae9225c7a3f/cdc/entry/schema_storage.go#L510-L512

@erwadba Awesome！
If you want, you can pull a request to help us fix this bug.
Then we can add integration tests for your PR as co-authors.（or may be you can add tests on your own）

[amyangfei]

Task list

• Bugfix *: fix dml not replicate when add partition about partition table without valid index #2841
• Add a unit test

[amyangfei]

closed by #2841