Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Kafka many_pk_or_uk fails #4895

Closed
Tracked by #2246
overvenus opened this issue Mar 15, 2022 · 3 comments
Closed
Tracked by #2246

Kafka many_pk_or_uk fails #4895

overvenus opened this issue Mar 15, 2022 · 3 comments
Assignees
@3AceShowHand
Labels
area/ticdc Issues or PRs related to TiCDC. component/test Unit tests and integration tests component. type/enhancement The issue or PR belongs to an enhancement.

Comments

@overvenus
Copy link
Member

overvenus commented Mar 15, 2022 •
edited
Loading

Which jobs are flaking?

kafka-integration-test

Which test(s) are flaking?

many_pk_or_uk

Jenkins logs or GitHub Actions link

https://ci2.pingcap.net/blue/organizations/jenkins/cdc_ghpr_kafka_integration_test/detail/cdc_ghpr_kafka_integration_test/9780/pipeline#step-1266-log-310

log-many_pk_or_uk-cyclic_ab.tar.gz

Anything else we need to know
@overvenus overvenus added component/test Unit tests and integration tests component. area/ticdc Issues or PRs related to TiCDC. labels Mar 15, 2022
@overvenus overvenus mentioned this issue Mar 15, 2022
Closed
@3AceShowHand 3AceShowHand self-assigned this Mar 17, 2022
@3AceShowHand 3AceShowHand added the type/enhancement The issue or PR belongs to an enhancement. label Mar 17, 2022
@3AceShowHand
Copy link
Contributor 

[2022/03/15 14:30:52.458 +08:00] [INFO] [main.go:542] ["ddl event received"] [ddl="{\"StartTs\":0,\"CommitTs\":431836587989925893,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"pk_or_uk_5_0\",\"TableID\":0,\"ColumnInfo\":null},\"PreTableInfo\":null,\"Query\":\"CREATE TABLE `pk_or_uk_5_0` (`id` SMALLINT UNIQUE KEY NOT NULL)\",\"Type\":3}"]
[2022/03/15 14:30:52.459 +08:00] [INFO] [main.go:530] ["ignore ddl event"] [maxDDLReceivedTs=431836587989925893] [ddl="{\"StartTs\":0,\"CommitTs\":431836587989925893,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"pk_or_uk_5_0\",\"TableID\":0,\"ColumnInfo\":null},\"PreTableInfo\":null,\"Query\":\"CREATE TABLE `pk_or_uk_5_0` (`id` SMALLINT UNIQUE KEY NOT NULL)\",\"Type\":3}"]
[2022/03/15 14:30:52.460 +08:00] [INFO] [main.go:530] ["ignore ddl event"] [maxDDLReceivedTs=431836587989925893] [ddl="{\"StartTs\":0,\"CommitTs\":431836587989925893,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"pk_or_uk_5_0\",\"TableID\":0,\"ColumnInfo\":null},\"PreTableInfo\":null,\"Query\":\"CREATE TABLE `pk_or_uk_5_0` (`id` SMALLINT UNIQUE KEY NOT NULL)\",\"Type\":3}"]
[2022/03/15 14:30:52.461 +08:00] [INFO] [main.go:530] ["ignore ddl event"] [maxDDLReceivedTs=431836587989925893] [ddl="{\"StartTs\":0,\"CommitTs\":431836587989925893,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"pk_or_uk_5_0\",\"TableID\":0,\"ColumnInfo\":null},\"PreTableInfo\":null,\"Query\":\"CREATE TABLE `pk_or_uk_5_0` (`id` SMALLINT UNIQUE KEY NOT NULL)\",\"Type\":3}"]
[2022/03/15 14:30:52.543 +08:00] [INFO] [mysql.go:332] ["start exec DDL"] [DDL="{\"StartTs\":0,\"CommitTs\":431836587989925893,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"pk_or_uk_5_0\",\"TableID\":0,\"ColumnInfo\":null},\"PreTableInfo\":null,\"Query\":\"CREATE TABLE `pk_or_uk_5_0` (`id` SMALLINT UNIQUE KEY NOT NULL)\",\"Type\":3}"]
[2022/03/15 14:30:52.634 +08:00] [INFO] [mysql.go:362] ["Exec DDL succeeded"] [sql="CREATE TABLE `pk_or_uk_5_0` (`id` SMALLINT UNIQUE KEY NOT NULL)"]
[2022/03/15 14:30:52.643 +08:00] [INFO] [main.go:632] ["update globalResolvedTs"] [ts=431836587989925893]

[2022/03/15 14:30:56.056 +08:00] [INFO] [main.go:542] ["ddl event received"] [ddl="{\"StartTs\":0,\"CommitTs\":431836588055199749,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"pk_or_uk_2_1\",\"TableID\":0,\"ColumnInfo\":null},\"PreTableInfo\":null,\"Query\":\"CREATE TABLE `pk_or_uk_2_1` (`id` INT UNSIGNED PRIMARY KEY)\",\"Type\":3}"]
[2022/03/15 14:30:56.057 +08:00] [INFO] [main.go:530] ["ignore ddl event"] [maxDDLReceivedTs=431836588055199749] [ddl="{\"StartTs\":0,\"CommitTs\":431836588055199749,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"pk_or_uk_2_1\",\"TableID\":0,\"ColumnInfo\":null},\"PreTableInfo\":null,\"Query\":\"CREATE TABLE `pk_or_uk_2_1` (`id` INT UNSIGNED PRIMARY KEY)\",\"Type\":3}"]
[2022/03/15 14:30:56.057 +08:00] [INFO] [main.go:530] ["ignore ddl event"] [maxDDLReceivedTs=431836588055199749] [ddl="{\"StartTs\":0,\"CommitTs\":431836588055199749,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"pk_or_uk_2_1\",\"TableID\":0,\"ColumnInfo\":null},\"PreTableInfo\":null,\"Query\":\"CREATE TABLE `pk_or_uk_2_1` (`id` INT UNSIGNED PRIMARY KEY)\",\"Type\":3}"]
[2022/03/15 14:30:56.059 +08:00] [INFO] [main.go:530] ["ignore ddl event"] [maxDDLReceivedTs=431836588055199749] [ddl="{\"StartTs\":0,\"CommitTs\":431836588055199749,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"pk_or_uk_2_1\",\"TableID\":0,\"ColumnInfo\":null},\"PreTableInfo\":null,\"Query\":\"CREATE TABLE `pk_or_uk_2_1` (`id` INT UNSIGNED PRIMARY KEY)\",\"Type\":3}"]
[2022/03/15 14:30:56.557 +08:00] [INFO] [mysql.go:332] ["start exec DDL"] [DDL="{\"StartTs\":0,\"CommitTs\":431836588055199749,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"pk_or_uk_2_1\",\"TableID\":0,\"ColumnInfo\":null},\"PreTableInfo\":null,\"Query\":\"CREATE TABLE `pk_or_uk_2_1` (`id` INT UNSIGNED PRIMARY KEY)\",\"Type\":3}"]
[2022/03/15 14:30:56.629 +08:00] [INFO] [mysql.go:362] ["Exec DDL succeeded"] [sql="CREATE TABLE `pk_or_uk_2_1` (`id` INT UNSIGNED PRIMARY KEY)"]
[2022/03/15 14:30:56.643 +08:00] [INFO] [main.go:632] ["update globalResolvedTs"] [ts=431836588055199749]

[2022/03/15 14:32:34.477 +08:00] [INFO] [main.go:542] ["ddl event received"] [ddl="{\"StartTs\":0,\"CommitTs\":431836588068569097,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"pk_or_uk_4_1\",\"TableID\":0,\"ColumnInfo\":null},\"PreTableInfo\":null,\"Query\":\"CREATE TABLE `pk_or_uk_4_1` (`id` SMALLINT UNSIGNED PRIMARY KEY)\",\"Type\":3}"]
[2022/03/15 14:32:34.485 +08:00] [INFO] [main.go:530] ["ignore ddl event"] [maxDDLReceivedTs=431836588068569097] [ddl="{\"StartTs\":0,\"CommitTs\":431836588068569097,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"pk_or_uk_4_1\",\"TableID\":0,\"ColumnInfo\":null},\"PreTableInfo\":null,\"Query\":\"CREATE TABLE `pk_or_uk_4_1` (`id` SMALLINT UNSIGNED PRIMARY KEY)\",\"Type\":3}"]
[2022/03/15 14:32:34.485 +08:00] [INFO] [main.go:530] ["ignore ddl event"] [maxDDLReceivedTs=431836588068569097] [ddl="{\"StartTs\":0,\"CommitTs\":431836588068569097,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"pk_or_uk_4_1\",\"TableID\":0,\"ColumnInfo\":null},\"PreTableInfo\":null,\"Query\":\"CREATE TABLE `pk_or_uk_4_1` (`id` SMALLINT UNSIGNED PRIMARY KEY)\",\"Type\":3}"]
[2022/03/15 14:32:34.485 +08:00] [INFO] [main.go:530] ["ignore ddl event"] [maxDDLReceivedTs=431836588068569097] [ddl="{\"StartTs\":0,\"CommitTs\":431836588068569097,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"pk_or_uk_4_1\",\"TableID\":0,\"ColumnInfo\":null},\"PreTableInfo\":null,\"Query\":\"CREATE TABLE `pk_or_uk_4_1` (`id` SMALLINT UNSIGNED PRIMARY KEY)\",\"Type\":3}"]
[2022/03/15 14:32:34.643 +08:00] [INFO] [mysql.go:332] ["start exec DDL"] [DDL="{\"StartTs\":0,\"CommitTs\":431836588068569097,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"pk_or_uk_4_1\",\"TableID\":0,\"ColumnInfo\":null},\"PreTableInfo\":null,\"Query\":\"CREATE TABLE `pk_or_uk_4_1` (`id` SMALLINT UNSIGNED PRIMARY KEY)\",\"Type\":3}"]
[2022/03/15 14:32:34.760 +08:00] [INFO] [mysql.go:362] ["Exec DDL succeeded"] [sql="CREATE TABLE `pk_or_uk_4_1` (`id` SMALLINT UNSIGNED PRIMARY KEY)"]
[2022/03/15 14:32:34.760 +08:00] [INFO] [main.go:632] ["update globalResolvedTs"] [ts=431836588068569097]

[2022/03/15 14:32:37.865 +08:00] [INFO] [main.go:542] ["ddl event received"] [ddl="{\"StartTs\":0,\"CommitTs\":431836588081938442,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"pk_or_uk_4_0\",\"TableID\":0,\"ColumnInfo\":null},\"PreTableInfo\":null,\"Query\":\"CREATE TABLE `pk_or_uk_4_0` (`id` SMALLINT UNSIGNED UNIQUE KEY NOT NULL)\",\"Type\":3}"]
[2022/03/15 14:32:37.870 +08:00] [INFO] [main.go:530] ["ignore ddl event"] [maxDDLReceivedTs=431836588081938442] [ddl="{\"StartTs\":0,\"CommitTs\":431836588081938442,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"pk_or_uk_4_0\",\"TableID\":0,\"ColumnInfo\":null},\"PreTableInfo\":null,\"Query\":\"CREATE TABLE `pk_or_uk_4_0` (`id` SMALLINT UNSIGNED UNIQUE KEY NOT NULL)\",\"Type\":3}"]
[2022/03/15 14:32:37.880 +08:00] [INFO] [main.go:530] ["ignore ddl event"] [maxDDLReceivedTs=431836588081938442] [ddl="{\"StartTs\":0,\"CommitTs\":431836588081938442,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"pk_or_uk_4_0\",\"TableID\":0,\"ColumnInfo\":null},\"PreTableInfo\":null,\"Query\":\"CREATE TABLE `pk_or_uk_4_0` (`id` SMALLINT UNSIGNED UNIQUE KEY NOT NULL)\",\"Type\":3}"]
[2022/03/15 14:32:37.881 +08:00] [INFO] [main.go:530] ["ignore ddl event"] [maxDDLReceivedTs=431836588081938442] [ddl="{\"StartTs\":0,\"CommitTs\":431836588081938442,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"pk_or_uk_4_0\",\"TableID\":0,\"ColumnInfo\":null},\"PreTableInfo\":null,\"Query\":\"CREATE TABLE `pk_or_uk_4_0` (`id` SMALLINT UNSIGNED UNIQUE KEY NOT NULL)\",\"Type\":3}"]
[2022/03/15 14:32:38.754 +08:00] [INFO] [mysql.go:332] ["start exec DDL"] [DDL="{\"StartTs\":0,\"CommitTs\":431836588081938442,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"pk_or_uk_4_0\",\"TableID\":0,\"ColumnInfo\":null},\"PreTableInfo\":null,\"Query\":\"CREATE TABLE `pk_or_uk_4_0` (`id` SMALLINT UNSIGNED UNIQUE KEY NOT NULL)\",\"Type\":3}"]
[2022/03/15 14:32:38.912 +08:00] [INFO] [mysql.go:362] ["Exec DDL succeeded"] [sql="CREATE TABLE `pk_or_uk_4_0` (`id` SMALLINT UNSIGNED UNIQUE KEY NOT NULL)"]
[2022/03/15 14:32:38.912 +08:00] [INFO] [main.go:632] ["update globalResolvedTs"] [ts=431836588081938442]

[2022/03/15 14:32:41.865 +08:00] [INFO] [main.go:542] ["ddl event received"] [ddl="{\"StartTs\":0,\"CommitTs\":431836588094521351,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"pk_or_uk_5_1\",\"TableID\":0,\"ColumnInfo\":null},\"PreTableInfo\":null,\"Query\":\"CREATE TABLE `pk_or_uk_5_1` (`id` SMALLINT PRIMARY KEY)\",\"Type\":3}"]
[2022/03/15 14:32:41.870 +08:00] [INFO] [main.go:530] ["ignore ddl event"] [maxDDLReceivedTs=431836588094521351] [ddl="{\"StartTs\":0,\"CommitTs\":431836588094521351,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"pk_or_uk_5_1\",\"TableID\":0,\"ColumnInfo\":null},\"PreTableInfo\":null,\"Query\":\"CREATE TABLE `pk_or_uk_5_1` (`id` SMALLINT PRIMARY KEY)\",\"Type\":3}"]
[2022/03/15 14:32:41.874 +08:00] [INFO] [main.go:530] ["ignore ddl event"] [maxDDLReceivedTs=431836588094521351] [ddl="{\"StartTs\":0,\"CommitTs\":431836588094521351,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"pk_or_uk_5_1\",\"TableID\":0,\"ColumnInfo\":null},\"PreTableInfo\":null,\"Query\":\"CREATE TABLE `pk_or_uk_5_1` (`id` SMALLINT PRIMARY KEY)\",\"Type\":3}"]
[2022/03/15 14:32:41.874 +08:00] [INFO] [main.go:530] ["ignore ddl event"] [maxDDLReceivedTs=431836588094521351] [ddl="{\"StartTs\":0,\"CommitTs\":431836588094521351,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"pk_or_uk_5_1\",\"TableID\":0,\"ColumnInfo\":null},\"PreTableInfo\":null,\"Query\":\"CREATE TABLE `pk_or_uk_5_1` (`id` SMALLINT PRIMARY KEY)\",\"Type\":3}"]
[2022/03/15 14:32:42.647 +08:00] [INFO] [mysql.go:332] ["start exec DDL"] [DDL="{\"StartTs\":0,\"CommitTs\":431836588094521351,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"pk_or_uk_5_1\",\"TableID\":0,\"ColumnInfo\":null},\"PreTableInfo\":null,\"Query\":\"CREATE TABLE `pk_or_uk_5_1` (`id` SMALLINT PRIMARY KEY)\",\"Type\":3}"]
[2022/03/15 14:32:42.770 +08:00] [INFO] [mysql.go:362] ["Exec DDL succeeded"] [sql="CREATE TABLE `pk_or_uk_5_1` (`id` SMALLINT PRIMARY KEY)"]
[2022/03/15 14:32:42.770 +08:00] [INFO] [main.go:632] ["update globalResolvedTs"] [ts=431836588094521351]

[2022/03/15 14:32:46.259 +08:00] [INFO] [main.go:542] ["ddl event received"] [ddl="{\"StartTs\":0,\"CommitTs\":431836588120735751,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"pk_or_uk_7_0\",\"TableID\":0,\"ColumnInfo\":null},\"PreTableInfo\":null,\"Query\":\"CREATE TABLE `pk_or_uk_7_0` (`id` TINYINT UNIQUE KEY NOT NULL)\",\"Type\":3}"]
[2022/03/15 14:32:46.260 +08:00] [INFO] [main.go:530] ["ignore ddl event"] [maxDDLReceivedTs=431836588120735751] [ddl="{\"StartTs\":0,\"CommitTs\":431836588120735751,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"pk_or_uk_7_0\",\"TableID\":0,\"ColumnInfo\":null},\"PreTableInfo\":null,\"Query\":\"CREATE TABLE `pk_or_uk_7_0` (`id` TINYINT UNIQUE KEY NOT NULL)\",\"Type\":3}"]
[2022/03/15 14:32:46.260 +08:00] [INFO] [main.go:530] ["ignore ddl event"] [maxDDLReceivedTs=431836588120735751] [ddl="{\"StartTs\":0,\"CommitTs\":431836588120735751,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"pk_or_uk_7_0\",\"TableID\":0,\"ColumnInfo\":null},\"PreTableInfo\":null,\"Query\":\"CREATE TABLE `pk_or_uk_7_0` (`id` TINYINT UNIQUE KEY NOT NULL)\",\"Type\":3}"]
[2022/03/15 14:32:46.260 +08:00] [INFO] [main.go:530] ["ignore ddl event"] [maxDDLReceivedTs=431836588120735751] [ddl="{\"StartTs\":0,\"CommitTs\":431836588120735751,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"pk_or_uk_7_0\",\"TableID\":0,\"ColumnInfo\":null},\"PreTableInfo\":null,\"Query\":\"CREATE TABLE `pk_or_uk_7_0` (`id` TINYINT UNIQUE KEY NOT NULL)\",\"Type\":3}"]
[2022/03/15 14:32:46.853 +08:00] [INFO] [mysql.go:332] ["start exec DDL"] [DDL="{\"StartTs\":0,\"CommitTs\":431836588120735751,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"pk_or_uk_7_0\",\"TableID\":0,\"ColumnInfo\":null},\"PreTableInfo\":null,\"Query\":\"CREATE TABLE `pk_or_uk_7_0` (`id` TINYINT UNIQUE KEY NOT NULL)\",\"Type\":3}"]
[2022/03/15 14:32:46.936 +08:00] [INFO] [mysql.go:362] ["Exec DDL succeeded"] [sql="CREATE TABLE `pk_or_uk_7_0` (`id` TINYINT UNIQUE KEY NOT NULL)"]
[2022/03/15 14:32:46.943 +08:00] [INFO] [main.go:632] ["update globalResolvedTs"] [ts=431836588120735751]

[2022/03/15 14:32:50.263 +08:00] [INFO] [main.go:542] ["ddl event received"] [ddl="{\"StartTs\":0,\"CommitTs\":431836588134105100,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"pk_or_uk_1_1\",\"TableID\":0,\"ColumnInfo\":null},\"PreTableInfo\":null,\"Query\":\"CREATE TABLE `pk_or_uk_1_1` (`id` BIGINT PRIMARY KEY)\",\"Type\":3}"]
[2022/03/15 14:32:50.266 +08:00] [INFO] [main.go:530] ["ignore ddl event"] [maxDDLReceivedTs=431836588134105100] [ddl="{\"StartTs\":0,\"CommitTs\":431836588134105100,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"pk_or_uk_1_1\",\"TableID\":0,\"ColumnInfo\":null},\"PreTableInfo\":null,\"Query\":\"CREATE TABLE `pk_or_uk_1_1` (`id` BIGINT PRIMARY KEY)\",\"Type\":3}"]
[2022/03/15 14:32:50.272 +08:00] [INFO] [main.go:530] ["ignore ddl event"] [maxDDLReceivedTs=431836588134105100] [ddl="{\"StartTs\":0,\"CommitTs\":431836588134105100,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"pk_or_uk_1_1\",\"TableID\":0,\"ColumnInfo\":null},\"PreTableInfo\":null,\"Query\":\"CREATE TABLE `pk_or_uk_1_1` (`id` BIGINT PRIMARY KEY)\",\"Type\":3}"]
[2022/03/15 14:32:50.273 +08:00] [INFO] [main.go:530] ["ignore ddl event"] [maxDDLReceivedTs=431836588134105100] [ddl="{\"StartTs\":0,\"CommitTs\":431836588134105100,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"pk_or_uk_1_1\",\"TableID\":0,\"ColumnInfo\":null},\"PreTableInfo\":null,\"Query\":\"CREATE TABLE `pk_or_uk_1_1` (`id` BIGINT PRIMARY KEY)\",\"Type\":3}"]
[2022/03/15 14:32:50.665 +08:00] [INFO] [mysql.go:332] ["start exec DDL"] [DDL="{\"StartTs\":0,\"CommitTs\":431836588134105100,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"pk_or_uk_1_1\",\"TableID\":0,\"ColumnInfo\":null},\"PreTableInfo\":null,\"Query\":\"CREATE TABLE `pk_or_uk_1_1` (`id` BIGINT PRIMARY KEY)\",\"Type\":3}"]
[2022/03/15 14:32:50.760 +08:00] [INFO] [mysql.go:362] ["Exec DDL succeeded"] [sql="CREATE TABLE `pk_or_uk_1_1` (`id` BIGINT PRIMARY KEY)"]
[2022/03/15 14:32:50.760 +08:00] [INFO] [main.go:632] ["update globalResolvedTs"] [ts=431836588134105100]

Look at the consumer's log, DDL of finish_mark is not received.

@3AceShowHand 3AceShowHand mentioned this issue Mar 17, 2022
Merged
@3AceShowHand
Copy link
Contributor 

[2022/03/15 14:27:39.121 +08:00] [INFO] [ddl.go:633] ["[ddl] start DDL job"] [job="ID:2, Type:create schema, State:none, SchemaState:queueing, SchemaID:1, TableID:0, RowCount:0, ArgLen:1, start time: 2022-03-15 14:27:39.101 +0800 CST, Err:<nil>, ErrCount:0, SnapshotVersion:0"] [query="CREATE DATABASE IF NOT EXISTS test"]
[2022/03/15 14:27:39.197 +08:00] [INFO] [ddl.go:697] ["[ddl] DDL job is finished"] [jobID=2]

[2022/03/15 14:30:51.425 +08:00] [INFO] [ddl.go:633] ["[ddl] start DDL job"] [job="ID:66, Type:create table, State:none, SchemaState:queueing, SchemaID:1, TableID:65, RowCount:0, ArgLen:1, start time: 2022-03-15 14:30:51.301 +0800 CST, Err:<nil>, ErrCount:0, SnapshotVersion:0"] [query="CREATE TABLE pk_or_uk_5_0(id SMALLINT SIGNED UNIQUE NOT NULL)"]
[2022/03/15 14:30:51.499 +08:00] [INFO] [ddl.go:697] ["[ddl] DDL job is finished"] [jobID=66]

[2022/03/15 14:30:51.651 +08:00] [INFO] [ddl.go:633] ["[ddl] start DDL job"] [job="ID:82, Type:create table, State:none, SchemaState:queueing, SchemaID:1, TableID:67, RowCount:0, ArgLen:1, start time: 2022-03-15 14:30:51.601 +0800 CST, Err:<nil>, ErrCount:0, SnapshotVersion:0"] [query="CREATE TABLE pk_or_uk_2_1(id INT UNSIGNED PRIMARY KEY)"]
[2022/03/15 14:30:51.715 +08:00] [INFO] [ddl.go:697] ["[ddl] DDL job is finished"] [jobID=82]

[2022/03/15 14:30:51.660 +08:00] [INFO] [ddl.go:633] ["[ddl] start DDL job"] [job="ID:83, Type:create table, State:none, SchemaState:queueing, SchemaID:1, TableID:68, RowCount:0, ArgLen:1, start time: 2022-03-15 14:30:51.651 +0800 CST, Err:<nil>, ErrCount:0, SnapshotVersion:0"] [query="CREATE TABLE pk_or_uk_4_1(id SMALLINT UNSIGNED PRIMARY KEY)"]
[2022/03/15 14:30:51.902 +08:00] [INFO] [ddl.go:697] ["[ddl] DDL job is finished"] [jobID=83]

[2022/03/15 14:30:51.660 +08:00] [INFO] [ddl.go:633] ["[ddl] start DDL job"] [job="ID:85, Type:create table, State:none, SchemaState:queueing, SchemaID:1, TableID:70, RowCount:0, ArgLen:1, start time: 2022-03-15 14:30:51.651 +0800 CST, Err:<nil>, ErrCount:0, SnapshotVersion:0"] [query="CREATE TABLE pk_or_uk_5_1(id SMALLINT SIGNED PRIMARY KEY)"]
[2022/03/15 14:30:51.964 +08:00] [INFO] [ddl.go:697] ["[ddl] DDL job is finished"] [jobID=85]

[2022/03/15 14:30:51.660 +08:00] [INFO] [ddl.go:633] ["[ddl] start DDL job"] [job="ID:84, Type:create table, State:none, SchemaState:queueing, SchemaID:1, TableID:69, RowCount:0, ArgLen:1, start time: 2022-03-15 14:30:51.651 +0800 CST, Err:<nil>, ErrCount:0, SnapshotVersion:0"] [query="CREATE TABLE pk_or_uk_4_0(id SMALLINT UNSIGNED UNIQUE NOT NULL)"]
[2022/03/15 14:30:52.028 +08:00] [INFO] [ddl.go:697] ["[ddl] DDL job is finished"] [jobID=84]

[2022/03/15 14:30:51.660 +08:00] [INFO] [ddl.go:633] ["[ddl] start DDL job"] [job="ID:87, Type:create table, State:none, SchemaState:queueing, SchemaID:1, TableID:72, RowCount:0, ArgLen:1, start time: 2022-03-15 14:30:51.651 +0800 CST, Err:<nil>, ErrCount:0, SnapshotVersion:0"] [query="CREATE TABLE pk_or_uk_1_1(id BIGINT SIGNED PRIMARY KEY)"]
[2022/03/15 14:30:52.161 +08:00] [INFO] [ddl.go:697] ["[ddl] DDL job is finished"] [jobID=87]

[2022/03/15 14:30:51.660 +08:00] [INFO] [ddl.go:633] ["[ddl] start DDL job"] [job="ID:86, Type:create table, State:none, SchemaState:queueing, SchemaID:1, TableID:71, RowCount:0, ArgLen:1, start time: 2022-03-15 14:30:51.651 +0800 CST, Err:<nil>, ErrCount:0, SnapshotVersion:0"] [query="CREATE TABLE pk_or_uk_7_0(id TINYINT SIGNED UNIQUE NOT NULL)"]
[2022/03/15 14:30:52.161 +08:00] [INFO] [ddl.go:697] ["[ddl] DDL job is finished"] [jobID=86]

================================================================================================================================
// belove is all lost DDLs.
[2022/03/15 14:30:51.660 +08:00] [INFO] [ddl.go:633] ["[ddl] start DDL job"] [job="ID:89, Type:create table, State:none, SchemaState:queueing, SchemaID:1, TableID:74, RowCount:0, ArgLen:1, start time: 2022-03-15 14:30:51.651 +0800 CST, Err:<nil>, ErrCount:0, SnapshotVersion:0"] [query="CREATE TABLE pk_or_uk_2_0(id INT UNSIGNED UNIQUE NOT NULL)"]
[2022/03/15 14:30:52.167 +08:00] [INFO] [ddl.go:697] ["[ddl] DDL job is finished"] [jobID=89]

[2022/03/15 14:30:51.660 +08:00] [INFO] [ddl.go:633] ["[ddl] start DDL job"] [job="ID:88, Type:create table, State:none, SchemaState:queueing, SchemaID:1, TableID:73, RowCount:0, ArgLen:1, start time: 2022-03-15 14:30:51.651 +0800 CST, Err:<nil>, ErrCount:0, SnapshotVersion:0"] [query="CREATE TABLE pk_or_uk_6_0(id TINYINT UNSIGNED UNIQUE NOT NULL)"]
[2022/03/15 14:30:52.167 +08:00] [INFO] [ddl.go:697] ["[ddl] DDL job is finished"] [jobID=88]

[2022/03/15 14:30:51.660 +08:00] [INFO] [ddl.go:633] ["[ddl] start DDL job"] [job="ID:90, Type:create table, State:none, SchemaState:queueing, SchemaID:1, TableID:75, RowCount:0, ArgLen:1, start time: 2022-03-15 14:30:51.651 +0800 CST, Err:<nil>, ErrCount:0, SnapshotVersion:0"] [query="CREATE TABLE pk_or_uk_0_0(id BIGINT UNSIGNED UNIQUE NOT NULL)"]
[2022/03/15 14:30:52.406 +08:00] [INFO] [ddl.go:697] ["[ddl] DDL job is finished"] [jobID=90]

[2022/03/15 14:30:51.667 +08:00] [INFO] [ddl.go:633] ["[ddl] start DDL job"] [job="ID:94, Type:create table, State:none, SchemaState:queueing, SchemaID:1, TableID:79, RowCount:0, ArgLen:1, start time: 2022-03-15 14:30:51.651 +0800 CST, Err:<nil>, ErrCount:0, SnapshotVersion:0"] [query="CREATE TABLE pk_or_uk_3_1(id INT SIGNED PRIMARY KEY)"]
[2022/03/15 14:30:52.470 +08:00] [INFO] [ddl.go:697] ["[ddl] DDL job is finished"] [jobID=94]

[2022/03/15 14:30:51.667 +08:00] [INFO] [ddl.go:633] ["[ddl] start DDL job"] [job="ID:95, Type:create table, State:none, SchemaState:queueing, SchemaID:1, TableID:80, RowCount:0, ArgLen:1, start time: 2022-03-15 14:30:51.651 +0800 CST, Err:<nil>, ErrCount:0, SnapshotVersion:0"] [query="CREATE TABLE pk_or_uk_3_0(id INT SIGNED UNIQUE NOT NULL)"]
[2022/03/15 14:30:52.607 +08:00] [INFO] [ddl.go:697] ["[ddl] DDL job is finished"] [jobID=95]

[2022/03/15 14:30:51.660 +08:00] [INFO] [ddl.go:633] ["[ddl] start DDL job"] [job="ID:93, Type:create table, State:none, SchemaState:queueing, SchemaID:1, TableID:78, RowCount:0, ArgLen:1, start time: 2022-03-15 14:30:51.651 +0800 CST, Err:<nil>, ErrCount:0, SnapshotVersion:0"] [query="CREATE TABLE pk_or_uk_6_1(id TINYINT UNSIGNED PRIMARY KEY)"]
[2022/03/15 14:30:52.661 +08:00] [INFO] [ddl.go:697] ["[ddl] DDL job is finished"] [jobID=93]

[2022/03/15 14:30:51.660 +08:00] [INFO] [ddl.go:633] ["[ddl] start DDL job"] [job="ID:91, Type:create table, State:none, SchemaState:queueing, SchemaID:1, TableID:76, RowCount:0, ArgLen:1, start time: 2022-03-15 14:30:51.651 +0800 CST, Err:<nil>, ErrCount:0, SnapshotVersion:0"] [query="CREATE TABLE pk_or_uk_0_1(id BIGINT UNSIGNED PRIMARY KEY)"]
[2022/03/15 14:30:52.661 +08:00] [INFO] [ddl.go:697] ["[ddl] DDL job is finished"] [jobID=91]

[2022/03/15 14:30:51.660 +08:00] [INFO] [ddl.go:633] ["[ddl] start DDL job"] [job="ID:92, Type:create table, State:none, SchemaState:queueing, SchemaID:1, TableID:77, RowCount:0, ArgLen:1, start time: 2022-03-15 14:30:51.651 +0800 CST, Err:<nil>, ErrCount:0, SnapshotVersion:0"] [query="CREATE TABLE pk_or_uk_7_1(id TINYINT SIGNED PRIMARY KEY)"]
[2022/03/15 14:30:52.661 +08:00] [INFO] [ddl.go:697] ["[ddl] DDL job is finished"] [jobID=92]


[2022/03/15 14:30:51.667 +08:00] [INFO] [ddl.go:633] ["[ddl] start DDL job"] [job="ID:96, Type:create table, State:none, SchemaState:queueing, SchemaID:1, TableID:81, RowCount:0, ArgLen:1, start time: 2022-03-15 14:30:51.651 +0800 CST, Err:<nil>, ErrCount:0, SnapshotVersion:0"] [query="CREATE TABLE pk_or_uk_1_0(id BIGINT SIGNED UNIQUE NOT NULL)"]
[2022/03/15 14:30:52.668 +08:00] [INFO] [ddl.go:697] ["[ddl] DDL job is finished"] [jobID=96]

[2022/03/15 14:30:52.686 +08:00] [INFO] [ddl.go:633] ["[ddl] start DDL job"] [job="ID:98, Type:create table, State:none, SchemaState:queueing, SchemaID:1, TableID:97, RowCount:0, ArgLen:1, start time: 2022-03-15 14:30:52.651 +0800 CST, Err:<nil>, ErrCount:0, SnapshotVersion:0"] [query="create table finish_mark(a int primary key);"]
[2022/03/15 14:30:52.749 +08:00] [INFO] [ddl.go:697] ["[ddl] DDL job is finished"] [jobID=98]

take a look at the logs above, all lost DDLs is can be separated by time line.

cdc exit after the last synced DDL executed 3s later, but there are still 10 DDLs need to be executed, I would prefer this is cause by unstable testing environment, close it now.

@3AceShowHand
Copy link
Contributor

close by the comment above, if you see this again, you can open this again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@3AceShowHand 3AceShowHand

Labels
area/ticdc Issues or PRs related to TiCDC. component/test Unit tests and integration tests component. type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@overvenus @3AceShowHand