Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When sync-diff verifies data, an error is reported if the upstream and downstream tables are inconsistent #680

Open
jt20190617 opened this issue Sep 23, 2022 · 1 comment

Comments

@jt20190617
Copy link

Bug Report

Please answer these questions before submitting your issue. Thanks!

  1. What did you do?
    upstream: tidb, and have 2 tables: t1,t2
    downstream: tidb, and have 1 tables: t2

Then check the data with sync-diff: ./sync_diff_inspector --config=config.toml

  1. What did you expect to see?
    Sync-diff notices that there is an additional T1 table upstream, records this information, and proceeds to check the other tables

  2. What did you see instead?
    Sync-diff logs out with an error.
    [2022/09/23 17:47:52.913 +08:00] [INFO] [printer.go:46] ["Welcome to sync_diff_inspector"] ["Release Version"=v6.1.1-4-g4572996] ["Git Commit Hash"=45729968487c9b4aa5820b3c00326be0c9bb89d5] ["Git Branch"=master] ["UTC Build Time"="2022-09-23 08:29:09"] ["Go Version"=go1.19.1] [2022/09/23 17:47:52.914 +08:00] [INFO] [main.go:101] [config="{\"check-thread-count\":4,\"split-thread-count\":5,\"export-fix-sql\":true,\"check-struct-only\":false,\"dm-addr\":\"\",\"dm-task\":\"\",\"data-sources\":{\"tidb0\":{\"host\":\"172.16.201.121\",\"port\":4023,\"user\":\"root\",\"password\":\"\",\"sql-mode\":\"\",\"snapshot\":\"\",\"security\":null,\"route-rules\":null,\"Router\":{\"Selector\":{}},\"Conn\":null},\"tidb1\":{\"host\":\"172.16.201.121\",\"port\":4021,\"user\":\"root\",\"password\":\"\",\"sql-mode\":\"\",\"snapshot\":\"\",\"security\":null,\"route-rules\":null,\"Router\":{\"Selector\":{}},\"Conn\":null}},\"routes\":null,\"table-configs\":null,\"task\":{\"source-instances\":[\"tidb1\"],\"source-routes\":null,\"target-instance\":\"tidb0\",\"target-check-tables\":[\"*.*\",\"!INFORMATION_SCHEMA.*\",\"!METRICS_SCHEMA.*\",\"!PERFORMANCE_SCHEMA.*\",\"!mysql.*\",\"!test.*\"],\"target-configs\":null,\"output-dir\":\"./output\",\"SourceInstances\":[{\"host\":\"172.16.201.121\",\"port\":4021,\"user\":\"root\",\"password\":\"\",\"sql-mode\":\"\",\"snapshot\":\"\",\"security\":null,\"route-rules\":null,\"Router\":{\"Selector\":{}},\"Conn\":null}],\"TargetInstance\":{\"host\":\"172.16.201.121\",\"port\":4023,\"user\":\"root\",\"password\":\"\",\"sql-mode\":\"\",\"snapshot\":\"\",\"security\":null,\"route-rules\":null,\"Router\":{\"Selector\":{}},\"Conn\":null},\"TargetTableConfigs\":null,\"TargetCheckTables\":[{},{},{},{},{},{}],\"FixDir\":\"output/fix-on-tidb0\",\"CheckpointDir\":\"output/checkpoint\",\"HashFile\":\"\"},\"ConfigFile\":\"config.toml\",\"PrintVersion\":false}"] [2022/09/23 17:47:53.874 +08:00] [INFO] [tidb.go:199] ["find router for tidb source"] [2022/09/23 17:47:53.874 +08:00] [FATAL] [main.go:120] ["failed to initialize diff process"] [error="from upstream: please make sure the filter is correct.: the target has no table to be compared. source-table is ``db1.t2``"] [errorVerbose="the target has no table to be compared. source-table is ``db1.t2``\ngithub.com/pingcap/tidb-tools/sync_diff_inspector/source.checkTableMatched\n\t/home/jenkins/agent/workspace/d_tidb_tools_multi_branch_master/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/source/source.go:389\ngithub.com/pingcap/tidb-tools/sync_diff_inspector/source.getSourceTableMap\n\t/home/jenkins/agent/workspace/d_tidb_tools_multi_branch_master/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/source/tidb.go:256\ngithub.com/pingcap/tidb-tools/sync_diff_inspector/source.NewTiDBSource\n\t/home/jenkins/agent/workspace/d_tidb_tools_multi_branch_master/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/source/tidb.go:263\ngithub.com/pingcap/tidb-tools/sync_diff_inspector/source.buildSourceFromCfg\n\t/home/jenkins/agent/workspace/d_tidb_tools_multi_branch_master/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/source/source.go:228\ngithub.com/pingcap/tidb-tools/sync_diff_inspector/source.NewSources\n\t/home/jenkins/agent/workspace/d_tidb_tools_multi_branch_master/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/source/source.go:206\nmain.(*Diff).init\n\t/home/jenkins/agent/workspace/d_tidb_tools_multi_branch_master/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/diff.go:139\nmain.NewDiff\n\t/home/jenkins/agent/workspace/d_tidb_tools_multi_branch_master/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/diff.go:97\nmain.checkSyncState\n\t/home/jenkins/agent/workspace/d_tidb_tools_multi_branch_master/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/main.go:117\nmain.main\n\t/home/jenkins/agent/workspace/d_tidb_tools_multi_branch_master/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/main.go:104\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:250\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1594\nplease make sure the filter is correct.\nfrom upstream"] [stack="main.checkSyncState\n\t/home/jenkins/agent/workspace/d_tidb_tools_multi_branch_master/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/main.go:120\nmain.main\n\t/home/jenkins/agent/workspace/d_tidb_tools_multi_branch_master/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/main.go:104\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:250"]

  3. What version of TiDB are you using (tidb-server -V or run select tidb_version(); on TiDB)?
    all version

  4. which tool are you using?
    sync-diff

  5. what versionof tool are you using (pump -V or tidb-lightning -V or syncer -V)?
    all version

@3pointer
Copy link
Contributor

The workaround is set the correct table name with table filter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants