lightning parallel import fail for single table with duplication data, duplicate-resolution = record

[seiya-annie]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

1. Launch 2 lightning instance to do parallel local backend import data for single data, with duplicate data, specify duplicate-resolution = record
2. Lightning should succeed with notification to users to let them know there is duplicate data

2. What did you expect to see? (Required)

lightning import successfully

3. What did you see instead (Required)

2023-01-11T14:37:20.575+0800    INFO    host/host.go:41 Execute command finished        {"command": "/tidb-lightning \"-c\" \"/tmp/tidb-lightning.toml\"", "execute duration": "1m10.798678809s"}
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x50 pc=0x230a54c]

goroutine 1695 [running]:
github.com/tikv/client-go/v2/internal/locate.(*CodecPDClient).GetRegion(0xc000e2b320, {0x4dec5f0, 0xc0b480d3b0}, {0xc00005ce70?, 0xc0010acde0?, 0x16ad03f?}, {0xc0b7876140, 0x1, 0x1})
        /go/pkg/mod/github.com/tikv/client-go/[email protected]/internal/locate/pd_codec.go:96 +0x4c
github.com/tikv/client-go/v2/internal/locate.(*RegionCache).loadRegion(0xc0011420e0, 0xc001169680, {0xc00005ce70, 0x13, 0x13}, 0x0)
        /go/pkg/mod/github.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:1456 +0x383
github.com/tikv/client-go/v2/internal/locate.(*RegionCache).findRegionByKey(0xc0011420e0, 0xc001169680, {0xc00005ce70, 0x13, 0x13}, 0x0?)
        /go/pkg/mod/github.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:948 +0x425
github.com/tikv/client-go/v2/internal/locate.(*RegionCache).LocateKey(0xc0010ad240?, 0x16ef032?, {0xc00005ce70?, 0x2?, 0xc0b60ea018?})
        /go/pkg/mod/github.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:917 +0x36
github.com/tikv/client-go/v2/internal/locate.(*RegionCache).GroupKeysByRegion(0x2c?, 0x4dc3360?, {0xc000fbe000, 0x8, 0xc0010ad2e0?}, 0x0)
        /go/pkg/mod/github.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:1142 +0x1a7
github.com/tikv/client-go/v2/txnkv/txnsnapshot.(*KVSnapshot).batchGetKeysByRegions(0xc0b95a6160, 0xc001169680, {0xc000fbe000, 0x8, 0x8}, 0xc0006227f8)
        /go/pkg/mod/github.com/tikv/client-go/[email protected]/txnkv/txnsnapshot/snapshot.go:323 +0x12a
github.com/tikv/client-go/v2/txnkv/txnsnapshot.(*KVSnapshot).BatchGet(0xc0b95a6160, {0x4dec548, 0xc0b773d940}, {0xc000fbe000, 0x8, 0x8})
        /go/pkg/mod/github.com/tikv/client-go/[email protected]/txnkv/txnsnapshot/snapshot.go:234 +0x4d5
github.com/pingcap/tidb/br/pkg/lightning/backend/local.(*DuplicateManager).saveIndexHandles(0xc0ae3ce380, {0x4dec548, 0xc0b773d940}, {{0xc000556780, 0x8, 0x8}, {0xc0360eeb80, 0x8, 0x8}, {0xc0360eec80, ...}, ...})
        /home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/lightning/backend/local/duplicate.go:472 +0x97
github.com/pingcap/tidb/br/pkg/lightning/backend/local.(*DuplicateManager).RecordIndexConflictError(0xc0ae3ce380, {0x4dec548, 0xc0b773d940}, {0x4dd0948?, 0xc0abd70140?}, 0x48?, 0xc000ee8f30)
        /home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/lightning/backend/local/duplicate.go:529 +0x8d0
github.com/pingcap/tidb/br/pkg/lightning/backend/local.(*DuplicateManager).processRemoteDupTaskOnce.func1.1({0x4dec548, 0xc0b773d940}, 0x0?, {{0xc0b6187a40, 0x39, 0x48}, {0xc0b6187a90, 0x3a, 0x48}, {}, ...}, ...)
        /home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/lightning/backend/local/duplicate.go:787 +0xc5
github.com/pingcap/tidb/br/pkg/lightning/backend/local.(*DuplicateManager).processRemoteDupTaskOnce.func1()
        /home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/lightning/backend/local/duplicate.go:793 +0x4a8
github.com/pingcap/tidb/br/pkg/utils.(*WorkerPool).Apply.func1()
        /home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/utils/worker.go:58 +0x71
created by github.com/pingcap/tidb/br/pkg/utils.(*WorkerPool).Apply
        /home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/utils/worker.go:56 +0x98
2023-01-11T14:37:22.222+0800    INFO    host/host.go:37 Execute command error   {"command": "/tidb-lightning \"-c\" \"/tmp/tidb-lightning.toml\"", "exit code": 2, "stdout": "Verbose debug logs will be written to /tmp/lightning.log.2023-01-11T06.36.10Z\n\n

"stderr": "panic: runtime error: invalid memory address or nil pointer dereference\n[signal SIGSEGV: segmentation violation code=0x1 addr=0x50 pc=0x230a54c]\n\ngoroutine 1695 [running]:\ngithub.com/tikv/client-go/v2/internal/locate.(*CodecPDClient).GetRegion(0xc000e2b320, {0x4dec5f0, 0xc0b480d3b0}, {0xc00005ce70?, 0xc0010acde0?, 0x16ad03f?}, {0xc0b7876140, 0x1, 0x1})\n\t/go/pkg/mod/github.com/tikv/client-go/[email protected]/internal/locate/pd_codec.go:96 +0x4c\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).loadRegion(0xc0011420e0, 0xc001169680, {0xc00005ce70, 0x13, 0x13}, 0x0)\n\t/go/pkg/mod/github.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:1456 +0x383\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).findRegionByKey(0xc0011420e0, 0xc001169680, {0xc00005ce70, 0x13, 0x13}, 0x0?)\n\t/go/pkg/mod/github.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:948 +0x425\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).LocateKey(0xc0010ad240?, 0x16ef032?, {0xc00005ce70?, 0x2?, 0xc0b60ea018?})\n\t/go/pkg/mod/github.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:917 +0x36\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).GroupKeysByRegion(0x2c?, 0x4dc3360?, {0xc000fbe000, 0x8, 0xc0010ad2e0?}, 0x0)\n\t/go/pkg/mod/github.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:1142 +0x1a7\ngithub.com/tikv/client-go/v2/txnkv/txnsnapshot.(*KVSnapshot).batchGetKeysByRegions(0xc0b95a6160, 0xc001169680, {0xc000fbe000, 0x8, 0x8}, 0xc0006227f8)\n\t/go/pkg/mod/github.com/tikv/client-go/[email protected]/txnkv/txnsnapshot/snapshot.go:323 +0x12a\ngithub.com/tikv/client-go/v2/txnkv/txnsnapshot.(*KVSnapshot).BatchGet(0xc0b95a6160, {0x4dec548, 0xc0b773d940}, {0xc000fbe000, 0x8, 0x8})\n\t/go/pkg/mod/github.com/tikv/client-go/[email protected]/txnkv/txnsnapshot/snapshot.go:234 +0x4d5\ngithub.com/pingcap/tidb/br/pkg/lightning/backend/local.(*DuplicateManager).saveIndexHandles(0xc0ae3ce380, {0x4dec548, 0xc0b773d940}, {{0xc000556780, 0x8, 0x8}, {0xc0360eeb80, 0x8, 0x8}, {0xc0360eec80, ...}, ...})\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/lightning/backend/local/duplicate.go:472 +0x97\ngithub.com/pingcap/tidb/br/pkg/lightning/backend/local.(*DuplicateManager).RecordIndexConflictError(0xc0ae3ce380, {0x4dec548, 0xc0b773d940}, {0x4dd0948?, 0xc0abd70140?}, 0x48?, 0xc000ee8f30)\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/lightning/backend/local/duplicate.go:529 +0x8d0\ngithub.com/pingcap/tidb/br/pkg/lightning/backend/local.(*DuplicateManager).processRemoteDupTaskOnce.func1.1({0x4dec548, 0xc0b773d940}, 0x0?, {{0xc0b6187a40, 0x39, 0x48}, {0xc0b6187a90, 0x3a, 0x48}, {}, ...}, ...)\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/lightning/backend/local/duplicate.go:787 +0xc5\ngithub.com/pingcap/tidb/br/pkg/lightning/backend/local.(*DuplicateManager).processRemoteDupTaskOnce.func1()\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/lightning/backend/local/duplicate.go:793 +0x4a8\ngithub.com/pingcap/tidb/br/pkg/utils.(*WorkerPool).Apply.func1()\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/utils/worker.go:58 +0x71\ncreated by github.com/pingcap/tidb/br/pkg/utils.(*WorkerPool).Apply\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/utils/worker.go:56 +0x98\n", "error": "command terminated with exit code 2"}

### 4. What is your TiDB version? (Required)

/ # ./tidb-lightning -V
Release Version: v6.6.0-alpha
Git Commit Hash: 10f00936b7c88ae7a62d39291b411abb5003c1aa
Git Branch: heads/refs/tags/v6.6.0-alpha
Go Version: go1.19.3
UTC Build Time: 2023-01-10 11:43:41
Race Enabled: false
/ # 

[GMHDBJD]

tikv/client-go#649 cc @AmoebaProtozoa