integration test get stuck during tidb cluster startup

[amyangfei]

Which jobs are flaking?

https://ci2.pingcap.net/blue/organizations/jenkins/cdc_ghpr_integration_test/detail/cdc_ghpr_integration_test/4858/pipeline

Which test(s) are flaking?

integration test, common_1

Jenkins logs or GitHub Actions link

upstream PD starts up failed

[2021/10/08 17:43:24.119 +08:00] [INFO] [etcd.go:299] ["starting an etcd server"] [etcd-version=3.4.3] [git-sha="Not provided (use ./build instead of go build)"] [go-version=go1.16.4] [go-os=linux] [go-arch=amd64] [max-cpu-set=40] [max-cpu-available=40] [member-initialized=true] [name=pd1] [data-dir=/tmp/tidb_cdc_test/common_1/pd1] [wal-dir=] [wal-dir-dedicated=] [member-dir=/tmp/tidb_cdc_test/common_1/pd1/member] [force-new-cluster=false] [heartbeat-interval=500ms] [election-timeout=3s] [initial-election-tick-advance=true] [snapshot-count=100000] [snapshot-catchup-entries=5000] [initial-advertise-peer-urls="[http://127.0.0.1:2380]"] [listen-peer-urls="[http://0.0.0.0:2380]"] [advertise-client-urls="[http://127.0.0.1:2379]"] [listen-client-urls="[http://0.0.0.0:2379]"] [listen-metrics-urls="[]"] [cors="[*]"] [host-whitelist="[*]"] [initial-cluster=] [initial-cluster-state=new] [initial-cluster-token=] [quota-size-bytes=8589934592] [pre-vote=true] [initial-corrupt-check=false] [corrupt-check-time-interval=0s] [auto-compaction-mode=periodic] [auto-compaction-retention=1h0m0s] [auto-compaction-interval=1h0m0s] [discovery-url=] [discovery-proxy=]
[2021/10/08 17:43:24.120 +08:00] [INFO] [backend.go:79] ["opened backend db"] [path=/tmp/tidb_cdc_test/common_1/pd1/member/snap/db] [took=396.712<C2><B5>s]
[2021/10/08 17:43:24.138 +08:00] [FATAL] [storage.go:88] ["failed to read WAL, cannot be repaired"] [error="walpb: crc mismatch"] [stack="go.etcd.io/etcd/etcdserver.readWAL\n\t/nfs/cache/mod/go.etcd.io/[email protected]/etcdserver/storage.go:88\ngo.etcd.io/etcd/etcdserver.restartNode\n\t/nfs/cache/mod/go.etcd.io/[email protected]/etcdserver/raft.go:503\ngo.etcd.io/etcd/etcdserver.NewServer\n\t/nfs/cache/mod/go.etcd.io/[email protected]/etcdserver/server.go:472\ngo.etcd.io/etcd/embed.StartEtcd\n\t/nfs/cache/mod/go.etcd.io/[email protected]/embed/etcd.go:211\ngithub.com/tikv/pd/server.(*Server).startEtcd\n\t/home/jenkins/agent/workspace/build_pd_multi_branch_master/go/src/github.com/pingcap/pd/server/server.go:272\ngithub.com/tikv/pd/server.(*Server).Run\n\t/home/jenkins/agent/workspace/build_pd_multi_branch_master/go/src/github.com/pingcap/pd/server/server.go:491\nmain.main\n\t/home/jenkins/agent/workspace/build_pd_multi_branch_master/go/src/github.com/pingcap/pd/cmd/pd-server/main.go:121\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:225"]

Anything else we need to know

The following check will enter infinite loop:

https://github.com/pingcap/ticdc/blob/82d922390825f29fda96091791e9da8973e509a2/tests/_utils/start_tidb_cluster_impl#L97-L109