Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

server_test.TestServer data race #3785

Closed
asddongmen opened this issue Dec 8, 2021 · 1 comment
Closed

server_test.TestServer data race #3785

asddongmen opened this issue Dec 8, 2021 · 1 comment
Labels
area/dm Issues or PRs related to DM. component/test Unit tests and integration tests component.

Comments

@asddongmen
Copy link
Contributor

Which jobs are flaking?

https://ci.pingcap.net/blue/organizations/jenkins/cdc-verify-ci/detail/cdc-verify-ci/1442/pipeline

Which test(s) are flaking?

server_test.TestServer
https://github.com/pingcap/ticdc/blob/master/dm/dm/worker/server_test.go#L139

Jenkins logs or GitHub Actions link

WARNING: DATA RACE
Read at 0x00c0011b2130 by goroutine 238:
  runtime.slicecopy()
      /usr/local/go/src/runtime/slice.go:247 +0x0
  go.etcd.io/etcd/clientv3.(*Client).Endpoints()
      /nfs/cache/mod/go.etcd.io/[email protected]/clientv3/client.go:157 +0x193
  go.etcd.io/etcd/clientv3.(*Client).roundRobinQuorumBackoff.func1()
      /nfs/cache/mod/go.etcd.io/[email protected]/clientv3/client.go:506 +0x78
  go.etcd.io/etcd/clientv3.waitRetryBackoff()
      /nfs/cache/mod/go.etcd.io/[email protected]/clientv3/retry_interceptor.go:276 +0x25b
  go.etcd.io/etcd/clientv3.(*Client).unaryClientInterceptor.func1()
      /nfs/cache/mod/go.etcd.io/[email protected]/clientv3/retry_interceptor.go:50 +0x1fc
  google.golang.org/grpc.(*ClientConn).Invoke()
      /nfs/cache/mod/google.golang.org/[email protected]/call.go:35 +0x1a6
  google.golang.org/grpc.Invoke()
      /nfs/cache/mod/google.golang.org/[email protected]/call.go:60 +0x110
  go.etcd.io/etcd/etcdserver/etcdserverpb.(*leaseClient).LeaseRevoke()
      /nfs/cache/mod/go.etcd.io/[email protected]/etcdserver/etcdserverpb/rpc.pb.go:3803 +0x55
  go.etcd.io/etcd/clientv3.(*retryLeaseClient).LeaseRevoke()
      /nfs/cache/mod/go.etcd.io/[email protected]/clientv3/retry.go:148 +0x186
  go.etcd.io/etcd/clientv3.(*lessor).Revoke()
      /nfs/cache/mod/go.etcd.io/[email protected]/clientv3/lease.go:231 +0x10c
  github.com/pingcap/ticdc/dm/pkg/ha.revokeLease()
      /home/jenkins/agent/workspace/atom-ut/ticdc/dm/pkg/ha/keepalive.go:180 +0x13d
  github.com/pingcap/ticdc/dm/pkg/ha.KeepAlive.func2()
      /home/jenkins/agent/workspace/atom-ut/ticdc/dm/pkg/ha/keepalive.go:120 +0x76
  github.com/pingcap/ticdc/dm/pkg/ha.KeepAlive()
      /home/jenkins/agent/workspace/atom-ut/ticdc/dm/pkg/ha/keepalive.go:138 +0x6b6
  github.com/pingcap/ticdc/dm/dm/worker.(*Server).KeepAlive()
      /home/jenkins/agent/workspace/atom-ut/ticdc/dm/dm/worker/join.go:104 +0x2eb
  github.com/pingcap/ticdc/dm/dm/worker.(*Server).doStartKeepAlive()
      /home/jenkins/agent/workspace/atom-ut/ticdc/dm/dm/worker/server.go:254 +0x8a
Previous write at 0x00c0011b2130 by goroutine 177:
  runtime.slicecopy()
      /usr/local/go/src/runtime/slice.go:247 +0x0
  github.com/pingcap/ticdc/dm/dm/worker.(*Server).syncMasterEndpoints.func1()
      /home/jenkins/agent/workspace/atom-ut/ticdc/dm/dm/worker/server.go:282 +0x2d3
  github.com/pingcap/ticdc/dm/dm/worker.(*Server).syncMasterEndpoints()
      /home/jenkins/agent/workspace/atom-ut/ticdc/dm/dm/worker/server.go:295 +0x156
  github.com/pingcap/ticdc/dm/dm/worker.(*Server).Start.func2()
      /home/jenkins/agent/workspace/atom-ut/ticdc/dm/dm/worker/server.go:129 +0x7a
Goroutine 238 (running) created at:
  github.com/pingcap/ticdc/dm/dm/worker.(*Server).startKeepAlive()
      /home/jenkins/agent/workspace/atom-ut/ticdc/dm/dm/worker/server.go:249 +0x12c
  github.com/pingcap/ticdc/dm/dm/worker.(*Server).Start()
      /home/jenkins/agent/workspace/atom-ut/ticdc/dm/dm/worker/server.go:133 +0x875
  github.com/pingcap/ticdc/dm/dm/worker.(*testServer).TestServer.func4()
      /home/jenkins/agent/workspace/atom-ut/ticdc/dm/dm/worker/server_test.go:139 +0x3c
Goroutine 177 (running) created at:
  github.com/pingcap/ticdc/dm/dm/worker.(*Server).Start()
      /home/jenkins/agent/workspace/atom-ut/ticdc/dm/dm/worker/server.go:128 +0x849
  github.com/pingcap/ticdc/dm/dm/worker.(*testServer).TestServer.func4()
      /home/jenkins/agent/workspace/atom-ut/ticdc/dm/dm/worker/server_test.go:139 +0x3c
==================

https://ci.pingcap.net/blue/organizations/jenkins/cdc-verify-ci/detail/cdc-verify-ci/1442/pipeline

Anything else we need to know

  • Does this test exist for other branches as well?

  • Has there been a high frequency of failure lately?
@asddongmen asddongmen added component/test Unit tests and integration tests component. area/dm Issues or PRs related to DM. labels Dec 8, 2021
@asddongmen asddongmen mentioned this issue Dec 8, 2021
Closed
@lance6716
Copy link
Contributor

syncMasterEndpoints is removed from code now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/dm Issues or PRs related to DM. component/test Unit tests and integration tests component.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@lance6716 @asddongmen