Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DATA RACE in the TestShuffleMergeJoinInDisk #53862

Closed
Tracked by #41316
hawkingrei opened this issue Jun 6, 2024 · 3 comments
Closed
Tracked by #41316

DATA RACE in the TestShuffleMergeJoinInDisk #53862

hawkingrei opened this issue Jun 6, 2024 · 3 comments
Labels
affects-8.5 component/test may-affects-5.4 This bug maybe affects 5.4.x versions. may-affects-6.1 may-affects-6.5 may-affects-7.1 may-affects-7.5 may-affects-8.1 severity/major sig/execution SIG execution type/bug The issue is confirmed as a bug.

Comments

@hawkingrei
Copy link
Member

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

2. What did you expect to see? (Required)

3. What did you see instead (Required)

==================
WARNING: DATA RACE
Write at 0x00c00978fb10 by goroutine 63:
  github.com/pingcap/tidb/pkg/util/chunk.(*Chunk).resetForReuse()
      pkg/util/chunk/chunk.go:153 +0x910
  github.com/pingcap/tidb/pkg/util/chunk.(*allocator).Reset()
      pkg/util/chunk/alloc.go:123 +0x124
  github.com/pingcap/tidb/pkg/testkit.(*TestKit).MustQuery.func1()
      pkg/testkit/testkit.go:169 +0x65
  runtime.deferCallSave()
      GOROOT/src/runtime/panic.go:798 +0x83
  testing.(*T).FailNow()
      <autogenerated>:1 +0x31
  github.com/stretchr/testify/require.NoError()
      external/com_github_stretchr_testify/require/require.go:1357 +0xba
  github.com/stretchr/testify/require.(*Assertions).NoError()
      external/com_github_stretchr_testify/require/require_forward.go:1072 +0xa6
  github.com/pingcap/tidb/pkg/testkit.(*TestKit).ResultSetToResultWithCtx()
      pkg/testkit/testkit.go:272 +0x14e
  github.com/pingcap/tidb/pkg/testkit.(*TestKit).MustQueryWithContext()
      pkg/testkit/testkit.go:197 +0x2e4
  github.com/pingcap/tidb/pkg/testkit.(*TestKit).MustQuery()
      pkg/testkit/testkit.go:172 +0x106
  github.com/pingcap/tidb/pkg/executor/join_test.checkMergeAndRun()
      pkg/executor/join/merge_join_test.go:38 +0x329
  github.com/pingcap/tidb/pkg/executor/join_test.TestShuffleMergeJoinInDisk()
      pkg/executor/join/merge_join_test.go:67 +0x61c
  github.com/pingcap/tidb/pkg/domain.(*Domain).rebuildSysVarCache()
      pkg/domain/sysvar_cache.go:145 +0x904
  github.com/pingcap/tidb/pkg/domain.(*Domain).LoadSysVarCacheLoop()
      pkg/domain/domain.go:1755 +0x93
  github.com/pingcap/tidb/pkg/session.bootstrapSessionImpl()
      pkg/session/session.go:3426 +0x864
  github.com/pingcap/tidb/pkg/domain.(*Domain).GetSessionCache()
      pkg/domain/sysvar_cache.go:61 +0x50
  github.com/pingcap/tidb/pkg/session.(*session).loadCommonGlobalVariablesIfNeeded()
      pkg/session/session.go:3794 +0x2ce
  github.com/pingcap/tidb/pkg/session.(*session).ExecuteStmt()
      pkg/session/session.go:2008 +0x184
  github.com/pingcap/tidb/pkg/session.(*session).ExecuteInternal()
      pkg/session/session.go:1519 +0x3b9
  github.com/pingcap/tidb/pkg/domain.(*Domain).LoadPrivilegeLoop()
      pkg/domain/domain.go:1698 +0x108
  github.com/pingcap/tidb/pkg/session.bootstrapSessionImpl()
      pkg/session/session.go:3419 +0x804
  github.com/pingcap/tidb/pkg/session.BootstrapSession()
      pkg/session/session.go:3318 +0xad
  github.com/pingcap/tidb/pkg/testkit.bootstrap()
      pkg/testkit/mockstore.go:254 +0x94
  github.com/pingcap/tidb/pkg/testkit.CreateMockStoreAndDomain()
      pkg/testkit/mockstore.go:225 +0xd2
  github.com/pingcap/tidb/pkg/executor/join_test.TestShuffleMergeJoinInDisk()
      pkg/executor/join/merge_join_test.go:46 +0xf9
  github.com/pingcap/failpoint.parseTerm()
      external/com_github_pingcap_failpoint/terms.go:149 +0x364
  github.com/pingcap/failpoint.parse()
      external/com_github_pingcap_failpoint/terms.go:126 +0xa5
  github.com/pingcap/failpoint.newTerms()
      external/com_github_pingcap_failpoint/terms.go:98 +0x3e
  github.com/pingcap/failpoint.(*Failpoint).Enable()
      external/com_github_pingcap_failpoint/failpoint.go:58 +0x3e
  github.com/pingcap/failpoint.(*Failpoints).Enable()
      external/com_github_pingcap_failpoint/failpoints.go:105 +0x296
  github.com/pingcap/failpoint.Enable()
      external/com_github_pingcap_failpoint/failpoints.go:255 +0x55
  github.com/pingcap/tidb/pkg/executor/join_test.TestShuffleMergeJoinInDisk()
      pkg/executor/join/merge_join_test.go:42 +0x56
  testing.tRunner()
      GOROOT/src/testing/testing.go:1595 +0x261
  testing.(*T).Run.func1()
      GOROOT/src/testing/testing.go:1648 +0x44
Previous read at 0x00c00978fb10 by goroutine 45341:
  github.com/pingcap/tidb/pkg/util/chunk.(*Chunk).Reset()
      pkg/util/chunk/chunk.go:285 +0x564
  github.com/pingcap/tidb/pkg/executor/sortexec.(*SortExec).Next()
      pkg/executor/sortexec/sort.go:273 +0x635
  github.com/pingcap/tidb/pkg/executor/internal/exec.Next()
      pkg/executor/internal/exec/executor.go:410 +0x3b0
  github.com/pingcap/tidb/pkg/executor/join.(*MergeJoinTable).fetchNextChunk()
      pkg/executor/join/merge_join.go:169 +0x26f
  github.com/pingcap/tidb/pkg/executor/join.(*MergeJoinTable).fetchNextInnerGroup()
      pkg/executor/join/merge_join.go:213 +0x744
  github.com/pingcap/tidb/pkg/executor/join.(*MergeJoinExec).Next()
      pkg/executor/join/merge_join.go:335 +0x3eb
  github.com/pingcap/tidb/pkg/executor/internal/exec.Next()
      pkg/executor/internal/exec/executor.go:410 +0x3b0
  github.com/pingcap/tidb/pkg/executor.(*shuffleWorker).run()
      pkg/executor/shuffle.go:409 +0x2aa
  github.com/pingcap/tidb/pkg/executor.(*ShuffleExec).prepare4ParallelExec.func2()
      pkg/executor/shuffle.go:215 +0x5d
Goroutine 63 (running) created at:
  testing.(*T).Run()
      GOROOT/src/testing/testing.go:1648 +0x845
  testing.runTests.func1()
      GOROOT/src/testing/testing.go:2054 +0x84
  testing.tRunner()
      GOROOT/src/testing/testing.go:1595 +0x261
  testing.runTests()
      GOROOT/src/testing/testing.go:2052 +0x8ad
  testing.(*M).Run()
      GOROOT/src/testing/testing.go:1925 +0xcd7
  main.main()
      bazel-out/k8-fastbuild/bin/pkg/executor/join/join_test_/testmain.go:159 +0x6dc
Goroutine 45341 (running) created at:
  github.com/pingcap/tidb/pkg/executor.(*ShuffleExec).prepare4ParallelExec()
      pkg/executor/shuffle.go:215 +0x264
  github.com/pingcap/tidb/pkg/executor.(*ShuffleExec).Next()
      pkg/executor/shuffle.go:230 +0x11b
  github.com/pingcap/tidb/pkg/executor/internal/exec.Next()
      pkg/executor/internal/exec/executor.go:410 +0x3b0
  github.com/pingcap/tidb/pkg/executor.(*SelectionExec).Next()
      pkg/executor/executor.go:1605 +0x208
  github.com/pingcap/tidb/pkg/executor/internal/exec.Next()
      pkg/executor/internal/exec/executor.go:410 +0x3b0
  github.com/pingcap/tidb/pkg/executor.(*ExecStmt).next()
      pkg/executor/adapter.go:1211 +0x84
  github.com/pingcap/tidb/pkg/executor.(*recordSet).Next()
      pkg/executor/adapter.go:160 +0x19a
  github.com/pingcap/tidb/pkg/session.(*execStmtResult).Next()
      <autogenerated>:1 +0x6c
  github.com/pingcap/tidb/pkg/session.GetRows4Test()
      pkg/session/tidb.go:346 +0xd2
  github.com/pingcap/tidb/pkg/session.ResultSetToStringSlice()
      pkg/session/tidb.go:364 +0xa7
  github.com/pingcap/tidb/pkg/testkit.(*TestKit).ResultSetToResultWithCtx()
      pkg/testkit/testkit.go:271 +0xac
  github.com/pingcap/tidb/pkg/testkit.(*TestKit).MustQueryWithContext()
      pkg/testkit/testkit.go:197 +0x2e4
  github.com/pingcap/tidb/pkg/testkit.(*TestKit).MustQuery()
      pkg/testkit/testkit.go:172 +0x106
  github.com/pingcap/tidb/pkg/executor/join_test.checkMergeAndRun()
      pkg/executor/join/merge_join_test.go:38 +0x329
  github.com/pingcap/tidb/pkg/executor/join_test.TestShuffleMergeJoinInDisk()
      pkg/executor/join/merge_join_test.go:67 +0x61c
  github.com/pingcap/tidb/pkg/domain.(*Domain).rebuildSysVarCache()
      pkg/domain/sysvar_cache.go:145 +0x904
  github.com/pingcap/tidb/pkg/domain.(*Domain).LoadSysVarCacheLoop()
      pkg/domain/domain.go:1755 +0x93
  github.com/pingcap/tidb/pkg/session.bootstrapSessionImpl()
      pkg/session/session.go:3426 +0x864
  github.com/pingcap/tidb/pkg/domain.(*Domain).GetSessionCache()
      pkg/domain/sysvar_cache.go:61 +0x50
  github.com/pingcap/tidb/pkg/session.(*session).loadCommonGlobalVariablesIfNeeded()
      pkg/session/session.go:3794 +0x2ce
  github.com/pingcap/tidb/pkg/session.(*session).ExecuteStmt()
      pkg/session/session.go:2008 +0x184
  github.com/pingcap/tidb/pkg/session.(*session).ExecuteInternal()
      pkg/session/session.go:1519 +0x3b9
  github.com/pingcap/tidb/pkg/domain.(*Domain).LoadPrivilegeLoop()
      pkg/domain/domain.go:1698 +0x108
  github.com/pingcap/tidb/pkg/session.bootstrapSessionImpl()
      pkg/session/session.go:3419 +0x804
  github.com/pingcap/tidb/pkg/session.BootstrapSession()
      pkg/session/session.go:3318 +0xad
  github.com/pingcap/tidb/pkg/testkit.bootstrap()
      pkg/testkit/mockstore.go:254 +0x94
  github.com/pingcap/tidb/pkg/testkit.CreateMockStoreAndDomain()
      pkg/testkit/mockstore.go:225 +0xd2
  github.com/pingcap/tidb/pkg/executor/join_test.TestShuffleMergeJoinInDisk()
      pkg/executor/join/merge_join_test.go:46 +0xf9
  github.com/pingcap/failpoint.parseTerm()
      external/com_github_pingcap_failpoint/terms.go:149 +0x364
  github.com/pingcap/failpoint.parse()
      external/com_github_pingcap_failpoint/terms.go:126 +0xa5
  github.com/pingcap/failpoint.newTerms()
      external/com_github_pingcap_failpoint/terms.go:98 +0x3e
  github.com/pingcap/failpoint.(*Failpoint).Enable()
      external/com_github_pingcap_failpoint/failpoint.go:58 +0x3e
  github.com/pingcap/failpoint.(*Failpoints).Enable()
      external/com_github_pingcap_failpoint/failpoints.go:105 +0x296
  github.com/pingcap/failpoint.Enable()
      external/com_github_pingcap_failpoint/failpoints.go:255 +0x55
  github.com/pingcap/tidb/pkg/executor/join_test.TestShuffleMergeJoinInDisk()
      pkg/executor/join/merge_join_test.go:42 +0x56
  testing.tRunner()
      GOROOT/src/testing/testing.go:1595 +0x261
  testing.(*T).Run.func1()
      GOROOT/src/testing/testing.go:1648 +0x44
==================

4. What is your TiDB version? (Required)
@hawkingrei hawkingrei added the type/bug The issue is confirmed as a bug. label Jun 6, 2024
@hawkingrei hawkingrei mentioned this issue Jun 6, 2024
Open
@hawkingrei
Copy link
Member Author

https://tiprow.hawkingrei.com/view/gs/pingcapprow/logs/tidb_data_race/1798732993120964608#1:build-log.txt%3A3524

@hawkingrei
Copy link
Member Author

We cannot see it for a long time. so we close it. if we can find it again. reopen this issue.

@zimulala
Copy link
Contributor

zimulala commented Nov 18, 2024 •
edited
Loading

In addition, The stack feels like a data race due to unstable testing

testing.(*T).FailNow()
<autogenerated>:1 +0x31

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
affects-8.5 component/test may-affects-5.4 This bug maybe affects 5.4.x versions. may-affects-6.1 may-affects-6.5 may-affects-7.1 may-affects-7.5 may-affects-8.1 severity/major sig/execution SIG execution type/bug The issue is confirmed as a bug.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@hawkingrei @zimulala @jebter