Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DATA RACE in the UpdateTableStatsLoop #33001

Closed
hawkingrei opened this issue Mar 10, 2022 · 1 comment
Closed

DATA RACE in the UpdateTableStatsLoop #33001

hawkingrei opened this issue Mar 10, 2022 · 1 comment
Labels
affects-6.0 severity/major sig/sql-infra SIG: SQL Infra type/bug The issue is confirmed as a bug.

Comments

@hawkingrei
Copy link
Member

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

==================
WARNING: DATA RACE
Read at 0x00c001193b60 by goroutine 201:
  github.com/pingcap/tidb/session.(*session).getInternalSession.func1()
      /go/tidb/session/session.go:1745 +0x426
  github.com/pingcap/tidb/session.(*session).withRestrictedSQLExecutor()
      /go/tidb/session/session.go:1777 +0x399
  github.com/pingcap/tidb/session.(*session).ExecRestrictedSQL()
      /go/tidb/session/session.go:1781 +0x124
  github.com/pingcap/tidb/statistics/handle.(*Handle).execRestrictedSQL()
      /go/tidb/statistics/handle/handle.go:132 +0x188
  github.com/pingcap/tidb/statistics/handle.(*Handle).HandleUpdateStats()
      /go/tidb/statistics/handle/update.go:731 +0x84
  github.com/pingcap/tidb/domain.(*Domain).updateStatsWorker()
      /go/tidb/domain/domain.go:1451 +0x97a
  github.com/pingcap/tidb/domain.(*Domain).UpdateTableStatsLoop·dwrap·19()
      /go/tidb/domain/domain.go:1304 +0x71
Previous write at 0x00c001193b60 by goroutine 43:
  github.com/pingcap/tidb/executor.ResetContextOfStmt()
      /go/tidb/executor/executor.go:1939 +0x2cde
  github.com/pingcap/tidb/session.(*session).ExecuteStmt()
      /go/tidb/session/session.go:1833 +0x396
  github.com/pingcap/tidb/session.(*session).ExecuteInternal()
      /go/tidb/session/session.go:1494 +0x484
  github.com/pingcap/tidb/statistics/handle.(*Handle).getStatsReader()
      /go/tidb/statistics/handle/handle.go:1449 +0x2f1
  github.com/pingcap/tidb/statistics/handle.(*Handle).getGlobalStatsReader()
      /go/tidb/statistics/handle/handle.go:1426 +0x125
  github.com/pingcap/tidb/statistics/handle.(*Handle).LoadNeededHistograms()
      /go/tidb/statistics/handle/handle.go:597 +0x6e
  github.com/pingcap/tidb/domain.(*Domain).loadStatsWorker()
      /go/tidb/domain/domain.go:1369 +0x9ba
  github.com/pingcap/tidb/domain.(*Domain).UpdateTableStatsLoop·dwrap·17()
      /go/tidb/domain/domain.go:1292 +0x39
Goroutine 201 (running) created at:
  github.com/pingcap/tidb/domain.(*Domain).UpdateTableStatsLoop()
      /go/tidb/domain/domain.go:1304 +0x470
  github.com/pingcap/tidb/session.BootstrapSession()
      /go/tidb/session/session.go:2886 +0x804
  github.com/pingcap/tidb/cmd/ddltest.createDDLSuite()
      /go/tidb/cmd/ddltest/ddl_test.go:128 +0x878
  github.com/pingcap/tidb/cmd/ddltest.TestColumn()
      /go/tidb/cmd/ddltest/column_test.go:118 +0x4a
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:1259 +0x22f
  testing.(*T).Run·dwrap·21()
      /usr/local/go/src/testing/testing.go:1306 +0x47
Goroutine 43 (running) created at:
  github.com/pingcap/tidb/domain.(*Domain).UpdateTableStatsLoop()
      /go/tidb/domain/domain.go:1292 +0x20a
  github.com/pingcap/tidb/session.BootstrapSession()
      /go/tidb/session/session.go:2886 +0x804
  github.com/pingcap/tidb/cmd/ddltest.createDDLSuite()
      /go/tidb/cmd/ddltest/ddl_test.go:128 +0x878
  github.com/pingcap/tidb/cmd/ddltest.TestColumn()
      /go/tidb/cmd/ddltest/column_test.go:118 +0x4a
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:1259 +0x22f
  testing.(*T).Run·dwrap·21()
      /usr/local/go/src/testing/testing.go:1306 +0x47
==================
==================
WARNING: DATA RACE
Write at 0x00c0011945c0 by goroutine 201:
  sync/atomic.CompareAndSwapInt32()
      /usr/local/go/src/runtime/race_amd64.s:334 +0xb
  sync/atomic.CompareAndSwapInt32()
      <autogenerated>:1 +0x1e
  github.com/pingcap/tidb/sessionctx/stmtctx.(*StatementContext).AppendWarnings()
      /go/tidb/sessionctx/stmtctx/stmtctx.go:563 +0x65
  github.com/pingcap/tidb/session.(*session).getInternalSession.func1()
      /go/tidb/session/session.go:1745 +0x449
  github.com/pingcap/tidb/session.(*session).withRestrictedSQLExecutor()
      /go/tidb/session/session.go:1777 +0x399
  github.com/pingcap/tidb/session.(*session).ExecRestrictedSQL()
      /go/tidb/session/session.go:1781 +0x124
  github.com/pingcap/tidb/statistics/handle.(*Handle).execRestrictedSQL()
      /go/tidb/statistics/handle/handle.go:132 +0x188
  github.com/pingcap/tidb/statistics/handle.(*Handle).HandleUpdateStats()
      /go/tidb/statistics/handle/update.go:731 +0x84
  github.com/pingcap/tidb/domain.(*Domain).updateStatsWorker()
      /go/tidb/domain/domain.go:1451 +0x97a
  github.com/pingcap/tidb/domain.(*Domain).UpdateTableStatsLoop·dwrap·19()
      /go/tidb/domain/domain.go:1304 +0x71
Previous write at 0x00c0011945c0 by goroutine 43:
  github.com/pingcap/tidb/sessionctx/variable.(*SessionVars).InitStatementContext()
      /go/tidb/sessionctx/variable/session.go:1019 +0x1f5
  github.com/pingcap/tidb/executor.ResetContextOfStmt()
      /go/tidb/executor/executor.go:1755 +0x13d
  github.com/pingcap/tidb/session.(*session).ExecuteStmt()
      /go/tidb/session/session.go:1833 +0x396
  github.com/pingcap/tidb/session.(*session).ExecuteInternal()
      /go/tidb/session/session.go:1494 +0x484
  github.com/pingcap/tidb/statistics/handle.(*Handle).getStatsReader()
      /go/tidb/statistics/handle/handle.go:1449 +0x2f1
  github.com/pingcap/tidb/statistics/handle.(*Handle).getGlobalStatsReader()
      /go/tidb/statistics/handle/handle.go:1426 +0x125
  github.com/pingcap/tidb/statistics/handle.(*Handle).LoadNeededHistograms()
      /go/tidb/statistics/handle/handle.go:597 +0x6e
  github.com/pingcap/tidb/domain.(*Domain).loadStatsWorker()
      /go/tidb/domain/domain.go:1369 +0x9ba
  github.com/pingcap/tidb/domain.(*Domain).UpdateTableStatsLoop·dwrap·17()
      /go/tidb/domain/domain.go:1292 +0x39
Goroutine 201 (running) created at:
  github.com/pingcap/tidb/domain.(*Domain).UpdateTableStatsLoop()
      /go/tidb/domain/domain.go:1304 +0x470
  github.com/pingcap/tidb/session.BootstrapSession()
      /go/tidb/session/session.go:2886 +0x804
  github.com/pingcap/tidb/cmd/ddltest.createDDLSuite()
      /go/tidb/cmd/ddltest/ddl_test.go:128 +0x878
  github.com/pingcap/tidb/cmd/ddltest.TestColumn()
      /go/tidb/cmd/ddltest/column_test.go:118 +0x4a
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:1259 +0x22f
  testing.(*T).Run·dwrap·21()
      /usr/local/go/src/testing/testing.go:1306 +0x47
Goroutine 43 (running) created at:
  github.com/pingcap/tidb/domain.(*Domain).UpdateTableStatsLoop()
      /go/tidb/domain/domain.go:1292 +0x20a
  github.com/pingcap/tidb/session.BootstrapSession()
      /go/tidb/session/session.go:2886 +0x804
  github.com/pingcap/tidb/cmd/ddltest.createDDLSuite()
      /go/tidb/cmd/ddltest/ddl_test.go:128 +0x878
  github.com/pingcap/tidb/cmd/ddltest.TestColumn()
      /go/tidb/cmd/ddltest/column_test.go:118 +0x4a
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:1259 +0x22f
  testing.(*T).Run·dwrap·21()
      /usr/local/go/src/testing/testing.go:1306 +0x47
==================
==================
WARNING: DATA RACE
Read at 0x00c001194610 by goroutine 201:
  github.com/pingcap/tidb/sessionctx/stmtctx.(*StatementContext).AppendWarnings()
      /go/tidb/sessionctx/stmtctx/stmtctx.go:565 +0xd0
  github.com/pingcap/tidb/session.(*session).getInternalSession.func1()
      /go/tidb/session/session.go:1745 +0x449
  github.com/pingcap/tidb/session.(*session).withRestrictedSQLExecutor()
      /go/tidb/session/session.go:1777 +0x399
  github.com/pingcap/tidb/session.(*session).ExecRestrictedSQL()
      /go/tidb/session/session.go:1781 +0x124
  github.com/pingcap/tidb/statistics/handle.(*Handle).execRestrictedSQL()
      /go/tidb/statistics/handle/handle.go:132 +0x188
  github.com/pingcap/tidb/statistics/handle.(*Handle).HandleUpdateStats()
      /go/tidb/statistics/handle/update.go:731 +0x84
  github.com/pingcap/tidb/domain.(*Domain).updateStatsWorker()
      /go/tidb/domain/domain.go:1451 +0x97a
  github.com/pingcap/tidb/domain.(*Domain).UpdateTableStatsLoop·dwrap·19()
      /go/tidb/domain/domain.go:1304 +0x71
Previous write at 0x00c001194610 by goroutine 43:
  github.com/pingcap/tidb/sessionctx/variable.(*SessionVars).InitStatementContext()
      /go/tidb/sessionctx/variable/session.go:1019 +0x1f5
  github.com/pingcap/tidb/executor.ResetContextOfStmt()
      /go/tidb/executor/executor.go:1755 +0x13d
  github.com/pingcap/tidb/session.(*session).ExecuteStmt()
      /go/tidb/session/session.go:1833 +0x396
  github.com/pingcap/tidb/session.(*session).ExecuteInternal()
      /go/tidb/session/session.go:1494 +0x484
  github.com/pingcap/tidb/statistics/handle.(*Handle).getStatsReader()
      /go/tidb/statistics/handle/handle.go:1449 +0x2f1
  github.com/pingcap/tidb/statistics/handle.(*Handle).getGlobalStatsReader()
      /go/tidb/statistics/handle/handle.go:1426 +0x125
  github.com/pingcap/tidb/statistics/handle.(*Handle).LoadNeededHistograms()
      /go/tidb/statistics/handle/handle.go:597 +0x6e
  github.com/pingcap/tidb/domain.(*Domain).loadStatsWorker()
      /go/tidb/domain/domain.go:1369 +0x9ba
  github.com/pingcap/tidb/domain.(*Domain).UpdateTableStatsLoop·dwrap·17()
      /go/tidb/domain/domain.go:1292 +0x39
Goroutine 201 (running) created at:
  github.com/pingcap/tidb/domain.(*Domain).UpdateTableStatsLoop()
      /go/tidb/domain/domain.go:1304 +0x470
  github.com/pingcap/tidb/session.BootstrapSession()
      /go/tidb/session/session.go:2886 +0x804
  github.com/pingcap/tidb/cmd/ddltest.createDDLSuite()
      /go/tidb/cmd/ddltest/ddl_test.go:128 +0x878
  github.com/pingcap/tidb/cmd/ddltest.TestColumn()
      /go/tidb/cmd/ddltest/column_test.go:118 +0x4a
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:1259 +0x22f
  testing.(*T).Run·dwrap·21()
      /usr/local/go/src/testing/testing.go:1306 +0x47
Goroutine 43 (running) created at:
  github.com/pingcap/tidb/domain.(*Domain).UpdateTableStatsLoop()
      /go/tidb/domain/domain.go:1292 +0x20a
  github.com/pingcap/tidb/session.BootstrapSession()
      /go/tidb/session/session.go:2886 +0x804
  github.com/pingcap/tidb/cmd/ddltest.createDDLSuite()
      /go/tidb/cmd/ddltest/ddl_test.go:128 +0x878
  github.com/pingcap/tidb/cmd/ddltest.TestColumn()
      /go/tidb/cmd/ddltest/column_test.go:118 +0x4a
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:1259 +0x22f
  testing.(*T).Run·dwrap·21()
      /usr/local/go/src/testing/testing.go:1306 +0x47
==================

2. What did you expect to see? (Required)

3. What did you see instead (Required)

4. What is your TiDB version? (Required)
@hawkingrei hawkingrei added the type/bug The issue is confirmed as a bug. label Mar 10, 2022
@hawkingrei
Copy link
Member Author

https://prow.hawkingrei.com/view/gs/prowpingcap/logs/tidb_race/1501951228630798336#1:build-log.txt%3A2171

@ti-chi-bot ti-chi-bot added may-affects-4.0 This bug maybe affects 4.0.x versions. may-affects-5.0 This bug maybe affects 5.0.x versions. may-affects-5.1 This bug maybe affects 5.1.x versions. may-affects-5.2 This bug maybe affects 5.2.x versions. may-affects-5.3 This bug maybe affects 5.3.x versions. may-affects-5.4 This bug maybe affects 5.4.x versions. labels Mar 11, 2022
@ChenPeng2013 ChenPeng2013 removed may-affects-4.0 This bug maybe affects 4.0.x versions. may-affects-5.1 This bug maybe affects 5.1.x versions. may-affects-5.2 This bug maybe affects 5.2.x versions. may-affects-5.3 This bug maybe affects 5.3.x versions. may-affects-5.4 This bug maybe affects 5.4.x versions. may-affects-5.0 This bug maybe affects 5.0.x versions. labels Mar 11, 2022
@hawkingrei hawkingrei added sig/sql-infra SIG: SQL Infra and removed component/test labels Mar 11, 2022
@hawkingrei hawkingrei mentioned this issue Mar 20, 2022
Merged
12 tasks
ti-chi-bot pushed a commit that referenced this issue Mar 25, 2022
@xuyifangreeneyes
statistics: fix data race in HandleUpdateStats (#33260)
9b9051f 
ref #32867, ref #33001, close #33108
@ti-srebot ti-srebot mentioned this issue Mar 25, 2022
Closed
12 tasks
@hawkingrei hawkingrei mentioned this issue Apr 2, 2022
Closed
12 tasks
@xuyifangreeneyes xuyifangreeneyes mentioned this issue Apr 6, 2022
Merged
12 tasks
ti-chi-bot pushed a commit that referenced this issue Apr 13, 2022
@xuyifangreeneyes
session, statistics, util: fix data race of Handle.mu.ctx (#33732)
9c836a5 
ref #32822, close #32867, ref #33001
@ti-srebot ti-srebot mentioned this issue Apr 13, 2022
Closed
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
affects-6.0 severity/major sig/sql-infra SIG: SQL Infra type/bug The issue is confirmed as a bug.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

domain: fix data race in the UpdateTableStatsLoop hawkingrei/tidb
4 participants
@hawkingrei @ChenPeng2013 @VelocityLight @ti-chi-bot