Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tidb occurred fatal error: concurrent map iteration #38994

Closed
AkiraXie opened this issue Nov 9, 2022 · 1 comment · Fixed by #38998
Closed

tidb occurred fatal error: concurrent map iteration #38994

AkiraXie opened this issue Nov 9, 2022 · 1 comment · Fixed by #38998
Labels
affects-6.4 may-affects-4.0 This bug maybe affects 4.0.x versions. may-affects-5.0 This bug maybe affects 5.0.x versions. may-affects-5.1 This bug maybe affects 5.1.x versions. may-affects-5.2 This bug maybe affects 5.2.x versions. may-affects-5.3 This bug maybe affects 5.3.x versions. may-affects-5.4 This bug maybe affects 5.4.x versions. may-affects-6.0 may-affects-6.1 may-affects-6.2 may-affects-6.3 severity/critical sig/execution SIG execution type/bug The issue is confirmed as a bug.

Comments

@AkiraXie
Copy link

AkiraXie commented Nov 9, 2022

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

  1. running ch workload:/go-tpc ch run --host *** -P 4000 --warehouses 2000 -T 16 -D longrun_tpcc --acThreads 0 --ignore-errorand randomly execute query in q1,q2,q3,q4,q5,q6,q7,q8,q10,q11,q12,q13,q14,q15,q16,q17,q18,q19,q20,q21,q22,q23

2. What did you expect to see? (Required)

  1. running with no error

3. What did you see instead (Required)

fatal error: concurrent map iteration and map write goroutine 208394 [running]: github.com/pingcap/tidb/util/execdetails.(*CopRuntimeStats).MergeBasicStats(...) /home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/util/execdetails/execdetails.go:400 github.com/pingcap/tidb/util/execdetails.(*CopRuntimeStats).String(0xc1a8592600) /home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/util/execdetails/execdetails.go:417 +0xf1 github.com/pingcap/tidb/planner/core.getRuntimeInfoStr({0x4eb7f88?, 0xc06d83aa00?}, {0x4e6ed30?, 0xc1a7272000?}, 0xc1a7272000?) /home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/planner/core/common_plans.go:842 +0x145 github.com/pingcap/tidb/planner/core.(*Explain).prepareOperatorInfo(0xc3cf154f28, {0x4e6ed30, 0xc1a7272000}, {0xc2d4fd7f57, 0x9}, {0xc167fa3410, 0x21}) /home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/planner/core/common_plans.go:895 +0x3c9 github.com/pingcap/tidb/planner/core.(*Explain).explainFlatOpInRowFormat(0xc3cf154f28, 0xc350197f80) /home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/planner/core/common_plans.go:815 +0x21d github.com/pingcap/tidb/planner/core.(*Explain).explainFlatPlanInRowFormat(0x4e6e970?, 0xc48857ec80) /home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/planner/core/common_plans.go:796 +0x5f github.com/pingcap/tidb/planner/core.(*Explain).RenderResult(0xc3cf154f28) /home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/planner/core/common_plans.go:763 +0xd4e github.com/pingcap/tidb/planner/core.GetExplainAnalyzeRowsForPlan(0xc3cf154f28) /home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/planner/core/common_plans.go:666 +0x2f github.com/pingcap/tidb/session.(*session).getCurrentAnalyzePlan(0xc06d83aa00, {0x46b60c0?, 0xc1a6f7bb80}, 0xc13b707998) /home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/session/session.go:1639 +0xd7 github.com/pingcap/tidb/util/memoryusagealarm.getCurrentAnalyzePlan(0x6b15e9f296b?) /home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/util/memoryusagealarm/memoryusagealarm.go:249 +0x53 github.com/pingcap/tidb/util/memoryusagealarm.(*memoryUsageAlarm).getTop10SqlInfo(0xc3cf155f20, 0x0?, {0xc3d0e2cc00?, 0x1, 0x36?}) /home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/util/memoryusagealarm/memoryusagealarm.go:291 +0x9e6 github.com/pingcap/tidb/util/memoryusagealarm.(*memoryUsageAlarm).getTop10SqlInfoByCostTime(0x0?, {0xc3d0e2cc00?, 0x36?, 0x0?}) /home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/util/memoryusagealarm/memoryusagealarm.go:317 +0x2e github.com/pingcap/tidb/util/memoryusagealarm.(*memoryUsageAlarm).printTop10SqlInfo(0xc3d0e1e140?, {0xc3d0e2cc00, 0x1, 0xb}, 0xc054f3d840?) /home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/util/memoryusagealarm/memoryusagealarm.go:268 +0x50a github.com/pingcap/tidb/util/memoryusagealarm.(*memoryUsageAlarm).recordSQL(0xc098d4b800?, {0x4e700e0?, 0xc01e038800?}, {0xc098d4b800, 0x3c}) /home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/util/memoryusagealarm/memoryusagealarm.go:342 +0x43a github.com/pingcap/tidb/util/memoryusagealarm.(*memoryUsageAlarm).doRecord(0xc3cf155f20, 0x4385310d0?, 0xffffffffffffffff?, {0x4e700e0, 0xc01e038800}, 0x1) /home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/util/memoryusagealarm/memoryusagealarm.go:228 +0x103e github.com/pingcap/tidb/util/memoryusagealarm.(*memoryUsageAlarm).alarm4ExcessiveMemUsage(0xc3cf155f20, {0x4e700e0, 0xc01e038800}) /home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/util/memoryusagealarm/memoryusagealarm.go:165 +0x18d github.com/pingcap/tidb/util/memoryusagealarm.(*Handle).Run(0xc001166780) /home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/util/memoryusagealarm/memoryusagealarm.go:66 +0x113 created by main.createServer /home/jenkins/agent/workspa ce/build-common/go/src/github.com/pingcap/tidb/tidb-server/main.go:761 +0x4d0 ...
tidb5err-1109.log

4. What is your TiDB version? (Required)

v6.4.0

@AkiraXie AkiraXie added the type/bug The issue is confirmed as a bug. label Nov 9, 2022
@AkiraXie
Copy link
Author

AkiraXie commented Nov 9, 2022

/severity major

@ti-chi-bot ti-chi-bot added severity/major may-affects-4.0 This bug maybe affects 4.0.x versions. may-affects-5.0 This bug maybe affects 5.0.x versions. may-affects-5.1 This bug maybe affects 5.1.x versions. may-affects-5.2 This bug maybe affects 5.2.x versions. may-affects-5.3 This bug maybe affects 5.3.x versions. may-affects-5.4 This bug maybe affects 5.4.x versions. may-affects-6.0 may-affects-6.1 may-affects-6.2 may-affects-6.3 labels Nov 9, 2022
@jebter jebter added the sig/execution SIG execution label Nov 9, 2022
@hawkingrei hawkingrei added affects-6.4 and removed may-affects-4.0 This bug maybe affects 4.0.x versions. may-affects-5.1 This bug maybe affects 5.1.x versions. may-affects-5.2 This bug maybe affects 5.2.x versions. may-affects-5.3 This bug maybe affects 5.3.x versions. may-affects-5.4 This bug maybe affects 5.4.x versions. may-affects-5.0 This bug maybe affects 5.0.x versions. may-affects-6.0 may-affects-6.1 may-affects-6.2 may-affects-6.3 labels Nov 9, 2022
ti-chi-bot pushed a commit that referenced this issue Nov 10, 2022
@ti-chi-bot ti-chi-bot added may-affects-4.0 This bug maybe affects 4.0.x versions. may-affects-5.0 This bug maybe affects 5.0.x versions. may-affects-5.1 This bug maybe affects 5.1.x versions. may-affects-5.2 This bug maybe affects 5.2.x versions. may-affects-5.3 This bug maybe affects 5.3.x versions. may-affects-5.4 This bug maybe affects 5.4.x versions. may-affects-6.0 may-affects-6.1 may-affects-6.2 may-affects-6.3 labels Nov 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects-6.4 may-affects-4.0 This bug maybe affects 4.0.x versions. may-affects-5.0 This bug maybe affects 5.0.x versions. may-affects-5.1 This bug maybe affects 5.1.x versions. may-affects-5.2 This bug maybe affects 5.2.x versions. may-affects-5.3 This bug maybe affects 5.3.x versions. may-affects-5.4 This bug maybe affects 5.4.x versions. may-affects-6.0 may-affects-6.1 may-affects-6.2 may-affects-6.3 severity/critical sig/execution SIG execution type/bug The issue is confirmed as a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants