Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

executor: fix data race in GetDirtyTable() #12767

Merged

Conversation

lzmhhh123
Copy link
Contributor

@lzmhhh123 lzmhhh123 commented Oct 16, 2019

What problem does this PR solve?

As the title says.

What is changed and how it works?

Add the lock for dirty DB.

Check List

Tests

  • Unit test
  • Integration test

Side effects

  • Possible performance regression

Related changes

  • Need to cherry-pick to the release branch

@lzmhhh123 lzmhhh123 added type/bugfix This PR fixes a bug. sig/execution SIG execution needs-cherry-pick-2.1 labels Oct 16, 2019
@codecov
Copy link

codecov bot commented Oct 16, 2019

Codecov Report

Merging #12767 into master will not change coverage.
The diff coverage is n/a.

@@             Coverage Diff             @@
##             master     #12767   +/-   ##
===========================================
  Coverage   80.0495%   80.0495%           
===========================================
  Files           465        465           
  Lines        107326     107326           
===========================================
  Hits          85914      85914           
  Misses        14966      14966           
  Partials       6446       6446

@zz-jason zz-jason changed the title executor: fix bug index join causes data race in building union scan executor: fix data race in GetDirtyTable() Oct 23, 2019
Copy link
Member

@zz-jason zz-jason left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zz-jason zz-jason added the status/LGT1 Indicates that a PR has LGTM 1. label Oct 23, 2019
// tables is a map whose key is tableID.
tables map[int64]*DirtyTable
}

// GetDirtyTable gets the DirtyTable by id from the DirtyDB.
func (udb *DirtyDB) GetDirtyTable(tid int64) *DirtyTable {
// The index join access the tables map paralelly.
// But the map throws panic in this case. So it's locked.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you also update the comment in table reader case of buildUnionScanFromReader? It is outdated now.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copy link
Contributor

@eurekaka eurekaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@eurekaka eurekaka added status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Oct 23, 2019
@sre-bot
Copy link
Contributor

sre-bot commented Oct 23, 2019

/run-all-tests

@sre-bot sre-bot merged commit 34b3c9a into pingcap:master Oct 23, 2019
@sre-bot
Copy link
Contributor

sre-bot commented Oct 23, 2019

cherry pick to release-2.1 failed

@sre-bot
Copy link
Contributor

sre-bot commented Oct 23, 2019

cherry pick to release-3.1 failed

@sre-bot
Copy link
Contributor

sre-bot commented Oct 23, 2019

cherry pick to release-3.0 failed

@lzmhhh123 lzmhhh123 deleted the bug-fix/dirty_table_map_read_write_conflict branch October 23, 2019 08:54
lzmhhh123 added a commit to lzmhhh123/tidb that referenced this pull request Oct 23, 2019
lzmhhh123 added a commit to lzmhhh123/tidb that referenced this pull request Oct 23, 2019
lzmhhh123 added a commit to lzmhhh123/tidb that referenced this pull request Oct 23, 2019
lfkdsk added a commit to JustProject/tidb that referenced this pull request Oct 26, 2019
…ect/tidb into feature-add-udf-support

* 'feature-add-udf-support' of https://github.com/JustProject/tidb: (26 commits)
  *: fix bug that the kill command doesn't work when the killed session is waiting for the pessimistic lock (pingcap#12852)
  executor: fix the projection upon the indexLookUp in indexLookUpJoin can't get result. (pingcap#12889)
  planner, executor: support create view on union (pingcap#12595)
  planner/cascades: introduce TransformationID in cascades planner (pingcap#12879)
  executor: fix data race in test (pingcap#12910)
  executor: reuse chunk row for insert on duplicate update (pingcap#12847)
  ddl: speed up tests (pingcap#12888)
  executor: speed up test (pingcap#12896)
  expression: implement vectorized evaluation for `builtinSecondSig` (pingcap#12886)
  expression: implement vectorized evaluation for `builtinJSONObjectSig` (pingcap#12663)
  expression: speed up builtinRepeatSig by using MergeNulls (pingcap#12674)
  expression: speed up unit tests under the expression package (pingcap#12887)
  store,kv: snapshot doesn't cache the non-exists kv entries lead to poor 'insert ignore' performance (pingcap#12872)
  executor: fix data race in `GetDirtyTable()` (pingcap#12767)
  domain: increase TTL to reduce the occurrence of reporting min startTS errors (pingcap#12578)
  executor: split test for speed up (pingcap#12881)
  executor: fix inconsistent of grants privileges with MySQL when executing `grant all on ...` (pingcap#12330)
  expression: implement vectorized evaluation for `builtinJSONUnquoteSig` (pingcap#12841)
  tune grpc connection count between tidb and tikv (pingcap#12884)
  Makefile: change test parallel to 8 (pingcap#12885)
  ...
XiaTianliang pushed a commit to XiaTianliang/tidb that referenced this pull request Dec 21, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sig/execution SIG execution status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. type/bugfix This PR fixes a bug.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants