tools/check/ut: support pattern matching on test names #33020

tangenta · 2022-03-13T06:07:13Z

What problem does this PR solve?

Issue Number: ref #30822

Problem Summary: In the past, we can only specify one test case to run at a time. It is kind of inconvenient when there are a few tests, for example:

TestColumnTypeChangeBetweenInteger
TestColumnTypeChangeStateBetweenInteger
TestColumnTypeChangeFromIntegerToOthers
...

We have to run these tests one by one.

What is changed and how it works?

$ ./tools/bin/ut list ddl 'TestColumnTypeChange.*'

TestColumnTypeChangeBetweenInteger
TestColumnTypeChangeStateBetweenInteger
TestColumnTypeChangeFromIntegerToOthers
TestColumnTypeChangeBetweenVarcharAndNonVarchar
TestColumnTypeChangeFromStringToOthers
TestColumnTypeChangeFromNumericToOthers
TestColumnTypeChangeIgnoreDisplayLength
TestColumnTypeChangeFromDateTimeTypeToOthers
TestColumnTypeChangeFromJsonToOthers
TestColumnTypeChangeBetweenFloatAndDouble
TestColumnTypeChangeTimestampToInt
testSerialDBSuite.TestColumnTypeChangeGenUniqueChangingName

$ ./tools/bin/ut run ddl 'TestColumnTypeChange.*'

2022/03/13 13:49:46 maxprocs: Leaving GOMAXPROCS=10: CPU quota undefined
building task finish, count=12, takes=29.051195667s
run all tasks takes 50.588586459s

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code

Side effects

Performance regression: Consumes more CPU
Performance regression: Consumes more Memory
Breaking backward compatibility

Documentation

Release note

None

ti-chi-bot · 2022-03-13T06:07:15Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

hawkingrei
tiancaiamao

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

sre-bot · 2022-03-13T06:15:56Z

Code Coverage Details: https://codecov.io/github/pingcap/tidb/commit/eb6465b6676ea834acff27510a5d0b54f482e2d6

tiancaiamao

I've planned to add the regex support, but found it's easy to match something unexpected, so I gave up...

For example, there are

TestColumnTypeChangeXXX
TestColumnTypeChangeYYY
testSerialDBSuite.TestColumnTypeChangeZZZ

where what I really mean '^TestColumnTypeChangeXXX'

Or ... what I mean is just the single function TestColumnTypeChangeYYY...
After supporting regex, it gives me the whole funcitons listed above.

Well... it doesn't matter, as long as one write the regexp exactly correct.

tangenta · 2022-03-14T02:55:22Z

@tiancaiamao How about using a prefix like "r:" to recognize whether it should match with regex?

tangenta · 2022-03-14T07:54:41Z

/hold

tiancaiamao · 2022-03-15T06:42:06Z

@tiancaiamao How about using a prefix like "r:" to recognize whether it should match with regex?

That's a good idea~

tangenta · 2022-03-21T02:39:59Z

/unhold

tangenta · 2022-03-22T09:42:18Z

The data race is unrelated to this PR:

[2022-03-21T02:52:25.366Z] ==================
[2022-03-21T02:52:25.366Z] WARNING: DATA RACE
[2022-03-21T02:52:25.366Z] Read at 0x00c00611c038 by goroutine 74:
[2022-03-21T02:52:25.370Z]   github.com/pingcap/tidb/domain/infosync.(*InfoSyncer).ReportMinStartTS()
[2022-03-21T02:52:25.370Z]       /home/jenkins/agent/workspace/tidb_ghpr_check_2/go/src/github.com/pingcap/tidb/domain/infosync/info.go:610 +0x68
[2022-03-21T02:52:25.370Z]   github.com/pingcap/tidb/domain.(*Domain).infoSyncerKeeper()
[2022-03-21T02:52:25.370Z]       /home/jenkins/agent/workspace/tidb_ghpr_check_2/go/src/github.com/pingcap/tidb/domain/domain.go:517 +0x304
[2022-03-21T02:52:25.370Z]   github.com/pingcap/tidb/domain.(*Domain).Init.func12()
[2022-03-21T02:52:25.370Z]       /home/jenkins/agent/workspace/tidb_ghpr_check_2/go/src/github.com/pingcap/tidb/domain/domain.go:868 +0x39
[2022-03-21T02:52:25.370Z] 
[2022-03-21T02:52:25.370Z] Previous write at 0x00c00611c038 by main goroutine:
[2022-03-21T02:52:25.370Z]   github.com/pingcap/tidb/domain/infosync.(*InfoSyncer).SetSessionManager()
[2022-03-21T02:52:25.370Z]       /home/jenkins/agent/workspace/tidb_ghpr_check_2/go/src/github.com/pingcap/tidb/domain/infosync/info.go:213 +0x593
[2022-03-21T02:52:25.370Z]   main.createServer()
[2022-03-21T02:52:25.370Z]       /home/jenkins/agent/workspace/tidb_ghpr_check_2/go/src/github.com/pingcap/tidb/tidb-server/main.go:695 +0x58a
[2022-03-21T02:52:25.370Z]   main.main()
[2022-03-21T02:52:25.370Z]       /home/jenkins/agent/workspace/tidb_ghpr_check_2/go/src/github.com/pingcap/tidb/tidb-server/main.go:205 +0x604
[2022-03-21T02:52:25.370Z] 
[2022-03-21T02:52:25.370Z] Goroutine 74 (running) created at:
[2022-03-21T02:52:25.370Z]   github.com/pingcap/tidb/domain.(*Domain).Init()
[2022-03-21T02:52:25.370Z]       /home/jenkins/agent/workspace/tidb_ghpr_check_2/go/src/github.com/pingcap/tidb/domain/domain.go:868 +0x1315
[2022-03-21T02:52:25.370Z]   github.com/pingcap/tidb/session.(*domainMap).Get.func1()
[2022-03-21T02:52:25.370Z]       /home/jenkins/agent/workspace/tidb_ghpr_check_2/go/src/github.com/pingcap/tidb/session/tidb.go:83 +0x664
[2022-03-21T02:52:25.370Z]   github.com/pingcap/tidb/util.RunWithRetry()
[2022-03-21T02:52:25.370Z]       /home/jenkins/agent/workspace/tidb_ghpr_check_2/go/src/github.com/pingcap/tidb/util/misc.go:65 +0x93
[2022-03-21T02:52:25.370Z]   github.com/pingcap/tidb/session.(*domainMap).Get()
[2022-03-21T02:52:25.370Z]       /home/jenkins/agent/workspace/tidb_ghpr_check_2/go/src/github.com/pingcap/tidb/session/tidb.go:71 +0x38b
[2022-03-21T02:52:25.370Z]   github.com/pingcap/tidb/session.createSessionWithOpt()
[2022-03-21T02:52:25.370Z]       /home/jenkins/agent/workspace/tidb_ghpr_check_2/go/src/github.com/pingcap/tidb/session/session.go:2958 +0x75
[2022-03-21T02:52:25.370Z]   github.com/pingcap/tidb/session.createSession()
[2022-03-21T02:52:25.370Z]       /home/jenkins/agent/workspace/tidb_ghpr_check_2/go/src/github.com/pingcap/tidb/session/session.go:2954 +0x246
[2022-03-21T02:52:25.370Z]   github.com/pingcap/tidb/session.BootstrapSession()
[2022-03-21T02:52:25.370Z]       /home/jenkins/agent/workspace/tidb_ghpr_check_2/go/src/github.com/pingcap/tidb/session/session.go:2791 +0x22f
[2022-03-21T02:52:25.370Z]   main.createStoreAndDomain()
[2022-03-21T02:52:25.370Z]       /home/jenkins/agent/workspace/tidb_ghpr_check_2/go/src/github.com/pingcap/tidb/tidb-server/main.go:298 +0x113
[2022-03-21T02:52:25.370Z]   main.main()
[2022-03-21T02:52:25.370Z]       /home/jenkins/agent/workspace/tidb_ghpr_check_2/go/src/github.com/pingcap/tidb/tidb-server/main.go:204 +0x5e9
[2022-03-21T02:52:25.370Z] ==================

tangenta · 2022-03-22T09:42:27Z

/merge

ti-chi-bot · 2022-03-22T09:42:29Z

This pull request has been accepted and is ready to merge.

Commit hash: ca370ea

tiancaiamao · 2022-03-22T10:10:54Z

You can create a issue for the data race @tangenta

ti-chi-bot · 2022-03-22T10:48:33Z

@tangenta: Your PR was out of date, I have automatically updated it for you.

At the same time I will also trigger all tests for you:

/run-all-tests

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

tangenta · 2022-03-23T03:26:58Z

/merge

tangenta · 2022-03-23T03:28:00Z

Here is the data race issue: #33335

tools/check/ut: support pattern matching on test names

d53c84d

ti-chi-bot added release-note-none Denotes a PR that doesn't merit a release note. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Mar 13, 2022

support list tests pattern match

0dae204

ti-chi-bot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Mar 13, 2022

tangenta requested a review from tiancaiamao March 13, 2022 06:08

tiancaiamao approved these changes Mar 14, 2022

View reviewed changes

ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Mar 14, 2022

hawkingrei approved these changes Mar 14, 2022

View reviewed changes

ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Mar 14, 2022

ti-chi-bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 14, 2022

tangenta added 2 commits March 21, 2022 10:38

ddl: use prefix 'r:' for pattern matching

c33e70f

Merge remote-tracking branch 'upstream/master' into ut-regex

ca370ea

ti-chi-bot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Mar 21, 2022

ti-chi-bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 21, 2022

tangenta requested review from tiancaiamao and hawkingrei March 21, 2022 02:40

tiancaiamao approved these changes Mar 22, 2022

View reviewed changes

ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Mar 22, 2022

ti-chi-bot added 2 commits March 22, 2022 17:42

Merge branch 'master' into ut-regex

8d20e24

Merge branch 'master' into ut-regex

faee08a

Merge branch 'master' into ut-regex

eb6465b

tangenta closed this Mar 23, 2022

tangenta reopened this Mar 23, 2022

ti-chi-bot merged commit 23d79d4 into pingcap:master Mar 23, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

tools/check/ut: support pattern matching on test names #33020

tools/check/ut: support pattern matching on test names #33020

tangenta commented Mar 13, 2022

ti-chi-bot commented Mar 13, 2022 •

edited

Loading

sre-bot commented Mar 13, 2022 •

edited

Loading

tiancaiamao left a comment

tangenta commented Mar 14, 2022

tangenta commented Mar 14, 2022

tiancaiamao commented Mar 15, 2022

tangenta commented Mar 21, 2022

tangenta commented Mar 22, 2022

tangenta commented Mar 22, 2022

ti-chi-bot commented Mar 22, 2022

tiancaiamao commented Mar 22, 2022

ti-chi-bot commented Mar 22, 2022

tangenta commented Mar 23, 2022

tangenta commented Mar 23, 2022

tools/check/ut: support pattern matching on test names #33020

tools/check/ut: support pattern matching on test names #33020

Conversation

tangenta commented Mar 13, 2022

What problem does this PR solve?

What is changed and how it works?

Check List

Release note

ti-chi-bot commented Mar 13, 2022 • edited Loading

sre-bot commented Mar 13, 2022 • edited Loading

tiancaiamao left a comment

Choose a reason for hiding this comment

tangenta commented Mar 14, 2022

tangenta commented Mar 14, 2022

tiancaiamao commented Mar 15, 2022

tangenta commented Mar 21, 2022

tangenta commented Mar 22, 2022

tangenta commented Mar 22, 2022

ti-chi-bot commented Mar 22, 2022

tiancaiamao commented Mar 22, 2022

ti-chi-bot commented Mar 22, 2022

tangenta commented Mar 23, 2022

tangenta commented Mar 23, 2022

ti-chi-bot commented Mar 13, 2022 •

edited

Loading

sre-bot commented Mar 13, 2022 •

edited

Loading