tools/check: adpot staged build to make ut a little bit faster

[tiancaiamao]

What problem does this PR solve?

Issue Number: ref #31880 #30822

Problem Summary:

Maybe we can 'make ut X=build' a little bit faster!

go clean -cache
time make ut X='build'

After vs Before:

real    11m18.792s
user    106m32.408s
sys     8m57.255s

real    10m14.162s
user    98m40.710s
sys     8m16.094s

=====

real    16m2.547s
user    93m58.212s
sys     7m51.199s

real    15m37.476s
user    91m56.299s
sys     8m11.068s

What changed and how does it work?

Behind the scene, it's a technique called 'staged build'.

If we build all the test binaries parallelly, the go compiler will detect that much of the pacakge cache are not available, so it would build package cache several times. For example, building pkg/executor and pkg/session parallelly, they all depends on a lot of other small packages, since those packages do not exist at the time. The package is built repeatedly because of the parallel.

If we build all the packages first to make the cache available, and then built the test binaries, we can avoid repeated package building. This make the whole process a little bit faster.

See also

• https://incident.io/blog/go-build-faster
• https://blog.howardjohn.info/posts/go-build-times/#staggered-builds

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)

go clean -cache
time make ut X='build'

• No need to test

  • I checked and no code files have been changed.

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

[tiprow]

Hi @tiancaiamao. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[codecov]

Codecov Report

Merging #51374 (77c33bc) into master (f4e758a) will increase coverage by 2.1398%.
Report is 20 commits behind head on master.
The diff coverage is n/a.

Additional details and impacted files

@@               Coverage Diff                @@
##             master     #51374        +/-   ##
================================================
+ Coverage   70.6793%   72.8191%   +2.1398%     
================================================
  Files          1460       1462         +2     
  Lines        434615     440564      +5949     
================================================
+ Hits         307183     320815     +13632     
+ Misses       108242      99654      -8588     
- Partials      19190      20095       +905     

Flag	Coverage Δ
integration	48.8880% <ø> (?)
unit	70.3644% <ø> (-0.1102%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
dumpling	53.9957% <ø> (ø)
parser	∅ <ø> (∅)
br	51.3338% <ø> (+5.4500%)	⬆️

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Defined2014, hawkingrei

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [Defined2014,hawkingrei]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ti-chi-bot]

[LGTM Timeline notifier]

Timeline:

• 2024-02-27 14:33:00.123963647 +0000 UTC m=+972468.871586759: ☑️ agreed by Defined2014.
• 2024-02-28 02:15:45.247923781 +0000 UTC m=+1014633.995546887: ☑️ agreed by hawkingrei.