Reduce micromatch overhead in jest-haste-map HasteFS #10132
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
I was profiling some Jest runs at Airbnb and noticed that on my MacBook Pro, we can spend over 30 seconds after running Jest with code coverage as the coverage reporter adds all of the untested files. I believe that this will grow as the size of the codebase increases. Looking at the call stacks, it appears to be calling micromatch repeatedly, which calls picomatch, which builds a regex out of the globs. It seems that the parsing and regex building also triggers the garbage collector frequently. Since this is in a tight loop and the globs won't change between checks, we can greatly improve the performance here by using micromatch.matcher. This optimization reduces the block of time here from about 30s to about 10s. The aggregated total time of coverage reporter's onRunComplete goes from 23s to 600ms.
|
I think this actually suffers from the same bug I addressed in #10131 Once that PR lands, I'll extract the code to a shared module and use it in both places. |
|
I'm going to fold this PR into #10131 |
|
This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I was profiling some Jest runs at Airbnb and noticed that on my
MacBook Pro, we can spend over 30 seconds after running Jest with code
coverage as the coverage reporter adds all of the untested files. I
believe that this will grow as the size of the codebase increases.
Looking at the call stacks, it appears to be calling micromatch
repeatedly, which calls picomatch, which builds a regex out of the
globs. It seems that the parsing and regex building also triggers the
garbage collector frequently.
Since this is in a tight loop and the globs won't change between
checks, we can greatly improve the performance here by using
micromatch.matcher.
This optimization reduces the block of time here from about 30s to
about 10s. The aggregated total time of coverage reporter's
onRunComplete goes from 23s to 600ms.
Before:
After:
Summary
Motivation: Improve Jest performance when collecting coverage
Test plan
I ran jest in the Airbnb frontend monorepo with and without coverage options, with a path argument.