format_tool: optimization pass #424
Conversation
We can save a lot (90%) of execution time on some repos with two changes: - Not recursing into directories that are a priori excluded from the formatter's list (e.g., hidden directories like .git). Note that this means we no longer build and log a comprehensive list of excluded files. I think it's worth it to run so much faster. - Removing the collapseDirectories feature, to be replaced with batch parallel invocations. Computing the collapsible directories was some kind of superlinear deduplication. This could maybe be optimized in a way that would allow it to remain, but it seems it was intended as a stopgap solution for argv limitations in the first place.
Security InsightsNo security relevant content was detected by automated scans. Action Items
Questions or Comments? Reach out on Slack: #support-infosec. |
|
I just pushed a commit to restore the deleted fields back onto |
|
Oops, I broke CI. I'll get that addressed today. |
|
|
||
| if (entry is Link) { | ||
| _log.finer('skipping link $relative\n'); | ||
| skippedLinks.add(relative); | ||
| _log.finer('skipping link $filename\n'); |
There was a problem hiding this comment.
Why did these checks not make it into the listSync forEach? Would they even be faster there?
There was a problem hiding this comment.
That would definitely be faster— I chose to keep this closer to the original logic for now to avoid churn. I tentatively plan to make another pass to reduce the amount of work being done in this function later.
|
@Workiva/release-management-pp |
|
Rosie doesn't require QA on this repo? :sus: |
We can save a lot (~90%) of execution time of
dart_dev formaton some repos with two changes:Not recursing into directories that are a priori excluded from the formatter's list (e.g., hidden directories like .git). Note that this means we no longer build and log a comprehensive list of excluded files. I think it's worth it to run so much faster.
Removing the
collapseDirectoriesfeature, to be replaced with batch parallel invocations. Computing the collapsible directories was some kind of superlinear deduplication. This could maybe be optimized in a way that would allow it to remain, but it seems it was intended as a stopgap solution for argv limitations in the first place.This is technically a breaking change to the API since we no longer return a full list of excluded files and hidden directories.
I have retained the
collapseDirectoriesparameter in the name of backward compatibility for now, but it's ignored and marked@deprecated.This moves the responsibility for handling argv limitations into
FormatTool.