-
-
Notifications
You must be signed in to change notification settings - Fork 32.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[core] Flatten imports to speed up webpack build & node resolution #35840
Comments
@oliviertassinari this issue isn't specific to icons, please see "Demo App 3":
|
We could enforce a few rules to improve the amount of dependencies listed. |
@flaviendelangle that would be great! |
I think Olivier added the icon label because it is the scenario with the most obvious gains. By the way, on the X components (data grid and pickers), we are probably even worse than that. |
Thanks for such a detailed report, @anthonyalayo! I agree with @flaviendelangle, we can start with creating an eslint rule to disallow imports from barrel files. It could be better than introducing a build transform, as anyone reading the source code will see the proper way of importing other modules. @anthonyalayo, would you be interested in working on this? |
@michaldudak thanks for going through it 😄 sure I'm interested, but I wanted to hit on that point you just mentioned:
I considered this (as it's the easier way to go code change wise), but I also noted in the proposed solution that the developer experience would be affected. A tangential discussion happened on a Next.js issue, and the audience was quite in favor of barrel files: vercel/next.js#12557 (comment) On top of that, Next.js recently released
In the absence of these features, I think the only solution would be an eslint rule to reject barrel file usage. But since we already are somewhat across the goal post for good DX and performance, why not take it all the way? |
By "taking it all the way" you mean fixing the issues that prevent babel and modularizeImports transforms to work, right? I haven't looked into this much, but I fear we won't be able to change the structure of our packages without introducing breaking changes. We can certainly look into it for the next release, as we're going to change how things are imported anyway (by introducing ESM and import maps) |
@michaldudak agreed, it would be a breaking change (since some import locations would move to be consistent), so I think looking into it for the next release sounds reasonable to me. With that being said, I can definitely do the eslint rule and import fixes associated with it. I'll make a PR for it an attach it to this issue. |
@michaldudak I did an initial attempt, but there's quite a bit of eslint configs/overrides setup already with I also noticed that
Is there anyone at @mui that could join in the conversation for how they would like it ideally? |
@michaldudak bumping the above message in case it got missed |
Unfortunately, cc @mui/code-infra for visibility and perhaps other opinions |
Sounds good, If no other opinions from @mui/code-infra i'll do that then |
@anthonyalayo I think that the number of modules isn't this relevant. We should focus more on the metrics that directly impact developers:
I have added the icon's label because so far, I don't think that it was ever proven that the problem goes beyond icons. https://mui.com/material-ui/guides/minimizing-bundle-size/#development-environment mentions a build time of x6 for icons, but for a button, back then, it was like 50%, mostly negligible. It could be great to measure again, it was a long time ago.
👍 agree, to keep doing it (I think that we started doing this a long time ago). |
I've been looking into what I think is the same root issue: I'm trying to speed up our Jest test suite. Jest by default executes each of its suites in an isolated Node context, which means all of the tests' dependencies have to be re-loaded, parsed, and executed for every test module. Tree-shaking doesn't apply, and Babel typically isn't run on node_modules for Jest (so babel-plugin-import or babel-plugin-direct-import won't help), and the costs are incurred on every test execution (in watch mode, every time a file is saved, versus just when webpack-dev-server is starting up). MUI packages' use of barrel imports from other MUI packages seems to have a noticeable impact here, so I'm interested in this area of work as well. Should If this isn't the same issue or would be better tracked separately, please let me know. |
This issue is up for grabs for the community. It should be relatively easy to make progress. It's a matter of have deeper imports, avoiding barrel index files. |
Has the change to avoid importing directly from the root of I'm experiencing issues with exports as mentioned in the issue description. By merely importing an icon with `import Abc from '@mui/icons-material/Abc', the build time in a simple React + Webpack app increased from 3 seconds to 17 because Material adds roughly 500 internal packages. "@emotion/react": "11.10.6", Stuck at "react": "17.0.2" 😅 |
@anthonyalayo or if anyone is still interested in tackling this. I believe a good starting point would be to update diff --git a/.eslintrc.js b/.eslintrc.js
index 038be57d26..c178890cb3 100644
--- a/.eslintrc.js
+++ b/.eslintrc.js
@@ -457,7 +457,17 @@ module.exports = {
'no-restricted-imports': [
'error',
{
- paths: NO_RESTRICTED_IMPORTS_PATHS_TOP_LEVEL_PACKAGES,
+ paths: [
+ ...NO_RESTRICTED_IMPORTS_PATHS_TOP_LEVEL_PACKAGES,
+ {
+ name: '@mui/system',
+ message: OneLevelImportMessage,
+ },
+ {
+ name: '@mui/utils',
+ message: OneLevelImportMessage,
+ },
+ ],
},
],
// TODO: Consider setting back to `ignoreExternal: true` when the expected behavior is fixed: And then address the ~250 warnings this generates. Most of them will be about rewriting things from e.g. import { getDisplayName } from '@mui/utils'; to import getDisplayName from '@mui/utils/getDisplayName'; Some of them may be a bit more challenging such as import { Interpolation } from '@mui/system'; There we may first have to push those re-exports deeper in import { Interpolation } from '@mui/system/styled'; |
I'm working on this, but firstly, it's good to establish the ideal end state. The imports in the MUI codebase are a hot mess of anti-patterns…
Some of the above anti-patterns I have listed how tooling can be used to enforce good patterns, but others I still have to figure out the right ESLint/TS config. |
This is a test file, it's not inside the package I agree with some of the proposals (sorting the imports is something I would love to for quite some time but never took the effort to do so. For others I have by doubts tbh |
It's a module within a package boundary; it just happens to not be published.
Sorting imports has the least functional improvement of all the recommendations. What do you doubt about the other points? Do you just doubt some, but not all points? |
Then I'm not following what is the point of your whole post.
I have doubts about the viability of something like |
@jaydenseric Thanks for looking into it! On the labeling for "hot mess of anti-patterns", for each point:
|
I tend to disagree on that one, I find that codebase with organized imports are more readable and maintainable, and this is easily handled by ESLint or even Prettier. But for me it's totally out of the scope of this issue though |
@flaviendelangle we used to sort all the imports on the codebase. I think that we should continue to do it. We would sort them in 3 buckets:
However, under "hot mess of anti-patterns" I disagree, I think it would be a distraction to work on this first. I also fail to see how its not already sorted in the example provided. Meaning, why would it be better with a different sorting?
👍 |
Regarding (1): It's very easy to autofix this project wide with ESLint, it improves quality so I don't see why not to enforce it. It makes code much easier to read and understand what imports are types and what represents runtime code. I don't feel a sense of urgency about actioning this because it can be done by anyone pretty easily; I only have a week right now to contribute my expertise for optimal module design to hopefully solve a bunch of problems we have been experiencing with MUI for some time now at work. We want it to work in native Node.js ESM projects, and we have been having a lot of issues with absurd bundle sizes for deep imports of specific components due to all the suboptimal internal imports via barrel files, etc. We have unit tests in our design system that assert the esbuild bundle size per component our UI library exports, and due to the way MUI is structured, regularly they expand for seemingly no reason because of how much unrelated code gets sucked in.
Firstly, the MUI codebase sometimes relative imports the main thing being tested, other times it uses the package name. There is no tooling enforcing consistency. Secondly, any modules authored in a Node.js project should conform to the Node.js resolution rules. Here are the rules for self referencing package imports: https://nodejs.org/api/packages.html#self-referencing-a-package-using-its-name For ESM, it's invalid unless the package has an exports field and what is being imported is exported. At this time, the packages do not have an material-ui/packages/mui-utils/package.json Lines 1 to 80 in 1c7f61f
Thirdly, even if in the future the packages were to have an
Finally, it's best for unit tests to be co-located next to the module being tested, with a relative import of the sibling module. You can move those modules together to a different directory or even package and you don't need to rewrite the import path. You can decide to change if the module is public or private via the package exports field without having to make code changes to module imports within the package. Conceptually, a unit test should only be testing that unit. What you are trying to do, is test the package itself, which composes such units. As such, you should have package level tests for that. Regarding (5):
It's always better to write valid ESM instead of using exotic tooling configurations to allow you to write invalid code in source modules. It's better for contributors to understand (and enforce with tooling) one set of rules for published code and source code. Regarding (6):
We can agree to disagree on this, and have it your way because it doesn't cause misery in MUI user's projects like the other issues do.
I don't massively care how the imports are deterministically ordered (although I have personal preferences which align with the default for Again, I don't feel urgency to push this like the other issues, and it could be actioned later very easily with a big ESLint autofix :) |
I will be curious to get code-infra view on this.
@jaydenseric Ok, so more about #43938
I don't know. It also feels that it's extra work for maintainers without clear DX value. The UX value of smaller bundle export makes sense, but it's more about being type aware when transpiling.
We didn't complet this migration yet.
We don't use Node.js resolution directly nor did we try to. Babel operates in between and rewrite those imports.
It definitely has its limits, but npm pack issue is workedaround by following stronge conventions, same for the exports package.js field. As for the different import ways, we only recommend the one level import, so only uses this one. Both are aliased.
I don't know about this. If Node.js default resolution is poor DX, and we have to transpile the source anyway, then I don't see the value.
It sound like how material-ui-pickers source was configured. I disabled his rule, it was a lot of busy work. Eslint is not an acceptable solution for this IMHO. It would at least need to be Prettier based so it's automated, but even then, it seems not important enough for developers to have to be distracted by this. |
What's the problem? 🤔
Background
While playing around with Next.js, I installed a package that was using @mui. I immediately noticed huge delays running
next dev
and the large module count (10K+) listed in the terminal. Searching for an answer, I landed on a long list of requests for help:But none of those had an actual answer with an actual solution, just guesses and workarounds. One of the popular answers was: https://mui.com/material-ui/guides/minimizing-bundle-size/
After attempting to fix the library I pulled in by following the first option on that guide:
https://mui.com/material-ui/guides/minimizing-bundle-size/#option-one-use-path-imports
I noticed that the module count was still in the thousands. Why?
After reading everything on webpack, modules, tree shaking, etc, I made a demo application using CRA and a single import to @mui to showcase the problem.
Problem
Reproduction Steps
npx create-react-app cra-test
to get the latest with Webpack 5I used the following configurations for
stats.toJson()
to balance the verbosity of the output.I tested with and without default imports for a single icon and component.
{assets: false, chunks: false, modulesSpace: 6, nestedModules: false, reasonsSpace: 10}
{assets: false, chunks: false})
Demo App 1 - Default Import Icon
This is the scenario that we are told to avoid when using @mui, and for good reason.
I created this as a baseline to see what webpack had to traverse.
Demo App 1 - Webpack Metrics Output
As expected, using the default import for
AbcRounded
ended up pulling in all icons.Webpack Module Summary
Demo App 2 - Path Import Icon
Following the docs, you would expect an import like this to be lightweight. It isn't.
Demo App 2 - Webpack Metrics Output
This is the problem. Importing a single icon resulted in:
Webpack Module Summary
Demo App 3 - Path Import Component
This problem isn't unique to
@mui/icons-material
either. Here's performing a path import of a button.Demo App 3 - Webpack Metrics Output
Again, the problem. Importing a single button resulted in:
Webpack Module Summary
We should not be importing hundreds of modules from a single icon or button.
But... Tree Shaking? Side Effects?
This was confusing for me too, and I had to go into the details to find the answer.
Yes the bundle will still be minimized successfully when following ESM practices, but thousands of modules being traversed bloats memory and slows down development servers.
What are the requirements? ❓
Importing from @mui should always result in a minimal dependency graph for that particular import.
What are our options? 💡
Option 1 - Proposal
Apply transformations to the @mui build process to ensure a minimal dependency graph for all files.
Option 2 - Alternative
Remove all barrel files from @mui. This option isn't great as the developer experience that they provide is desired by both library maintainers and library users alike.
Proposed solution 🟢
Showcased above, even when importing a component or icon directly, thousands of downstream modules get pulled in. This happens because within @mui itself, barrel files are being utilized for their developer experience. In that case, why not follow the same recommendation that @mui gives, and add these transforms to the build process?
In [docs] Modularize Imports for Nextjs, the comment #35457 (comment) requested that the docs don't include
@mui/material
for import transformations via babel or swc, since there are outliers that cause the transformation to fail.Instead of backing off here, the work should be put in to fix it. The same barrel files that @mui is using internally for better developer experience is what users of the library need as well. By fixing point 1, this will come for free.
Resources and benchmarks 🔗
Below are the webpack metrics I collected for the applications in the background statement:
mui-default-import-icon-truncated-metrics.txt
mui-path-import-button-metrics.txt
mui-path-import-icon-metrics.txt
The text was updated successfully, but these errors were encountered: