Slow start times due to use of barrel files #11234
Comments
|
On our project we have ~100 test suites, ~600 tests. it take about 6 minutes to run all the tests. I've created a script that changes all the imports from barrel files to explicit import, and the tests run 10x faster (Still not optimal, but shows the problem) Anyone with a solution/workaround/idea? |
|
I'm happy to take a PR outlining this issue. 👍 In Jest itself we've made lots of |
|
@gilamran could you please share that script for changing the barrel files import? I'm in a similar bind where changing one reusable component is running 100s of tests. I've tried using babel transform plugin imports but it was not working for me. |
|
@fadi-george sorry I don't have the script anymore (Maybe I can look it up in the git history), but I've found even faster solution. I had to do some juggling to make esbuild and jest work together, like importing svgs etc. |
|
@gilamran ah interesting approach, thank you for sharing! I'll definitely play around with that idea. |
|
@gilamran I assume you are doing something like this? Have you ran into any memory issues? bundle.spec.ts: |
|
I believe I fixed my memory issues but now I ran into this problem with jest mocks and mockImplementations failing. |
|
@gilamran did you use any jest.mock calls? For me, it forces my tests to fail It might be supported later on with esbuild or jest, I tried top-level-await but I couldn't get it to work |
|
I don't use |
|
I have made a simple repo that reproduces this issue using a barrel file provided by a large third-party library, in this case Material UI icons: https://github.com/rsslldnphy/jest-barrel-files On my machine, the test that imports icons as I do not have a good understanding of the mechanics of what goes on (with treeshaking etc) when importing files in this way, but I'm not encountering this slowness issue the other tools I'm using - is there a way to make jest aware of code that can be ignored? |
|
@rsslldnphy for material specifically, which is a quite big library, I isolate the parts of the library my code actually uses in a re-exported barrel file: // components/index.ts
export { default as AppBar, type AppBarProps } from "@mui/material/AppBar";
export { default as Button, type ButtonProps } from "@mui/material/Button";
// ...
export { default as DateRangeIcon } from "@mui/icons-material/DateRange";
export { default as CheckCircleIcon } from "@mui/icons-material/CheckCircle";
export { default as LocalShippingIcon } from "@mui/icons-material/LocalShipping";
// ... keep in mind that even 50-100 icons is only a small part of the 2000+ icons material exportsTo be fair, we did this to mark to other developers in the team what parts of the material ui they could use without having to have a talk with our UI/UX designer beforehand. But it also mitigates this issue as a side effect. |
I've had this problem. Problem is barreling, and the way the import mapping works. Unfortunately, it seems that transpilers (TypeScript), bundlers (WebPack, SnowPack, Vite, Rollup) can't differentiate exactly which is the import you're trying to use, and it will pull all the other files that were specified in the barrel index file. I'm currently refactoring a somewhat big app because I was waiting for 3 minutes until I ran 3 tests, and it was because my entire app was barreled. |
Nice idea on both counts @PupoSDC! May well implement this. Will be a bit of a faff to set up and maintain with icons especially but not a bad trade-off at all for mitigating this issue. Thanks! |
|
Not sure if it will help, but here's my 2 cents: I face problems with barrel files not only in jest, but on some external and internal libs that have optional peer dependencies. Including these libs which do all exports from a barrel causes compile errors due to indirect import of unused components that use optional not installed dependencies To fix this, I was using the already mentioned babel-plugin-transform-imports, which IMO works great (although I had to fork it and fix import aliases issues) I even tried to improve the transform imports solution writing babel-plugin-resolve-barrel-files, but its a simple solution for ESM modules only. But I guess that for jest, people could try implementing a lazy import with mocks: jest.mock('module-with-barrel', () => {
const RequiredComponent = jest.requireActual('module-with-barrel/RequiredComponent').default;
return {
__esModule: true,
RequiredComponent
// or use a more lazy approach...
// get RequiredComponent() {
// return jest.requireActual('module-with-barrel/RequiredComponent').default;
// }
}
})
They probably can, but since barrel files are normal source code files, they can execute side effects (read the tip) (eg: some dep exporting global stuff).. so it's more safe to just don't optimize unless you explicit tell them to do it (the case for the babel plugins and webpack configs) |
|
I confirm. Using barrel imports extreamly slows down starting test suites. Test cases run very fast. Please manage this problem. |
|
Is there any solution for this? I have more than 3000 imports to change if i want to reverse the barrels import to full file path import, which would take very long time. |
|
No solution, but there's a workaround |
|
Just a thought on this as recently i was facing a similar issue and the way i fixed it is by creating a custom barrel import transformer. The way it works is in first step we iterate all files to determine all the exports of files in project making a Map of import to lookup later. Now when the test start to execute, using jest transform configuration, then execute a transform which uses the import lookup map created in first step to rewrite the import statements to specific imports statements and then jest executes on the transformed code. Was able to significant improvement with this approach when using jest with ts-jest (isolatedModules enabled). |
|
How did you manage it? Do you have any example somewhere? I am very interested as i can't manage to reduce the time execution of my spec files due to the size of the project and barrel imports everywhere. |
|
@boubou158 added a typescript based sample here with some readme- https://github.com/dsmalik/ts-barrel-import-transformer |
|
After spending a month trying to make esm working on our project trying to speed up the jest performance, it seems impossible to have 100% working. I am giving up on the esm option. I am now checking your solution (thanks a lot for that by the way !) but i am a little confused on how i could integrate this transformer with the transformer used by jest-preset-angular? |
How would you apply your approach on an angular project? I am running esbuild on a single spec file with --bundle option, it is then throwing errors TypeError: Cannot read properties of null (reading 'ngModule') every where when i run this file with jest. |
I know it's been a long time since you've posted this solution, but I hope you can help me out. I have been trying to use this approach and the bundling seems to work fine. Also, when passing the single bundled file to jest, it doesn't work for tests that are using For instance, we have this piece of code in one of our tests: The corresponding file (both the actual implementation and mocked version) does exist, but when running Jest, it gives the following error: |
|
We also face long startup times in our tests. After trying a lot of "solutions" mentioned on the internet we discovered that in our case the problem are the barrel files. I created two simple unit tests in our code base. Test
To be able to dig deeper into the problem you have to understand what is going on during the long startup time. Modify Jest Runtime ClassInstead of a patch I link the code-pointers and the added code, in case you run a different jest version. The file to modify is: Step 1Add additional class properties to track the number of loaded modules. Step 2Count and log loaded modules. code pointer 2 OutcomeAfter patching the
And you can clearly see by the logged output that jest is importing everything in case of barrel files before running a test. After adjusting only some of the imports (to use named imports) in our code base as a PoC, I was able to reduce the build time from 15 minutes to 6 minutes. |
|
We have around 3200 test suites and 35k tests. We had the similar issue. @mui/icons-material is the culprit in our case. We simply mocked the package and time came down from 50 minutes to 18 minutes. We are still going through other packages but that's a significant time improvement for mocking just one package, |
This is fantastic. Thanks for sharing your patch. I've got it printing out the results, but I'm not sure what to look for. I can see that it's certainly loading a ton of modules, but nothing stands out as something that can be optimized. Do you have any advice? Currently taking >20s to run a <3s test. |
To investigate, you should select a simple and rather small test that has an unexpectedly long start time. Maybe you can see the difference better in the following react example. As mentioned by others If you are not already using Create a test tsx-file and add the following code: The test with the second import for Now run the single test with only one of the two imports for With this knowledge, you should be able to find problematic imports in your codebase. |
|
is there a plugin or something that can solve all barrel files usages? |
Unfortunately not. There are several workarounds in this thread, but none are low-effort. FYI, if you're looking at any of the above issues, here are some limitations and things to experiment with: - Remove barrel files and only test what you can see Sure-fire way to fix this. However, not feasible in larger codebases using barrel files liberally. (Plus, it's tilting at windmills. Build for the actual need, not the ideal.) In addition, mock things your test subject is using, so that your test is focused on exactly the logic you wrote. Let TypeScript make sure your interfaces continue to align, and if you're worried, you can have another layer of tests to ensure interop between modules. (We have a test capital-i, where we have 100% unit test coverage, and a comprehensive set of end-to-end tests that validate high-level module interop. Almost no integration tests because those often end up slower than our end-to-end tests!). - Try out the various Babel plugins listed above This may work for you on relatively small projects without using a lot of mocks and spies. However, most of these are opt-in, and are only useful if you can identify up-front where a lot of your heftier barrel files are. In many cases, this isn't feasible. - Try another test runner Something like Vite might work better, but most people won't have a 1:1 transition from Jest. Vite uses enforces strict modules (e.g., not mockable/spyable in the same ways Jest is). Also, it suffers from the same issues as those described below, since it also uses ESBuild. But you might have luck here if you don't do anything fancy with mocks/spies. - ESBuild/SWC all the things If you barely use Jest's mocking/spy features, this might be feasible. However, any alternative transformer to Babel will be a pain if you use them. Synthetic exports in babel are loosely constructed (e.g., they're mutable references, so can be replaced spied on). Tools like SWC and ESBuild adhere more to official specs for modules (e.g., modules are immutable, can't be monkey patched, etc.). I have not yet tried Rollup for our tests. Might be worth a shot if someone has the energy, but I'd be surprised if it's any faster/better/useable than any of the above examples. I have experimented with a custom Jest transformer, too, that uses Babel to hoist In my case, we're a bit stuck. We have a huge codebase with tons of mocks and spies (we require 100% code coverage, and suggest that developers have 1:1 test -> adjacent test file with 100% coverage, so our codebase is at least half test code). This has been excellent for maintaining high code quality based on the standards when we began (circa 2019). But the barrel file issue is a thorny one. In new code, we don't allow them, but we've got thousands of tests that drag because of this. To put it simply: there's no easy solution here. In hindsight, it was probably engineering malpractice to suggest the use of barrel files, and we're paying for our past oversight! 😆 The only clear way forward at the moment is to suck it up, deal with slow tests for a while, and refactor them as you get the opportunity. I've been experimenting on the side for my team for years looking for ways to make this better, especially as our lazier engineers realize how easy we've made it to write thick tests that exercise huge swaths of the platform. Yeah, it's less test code, but it takes two minutes to get feedback. Some folks don't mind waiting, but I sure do. Despite our best efforts, even the best engineers will take shortcuts to shave off a few minutes from their tickets, and make everyone pay for that in the time it takes to run tests. Amortized over several years and several dozen developers, and you're looking at 3 minute startup times for a single, 3ms test that ensures an I almost think Jest (or some other tool) needs a way to elevate how much code a certain test is using and give us the ability to set upper limits. It's a waste of time and CPU to load 20,000 modules to test a couple of logical branches. No matter what Jest does to make this better, without the automatic tooling for us tech leads to set upper bounds on this, I don't foresee this being all that fixable with some out of the box solution... 🤔 |
|
I have taken a stab at writing my own jest transformer to accomplish this. I'm using an Nx mono repo with about 11 Angular libraries inside (v17.1), I have about 1600 unit tests, and it takes about 5 minutes to run. The jest transformer I've created will correctly replace all of the paths in my ts config with the import pointing to the actual file of where it's exported. However, my solution doesn't work when it comes to transforming 3rd-party paths like @angular/core, @ngrx/store, etc., because ts.createProgram will only return declaration files for any path I pass in that points to a node_module file. With this transformer only changing local imports, it's only shaved about 30 seconds off the execution time. If there is anyone who might be able to lend a helping hand with this transformer, I would be very appreciative. I'm sure it could be very beneficial to others coming here as well. I only started working with the TypeScript compiler api a few days ago, so I'm sure that my solution is far from optimal. // jest.preset.js
{
...
transform: {
'^.+\\.(ts|mjs|js|html)$': [
'<rootDir>/jest-import-transformer.js'
]
},
...
}// jest-import-transformer.js
var fs = require('fs');
const path = require('path');
const mkdirp = require('mkdirp');
var angularPreset = require('jest-preset-angular');
const presetConfig = {
// NOTE: All tsconfig.spec.json files used have their target set to ES2016 to deal with the following problem
// https://github.com/nrwl/nx/issues/15390#issuecomment-1457792176
tsconfig: '<rootDir>/tsconfig.spec.json',
stringifyContentPathRegex: '\\.(html|svg)$',
isolatedModules: true
};
var baseTransformer = angularPreset.default.createTransformer(presetConfig);
const ts = require('typescript');
const tsConfig = require('./tsconfig.base.json');
const pathCache = {};
const filePaths = {};
function extractAllExportStatements(filePath) {
const program = ts.createProgram([filePath], {
target: tsConfig.compilerOptions.target,
module: tsConfig.compilerOptions.module
});
const output = {};
for (const sourceFile of program.getSourceFiles()) {
if (!sourceFile.isDeclarationFile) {
// Walk the tree to search for exported nodes
if (!filePaths[sourceFile.fileName]) {
filePaths[sourceFile.fileName] = path
.relative(__dirname, sourceFile.fileName)
.replaceAll('\\', '/');
}
const fileName = filePaths[sourceFile.fileName];
ts.forEachChild(sourceFile, node => visit(node, fileName));
}
}
return output;
function visit(node, fileName) {
// Only consider exported nodes
if (!isNodeExported(node)) {
return;
}
if (ts.isModuleDeclaration(node)) {
// This is a namespace, visit its children
ts.forEachChild(node, visit);
} else if (node.name) {
output[node.name.text] = fileName;
} else if (ts.isVariableStatement(node)) {
output[node.declarationList?.declarations[0]?.name?.text] = fileName;
} else if (ts.isExportDeclaration(node)) {
output[node.exportClause?.name?.text] = fileName;
} else {
debugger;
}
}
function isNodeExported(node) {
return (
(ts.getCombinedModifierFlags(node) & ts.ModifierFlags.Export) !== 0 ||
(!!node.parent && node.parent.kind === ts.SyntaxKind.SourceFile) ||
node.exportClause?.name?.text // export * as [Name] syntax
);
}
}
function ensureBarrelIsCached(oldPath, barrelPath, options) {
if (!pathCache[barrelPath]) {
// Not in the cache, let's check the cacheDirectory
const cacheFilePath = path.join(options.config.cacheDirectory, oldPath + '.json');
if (fs.existsSync(cacheFilePath)) {
pathCache[barrelPath] = JSON.parse(fs.readFileSync(cacheFilePath));
} else {
// https://github.com/microsoft/TypeScript/wiki/Using-the-Compiler-API#using-the-type-checker
pathCache[barrelPath] = extractAllExportStatements(barrelPath);
const dirName = path.dirname(cacheFilePath);
mkdirp.sync(dirName);
fs.writeFileSync(cacheFilePath, JSON.stringify(pathCache[barrelPath]));
}
}
return pathCache[barrelPath];
}
function invalidateAndReloadBarrelCache(oldPath, barrelPath, options) {
const cacheFilePath = path.join(options.config.cacheDirectory, oldPath + '.json');
if (fs.existsSync(cacheFilePath)) {
fs.rmSync(cacheFilePath, { force: true, recursive: true });
}
pathCache[barrelPath] = null;
return ensureBarrelIsCached(oldPath, barrelPath, options);
}
function modifyImportDeclaration(node, src, fileName, options) {
const oldPath = node.moduleSpecifier.text;
// only transform paths defined in our tsConfig
const barrelPath = tsConfig.compilerOptions.paths[oldPath]?.[0];
let barrelExports;
if (barrelPath) {
barrelExports = ensureBarrelIsCached(oldPath, barrelPath, options);
}
if (!barrelExports) {
return src;
}
if (node.importClause && node.importClause.namedBindings) {
var directImports = [];
for (importNode of node.importClause.namedBindings.elements) {
importNode = importNode.getFullText().trim();
if (!barrelExports[importNode]) {
barrelExports = invalidateAndReloadBarrelCache(oldPath, barrelPath, options);
if (!barrelExports[importNode]) {
console.warn('Import not found!', importNode, fileName);
break;
}
}
var relPath = path
.relative(fileName, __dirname + '/' + barrelExports[importNode])
.replaceAll('\\', '/')
.replace('.ts', '')
.replace('../', '');
directImports.push(`import {${importNode}} from '${relPath}';`);
}
if (directImports.length) {
var transformedFileSrc = src.replace(node.getFullText(), directImports.join('\r\n'));
return transformedFileSrc;
}
}
return src;
}
function processFile(src, filename, options) {
var transformedFileContent = src;
ts.transpileModule(src, {
compilerOptions: tsConfig.compilerOptions,
fileName: filename,
transformers: {
before: [
context => {
return sourceFile => {
function visit(node) {
if (ts.isImportDeclaration(node)) {
transformedFileContent = modifyImportDeclaration(
node,
transformedFileContent,
filename,
options
);
// TODO find a way to replace the node with multiple import nodes instead
}
return ts.visitEachChild(node, visit, context);
}
return ts.visitNode(sourceFile, visit);
};
}
]
}
});
return transformedFileContent;
}
function hasDebuggerAttached() {
// We don't want to run when the debugger is attached, because it would throw off sourcemaps
return !!process.env.VSCODE_INSPECTOR_OPTIONS;
}
module.exports = {
process(src, fileName, options) {
let newSrc = src;
if (
!hasDebuggerAttached() &&
fileName.endsWith('.ts') &&
!ignoreFiles.some(f => fileName.includes(f))
) {
newSrc = processFile(src, fileName, options);
}
return baseTransformer.process(newSrc, fileName, {
...options,
transformConfig: presetConfig
});
}
};
const ignoreFiles = ['test-setup', 'test-polyfills']; |
|
@mikerentmeister I have also embarked on this endeavor myself. I have been building a babel plugin that replaces all the import statements to point directly to the original exporter. I have also modified the In order for my plugin to be practical I need to modify the parser I'm spawning so that it dynamically uses the jest/babel configuration of the project. As of right now I have sort of hardcoded my project's config within my plugin |
|
Here is a first version of the plugin I mentioned above (as https://www.npmjs.com/package/@gtsopanoglou/babel-jest-boost Edit: If you've used the plugin above successfully, or have trouble using it, please open an issue and i'll try to help you. I intend for it be an actual solution to this problem we are discussing here, not just a tailor-made solution for my codebase. Thanks 🙏🏻 AUG 13 Update: We have been successfully using this plugin in our production codebase since it's release (3+ months as of writing this). We have upgraded to react 18 and it still works great. OCT 30 Update: I now consider this plugin production ready. We've been continually using it in an ever evolving codebase for many months and it has proven to be an actual solution to the problem discussed here. |
|
@gtsop I have tried your plugin quickly and so far on my nx monorepo project that uses barrel files everywhere, the performance gain is massive! |
|
@gtsop your license stops us from trying your plugin. As it will probably do for any commercial project. |
|
@Oize Not sure I can think of any use cases other than speeding up your own tests as part of your internal tooling, in which case the licence doesn't block you at all. If for some reason you wish to sell jest test runs as a service then maybe there is a problem there. |
|
I mean, afaik, I have to make open source any repo that has your package as a dependency. And I definitely can't make open source my company's monorepo xD. Maybe I am reading it wrong, idk. |
|
@gtsop I'd love to use your plugin at work as well, where our product code is closed source. Can you please update the license to MIT or something less restrictive? Thanks for your work on this — I'm genuinely extremely excited to try this out and see if it helps |
|
@TSMMark @Oize The licence does not prevent you from using it as long as it's for internal tooling, and let's not make this thread about licencing, mongodb was agplv3 up to 2018 and had thousands of commercial users, I highly doubt anyone has a case that would trigger the copyleft clause for this plugin |
|
Hey, I am facing a similar issue, I use ts-jest in my config, does anyone have any ideas on how to improve the performace. We have ~1800 test cases, which take anywhere between 15-20 mins |
|
Hi everyone, |
|
@FogelAI do you mind scaffolding a project on I followed the readme file & tried to apply in my project, and I didn't see any improvement. I might be doing something wrong 😞 The project that I tried is using |
|
Hi @guilhermecastros |
|
It is not public, but I have it like this:
|
|
Try this in Jest: "transform": {
"^.+.tsx?$": [
"ts-jest",
{
"isolatedModules": true,
"babelConfig": {
"plugins": [
[
"babel-plugin-transform-barrels",
{
"executorName": "jest"
}
]
]
}
}
],
},Let me know if it helps. |
|
@FogelAI I have tried your lib, for my codebase it make test results slower ;D NX monorepo with 120+ libs, tests 290s (with plugin) vs 250s (without) Isolated modules enabled. |
|
As a re-cap to this mega-thread:
https://github.com/gtsop/babel-jest-boost Since this problem is officially out of scope for jest, I am wondering whether jest should add a reference in the documentation for this topic and how to approach it. |
and others
🐛 Bug Report
It is unclear that large dependency graphs caused by the use of barrel files can slow test initialization dramatically. There are some tangential mentions of this in the documentation, but it is not outlined in a clear and direct manner. This likely leads to a lot of inefficiency in many projects.
Here are the most important points to stress:
It is the last bullet that leads to the largest reduction in efficiency, due mainly to barrel files. Barrel files are
indexfiles that re-export the exports of other files in a directory. They make it possible to import multiple related dependencies in a singleimportstatement. The downside of this is that Jest sees all of the re-exported contents of the barrel file as dependencies and crawls through them, used or not.Reducing the use of barrel files can help quite a bit in reducing the amount of time it takes before a test can start. This is especially true of dependencies pulled from NPM packages. Packages are typically developed with a root index file that acts as a barrel file for the entire package. If the package is rolled up into a single file at build time, there is only one file for the Jest runtime to open and parse. If the package publishes its source files independently, without a rollup stage, the Jest runtime will need to open and parse every file in the package independently.
In an enterprise setting, there are often internally developed tools and libraries. These packages can grow to be fairly large, and given their internal use, it can be tempting to provide the source files as the output of the packages. In fact, this can improve tree shaking when building the applications that depend on them. Jest, however, can suffer greatly in this environment because the raw number of files can grow without bounds. This problem becomes exponentially worse when overly tight internal dependency semvers reduce the ability of package managers to de-duplicate their installs.
Resolving these issues can lead to tremendous decreases in Jest's test suite initialization and this should be highlighted in the documentation. Barrel files can have a huge impact on the number of files that the Jest runtime needs to parse, and that is not clear without a deep dive into the way dependencies are evaluated. Sharing this knowledge more broadly could make an already fantastic test runner that much better and improve the quality of many products that rely upon it in the process.
To Reproduce
As this is a documentation issue and not a code issue, this may not apply. However, in the spirit of completeness:
Expected behavior
The expectation is that the Jest documentation should more explicitly explain the impact that barrel files can have on test start times.
Link to repl or repo (highly encouraged)
Given that this issue is manifested in large scale applications with many inter-related dependencies, it is not feasible to provide a replication of the issue.
envinfo
N/A
The text was updated successfully, but these errors were encountered: