Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(enhanced): Recursive search for versions of shared dependencies #3078

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/smooth-goats-knock.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@module-federation/enhanced': minor
---

Added recursively search for shared dependency versions
36 changes: 24 additions & 12 deletions packages/enhanced/src/lib/sharing/ConsumeSharedPlugin.ts
Original file line number Diff line number Diff line change
Expand Up @@ -235,19 +235,29 @@ class ConsumeSharedPlugin {
compilation.inputFileSystem,
context,
['package.json'],
(err, result) => {
(err, result, checkedDescriptionFilePaths) => {
if (err) {
requiredVersionWarning(
`Unable to read description file: ${err}`,
);
return resolve(undefined);
}
//@ts-ignore
const { data, path: descriptionPath } = result;
const { data } = result || {};
if (!data) {
requiredVersionWarning(
`Unable to find description file in ${context}.`,
);
if (checkedDescriptionFilePaths) {
requiredVersionWarning(
[
`Unable to find required version for "${packageName}" in description file/s`,
checkedDescriptionFilePaths.join('\n'),
'It need to be in dependencies, devDependencies or peerDependencies.',
].join('\n'),
);
} else {
requiredVersionWarning(
`Unable to find description file in ${context}.`,
);
}

return resolve(undefined);
}
//@ts-ignore
Expand All @@ -259,15 +269,17 @@ class ConsumeSharedPlugin {
data,
packageName,
);
if (typeof requiredVersion !== 'string') {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should still keep the required version warning if the recursive lookup doesnt return the package for whatever reason.

What happens if someone has a package.json outside their app project, like in the root of their FS? This will crawl upwards across the whole fs?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

recursive lookup will stop under the following condition:

if (!parentDirectory || parentDirectory === directory) {

and this warning will be registered
`Unable to find description file in ${context}.`,

but we can actually reach the root of fs. Maybe we should stop searching at some upwards?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, maybe try with compiler.context as the edge boundary? Im not sure if that will work in your use case, but it probbably will be a good idea to somehow prevent it from searching the entire fs all the way up to root if it doesnt find something. Like keep it from leaving the current repo, usually i use compiler.context to find the boundary of the compiler.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have any ideas yet on how to limit the search tree upwards😔 compiler.context will help with dual-packages, but for monorepos search should go up to the monorepo package.json

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. Okay well I think we can run with this. I'll review and merge. Then will port to rust

requiredVersionWarning(
`Unable to find required version for "${packageName}" in description file (${descriptionPath}). It need to be in dependencies, devDependencies or peerDependencies.`,
);
return resolve(undefined);
}
// @ts-ignore webpack internal semver has some issue, use runtime semver , related issue: https://github.com/webpack/webpack/issues/17756
resolve(requiredVersion);
},
({ data }) => {
const maybeRequiredVersion =
getRequiredVersionFromDescriptionFile(data, packageName);
return (
data['name'] === packageName ||
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if no version warning should still get logged

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same warning will be registered

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but i see this messge removed

Unable to find required version for "${packageName}" in description file (${descriptionPath}). It need to be in dependencies, devDependencies or peerDependencies.,
);

                will same message report or does it fall back to unable to find description file only? 

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh yeah, I thought about the self-referencing warning that doesn't get registered.

Maybe accumulate warnings for all description file names (descriptionPath ) visited during recursive lookup to create a summary warning?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea, if you want to update the PR to do so, feel free. Otherwise just make sure that it reports back that warning if it comes up dry, rather than removing it completely. Its useful to tell dev they should list the package in their deps etc if it cant find one, especially now that it can recurse upwards

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added accumulation of checked files to the warning if the version is not found in them. I think the output looks clearer now. But the implementation looks a bit ugly 😅

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All good. I'll pull locally and touch it up a little then merge

typeof maybeRequiredVersion === 'string'
);
},
);
}),
]).then(([importResolved, requiredVersion]) => {
Expand Down
46 changes: 42 additions & 4 deletions packages/enhanced/src/lib/sharing/utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -335,27 +335,48 @@ export { normalizeVersion };
* @param {InputFileSystem} fs file system
* @param {string} directory directory to start looking into
* @param {string[]} descriptionFiles possible description filenames
* @param {function((Error | null)=, {data: object, path: string}=): void} callback callback
* @param {function((Error | null)=, {data?: object, path?: string}=, (checkedDescriptionFilePaths: string[])=): void} callback callback
* @param {function({data: Record<string, any>, path: string}=): boolean} satisfiesDescriptionFileData file data compliance check
*/
const getDescriptionFile = (
fs: InputFileSystem,
directory: string,
descriptionFiles: string[],
callback: (err: Error | null, data?: { data: object; path: string }) => void,
callback: (
err: Error | null,
data?: { data: object; path: string },
checkedDescriptionFilePaths?: string[],
) => void,
satisfiesDescriptionFileData?: (data: {
data: Record<string, any>;
path: string;
}) => boolean,
) => {
let i = 0;

// use a function property to store the list of visited files inside the recursive lookup
// instead of the public getDescriptionFile property
const satisfiesDescriptionFileDataInternal:
| (typeof satisfiesDescriptionFileData & { checkedFilePaths?: Set<string> })
| undefined = satisfiesDescriptionFileData;

const tryLoadCurrent = () => {
if (i >= descriptionFiles.length) {
const parentDirectory = dirname(fs, directory);
if (!parentDirectory || parentDirectory === directory) {
//@ts-ignore
return callback();
return callback(
null,
undefined,
satisfiesDescriptionFileDataInternal?.checkedFilePaths &&
Array.from(satisfiesDescriptionFileDataInternal?.checkedFilePaths),
);
}
return getDescriptionFile(
fs,
parentDirectory,
descriptionFiles,
callback,
satisfiesDescriptionFileDataInternal,
);
}
const filePath = join(fs, directory, descriptionFiles[i]);
Expand All @@ -372,6 +393,23 @@ const getDescriptionFile = (
new Error(`Description file ${filePath} is not an object`),
);
}
if (typeof satisfiesDescriptionFileDataInternal === 'function') {
if (!satisfiesDescriptionFileDataInternal({ data, path: filePath })) {
i++;

if (
satisfiesDescriptionFileDataInternal.checkedFilePaths instanceof Set
) {
satisfiesDescriptionFileDataInternal.checkedFilePaths.add(filePath);
} else {
satisfiesDescriptionFileDataInternal.checkedFilePaths = new Set([
filePath,
]);
}

return tryLoadCurrent();
}
}
callback(null, { data, path: filePath });
});
};
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
it('should provide own dependency', async () => {
expect(await import('lib')).toEqual(
expect.objectContaining({
default: '[email protected] with [email protected]',
}),
);
});
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"type": "commonjs"
}

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"dependencies": {
"lib": "^1.0.0"
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
const { SharePlugin } = require('../../../../dist/src');

/** @type {import("../../../../").Configuration} */
module.exports = {
context: `${__dirname}/cjs`,
plugins: [
new SharePlugin({
shared: {
lib: {},
transitive_lib: {},
},
}),
],
};
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
it('should provide library from own package.json', async () => {
expect(await import('lib1')).toEqual(
expect.objectContaining({
default: '[email protected]',
}),
);
});

it('should provide library from parent package.json', async () => {
expect(await import('lib2')).toEqual(
expect.objectContaining({
default: '[email protected]',
}),
);
});

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"dependencies": {
"lib2": "^2.0.0"
}
}

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"dependencies": {
"lib1": "^1.0.0",
"lib2": "^1.0.0"
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
const { SharePlugin } = require('../../../../dist/src');

/** @type {import("../../../../").Configuration} */
module.exports = {
context: `${__dirname}/app1`,
plugins: [
new SharePlugin({
shared: {
lib1: {},
lib2: {
singleton: true,
},
},
}),
],
};