Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(lambda-nodejs): esbuild bundling #11289

Merged
merged 48 commits into from
Nov 18, 2020
Merged

Conversation

jogold
Copy link
Contributor

@jogold jogold commented Nov 4, 2020

Replace Parcel with esbuild for bundling.

esbuild offers impressive performances compared to Parcel.
Moreover everything can be configured via the CLI. This means that
we don't need to play with the user package.json file anymore.

Add full Windows support for local bundling.

Refactor and clean-up.

Closes #10286
Closes #9130
Closes #9312
Resolves #11222

BREAKING CHANGE: local bundling now requires esbuild to be installed.

  • lambda-nodejs: projectRoot has been replaced by depsLockFilePath. It should point to your dependency lock file (package-lock.json or yarn.lock)
  • lambda-nodejs: parcelEnvironment has been renamed to bundlingEnvironment
  • lambda-nodejs: sourceMaps has been renamed to sourceMap

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@gitpod-io
Copy link

gitpod-io bot commented Nov 4, 2020

@jogold
Copy link
Contributor Author

jogold commented Nov 9, 2020

@floydspace this is now ready for review, your feedback is more than welcome, thx!

@jogold jogold marked this pull request as ready for review November 9, 2020 10:58

for (const mod of modules) {
if (!pkgDependencies[mod]) {
throw new Error(`Cannot extract version for ${mod} in package.json`);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we also/only check version in the package-lock.json file so it would not fail in case we do not have excluding package installed in our package.json
for instance, I have aws-xray-sdk-core as my dependency and willing to bundle it, but to successfully bundle it I have to exclude the child dependency pkginfo from the bundle and push it to node_modules

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will look into this. Maybe the right way to do this is to use npm list or yarn list and get the versions from there.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you detail your use case with aws-xray-sdk-core? Why do you need to exclude pkginfo in this case?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when I try to bundle aws-xray-sdk-core esbuild warns:

node_modules/pkginfo/lib/pkginfo.js:104:15: warning: This call to "require" will not be bundled because the argument is not a string literal
    contents = require(dir + '/package.json');

so it will not be able to also bundle content from require(dir + '/package.json'); and it could lead to unpredicted behavior, so I better exclude it and put the package pkginfo with its dependencies as it is in node_modules to avoid unpredicted behavior

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

due to this require condition, I also have to exclude 'import-from' for example:

node_modules/import-from/index.js:4:46: warning: This call to "require" will not be bundled because the argument is not a string literal
...ts = (fromDirectory, moduleId) => require(resolveFrom(fromDirectory, modul...
                                     ~~~~~~~
node_modules/import-from/index.js:8:9: warning: This call to "require" will not be bundled because the argument is not a string literal
    return require(resolveFrom(fromDirectory, moduleId));

Copy link

@floydspace floydspace Nov 10, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these are all inherited dependencies, not my project's directly ones

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, got it. But why don't you set aws-xray-sdk-core as external then? Is there a big difference?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, it would work, aws-xray-sdk-core is not so heavy package. But we should also take into account that it will bring also these packages along:

        "continuation-local-storage": "^3.2.0",
        "moment": "^2.15.2",
        "pkginfo": "^0.4.0",
        "semver": "^5.3.0",
        "underscore": "^1.8.3",
        "winston": "^2.2.0"

so it could lead to a tangible difference

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I think the best solution is to parse the result of npm list <mod1> <mod2> --depth 0 --json or yarn list --pattern "<mod1>|<mod2>" --depth=0 --json.

The JSON format is different for npm and yarn.

I suggest doing this in another PR.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed

packages/@aws-cdk/aws-lambda-nodejs/lib/bundling.ts Outdated Show resolved Hide resolved
packages/@aws-cdk/aws-lambda-nodejs/lib/esbuild-bundler.ts Outdated Show resolved Hide resolved
@jogold
Copy link
Contributor Author

jogold commented Nov 14, 2020

@Garethp

Ideally it'd be nice if NodejsFunctionProps had an optional property for an instance of cdk.ILocalBundling so that we could also take advantage of the discovered project root, installer and relative entry path.

Can you open an issue for that?

@hoegertn
Copy link
Contributor

@eladb @jogold Do we have any ETA on this change?

@adamdottv
Copy link
Contributor

@eladb @jogold Do we have any ETA on this change?

Also interested in timing here; pretty sure this PR is going to save me ~30 minutes a day 😅

@jogold jogold requested a review from eladb November 17, 2020 16:46
Copy link
Contributor

@eladb eladb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor

packages/@aws-cdk/aws-lambda-nodejs/README.md Outdated Show resolved Hide resolved
packages/@aws-cdk/aws-lambda-nodejs/lib/function.ts Outdated Show resolved Hide resolved
@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: 1a91fc2
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@mergify
Copy link
Contributor

mergify bot commented Nov 18, 2020

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@jogold
Copy link
Contributor Author

jogold commented Nov 18, 2020

@eladb https://github.com/aws/aws-cdk/pull/11289/checks?check_run_id=1417379312

GitHub App like Mergify are not allowed to merge pull request where .github/workflows is changed.
This pull request must be merged manually.

@eladb eladb merged commit 7a82850 into aws:master Nov 18, 2020
@jogold jogold deleted the lambda-nodejs-esbuild branch November 18, 2020 10:04
jogold added a commit to jogold/aws-cdk that referenced this pull request Nov 18, 2020
Lookup the version in the `package.json` and then fallback to requiring
the module's `package.json`. This allows to mark transitive dependencies
as externals and install them.

See aws#11289 (comment)
mergify bot pushed a commit that referenced this pull request Nov 23, 2020
Lookup the version in the `package.json` and then fallback to requiring
the module's `package.json`. This allows to mark transitive dependencies
as externals and install them.

See #11289 (comment)


----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
9 participants