Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(lambda-nodejs): esbuild bundling #11289

Merged
merged 48 commits into from
Nov 18, 2020
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
6a16cf2
feat(lambda-nodejs): esbuild bundling
jogold Nov 4, 2020
1c8639b
Merge branch 'master' into lambda-nodejs-esbuild
jogold Nov 4, 2020
36e29f8
minify and sourcemap
jogold Nov 4, 2020
ba7c830
README and target
jogold Nov 4, 2020
7d03889
README
jogold Nov 4, 2020
4f27d7a
Merge branch 'master' into lambda-nodejs-esbuild
jogold Nov 4, 2020
2195f60
env
jogold Nov 4, 2020
2d6b25c
Merge branch 'master' into lambda-nodejs-esbuild
jogold Nov 4, 2020
087d07f
Merge branch 'lambda-nodejs-esbuild' of github.com:jogold/aws-cdk int…
jogold Nov 4, 2020
703efd5
packagejsonmanager
jogold Nov 4, 2020
893af67
Update packages/@aws-cdk/aws-lambda-nodejs/README.md
jogold Nov 5, 2020
a087a89
Merge branch 'master' into lambda-nodejs-esbuild
jogold Nov 8, 2020
39e22bd
clean up / refactor
jogold Nov 8, 2020
6176343
Merge branch 'master' into lambda-nodejs-esbuild
jogold Nov 8, 2020
815ce59
EsBuildBundler
jogold Nov 8, 2020
de6ad71
do not docker build during tests
jogold Nov 9, 2020
3720e2f
win32 support
jogold Nov 9, 2020
f31e054
Merge branch 'master' into lambda-nodejs-esbuild
jogold Nov 9, 2020
3ee5915
get target from runtime
jogold Nov 9, 2020
a61dbf1
JSDoc
jogold Nov 9, 2020
c634f26
shouldBuildImage
jogold Nov 9, 2020
712801e
Merge branch 'master' into lambda-nodejs-esbuild
jogold Nov 9, 2020
8e94fc9
branch coverage to 70
jogold Nov 9, 2020
b2f4d86
refactor and loaders
jogold Nov 10, 2020
a19fb3a
fix npx on Windows
jogold Nov 10, 2020
8f1fc7a
fix json writing on Windows
jogold Nov 10, 2020
d6d81b6
Merge branch 'master' into lambda-nodejs-esbuild
jogold Nov 10, 2020
3b47df1
Merge branch 'master' into lambda-nodejs-esbuild
jogold Nov 10, 2020
0843335
Remove nohoist on parcel-bundler
jogold Nov 11, 2020
4646273
Merge branch 'master' into lambda-nodejs-esbuild
jogold Nov 11, 2020
72da21f
remove parcel special case from yarn-upgrade workflow
jogold Nov 11, 2020
ef20456
esbuild -> Parcel in JSDoc
jogold Nov 11, 2020
d10d281
README
jogold Nov 11, 2020
c4e46f7
change loader interface
jogold Nov 11, 2020
27fa4af
Merge branch 'master' into lambda-nodejs-esbuild
jogold Nov 11, 2020
5d47fce
Merge branch 'master' into lambda-nodejs-esbuild
jogold Nov 12, 2020
5367675
Merge branch 'master' into lambda-nodejs-esbuild
jogold Nov 13, 2020
a4c00bb
Merge branch 'master' into lambda-nodejs-esbuild
jogold Nov 15, 2020
f366982
Merge branch 'master' into lambda-nodejs-esbuild
Nov 17, 2020
be89560
Update packages/@aws-cdk/aws-lambda-nodejs/lib/util.ts
jogold Nov 17, 2020
68c4d64
depsLockFilePath
jogold Nov 17, 2020
8a7dd2a
Merge branch 'master' into lambda-nodejs-esbuild
jogold Nov 17, 2020
451d3f4
startsWith/endsWith
jogold Nov 17, 2020
1ccb052
latest esbuild
jogold Nov 17, 2020
5623bd5
Merge branch 'master' into lambda-nodejs-esbuild
jogold Nov 17, 2020
ad3a010
Merge branch 'master' into lambda-nodejs-esbuild
jogold Nov 17, 2020
c3eb7f6
Merge branch 'master' into lambda-nodejs-esbuild
jogold Nov 18, 2020
1a91fc2
README and Error
jogold Nov 18, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 12 additions & 18 deletions packages/@aws-cdk/aws-lambda-nodejs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,7 @@ up the entry file:
├── stack.my-handler.ts # exports a function named 'handler'
```

This file is used as "entry" for [Parcel](https://parceljs.org/). This means that your code is
automatically transpiled and bundled whether it's written in JavaScript or TypeScript.
This file is used as "entry" for [esbbuild](https://esbuild.github.io/). This means that your code is automatically transpiled and bundled whether it's written in JavaScript or TypeScript.
jogold marked this conversation as resolved.
Show resolved Hide resolved

Alternatively, an entry file and handler can be specified:

Expand All @@ -45,11 +44,11 @@ All other properties of `lambda.Function` are supported, see also the [AWS Lambd
The `NodejsFunction` construct automatically [reuses existing connections](https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/node-reusing-connections.html)
when working with the AWS SDK for JavaScript. Set the `awsSdkConnectionReuse` prop to `false` to disable it.

Use the `parcelEnvironment` prop to define environments variables when Parcel runs:
Use the `bundlingEnvironment` prop to define environments variables when esbuild runs:

```ts
new lambda.NodejsFunction(this, 'my-handler', {
parcelEnvironment: {
bundlingEnvironment: {
NODE_ENV: 'production',
},
});
Expand All @@ -73,10 +72,10 @@ new lambda.NodejsFunction(this, 'my-handler', {
});
```

This image should have Parcel installed at `/`. If you plan to use `nodeModules` it
This image should have esbuild installed globally. If you plan to use `nodeModules` it
should also have `npm` or `yarn` depending on the lock file you're using.

Use the [default image provided by `@aws-cdk/aws-lambda-nodejs`](https://github.com/aws/aws-cdk/blob/master/packages/%40aws-cdk/aws-lambda-nodejs/parcel/Dockerfile)
Use the [default image provided by `@aws-cdk/aws-lambda-nodejs`](https://github.com/aws/aws-cdk/blob/master/packages/%40aws-cdk/aws-lambda-nodejs/lib/Dockerfile)
as a source of inspiration.

### Project root
Expand All @@ -97,13 +96,8 @@ Alternatively, you can specify the `projectRoot` prop manually. In this case you
need to ensure that this path includes `entry` and any module/dependencies used
by your function. Otherwise bundling will fail.

### Configuring Parcel
The `NodejsFunction` construct exposes some [Parcel](https://parceljs.org/) options via properties: `minify`, `sourceMaps` and `cacheDir`.

Parcel transpiles your code (every internal module) with [@babel/preset-env](https://babeljs.io/docs/en/babel-preset-env) and uses the
runtime version of your Lambda function as target.

Configuring Babel with Parcel is possible via a `.babelrc` or a `babel` config in `package.json`.
### Configuring esbuild
The `NodejsFunction` construct exposes some [esbuild](https://esbuild.github.io/) options via properties: `minify`, `sourceMaps` and `target`.

### Working with modules

Expand Down Expand Up @@ -137,21 +131,21 @@ same version will be used for installation. If a lock file is detected (`package
`yarn.lock`) it will be used along with the right installer (`npm` or `yarn`).

### Local bundling
If Parcel v2.0.0-beta.1 is available it will be used to bundle your code in your environment. Otherwise,
If esbuild is available it will be used to bundle your code in your environment. Otherwise,
bundling will happen in a [Lambda compatible Docker container](https://hub.docker.com/r/amazon/aws-sam-cli-build-image-nodejs12.x).

For macOS the recommendend approach is to install Parcel as Docker volume performance is really poor.
For macOS the recommendend approach is to install esbuild as Docker volume performance is really poor.

Parcel v2.0.0-beta.1 can be installed with:
esbuild can be installed with:

```bash
$ npm install --save-dev --save-exact [email protected]
$ npm install --save-dev esbuild@0
```

OR

```bash
$ yarn add --dev --exact [email protected]
$ yarn add --dev esbuild@0
```

To force bundling in a Docker container, set the `forceDockerBundling` prop to `true`. This
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,10 @@ FROM $IMAGE
# Install yarn
RUN npm install --global [email protected]

# Install parcel 2 (fix the version since it's still in beta)
# install at "/" so that node_modules will be in the path for /asset-input
ARG PARCEL_VERSION=2.0.0-beta.1
RUN cd / && npm install parcel@$PARCEL_VERSION --no-package-lock
# Install esbuild
# (unsafe-perm because esbuild has a postinstall script)
ARG ESBUILD_VERSION=0
RUN npm install --global --unsafe-perm=true esbuild@$ESBUILD_VERSION

# Ensure all users can write to npm cache
RUN mkdir /tmp/npm-cache && \
Expand All @@ -22,4 +22,4 @@ RUN npm config --global set update-notifier false
# create non root user and change allow execute command for non root user
RUN /sbin/useradd -u 1000 user && chmod 711 /

CMD [ "parcel" ]
CMD [ "esbuild" ]
114 changes: 50 additions & 64 deletions packages/@aws-cdk/aws-lambda-nodejs/lib/bundlers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,16 @@ import { Runtime } from '@aws-cdk/aws-lambda';
import * as cdk from '@aws-cdk/core';
import { exec } from './util';

const PARCEL_VERSION = '2.0.0-beta.1';
const ESBUILD_VERSION = '0';

interface BundlerProps {
relativeEntryPath: string;
cacheDir?: string;
minify?: boolean;
sourceMap?: boolean;
target?: string;
environment?: { [key: string]: string };
dependencies?: { [key: string]: string };
externals?: string[];
installer: Installer;
lockFile?: LockFile;
}
Expand All @@ -21,51 +24,44 @@ interface LocalBundlerProps extends BundlerProps {
}

/**
* Local Parcel bundler
* Local esbuild bundler
*/
export class LocalBundler implements cdk.ILocalBundling {
public static runsLocally(resolvePath: string): boolean {
if (LocalBundler._runsLocally[resolvePath] !== undefined) {
return LocalBundler._runsLocally[resolvePath];
}
if (os.platform() === 'win32') { // TODO: add Windows support
return false;
public static get runsLocally(): boolean {
if (LocalBundler._runsLocally !== undefined) {
return LocalBundler._runsLocally;
}
try {
const parcel = spawnSync(require.resolve('parcel', { paths: [resolvePath] }), ['--version']);
const version = parcel.stdout.toString().trim();
LocalBundler._runsLocally[resolvePath] = new RegExp(`^${PARCEL_VERSION}`).test(version); // Cache result to avoid unnecessary spawns
if (!LocalBundler._runsLocally[resolvePath]) {
process.stderr.write(`Incorrect parcel version detected: ${version} <> ${PARCEL_VERSION}. Switching to Docker bundling.\n`);
const esbuild = spawnSync('npx', ['--no-install', 'esbuild', '--version']);
const version = esbuild.stdout.toString().trim();
LocalBundler._runsLocally = new RegExp(`^${ESBUILD_VERSION}`).test(version); // Cache result to avoid unnecessary spawns
if (!LocalBundler._runsLocally) {
process.stderr.write(`Incorrect esbuild version detected: ${version} <> ${ESBUILD_VERSION}. Switching to Docker bundling.\n`);
}
return LocalBundler._runsLocally[resolvePath];
return LocalBundler._runsLocally;
} catch (err) {
process.stderr.write('Using Docker bundling');
return false;
}
}

public static clearRunsLocallyCache(): void { // for tests
LocalBundler._runsLocally = {};
LocalBundler._runsLocally = undefined;
}

private static _runsLocally: { [key: string]: boolean } = {};
private static _runsLocally?: boolean;

constructor(private readonly props: LocalBundlerProps) {}

public tryBundle(outputDir: string) {
if (!LocalBundler.runsLocally(this.props.projectRoot)) {
if (!LocalBundler.runsLocally) {
return false;
}

const localCommand = createBundlingCommand({
projectRoot: this.props.projectRoot,
relativeEntryPath: this.props.relativeEntryPath,
cacheDir: this.props.cacheDir,
...this.props,
outputDir,
dependencies: this.props.dependencies,
installer: this.props.installer,
lockFile: this.props.lockFile,
bundlingEnvironment: BundlingEnvironment.LOCAL,
forcePosixPath: os.platform() !== 'win32',
});

exec('bash', ['-c', localCommand], {
Expand All @@ -86,7 +82,7 @@ interface DockerBundlerProps extends BundlerProps {
buildImage?: boolean;
buildArgs?: { [key: string]: string };
runtime: Runtime;
parcelVersion?: string;
esbuildVersion?: string;
}

/**
Expand All @@ -97,24 +93,19 @@ export class DockerBundler {

constructor(props: DockerBundlerProps) {
const image = props.buildImage
? props.bundlingDockerImage ?? cdk.BundlingDockerImage.fromAsset(path.join(__dirname, '../parcel'), {
? props.bundlingDockerImage ?? cdk.BundlingDockerImage.fromAsset(path.join(__dirname, '../lib'), {
buildArgs: {
...props.buildArgs ?? {},
IMAGE: props.runtime.bundlingDockerImage.image,
PARCEL_VERSION: props.parcelVersion ?? PARCEL_VERSION,
ESBUILD_VERSION: props.esbuildVersion ?? ESBUILD_VERSION,
},
})
: cdk.BundlingDockerImage.fromRegistry('dummy'); // Do not build if we don't need to

const command = createBundlingCommand({
...props,
projectRoot: cdk.AssetStaging.BUNDLING_INPUT_DIR, // project root is mounted at /asset-input
relativeEntryPath: props.relativeEntryPath,
cacheDir: props.cacheDir,
outputDir: cdk.AssetStaging.BUNDLING_OUTPUT_DIR,
installer: props.installer,
lockFile: props.lockFile,
dependencies: props.dependencies,
bundlingEnvironment: BundlingEnvironment.DOCKER,
});

this.bundlingOptions = {
Expand All @@ -127,42 +118,37 @@ export class DockerBundler {
}

interface BundlingCommandOptions extends LocalBundlerProps {
forcePosixPath?: boolean;
outputDir: string;
bundlingEnvironment: BundlingEnvironment;
}

enum BundlingEnvironment {
DOCKER = 'docker',
LOCAL = 'local',
}

/**
* Generates bundling command
*/
function createBundlingCommand(options: BundlingCommandOptions): string {
const entryPath = path.join(options.projectRoot, options.relativeEntryPath);
const distFile = path.basename(options.relativeEntryPath).replace(/\.(jsx|tsx?)$/, '.js');
const parcelResolvePath = options.bundlingEnvironment === BundlingEnvironment.DOCKER
? '/' // Force using parcel installed at / in the image
: entryPath; // Look up starting from entry path

const parcelCommand: string = chain([
[
`$(node -p "require.resolve(\'parcel\', { paths: ['${parcelResolvePath}'] })")`, // Parcel is not globally installed, find its "bin"
'build', entryPath.replace(/\\/g, '/'), // Always use POSIX paths in the container
'--target', 'cdk-lambda',
'--dist-dir', options.outputDir, // Output bundle in outputDir (will have the same name as the entry)
'--no-autoinstall',
'--no-scope-hoist',
...options.cacheDir
? ['--cache-dir', path.join(options.projectRoot, options.cacheDir)]
: [],
].join(' '),
// Always rename dist file to index.js because Lambda doesn't support filenames
// with multiple dots and we can end up with multiple dots when using automatic
// entry lookup
distFile !== 'index.js' ? `mv ${options.outputDir}/${distFile} ${options.outputDir}/index.js` : '',
]);
let entryPath = path.join(options.projectRoot, options.relativeEntryPath);

if (options.forcePosixPath ?? true) {
entryPath = entryPath.replace(/\\/g, '/');
}

const esbuildCommand: string = [
'npx',
'esbuild',
'--bundle', entryPath,
`--target=${options.target ?? 'es2017'}`,
'--platform=node',
`--outfile=${options.outputDir}/index.js`,
...options.minify
? ['--minify']
: [],
...options.sourceMap
? ['--sourcemap']
: '',
...options.externals
? options.externals.map(external => `--external:${external}`)
: [],
].join(' ');

let depsCommand = '';
if (options.dependencies) {
Expand All @@ -175,7 +161,7 @@ function createBundlingCommand(options: BundlingCommandOptions): string {
]);
}

return chain([parcelCommand, depsCommand]);
return chain([esbuildCommand, depsCommand]);
}

export enum Installer {
Expand Down
Loading