Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DO-NOT-MERGE] RFC: use esbuild to significantly speedup building #4487

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion packages/docusaurus-plugin-content-docs/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,9 @@
"main": "lib/index.js",
"types": "src/plugin-content-docs.d.ts",
"scripts": {
"build": "tsc",
"build": "tsc && yarn build-theme-esm && yarn build-client-esm",
"build-theme-esm": "esbuild src/theme/hooks/useDocs.ts > lib/theme/hooks/useDocs.js",
"build-client-esm": "esbuild src/client/docsClientUtils.ts > lib/client/docsClientUtils.js",
Comment on lines +8 to +10
Copy link
Contributor Author

@SamChou19815 SamChou19815 Mar 22, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Somehow esbuild doesn't like that the client files are not in esm, so I setup the script to compile the client files to esm.

"watch": "tsc --watch"
},
"publishConfig": {
Expand All @@ -21,6 +23,7 @@
"@docusaurus/module-type-aliases": "2.0.0-alpha.72",
"@types/picomatch": "^2.2.1",
"commander": "^5.1.0",
"esbuild": "^0.10.2",
"picomatch": "^2.1.1"
},
"dependencies": {
Expand Down
1 change: 1 addition & 0 deletions packages/docusaurus-theme-common/tsconfig.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
"compilerOptions": {
"incremental": true,
"tsBuildInfoFile": "./lib/.tsbuildinfo",
"module": "esnext",
"rootDir": "src",
"outDir": "lib"
}
Expand Down
13 changes: 5 additions & 8 deletions packages/docusaurus-theme-live-codeblock/src/custom-buble.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,10 @@
// fork of Buble which removes Buble's large dependency and weighs in
// at a smaller size of ~51kB
// https://github.com/FormidableLabs/react-live#what-bundle-size-can-i-expect
const {transform, features: bubleFeatures} = require('@philpl/buble');
import {transform, features as bubleFeatures} from '@philpl/buble';

// This file is designed to mimic what's written in
// https://github.com/kitten/buble/blob/mini/src/index.js, with custom transforms options,
// so that webpack can consume it correctly.
exports.features = bubleFeatures;

exports.transform = function customTransform(source, options) {
function customTransform(source, options) {
return transform(source, {...options, transforms: {asyncAwait: false}});
};
}

export {bubleFeatures as features, customTransform as transform};
Comment on lines -11 to +17
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to turn this into esm to make esbuild happy.

1 change: 1 addition & 0 deletions packages/docusaurus/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@
"css-loader": "^5.1.1",
"del": "^6.0.0",
"detect-port": "^1.3.0",
"esbuild-loader": "^2.11.0",
"eta": "^1.12.1",
"express": "^4.17.1",
"file-loader": "^6.2.0",
Expand Down
90 changes: 15 additions & 75 deletions packages/docusaurus/src/webpack/utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,10 @@ import webpack, {
Stats,
} from 'webpack';
import fs from 'fs-extra';
import TerserPlugin from 'terser-webpack-plugin';
import OptimizeCSSAssetsPlugin from 'optimize-css-assets-webpack-plugin';
import CleanCss from 'clean-css';
// import TerserPlugin from 'terser-webpack-plugin';
import {ESBuildMinifyPlugin} from 'esbuild-loader';
// import OptimizeCSSAssetsPlugin from 'optimize-css-assets-webpack-plugin';
// import CleanCss from 'clean-css';
import path from 'path';
import crypto from 'crypto';
import chalk from 'chalk';
Expand All @@ -29,7 +30,7 @@ import {
ConfigurePostCssFn,
PostCssOptions,
} from '@docusaurus/types';
import CssNanoPreset from '@docusaurus/cssnano-preset';
// import CssNanoPreset from '@docusaurus/cssnano-preset';
import {version as cacheLoaderVersion} from 'cache-loader/package.json';
import {
BABEL_CONFIG_FILE_NAME,
Expand Down Expand Up @@ -143,11 +144,16 @@ export function getBabelOptions({

export function getBabelLoader(
isServer: boolean,
// eslint-disable-next-line @typescript-eslint/no-unused-vars
babelOptions?: TransformOptions | string,
): Loader {
return {
loader: require.resolve('babel-loader'),
options: getBabelOptions({isServer, babelOptions}),
loader: require.resolve('esbuild-loader'),
options: {
loader: 'tsx',
format: isServer ? 'cjs' : 'esm',
target: 'es2017',
},
};
}
Comment on lines 145 to 158
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I plan to make this function be like

if (process.env.ESBUILD) {
  return {
    loader: require.resolve('esbuild-loader'),
    options: {
      loader: 'tsx',
      format: isServer ? 'cjs' : 'esm',
      target: 'es2017',
    },
  };
  return {
    loader: require.resolve('babel-loader'),
    options: getBabelOptions({isServer, babelOptions}),
  }
}

In this way all the existing code that uses the getBabelLoader can work without modification. This is somehow needed since this function is exposed here. However, this is bad since the function should then really be named as getJSLoader. Or we can create a new function called getJSLoader and deprecate the getBabelLoader function.

Do maintainers have any options?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

getJSLoader is also something I had in mind. Would be fine to deprecate getBabelLoader()


Expand Down Expand Up @@ -463,6 +469,7 @@ export function getHttpsConfig(): boolean | {cert: Buffer; key: Buffer} {
}

// See https://github.com/webpack-contrib/terser-webpack-plugin#parallel
// eslint-disable-next-line @typescript-eslint/no-unused-vars
function getTerserParallel() {
let terserParallel: boolean | number = true;
if (process.env.TERSER_PARALLEL === 'false') {
Expand All @@ -476,74 +483,7 @@ function getTerserParallel() {
return terserParallel;
}

// eslint-disable-next-line @typescript-eslint/no-unused-vars
export function getMinimizer(useSimpleCssMinifier = false): Plugin[] {
const minimizer = [
new TerserPlugin({
cache: true,
parallel: getTerserParallel(),
sourceMap: false,
terserOptions: {
parse: {
// we want uglify-js to parse ecma 8 code. However, we don't want it
// to apply any minification steps that turns valid ecma 5 code
// into invalid ecma 5 code. This is why the 'compress' and 'output'
// sections only apply transformations that are ecma 5 safe
// https://github.com/facebook/create-react-app/pull/4234
ecma: 8,
},
compress: {
ecma: 5,
warnings: false,
},
mangle: {
safari10: true,
},
output: {
ecma: 5,
comments: false,
// Turned on because emoji and regex is not minified properly using default
// https://github.com/facebook/create-react-app/issues/2488
ascii_only: true,
},
},
}),
];

if (useSimpleCssMinifier) {
minimizer.push(
new OptimizeCSSAssetsPlugin({
cssProcessorPluginOptions: {
preset: 'default',
},
}),
);
} else {
minimizer.push(
...[
new OptimizeCSSAssetsPlugin({
cssProcessorPluginOptions: {
preset: CssNanoPreset,
},
}),
new OptimizeCSSAssetsPlugin({
cssProcessor: CleanCss,
cssProcessorOptions: {
inline: false,
level: {
1: {
all: false,
},
2: {
all: true,
restructureRules: true,
removeUnusedAtRules: false,
},
},
},
}),
],
);
}

return minimizer;
return [new ESBuildMinifyPlugin({target: 'es2017', css: true})];
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wonder how much different this minification is compared to the old one?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Earlier stat because I mess up the build shows that it reduces the output by 1%.

}
45 changes: 40 additions & 5 deletions yarn.lock
Original file line number Diff line number Diff line change
Expand Up @@ -8224,6 +8224,23 @@ es6-promisify@^5.0.0:
dependencies:
es6-promise "^4.0.3"

esbuild-loader@^2.11.0:
version "2.11.0"
resolved "https://registry.yarnpkg.com/esbuild-loader/-/esbuild-loader-2.11.0.tgz#65efc9c8ab14f2da06b9d165396f9520b4aeac96"
integrity sha512-yFzrpIvhHRtV1Z8V1VtW6xm0dmEHlBheJjhx+EJPTcSKBhVMeIKC2FVyZ+N1ZgSBZEKgky9vtD4q455HgnT/3g==
dependencies:
esbuild "^0.10.2"
joycon "^3.0.1"
json5 "^2.2.0"
loader-utils "^2.0.0"
type-fest "^1.0.1"
webpack-sources "^2.2.0"

esbuild@^0.10.2:
version "0.10.2"
resolved "https://registry.yarnpkg.com/esbuild/-/esbuild-0.10.2.tgz#caa65a8f3096d547d89159918039df6c5c6c90be"
integrity sha512-/5vsZD7wTJJHC3yNXLUjXNvUDwqwNoIMvFvLd9tcDQ9el5l13pspYm3yufavjIeYvNtAbo+6N/6uoWx9dGA6ug==

escalade@^3.0.2, escalade@^3.1.1:
version "3.1.1"
resolved "https://registry.yarnpkg.com/escalade/-/escalade-3.1.1.tgz#d8cfdc7000965c5a0174b4a82eaa5c0552742e40"
Expand Down Expand Up @@ -12029,6 +12046,11 @@ joi@^17.3.0, joi@^17.4.0:
"@sideway/formula" "^3.0.0"
"@sideway/pinpoint" "^2.0.0"

joycon@^3.0.1:
version "3.0.1"
resolved "https://registry.yarnpkg.com/joycon/-/joycon-3.0.1.tgz#9074c9b08ccf37a6726ff74a18485f85efcaddaf"
integrity sha512-SJcJNBg32dGgxhPtM0wQqxqV0ax9k/9TaUskGDSJkSFSQOEWWvQ3zzWdGQRIUry2j1zA5+ReH13t0Mf3StuVZA==

jpeg-js@^0.3.4:
version "0.3.6"
resolved "https://registry.yarnpkg.com/jpeg-js/-/jpeg-js-0.3.6.tgz#c40382aac9506e7d1f2d856eb02f6c7b2a98b37c"
Expand Down Expand Up @@ -12180,10 +12202,10 @@ json5@^1.0.1:
dependencies:
minimist "^1.2.0"

json5@^2.1.2:
version "2.1.3"
resolved "https://registry.yarnpkg.com/json5/-/json5-2.1.3.tgz#c9b0f7fa9233bfe5807fe66fcf3a5617ed597d43"
integrity sha512-KXPvOm8K9IJKFM0bmdn8QXh7udDh1g/giieX0NLCaMnb4hEiVFqnop2ImTXCc5e0/oHz3LTqmHGtExn5hfMkOA==
json5@^2.1.2, json5@^2.2.0:
version "2.2.0"
resolved "https://registry.yarnpkg.com/json5/-/json5-2.2.0.tgz#2dfefe720c6ba525d9ebd909950f0515316c89a3"
integrity sha512-f+8cldu7X/y7RAJurMEJmdoKXGB/X550w2Nr3tTbezL6RwEE/iMcm+tZnXeoZtKuOq6ft8+CqzEkrIgx1fPoQA==
dependencies:
minimist "^1.2.5"

Expand Down Expand Up @@ -18283,7 +18305,7 @@ sort-keys@^2.0.0:
dependencies:
is-plain-obj "^1.0.0"

source-list-map@^2.0.0:
source-list-map@^2.0.0, source-list-map@^2.0.1:
version "2.0.1"
resolved "https://registry.yarnpkg.com/source-list-map/-/source-list-map-2.0.1.tgz#3993bd873bfc48479cca9ea3a547835c7c154b34"
integrity sha512-qnQ7gVMxGNxsiL4lEuJwe/To8UnK7fAnmbGEEH8RpLouuKbeEm0lhbQVFIrNSuB+G7tVrAlVsZgETT5nljf+Iw==
Expand Down Expand Up @@ -19708,6 +19730,11 @@ type-fest@^0.8.0, type-fest@^0.8.1:
resolved "https://registry.yarnpkg.com/type-fest/-/type-fest-0.8.1.tgz#09e249ebde851d3b1e48d27c105444667f17b83d"
integrity sha512-4dbzIzqvjtgiM5rw1k5rEHtBANKmdudhGyBEajN01fEyhaAIhsoKNy6y7+IN93IfpFtwY9iqi7kD+xwKhQsNJA==

type-fest@^1.0.1:
version "1.0.1"
resolved "https://registry.yarnpkg.com/type-fest/-/type-fest-1.0.1.tgz#2494455e65c59170ec98bdda05b7d7184f5b74ad"
integrity sha512-+UTPE7JT3O+sUpRroRgQAbbSfIRBwOHh+o/oruB1JJE6g6uBm3Y0D82fO3xu8VHfxJLQjeRp0PEY6mRmh/lElA==

type-is@~1.6.17, type-is@~1.6.18:
version "1.6.18"
resolved "https://registry.yarnpkg.com/type-is/-/type-is-1.6.18.tgz#4e552cd05df09467dcbc4ef739de89f2cf37c131"
Expand Down Expand Up @@ -20530,6 +20557,14 @@ webpack-sources@^1.1.0, webpack-sources@^1.4.0, webpack-sources@^1.4.1, webpack-
source-list-map "^2.0.0"
source-map "~0.6.1"

webpack-sources@^2.2.0:
version "2.2.0"
resolved "https://registry.yarnpkg.com/webpack-sources/-/webpack-sources-2.2.0.tgz#058926f39e3d443193b6c31547229806ffd02bac"
integrity sha512-bQsA24JLwcnWGArOKUxYKhX3Mz/nK1Xf6hxullKERyktjNMC4x8koOeaDNTA2fEJ09BdWLbM/iTW0ithREUP0w==
dependencies:
source-list-map "^2.0.1"
source-map "^0.6.1"

webpack@^4.44.1:
version "4.44.1"
resolved "https://registry.yarnpkg.com/webpack/-/webpack-4.44.1.tgz#17e69fff9f321b8f117d1fda714edfc0b939cc21"
Expand Down