Upgrade CRA, webpack5, storybook 4.6

[illume]

Upgrade dependencies

Create React App 5 (which uses Webpack 5) and storybook 4.6, and npm audit --fix.

Some dependencies haven't been updated to work with Webpack 5, because webpack 5 doesn't pollyfill node modules with their browser equivalents anymore (it is considered a bug by Create React 5 which they are going to fix inside CRA but they haven't fixed it yet). To work around this until the bug is fixed upstream, patch-package is used to hotfix this issue.

How to use

Everything should work as before.

Testing done

• storybook runs
• tests run
• storyshot tests output in src/src for some reason (fixed)
• lint check
• type check
• make run-front still works
• make docs
• apps build on CI,
• app runs on windows
• app runs on linux (WSL2 ubuntu win11)

---

Create React 5, Webpack 5, Storybook 6.4 upgrade notes

Below are notes made during upgrading and working through issues.

Create React Webpack 5 issues with deps on node modules
facebook/create-react-app#11756

package aliaseshttps://github.com/facebook/create-react-app/issues/11756#issuecomment-998418988

Storybook webpack 5
https://storybook.js.org/blog/storybook-for-webpack-5/

@axe-core
crypto-browserify

"Failed to parse source map"

'@storybook/preset-create-react-app' doesn't work with storybook. Hasn't been updated for CRA 5.

tools which allow own configuration

• patch-package (patches webconfig, but does not require a whole new replacement for CRA) v5 used to include polyfills for node.js core modules by default facebook/create-react-app#11756 (comment)
• craco https://github.com/gsoft-inc/craco
• react-app-rewired https://github.com/timarney/react-app-rewired (not so well maintained as craco)

issues raised

• axe-core/react
  • issue submitted: Module not found: Error: Can't resolve 'crypto' dequelabs/axe-core#2873 (comment)
• @apidevtools/json-schema-ref-parser via @apidevtools/swagger
  • issue submitted: No polyfills for node modules http and https with Webpack 5 (via CRA 5) APIDevTools/json-schema-ref-parser#254
  • ERROR in ./node_modules/@apidevtools/json-schema-ref-parser/lib/resolvers/http.js 3:13-28
    Module not found: Error: Can't resolve 'http' in '/home/rene/dev/headlamp/frontend/node_modules/@apidevtools/json-schema-ref-parser/lib/resolvers'
    If you want to include a polyfill, you need to:
    - add a fallback 'resolve.fallback: { "http": require.resolve("stream-http") }'
    - install 'stream-http'
  • ERROR in ./node_modules/@apidevtools/json-schema-ref-parser/lib/resolvers/http.js 5:14-30
    Module not found: Error: Can't resolve 'https' in '/home/rene/dev/headlamp/frontend/node_modules/@apidevtools/json-schema-ref-parser/lib/resolvers'
    • add a fallback 'resolve.fallback: { "https": require.resolve("https-browserify") }'
    • install 'https-browserify'
• github-markdown (and some other css) fails to compile because of postcss problem.
  • https://github.com/postcss/postcss/wiki/PostCSS-8-for-end-users
  • fixed by installing postcss as a dependency
    • npm install postcss
• ran npm audit fix
• ERROR in Plugin "react" was conflicted between "package.json » @kinvolk/eslint-config" and "BaseConfig » /home/rene/dev/headlamp/frontend/node_modules/eslint-config-react-app/base.js".
  • removing package.json.eslintConfig. "@kinvolk" from there stops it breaking
  • probably have to look at kinvolk/eslint-config
  • after putting it back in... it worked? wtf?
  • also came back after adding a module with npm install.
    • Fixed again by removing that "@kinvolk" line, and adding it back.

Least bad way to use CRA 5

I've added issues to packages affected by no-node-polyfills-in-webpack5, and there might be an option to use polyfills in CRA 5

However, what to do for now?

• wait some months
• drop those dependencies
• use a CRA scripts replacement that allows config like react-app-rewired or craco
• use temporary patch to CRA with patch-package
  • v5 used to include polyfills for node.js core modules by default facebook/create-react-app#11756 (comment)

The least big change which doesn't block us is to use patch-package.

vi node_modules/react-scripts/config/webpack.config.js

  fallback: {
    crypto: require.resolve('crypto-browserify'),
    http: require.resolve('stream-http'),
    https: require.resolve('https-browserify'),
    process: require.resolve('process/browser'),
    stream: require.resolve('stream-browserify'),
  },

process

Uncaught ReferenceError: process is not defined
at Object../node_modules/@apidevtools/json-schema-ref-parser/lib/util/url.js (bundle.js:31294:29)
at Object.options.factory (bundle.js:370810:31)
at webpack_require (bundle.js:370203:33)
at fn (bundle.js:370481:21)
at Object../node_modules/@apidevtools/json-schema-ref-parser/lib/util/errors.js (bundle.js:30998:5)
at Object.options.factory (bundle.js:370810:31)
at webpack_require (bundle.js:370203:33)
at fn (bundle.js:370481:21)
at Object../node_modules/@apidevtools/json-schema-ref-parser/lib/parsers/json.js (bundle.js:29495:5)
at Object.options.factory (bundle.js:370810:31)

node_modules/@apidevtools/json-schema-ref-parser/lib/util/url.js

• @apidevtools/json-schema-ref-parser and @apidevtools/swagger were problematic
  • they don't publish a browser build by default
• Had to do a fresh npm install
• install some dependencies with npm install, so they will run correctly.
  • is it because then webpack builds them?
    "@apidevtools/json-schema-ref-parser
• node-polyfill-webpack-plugin is an option.
  • didn't try it too much
• npm install process stream-browserify stream-http https-browserify crypto-browserify console-browserify

fullySpecified: false

            {
              test: /\.m?js/,
              resolve: {
                fullySpecified: false
              }
            },
const NodePolyfillPlugin = require("node-polyfill-webpack-plugin");
    plugins: [
      // new NodePolyfillPlugin({
      //   excludeAliases: ["console"]
      // }),
      // new webpack.ProvidePlugin({
      //   process: 'process/browser.js',
      //   Buffer: ['buffer', 'Buffer'],
      // }),
      // new webpack.ProvidePlugin({
      //   process: 'process/browser',
      //   // Buffer: ['buffer', 'Buffer'],
      // }),

Issue with npm run storybook

https://storybook.js.org/blog/storybook-for-webpack-5/

• storybook with webpack 4 is not compatible
  • doing an upgrade changes the config to webpack 5
    • npx sb@next upgrade
  • update some fallback webpack config.
    • https://gist.github.com/shilman/8856ea1786dcd247139b47b270912324

This is one of the original issues...

  10% building 1/1 modules 0 active/home/rene/dev/headlamp/frontend/node_modules/html-webpack-plugin/lib/webpack5/file-watcher-api.js:13
      mainCompilation.fileSystemInfo.createSnapshot(
                                    ^

  TypeError: Cannot read properties of undefined (reading 'createSnapshot')
      at /home/rene/dev/headlamp/frontend/node_modules/html-webpack-plugin/lib/webpack5/file-watcher-api.js:13:36
      at new Promise (<anonymous>)
      at Object.createSnapshot (/home/rene/dev/headlamp/frontend/node_modules/html-webpack-plugin/lib/webpack5/file-watcher-api.js:12:10)
      at /home/rene/dev/headlamp/frontend/node_modules/html-webpack-plugin/lib/cached-child-compiler.js:219:35

Issue with Jest

Jest doesn't support ESM modules yet.
remarkjs/react-markdown#635 (comment)

react-markdown is only releasing as ESM.

So we need to add a bunch of jest config in package.json with transformIgnorePatterns to ignore dependencies of react-markdown.

eslint-plugin-react dependency

There was an issue with eslint-plugin-react version conflict: jsx-eslint/eslint-plugin-react#3128 (comment)

typedoc

Had to upgrade typedoc. Re: typedoc2md/typedoc-plugin-markdown#283 (comment)

node 10 EOL, node 16 active LTS

Seems lots of packages are dropping node 10 support, since it's no longer supported. This is the default on Ubuntu 20.04.

Current LTS is node 16. https://nodejs.org/en/about/releases/
Node 12 End of life is in a few months April 30(2022-04-30.)

npm install prod build

• Some production build related dependencies for the apidev json package: npm install util url buffer
• In the production build it also was missing this: npm install postcss-flexbugs-fixes

mac electron build with nodejs 16

From this issue, there appears to be a merged PR with a fix. electron-userland/electron-builder#5862

electron-builder 22.11.2 contains the fix: https://github.com/electron-userland/electron-builder/releases/tag/v22.11.2

22.11.2 failed to build on linux, using [email protected]

  ⨯ editions-autoloader-none-broadened: Unable to determine a suitable edition, even after broadening.  stackTrace=
                                                                                                          Error: editions-autoloader-none-broadened: Unable to determine a suitable edition, even after broadening.
                                                                                                              at new Errlop (/Users/runner/work/headlamp/headlamp/app/node_modules/errlop/edition-es5/index.js:61:18)
                                                                                                              at Object.errtion (/Users/runner/work/headlamp/headlamp/app/node_modules/editions/edition-es5/util.js:23:14)
                                                                                                              at determineEdition (/Users/runner/work/headlamp/headlamp/app/node_modules/editions/edition-es5/index.js:317:21)
                                                                                                              at solicitEdition (/Users/runner/work/headlamp/headlamp/app/node_modules/editions/edition-es5/index.js:350:16)
                                                                                                              at Object.requirePackage (/Users/runner/work/headlamp/headlamp/app/node_modules/editions/edition-es5/index.js:364:9)
                                                                                                              at Object.<anonymous> (/Users/runner/work/headlamp/headlamp/app/node_modules/istextorbinary/index.cjs:4:38)

[illume]

A number of issues for this have been fixed externally, and internally to headlamp in separate PRs.

Next step is rebase this so it is much smaller, and also to try and find a solution to the remaining issue found after the upgrade.

[illume]

The failure here is related to a headlamp-plugin types issue with notifications.

[illume]

@joaquimrocha this one is ready to review.

[joaquimrocha]

When running make frontend it asked me if I wanted to install patch-project. I think it's worth adding this as a dev dependency to avoid that, WDYT?

Otherwise, since it seems to work, and as you said, there's no great alternative to the patching. I think we can go ahead with this (as much as I'd like to avoid adding more complexity to the project).

[illume]

When running make frontend it asked me if I wanted to install patch-project. I think it's worth adding this as a dev dependency to avoid that, WDYT?

This is fixed. I updated how this works according to the latest instructions (adding the package and removing the npx part in the postinstall).