[gatsby] core-js@3 incompatibility: Compile error, dependency not found

[MarcusCemes]

Description

This seems like a webpack error, but I thought it might be more relevant here, as gatsby heavily manages webpack?

Unable to install and use Redoc. The compiler fails on line 6454 of node_modules/redoc/bundles/redoc.lib.js:

module.exports = require("core-js/es/promise");

After checking node_modules, I can confirm the existence of core-js/es/promise/index.js with the following export:

var path = require('../../internals/path');
module.exports = path.Promise;

VS Code is also able to resolve the required path with a CTRL+Click.

Gatsby develop error

 ERROR  Failed to compile with 1 errors                                                                                                                                                       
This dependency was not found:
⠀
* core-js/es/promise in ./node_modules/redoc/bundles/redoc.lib.js
⠀
To install it, you can run: npm install --save core-js/es/promise
× ｢wdm｣:
ERROR in ./node_modules/redoc/bundles/redoc.lib.js
Module not found: Error: Can't resolve 'core-js/es/promise' in 'C:\<PROJECT_PATH>\node_modules\redoc\bundles'
 @ ./node_modules/redoc/bundles/redoc.lib.js 6454:17-46
 @ ./src/pages/api.tsx
 @ ./.cache/sync-requires.js
 @ ./.cache/app.js
 @ multi ./node_modules/event-source-polyfill/src/eventsource.js (webpack)-hot-middleware/client.js?path=/__webpack_hmr&reload=true&overlay=false C://<PROJECT_PATH>//.cache//app
i ｢wdm｣: Failed to compile.

Gatsby build error

ERROR #98123  WEBPACK

Generating JavaScript bundles failed

Can't resolve 'core-js/es/promise' in 'C:\<PROJECT_PATH>\node_modules\redoc\bundles'

File: node_modules\redoc\bundles\redoc.lib.js

⠙ Building production JavaScript and CSS bundles

Steps to reproduce

Absolute minimal example (may need to fork to run the sandbox):

https://codesandbox.io/embed/gatsbyredoc-dependancy-not-found-bug-csr12

1. Install redoc npm i redoc, and its peer dependancies npm i mobx styled-components core-js
2. Import redoc into a project and try to run gatsby

import { RedocStandalone } from "redoc";
...
<RedocStandalone ... />

Expected result

The project compiles successfully

Actual result

Webpack is unable to resolve a require(...) dependency.

Environment

My enviroment

  System:
    OS: Windows 10
    CPU: (16) x64 AMD Ryzen 7 2700X Eight-Core Processor
  Binaries:
    npm: 6.10.2 - C:\Program Files\nodejs\npm.CMD
  Browsers:
    Edge: 44.18362.267.0
  npmPackages:
    gatsby: ^2.14.0 => 2.14.0
    gatsby-image: ^2.2.14 => 2.2.14
    gatsby-plugin-create-client-paths: ^2.1.5 => 2.1.5
    gatsby-plugin-import: ^2.1.5 => 2.1.5
    gatsby-plugin-manifest: ^2.2.9 => 2.2.9
    gatsby-plugin-offline: ^2.2.10 => 2.2.10
    gatsby-plugin-react-helmet: ^3.1.5 => 3.1.5
    gatsby-plugin-remove-trailing-slashes: ^2.1.4 => 2.1.4
    gatsby-plugin-sharp: ^2.2.18 => 2.2.18
    gatsby-plugin-typescript: ^2.1.5 => 2.1.5
    gatsby-plugin-webpack-bundle-analyzer: ^1.0.5 => 1.0.5
    gatsby-source-filesystem: ^2.1.18 => 2.1.18
    gatsby-transformer-sharp: ^2.2.12 => 2.2.12
    gatsby-transformer-sqip: ^2.1.21 => 2.1.21

Codesandbox minimal enviroment

  System:
    OS: Linux 4.19 Debian GNU/Linux 9 (stretch) 9 (stretch)
    CPU: (8) x64 Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz
    Shell: 4.4.12 - /bin/bash
  Binaries:
    Node: 10.16.0 - /tmp/yarn--1566994773572-0.7409837003023974/node
    Yarn: 1.16.0 - /tmp/yarn--1566994773572-0.7409837003023974/yarn
    npm: 6.9.0 - /usr/local/bin/npm
  Languages:
    Python: 2.7.13 - /usr/bin/python
  npmPackages:
    gatsby: ^2.13.73 => 2.13.73

[MarcusCemes]

Forgot to add:

Node.js 12.9.1, npm 6.10.2, Gatsby CLI 2.7.40, Gatsby 2.14.0

gatsby-config.js, gatsby-node.js, gatsby-browser.js and gatsby-ssr.js are all empty in the code sandbox example, void of any plugins.

I will continue diagnosing the issue and attempt to submit a PR, it is probably a fault with core-js or redoc, but my understanding of bundlers require(...) parsing is very limited.

[MarcusCemes]

I have new reduced examples.

Reduced gatsby example, webpack error:
https://codesandbox.io/embed/gatsbywebpack-require-bug-ngkcy

Only webpack, works fine:
https://github.com/MarcusCemes/webpack-require-bug

I assume this is a problem with Gatsby's Webpack configuration, as regular webpack works, as per the documentation. I will continue to do some digging.

[jonniebigodes]

@MarcusCemes i'm inclined to believe that this is more of a incompatibility between the package and gatsby. I've tinkered with both and the only version i was able work with gatsby was 2.0.0-alpha.30, between that and the latest release. I wasn't able to make it work. The error you mentioned was surfacing.

Did some tests with create react app and i was able make it work, also did some tests with another ssr framework, namely nextjs and i was able to make it work.

Also applied some webpack configuration in gatsby-node.js like so:

exports.onCreateWebpackConfig = ({ stage, loaders, actions }) => {
    if (stage === "build-html") {
      actions.setWebpackConfig({
        module: {
          rules: [
            {
              test: /redoc/,
              use: loaders.null(),
            },
          ],
        },
      })
    }
  }

Also tried using it with react-loadable and the issue still persisted. In the meantime i think it's a good idea to post a issue in the package repo and mention this one and see what the feedback is.

I have the version with the alpha version hoisted here.

[MarcusCemes]

Thanks @jonniebigodes for the digging. There is a recent closed (but unresolved and ongoing) issue on what seems to be a related issue (it was deemed "fixed" in 2.0.0-rc14, but the same import is still throwing).

I'll follow and assist in the referenced issue and keep this updated, I'll leave it to a moderator to decide whether to close/keep this issue open, as I still don't know where the problem resides.

[jonniebigodes]

@MarcusCemes no need to thank. i'll keep an eye on this aswell

[thovden]

Also hit this problem today. I had some additional issues importing specific core-js polyfills from my own code as well, with the same error - i.e., import 'core-js/es/some-polyfill' results in error described here. import 'core-js' works fine. So my hunch is that has something to do with the core-js and babel integration, and Gatsby heavily configures babel. https://github.com/zloirock/core-js/blob/master/docs/2019-03-19-core-js-3-babel-and-a-look-into-the-future.md

Will investigate a little bit today.

[thovden]

The problem is that Gatsby uses/hardcodes core-js@2 in the babel config:

gatsby/packages/babel-preset-gatsby/src/index.js

Line 54 in c5cc361

corejs: 2,

The import above is for core-js@3.

[thovden]

There is no simple way out of this, since core-js@2 is hardcoded several places in gatsby. So code to be compiled with gatsby cannot use direct core-js@3 imports for now.

[gatsbot]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.

If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!

As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[gatsbot]

Hey again!

It’s been 30 days since anything happened on this issue, so our friendly neighborhood robot (that’s me!) is going to close it.

Please keep in mind that I’m only a robot, so if I’ve closed this issue in error, I’m HUMAN_EMOTION_SORRY. Please feel free to reopen this issue or create a new one if you need anything else.

As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks again for being part of the Gatsby community!

[MarcusCemes]

As far as I'm concerned, this is still an open issue, as it seems that libraries using core-js@3 (such as redoc) are not compatible with Gatsby. This seems like it would require some refactoring of Gatsby's internal code, which I'm not familiar with.

I'll leave the question of to close or not to close to someone with more of an idea of the complexity of the issue.

[wardpeet]

@MarcusCemes As a workaround you could remove our core-js alias from the webpack config. This might increase your javascript bundle.

exports.onCreateWebpackConfig = ({ actions, loaders, getConfig }) => {
  const config = getConfig()
  delete config.resolve['core-js']
  actions.replaceWebpackConfig(config)
}

I'm not 100% sure the code is in 100% working state but something along these lines should do the trick

[MarcusCemes]

@wardpeet Thank you for the comment, although I was unable to solve the issue.

I am not familiar with Gatsby's under-the-hood configurations, and wouldn't feel safe trying to monkey-patch the configuration. Perhaps it may help someone else. I have also updated the code sandbox with the above-proposed workaround (does not resolve the issue).

[wardpeet]

I think I missed the chaining.

exports.onCreateWebpackConfig = ({ actions, getConfig }) => {
  const config = getConfig();
  delete config.resolve.alias["core-js"];
  actions.replaceWebpackConfig(config);
};

[wardpeet]

Ok sorry for the late response but this came up again, this should fix your issue:

In gatsby-node.js

const path = require('path');

exports.onCreateWebpackConfig = ({ actions, getConfig, stage }) => {
  const config = getConfig();

  const coreJs2config = config.resolve.alias['core-js'];
  delete config.resolve.alias['core-js']
  config.resolve.alias[`core-js/modules`] = `${coreJs2config}/modules`
  try {
    config.resolve.alias[`core-js/es`] = path.dirname(require.resolve('core-js/es'));
  } catch (err) { }
  actions.replaceWebpackConfig(config);
};

so what are we doing here?
We remove the default core-js rule from our webpack config so we don't alias everything to core-js@2. We create 2 new aliases. We create one that aliases all modules inside core-js/modules to core-js@2. We create another one that aliases all modules inside core-js/es to core-js@3.

The try-catch is needed because core-js@3 might not be a direct child of your gatsby site. Technically it's not needed but it will make your bundle a bit smaller as everything will point to the same core-js@3 library so webpack won't load the same file more than once in the same bundle.

[MarcusCemes]

Thank you for your continued support.

The above configuration does allow gatsby to start correctly, but the problem (at least with redoc) has been shifted into React's domain. The redoc component returns {$$typeof: Symbol(react.element), type: undefined, key: null, ref: null, props: {…}, …}, causing a Element type is invalid React error. The code sandbox has again been updated;

This may be an internal issue with the redoc library or a more general issue due to how aliasing between two different core-js (major) versions works. It seems like more effort to solve than it's worth.

After having done another search of gatsby issues (now knowing the cause of this one), this seems to be a duplicate of (or at least solvable by) #15601. I'm closing this issue in favour of that one.

Supporting core-js@3 will most likely also include performance/size improvements for gatsby (not to mention actually being able to use certain libraries without patchwork configurations), I'm all for breaking changes if it results in overall gains.