Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"gatsby build" breaks when dependencies use es6 code (which Uglify doesn't support) #3780

Closed
szimek opened this issue Jan 30, 2018 · 47 comments
Labels
type: question or discussion Issue discussing or asking a question about Gatsby

Comments

@szimek
Copy link
Contributor

szimek commented Jan 30, 2018

Description

After upgrading Gatsby to 1.9.167, I'm getting the following error when I try to run gatsby build:

error Generating JavaScript bundles failed

  Error: component---src-templates-post-jsx-dbdaebf346a9292b96a8.js from UglifyJs
  SyntaxError: Unexpected token: name (isLastCharLower) [./~/camelcase/index.js:4,0]

The line mentioned in the error contains keyword let. Looks like (at least) dependencies are no longer compiled to ES5 and UglifyJS version used by Gatsby doesn't understand ES6+, so it throws an error.

The same app works fine with 1.9.166. It's most likely caused by some changes introduced in #3369, other changes introduced in this version don't look suspicious.

To replicate this issue create a new Gatsby project, run yarn add camelcase, import and call camelcase in any component and run gatsby build.

@617dev
Copy link
Contributor

617dev commented Jan 30, 2018

i am also seeing this. except for me it's running out of memory when trying to build the JS bundles, however pinning to the version you mentioned 1.9.166 works fine...
@szimek when you say some code isn't being compiled to ES5, do you know which code? I'd like to see if ES5-ing it would fix the error with the newer versions to prove your theory...

@szimek
Copy link
Contributor Author

szimek commented Jan 31, 2018

In my case it's caused by camelcase package (which is a dependency of a markdown to html library I'm using), but there are others available only in ES6 as well. This error has already been reported sindresorhus/camelcase#21 and it won't be "fixed" - more info about that sindresorhus/ama#446, which makes perfect sense from the maintainer point of view.

However, this means that all dependencies should also be compiled by Babel to ES5, because they might be written in ES6, so even if Gatsby started using UglifyJS2, which does support ES6 syntax, it would break in older browsers that don't support ES6.

@szimek szimek changed the title v1.9.167 throws UglifyJS error when running "gatsby build" v1.9.167 no longer compiles dependencies to ES5; breaks "gatsby build" command Feb 2, 2018
@ryami333
Copy link

ryami333 commented Feb 4, 2018

Experiencing this too - big dealbreaker as far as I'm concerned.

@KyleAMathews
Copy link
Contributor

Gatsby has never compiled code in node_modules to es5. If you're seeing problems then it's because either an existing dependency of yours started shipping es6 code or (more likely) you added a new dependency with es6 cod which broke uglify.

CRA will be compiling code in node_modules w/ babel starting in their v2 which we can consider as well.

facebook/create-react-app#3776

@szimek that PR only affects the Typescript plugin. We haven't changed anything related to Babel in a long time.

@alexjsdev are you using the typescript plugin?

@KyleAMathews KyleAMathews changed the title v1.9.167 no longer compiles dependencies to ES5; breaks "gatsby build" command "gatsby build" breaks when dependencies use es6 code (which Uglify doesn't support) Feb 4, 2018
@ryami333
Copy link

ryami333 commented Feb 4, 2018

I would have thought so too @KyleAMathews, but I can verify that it builds correctly on Gatsby 1.9.166 but not on the latest (1.9.178 at time of writing), so whether or not the dependency in question has changed is irrelevant (and it hasn't, I checked).

@KyleAMathews
Copy link
Contributor

@ryami333 fair enough — could you post a link to a site reproducing this problem? All the example sites and gatsbyjs.org are still building correctly AFAIK so something reproducing this would be great.

@ryami333
Copy link

ryami333 commented Feb 4, 2018

I'll see what I can do, but it might take a little while. In the meantime, I verified that the break is between builds 1.9.166 and 1.9.167.

@617dev
Copy link
Contributor

617dev commented Feb 4, 2018

@KyleAMathews no we are not using typescript, just the out of the box gatsby starter setup, with what was the latest Gatsby at the time which was 1.9.176 (i believe). We would get an UglifyJS running out of memory error when Gatsby would try to compile our JS bundles, but then i found this thread and based on @szimek suggestion tried pinning the version to 1.9.166 and voila! it works... I'm not sure what changed, do you have a changelog for each version, or like a tagged merged commit for each where I could try to help figure this out? I really don't want to leave the version pinned.

On a side note, I tried updating to the latest a few days ago (1.9.177) and although gatsby would successfully run develop if i ran a build it would hang on createPagesStatefully forever... so different behavior (than the uglify JS error but still no functional build)

Thanks for all your help! And btw Gatsby is truly genius!

@617dev
Copy link
Contributor

617dev commented Feb 4, 2018

@KyleAMathews just saw that there's a newer version 1.9.178 and it looked like there was a fix merged for the hanging i was talking about... well it's true it doesn't hang there anymore but this is back now 😢:

error Generating JavaScript bundles failed

  Error: app-45a58aeb1a29bc012abd.js from UglifyJs
  RangeError: Maximum call stack size exceeded```

@ryami333
Copy link

ryami333 commented Feb 4, 2018

@alexjsdev is your issue actually related to the OP? Have you verified that your dependency is indeed ES6?

@ryami333
Copy link

ryami333 commented Feb 4, 2018

@KyleAMathews I've created a demo repo here, based on gatsby-starter-default.

  • Commit 1, initial commit (fresh gatsby-starter-default).
  • Commit 2, demonstrate successful build with ES6 dependency on 1.9.166.
  • Commit 3, demonstrate failing build with 1.9.167.
  • Commit 4, demonstrate failing build on latest release.

@szimek
Copy link
Contributor Author

szimek commented Feb 4, 2018

@KyleAMathews

Gatsby has never compiled code in node_modules to es5.

You're right. Turns out that Gatsby 1.9.166 and earlier versions silently skip the minification of a file, if it uses ES6 syntax after bundling, e.g. when I use camelcase library in one of pages. If I use it in pages/index.js, all files will be minified except public/component---src-pages-index-js-xxx.js. Gatsby 1.9.167 and later versions throw an error in this case.

According to facebook/create-react-app#1125 (comment) it's not safe to run Babel 6 on node_modules and Babel 7 will be added in Gatsby v2, right?

Here's PR that adds compiling dependencies with babel-preset-env to create-react-app: facebook/create-react-app#3776

@danoc
Copy link
Contributor

danoc commented Feb 5, 2018

I can also confirm that this is happening.

In my particular case, it is caused by a let used in buble, a dependency included within react-live.

Downgrading to 1.9.166 does fix the problem. Haven't dug into what the proper fix should be.

@KyleAMathews
Copy link
Contributor

@szimek yeah, planning on copying CRA's implementation for Gatsby v2.

@KyleAMathews
Copy link
Contributor

Folks running into this — what do think we should do? Remove the error? Add easy way to compile problematic packages? Just removing the error doesn't really solve much as now you're shipping unminified es6 code which is a) slower cause it's unminified and b) will break older browsers that don't support whatever es6 features are used in the code.

@ryami333
Copy link

ryami333 commented Feb 6, 2018 via email

@KyleAMathews
Copy link
Contributor

That could be a possibility too — I haven't researched it. Actually, just researched it :-) and it looks like it might require webpack v3 — so we should do this upgrade along with the remainder of the Gatsby v2 work facebook/create-react-app#3618

@szimek
Copy link
Contributor Author

szimek commented Feb 6, 2018

Why isn't using an es6-compatible version of Uglify an option?

There are 2 issues. When Gatsby switches to babel-preset-env it will need to use ES6-compatible minifier (most likely UglifyJS2), because user's code can end up with ES6+ syntax as well. However, if I configure babel-preset-env to support IE9 or Safari 9, I'd expect all code to work in these browsers, including third-party code that might use ES6+ syntax, not just the code I wrote.

@Kikobeats
Copy link
Contributor

Kikobeats commented Feb 21, 2018

@KyleAMathews Could be possible setup an Uglify ES6-compatible on gatsby-node.js?

Just as workaround to avoid not update gatsby dependency.

Looks like, if I don't update gatsby this error appears:

GraphQL Error There was an error while compiling your site's GraphQL queries.
  Error: Cannot use GraphQLSchema "[object Object]" from another module or realm.

Ensure that there is only one instance of "graphql" in the node_modules
directory. If different versions of "graphql" are the dependencies of other
relied on modules, use "resolutions" to ensure only one version is installed.

https://yarnpkg.com/en/docs/selective-version-resolutions

Duplicate "graphql" modules cannot be used at the same time since different
versions may have different capabilities and behavior. The data from one
version used in the function from another could produce confusing and
spurious results.

EDIT: In my case looks a very isolate problem related with a subdependency from react-live
I opened an issue FormidableLabs/react-live#61. If you want to reproduce the problem just:

git clone -b update [email protected]:microlinkhq/www.git
npm install && npm run build

@KyleAMathews
Copy link
Contributor

@Kikobeats it's best to fix your dependency by compiling it to es5. Not updating Gatsby just means your build succeeds not that the underlying problem, that your code isn't minifying and that it's not runnable on older browsers, is fixed.

@KyleAMathews
Copy link
Contributor

Working on fixes for this in Gatsby v2. We'll be compiling all code in node_modules + upgrading to the latest version of uglify.

@chadwatson
Copy link

@m-allanson thank you so much for your help!

@RobertoMSousa
Copy link
Contributor

Did anyone found a solution for this? I'm having the same problem. Here are my gatsby.config

module.exports =  {
  siteMetadata: {
    title: '4Humans',
  },
  plugins: [
    `gatsby-plugin-react-next`,
    `gatsby-plugin-core-js`,
    'gatsby-plugin-react-helmet',
    `gatsby-plugin-sass`,
    `gatsby-plugin-typescript`
  ],
}

and my dependencies

"dependencies": {
    "gatsby": "^1.9.166",
    "gatsby-link": "^1.6.40",
    "gatsby-plugin-catch-links": "^1.0.22",
    "gatsby-plugin-core-js": "^1.0.2",
    "gatsby-plugin-google-analytics": "^1.0.31",
    "gatsby-plugin-mailchimp": "^2.2.3",
    "gatsby-plugin-react-helmet": "^2.0.11",
    "gatsby-plugin-react-next": "^1.0.11",
    "gatsby-plugin-sass": "^1.0.26",
    "gatsby-plugin-sharp": "^1.6.46",
    "gatsby-plugin-typescript": "^1.4.20",
    "gatsby-source-filesystem": "^1.5.36",
    "gatsby-transformer-remark": "^1.7.41",
    "gatsby-transformer-sharp": "^1.6.26",
    "react-helmet": "^5.2.0",
    "react-particles-js": "^2.2.0",
    "tslint": "^5.10.0",
    "typescript": "^2.9.2",
    "typography-theme-grand-view": "^0.15.10"
  },

@natesjacobs
Copy link

Hi all. I'm having similar issue with 'query-string' package. I'm fine with not having my site working on older browsers... what are people doing to get to solution where it just skips over the ES6 packages? I saw @KyleAMathews is pushing out v2 (woohoo! thanks dude!) but that's causing other errors for me so wanted to find a solution while v2 settles in :)

@RobertoMSousa
Copy link
Contributor

@natesjacobs I solved my problem my removing async/await from my app. Apparently, the v1 does not support it yet. Check if you have any and change it to the good old callbacks ;-)

@fsegouin
Copy link

@natesjacobs I ran into the same issue and had to use query-string@5.

@greypants
Copy link

greypants commented Aug 7, 2018

I had same issue upgrading to to a module dependency that uses let and const. The solution that finally worked for me was whitelisting specific modules in the babel loader via the weback config. I used the example in the docs as a starting point, and ended up with something like this in gatsby-node.js:

exports.modifyWebpackConfig = ({ config, stage }) => {
  // https://www.gatsbyjs.org/docs/add-custom-webpack-config/#modifying-the-babel-loader
  const program = {
    directory: __dirname,
    browserslist: ["> 1%", "last 2 versions", "IE >= 9"]
  };

  return generateBabelConfig(program, stage).then(babelConfig => {
    config.removeLoader("js").loader("js", {
      test: /\.jsx?$/,
      exclude: modulePath => {
        return (
          /node_modules/.test(modulePath) &&
          // whitelist specific es6 modules
          !/node_modules\/(react-docgen|ast-types|recast)/.test(modulePath)
        );
      },
      loader: "babel",
      query: babelConfig
    });
  });
}

@greypants
Copy link

We'll be compiling all code in node_modules

@KyleAMathews Won't that significantly increase the build time? Sticking with exclude and just whitelisting (like I've done above) prevents babel from evaluating more than it needs to, right? Maybe just needs to be documented. Plus I think the webpack + uglify upgrade takes care of this specific issue anyway.

@KyleAMathews
Copy link
Contributor

Yes — we've actually decided to wait and see a bit more on the issue of compiling node_modules. Both because of the slowdowns and because there's a lot of active discussions around how it should work.

@Chuloo Chuloo added the type: question or discussion Issue discussing or asking a question about Gatsby label Aug 8, 2018
@robwalkerco
Copy link
Contributor

If anyone else comes across this issue when they are using Gatsby V2, the above examples will not work due to Gatsby API changes, but the below example did (at least for me).

exports.onCreateWebpackConfig = ({ loaders, actions }) => {
  actions.setWebpackConfig({
    module: {
      rules: [
        {
          test: /\.js$/,
          exclude: modulePath =>
            /node_modules/.test(modulePath) &&
            // whitelist specific es6 module
            !/node_modules\/@papertrailio\/js-core/.test(modulePath),
          use: loaders.js(),
        },
      ],
    },
  })
}

In the above example, the @papertrailio/js-core module included Flow definitions, so adding the above code to gatsby-node.js, and adding the gatsby-plugin-flow plugin to gatsby-config.js worked perfectly.

@angeloashmore
Copy link
Contributor

Could this functionality be outsourced to a plugin? Something where you could just provide a list of node_module packages that need to be transpiled. In the implementation, it just does the onCreateWebpackConfig above.

@KyleAMathews
Copy link
Contributor

Yeah! This should totally be a plugin. Anyone up for creating it?

@robwalkerco
Copy link
Contributor

@KyleAMathews I will have a crack at a plugin for Gatsby V2 using something like my example above (#3780 (comment))

Perhaps I should call it gatsby-plugin-es6-webpack-package?

@KyleAMathews
Copy link
Contributor

@robwalkerco awesome! How about maybe gatsby-plugin-compile-es6-packages?

@robwalkerco
Copy link
Contributor

robwalkerco commented Sep 20, 2018

@KyleAMathews Ah, thats a good name.

I published it as gatsby-plugin-es6-webpack-modules a few hours ago and the Gatsby Plugins directory has picked it up already at https://www.gatsbyjs.org/packages/gatsby-plugin-es6-webpack-modules/

Could try to get the name changed though.

If anyone else finds this issue, then the plugin will help you to customise the webpack config to compile es6 modules in node_modules.

Open to PR's to improve the readme etc

@robwalkerco
Copy link
Contributor

After sleeping on it, I've decided that @KyleAMathews plugin name makes much more sense, so have moved the plugin to https://www.npmjs.com/package/gatsby-plugin-compile-es6-packages

I hope you find it useful 😄

Also closing this issue now, as using this plugin and configuring it with the npm packages that have es6 code in them will allow them to be compiled by Babel.

@dustinhorton
Copy link
Contributor

dustinhorton commented Dec 3, 2018

I'm working in a 'monorepo' and trying to figure out if I need @robwalkerco's plugin. The app is structured like:

monorepo-app/
  |- packages/
     | - common/
     | - gatsby-app-1/
     | - gatsby-app-2/

…where common includes shared components, libs, etc.

Should I be adding common as a package to this plugin if I'm using es6, or is there something more to it? gatsby build doesn't error with the UglifyJS message mentioned here.

indiesquidge pushed a commit to indiesquidge/mdx that referenced this issue Aug 29, 2019
Unclear whether this is a steadfast solution or a patch for the issue of
having ES6+ code in a npm dependency, but for now it allows the docs
site to be run locally.

Ref: gatsbyjs/gatsby#3780
Ref: mdx-js#542
indiesquidge pushed a commit to indiesquidge/mdx that referenced this issue Aug 29, 2019
Unclear whether this is a steadfast solution or a patch for the issue of
having ES6+ code in a npm dependency, but for now it allows the docs
site to be run locally.

Ref: gatsbyjs/gatsby#3780
Ref: mdx-js#542
indiesquidge pushed a commit to indiesquidge/mdx that referenced this issue Aug 29, 2019
Unclear whether this is a steadfast solution or a patch for the issue of
having ES6+ code in a npm dependency, but for now it allows the docs
site to be run locally.

Ref: gatsbyjs/gatsby#3780
Ref: mdx-js#542
johno pushed a commit to mdx-js/mdx that referenced this issue Aug 30, 2019
Unclear whether this is a steadfast solution or a patch for the issue of
having ES6+ code in a npm dependency, but for now it allows the docs
site to be run locally.

Ref: gatsbyjs/gatsby#3780
Ref: #542
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: question or discussion Issue discussing or asking a question about Gatsby
Projects
None yet
Development

No branches or pull requests