[Build Error] Out of memory when pages exceed 95

[siriwatknp]

Description

Everything works fine when pages are 95, but after adding one more simple page and run build I got this error.

I searched and tried this node --max_old_space_size=12000 ./node_modules/.bin/gatsby build. There is no error but the build time is very long (from 100s to around 600s with 1 more page) and still fail in Circle CI, so I don't think that is a solution. I totally stuck and need help.

Steps to reproduce

Clone this project. https://github.com/siriwatknp/mui-treasury

// at master branch
yarn install
yarn build

Expected result

It should be able to build. If delete some page in /website/src/pages/styles/... and then run yarn build it will pass. 😱

Actual result

Build fail with the error above.

Environment

System:
OS: macOS 10.15.1
CPU: (12) x64 Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
Shell: 5.7.1 - /bin/zsh
Binaries:
Node: 10.16.3 - ~/.nvm/versions/node/v10.16.3/bin/node
Yarn: 1.21.1 - /usr/local/bin/yarn
npm: 6.9.0 - ~/.nvm/versions/node/v10.16.3/bin/npm
Languages:
Python: 2.7.16 - /usr/bin/python
Browsers:
Chrome: 80.0.3987.149
Firefox: 73.0.1
Safari: 13.0.3

[micaelr95]

This doesn't seem like a bug or problem in Gatsby but rather a limitation in Nodejs.

As you pointed out, using the flag --max-old-space-size to increase the amount of RAM used by Nodejs seems to do the trick.

[pvdz]

Well, that kind of depends. Considering this is <100 pages nothing should OOM with default RAM settings. It is possible, of course, if you put enough content in each page. That's not generally the case, so I'll check this out.

Additionally I'm interested to see which stringify is OOMing here, since the heavy serialization goes through v8.serialize these days :)

Thanks for the report

[siriwatknp]

@pvdz Thanks, it could possibly my fault that I make mistake somewhere but I have know idea right now.

[kuworking]

I've opened a new issue #22717 that sounds very much like this one, could you test if downgrading to gatsby 2.20.5 solves your issue?

[siriwatknp]

@kuworking I tried but it does not solve my issue.

[freiksenet]

Potential fix published in 2.20.12, does it work for you?

[pvdz]

@siriwatknp when I try to build your site it OOMs when creating a webpack bundle (this is the "Building production JavaScript and CSS bundles" step). Might be caused by minification (although running it with --no-uglify runs into different problems) but either way indicates there's just too much JS For the site.

So this is unrelated to page count, or the telemetry issue mentioned by @freiksenet, but related to how much client side JS webpack is processing.

Try to have a look at how you might reduce that. Your config contains a lot of plugins and I think you're including a bunch as well. Was that intentional?

[siriwatknp]

@pvdz Have u ever seen other use case that use a lot of JS, do they face the same issue I have? If I want to handle a lot of pages, I should do it with createPage API instead of creating js under pages folder, is that correct?

[kuworking]

I've tried (287 pages) with gatsby 2.20.12 and gatsby develop works, and gatsby build works

Still, I get an ERROR label that I guess it's because in my core theme requires gatsby 2.20.5 (see image)

Other than that gatsby serve looks well

[pvdz]

Yeah that error seems to be caused by the update notification that was injected by yarn/npm. Annoying but kind of out of Gatsby's control :(

[pvdz]

@siriwatknp eh, tbh, I actually don't know the perf implications of that choice. All I see is createPage api, though. I'm not sure if that should affect the JS bundle, or why it would, but maybe.

I've seen issues like this before and it's usually a configuration issue of some kind. Perhaps @wardpeet has a better insight.

[siriwatknp]

I will close this issue for now.