Module build failed: unable to use this non-serializable babel config in this node process

[vlascik]

After updating from embroider 2.x some addons with babel transforms started to break the build.

Were there any changes to those (couldn't find any documentation), or is it a bug?

Repro:

git clone https://github.com/vlascik/repro-babel-transforms
cd repro-babel-transforms
npm install
ember serve

Build Error (PackagerRunner) in assets\repro-babel-transforms.js

Module build failed (from ../../thread-loader/dist/cjs.js):
Thread Loader (Worker 0)
[BABEL] P:\Projects\repro-babel-transforms\node_modules\.embroider\rewritten-app\assets\repro-babel-transforms.js\repro-babel-transforms.js: unable to use this non-serializable babel config in this node process (While processing: "P:\\Projects\\repro-babel-transforms\\node_modules\\@embroider\\core\\src\\portable-babel-launcher.js")

    at Portable.hydrate (P:\Projects\repro-babel-transforms\node_modules\@embroider\core\src\portable.js:111:35)
    at P:\Projects\repro-babel-transforms\node_modules\@embroider\core\src\portable.js:103:55
    at Array.map (<anonymous>)
    at Portable.hydrate (P:\Projects\repro-babel-transforms\node_modules\@embroider\core\src\portable.js:103:26)
    at P:\Projects\repro-babel-transforms\node_modules\@embroider\core\src\portable.js:123:77
    at P:\Projects\repro-babel-transforms\node_modules\lodash\mapValues.js:38:34
    at P:\Projects\repro-babel-transforms\node_modules\lodash\_createBaseFor.js:17:11
    at baseForOwn (P:\Projects\repro-babel-transforms\node_modules\lodash\_baseForOwn.js:13:20)
    at mapValues (P:\Projects\repro-babel-transforms\node_modules\lodash\mapValues.js:37:3)
    at Portable.hydrate (P:\Projects\repro-babel-transforms\node_modules\@embroider\core\src\portable.js:123:48)
    at P:\Projects\repro-babel-transforms\node_modules\@embroider\core\src\portable.js:123:77
    at P:\Projects\repro-babel-transforms\node_modules\lodash\mapValues.js:38:34
    at P:\Projects\repro-babel-transforms\node_modules\lodash\_createBaseFor.js:17:11
    at baseForOwn (P:\Projects\repro-babel-transforms\node_modules\lodash\_baseForOwn.js:13:20)
    at mapValues (P:\Projects\repro-babel-transforms\node_modules\lodash\mapValues.js:37:3)
    at Portable.hydrate (P:\Projects\repro-babel-transforms\node_modules\@embroider\core\src\portable.js:123:48)
    at babelLauncher (P:\Projects\repro-babel-transforms\node_modules\@embroider\core\src\portable-babel-launcher.js:6:22)
    at async (P:\Projects\repro-babel-transforms\node_modules\@babel\core\lib\gensync-utils\async.js:36:33)
    at async (P:\Projects\repro-babel-transforms\node_modules\gensync\index.js:186:15)
    at P:\Projects\repro-babel-transforms\node_modules\gensync\index.js:216:13
    at Generator.next (<anonymous>)
    at P:\Projects\repro-babel-transforms\node_modules\@babel\core\lib\config\full.js:181:21
    at Generator.next (<anonymous>)
    at Function.<anonymous> (P:\Projects\repro-babel-transforms\node_modules\@babel\core\lib\gensync-utils\async.js:21:3)
    at Generator.next (<anonymous>)
    at step (P:\Projects\repro-babel-transforms\node_modules\gensync\index.js:269:25)
    at evaluateAsync (P:\Projects\repro-babel-transforms\node_modules\gensync\index.js:291:5)
    at Function.errback (P:\Projects\repro-babel-transforms\node_modules\gensync\index.js:113:7)
    at errback (P:\Projects\repro-babel-transforms\node_modules\@babel\core\lib\gensync-utils\async.js:66:18)
    at async (P:\Projects\repro-babel-transforms\node_modules\gensync\index.js:188:17)
    at onFirstPause (P:\Projects\repro-babel-transforms\node_modules\gensync\index.js:216:13)
    at Generator.next (<anonymous>)
    at cachedFunction (P:\Projects\repro-babel-transforms\node_modules\@babel\core\lib\config\caching.js:52:46)
    at cachedFunction.next (<anonymous>)
    at loadPluginDescriptor (P:\Projects\repro-babel-transforms\node_modules\@babel\core\lib\config\full.js:260:42)
    at loadPluginDescriptor.next (<anonymous>)
    at loadPluginDescriptors (P:\Projects\repro-babel-transforms\node_modules\@babel\core\lib\config\full.js:129:33)
    at loadPluginDescriptors.next (<anonymous>)
    at P:\Projects\repro-babel-transforms\node_modules\@babel\core\lib\config\full.js:157:21
    at Generator.next (<anonymous>)
    at loadFullConfig (P:\Projects\repro-babel-transforms\node_modules\@babel\core\lib\config\full.js:141:5)
    at loadFullConfig.next (<anonymous>)
    at transform (P:\Projects\repro-babel-transforms\node_modules\@babel\core\lib\transform.js:20:45)
    at transform.next (<anonymous>)
    at step (P:\Projects\repro-babel-transforms\node_modules\gensync\index.js:269:25)
    at P:\Projects\repro-babel-transforms\node_modules\gensync\index.js:273:13
    at async.call.result.err.err (P:\Projects\repro-babel-transforms\node_modules\gensync\index.js:223:11)
    at P:\Projects\repro-babel-transforms\node_modules\gensync\index.js:189:28
    at P:\Projects\repro-babel-transforms\node_modules\@babel\core\lib\gensync-utils\async.js:68:7
    at P:\Projects\repro-babel-transforms\node_modules\gensync\index.js:113:33

[ef4]

Nothing deliberately changed in 3.x.

I wonder if you're getting some state leakage across separate builds. If you see this error, then rm -rf node_modules/.embroider and then try again, does it go away?

[vlascik]

I wonder if you're getting some state leakage across separate builds. If you see this error, then rm -rf node_modules/.embroider and then try again, does it go away?

Nope, the error stays the same regardless of whether that folder exists or not. From what I have seen in the debugger, this htmlbars ast transform/plugin gets marked as "global", whatever that means in embroider parlance, and then throws at node_modules\@embroider\core\src\portable.js:111

[mansona]

So we debugged this today in the triage meeting. It looks like there is something going on with the addon ember-bind-helper in the way that it's registering it's ast transform

It looks like they are passing an object to registry.add() https://github.com/Serabe/ember-bind-helper/blob/master/index.js#L7-L13

but that required object won't be serializable. I don't know how or why that's a requirement or even if something has changed in this regard recently 🤷 but I found just passing the plugin by name seems to work, even though the repro repo didn't have an example of using the ember-bind-helper so I wasn't able to verify.

setupPreprocessorRegistry(type, registry) {
    registry.add("htmlbars-ast-plugin", {
      name: "bind",
      plugin: "ember-bind-helper/lib/bind-transform"
    });
  }

[vlascik]

Thank you! This fixed the problem in this case, but I suspect it will reappear with other addons later: according to EmberObserver codesearch, passing require into ast transform's plugin is fairly common (20 addons).

https://emberobserver.com/code-search?codeQuery=plugin%3A%20require

[ef4]

@mansona thanks, we should add this to the list of things to officially deprecate upstream. We aren't ever going to be able to fully support non-serializable plugins safely. It definitely won't work in vite.

Embroider also has a pluginHints option that lets you deal with an addon that is injecting non-serializable plugins. In this case, assuming ember-bind-helper is a direct dependency of the app, it would be pluginHints: [ { resolve: ["ember-bind-helper/lib/bind-transform"] } ]

That still didn't explain why this changed from a warning to an error, because we used to detect whether the config was serializable (and thus safe to parallelized across processes) or not, and force everything to run single-process when it's not serializable. So I went looking and, well, oops:

embroider/packages/compat/src/compat-app-builder.ts

Line 970 in b6dee02

isParallelSafe: true, // TODO

I guess we need test coverage for this case.

[ef4]

passing require into ast transform's plugin is fairly common (20 addons).

Many of those are serializable though because they use an explicit API to become so. The first hit I saw on that list is our own @embroider/addon-shim and its plugin uses parallelBabel to make itself serializable:

https://github.com/embroider-build/embroider/blob/main/test-packages/sample-transforms/index.js#L23-L30

The ability to pass a name instead of a function is relatively recent. I added it in babel-plugin-ember-template-compilation. For most of time, addons had to pass a function, but they could annotate it with parallelBabel to make it serializable. So that is the most common case you'll find in the wild.

Since many large apps run with the ember-cli-babel setting throwUnlessParallelizable: true for the performance benefits, a lot of the non-serializable cases in the ecosystem have been fixed. But of course not all of them.

[boris-petrov]

I get the same error when trying my project with embroider. I do have ember-box (which has the same "problem" as ember-bind-helper) but even if I apply the fix from above or even remove it from my package.json- the error remains. How can I figure out what the problem is?

[ef4]

How can I figure out what the problem is?

Without embroider, add throwUnlessParallelizable: true to your ember-cli-babel settings. It should print the source code of the unserializable function. You can search for that to see which addon it's coming from.

[boris-petrov]

Thanks! I have 2:

1: name: unknown, location: unknown
2: name: unknown, location: unknown

😄

I have a slight idea that it might be ember-cli-styles but how do I find for sure?

[boris-petrov]

I'm confused. Having the simplest Ember app - when one adds includeExternalHelpers - it stops working with parallelization:

[Babel: Babel: foo] was configured to `throwUnlessParallelizable` and was unable to parallelize a plugin.
plugins:
1: name: unknown, location: unknown

Is that known and how does one workaround it?

[runspired]

I hit this tonight and realized in my case it was because objects with null prototypes are being treated as unserializable.

I had an object with a null prototype because I was using import * as Settings from './' as a pattern. Settings in this case gets a null prototype. Object.assign({}, Settings) was enough to fix it.