Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Node 20: errors produced by reportTSError are not serialised correctly when using ESM loader #2026

Open
danrr opened this issue May 23, 2023 · 31 comments

Comments

@danrr
Copy link

danrr commented May 23, 2023

Search Terms

Node 20, reportTSError, error

Expected Behavior

When using the ts-node/esm loader with Node 18, TS errors are reported correctly. The file below when run with node --experimental-loader ts-node/esm test.ts

const a = (b: 1 | 2) => b + 1

console.log(a(3))

produces the following correct error:

TSError: ⨯ Unable to compile TypeScript:
test.ts:3:15 - error TS2345: Argument of type '3' is not assignable to parameter of type '1 | 2'.

3 console.log(a(3))
<stack trace>
  diagnosticCodes: [ 2345 ]

Actual Behavior

On Node 20, the error is serialised to only retain diagnosticCodes and no other info

node:internal/process/esm_loader:42
      internalBinding('errors').triggerUncaughtException(
                                ^
{ diagnosticCodes: [ 2345 ] }

Likely related to https://github.com/nodejs/node/blob/main/doc/changelogs/CHANGELOG_V20.md#custom-esm-loader-hooks-run-on-dedicated-thread

Steps to reproduce the problem

Run command above on Node 20

Minimal reproduction

TypeStrong/ts-node-repros#33

https://github.com/TypeStrong/ts-node-repros/actions/runs/5057165129/jobs/9075579140

Specifications

  • ts-node v10.9.1
  • node v18.16.0
  • TS compiler v5.0.4

and

  • ts-node v10.9.1

  • node v20.2.0

  • TS compiler v5.0.4

  • package.json:

{
  "type": "module",
}
  • Operating system and version: MacOS 13.4
  • If Windows, are you using WSL or WSL2?:
@cspotcode
Copy link
Collaborator

cspotcode commented May 23, 2023

I think you're right, the errors are being marshalled from one thread to another, so our INSPECT_CUSTOM is being lost.

ts-node/src/index.ts

Lines 432 to 464 in 7af5c48

/**
* TypeScript diagnostics error.
*/
export class TSError extends BaseError {
name = 'TSError';
diagnosticText!: string;
diagnostics!: ReadonlyArray<_ts.Diagnostic>;
constructor(
diagnosticText: string,
public diagnosticCodes: number[],
diagnostics: ReadonlyArray<_ts.Diagnostic> = []
) {
super(`⨯ Unable to compile TypeScript:\n${diagnosticText}`);
Object.defineProperty(this, 'diagnosticText', {
configurable: true,
writable: true,
value: diagnosticText,
});
Object.defineProperty(this, 'diagnostics', {
configurable: true,
writable: true,
value: diagnostics,
});
}
/**
* @internal
*/
[INSPECT_CUSTOM]() {
return this.diagnosticText;
}
}

Can you think of a good solution here? Is there a way to rehydrate the errors on the main thread so that they're logged correctly?

As far as I remember, they errors are never "caught" per-se, they simply have a custom formatting. So when node prints them to stdout, they are formatted the way we want.

@danrr
Copy link
Author

danrr commented May 24, 2023

OK, I understand, so it's the correct exception, hence the { diagnosticCodes: [ 2345 ] } output but the diagnosticText is being lost in the serialisation of the exception.

I had a look at the code in https://github.com/nodejs/node/blob/main/lib/internal/error_serdes.js and I think I see what's happening.

The if on line https://github.com/nodejs/node/blob/main/lib/internal/error_serdes.js#LL119 is skipped as the type is [object Object], not [object Error].

The if on line https://github.com/nodejs/node/blob/main/lib/internal/error_serdes.js#L137 is skipped because ObjectPrototypeHasOwnProperty(error, customInspectSymbol) returns false -- the check here is done using Object.prototype.hasOwnProperty but, because the error is created from TSError, the INSPECT_CUSTOM is a property of the prototype of the error, not of the error itself. .

This means it falls into the try block here https://github.com/nodejs/node/blob/main/lib/internal/error_serdes.js#LL145 which only serialises the diagnosticCodes (as that's the only own property on the TSError).

The fallback on https://github.com/nodejs/node/blob/main/lib/internal/error_serdes.js#LL150 actually serialises it correctly but it's never reached normally as the block above returns.

I tried to change TSError to move

  this[INSPECT_CUSTOM] = () => {
      return this.diagnosticText;
    }

to the constructor, which caused it to serialise correctly and deserialise to an object with the inspect output of TSError as its inspect output. BUT the output from node is now this:

node:internal/process/esm_loader:42
      internalBinding('errors').triggerUncaughtException(
                                ^
[Object: null prototype] {
  [Symbol(nodejs.util.inspect.custom)]: [Function: [nodejs.util.inspect.custom]]
}

from this line https://github.com/nodejs/node/blob/main/lib/internal/process/esm_loader.js#L42.

I even put console.log("loadESM", err) on the line above the call to internalBinding('errors').triggerUncaughtException and that printed the TSError output correctly, so I'm not sure what is happening.

I tried to set process.setUncaughtExceptionCaptureCallback() but I think I ran into #2024.

Any idea what's happening with this?

@danrr danrr changed the title Node 20: errors produced by reportTSError are not caught when using ESM loader Node 20: errors produced by reportTSError are not serialised correctly when using ESM loader May 25, 2023
@azasypkin
Copy link

Thanks for filing this issue! I just wanted to mention that it makes debugging the tests running with Node.js's Test Runner (node --loader=ts-node/esm --test ./src/tests.ts, from secutils-web-scraper) incredibly painful, as it is common to rely on errors while writing tests. One has to switch to Node.js 18 just to see the actual error.

@danrr
Copy link
Author

danrr commented Jul 2, 2023

@cspotcode serialisation has been fixed in the nightly version of Node. The problem is that internalBinding('errors').triggerUncaughtException( ignores Symbol(nodejs.util.inspect.custom) but with process.setUncaughtExceptionCaptureCallback(console.error) the type error info is shown again in the console.

@danrr
Copy link
Author

danrr commented Jul 7, 2023

@cspotcode Node 20.4 is out now with the serialisation fix in place. Could you please release a new version of ts-node so it can be used like this?

//logError.js
import { setUncaughtExceptionCaptureCallback } from "node:process"
setUncaughtExceptionCaptureCallback(console.log)

node --loader=ts-node/esm --import=./logError.js

Unless there's another way of getting this to work that doesn't require the fix in #2025.

@Maikpwwq
Copy link

I am using ts-node this way on Node 20 and works. Config ["type": "module"] in package.json, then update tsconfig.json to ts-node's ESM support and pass the loader flag to node in scripts, node --loader ts-node/esm ./index.ts. tsconfig.json:

{
  "compilerOptions": {
    "strict": true,
    "module": "ESNext", // ES2020
    "target": "ES2020",
    "moduleResolution": "Node",
    "lib": ["DOM", "DOM.Iterable", "ESNext"],
    "types": ["vite/client"],
    "jsx": "react-jsx",
    "skipLibCheck": true,
    "esModuleInterop": true,
    "allowSyntheticDefaultImports": true
  },
  "ts-node": {
    "experimentalSpecifierResolution": "node",
    "transpileOnly": true,
    "esm": true,
  }
}

@brianjenkins94
Copy link

brianjenkins94 commented Sep 15, 2023

I am seeing this with:

$ npx ts-node -vvv
ts-node v10.9.1 C:\project\node_modules\ts-node
node v20.6.0
compiler v5.2.2 C:\project\node_modules\typescript\lib\typescript.js

And on Mac:

% npx ts-node -vvv
ts-node v10.9.1 /project/node_modules/ts-node
node v20.6.1
compiler v5.2.2 /project/node_modules/typescript/lib/typescript.js

@Pyrolistical
Copy link

Pyrolistical commented Sep 18, 2023

Thanks @danrr.

This worked for me on node v20.6.1

//logError.js
import { setUncaughtExceptionCaptureCallback } from "node:process"
setUncaughtExceptionCaptureCallback(console.log)

Full command: node --test --no-warnings=ExperimentalWarning --loader ts-node/esm --import=./logError.js test-*.ts

@Pyrolistical
Copy link

Wait, that workaround only correctly logs out the error, but the test passes. How do we also fail the test?

@gsimko
Copy link

gsimko commented Oct 4, 2023

@Pyrolistical your solution doesn't work for me, I get the "Error [ERR_DOMAIN_CANNOT_SET_UNCAUGHT_EXCEPTION_CAPTURE]: The domain module is in use, which is mutually exclusive with calling process.setUncaughtExceptionCaptureCallback()"

Would it be possible to log the error from reportTSError instead of just throwing it? In my case it didn't even produce the diagnosticCodes or a stack trace or even a line where the error occurred. The ts-node process just hang up and printed

Object: null prototype] {
  [Symbol(nodejs.util.inspect.custom)]: [Function: [nodejs.util.inspect.custom]]
}

It took some non-trivial amount of effort to figure that the problem is related to this bug report.

@Pyrolistical
Copy link

@gsimko i gave up trying to use ts-node/esm and just tsc the test code and ran it normally with node --test

@alpharder
Copy link

I also get this when any TSC error happens under the hood:

Restarting 'app.ts server:start'
(node:42858) ExperimentalWarning: `--experimental-loader` may be removed in the future; instead use `register()`:
--import 'data:text/javascript,import { register } from "node:module"; import { pathToFileURL } from "node:url"; register("ts-node/esm", pathToFileURL("./"));'
(Use `node --trace-warnings ...` to show where the warning was created)

node:internal/process/esm_loader:40
      internalBinding('errors').triggerUncaughtException(
                                ^
[Object: null prototype] {
  [Symbol(nodejs.util.inspect.custom)]: [Function: [nodejs.util.inspect.custom]]
}

Node.js v21.1.0
Failed running 'app.ts server:start'

The command is

node --watch --loader ts-node/esm app.ts

ts-node v 10.9.1

@alpharder
Copy link

The world was not ready for ESM 7 years ago and isn't yet quite ready

@Kagami
Copy link

Kagami commented Nov 12, 2023

Took me about 20 minutes to come with this solution:

// package.json
"scripts": {
  "test": "TS_NODE_PROJECT=test/tsconfig.json node --test --import=./test/register.js test/**/*.spec.*"
},
// test/register.js
import { register } from "node:module";
import { pathToFileURL } from "node:url";
import { setUncaughtExceptionCaptureCallback } from "node:process";
setUncaughtExceptionCaptureCallback((err) => {
  console.error(err);
  process.exit(1);
});
register("ts-node/esm", pathToFileURL("./"));
// test/tsconfig.json
{
  "extends": "../tsconfig.json",
  "compilerOptions": {
    "allowImportingTsExtensions": true,
    "types": ["node"]
  }
}
// test/test.spec.ts
import test from "node:test";
import { deepStrictEqual } from "assert";

import { myfn } from "../lib/utils.ts";

test("test array", () => {
  deepStrictEqual(myfn([1, 2, 3]), [1, 2, 3]);
});

ESM + TypeScript + node is definitely quite tiresome combination 😅

@ArmorDarks
Copy link

@Kagami unfortunately it's not a solution for real applications, because you can easily land in this fatal error if nothing throws:

Error [ERR_DOMAIN_CALLBACK_NOT_AVAILABLE]: A callback was registered through process.setUncaughtExceptionCaptureCallback(), which is mutually exclusive with using the `domain` module

@Kagami
Copy link

Kagami commented Nov 13, 2023

I switched to https://github.com/swc-project/swc-node because imports without .ts extension doesn't work with ts-node.

// package.json
"scripts": {
  "test": "node --test --import=./test/register.js test/**/*.spec.*"
},
// test/register.js
import { register } from "node:module";
import { pathToFileURL } from "node:url";
register("@swc-node/register/esm", pathToFileURL("./"));

Much simpler and also faster. But I'm only using it to run tests...

@alpharder
Copy link

For anyone who just wants to see what the TSC errors are without any fanciness, here's how I patched ts-node's TSError impl:

class TSError extends Error {
    constructor(diagnosticText, diagnosticCodes, diagnostics = []) {
        super(`⨯ Unable to compile TypeScript:\n${diagnosticText}`);
        this.diagnosticCodes = diagnosticCodes;
        this.name = 'TSError';
        Object.defineProperty(this, 'diagnosticText', {
            configurable: true,
            writable: true,
            value: diagnosticText,
        });
        Object.defineProperty(this, 'diagnostics', {
            configurable: true,
            writable: true,
            value: diagnostics,
        });
    }
}

Now I can see the error:

node:internal/process/esm_loader:40
      internalBinding('errors').triggerUncaughtException(
                                ^

TSError: ⨯ Unable to compile TypeScript:
src/framework-extras/streaming-file-uploads/streaming-file-upload.model.ts(48,21): error TS2339: Property 'warning' does not exist on type 'LoggerInterface'.

You can use https://github.com/ds300/patch-package for replicating this workaround locally.

@mogzol
Copy link

mogzol commented Nov 22, 2023

Thanks @danrr.

This worked for me on node v20.6.1

//logError.js
import { setUncaughtExceptionCaptureCallback } from "node:process"
setUncaughtExceptionCaptureCallback(console.log)

Full command: node --test --no-warnings=ExperimentalWarning --loader ts-node/esm --import=./logError.js test-*.ts

Wait, that workaround only correctly logs out the error, but the test passes. How do we also fail the test?

Just doing process.exit(1) in the error handler seems to work for me:

// logError.js
setUncaughtExceptionCaptureCallback((error) => {
  console.error(error);
  process.exit(1);
});

@karlhorky
Copy link

karlhorky commented Nov 24, 2023

For anyone who just wants to see what the TSC errors are without any fanciness, here's how I patched ts-node's TSError impl:

This is great, thanks for the patch, @alpharder !

Unfortunately, I couldn't get your patch working as-is.

Do you think that you could also:

  1. Add the filename of the file you changed
  2. Change the syntax highlighting on your code above to diff and then add - and + lines to the lines that have been removed and added? A bit nicer to be able to check the patch...

@BenjaminGaymay
Copy link

For anyone who just wants to see what the TSC errors are without any fanciness, here's how I patched ts-node's TSError impl:

This is great, thanks for the patch, @alpharder !

Unfortunately, I couldn't get your patch working as-is.

Do you think that you could also:

  1. Add the filename of the file you changed
  2. Change the syntax highlighting on your code above to diff and then add - and + lines to the lines that have been removed and added? A bit nicer to be able to check the patch...

After a few research in found the fix in "dist/index.js" l97, he replaced the whole TSError class.
That's a small win but is there an other fix we can make to get the error cleaner ?

@karlhorky
Copy link

After a few research in found the fix in "dist/index.js" l97, he replaced the whole TSError class

I also did this research already, but applying the patch the way it is above did not work. So it would be good to get specific instructions.

@ArmorDarks
Copy link

ArmorDarks commented Dec 9, 2023

  1. Put this patch in patches/ts-node+10.9.1.patch (you might need to correct the version in file name)

    diff --git a/node_modules/ts-node/dist/index.js b/node_modules/ts-node/dist/index.js
    index c03afbf..0370067 100644
    --- a/node_modules/ts-node/dist/index.js
    +++ b/node_modules/ts-node/dist/index.js
    @@ -94,7 +94,7 @@ exports.DEFAULTS = {
    /**
     * TypeScript diagnostics error.
     */
    -class TSError extends make_error_1.BaseError {
    +class TSError extends Error {
        constructor(diagnosticText, diagnosticCodes, diagnostics = []) {
            super(`⨯ Unable to compile TypeScript:\n${diagnosticText}`);
            this.diagnosticCodes = diagnosticCodes;
    @@ -110,13 +110,8 @@ class TSError extends make_error_1.BaseError {
                value: diagnostics,
            });
        }
    -    /**
    -     * @internal
    -     */
    -    [exports.INSPECT_CUSTOM]() {
    -        return this.diagnosticText;
    -    }
    }
    +
    exports.TSError = TSError;
    const TS_NODE_SERVICE_BRAND = Symbol('TS_NODE_SERVICE_BRAND');
    function register(serviceOrOpts) {
  2. Add to package.json scripts this:

    "postinstall": "npx patch-package"
    

    It will auto-apply patch on npm install

  3. Do npm istall. That's it, patch applied.

@bennycode
Copy link

I switched to https://github.com/swc-project/swc-node because imports without .ts extension doesn't work with ts-node.

I just tried node --loader @swc-node/register/esm ./src/start.ts. Instead of providing incorrect stack traces, it runs without performing typechecking. Do you know how to enable it?

@Kagami
Copy link

Kagami commented Jan 3, 2024

it runs without performing typechecking. Do you know how to enable it?

I believe it's by design. For my use case (unit tests) it's perfect. Also you can check types just by building TypeScript project with noEmit: true if needed. Although for some use cases ts-node might be actually better, not sure here.

@gurpreetatwal
Copy link

@Kagami unfortunately it's not a solution for real applications, because you can easily land in this fatal error if nothing throws:

Error [ERR_DOMAIN_CALLBACK_NOT_AVAILABLE]: A callback was registered through process.setUncaughtExceptionCaptureCallback(), which is mutually exclusive with using the `domain` module

@ArmorDarks you can use process.on('uncaughtException') to get the same behavior if your code is using the domain module.

e.g.

import process from 'node:process';

process.on('uncaughtException', function (err) {
  console.error(err);
  process.exit(1);
});

@Nerdman4U
Copy link

I had this error with node 21.7.1 instead of real error:

node:internal/process/esm_loader:34
      internalBinding('errors').triggerUncaughtException(
                                ^
[Object: null prototype] {
  [Symbol(nodejs.util.inspect.custom)]: [Function: [nodejs.util.inspect.custom]]
}

and i started to get correct error messages after adding this to tsconfig.json (found solution from this thread):

"transpileOnly": true,

@kollein
Copy link

kollein commented Apr 10, 2024

I am using ts-node this way on Node 20 and works. Config ["type": "module"] in package.json, then update tsconfig.json to ts-node's ESM support and pass the loader flag to node in scripts, node --loader ts-node/esm ./index.ts. tsconfig.json:

{
  "compilerOptions": {
    "strict": true,
    "module": "ESNext", // ES2020
    "target": "ES2020",
    "moduleResolution": "Node",
    "lib": ["DOM", "DOM.Iterable", "ESNext"],
    "types": ["vite/client"],
    "jsx": "react-jsx",
    "skipLibCheck": true,
    "esModuleInterop": true,
    "allowSyntheticDefaultImports": true
  },
  "ts-node": {
    "experimentalSpecifierResolution": "node",
    "transpileOnly": true,
    "esm": true,
  }
}

Thanks, it works well with my custom loader too 👍

@ert78gb
Copy link

ert78gb commented May 3, 2024

Hi All,

I have similar experience.
My setup is a mono-repo. There is a root ts-config.json and every package contains it owns ts-config.json. The packages extends the root ts-config.json.
The package that contains the ts script contains the following extension.

"ts-node": {
    "esm": true,
    "transpileOnly": true
  }

When I ran ./packages/package1/my-script.ts with node 18.17.1 then worked perfectly.
After I upgraded to 20.9.0 I got the following error.

node:internal/process/esm_loader:40
      internalBinding('errors').triggerUncaughtException(
                                ^
[Object: null prototype] {
  [Symbol(nodejs.util.inspect.custom)]: [Function: [nodejs.util.inspect.custom]]
}

I use the #!/usr/bin/env -S node --loader ts-node/esm --no-warnings=ExperimentalWarning shebang in the scripts.

I tried to figure it out what is the problem because only the node version changed.
If I run the my-script.ts in the packages/package1 folder then the script runs successfully.
If I run the script in the root folder of the repo then fails.

After that I moved the ts-node specific setup to the root ts-config.json and run the ./packages/package1/my-script.ts from the repo root works again. PR

I don't know what changed but looks like the ts-config resolver starts from the cwd instead of the script directory.

@vladtreny
Copy link

transpileOnly is not a solution, then what is the point of using typescript.

As for now, stick with node 19. Nothing to do.
20, 21 do not provide any must-have features.

@ert78gb
Copy link

ert78gb commented May 3, 2024

transpileOnly is not the solution. I just wanted to share the ts-config.json resolving logic changed too.

@lortonx
Copy link

lortonx commented Aug 29, 2024

bump

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests