SyntaxError: Unexpected token import when installed globally and registered programmatically

[astorije]

We have been losing our hair on this for a couple days, in a rather sizable project, so I made a "minimal" reproduction repository at https://github.com/astorije/create-repro-ts-node-issue

All the details of how to reproduce, what the error is, what we tried, etc. should be explained in the README of this repo, but I'll summarize it here:

• When running this repo locally, either directly by calling the script, or by using yarn link (mimicking global install), everything works fine.

• When running this globally (yarn global add create-repro-ts-node-issue) or using yarn create (yarn create repro-ts-node-issue), it seems there is something wrong that causes Node to run running an ES module:

/Users/astorije/.config/yarn/global/node_modules/create-repro-ts-node-issue/src/index.ts:1
(function (exports, require, module, __filename, __dirname) { import inquirer from "inquirer";
                                                              ^^^^^^

SyntaxError: Unexpected token import
    at createScript (vm.js:80:10)
    at Object.runInThisContext (vm.js:139:10)
    at Module._compile (module.js:616:28)
    at Module._extensions..js (module.js:663:10)
    at Object.require.extensions.(anonymous function) [as .ts] (/Users/astorije/.config/yarn/global/node_modules/create-repro-ts-node-issue/node_modules/ts-node/src/index.ts:431:14)
    at Module.load (module.js:565:32)
    at tryModuleLoad (module.js:505:12)
    at Function.Module._load (module.js:497:3)
    at Module.require (module.js:596:17)
    at require (internal/module.js:11:18)

I'm happy to provide more info, I think I provided as much as I could in the repro repo, but let me know if I missed anything.

[astorije]

Maaaaaybe related to #617?

[astorije]

New attempt: instead of using programmatic access, I used a child process with:

#!/usr/bin/env node

const { resolve } = require('path');
const { spawn } = require('child_process');

const tsnode = spawn(
  resolve(__dirname, 'node_modules', '.bin', 'ts-node'),
  [
    '--project',
    resolve(__dirname, 'tsconfig.json'),
    resolve(__dirname, 'src', 'scripts'),
  ],
  { stdio: [process.stdin, process.stdout, process.stderr] },
);

tsnode.on('error', data => {
  console.log(`error: ${data}`);
});

Getting the exact same error (!):

(function (exports, require, module, __filename, __dirname) { import inquirer from 'inquirer';
                                                              ^^^^^^
SyntaxError: Unexpected token import

[blakeembrey]

I was confused for a bit but then realised your issue. It's because your module is inside node_modules. TS Node does not compile your node modules for you, they are expected to be in a node.js consumable format. I'll need to update the README to reflect this, but for now you can alter the behaviour by messing with --ignore (https://github.com/TypeStrong/ts-node#cli-and-programmatic-options). In general, it's not recommended to compile or use TypeScript directly from a node.js module which is why it's not compiled automatically for you.

[astorije]

Hey @blakeembrey, thanks for your reply!
Unfortunately, I'm not sure I'm following you.

As you can see here, the import inquirer from 'inquirer'; line is not in a Node module but in the source of that repo.
This file is loaded and run by ts-node programmatically here.

If this is an issue on our end, how would you go about fixing it? I'm trying to figure out how --ignore can help us, and unfortunately I'm not sure it would be of any help, I'm still stuck.
What is puzzling to me is that, as shown in the README, it works fine when called directly (using $ ./index for example) but not in a more global state (such as global install or running through yarn create).

[blakeembrey]

When you make it a global it’s in node modules, therefore it is not compiled. You can see the path in the README of your example that fails.

[astorije]

@blakeembrey, I'm playing with ignore but so far, no luck. I'm using a negative lookahead to mean "ignore all in /node_modules/ unless it's that package" using
ignore: '/node_modules/(?!create-repro-ts-node-issue/)', but it unignored outside of this package too:

/Users/astorije/.config/yarn/global/node_modules/ts-node/src/index.ts:261
    return new TSError(diagnosticText, diagnosticCodes)
           ^
TSError: ⨯ Unable to compile TypeScript:
../../../.config/yarn/global/node_modules/create-repro-ts-node-issue/src/index.ts(6,21): error TS7016: Could not find a declaration file for module 'request'. '/Users/astorije/.config/yarn/global/node_modules/request/index.js' implicitly has an 'any' type.
  Try `npm install @types/request` if it exists or add a new declaration (.d.ts) file containing `declare module 'request';`
../../../.config/yarn/global/node_modules/create-repro-ts-node-issue/src/index.ts(20,7): error TS7006: Parameter 'err' implicitly has an 'any' type.

(request is not part of create-repro-ts-node-issuebecause it's just a minimal test case for the initial issue, but this is was I'm seeing on our private package that uses otherrequest` and others)

Do you have any suggestion to get past this?

[astorije]

FYI the top of our index.ts file is:

import inquirer from 'inquirer';
import chalk from 'chalk';
import { readFileSync } from 'fs';
import { resolve } from 'path';
import { spawnSync } from 'child_process';
import request from 'request';

import { ... } from './ourStuff';

so somehow it chokes on request but not the rest.

[astorije]

Nevermind, I didn't have my @types/... dependency ducks in a row, but now it's choking on:

TypeError: Unable to require `.d.ts` file.
This is usually the result of a faulty configuration or import. Make sure there is a `.js`, `.json` or another executable extension and loader (attached before `ts-node`) available alongside `feature.ts`.

This is getting out of hand, I'm not sure why I'm getting this now.

[blakeembrey]

So we discovered recently that error occurs because TypeScript won’t actually compile anything in node modules. There’s no workaround right now. You can probably find the issue where it was discovered to learn more (I’m on mobile now).

[trusktr]

TypeScript won’t actually compile anything in node modules.

That isn't true, at least lately. I am able to ship private modules on NPM with source code entirely in TypeScript, then have my app compile them with TypeScript just fine. (using ts-node or ts-loader in Webpack).

I am however running into this problem, with files that are not in node_modules, but it happens when I try to run an .html file in Electron using <webview> instead of new BrowserWindow.

This HTML file look like

<!DOCTYPE html>
<html lang="en">
	<head>
		<meta charset='utf-8' />
		<title>Visualizer</title>
		<meta name="viewport" content="width=device-width, initial-scale=1" />
	</head>
 	<body>
		<div id="root"></div>
 		<script>
			require("./").start()
		</script>
	</body>
</html>

The ./ import registers ts-node then import a .tsx file:

require('ts-node').register({
	typeCheck: false,
	transpileOnly: true,
	files: true,
	ignore: [
		// ignore all node_modules except package-with-ts-source
		/node_modules(?!\/package-with-ts-source)/,
	],
})

require('./entry') // ./entry.tsx

which then throws the error

SyntaxError: Unexpected token import

on the first line of my entry.tsx. My tsconfig module option is set to commonjs.

This all works fine when I load the HTML file using Electron's new BrowserWindow API, but apparently not when I'm using a <webview> element (similar to an iframe, with Electron APIs) in an existing window's DOM.

I'm stuck trying to figure out why this is happening...

[john-kurkowski]

TypeError: Unable to require `.d.ts` file.
This is usually the result of a faulty configuration or import. Make sure there is a `.js`, `.json` or another executable extension and loader (attached before `ts-node`) available alongside `feature.ts`.

So we discovered recently that error occurs because TypeScript won’t actually compile anything in node modules. There’s no workaround right now. You can probably find the issue where it was discovered to learn more (I’m on mobile now).

I think the discovery Blake is referring to started with this comment in #624. The thread suggests the root cause is microsoft/TypeScript#11946.