Re: Re: Move to ESM

[mizdra]

ref: #84, #85

BREAKING CHANGES

• Drop node < 14.3.1

Tasks

Phase 1: Read guides

The conversion to ESM can cause a lot of BREAKING CHANGES. Also, support for various toolchains is partial. I will read this guide and check if I can really convert eslint-interactive to ESM and not have any problems.

• TypeScript 4.5 以降で ESM 対応はどうなるのか？ (in Japanese)
  • This is a most excellent guide!
  • This guide explains how to create a dual package that supports both CJS and ESM.
    • But eslint-interactive is a tool, not a library.
    • Therefore, it is no need to make eslint-interactive a dual package.
    • This will make the task easier.
• Node Modules at War: Why CommonJS and ES Modules Can’t Get Along | by Dan Fabulich | Code Red
• Pure ESM package
  • According to this guide, the minimum required version of Node.js is 14.13.1.

Phase 2: Check support status of toolchains

At this time, many toolchains have partial support for ESM. While many applications can be converted to ESM, some may not be able to be converted due to lack of support in the toolchain. Therefore, it is necessary to check the support status of various toolchains.

• TypeScript
  • Announcing TypeScript 4.5 Beta - TypeScript
  • To use TypeScript's ESM support, we need to use typescript@nightly.
  • The nightly is not a stable version and should be carefully considered for inclusion in dependencies.
  • In eslint-interactive, typescript is devDeps, so there was no problem.
• Jest
  • ECMAScript Modules · Jest
  • Meta: Native support for ES Modules jestjs/jest#9430
  • I found that jest.requireActual is not yet supported in ESM.
  • We may have to rewrite tests to not use jest.requireActual, or remove the tests.
• ts-jest
  • ESM Support | ts-jest
• ESLint
  • eslint-interactive uses ESLint's Node.js API to dynamically load custom rules.
  • Therefore, we need to check if ESLint supports the ESM custom rule.
  • I opened the issue and checked, and found that there is no problem.: Change Request: Support ESM plugins eslint/eslint#15453
• Node.js
  • Modules: ECMAScript modules | Node.js v17.3.0 Documentation
  • Modules: Packages | Node.js v17.3.0 Documentation

Phase 3: Setup

Add some basic configs.

1. Add "type": "module" in package.json: 1d0756d
2. Update typescript@nightly: cae48aa
3. Use nightly version tsserver in VSCode: 44a1054
4. Change "module": "node12" and "moduleResolution": "node12" of tsconfig.json: 03e2117

Phase 4: Fix type errors

Fix type errors by checking the results of yarn run lint:tsc.

1. Change import ... from './path/to/module'; to import ... from './path/to/module.js';: 888b431
   • It is because ESM does not allow the omission of the extension.
2. Change import ... from './path/to/dir'; to import ... from './path/to/dir/index.js';: bfd8558
   • It is because ESM does not allow the omission of index.js.
3. Update dependencies to supported versions of ESM: 6fb215f, 25a66f0
   • Depending on the situation, CJS packages may not be importable with the import statement (import ... from '...';).
   • If it is an ESM package, it can be easily imported with an import statement.
   • So, update dependencies to supported versions of ESM.
4. Import non-ESM (CJS) packages with the import ... = require('...') syntax: ab398aa, be4d269
   • Import the rest of the packages using require

Phase 5: Fix lint errors

Fix lint errors by checking the results of yarn run lint:eslint.

1. Rename .eslintrc.js to .eslintrc.cjs: b8cfc2e
   • ESM config is not yet supported.
   • ref: https://eslint.org/docs/user-guide/configuring/configuration-files#configuration-file-formats
2. Disable import/no-unresolved rule: 9e91c7c
   • This rule does not yet support to resolve pkg.exports.
   • Therefore, it warns the import statement to import Pure ESM packages.
   • So I will disable it because it's incompatible.
   • ref: [import/extensions] - importing TS code with .js extension import-js/eslint-plugin-import#2170
3. Include .cjs, .mjs, .cts and .mts as lint targets: 36e4c54, d97a2b4
   • In my case, I had to modify the shared config too.
   • ref: Support *.cts and *.mts eslint-config-mizdra#108

Phase 6: Fix runtime errors

Fix runtime errors by checking the results of yarn run dev.

1. Add an extension (.js) to the path in the bin field of package.json: a932cb1
   • It is because without the extension, the runtime error occurs
     • log: TypeError: Unknown file extension "" for /Users/mizdra/src/github.com/mizdra/eslint-interactive/bin/eslint-interactive
   • According to Node.js documentation, adding --input-type=module to shebang looks like it should work, but it doesn't. We have to add an extension.
   • ref: node should execute files without extension as ESM if "type": "module", is specified nodejs/node#34049
2. Import the CJS package with ESM-interop: 69595d7
   • Node.js output a hint at runtime, so follow it to fix the problem.

   • $ yarn run dev
     file:///Users/mizdra/src/github.com/mizdra/eslint-interactive/dist/cli/prompt.js:2
     import { prompt } from 'enquirer';
              ^^^^^^
     SyntaxError: Named export 'prompt' not found. The requested module 'enquirer' is a CommonJS module, which may not support all module.exports as named exports.
     CommonJS modules can always be imported via the default export, for example using:
     
     import pkg from 'enquirer';
     const { prompt } = pkg;
     
     at ModuleJob._instantiate (node:internal/modules/esm/module_job:127:21)

3. Convert __dirname to dirname(fileURLToPath(import.meta.url)): 6db3b65
   • __dirname is not supported in ESM
4. Fix mismatch between type definitions and runtime types: c5dc7b0
   • I don't know the cause, but sometimes the type definition and the runtime do not match.
   • I used a cast to force a fix.
5. Remove module.exports: 0a980ad
   • eslint-interactive used module.exports in some packages to support both CJS/ESM.
   • This will cause a runtime error since module.exports is not supported in ESM.
   • Since the whole eslint-interactive will be Pure ESM, this hack will no longer be necessary.
   • So, I will remove it.
6. Make the entire fixtures/ CJS: bca4371
   • For testing purposes, fixtures/ must be treated as CJS.
   • So I put fixtures/package.json with {"type": "commonjs"}.
   • This treats fixtures/ as CJS.

Phase 7: Fix tests

Fix tests by checking the results of yarn run test.

1. Convert jest.config.js to jest.config.mjs: cdc1672
2. Update jest toolchain: 1afa2db
3. Setup jest config to support ESM: c550aec
   • ref: https://jestjs.io/docs/ecmascript-modules
   • ref: https://kulshekhar.github.io/ts-jest/docs/guides/esm-support/
4. Use custom resolver: f5f57f0, 9a33888
   • Jest is still unable to resolve pkg.exports.
   • So we will use a custom resolver.
   • ref: Support package exports in jest-resolve jestjs/jest#9771 (comment)
5. Insert import { jest } from '@jest/globals';: deea149, 7683a9d
   • In ESM, jest objects are not automatically assigned.
   • So if globalThis.jest is used, it causes ReferenceError: jest is not defined.
   • Need to insert import { jest } from '@jest/globals';.
   • ref: Meta: Native support for ES Modules jestjs/jest#9430 (comment)
   • ref: https://archive.jestjs.io/docs/ja/api
6. Replace jest.requireActual with another solutions: 009081e, 45b5015
   • jest.requireActual is not yet supported in ESM.
7. Include .cjs, .mjs, .cts and .mts as coverage targets: 2b4bd5e

Phase 8: Refactor codes

Refactoring bad code.

1. Use ESM version of comlink: 19016e1
   • comlink provides an ESM module (ex. comlink/dist/esm/node-adapter.mjs)
   • Of course, comlink also provide type definitions (ex. comlink/dist/esm/node-adapter.d.ts)
   • However, the corresponding type definition for xxx.mjs is xxx.d.mts in "module": "node12" mode
   • Therefore, importing the ESM module causes compilation errors
   • So, use paths option of tsconfig.json to force xxx.mjs to be mapped to xxx.d.ts.

Phase 9: Confirm operation

1. npm pack
2. npm install -D xxx.zip
3. Confirm operation of the package
   • Confirm on each OS and Node.js version supported by eslint-interactive

TODO

• Confirm operation
• Merge this PR
• Release new version
• Report problems of toolchains that I found in this work
  • It is a good idea to send feedback for the future of the ecosystem 👍
  • Compatibility issues with type definitions of comlink

[mizdra]

It works! It's perfect...

[mizdra]

Everything is ready to go. We're moving forward!