Make JSDoc skipping public, plus more configurable #55739

jakebailey · 2023-09-13T23:03:36Z

This is a follow-up to #52921.

This changes the way we do the new JSDoc skipping, instead replacing it with a publicly-available option which lets you control how much info you want. Specifically, you can specify:

JSDocParsingMode.ParseAll, the default, which is to keep parsing JSDoc and including it in the tree. This is the default used by tsserver, services, etc, where we definitely need all of that info.
JSDocParsingMode.ParseForTypeErrors, which only parses JSDoc tags which have a semantic meaning to the checker which impact error calculation, that is, any JSDoc in a non-TS/TSX file, or any JSDoc containing @see or @link. This is the new default when calling from tsc, and would be a good default for anyone orchestrating TS builds, e.g. heft, nx.
JSDocParsingMode.ParseForTypeInfo, if you only query type info, but never care about its errors. This parses JSDoc in JS files, but doesn't in TS files at all (as they are never used for type info there). This is good for something like ts-eslint with type info (which does not surface errors).
JSDocParsingMode.ParseNone, which always skips JSDoc, no matter what. This is good for those who only want the parser, e.g. formatters like prettier, ts-eslint without type information, transpileModule, maybe others. Set this at your own risk.

To use this when calling createSourceFile, set jsDocParsingMode on the create options:

const sourceFile = ts.createSourceFile(
    filename,
    content,
    { languageVersion: ts.ScriptTarget.ESNext, jsDocParsingMode: ts.JSDocParsingMode.ParseNone },
);

To use this when constructing a host, set jsDocParsingMode:

const host = ts.createCompilerHost(options);
host.jsDocParsingMode = ts.JSDocParsingMode.ParseForTypeInfo;
// or
const host = ts.createIncrementalCompilerHost(options);
host.jsDocParsingMode = ts.JSDocParsingMode.ParseForTypeErrors;
// or
const host = ts.createWatchCompilerHost(...);
host.jsDocParsingMode = ts.JSDocParsingMode.ParseForTypeErrors;

FYI @fisker @bradzacher @JoshuaKGoldberg @JamesHenry @dmichon-msft

typescript-bot · 2023-09-13T23:03:49Z

Looks like you're introducing a change to the public API surface area. If this includes breaking changes, please document them on our wiki's API Breaking Changes page.

Also, please make sure @DanielRosenwasser and @RyanCavanaugh are aware of the changes, just as a heads up.

jakebailey · 2023-09-13T23:24:25Z

Bleh, there's that dprint ternary annoyance again that Sheetal noted (https://github.com/microsoft/TypeScript/actions/runs/6178742090/job/16772520512?pr=55739), maybe that's dprint/dprint-plugin-typescript#450 ?

bradzacher · 2023-09-13T23:53:40Z

Copying from the other PR

I think my main concern is that people will want to read out the .jsDoc properties by asking typescript-eslint for the original AST nodes (e.g. #55139), and then find that the info was never populated, or similar. I also wanted to try and make .jsDoc lazy and only create it when accessed (since we have the text anyway), but since JSDoc is bound by the binder, you can't really do that.

I think that this is going to be the rare case, if it even exists across the ecosystem.
If this is a configurable thing then we can easily set it up to be configurable on our side too - eg a user can pass parserOptions.tsJSdoc or something in their eslint config if a plugin needs it (and plugins can add it to their recommended config).

The vast majority of the ecosystem would just get faster-by-default lint runs which would be a big win for everyone!

jakebailey · 2023-09-14T00:36:45Z

On that front, I probably need to figure out a way to plumb this through ProjectService too.

… here

jakebailey · 2023-09-15T18:22:33Z

Marking as ready for review, though I need to double check that I have plumbed this up through createProgram and so on.

jakebailey · 2023-09-15T18:34:54Z

It turns out that I missed a lot of places that should plumb this. This will be tricky. ☹️

src/services/types.ts

src/executeCommandLine/executeCommandLine.ts

tests/baselines/reference/api/typescript.d.ts

src/compiler/tsbuildPublic.ts

sheetalkamat · 2023-09-15T21:34:07Z

src/compiler/program.ts

@@ -393,16 +394,16 @@ export function computeCommonSourceDirectoryOfFilenames(fileNames: readonly stri
    return getPathFromPathComponents(commonPathComponents);
 }

-export function createCompilerHost(options: CompilerOptions, setParentNodes?: boolean): CompilerHost {
-    return createCompilerHostWorker(options, setParentNodes);
+export function createCompilerHost(options: CompilerOptions, setParentNodes?: boolean, jsDocParsingMode?: JSDocParsingMode): CompilerHost {


I dont think we should be threading in this to defaults. User of this API can always call thius method and override getSourceFile on their own .. there is nothing special here.

I think that not providing this parameter will make it very inconvenient; this function and createCompilerHostWorker are what call createGetSourceFile, which is internal and non-trivial, and then wrap it again in another closure. Setting the JSDoc parsing mode is a lot like setting a default setParentNodes which this function already does.

See: https://github.com/typescript-eslint/typescript-eslint/blob/7826910d8c6ba4d2d40b8d55fcf7fd39fe290d88/packages/typescript-estree/src/create-program/useProvidedPrograms.ts#L84

Generally speaking I think this means that anywhere we can expect to find setParentNodes, we probably need to be expecting jsDocParsingMode.

Hmm.. Generally i like to go with conservative approach and avoid having to add additional surface area that we have to keep maintaining. This is ok i guess though , user can just create source file from file text easily to override but ok.

If the option is fed in via the CreateSourceFileOptions, rather than a top-level field, callers will already have the opportunity to request it on a per-file basis when making the file, rather than needing to set it on the host as a whole.

src/compiler/watch.ts

src/compiler/watchPublic.ts

src/services/services.ts

src/services/types.ts

jakebailey · 2023-09-20T23:51:20Z

FYI @nonara; in executeCommandLine.ts, there's a new global defaultJSDocParsingMode; in tsc here that's set to ParseForTypeErrors, but it's likely that someone's plugin will actually need this to be reset to ParseAll in order to get all of the JSDoc nodes they expect. e.g. I suspect typia reads JSDoc, somehow. Probably others. Just something to watch out for!

jakebailey · 2023-09-20T23:58:19Z

src/compiler/types.ts

+     * This will always parse JSDoc in non-TS files, but only parse JSDoc comments
+     * containing `@see` and `@link` in TS files.
+     */
+    ParseForTypeErrors,


Last minute bikeshedding; ParseForErrors? Technically, this covers grammar checks.

But, not actually sure we do grammar checks which relate to JSDoc. I assume not.

Not in TS files, no. Just in JS files.

nonara · 2023-09-21T01:44:16Z

Thanks for the heads up!

jakebailey · 2023-09-21T22:31:03Z

Alright, seems like everything is alright; I'm going to merge so we can get this in for 5.3. Thank you everyone for the feedback, hopefully it works out (and tell me if it doesn't).

Test new property `CompilerHost.jsDocParsingMode` on playground following below guide: microsoft/TypeScript#55739

Since TypeScript v5.3 update, TypeScript compiler no more analyzes `JsDocComment` as default to boost up compilation speed by not parsing the JS Doc Comments. Therefore, to parepare the future TypeScript v5.3 release, I've updated `@nestia/sdk` to configure the `jsDocParsingMode` to parse every comments manually. microsoft/TypeScript#55739

samchon · 2023-11-08T12:47:03Z

Thanks for detailed documentation.

As TypeScript starts skipping parsing `JSDocComment`s since TypeScript v5.3 update but it is possible to revive the `JSDocComment`s' parsing, by hacking the `jsDocParsingMode` value of `tsc.js`, I've decided to print a warning message from `typia` when the `jsDocParsingMode` value is not hacked. Therefore, it is possible to running the `typia` as before even when the `jsDocParsingMode` is not hacked, and user can ignore the warning message if he (or she) does not use any comment tags or not generating JSON schema with description comments. - Related issue: microsoft/TypeScript#55739

Since TypeScript v5.3 update, `tsc` no more parses `JSDocComment`s. Therefore, Therefore, `typia` and `@nestia/core` also cannot utilize those `JSDocComment` related features too, especially "Comment Tags" and "JSON schema generator". However, in relation to this, the upgrade of `ts-patch` continues to be delayed, and I still don't know how long the delay would be continue. Furthermore, there're some `typia`/`nestia` users urging to resolve the `peerDependencies` of `typia`/`nestia` that blocking the TypeScript v5.3 update. Therefore, before the `ts-patch` being prepared, I've decoded to provide `typia`'s own solution for a while. It is the new CLI command `npx typia patch`, and `nestia` also adapts it (`npx nestia setup` command performs it). Also, if the `defaultJSDocParsingMode` value not being patched, `typia` will generate an warning message of TypeScript compiler API. For reference, as it is an warning message, it does not interrupt the TypeScript compilation like the compilation error case. If there're some `typia`/`nestia` users never using "Comment Tags" or "JSON schema generator" at all, they don't need to run the CLI command. This is not mandatory command, but just optional command. Of course, when `ts-patch` being updated, this CLI command would be disabled immediately, if the installed `ts-patch` version is the latest one. Related issues: - samchon/typia#883 - microsoft/TypeScript#55739 - nonara/ts-patch#134

Related issues - samchon/nestia#696 - samchon/typia#883 - microsoft/TypeScript#55739 - nonara/ts-patch#134

Make JSDoc skipping more configurable

442b467

typescript-bot assigned jakebailey Sep 13, 2023

typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Sep 13, 2023

This was referenced Sep 13, 2023

Skip parsing JSDoc when not needed #52921

Merged

Investigate making JSDoc parsing lazier and more optimal #52959

Closed

I guess it's a big binary expression now

9119d2f

jakebailey added 2 commits September 13, 2023 20:45

An attempt at ProjectService plumbing, not sure what should be public…

ba9dcab

… here

Merge branch 'main' into skip-jsdoc-2

8a4837c

jakebailey requested review from sandersn, DanielRosenwasser and andrewbranch September 15, 2023 18:19

jakebailey marked this pull request as ready for review September 15, 2023 18:22

jakebailey marked this pull request as draft September 15, 2023 18:34

jakebailey commented Sep 15, 2023

View reviewed changes

src/services/types.ts Outdated Show resolved Hide resolved

Preemptive bikeshedding

66a0b09

DanielRosenwasser mentioned this pull request Sep 15, 2023

Design Meeting Notes, 9/15/2023 #55754

Closed

jakebailey added 3 commits September 15, 2023 12:29

Temporarily make stuff required to try and get plumbing correct

46bbf0e

Revert by moving parameters back to optional

88bfd9e

revert a formatting change

2abd21b

jakebailey commented Sep 15, 2023

View reviewed changes

src/executeCommandLine/executeCommandLine.ts Show resolved Hide resolved

Remove TODOs, tests pass

b691830

jakebailey commented Sep 15, 2023

View reviewed changes

tests/baselines/reference/api/typescript.d.ts Outdated Show resolved Hide resolved

sheetalkamat requested changes Sep 15, 2023

View reviewed changes

sheetalkamat approved these changes Sep 20, 2023

View reviewed changes

Rename global tsc JSDocParsingMode for better tsc.js emit

9392280

jakebailey commented Sep 20, 2023

View reviewed changes

jakebailey merged commit a3c5d5d into microsoft:main Sep 21, 2023
19 checks passed

jakebailey deleted the skip-jsdoc-2 branch September 21, 2023 22:31

snovader pushed a commit to mestro-se/TypeScript that referenced this pull request Sep 23, 2023

Make JSDoc skipping public, plus more configurable (microsoft#55739)

99afd53

dummdidumm mentioned this pull request Sep 26, 2023

Performance: skip jsdoc parsing in various places sveltejs/language-tools#2170

Open

DanielRosenwasser mentioned this pull request Sep 27, 2023

Design Meeting Notes, 9/20/2023 #55897

Closed

samchon added a commit to samchon/typia that referenced this pull request Nov 8, 2023

Test CompilerHost.jsDocParsingMode option on playground.

72abdfb

Test new property `CompilerHost.jsDocParsingMode` on playground following below guide: microsoft/TypeScript#55739

samchon mentioned this pull request Nov 8, 2023

Enable defaultJSDocParsingMode of TypeScript v5.3 nonara/ts-patch#134

Closed

samchon mentioned this pull request Nov 8, 2023

ts.CompilerHost.jsDocParsingMode samchon/nestia#683

Merged

uid11 mentioned this pull request Nov 21, 2023

[5.3] Type error in typescript.d.ts with exactOptionalPropertyTypes = true #56478

Closed

This was referenced Nov 22, 2023

Warning message when jsDocParsingMode is not hacked. samchon/typia#879

Merged

New cli command npx typia patch for hacking defaultJSDocParsingMode value samchon/typia#883

Merged

samchon mentioned this pull request Nov 26, 2023

Adapt new cli command npx typia patch. samchon/nestia#696

Merged

samchon added a commit to samchon/payments that referenced this pull request Nov 28, 2023

Upgrade dependencies for TS 5.3

cd93dd8

Related issues - samchon/nestia#696 - samchon/typia#883 - microsoft/TypeScript#55739 - nonara/ts-patch#134

samchon mentioned this pull request Nov 28, 2023

Upgrade dependencies for TS 5.3 samchon/payments#182

Merged

mpodwysocki mentioned this pull request Nov 30, 2023

Make JSDoc Parsing Configurable using the transpileModule method #56625

Closed

6 tasks

dominikg mentioned this pull request Dec 2, 2023

Support TypeScript 5.3 Rich-Harris/dts-buddy#66

Closed

captbaritone mentioned this pull request Jul 6, 2024

VSCode plugin not working with TS 5.5.2 captbaritone/grats#142

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make JSDoc skipping public, plus more configurable #55739

Make JSDoc skipping public, plus more configurable #55739

jakebailey commented Sep 13, 2023 •

edited

Loading

typescript-bot commented Sep 13, 2023

jakebailey commented Sep 13, 2023

bradzacher commented Sep 13, 2023

jakebailey commented Sep 14, 2023

jakebailey commented Sep 15, 2023

jakebailey commented Sep 15, 2023

sheetalkamat Sep 15, 2023

jakebailey Sep 15, 2023

jakebailey Sep 15, 2023

jakebailey Sep 15, 2023

sheetalkamat Sep 18, 2023

weswigham Sep 18, 2023

jakebailey commented Sep 20, 2023

jakebailey Sep 20, 2023

jakebailey Sep 20, 2023 •

edited

Loading

sandersn Sep 21, 2023

nonara commented Sep 21, 2023

jakebailey commented Sep 21, 2023

samchon commented Nov 8, 2023

Make JSDoc skipping public, plus more configurable #55739

Make JSDoc skipping public, plus more configurable #55739

Conversation

jakebailey commented Sep 13, 2023 • edited Loading

typescript-bot commented Sep 13, 2023

jakebailey commented Sep 13, 2023

bradzacher commented Sep 13, 2023

jakebailey commented Sep 14, 2023

jakebailey commented Sep 15, 2023

jakebailey commented Sep 15, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jakebailey commented Sep 20, 2023

Choose a reason for hiding this comment

jakebailey Sep 20, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nonara commented Sep 21, 2023

jakebailey commented Sep 21, 2023

samchon commented Nov 8, 2023

jakebailey commented Sep 13, 2023 •

edited

Loading

jakebailey Sep 20, 2023 •

edited

Loading