Convert the codebase to modules #51387

This step makes further commits look clearer by unindenting all of the top level namespaces preemptively.

This step makes all implicit namespace accesses explicit, e.g. "Node" turns into "ts.Node".

This step converts each file into an exported module by hoisting the namespace bodies into the global scope and transferring internal markers down onto declarations as needed. The namespaces are reconstructed as "barrel"-style modules, which are identical to the old namespace objects in structure. These reconstructed namespaces are then imported in the newly module-ified files, making existing expressions like "ts." valid.

This step converts as many explicit accesses as possible in favor of direct imports from the modules in which things were declared. This restores the code (as much as possible) back to how it looked originally before the explicitify step, e.g. instead of "ts.Node" and "ts.Symbol", we have just "Node" and "Symbol".

While GitHub automatically uses this file if present, GitLens in VS Code does not. Add the right option to our example settings.json for those who use the extension. Unfortunately, you can't leave this enabled if you want to look at the repo _without_ the file; git blame just crashes when the file isn't present. I'm not sure that there's a workaround for that.

Now that we are modules, there's no reason to ban multiple namespaces per file; each file is its own scope and declaring a namespace won't merge it into any other files.

See the next commit for a more fleshed-out description.

If these are regular comments, then they won't appear in our d.ts files. But, now we are relying on an external d.ts bundler to produce our final merged, so they need to be present in the "input" d.ts files, meaning they have to be JSDoc comments. These comments only work today because all of our builds load their TS files from scratch, so they see the actual source files and their non-JSDoc comments. The comments also need to be attached to a declaration, not floating, otherwise they won't be used by api-extractor, so move them if needed.

Although the existing require calls would work fine, it's more consistent to make them actual imports and let them be converted at compile/bundle time. It also lets us emit ESM if needed. There are still conditional requires present for things like the node System, but that's a more difficult problem to solve.

This is the same as TypeScript PR 51026.

This isn't totally complete, but enough to get everything building and tests running manually. The next commit will make the build work fully.

This configures the existing build tasks to use esbuild by default. If using the plain files is desired, passing `--bundle=false` will build using plain files and still produce a runnable system. This is only a basic build; a more efficient build is provided later when gulp is replaced by hereby.

This adds a "small" d.ts bundler script. This script is very basic, using Node printing to produce its output. Generally speaking, this is inadvisable as it completely disregards name shadowing, globals, etc. However, in our case, we don't care about the globals, and we can opt to restructure our codebase in order to avoid conflict, which we largely had to do anyway when we were namespaces and everything was in scope.

When bundled, it's very likely that the function "require" will actually exist at runtime, so we can't use this to determine if we are running in Node. Consolidate that logic and use other things to check instead. This is still not perfectly accurate, but I don't want to change this _too_ much, lest someone downstream depend on our inconsistent logic. There are yet other places this commit does not fix; search for "typeof process" for more examples.

In namespaces, we'd have access to a variable named "ts" when debugging, because that was just the ts namespace in the current file. But when modules, that variable doesn't exist, either because we are pure modules (so no ts variable), or named something else (when bundled, for export). Tack a "ts" object onto the globals, which should make this easier. Of course, someone may eventually declare a variable in the top level scope named "ts" and break this, but we can cross that bridge when we come to it.

The next commit will switch the build system, but git doesn't detect that the two config files are related.

This eliminates a significant number of dependencies, eliminating all npm audit issues, speeding up `npm ci` by 20%, and overall making the build faster (faster startup, direct code is faster than streams, etc) and clearer to understand. I'm finding it much easier to make build changes for the module transform with this; I can more clearly indicate task dependencies and prevent running tasks that don't need to be run. Given we're changing our build process entirely (new deps, new steps), it seems like this is a good time to change things up.

The old gulpfile produced lib.d.ts files with mixed newlines, thanks to the files containing CRLF, but the gulp streams adding in LF. Now they're all LF, which matches every other file in built / lib, but our baselines are sensitive to this. Rerun the tests and accept them.

Profiling the build roughly half of the time spent loading the build is spent importing typescript.js, for this one function. Since this stack is already adding required devDependencies, switch readJson to use jsonc-parser (published by the VS Code team), rather than importing the entire LKG typescript.js library.

This file is pretty much the same as it was when it was committed in 2017; these days, we can write clearer code with async/await and new FS APIs. Additionally, we can improve the performance of this script by using a newer/faster/maintained XML library. This will enable us to run the script unconditionally in a later commit.

needsUpdate may be wrong when the branch changes; these ones are now so fast thanks to being pure JS that we can just always run their contents and be sure that the outputs are right.

If our build scripts target Node 10+, we can use the builtin mkdir function in recursive mode instead.

Unlike our other exectuables, tsserver is used in web, which means that there actually would have been a "ts" variable declared for those users (e.g. in a web worker). It looks like VS Code needs this variable to be declared, so change this bundle to look the same as other libraries. Note that in Node, the IIFE will never actually return, so this can have no effect. On web, it will return in order to yield control back to the event loop (as on web, postMessage is used to communicate).

I should report this upstream, if I can manage to minimize this.

…ions We pass the entire "ts" object into plugins. However, we need to make sure that that object contains the debug compat helpers. In the old codebase, the deprecated compat code would tack things onto the actual ts object, after the server code was executed, and later that same object would be given to plugins. But in modules, each TS project only sees the view of the "ts" namespace that their references implied, not the ts object as some sort of singleton. To ensure that plugins get the debug compat code, we have to bring that into each project's view of the ts namespace, and not add it on later in the executable projects.

…o be tree shaken This debug code was added quite a while ago, constructed such that we wouldn't have to ship this code to our users. However, this is the sole place in the compiler project where the ts namespace "escapes" the bundle. By moving this debug code into the compiler itself, we no longer have any references to the ts namespace itself for our bundles that don't export anything (tsc, typingsInstaller). This lets bundlers tree shake the compiler, reducing the size of our output by _5.7 MB_ (a ridiculous improvement for _adding_ code).

Now that we target ES2018, we don't need to declare this ourselves.

…cesFactory Looking at github, sourcegraph, and internally, nobody appears to use this factory at all. It's still used within our testing framework, so I won't remove it, but we shouldn't be polyfilling globalThis and sticking things onto it these days.

See issue 10937 on github.com/github/codeql.

The troubleshooting wizard for the debugger actually says that this option is no longer recommended; removing it enables us to use source maps if we happen to be debugging while in --bundle=false mode.

It turns out that the import expression won't actally be rewritten by esbuild, so we can just write it directly. While this won't help CJS emit, that already didn't work anyway, and it's likely that this code is going to be moved outside of the codebase into VS Code or a shared package elsewhere anyway.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Convert the codebase to modules #51387

Convert the codebase to modules #51387

Commits on Nov 7, 2022