Strict null checking VS Code

[mjbvz]

Tracks engineering work to enable strictNullChecks in the VS Code code base.

Problem

A steady number of issues are reported each month that are caused by null reference exceptions. Most of these are just simple programming errors, such as forgetting to initialize a value or not realizing that a function may return undefined.

Proposal

To address this class of errors, I propose that we move to enable strictNullChecks in the main VS Code codebase. We have already successfully enabled strictNullChecks for our build scripts and for our built-in extensions, and I believe that these changes have been highly beneficial in catching common errors and enforcing better programming practices.

Today, building VS Code with strictNullChecks produces thousands of errors. Addressing these in a single pass is both impractical and would risk major regressions. I therefore propose we turn on strictNullChecks in our code base incrementally, opting in individual files until eventually strictNullChecks is enabled for the entire codebase.

Implementation

To accomplish this, a new tsconfig called tsconfig.strictNullChecks.json will be introduced. This tsconfig will only be used for error checking. It will use include and files to compile only files that should be strict null checked.

We will build tsconfig.strictNullChecks.json as part of our normal build process. A strictNullChecks failure in tsconfig.strictNullChecks.json should cause the entire VS Code build to fail. This should prevent us from regressing strictNullChecks changes that have already been added

How to help

This will be an long term engineer effort. The good news is that I believe we can do this incrementally and that each incremental change will improve the codebase in its own right.

Here's what's needed to make this work:

For regular development
Make sure your changes do not break the tsconfig.strictNullChecks.json build. You can check this manually by running: yarn strict-null-check

This script s part of our build pipelines. It is not run as part of yarn compile or yarn watch for performance reasons (doing so would essentially have us building the VS Code codebase twice)

Add a new file to be strict null checked

1. Add the file in the files section of src/tsconfig.strictNullChecks.json

   "files": [
   	"./vs/base/common/iterator.ts"
   ]

   Keep in mind that this will also strict null check all files that the added file imports. Therefore it's best to start with files that either have very few imports or that only import files that are strict null checked.

2. Run yarn strict-null-check or yarn strict-null-check -- --watch

3. Fix all null check related errors. In general, this should involve:

   • Annotate nullable types and fix type errors
   • Use the ! not null operator to tell TypeScript that a value cannot be null. Only use this if you are certain that a value absolutely cannot be null or undefined (i.e. you already checked this in code)
   • Add conditionals to handle null. Only change the actual logic if you believe that strict null checks has revealed a real programming error that must be handled.

4. Make sure you have not broken the normal VS Code build.

5. Check in your changes

[roblourens]

Will I see errors inline from this? Basically can VS Code check two overlapping tsconfig projects at the same time?

[mjbvz]

No, unfortunately that's not possible using multiple, overlapping tsconfigs . I'll bring this up to the TS team to see if there is a better approach

[Tyriar]

@mjbvz I add xterm.js' strict null check project as a pretest script, I assume most people run this before committing to that should help keep the build green.

[mjbvz]

@Tyriar Yes that's a good idea. At some point running the check may become quite slow but we can worry about that when we hit it

[mjbvz]

Status:

• Checked in src/tsconfig.strictNullChecks.json and opted in a few files
• Added a yarn strict-null-check script to build this project.
• Run this script in the precommit checks
• Run this script in our Azure pipelines builds

[mjbvz]

Note: I'll try to update this list as the set of files changes

---

Here's a list of files that only import files that are already strict null checked. These are a good starting point if you are looking to help out with conversion:

updated: March 14, 2019

• "./vs/workbench/api/electron-browser/mainThreadTask.ts" — Depended on by 1 files
• "./vs/workbench/api/node/extHostComments.ts" — Depended on by 1 files
• "./vs/workbench/api/node/extHostTerminalService.ts" — Depended on by 3 files
• "./vs/workbench/api/node/extHostTreeViews.ts" — Depended on by 1 files
• "./vs/workbench/contrib/preferences/browser/keybindingWidgets.ts" — Depended on by 2 files
• "./vs/workbench/contrib/preferences/browser/preferencesRenderers.ts" — Depended on by 1 files
• "./vs/workbench/services/configurationResolver/common/variableResolver.ts" — Depended on by 2 files
• "./vs/workbench/services/extensions/electron-browser/extensionHostProcessManager.ts" — Depended on by 1 files
• "./vs/workbench/services/search/node/searchService.ts" — Depended on by 3 files
• "./vs/workbench/services/search/test/node/rawSearchService.test.ts" — Depended on by 0 files

File list is generated using this script

[roblourens]

If it ends up taking more than a few seconds to run, I would move it out of the precommit hook to scripts/test or something

[roblourens]

What if we have a simple extension that runs the script and shows errors inline?

Or like a clone of the TS extension that basically runs a second tsserver instance for the tsconfig

[mjbvz]

Yes I just used a script to opt a bunch of files in to strict null checks ( a2738f7 ) so I guess it's already time to worry :) I'll move the check to run in the test scripts instead

Extension is an interesting idea. I think we could also use a task and a problem matcher

[joaomoreno]

Love this. Yeah, I was coming in saying I am worried about the pre-commit check. Tests is fine, we are always watching the build.

[alexdima]

👎 on running the compiler every time I invoke scripts/test.sh. I run the tests very often, it is part of the way I develop, and the extra slowness brought in is noticeable.

[bpasero]

Done with vs/workbench/(common|browser|electron-browser)/**

[mjbvz]

Just 325 strict null errors left in just a handful of files. Owners of these files should have an issue open. Please take a look so that we can enable strict null in our normal build

[ghost]

@mjbvz It could be very interesting if you publicized some behind-the-scenes data for this effort, such as roughly how many bugs was found in the codebase as a result of this and how much effort it took ?

[mjbvz]

Zero. I can't think of a single instance where we knew of a bug and strict null checking fixed it.

But that's not why this work is valuable. By better specifying our code's behavior with strict null checks, we (mostly) eliminated an entire class of future potential bugs, both in existing our code and in our future written code, enabling us to work more safely and quickly going forward.

So: 0 known bugs fixed. ∞ potential bugs prevented

[rudi-c]

@mjbvz Amazing effort! Do you think you might write a blog post about the process at some point? I'm sure lots of people would be interested.

[mjbvz]

Blog post