Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Control flow for element access expressions #31478

Merged
merged 9 commits into from
Jul 16, 2019

Conversation

sandersn
Copy link
Member

@sandersn sandersn commented May 20, 2019

Fixes #28081

I did some additional performance investigation, using the same synthetic stress test as above with 3 variants:

Variant 1 - Literal, Narrowing

function stress(o: { [s: string]: "aa" | "ab" | "ac" | ... }) {
  o["x"] = o["z"] === "de" ? "ff" : "ip";
  // many more lines like this...
  return [o['x'], o['y'], o['z']] as const
}

Variant 2 - Primitive, Non-Narrowing

function stress(o: { [s: string]: "aa" | "ab" | "ac" | ... }, x: string, y: string, z: string) {
  o[x] = o[z] === "de" ? "ff" : "ip";
  // many more lines like this...
  return [o[x], o[y], o[z]] as const
}

Variant 3 - Literal Type, Non-Narrowing

function stress(o: { [s: string]: "aa" | "ab" | "ac" | ... }, x: 'x', y: 'y', z: 'z') {
  o[x] = o[z] === "de" ? "ff" : "ip";
  // many more lines like this...
  return [o[x], o[y], o[z]] as const
}

The most important result is that in real-world scenarios, this change never makes a significant difference in compile time: the second two variants don't actually create flow nodes in the binder, so control flow quits early as there's nothing to analyze. This means that no narrowing happens, of course. Inserting flow nodes only for element accesses with literal access expressions was a performance decision from quite some time ago. As far as I know, we have no reason to revisit that decision.

The first variant does narrow, but only 1000 lines. After that, control flow throws an error "The containing function or module body is too large for control flow analysis."

So I cut the stress tests down to 1000 lines and did a before/after comparison of them:
Selection_046

As you can see, the worst case is less than 50% increase per very-long function. I think that's acceptable given the improvement in narrowing for the common case.

Draft version, just want to see how performance is
@sandersn
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented May 20, 2019

Heya @sandersn, I've started to run the perf test suite on this PR at bb3edc8. You can monitor the build here. It should now contribute to this PR's status checks.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@sandersn
The results of the perf run you requested are in!

Here they are:

Comparison Report - master..31478

Metric master 31478 Delta Best Worst
Angular - node (v12.1.0, x64)
Memory used 314,958k (± 0.05%) 314,934k (± 0.02%) -24k (- 0.01%) 314,855k 315,156k
Parse Time 1.38s (± 1.02%) 1.38s (± 0.53%) -0.00s (- 0.14%) 1.37s 1.40s
Bind Time 0.72s (± 0.68%) 0.72s (± 0.82%) 0.00s ( 0.00%) 0.71s 0.74s
Check Time 4.04s (± 0.35%) 4.03s (± 0.43%) -0.01s (- 0.20%) 4.00s 4.08s
Emit Time 5.14s (± 1.00%) 5.16s (± 0.87%) +0.02s (+ 0.29%) 5.10s 5.30s
Total Time 11.29s (± 0.60%) 11.30s (± 0.50%) +0.00s (+ 0.04%) 11.19s 11.44s
Monaco - node (v12.1.0, x64)
Memory used 343,749k (± 0.02%) 343,747k (± 0.01%) -2k (- 0.00%) 343,668k 343,815k
Parse Time 1.18s (± 0.81%) 1.18s (± 0.74%) +0.00s (+ 0.17%) 1.16s 1.20s
Bind Time 0.67s (± 0.86%) 0.67s (± 0.70%) 0.00s ( 0.00%) 0.66s 0.68s
Check Time 4.15s (± 0.52%) 4.14s (± 0.36%) -0.00s (- 0.05%) 4.12s 4.19s
Emit Time 2.78s (± 0.75%) 2.77s (± 0.34%) -0.00s (- 0.14%) 2.75s 2.79s
Total Time 8.77s (± 0.42%) 8.77s (± 0.18%) +0.00s (+ 0.01%) 8.73s 8.81s
TFS - node (v12.1.0, x64)
Memory used 300,826k (± 0.02%) 300,784k (± 0.02%) -42k (- 0.01%) 300,664k 300,962k
Parse Time 0.91s (± 0.66%) 0.90s (± 1.04%) -0.00s (- 0.22%) 0.88s 0.92s
Bind Time 0.62s (± 0.93%) 0.62s (± 0.90%) -0.00s (- 0.32%) 0.61s 0.63s
Check Time 3.72s (± 0.57%) 3.71s (± 0.44%) -0.01s (- 0.32%) 3.68s 3.74s
Emit Time 2.87s (± 1.12%) 2.87s (± 0.75%) +0.00s (+ 0.10%) 2.83s 2.92s
Total Time 8.11s (± 0.36%) 8.10s (± 0.41%) -0.01s (- 0.15%) 8.01s 8.16s
Angular - node (v8.9.0, x64)
Memory used 332,914k (± 0.02%) 332,904k (± 0.02%) -10k (- 0.00%) 332,745k 333,128k
Parse Time 1.78s (± 0.46%) 1.78s (± 0.51%) +0.00s (+ 0.06%) 1.76s 1.80s
Bind Time 0.79s (± 1.21%) 0.79s (± 1.07%) +0.00s (+ 0.38%) 0.77s 0.81s
Check Time 4.76s (± 1.41%) 4.73s (± 1.17%) -0.03s (- 0.57%) 4.65s 4.90s
Emit Time 5.93s (± 2.21%) 5.85s (± 2.80%) -0.08s (- 1.40%) 5.57s 6.17s
Total Time 13.26s (± 0.75%) 13.16s (± 0.99%) -0.10s (- 0.79%) 12.91s 13.42s
Monaco - node (v8.9.0, x64)
Memory used 360,902k (± 0.03%) 360,846k (± 0.01%) -57k (- 0.02%) 360,688k 360,932k
Parse Time 1.43s (± 0.47%) 1.43s (± 0.47%) +0.00s (+ 0.00%) 1.42s 1.45s
Bind Time 0.91s (± 2.34%) 0.91s (± 1.90%) +0.00s (+ 0.44%) 0.87s 0.94s
Check Time 4.93s (± 1.58%) 4.94s (± 1.66%) +0.01s (+ 0.20%) 4.82s 5.17s
Emit Time 3.24s (± 6.16%) 3.28s (± 5.43%) +0.04s (+ 1.11%) 2.79s 3.41s
Total Time 10.52s (± 1.45%) 10.57s (± 1.15%) +0.05s (+ 0.48%) 10.22s 10.71s
TFS - node (v8.9.0, x64)
Memory used 316,223k (± 0.01%) 316,281k (± 0.01%) +58k (+ 0.02%) 316,191k 316,347k
Parse Time 1.13s (± 0.46%) 1.13s (± 0.62%) 0.00s ( 0.00%) 1.12s 1.15s
Bind Time 0.67s (± 0.75%) 0.66s (± 0.67%) -0.00s (- 0.45%) 0.65s 0.67s
Check Time 4.36s (± 0.62%) 4.37s (± 0.51%) +0.01s (+ 0.23%) 4.32s 4.42s
Emit Time 3.14s (± 0.68%) 3.12s (± 0.45%) -0.02s (- 0.64%) 3.10s 3.17s
Total Time 9.30s (± 0.44%) 9.28s (± 0.42%) -0.01s (- 0.15%) 9.23s 9.41s
Angular - node (v8.9.0, x86)
Memory used 188,543k (± 0.02%) 188,575k (± 0.02%) +32k (+ 0.02%) 188,472k 188,668k
Parse Time 1.72s (± 1.00%) 1.73s (± 0.72%) +0.00s (+ 0.23%) 1.70s 1.76s
Bind Time 0.93s (± 1.15%) 0.94s (± 1.02%) +0.01s (+ 1.08%) 0.91s 0.96s
Check Time 4.42s (± 0.79%) 4.40s (± 0.50%) -0.02s (- 0.47%) 4.36s 4.45s
Emit Time 5.66s (± 1.37%) 5.64s (± 0.95%) -0.02s (- 0.32%) 5.51s 5.77s
Total Time 12.73s (± 0.97%) 12.71s (± 0.37%) -0.03s (- 0.21%) 12.64s 12.82s
Monaco - node (v8.9.0, x86)
Memory used 201,333k (± 0.02%) 201,330k (± 0.02%) -3k (- 0.00%) 201,269k 201,428k
Parse Time 1.49s (± 0.61%) 1.49s (± 0.37%) -0.00s (- 0.07%) 1.48s 1.50s
Bind Time 0.72s (± 0.95%) 0.72s (± 0.96%) -0.00s (- 0.14%) 0.70s 0.73s
Check Time 4.74s (± 0.52%) 4.73s (± 0.66%) -0.01s (- 0.25%) 4.66s 4.79s
Emit Time 3.10s (± 0.89%) 3.09s (± 0.64%) -0.01s (- 0.26%) 3.03s 3.13s
Total Time 10.04s (± 0.42%) 10.02s (± 0.47%) -0.02s (- 0.21%) 9.93s 10.15s
TFS - node (v8.9.0, x86)
Memory used 177,375k (± 0.02%) 177,401k (± 0.02%) +25k (+ 0.01%) 177,350k 177,477k
Parse Time 1.18s (± 1.05%) 1.19s (± 0.73%) +0.00s (+ 0.42%) 1.17s 1.21s
Bind Time 0.63s (± 1.30%) 0.63s (± 0.82%) -0.00s (- 0.47%) 0.62s 0.64s
Check Time 4.16s (± 0.54%) 4.20s (± 0.93%) +0.04s (+ 1.01%) 4.11s 4.29s
Emit Time 2.76s (± 1.39%) 2.79s (± 1.49%) +0.03s (+ 1.05%) 2.69s 2.89s
Total Time 8.73s (± 0.60%) 8.80s (± 0.82%) +0.07s (+ 0.80%) 8.62s 8.94s
Angular - node (v9.0.0, x64)
Memory used 332,518k (± 0.01%) 332,545k (± 0.03%) +27k (+ 0.01%) 332,339k 332,697k
Parse Time 1.63s (± 0.90%) 1.63s (± 0.51%) +0.00s (+ 0.00%) 1.61s 1.64s
Bind Time 0.74s (± 0.92%) 0.74s (± 1.02%) +0.01s (+ 1.09%) 0.73s 0.76s
Check Time 4.40s (± 0.25%) 4.39s (± 0.51%) -0.01s (- 0.11%) 4.34s 4.43s
Emit Time 5.65s (± 0.95%) 5.69s (± 1.27%) +0.04s (+ 0.67%) 5.58s 5.88s
Total Time 12.41s (± 0.44%) 12.45s (± 0.51%) +0.04s (+ 0.29%) 12.31s 12.59s
Monaco - node (v9.0.0, x64)
Memory used 360,559k (± 0.03%) 360,723k (± 0.03%) +164k (+ 0.05%) 360,488k 360,859k
Parse Time 1.28s (± 0.64%) 1.28s (± 0.59%) +0.00s (+ 0.16%) 1.27s 1.30s
Bind Time 0.85s (± 0.68%) 0.85s (± 0.72%) +0.00s (+ 0.12%) 0.84s 0.87s
Check Time 4.77s (± 0.47%) 4.76s (± 0.39%) -0.01s (- 0.13%) 4.72s 4.81s
Emit Time 3.28s (± 0.60%) 3.28s (± 0.47%) -0.00s (- 0.03%) 3.25s 3.31s
Total Time 10.18s (± 0.23%) 10.18s (± 0.31%) -0.00s (- 0.01%) 10.10s 10.27s
TFS - node (v9.0.0, x64)
Memory used 316,082k (± 0.02%) 316,097k (± 0.02%) +15k (+ 0.00%) 316,001k 316,192k
Parse Time 1.01s (± 1.11%) 1.01s (± 0.74%) +0.00s (+ 0.20%) 1.00s 1.03s
Bind Time 0.61s (± 0.73%) 0.61s (± 0.97%) +0.00s (+ 0.33%) 0.60s 0.62s
Check Time 4.28s (± 1.68%) 4.30s (± 2.14%) +0.01s (+ 0.33%) 4.17s 4.49s
Emit Time 3.04s (± 2.41%) 3.01s (± 3.40%) -0.03s (- 1.02%) 2.77s 3.15s
Total Time 8.94s (± 0.30%) 8.93s (± 0.37%) -0.01s (- 0.11%) 8.89s 9.04s
Angular - node (v9.0.0, x86)
Memory used 188,644k (± 0.03%) 188,667k (± 0.03%) +23k (+ 0.01%) 188,574k 188,804k
Parse Time 1.53s (± 0.63%) 1.54s (± 0.66%) +0.00s (+ 0.26%) 1.51s 1.56s
Bind Time 0.86s (± 1.17%) 0.86s (± 0.97%) -0.00s (- 0.12%) 0.84s 0.88s
Check Time 4.10s (± 0.48%) 4.07s (± 0.57%) -0.03s (- 0.73%) 4.03s 4.12s
Emit Time 5.36s (± 0.70%) 5.35s (± 0.47%) -0.01s (- 0.13%) 5.29s 5.40s
Total Time 11.85s (± 0.37%) 11.81s (± 0.40%) -0.03s (- 0.29%) 11.73s 11.92s
Monaco - node (v9.0.0, x86)
Memory used 201,332k (± 0.03%) 201,364k (± 0.02%) +32k (+ 0.02%) 201,306k 201,505k
Parse Time 1.30s (± 0.36%) 1.30s (± 0.58%) +0.00s (+ 0.31%) 1.29s 1.32s
Bind Time 0.65s (± 1.17%) 0.64s (± 1.21%) -0.01s (- 0.77%) 0.63s 0.67s
Check Time 4.60s (± 0.64%) 4.58s (± 0.48%) -0.01s (- 0.28%) 4.55s 4.64s
Emit Time 3.01s (± 0.76%) 3.00s (± 0.64%) -0.01s (- 0.23%) 2.97s 3.06s
Total Time 9.55s (± 0.43%) 9.53s (± 0.26%) -0.02s (- 0.19%) 9.48s 9.57s
TFS - node (v9.0.0, x86)
Memory used 177,465k (± 0.02%) 177,424k (± 0.02%) -41k (- 0.02%) 177,365k 177,517k
Parse Time 1.02s (± 1.04%) 1.03s (± 1.36%) +0.00s (+ 0.29%) 1.01s 1.07s
Bind Time 0.57s (± 0.78%) 0.57s (± 1.01%) +0.00s (+ 0.17%) 0.56s 0.58s
Check Time 4.00s (± 0.60%) 4.01s (± 0.71%) +0.01s (+ 0.28%) 3.94s 4.09s
Emit Time 2.70s (± 0.61%) 2.69s (± 0.57%) -0.02s (- 0.67%) 2.63s 2.71s
Total Time 8.30s (± 0.26%) 8.29s (± 0.32%) -0.01s (- 0.10%) 8.24s 8.37s
System
Machine Namets-ci-ubuntu
Platformlinux 4.4.0-142-generic
Architecturex64
Available Memory16 GB
Available Memory1 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v12.1.0, x64)
  • node (v8.9.0, x64)
  • node (v8.9.0, x86)
  • node (v9.0.0, x64)
  • node (v9.0.0, x86)
Scenarios
  • Angular - node (v12.1.0, x64)
  • Angular - node (v8.9.0, x64)
  • Angular - node (v8.9.0, x86)
  • Angular - node (v9.0.0, x64)
  • Angular - node (v9.0.0, x86)
  • Monaco - node (v12.1.0, x64)
  • Monaco - node (v8.9.0, x64)
  • Monaco - node (v8.9.0, x86)
  • Monaco - node (v9.0.0, x64)
  • Monaco - node (v9.0.0, x86)
  • TFS - node (v12.1.0, x64)
  • TFS - node (v8.9.0, x64)
  • TFS - node (v8.9.0, x86)
  • TFS - node (v9.0.0, x64)
  • TFS - node (v9.0.0, x86)
Benchmark Name Iterations
Current 31478 10
Baseline master 10

let assumeUninitialized = false;
if (strictNullChecks && strictPropertyInitialization && node.expression.kind === SyntaxKind.ThisKeyword) {
const declaration = prop && prop.valueDeclaration;
if (declaration && isInstancePropertyWithoutInitializer(declaration)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this be unified with the similar code in checkPropertyAccessExpressionOrQualifiedName? Maybe a shared worker, like getFlowTypeOfReferenceWithImpliedInitialType?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes! It is basically a cut+paste+rename, so it is next on my list after I find out how much this hurts performance.

@sandersn
Copy link
Member Author

Hm, no change in perf? That can't be right. I'm going to add a micro-benchmark to the perf test suite and report its results.

@sandersn
Copy link
Member Author

I just wrote a stress test for this code. It's of the form:

function stress(o: { [s: string]: "aa" | "ab" | "ac" | ... }) {
  o["x"] = o["z"] === "de" ? "ff" : "ip";
  // many more lines like this...
  return [o['x'], o['y'], o['z']] as const
}

It scales quite well; check time at 100 lines was 0.28 seconds, at 1000 lines it was 0.5 and at 10,000 it was 0.6. Memory increases from 128 MB to 157 MB which is not bad.

What didn't work as well was code like this:

function stress(o: { [s: string]: "aa" | "ab" | "ac" | ... }, x: string, y: string, z: string) {
  o[x] = o[z] === "de" ? "ff" : "ip";
  // many more lines like this...
  return [o[x], o[y], o[z]] as const
}

It scales linearly or a little worse, up to 5.5 seconds and 178 MB for 16,000 lines.

I'll see if I can get a newer code base or two added to our performance suite although I'm not sure what is a good test for this change.

@sandersn
Copy link
Member Author

Update: with more data points, it's pretty clear that the second example scales exponentially, not linearly.

@sandersn sandersn requested a review from ahejlsberg May 29, 2019 21:33
@sandersn
Copy link
Member Author

Well, this PR is as ready to go as it'll ever be. Any opinions @ahejlsberg?

@rbuckton any idea for good code bases that would show off real-world perf? My synthetic benchmark is intentionally unrepresentative.

@sandersn
Copy link
Member Author

From the brief discussion in the 6/28 design meeting (#32357), @rbuckton raised the question of performance on a stress test, which is a variant of the first where identifiers of literal type are substituted for literals. I expect performance to be about the same.

function stress(o: { [s: string]: "aa" | "ab" | "ac" | ... }, x: 'x', y: 'y', z: 'z') {
  o[x] = o[z] === "de" ? "ff" : "ip";
  // many more lines like this...
  return [o[x], o[y], o[z]] as const
}

@sandersn
Copy link
Member Author

I did some additional performance investigation, using the same synthetic stress test as above with 3 variants:

Variant 1 - Literal, Narrowing

function stress(o: { [s: string]: "aa" | "ab" | "ac" | ... }) {
  o["x"] = o["z"] === "de" ? "ff" : "ip";
  // many more lines like this...
  return [o['x'], o['y'], o['z']] as const
}

Variant 2 - Primitive, Non-Narrowing

function stress(o: { [s: string]: "aa" | "ab" | "ac" | ... }, x: string, y: string, z: string) {
  o[x] = o[z] === "de" ? "ff" : "ip";
  // many more lines like this...
  return [o[x], o[y], o[z]] as const
}

Variant 3 - Literal Type, Non-Narrowing

function stress(o: { [s: string]: "aa" | "ab" | "ac" | ... }, x: 'x', y: 'y', z: 'z') {
  o[x] = o[z] === "de" ? "ff" : "ip";
  // many more lines like this...
  return [o[x], o[y], o[z]] as const
}

The most important result is that in real-world scenarios, this change never makes a significant difference in compile time: the second two variants don't actually create flow nodes in the binder, so control flow quits early as there's nothing to analyze. This means that no narrowing happens, of course. Inserting flow nodes only for element accesses with literal access expressions was a performance decision from quite some time ago. As far as I know, we have no reason to revisit that decision.

The first variant does narrow, but only 1000 lines. After that, control flow throws an error "The containing function or module body is too large for control flow analysis."

So I cut the stress tests down to 1000 lines and did a before/after comparison of them:
Selection_046

As you can see, the worst case is less than 50% increase per very-long function. I think that's acceptable given the improvement in narrowing for the common case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Type guard using square bracket notation does not narrow type
4 participants