JCC Erratum Impact on .NET Core

[brianrob]

JCC Erratum Impact on .NET Core

In November 2019, Intel announced the Jump Conditional Code (JCC) erratum which may occur under conditions involving jump instructions that span 64-byte boundaries. Intel is resolving the JCC erratum via a microcode update that has a potential performance side-effect. The goal of this investigation was to determine whether or not we need to do any work in the runtime or framework to mitigate the impact of the microcode update.

For more details on the JCC erratum, or how applications can mitigate the performance impact, see https://www.intel.com/content/dam/support/us/en/documents/processors/mitigations-jump-conditional-code-erratum.pdf.

Thanks to everyone who participated in discussions around the JCC erratum in dotnet/runtime#13794 and dotnet/runtime#13795.

Discussion

Please share your thoughts on this change and see what others are saying at:

• JCC Erratum Impact on .NET Core runtime#35730

The Bottom Line

While we do see some impact from the JCC microcode update on our microbenchmark results on both Windows and Linux, we don’t see any negative impact on the “real world” scenarios that we A/B tested. As a result, we do not intend to implement any mitigations in the runtime or framework at this time. Instead, our ask of you, is to help us identify your business-critical scenarios where you are seeing an unacceptable impact. We will use this data to inform future work in this space.

Comparison Setup

In order to do a clean comparison, we setup two identical machines each with an identical Skylake processor. One machine was setup with a internal-only test build of Windows, and the other was setup with a publicly available build of Linux (Ubuntu 18.04). Both were able to deploy and revert the microcode update (MCU). This allowed for A/B comparisons and switching between the two versions of microcode on each machine.

Testing Strategy

The testing strategy was to run a selection of microbenchmarks and a set of larger scenarios across Windows and Linux, with the goal of understanding how impactful the JCC microcode update is to .NET Core.

Scenario Results

The scenarios used for this test are:

• Linux: Cloud-scale app throughput - TechEmpower Plaintext and JSON
• Windows: WPF app start-up - Real world app (Name withheld)
• Windows: ASP.NET app start-up - Link-A-Thon (This is an ASP.NET Core app that has been used for many experiments recently)

Most scenario averages are within 1%, with the exception of TechEmpower JSON which improves with the microcode update by 2.45%.

Note: Negative percentages are regressions and positive percentages are improvements.

TechEmpower Plaintext - Linux

	Before	After	Diff (%)
Avg	6,628,782.80	6,608,739.60	-0.30%
Stdev	10,334.67	40,146.37
Min	6,605,400.00	6,496,095.00	-1.65%
Max	6,645,216.00	6,632,308.00	-0.19%

Techempower Json - Linux

	Before	After	Diff (%)
Avg	568,352.00	582,285.40	2.45%
Stdev	60,443.60	38,572.44
Min	437,075.00	513,170.00	17.41%
Max	667,430.00	631,945.00	-5.32%

WPF Real-World App Startup - Windows

	Before	After	Diff (%)
Avg	2740.761	2734.087	-0.24%
Min	2725.215	2696.806	-1.04%
Max	2773.128	2758.793	-0.52%

ASP.NET Core Link-A-Thon Startup - Windows

	Before	After	Diff (%)
Avg	340.604	338.521	-0.61%
Min	325.59	329.391	0.12%
Max	375.822	346.351	-7.84%

Microbenchmark Results

The microbenchmark suite contained 1673 benchmarks when run for this investigation. The benchmarks were run both with and without the microcode update applied. The graphs below represent the grouping of differences between the benchmarks with and without the microcode update applied.

Axes:

• X: Ratio of after/before. Stated another way, the ratio is (withmcu/withoutmcu). Ratios less than 1 mean the benchmark performed better with the JCC microcode update applied. Ratios greater than 1 mean the benchmark performed worse with the JCC microcode update applied.
• Y: Count of benchmarks in the bucket.
  
  While there are some outliers, for the most part, the benchmarks are clustered around a ratio of 1 with the most benchmarks in the 1.038 – 1.085 bucket, which represents single digit percentage degradation.