superpmi-diffs results improvements

[jakobbotsch]

Some suggested follow-ups from #74584

• Use HTML tables to delineate rows in tables more
• Display net contributions in +/- style (done in SPMI: Write out improvements/regressions breakdown details #76711)
• Display sample diffs (done in SPMI: Include example diffs in markdown summary #82364)
• When there are both improvements and regressions in the different collections this should be clear in the header (e.g. maybe display as a range instead of average, see e.g. Cache allregs to avoid checking the type repeatedly #76850 (comment)) (done in SPMI: Display TP diffs as a range in the header #81661)

category:eng-sys
theme:super-pmi
skill-level:beginner
cost:small
impact:small

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[ghost]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Some suggested follow-ups from #74584

• Use HTML tables to delineate rows in tables more (cf. JIT: Enhance metrics reported by superpmi diffs #74584 (comment))
• Display net contributions in +/- style (cf. JIT: Enhance metrics reported by superpmi diffs #74584 (comment))

Author:	jakobbotsch
Assignees:	jakobbotsch
Labels:	area-CodeGen-coreclr, untriaged
Milestone:	8.0.0

[EgorBo]

Can we display a couple of real diffs on the summary page? e.g. for a few smallest methods

[jakobbotsch]

Can we display a couple of real diffs on the summary page? e.g. for a few smallest methods

Would be nice too.

Also if we see any TP diffs in FullOpts we currently display the rows for those collections in all tables; we should consider not showing the rows in the MinOpts table if there are no diffs (for asm diffs it's harder, as we cannot tell if there are diffs or not just based on "0 byte" diffs).

[BruceForstall]

If there are no diffs, I'd like to see that displayed prominently at the top. Currently, you need to visually examine each platform to notice that. Also, I think "No diffs" (or maybe the corresponding "X diffs"?) should be slightly larger font and first, per-section.

It seems like "Diffs are based on ..." can be in the "Detail" expansion. I understand if we want to keep "MISSING" in the main view.

e.g.,:

ASM diffs generated on Windows x64
No diffs found on any platform or Diffs found on 3 of 5 platforms

Linux arm64
No diffs found.
MISSED contexts: base: 846, diff: 846
Details
...

[jakobbotsch]

Sorry @BruceForstall, I have forgotten to respond to that.

If there are no diffs, I'd like to see that displayed prominently at the top. Currently, you need to visually examine each platform to notice that.

I agree something like that would be nice, but currently we do not have a model of the results of the other runs. The overall diffs summary is just a concatenation of the summaries created by child SPMI runs. I suppose we could check for "No diffs found" or something similar in those markdown files.

It seems like "Diffs are based on ..." can be in the "Detail" expansion. I understand if we want to keep "MISSING" in the main view.

The original intent was to make it clear that there is a reasonable set of methods in the collection to make conclusions based on the results, but I agree that it could be moved under the details section.

[BruceForstall]

but currently we do not have a model of the results of the other runs.

I previously opened #80719, which is related, suggesting some alternatives to how summarization is implemented. E.g., have the individual jobs create a data model of their results and a "Summarization" job combine them into a report. Would probably be a fair amount of work, with possibly not a huge amount of benefit.

[jakobbotsch]

Most of this was addressed, so will close this. I think we can consider changing summarization once we address #80719.