Refactor the renderer color calculations

[j4james]

This is a refactoring of the renderer color calculations to simplify the
implementation, and to make it easier to support additional
color-altering rendition attributes in the future (e.g. faint and
conceal).

References

• This is a followup to PRs Add support for the DECSCNM screen mode #3817 and Add support for DECSCNM in Windows Terminal #6809, which introduced additional
  complexity in the color calculations, and which suggested the need for
  refactoring.

Detailed Description of the Pull Request / Additional comments

When we added support for DECSCNM, that required the foreground and
background color lookup methods to be able to return the opposite of
what was requested when the reversed mode was set. That made those
methods unnecessarily complicated, and I thought we could simplify them
considerably just by combining the calculations into a single method
that derived both colors at the same time.

And since both conhost and Windows Terminal needed to perform the same
calculations, it also made sense to move that functionality into the
TextAttribute class, where it could easily be shared.

In general this way of doing things is a bit more efficient. However, it
does result in some unnecessary work when only one of the colors is
required, as is the case for the gridline painter. So to make that less
of an issue, I've reordered the gridline code a bit so it at least
avoids looking up the colors when no gridlines are needed.

Validation Steps Performed

Because of the API changes, quite a lot of the unit tests had to be
updated. For example instead of verifying colors with two separate calls
to LookupForegroundColor and LookupBackgroundColor, that's now
achieved with a single LookupAttributeColors call, comparing against a
pair of values. The specifics of the tests haven't changed though, and
they're all still working as expected.

I've also manually confirmed that the various color sequences and
rendition attributes are rendering correctly with the new refactoring.

[DHowett]

This will also make it easier to do the perceptual color nudging suggested in #2638. Thank you for separating this out of DECSCNM 😄

[j4james]

This will also make it easier to do the perceptual color nudging suggested in #2638.

Actually I was thinking it might be better to handle the perceptual color adjustments at the time that palette was set, so you're not constantly doing those calculations on every paint. Color scheme changes should be comparatively rare.

[DHowett]

I was wondering about that. I'm worried that we'd need to do it for every foreground * every background (from the first 16 colors) and store that somewhere. I'm not too bothered by doing that, but I do want to make sure we understand the risks.

Also, what if an application requests a 16-color foreground on a nearly identical 256-color or RGB background? Is the point of perceptual nudging to solve bad color palettes or to solve bad runtime accessibility?

I'm not certain!

[j4james]

Also, what if an application requests a 16-color foreground on a nearly identical 256-color or RGB background? Is the point of perceptual nudging to solve bad color palettes or to solve bad runtime accessibility?

I'm not certain!

Me neither. But my impression from playing with the ConEmu implementation was that they were just fixing bad palettes. Interestingly, though, whatever they're doing didn't seem to help with Solarized, which is what I thought it would be most useful for, so that kind of put me off the idea. Although there's no reason why we need to mimic their behaviour exactly - it needs a bit of thought.

[miniksa]

I didn't know how much I wanted this until I read it. Thank you very much, @j4james.

[miniksa]

The only thought that floated through my head was "I wonder if this is worse for perf in a circumstance where someone only wants the fg or only wants the bg and we're working on both. But then I thought "huh I bet that's never really a thing, now is it." Or it's not a common enough case for it to matter.

[ghost]

Hello @DHowett!

Because this pull request has the AutoMerge label, I will be glad to assist with helping to merge this pull request once all check-in policies pass.

p.s. you can customize the way I help with merging this pull request, such as holding this pull request until a specific person approves. Simply @mention me (@msftbot) and give me an instruction to get started! Learn more here.

[DHowett]

This PR, and the other followups around color handling, finally shipped in the traditional console with insider build 20197. 😄 Thanks again