Use active font metrics for grid line rendering

[j4james]

Description of the new feature/enhancement

I was looking at adding support for the crossed-out attribute, but found that it doesn't look very good without getting the font metrics to draw the line in the right position, and ideally the right size. And I know there have also been discussions before about the need to draw the underline in the right position and size.

So as a first step, I thought it might be good idea to put together a PR just gathering the font line metrics, and updating the grid line code to take those metrics into account. This is not about drawing the underline in the right position yet - just getting the existing grid lines to scale with the font size.

Proposed technical implementation details (optional)

My first thought was to use the underline thickness metric for the width of all grid lines, and you can see an example of what that might look like below:

But based on those results, I'm now more inclined to think a hard coded size might be preferable. Either option would be better than the current single pixel width, though, which is almost invisible on high dpi displays. If we do go with a hard coded value, I think something around the thickness of Cascadia or Fira Code would be reasonable (that's about 0.025em).

And while looking at this area of the code, I also wanted to propose some refactoring of the grid line renderer to be more efficient. In the current implementation, if you're drawing an underline across 20 cells, it actually renders it with 20 separate strokes, when it could easily be done in one. The left and right grid lines still need to be rendered one cell at a time, but that's a much less common occurrence.

So just to summarize:

1. Is is OK if I do a PR to make the grid lines scale with the font size?
2. If so, should I use the font's underline thickness or a fixed em size?
3. Can I do a refactor/optimisation of the rendering code at the same time?

[j4james]

I should mention that I tried to see what the grid lines looked like in the legacy console, but I couldn't get any of the COMMON_LVB_GRID attributes to work there. I suspect that's because they were originally only supported on FE versions of Windows (or something along those lines). But the upshot is, I'm not actually sure how these lines were originally intended to be rendered, so perhaps they should always be a single pixel wide regardless of font size. But even then, I would still expect some kind of DPI scaling to apply. And if that's the preferred approach, I'd be happy to do that too.

[DHowett]

This one probably needs @miniksa's input. I think the gridlines were intended to be 1 px or 1 dip (pixel * scale).

Personally, I'm fine with making them scale based on font size at a fixed em proportion.

What's the optimization you're thinking about for 3? Niksa's knocking about in rendering recently, and I poked at it for #6193 (never landed, but might inform future direction).

Idly, I wonder if we're getting closer to the time when we need to decouple underline/overline from gridlines and render them differently (per the callout in #2916)...

[DHowett]

I ought to have mentioned #2915 instead of #2916.

[j4james]

This one probably needs @miniksa's input. I think the gridlines were intended to be 1 px or 1 dip (pixel * scale).

OK, I'll wait to see what he says. I'm not in any rush to submit anything yet - I'm still looking into the GDI side of things.

What's the optimization you're thinking about for 3?

My main objective was to change the grid line rendering from this pattern:

for (size_t i = 0; i < cchLine; i++)
{
    if (lines & GridLines::Top)
        drawline
    if (lines & GridLines::Left)
        drawline
    if (lines & GridLines::Bottom)
        drawline
    if (lines & GridLines::Right)
        drawline
}

to this sort of thing:

if (lines & GridLines::Top)
    drawline
if (lines & GridLines::Bottom)
    drawline
if (lines & GridLines::Left)
    for (size_t i = 0; i < cchLine; i++)
        drawline
if (lines & GridLines::Right)
    for (size_t i = 0; i < cchLine; i++)
        drawline

So in the primary use case of a top or bottom line, you only need a single line being drawn, instead of a loop with lots of little strokes joined together. And pulling the conditions out of the loops in the left and right cases make those a little more efficient too (you can also pull up several of the offset calculations).

These things probably aren't going to be a big deal in terms of performance, but I thought it was worth trying to clean it up a bit if I was going to be extending it anyway to support varying line widths (which is what we'll need for the underline and crossed-out attributes). Oh and I also noticed an off-by-one error in the top line rendering which needed fixing.

Niksa's knocking about in rendering recently, and I poked at it for #6193 (never landed, but might inform future direction).

If you want me to wait for you guys to finish anything there, just let me know. Most of what I'm looking at is in the PaintBufferGridLines methods in the Dx and Gdi render engines, in case that's relevant.

Idly, I wonder if we're getting closer to the time when we need to decouple underline/overline from gridlines and render them differently (per the callout in #2916)...

That was one my reasons for proposing this refactoring. Even if we keep the current grid lines at 1px, I'd still like to get the code to a state where it's easy to add lines of different widths and at different offsets, and also get the metadata in place for the other line types.

If you prefer, we could just include this work as part of the PR for the crossed-out attribute, or the underline decoupling, but I thought it might be cleaner to do it as a separate, preparatory step.

[mdtauk]

Bold grid lines for instance, would benefit from changeable widths

[miniksa]

My first thought was to use the underline thickness metric for the width of all grid lines, and you can see an example of what that might look like below:

I actually really like this as it harnesses the style imparted by each font author. That was one of the reasons why I tried scaling on the box/line characters instead of ignoring them and drawing our own like some Terminals do.

I am willing to be overridden, though, by the rest of you if you believe that I'm crazypants and we should do a specific pixel or em count (scaled by font and DPI) for all fonts.

I should mention that I tried to see what the grid lines looked like in the legacy console, but I couldn't get any of the COMMON_LVB_GRID attributes to work there. I suspect that's because they were originally only supported on FE versions of Windows (or something along those lines).

You have to use the ENABLE_LVB_GRID_WORLDWIDE flag on SetConsoleMode to make that happen. I had initially made it work worldwide when we took over this. But there was one Delphi app out there that used the "unused" flags in our buffer for its own scratch space in Western countries. So when we decided to pay attention to the flags, that app looked weird.

But the upshot is, I'm not actually sure how these lines were originally intended to be rendered, so perhaps they should always be a single pixel wide regardless of font size.

Per conhostv1, they were always one pixel.

But even then, I would still expect some kind of DPI scaling to apply. And if that's the preferred approach, I'd be happy to do that too.

conhostv1 didn't have High DPI support. It would have been system scaled. At which point the 1px would have been coarsely scaled into whatever made sense by DWM and it would have been bigger than 1px. So that would be my preference: to scale whatever we choose.

Personally, I'm fine with making them scale based on font size at a fixed em proportion.

I'm fine with this.

This one probably needs @miniksa's input. I think the gridlines were intended to be 1 px or 1 dip (pixel * scale).

OK, I'll wait to see what he says. I'm not in any rush to submit anything yet - I'm still looking into the GDI side of things.

Given that the decision came from conhostv1 era, I'm 99% sure that no one made a conscious decision here and just pulled "1px" out of their butt to make it happen and there is no documentation of why it was chosen.

We probably should preserve that 1px for the GDI side though (and the other oddball renderers, if they even handle grid lines) and only change the DX one to some other proportion.

What's the optimization you're thinking about for 3?

My main objective was to change the grid line rendering from this pattern:

<snip>

So in the primary use case of a top or bottom line, you only need a single line being drawn, instead of a loop with lots of little strokes joined together. And pulling the conditions out of the loops in the left and right cases make those a little more efficient too (you can also pull up several of the offset calculations).

These things probably aren't going to be a big deal in terms of performance, but I thought it was worth trying to clean it up a bit if I was going to be extending it anyway to support varying line widths (which is what we'll need for the underline and crossed-out attributes). Oh and I also noticed an off-by-one error in the top line rendering which needed fixing.

Yes, I like this. Fewer draw commands is always better.

Niksa's knocking about in rendering recently, and I poked at it for #6193 (never landed, but might inform future direction).

If you want me to wait for you guys to finish anything there, just let me know. Most of what I'm looking at is in the PaintBufferGridLines methods in the Dx and Gdi render engines, in case that's relevant.

No, don't wait. Go ahead.

Idly, I wonder if we're getting closer to the time when we need to decouple underline/overline from gridlines and render them differently (per the callout in #2916)...

That was one my reasons for proposing this refactoring. Even if we keep the current grid lines at 1px, I'd still like to get the code to a state where it's easy to add lines of different widths and at different offsets, and also get the metadata in place for the other line types.

If you prefer, we could just include this work as part of the PR for the crossed-out attribute, or the underline decoupling, but I thought it might be cleaner to do it as a separate, preparatory step.

I also agree with having them go in their own pass if necessary.

I am OK with refactoring going in its own preparatory step to be cleaner.

[j4james]

We probably should preserve that 1px for the GDI side though (and the other oddball renderers, if they even handle grid lines) and only change the DX one to some other proportion.

I'm not thrilled by this idea. At the moment DX is only used in Windows Terminal, and that doesn't actually use grid lines other than as a way to emulate underline and overline. Underline we're going to replace with a real underline with font-specific position and width. But overline isn't covered by font metrics, so it would be convenient if we could just leave it an alias for the top grid line.

But if we go with that plan, then I'd expect the grid line width to be consistent, so that overline looked the same in both renderers. The other option would be for overline to get its own attribute bit, but then the DX renderer has no need to show grid lines at all, so there's still no point in making it render things differently.

Bottom line is I don't really care what width we use, but I do think it would be preferable if it were consistent across renderers.

[miniksa]

We probably should preserve that 1px for the GDI side though (and the other oddball renderers, if they even handle grid lines) and only change the DX one to some other proportion.

I'm not thrilled by this idea. At the moment DX is only used in Windows Terminal, and that doesn't actually use grid lines other than as a way to emulate underline and overline. Underline we're going to replace with a real underline with font-specific position and width. But overline isn't covered by font metrics, so it would be convenient if we could just leave it an alias for the top grid line.

But if we go with that plan, then I'd expect the grid line width to be consistent, so that overline looked the same in both renderers. The other option would be for overline to get its own attribute bit, but then the DX renderer has no need to show grid lines at all, so there's still no point in making it render things differently.

Bottom line is I don't really care what width we use, but I do think it would be preferable if it were consistent across renderers.

Well, okay. We can make it consistent across renderers. I just try to avoid changing any of the other ones when possible because leaving them alone is a 0% chance of bugs/regression where as changing them in any way is now non-zero. I feel it's easier to avoid the problem. But if you are passionate about it being consistent, that's fine with me.

[j4james]

First off, I just want to be clear that I am happy to let you guys make the final decision here - don't let me push you into doing something you're not comfortable with. That said, I've been playing around with a couple of different renderings to see how they feel, and think this might help with the decision making.

In the image below you can see some test cases with three different fonts (Cascadia Mono, Consolas, and MS Gothic) in a range of "normal" font sizes (12pt to 18pt). It's rendered on a high DPI display (at 200%) so you'll want to scale the image to 50% in the browser to get an accurate feel for the size.

In each case I'm rendering both top and bottom grid lines as well as a proper underline and strikethrough line. The grid lines are rendered in 3 different ways: 1 physical pixel (on the left); 1 device-independent pixel, i.e. 2 physical pixels (in the centre); and using the underline width (on the right).

I found the middle one a bit odd, because with Cascadia you end up with grid lines wider than the standard underline, which just seems wrong. And then if you scale the font size up far enough the opposite happens - the underline becomes wider than the grid lines - so it's not even consistent.

The one on the right (using the underline width) is nice in some ways, but with MS Gothic if feels like the grid lines are smothering the text, and this is one of the fonts I suspect is most likely to be used with grid lines. And in the case of Cascadia, the underline width is so narrow that's it's no different from a 1 pixel grid line anyway, at least for most "normal" font sizes.

So after all that I'm starting to think our current rendering (i.e. 1 physical pixel) may well be the best option. However, I'd be a little happier with something like 0.025 em. That's still going to be 1 pixel for most normal font sizes (i.e. it'll look exactly like the left column), but if you did happen to zoom in a large amount, it would feel more natural to see the grid lines scaling.

In short: first preference 0.025 em, second preference 1 physical pixel, but I'll leave it to you guys to make the final choice (and DX and GDI being different is not the end of the world for me). The way the code is now written it's essentially a one line change in each renderer to switch between widths, so it's not a big deal to play around with the different options if you change your mind.

[miniksa]

You're right, MS Gothic is most likely to be used with gridlines so that's the one we should probably make look the best with them.

I'm alright with doing the 0.025em option across all the renderers. If it's that close to how it already is in a majority of cases, I can't see it causing me much if any trouble with the other renderers also changing.

Also, I think I like the way the left most column of these looks as the best option. I'd rather it be based on a text metric though like a fractional em than on a pixel though, so I agree with your priority ranking.

[DHowett]

Yeah, I agree with that order as well. 👍

max(0.025em, 1px), presumably? actually, you covered this

[ghost]

🎉This issue was addressed in #7107, which has now been successfully released as Windows Terminal Preview v1.3.2382.0.:tada:

Handy links:

• Release Notes
• Store Download