Oversampling causes DynamicFont metrics to lock to (onscreen) integer pixel height font sizes

[wareya]

Godot version

3.x

System information

Windows

Issue description

Godot applies font oversampling by changing the font size it asks freetype to render the font with, via FT_Set_Pixel_Sizes. But because FT_Set_Pixel_Sizes only accepts integer font sizes, this has snowball effects everywhere else in all text flow logic. Everything gets its metrics from freetype, which are based on the floored font size, instead of the theoretical fractional font size. This means that the effective size of the metrics used for cursor advancement and line height (etc) go down between onscreen integer font heights.

This would not be an issue if godot used harfbuzz for all text flow, but AFAIK that's a 4.0 thing. If I had to guess, I'd reckon that the "right" thing to do might be to have two copies of the font loaded in freetype, one with and one without the oversampling adjustment, and to get all metrics (especially advancement, the most critical one) from the non-oversampling-adjusted font, but rendered glyph data from the oversampling-adjusted copy.

Steps to reproduce

Add a text label with a large-ish (e.g. 32+ pixels) DynamicFont font and a lot of text (preferably enough to linewrap multiple times), enable the 2d window scaling mode, then run the project and resize the window. Observe that non-linewrapping text stays mostly the same (maybe with some jitter) until it hits a new font pixel height, at which point it completely changes, and that linewrapping text reflows constantly as its size relative to the label it's in keeps changing.

I don't have a minimal reproduction project, but I have something that's probably better, a diff for the 3.x branch that partially works around this bug to demonstrate that this is indeed where it's coming from, which you can use to compare:

godot_font_oversampling_demonstration_diff.txt

without diff:

2022-01-01_07-56-27.mp4

with diff:

2022-01-01_07-28-29.mp4

(this diff is purely for the sake of demonstration, it is not MR-ready. for example, not every usage of the new oversampling_error property is correct and it's not used in every place that it should be used in (e.g. it might be a good idea to use it for glyph rect size if the font has filtering enabled). it also doesn't completely fix advancement changing between font sizes when hinting is enabled. it's merely enough to demonstrate the nature of the problem.)

Minimal reproduction project

No response

[Calinou]

Related to #47957. This is now fixed in master thanks to #55905.

[wareya]

This is also probably the underlying cause for #48415.