piecewise_linear_map: perform internal computations as f64 to increase precision #1120
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
but keep the outward interface as f32 to limit the scope of changes (otherwise callers would have to deal with f64s and there are a lot)
This seems to fix #1117 in the sense that ttx_diff.py now reports Grenze-Gotisch as "output is identical".
Basically, the loss of precision happened while converting glyphs sources to ir: the instance locations are in design-space in the source; we store them as normalized location
fontc/glyphs2fontir/src/toir.rs
Line 288 in 53151a1
... but then in the ir::NamedInstance they are in user-space (like in fvar) so they must be converted again
to_user
:fontc/glyphs2fontir/src/source.rs
Lines 397 to 404 in 53151a1
In this particular font, design- and user-space location are the same (there are no axis mappings). The instance at wght=200 (design) is converted to 0.666667 (normalized) and then again to 200 (user). If the lerp is done with f32 we lose precision and instead of 200 we get back 200.00002
I'm not entierly sure this is the best fix but it does seem to improve the situation at least.