CanvasRenderingContext2D API Improvements

[mysteryDate]

I'm requesting a TAG review of nine improvements to the CanvasRenderingContext2D API.

Canvas is the web’s direct mode rendering solution that closely matches traditional programming models. This is particularly targeted for games and full featured apps. Modern 2D developers sometimes have to fallback to GL for features that are expected to be available in 2D but currently aren’t supported by Canvas 2D. There's a always a balance to be struck when adding new APIs to the web. That said, it's important that Canvas2D is able to address developer's use cases in game development and text manipulation.

The current Canvas 2D API was originally proposed in 2013. Since then, a lot of 2D graphics APIs have appeared and changed what developers expect from a good 2D API. This proposal tries to modernize Canvas 2D API, considering current and future usage of Canvas and considering 3 pillars:

1. feature parity with other 2D APIs;
2. access to current capabilities of the Web/CSS;
3. performance improvement.

• Specification URL: Spec to WHATWG is currently in progress. Propsal repo: https://github.com/fserb/canvas2d
• Tests:
  • Round Rect
  • Reset
  • Perspective Transforms
  • Context Loss (This is hard to test on wpt)
  • Filters
  • CSSColorValue input (WIP)
  • WillReadFrequently (WIP to add WPT tests)
  • Conic Gradient
  • Text Modifiers
• Security and Privacy self-review²: Canvas2D is a potential target for fingerprinting. This is actively being studied internally at Google. These changes are on the front-end of the API and do not change font availability, winding, text rendering or compositing.
• GitHub repo: https://github.com/fserb/canvas2d, WHATWG is probably the better place for feedback at this point in the process.
• Primary contacts (and their relationship to the specification):
  • Aaron Krajeski, Chromium author/implementer
  • Fernando Serboncini, Chromium author/implementer
• Organization(s)/project(s) driving the specification: Chromium
• Key pieces of existing multi-stakeholder review or discussion of this specification: See WHATWG issues.
• External status/issue trackers for this specification (publicly visible, e.g. Chrome Status):
  • chromestatus,
  • crbug

Further details:

• I have reviewed the TAG's Web Platform Design Principles
• Relevant time constraints or deadlines: Trying to begin an origin trial in chrome for M92, which has a feature freeze on May 7.

We'd prefer the TAG provide feedback as:

💬 leave review feedback as a comment in this issue and @-notify mysteryDate and fserb.

[LeaVerou]

Hi @mysteryDate & @fserb!

@hober and I took a look at this during a breakout this week.

Overall, we're happy to see that each of these additions is being discussed in various WHATWG HTML issues where you've gotten a lot of feedback from other stakeholders. For instance, the ongoing discussion on the reset() function has been fruitful and we look forward to seeing what conclusion everyone comes to.

Regarding using CSSColorValue for input, and as @plinss already expressed in the Houdini TF discussion on this, we don't think CSSColorValue, as currently specified, is an appropriate object to use here. CSSColorValue is intended to represent CSS syntax in Typed OM, not as a general purpose Color object for the Web Platform. It would require substantial changes to be used in that way. However, there are plans to eventually define a Color object that other APIs can use as input.

On the proposed CSS Text Modifiers, we're concerned about the choice to use camelCased names instead of the actual CSS property names. The names currently proposed are not just camelCased versions of their CSS counterparts (e.g. textLetterSpacing instead of letterSpacing) , but even if they were, there is both mental and coding overhead to converting, compared to being able to use the CSS property verbatim. It might be better to instead expose an API that accepts CSS property names if you are looking to expand the set of CSS properties that you support. The set of properties proposed is also a little surprising, for example font-variant-caps is exposed but none of the other font-variant-* properties, or even font-variant itself.

You've gotten strong implementer feedback that perspective transforms aren't feasible in at least one implementation. Perhaps it's best to leave this for the future, in case that stops being the case later on.

The syntax proposed for SVG filters is strictly less powerful than SVG filters, as pointed out previously by @smfr. Adding APIs that are very similar to other APIs but different in subtle ways is likely to be confusing to authors.

We did not see any issues with any of the other proposed additions.

[mysteryDate]

Hey @LeaVerou! Thank you so much for the feedback. Very glad to hear that you all are on board with the bulk of these changes. It’s exciting to have the opportunity to make the web platform more expressive and useful! As per your concerns:

CSSColorValue as input:
As @annevk has pointed out in this issue if CSSColorValues can stringify then we can use them as style input without changing anything in the specification. A User Agent could short-circuit this and accept the object as an input, but the code would still work the same with the string translation. Would it be ok to add a note to that effect on the spec?

TextModifiers:
With the original proposal, we were trying to make sure that all text related attributes were prepended by the string text or font, which is the case for current text/font properties on CanvasRenderingContext2D. But we also see the rationale behind keeping the names equivalent to those found in HTMLElement style. Unless there’s any other mention to the contrary, we’ll go ahead and change the two affected ones: textLetterSpacing and textWordSpacing will become letterSpacing and wordSpacing, respectively.

Perspective Transforms:
We’re okay to table this feature for the time being.

SVG Filters:
The under-powering is intentional and based on implementer feedback. As @litherum mentioned here:

if we're going to add filter support, we should start with a small set of popular filters, and only expand the set if compelling needs arise. Also, we should start with the "linked-list" model of filters that the css filter() function accepts, rather than arbitrary graphs that are allowed by SVG.

This is a rather large change and it made sense to bite off a smaller chunk for the time being. The proposed syntax could be trivially extended to support DAGs in the future, by allowing CanvasFilters as parameters values.

One of the major motivations for this feature is to make filters available on WebWorkers with OffscreenCanvas. SVG and CSS are DOM dependent, and not trivially available on WebWorkers and thus a new syntax would already be necessary. This proposal has the advantage of being trivially parseable (as a regular Javascript object) and familiar to Javascript developers. Attribute names are copied verbatim from the SVG implementation also with the goal of being as un-surprising as possible.

[LeaVerou]

Hi @mysteryDate 👋🏼

@atanassov and I looked at this during a breakout today at our virtual f2f meeting.

CSSColorValue as input:
As @annevk has pointed out in this issue if CSSColorValues can stringify then we can use them as style input without changing anything in the specification. A User Agent could short-circuit this and accept the object as an input, but the code would still work the same with the string translation. Would it be ok to add a note to that effect on the spec?

It seems like an implementation detail, so we aren't sure a note is needed, but if you want to add one, sure, go ahead.

TextModifiers:
With the original proposal, we were trying to make sure that all text related attributes were prepended by the string text or font, which is the case for current text/font properties on CanvasRenderingContext2D. But we also see the rationale behind keeping the names equivalent to those found in HTMLElement style. Unless there’s any other mention to the contrary, we’ll go ahead and change the two affected ones: textLetterSpacing and textWordSpacing will become letterSpacing and wordSpacing, respectively.

We saw there was some pushback to that, because these attributes do not have the same syntax as the corresponding CSS properties.
We still feel these should be named the same as CSS, since the functionality is the same, even if the values are not. It is more consistent that way, and it means that if the syntax is extended in the future to support more types of values, the property name is already harmonized. The fewer concepts that people need to learn, the better. Keeping names the same is better for discoverability and education.

SVG Filters:
The under-powering is intentional and based on implementer feedback. As @litherum mentioned here:

if we're going to add filter support, we should start with a small set of popular filters, and only expand the set if compelling needs arise. Also, we should start with the "linked-list" model of filters that the css filter() function accepts, rather than arbitrary graphs that are allowed by SVG.

This is a rather large change and it made sense to bite off a smaller chunk for the time being. The proposed syntax could be trivially extended to support DAGs in the future, by allowing CanvasFilters as parameters values.

One of the major motivations for this feature is to make filters available on WebWorkers with OffscreenCanvas. SVG and CSS are DOM dependent, and not trivially available on WebWorkers and thus a new syntax would already be necessary. This proposal has the advantage of being trivially parseable (as a regular Javascript object) and familiar to Javascript developers. Attribute names are copied verbatim from the SVG implementation also with the goal of being as un-surprising as possible.

That seems reasonable to us.

Thank you for working with us and being receptive to feedback. Good luck, we look forward to seeing this progress!