PointerEvents should have fractional coordinates

[mustaqahmed]

PointerEvent spec doesn't override the MouseEvent spec definition of (clientX, clientY) and (screenX, screenY) coordinates: all of them are integers. In Edge, PointerEvents (and not MouseEvents) have fractional client coordinates even though screen coordinates are always integers.

Chrome follows the spec but wants to follow Edge.

1. Should we spec this?
2. If yes, does it make sense to have both screen and client coordinates fractional, instead of only client coordinate as in Edge?

[scottgonzalez]

+1 for all fractional values

[patrickhlauke]

i'd be for changing those to float, unless there's some compat risk? @teddink ?

[teddink]

I agree with changing them to float for consistency reasons. I am not overly concerned about compat risks since we have had fractional values for those devices that support it for deviceX/Y for quite some time.

On the other hand, we are fairly sure that the underlying system does not provide fractional values for screenX/Y.

What's happening is that screenX directly takes the value passed to us from the system, specifically the ptPixelLocation in the POINTER_INFO.

For clientX/Y we do ScreenToClient() conversion of the screen, then convert to layout coordinates, then convert back to document coordinates. It seems like this process allows subpixel transforms.

[staktrace]

I'm in favor of making everything (including screenX/Y) float where feasible, for the simple reason that it's easy to always provide int values even if the type is spec'd as float, but not so the other way around. Also for example on some Firefox OS devices we had hardware actually reporting subpixel-valued touch events, and if we ever want to expose that level of precision to content then it makes sense to have floats instead of ints.

[RByers]

Discussed in the hackathon that Edge's behavior with screenX/Y is a bug - it's using physical pixels instead of CSS px units. Andrew (Edge) has a bug to fix this, then it will be fractional too.

[RByers]

Note that CSS OM View already updates MouseEvent to define the co-ordinates as fractional. Once implementations have aligned on this, I'm sure we can get it moved into the UI Events spec. If, for web compat, we have to say that mouse* events always truncate, that's fine - then our spec can clarify that pointer* events have no such restriction.

Let's leave this issue open to remind ourselves to get the right spec updates for this. I'm guessing it's probably not practical to write web-platform-tests for this (until we have #120), right?

/cc @zcorpan @garykac

[RByers]

In our call today I said I'd file a bug on UI Events to adopt the double values. There's already a bug at w3c/uievents#40, but I don't think we can quite claim that implementations match this yet.

In Chrome, (new MouseEvent('mousedown',{clientX:5.5})).clientX still returns 5 (though @mustaqahmed is fixing this.

In Edge, even (new PointerEvent('pointerdown',{clientX:5.5})).clientXreturns 5 (but 5.5 as expected in Chrome). What's up with that @teddink, surely that's just a bug (given that the browser itself does generate events with non-integer co-ordinates) right?

So let's wait until we've got at least these implementation issues addressed and behaving consistently between Edge and Chrome before we push to adopt the IDL change into the UI Events spec.

WebKit and Gecko still use long, not double in their MouseEvent IDL.

[mustaqahmed]

Looks like we can partially test fractional coordinates even without #120 , through trivial changes in the web-platform-test pointerevent_constructor.html.

@RByers: Do you think it's a reasonable thing to do even without a change in spec wording? I am assuming that PointerEvents can be inferred to be based on the MouseEvent overrides in CSS OM View spec.

[RByers]

Yes, I think we should just test this here.

Ideally the the CSS OM View spec would have tests for fractional values passed to MouseEvent constructors (maybe it does already @zcorpan?). But regardless I thin it's fine to just update pointerevent_constructor.html for this. It's already depending on the UI Events spec eg. for the test for clientX, no reason it can't depend on CSS OM View also. I'd just add a comment with the relevant spec link.

We should probably also be pushing for the OM View MouseEvent changes to be merged back into the UI Events spec now that there are at least 2 shipping implementations /cc @garykac.

[RByers]

Note that testing that fractional co-ordinates get passed through the constructor isn't quite the same thing as testing real browser-generated events. So ideally we'd also use our automation hooks to test that too. But rather than create but-yet more -manual tests, I think it's fine to just block that on #120 (which is hopefully happening soonish - at least for Chrome).

[patrickhlauke]

@RByers (or anybody else) ... anything more on this than can/needs to be done?

[RByers]

I believe it should now be possible to test this, but I'm not entirely up-to-date on the status. Maybe @NavidZ can take a look?

[NavidZ]

The current automation API only allows an element as the target and not the exact coordinates.
But I believe I should be able to make an element that the target coordinate lands on a fractional coordinate. I'll look into this.

[NavidZ]

We already have a manual test for this usecase. I was going to use our click api but I realized it also expect the touch and pen inputs. I suggest waiting for this PR to land to have synthetic pointer action API and then directly use that instead for the automation. I'll keep an eye on the progress on that front and will take care of updating pointerevent tests accordingly.