This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.

Bug 28029 - Mouse coordinates represented in CSS pixels do not account for retina displays with window.devicePixelRatio > 1.
Summary: Mouse coordinates represented in CSS pixels do not account for retina display...
Status: RESOLVED MOVED
Alias: None
Product: WebAppsWG
Classification: Unclassified
Component: HISTORICAL - UI Events (show other bugs)
Version: unspecified
Hardware: All All
: P2 normal
Target Milestone: ---
Assignee: Travis Leithead [MSFT]
QA Contact: public-webapps-bugzilla
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-02-14 18:52 UTC by Jukka Jylänki
Modified: 2015-10-07 04:36 UTC (History)
11 users (show)

See Also:
zcorpan: needinfo? (travil)

Attachments

Description Jukka Jylänki 2015-02-14 18:52:55 UTC 
In e.g. the MouseEvent structure, the mouse xy coordinates are stored with the data type 'long', see here: http://www.w3.org/TR/DOM-Level-3-Events/#interface-MouseEvent .

There does not seem to be a clear documentation on how these coordinates relate to the value of window.devicePixelRatio (or I'm missing it). Looking at current implementation in Firefox, Safari and Chrome, the coordinate system that the fields clientX, clientY, screenX and screenY are specified in are the hardware-independent "CSS pixel" units meaning that they do not directly correspond to physical pixels on the display.

That is, testing on a Macbook Pro with 2880x1800 retina display, in each Firefox, Chrome and Safari, window.devicePixelRatio gives 2, and running this page

http://www.w3schools.com/jsref/tryit.asp?filename=tryjsref_event_mouse_screenxy

the extents of screenX and screenY range between (0, 0) and (1440, 900). In order to get the exact pixel coordinate where the mouse lies in, one must multiply the screenX and screenY parameters by window.devicePixelRatio, i.e. by a factor of 2 in this case.

However, since screenX and screenY (and all the other fields that represent mouse coordinates are of type 'long', this means that after multiplying, in both X and Y directions every second pixel is missed and only 25% of all pixels on the display are addressable by screenX and screenY.

4K resolutions are coming in fast, and I expect that browsers might use a window.devicePixelRatio of 4 on those displays. This would mean that only 1/16th or 6.25% of all hardware pixels would be addressable by the mouse event structure!

It looks like all the mouse coordinates throughout all events in the DOM specs should be passed as type 'double' instead of 'long' in places where the coordinates are represented in CSS pixels so that each hardware pixel will be addressable.

I wonder if I'm missing something here, and if this issue can be (has been?) resolved in some other way?
Comment 1 Boris Zbarsky 2015-02-14 19:26:10 UTC 
We should probably add accessors with new names that return the actual coordinate as a double.  Changing the behavior of the existing names is unlikely to be web-compatible.
Comment 2 Boris Gjenero 2015-02-15 05:19:51 UTC 
Note that movementX and movementY values described in the pointer lock API are also affected: https://dvcs.w3.org/hg/pointerlock/raw-file/default/index.html#extensions-to-the-mouseevent-interface

There is the same loss of precision as with position plus there is also the possibility that slow movement gets rounded to zero and ignored. It may be possible to slowly move the mouse as far as you want without causing any movement to be reported. I can reproduce this in Firefox 35 when using increased layout.css.devPixelsPerPx. A browser could work around this by accumulating sub-pixel motion, but floating point values are a better solution.
Comment 3 Anne 2015-02-15 08:43:30 UTC 
Note that the new iMac with a 5K screen simply uses a ratio of 2 so it's not that big of deal yet, but we should probably do something now so it's deployed and ready a couple years from now.
Comment 4 Jukka Jylänki 2015-02-15 10:06:25 UTC 
I would be very cautious to state that this is not currently a big deal, since already now only one quarter of the pixels on the screen are representable by the DOM events on high DPI displays, and Macbook Pro retina displays came out already in 2012.

I believe this issue also exists for all touch events http://www.w3.org/TR/touch-events/#touch-interface , and currently the majority of mobile devices out there have a window.devicePixelRatio greater than one. For example, Google Nexus 4 has a devicePixelRatio of 2, and iPhone 6 Plus has a devicePixelRatio of 3.

Also like Boris Gjenero mentions, this issue causes precision and stuttering to pointer lock and relative mouse movement.

In my opinion, this is definitely a critical P1 bug. Most web APIs do not properly handle high DPI, and the invention of hardware-independent pixels was not done with a good understanding or design of its wide-spanning implications on the different web APIs. This is a surface area of bugs that keeps popping up in different places, here's a small example of recent issues to illustrate:

https://github.com/KhronosGroup/WebGL/issues/587
https://www.khronos.org/webgl/public-mailing-list/archives/1406/msg00019.html
https://bugzilla.mozilla.org/show_bug.cgi?id=1021547
https://bugzilla.mozilla.org/show_bug.cgi?id=1024493

And the common underlying issue in all of these is that the web APIs were not designed to account for window.devicePixelRatio > 1.

My point is that everything is already broken now, and not just in a few years. I would like to plea that this issue is taken seriously in W3, and even further, I would like to see that non-high DPI APIs would get aggressively documented as being broken and deprecated in the future.
Comment 5 Simon Pieters 2015-02-16 19:05:28 UTC 
http://dev.w3.org/csswg/cssom-view/#extensions-to-the-mouseevent-interface specifies MouseEvent's members as double. This has been shipping in IE without opt-in since IE10. Blink plans to implement it also, see http://crbug.com/456625

Also see this thread: https://lists.w3.org/Archives/Public/www-style/2015Feb/thread.html#msg193
Comment 6 Travis Leithead [MSFT] 2015-02-16 20:05:36 UTC 
(In reply to Simon Pieters from comment #5)
> http://dev.w3.org/csswg/cssom-view/#extensions-to-the-mouseevent-interface
> specifies MouseEvent's members as double. This has been shipping in IE
> without opt-in since IE10. Blink plans to implement it also, see
> http://crbug.com/456625
> 
> Also see this thread:
> https://lists.w3.org/Archives/Public/www-style/2015Feb/thread.html#msg193

Note, we just reverted this behavior in our latest IE builds due to compat issues (with web content that works on Chrome). Specifically, for MouseEvent instances, IE will no longer return coordinate members as double. (We still do for Touch/Pointer though.)
Comment 7 Simon Pieters 2015-02-17 09:44:52 UTC 
Thanks Travis. Do you have information on which URLs broke? It is sad that MouseEvent has not been prioritized for Blink before you had to revert it.
Comment 8 Travis Leithead [MSFT] 2015-03-12 22:57:42 UTC 
(In reply to Simon Pieters from comment #7)
> Thanks Travis. Do you have information on which URLs broke? It is sad that
> MouseEvent has not been prioritized for Blink before you had to revert it.

Looking back through the checkins... looks like:
* http://goosecode.com/shapdar was causing the site to hang due to this.
(It was originally reported in Sept of 2013.)
Comment 9 Simon Pieters 2015-03-16 11:18:30 UTC 
OK, that's the same as https://www.w3.org/Bugs/Public/show_bug.cgi?id=24159#c2

Do I understand correctly that there were no other sites reported broken? Would you consider implementing it again if Blink implemented it?
Comment 10 Arthur Barstow 2015-03-21 15:11:54 UTC 
Bulk move of all D3E bugs to the UI Events component.
Comment 11 Gary Kacmarcik 2015-10-07 04:36:30 UTC 
Now tracking as: https://github.com/w3c/uievents/issues/40