User:Eoconnor/ISSUE-201

From HTML WG Wiki
< User:Eoconnor
Revision as of 21:41, 11 April 2012 by Eoconnor (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search


Provide for canvas location and hit testing capability with an exposed Path object and related alterations to the 2d Context API

Summary

A new hit testing method should be added to canvas elements to simplify hit testing and solve accessibility issues inherent to the fallback DOM concept.

Canvas authors are starting to create more complex user interface in canvas that can only could only map to several underlying elements (A spline editor, for example, would map to several range controls). To enable a better canvas+fallback DOM coding pattern, it seems prudent to enable easier canvas hit testing and event handling.

In the current HTML5 specification, authors are advised to create a fallback DOM under the canvas element to enable screen readers to interact with canvas user interfaces. The size and position of those elements are not defined, which causes problems for accessibility tools - what size/position should they report for these elements?

Because canvas elements usually respond to user input, it seems prudent to solve the hit testing and accessibility issues with the same mechanism.

This is for ISSUE-201 (canvas-fallback).

Rationale

Rationale common to both proposals

Authors will create two kind of canvas user interfaces:

  1. Simple canvas user interfaces that map well to a single regular input elements

A canvas checkbox (http://www.htmlstack.com/checkbox/ ) is the quintessential example here. A canvas bitmap is represented by a single checkbox in the DOM.

  1. Complex canvas user interfaces that maps to several regular input elements

Authors will create canvas UIs that contain multiple complex visual input controls that should map to several regular elements in the fallback dom.)

(Examples: See the customize-timing-function canvas UI in http://ie.microsoft.com/testdrive/Graphics/hands-on-css3/hands-on_transitions.htm.

These user interfaces don't match well to regular input controls - which might be why the author chose to use canvas.

The best approach to solving the accessibility issues in these complex canvas user interfaces would be to map the individual user interface elements in the canvas bitmap to multiple tabbable/focusable/etc input elements in the fallback DOM. We should enable authors to easily define hit testing regions on the canvas and map events to the underlying elements in the fallback dom. This will reduce author burden, and also solve the issue of reporting meaningful position+size information to accessibility tools.

Additional rationale for addHitRegion and the Path object

DOM elements, when created in large quantities, can be very expensive objects which negatively impact the performance of Web applications. Relying on the fallback DOM for hit testing is problematic when <canvas> is being used to represent something that would require a very large number of DOM nodes to acheive an equivalent representation. In such cases, it should be possible to hit test and provide location and size information to AT while not bearing the memory or other performance costs of an excessive number of DOM elements. The proposed addHitRegion() method allows for tying hit regions to either elements or to lightweight objects which simply provide a label and a WAI-ARIA role, which helps authors to make their canvas content accessible without incurring an unacceptable performance penalty.

It should be easy for authors to hit test text drawn onto <canvas>, and also to provide location and size data to AT. A text and font metrics API allows designers to acheive many interesting effects on text while simultaneously enabling hit testing and other A11Y needs.

Having an exposed Path object and encouraging authors to use such Path objects for describing hit regions helps authors to write accessible and maintainable canvas applications. Authors use the 2d context object's current default path for drawing operations, and the 2d context API is designed around the assumption that the current default path is used solely for drawing. Using the current default path to create paths used not for drawing but only for hit testing is awkward; it would be a cleaner, more comprehensible API for authors to be able to explicitly create a hit testing path without pretending that it will be used for drawing.

Details

Re-apply the changes which were reverted by a request of the Chairs, namely the changes in r7023, r7024, r7025, r7026, r7028, and r7029.

Additionally, apply to the W3C version of the spec the related changes in r7033, r7034, r7037, r7038, and r7039.

Also, apply any revisions necessary for the clean application of the above changes. I believe r7037 depends on the canvas transform work in r7035 and 7036, but determining which other revisions are required to make the above list of revisions workable is left to the discretion of the editor.

Impact

Positive Effects

  • Authors can use the microsyntax from SVG's <path d> to easily create paths, for hit testing or other purposes.
  • Authors can easily draw dashed lines.
  • New text metrics allow authors to write code that can reason about how text is drawn, for better hit testing and many other use cases.

Postive effects common to both proposals

  • Authors will have a standard method to implement hit-testing in their applications.
  • Authors will have a standard method to define regions of interest on a canvas bitmap, improving accessibility for users.
  • Allows a screen magnifier user to locate and zoom to any element rendered on canvas
  • Allows a screen reader user with a Braille better fill a refreshable Braille display by using the coordinates supplied to place rendered objects on the same line facilitating a determination of what goes on line of Braille text.
  • Allows assistive reading tools to speak the appropriate content when the user points to items on canvas.

Negative Effects

None.

Conformance Classes Changes

  • Existing implementations of the 2d Context API need to be updated to implement the new objects, interfaces, and methods.

Risks

  • As with all author-facing features, if mainstream User Agents do not implement these API enhancements, we would need to drop them in the future.

References

Most references are inline. Other sources used while compiling this proposal:


Local Variables: mode: text mode: longlines End: