Warning:
This wiki has been archived and is now read-only.

ChangeProposals/hitregions

From HTML WG Wiki
Jump to: navigation, search


Provide for canvas location and hit testing capability with an exposed Path object and related alterations to the 2d Context API

Revision of Ted Oconnors proposal

Editor: Steve Faulkner

Summary

As stated in the other Change Proposal for ISSUE-201:

A new hit testing method should be added to canvas elements to simplify hit testing and solve accessibility issues inherent to the fallback DOM concept.

Canvas authors are starting to create more complex user interface in canvas that can only could only [sic] map to several underlying elements ([a] spline editor, for example, would map to several range controls). To enable a better canvas+fallback DOM coding pattern, it seems prudent to enable easier canvas hit testing and event handling.

In the current HTML5 specification, authors are advised to create a fallback DOM under the canvas element to enable screen readers and screen magnifiers to interact with canvas user interfaces. The size and position of those elements are not defined, which causes problems for accessibility tools - what size/position should they report for these elements?

Because canvas elements usually respond to user input, it seems prudent to solve the hit testing and accessibility issues with the same mechanism.

This is for ISSUE-201 (canvas-fallback).

Rationale

Rationale common to both proposals

From the other Change Proposal for ISSUE-201:

Authors will create two kind of canvas user interfaces:

  1. Simple canvas user interfaces that map well to a single regular input element

A canvas checkbox (http://www.htmlstack.com/checkbox/ ) is the quintessential example here. A canvas bitmap is represented by a single checkbox in the DOM.

  1. Complex canvas user interfaces that maps to several regular input and static elements

Authors will create canvas UIs that contain multiple complex visual input controls that should map to several regular elements in the fallback dom.)

(Examples: See the customize-timing-function canvas UI in http://ie.microsoft.com/testdrive/Graphics/hands-on-css3/hands-on_transitions.htm.

These user interfaces don't match well to regular input controls - which might be why the author chose to use canvas.

Also, screen magnifiers must be able to determine the location of fallback elements rendered on the physical canvas in order to be able zoom to these elements whether or not they have focus. Also, screen readers need the location of these objects to assist in placing accessibility information such as text and role on a line in a refreshable Braille display.

The best approach to solving the accessibility issues in these complex canvas user interfaces would be to map the location of the individual user interface elements in the canvas bitmap to multiple tabbable/focusable/etc input elements in the fallback DOM. We should enable authors to easily define hit testing regions on the canvas and map events to the underlying elements in the fallback dom. This will reduce author burden, and also solve the issue of reporting meaningful position+size information to assistive technologies and accessibility test tools. It should be noted that this is not a new concept and the use of hit testing to bind location information to accessibility API has been and continues to be used on every desktop GUI today that supports accessibility services.

Additional rationale for addHitRegion and the Path object

DOM elements are expensive and can negatively impact the performance of Web applications when created in large numbers. Relying on the fallback DOM for hit testing is problematic when <canvas> is being used to represent something that would require a very large number of DOM elements to acheive an equivalent representation. In such cases, it should be possible to hit test and provide location and size information to AT while not bearing the performance and memory costs of an excessive number of DOM elements. The proposed addHitRegion() method allows for tying hit regions to either elements or to lightweight objects which simply provide a label and a WAI-ARIA role. This helps authors to make their canvas content accessible without incurring an unacceptable performance penalty.

ADDED start

As the 'lightweight objects which simply provide a label and a WAI-ARIA role' are not focusable and cannot have author assigned states and properties, the allowed ARIA roles should be limited to a subset of the non interactive ARIA roles that do not require or benefit from the addition of states and properties.

For example, A region with role=button could not be considered accessible unless it is able to receive focus. Or a role=checkbox has a required state of aria-checked, and must be able to recieve focus for device indepedent operation, so it makes no sense to allow authors to assign such roles on lightweight objects.

ADDED end


It should be easy for authors to hit test text drawn onto <canvas>, and also to provide location and size data to AT. A text and font metrics API allows designers to acheive many interesting effects on text while simultaneously enabling hit testing and other A11Y needs.

Having an exposed Path object and encouraging authors to use such Path objects for describing hit regions helps authors to write accessible and maintainable canvas applications. Authors use the 2d context object's current default path for drawing operations, and the 2d context API is designed around the assumption that the current default path is used solely for drawing. Using the current default path to create paths used not for drawing but only for hit testing is awkward; it would be a cleaner, more comprehensible API for authors to be able to explicitly create a hit testing path without pretending that it will be used for drawing.

Details

Re-apply the changes which were reverted by a request of the Chairs, namely the changes in r7023, r7024, r7028, and r7029.

Additionally, apply to the W3C version of the spec the related changes in r7037, r7038, and r7039.

Define a new method removeHitRegion(object) which disassociates any pixels associated with the given object. When invoked, this method must run the following steps:

  1. For each hit region in the canvas' hit region list, run these substeps:
    1. If object is the hit region's control or unbacked region description, invoke the clear regions that cover the pixels algorithm on the hit region's set of pixels.
  2. Return true.

ADDED start

In addition define the ARIA roles that many be assigned to unbacked hit regions:

ARIA role restrictions for unbacked hit regions

Unbacked hit regions cannot be focusable and cannot have the necessary states and properties assigned to adequately convey the semantics of interactive controls. Therefor authors may only assign the following roles using the addHitRegion() > role() method. For all interactive controls associate a hit region with an HTML element using the addHitRegion() > control() method.

subset of document structure roles

  • article
  • group
  • note
  • region
  • separator

landmark roles

  • application
  • banner
  • complementary
  • contentinfo
  • form
  • main
  • navigation
  • search

ADDED end

Finally, apply any revisions necessary for the clean application of the above changes. I believe r7037 depends on the canvas transform work in r7035 and 7036, but determining which other revisions are required to make the above list of revisions workable is left to the discretion of the editor.

Impact

Positive Effects

  • Authors can use the microsyntax from SVG's <path d> to easily create paths, for hit testing or other purposes.
  • Authors can easily draw dashed lines and ellipses.
  • New text metrics allow authors to reliably know where text is drawn onto the canvas, for better hit testing and many other use cases.

Postive effects common to both proposals

  • Authors will have a standard method to implement hit-testing in their applications.
  • Authors will have a standard method to define regions of interest on a canvas bitmap, improving accessibility for users.
  • Allows a screen magnifier user to locate and zoom to any element rendered on canvas
  • Allows a screen reader user with a Braille better fill a refreshable Braille display by using the coordinates supplied to place rendered objects on the same line facilitating a determination of what goes on line of Braille text.
  • Allows assistive reading tools to speak the appropriate content when the user points to items on canvas.

Negative Effects

None.

Conformance Classes Changes

  • Existing implementations of the 2d Context API need to be updated to implement the new objects, interfaces, and methods.

Risks

  • As with all author-facing features, if mainstream User Agents do not implement these API enhancements, we would need to drop them in the future.

References

Most references are inline. See also the Canvas page on the WHATWG's wiki.


Local Variables: mode: text mode: longlines End: