Warning:
This wiki has been archived and is now read-only.

ChangeProposals/CaretTracking

From HTML WG Wiki
Jump to: navigation, search

Augment the Canvas 2D API to include caret navigation and selection

Summary

Augment the Canvas 2D API to allow caret, and selection position information to be conveyed to third-party applications, with a focus on Screen magnification products per WCAG and UAAG designed to support users with low vision. This in response to Issue 131.

Rationale

The existing Canvas 2D API specification does not provide an API to allow an author to notify a screen magnifier as of location changes in the caret position and content selection boundaries. AI Squared, a leading screen magnifier vendor for Windows, has provided feedback on this issue. That feedback is located in the WhatWG use cases WIKI set up by the HTML5 editor. The Canvas 2D API introduced fillText and strokeText mechanisms for the display of text. While authors may superimpose text fields onto canvas, using "color: transparent" and close alignment, the solution has varies drawbacks. This solution has been used with some success by pdf.js. It is not being used by other Canvas projects. While the method is an appropriate solution for static rendering of PDF, it lacks flexibility and accuracy when applied to other scenarios and requires precise font management and conditions. It is insufficient for rich text, embedded font layout, custom fonts, and arbitrary objects and glyphs. The scope of SVG and HTML does not include arbitrary text decoration or non-standard language, font styles or glyphs.

Based on the current canvas 2D API specification we have the following deficiencies in meeting the requirements of a screen magnifier vendor like AI Squared:

  1. ZoomText tracks alignment of objects based on information made available by the author and UA. ZoomText users can specify center, parent or edge alignment for text cursor, focus and window objects differently. In order to provide the correct alignment for the user we need a way to understand the role of a given path. the current scrollPathIntoView function does not indicate whether the path applies to a cursor nor does it associate an element in canvas fallback content which would provide the role of the corresponding object. Because of this deficiency, ZoomText can not identify the object type being scrolled into view.
  2. ZoomText may use parent alignment: we need to be able to retrieve the location of the parent element of the path. The goal of parent alignment is to keep the parent object in view (or partial view) while making sure the current object is displayed within the magnified view. scrollPathIntoView does not provide an association to the element in fallback content where the role of the object and its parent would be available.
  3. ZoomText often replaces the text cursor with that of a different shape based on the role of the element. ZoomText users will select different text cursors based on the role.
  4. Not all platforms move the caret with the selection point. In fact, Mac OSX allows the selection point (where the user is pointing during a selection drag) to move while the caret location is fixed. scrollPathIntoView does not provide this.
  5. Mac OS X has highlight options for highlighting the current caret position for keyboard+caret navigation built into the OS. This feature would work with Canvas if the information is exposed by the author, through the UA. The feature is part of VoiceOver.
  6. The current drawSystemFocusRing and drawCustomFocusRing functions address either caret or selection notification changes to the assistive technology.
  7. Some platforms are entirely browser-based, such as Google's Chrome OS. These platforms use additional accessibility APIs often under a private namespace. With Chrome OS, it is the chrome.* namespace. Event listeners on this namespace must be notified of positions in caret and selection for ATs to work at all on the platform.
  8. Some platforms have browser extensions which implement assistive technology techniques. These extensions can be used instead of an OS-based AT. These extensions should still follow WCAG and expose data; further, they need programmatic access to data on target sites. It's reasonable that an AT would be written in Canvas; it's also reasonable that the AT would work with an appropriately marked up Canvas-based application.

To address this issue government procurement standards, such as 508, require applications to support system font and color settings. If your application does not comply you fail purchasing requirements. It should be noted that, in our proposal, if no system setting has been set for focus ring then the drawFocusRing simply draws along the path as the author specified while at the same time updating the magnifier based on the focus ring.

Details

I would like to resubmit the setCaretSelectionRect method from the modifications to the Canvas 2D Context API Specification, in the 2011 Issue 131 change proposal, for inclusion in section 9 Complex Shapes (paths) in light of the growing use of canvas for rich text editing. The changes are included here:

success = context . setCaretSelectionRect(element, x, y, w, h) Returns true if the given element is focused or a document descendant of an element with focus. Otherwise, returns false.

The setCaretSelectionRect(element, x, y, w, h) method, when invoked, must run the following steps:

  • If element does not have selected content and is not focused return false and abort these steps.
  • If the element is not a descendant of the element with whose context the method is associated, then return false and abort these step.
  • Transform the given point (x, y), the width w, and the height h according to the current transformation matrix.
  • If the user agent supports a platform accessibility API the user agent must use the element, transformed coordinates and transformed bounding box, and provide it through the supported accessibility API implementation.
  • Return true.

Impact

The impact to the specification is minor and it minimizes the impact to authors while producing a more reliably accessible solution.

Positive Effects

  • Provides for both screen magnifier tracking of caret and selection location changes on canvas separate from focus rings to allow a user agent to drive the appropriate platform accessibility APIs.
  • Provides a vehicle to determine the associated role of the element being drawn on canvas in fallback content as well as vehicle to determine the role of its parent element.
  • Changes the caret information provided to a screen magnifier to include width and height information to aid magnifiers in zooming around a caret or selection position.
  • Caret and Selection position APIs are lite weight. The APIs support screen magnification without restricting the user's drawing ability. This also allows this same API to drive magnification zooming when the author wants to draw the magnifier zoom to other drawing constructs.
  • Enables emerging assistive software, such as ChromeVox, to notify a magnifier of caret and selection changes so that the users can change the corresponding pointer cursor (it may be smaller for different drawing objects at different magnification levels as an example) and follow the point of user interaction during caret navigation and selection, often via keyboard.

Negative Effects

  • Caret and Selection are not integrated with actual drawing. However, it is questionable that we could produce something that is rich enough for canvas

Risks

  • Although outside the scope of this issue, this proposal does not address another need for screen magnification involving conveying screen screen coordinates of all content rendered from the fallback content on canvas. There is a separate change proposal to address this for Canvas Hit Testing Change Proposal.

References

Retrieved from "https://www.w3.org/html/wg/wiki/index.php?title=ChangeProposals/CaretTracking&oldid=12117"