Modify existing Canvas 2D API to include APIs that allow a screen magnifier to track caret and selection change movements

Summary

Modify the Canvas 2D API to properly handle caret, and selection position to assist screen magnification products to support users with low vision in using canvas. This in response to Issue 131.

Rationale

The existing Canvas 2D API specification does not provide an API to allow an author to notify a screen magnifier as of location changes in the caret position and content selection boundaries. AI Squared, a leading screen magnifier vendor for Windows, has provided feedback on this issue. From the WhatWG use cases WIKI we have the following use cases of product that employ text and rich text editing:

1. Lucidart Drawing Tool Is a diagramming tool that allows an author to edit text labels for drawing objects just like Visio and PowerPoint. The caret is drawn using a separate canvas element with no means to provide for caret and selection tracking during editing.
2. LibreOffice uses HTML5 canvas for its office editors is an HTML5 Canvas office suite with multi-language support. The Office editors need caret and selection tracking to allow for blind access.

In some cases authors may superimpose a text field onto a canvas and allow the user to enter text there and benefit from operating system features such as built in Input Method Editors. This method is used for text selection by the Mozilla PDF.js project. While the method is an appropriate solution for simple text entry, such as providing labels for a drawing object, it lacks flexibility and accuracy when applied to other text editing scenarios and is insufficient for rich text editors which embedded font layout, custom fonts, and arbitrary objects and glyphs. The scope of SVG and HTML does not include arbitrary text decoration or non-standard language, font styles or glyphs.

Based on the current canvas 2D API specification we have the following deficiencies in meeting the requirements of a screen magnifier vendor like AI Squared:

1. ZoomText tracks alignment of objects based on information made available by the author and UA. ZoomText users can specify center, parent or edge alignment for text cursor, focus and window objects differently. In order to provide the correct alignment for the user we need a way to understand the role of a given path. the current scrollPathIntoView function does not indicate whether the path applies to a cursor nor does it associate an element in canvas fallback content which would provide the role of the corresponding object. Because of this deficiency, ZoomText can not identify the object type being scrolled into view.
2. ZoomText may use parent alignment: we need to be able to retrieve the location of the parent element of the path. The goal of parent alignment is to keep the parent object in view (or partial view) while making sure the current object is displayed within the magnified view. scrollPathIntoView does not provide an association to the element in fallback content where the role of the object and its parent would be available.
3. ZoomText often replaces the text cursor with that of a different shape based on the role of the element. ZoomText users will select different text cursors based on the role.
4. Not all platforms move the caret with the selection point. In fact, Mac OSX allows the selection point (where the user is pointing during a selection drag) to move while the caret location is fixed. scrollPathIntoView does not provide this.
5. The current drawSystemFocusRing and drawCustomFocusRing functions address either caret or selection notification changes to the assistive technology.

To address this issue government procurement standards, such as 508, require applications to support system font and color settings. If your application does not comply you fail purchasing requirements. It should be noted that, in our proposal, if no system setting has been set for focus ring then the drawFocusRing simply draws along the path as the author specified while at the same time updating the magnifier based on the focus ring.

Details

I would like to resubmit the setCaretSelectionRect method from the modifications to the Canvas 2D Context API Specification, in the 2011 Issue 131 change proposal, for inclusion in section 9 Complex Shapes (paths) in light of the growing use of canvas for rich text editing. The changes are included here:

• Include the following definition of setCaretSelectionRect in the set of functions highlighted in green at the beginning of section 9 of the Editor's draft (http://dev.w3.org/html5/2dcontext/#complex-shapes-paths)

success = context . setCaretSelectionRect(element, x, y, w, h) Returns true if the given element is focused or a document descendant of an element with focus. Otherwise, returns false.

• Include the following text for setCaretSelectionRect between the invocation processing specifications for drawCustomFocusRing and scrollPathIntoView in section 9:

The setCaretSelectionRect(element, x, y, w, h) method, when invoked, must run the following steps:

• If element does not have selected content and is not focused return false and abort these steps.
• If the element is not a descendant of the element with whose context the method is associated, then return false and abort these step.
• Transform the given point (x, y), the width w, and the height h according to the current transformation matrix.
• If the user agent supports a platform accessibility API the user agent must use the element, transformed coordinates and transformed bounding box, and provide it through the supported accessibility API implementation.
• Return true.

Impact

The impact to the specification is minor and it minimizes the impact to authors while producing a more reliably accessible solution.

Positive Effects

• Provides for both screen magnifier tracking of caret and selection location changes on canvas separate from focus rings to allow a user agent to drive the appropriate platform accessibility APIs.
• Provides a vehicle to determine the associated role of the element being drawn on canvas in fallback content as well as vehicle to determine the role of its parent element.
• Changes the caret information provided to a screen magnifier to include width and height information to aid magnifiers in zooming around a caret or selection position.
• Caret and Selection position APIs are lite weight. The APIs support screen magnification without restricting the user's drawing ability. This also allows this same API to drive magnification zooming when the author wants to draw the magnifier zoom to other drawing constructs.
• Enables emerging rich text editors, such as LibreOffice, to notify a magnifier of caret and selection changes so that the users can change the corresponding pointer cursor (it may be smaller for different drawing objects at different magnification levels as an example) and follow the point of user interaction during rich text editing.

Negative Effects

• Caret and Selection are not integrated with actual drawing. However, it is questionable that we could produce something that is rich enough for canvas

Risks

• Although outside the scope of this issue, this proposal does not address another need for screen magnification involving conveying screen screen coordinates of all content rendered from the fallback content on canvas. There is a separate change proposal to address this for Canvas Hit Testing Change Proposal.

References

• Issue 131
• Original Working Group Decision on Issue 131
• HTML Canvas 2D Context Editor's Draft
• Original Caret, Selection, and FocusRing Change Proposal for Issue 131
• New Canvas Hit Testing Change Proposal for Issue 131
• New Focus Ring Text Baseline Proposal for Issue 131
• Modifications to 2D Context (earlier version but still applicable)
• Bug 11342 Request to expose text baseline in TextMetrics
• Bug 11239 Canvas support accessible caret tracking independent of Focus Ring tracking (with refrence to code to implement Bug 11342