Accessibility Object Model -- 21 Sep 2016

along with my colleagues Alice Boxhall from Google, Dominic Mazzoni from Google, and Alex Surkov from Mozilla

[reviews agenda]

History, Common PRoblens, code samples, Scoping considerations, Audience, and Development plan

History

Intentional Events, Canvas Sub-DOM, Apple: UI Independence (spawned IndieUI) and Accessibile RTE

scribe: which has all been abandoned
... Google chrome. automation which is a view of Accessibility tree API extension

<aboxhall> Mozilla a11yapi: https://github.com/WICG/a11yapi

scribe: Mozilla proposed A11y API proposal, Microsoft - WAPA proposal
... 2016 - Offline vendor discussions leading to the WICG incubation AOM effort

<aboxhall> WAPA: https://rawgit.com/cyns/wapa/master/wapa.html

scribe: Dominic sent this prooposal to WICG yesterday

<aboxhall> chrome.automation: https://developer.chrome.com/extensions/automation

Common Problems

DOM manipulation has poor performance with ARIA in large datasets

scribe: reliance on IDREF prevents some usage - want to use element refs instead of idrefs
... No way to receive accessibility vents, e.g. increment a custom slider
... some complex office suites are challenging to make accessbile through standard techniques of POSH + ARIA
... Bonus: Accessibility Inspection and Testing

Example

Standard Assistive Technology Interaction

[describes diagram - application -> Accessibility tree -> assistive technology -> user

scribe: the application refreshes a button on the accessibility tree, which passes role to the assistive technology which passes the role "button" to the user.
... the user presses the "button" (independent event) which passes it back to the accessibility tree which translates it to a click to the application

Next Slide diagram

scribe: application sends "role=slider" to the accessibility tree which passes it to the assitive technology which gives it to the user. The user increases the sliderto 50%, passes it the assitive technology to the accessibility tree. The tree fails in giving it back to the application. That problem is where the AOM needs to live.

Doug: Question: Why doesn't ARIA role=slider work?

<patrick_h_lauke_> for context, SteveF and I (plus some help from others) made a first attempt ages ago to document ARIA patterns that fall apart on touch+AT scenarios https://docs.google.com/spreadsheets/d/1gN9oRZPdrJxLDNtT6nVO4fn7E7sn1061L9Xl3__slZ4/edit

<patrick_h_lauke_> and this is exactly the problem outlined here with the "faked" ARIA slider

James: This is something we tried to bring into INdieUI, and I think the scope was too broad.

<patrick_h_lauke_> tl;dr: even if custom controls follow EXACTLY the patterns for kbd usage outlined in ARIA, they will be completely inaccessible for touch+AT scenarios

<patrick_h_lauke_> due to the assumption that keyboard listeners will be enough

<alastairc> patrick_h_lauke_ is there a place where this is being worked on?

Matt King: The patterns we are creating for ARIA widgets, that process keypress for accessibility

James: WE aren't all the way there yet.
... Alice will talk about more examples

<patrick_h_lauke_> so for keyboard (without AT) authors will still have to do key handling, and then for AT usage AOM will provide more input-agnostic event handling

Alice: ... [shows code for an example of an autocomplete list]

<patrick_h_lauke> el.accessibleNode.labelledBy = [el1]

Alice: el.accessibileNode.labelledBy=[el1];

<patrick_h_lauke> const input = combobox.shadowRoot.querySelector("input")

<patrick_h_lauke> const optionList = comboBox.querySelector("custom-optionlist");

<patrick_h_lauke> input.accessibleNode.activeDescendant = optionList;

James: labelledBy is an ARIA property, there is no attempt to override ARIA, just to use it in javascript.
... none of the smaples being proposed are possible today

Wilco: Could you insert an ID so you change it later?

James: Use the ARIA DOM manipulation thatwe have today
... this extends the existing ARIA -- this is not for most developers

Rich: You are going to have two trees, isn't this going to cause problems?

Brian: As a developer, there are two trees. Having something that explains it is better than magic that we have no insight into it.

Alice: WE are trying to keep this tightly scoped so we can ship something as quickly as possible.
... readily implementable
... must not inadvertently expose user details (privacy)
... must not negatively affect performance
... must be easy to author, but it is not for all authors.

Audience for AOM

Primarily for experts

scribe: frameworks engineers
... web component authors
... Doc suite developers (Google Docs, iWork for iCloud, etc)
... JS-expert Accessibility experts

Tightly Scoped Developments

1) Modifying Accesible Propoerties

2) Accessible Actions and events

3) Virtual accessibility nodes (e.g. canvas, WebGL)

4) Introspection of the Fully Computed Accessibility Tree. Need to very careful for performance

Domenic: To be able to set a property and not read it back is a new condition on the web.

@@@: gives example of when it has been done before

Alice: Style and Computed Style is a g ood example

Domenic: How does stage 2 (accessibility events) avoid exposing user details?

Alice: It's a outstanding issue (TBD), it may likely require explicit user consent.

James: If you had an example of a slider, it would have it own @@ and the average user would not receive the event.
... and example is location services. Most developers will set up an opt-in process, and I anticipate that is the way it could work

github.com/a11y-api

scribe: It will move to WICG soon.

<aboxhall> https://discourse.wicg.io/t/contributing-the-accessibility-object-model-specification/1702

Lisa: This is giving developing developers direct access to the accessibility tree

James: IN phase 4. Maybe

Lisa: there is a fuzzy area between assistive techology and the app
... example, people will still use HTML scraping instead of this tecchnology

James: In phase 4 people may be able to do that.
... This is not the platform tree, it is a browser tree. Windows, Mac and iOS all use different technologies

Jesse: Ifk you take away the assistive tech, we still need to make it keyboard accessible. If we mix accessible events into this mix, we aren't getting extra benefit for getting @@

<patrick_h_lauke> Jesse: seems like there a re two proposals: access to accessibility tree, and accessibility events

Jesse: There are two things happening. Accessible events that can be listed to.

Matt: Mousedown and mouse events -- that should still work. Nothing breaks, nothing changes (knock wood)
... the benefit is only for assistive technologies that do not use keyboard events to increment
... [example]

<patrick_h_lauke> ...AOM doesn't add anything new for keyboard/desktop (with/without AT) users?

<patrick_h_lauke> as in these cases even with AT, key events are sent, and those are listened to (using example of custom slider)

James: On the desktop, there are other keyboard commands to increment and decrement sliders

<patrick_h_lauke> james: but on mobile + AT

Domenic: Make it as ergonomic as possible

James: @@ doesn't work in Web Components, WebGL

Doug: The popular ones will be the performing ones

frremy (MS): One propsal I see, in PHase 3. The Accessibility Tree is not independent of the DOM tree, it is produced by the DOM tree. WHen you change the root, it has to be regenerated.

scribe: you cannot decide the tree will be generated by the browser. It is something at risk

James: Here's the current thinking. One of the things we would like (but may not get) -- example of large data set of music, we want to say there if there were virtual nodes, it would be on the subDOM of a specific node.
... it would come along with the DOM node, but would not mix children into the same parent.

Francois: that doesn't address @@@@

Patrick: 1) excited about accessible action events - in IRC I put a link to documentation of the ARIA patterns and where they fail with touch events

<frremy> @@@@/the case where you want to have list-items directly as children of a list

James: Indie UI would have been nice

Patrick: Asking user permission and the comparison with Geolocation. With Geolocation, hopefully it will still work. If the user can say no with custom widgets, then they can't use the site. It's like blackmail, if you tell us you are an AT user, then it can work.

James: I want developers to understand more about the assistive technologies
... the understanding may allow us to make better generalized APIs in the future

Rich: Are you expecting to have more accessible nodes?

James: There could be, the way the tree is made right now.

Rich: ARe you planning to build another sub-tree? is there a way to make it more simple

James: a concern of all the implementers is performance. accessibleNode is a property of element, and you are just reading/writing to it.

Alice: If kyou set a style and give invalid properties and it will be ignored. This is the same, if you give it invalid properties, it will be ignored.

<patrick_h_lauke> bikeshedUI

Slide text content follows

<jcraig> Accessibility Object Model

<jcraig> James Craig Apple Accessibility

<jcraig> Alice Boxhall Google Chrome Accessibility

<jcraig> Dominic Mazzoni Google Chrome Accessibility

<jcraig> Alex Surkov Mozilla Accessibility

<jcraig> Agenda

<jcraig> • History: previous proposals and vendor discussions

<jcraig> • Common problems for noticed by browser vendors

<jcraig> • Example problems and code samples

<jcraig> • Considerations for scoping

<jcraig> • Audience for this technology

<jcraig> • Tightly-scoped, phased development plan

<jcraig> • Q &A

<jcraig> History: Previous Attempts

<jcraig> • Intentional Events

<jcraig> • Canvas Sub-DOM: technically possible, but prohibitively tedious and expensive • Apple: UI Independence (spawned IndieUI) and Accessible RTE proposals

<jcraig> • Google chrome.automation Extension API

<jcraig> • Mozilla: A11y API proposal

<jcraig> • Microsoft: WAPA proposal

<jcraig> • 2016: Offline vendor discussions leading to WICG incubation AOM effort

<jcraig> Common Problems

<jcraig> • DOM manipulation requirement of ARIA can perform poorly with large datasets

<jcraig> • Reliance on in-document IDREF prevents some usage (e.g. Shadow DOM)

<jcraig> • No way to receive accessibility events (e.g. increment a custom slider)

<jcraig> • Some complex office suites (Google Docs, iWork for iCloud, etc) are challenging to make accessible through standard techniques: POSH+ARIA doesn't cut it.

<jcraig> • Bonus: Accessibility Inspection and Testing

<jcraig> Code Samples

<jcraig> el.accessibleNode.labelledBy = [el1];

<jcraig> const input = comboBox.shadowRoot.querySelector("input");

<jcraig> const optionList = comboBox.querySelector("custom-optionlist");

<jcraig> input.accessibleNode.activeDescendant = optionList;

<jcraig> Considerations for Scoping

<jcraig> • Must be readily implementable

<jcraig> • Must not expose user details (privacy)

<jcraig> • Must not negatively affect performance • Must be easy to author *

<jcraig> Audience for AOM

<jcraig> s/@@ doesn't work in Web Components/IDREF doesn't work across shadow root boundaries in Web Components/

- DRAFT -

Accessibility Object Model

21 Sep 2016

Attendees

Contents

History

Common Problems

Example

Audience for AOM

Tightly Scoped Developments

Slide text content follows

Summary of Action Items

Summary of Resolutions

Scribe.perl diagnostic output