Meeting minutes
cwilso: <Does introductions>
jcraig: <Shares slides.>
jcraig: Accessibility APIs in the web context are dependent on the browser rendering engine.
… Browsers vend things to the platform API
… People pick up different parts.
… Talking about platform-specific automation
… Not talking about client automation.
… We are talking about layout tests == engine-internal automation.
… Added a bunch of new tests web platform tests to test accessibility across browserrs.
… ~600 new tests.
jcraig: What roles and labels can do.
… Got thousands of tests unblocked from adding some new properties.
… Talking about AOM #197 and #203, to extend testing beyond role and label.
jcraig: We're testing 3 of lots of aria-* properties.
… Want to test all aria-* attributes and native features. e.g. if you have both required= and aria-required=, who wins?
… Accessibility notifications, tree walker, etc
… We need a test-only web API.
… Everyone agrees on these goals.
… (Please speak up if you don't.)
… Need a way to get an accessibility object. Computed Accessible Node. And its attributes.
… Reconcile the backing node with the element.
jcraig: Non-goals:
… Not writable == inspectable only. Can send an event, but not change properties.
… Don't think it needs to be live.
… Property bag as of the time of the request.
… IDs might not be persistent after DOM changes.
… Can't connect the previous accessiblity node with the new one across a DOM change.
… There are known differences between implementation accessibility trees, and we don't think we need to eliminate that to enable testing.
… Not making the testing interface specific to the platform, because we don't see a way to do that in WPT.
… Because we want to write cross-platform tests, and the platform-specific APIs are too different.
… Don't want to automate assistive technologies.
jcraig: Skipping discussion of a WebDriver extension.
<jcraig> WICG/
jcraig: There's been a lot of discussion in the issue; tried to summarize it.
… Chrome devtools have a related use case for accessibility trees to build selectors.
… Tree dump tests.
… Got good feedback from WebDriver group.
… Not unanimous about doing an extension to WebDriver vs in the core spec.
jcraig: Certain aspects work in classic & bidi, but others are easier in bidi.
… I'm going to focus on the core features in classic, and later add to bidi.
… What are the next steps?
jyasskin: thinking about next steps, what do you want next
jcraig: we're doing this in AOM; but AOM became a general discussion group, not just OM.
jcraig: probably next steps is to get these into the WebDriver core spec
<Zakim> tomayac, you wanted to ask about the state of AOM in general
tomayac: A possible solution: I was soft told AOM isn't the future. People who use assistive tech can be detected. But it's also perfect because it does what it says on the tin.
… Do all browser vendors have to agree on that? Where is it going?
jcraig: People ask "whatever happened to AOM?" Most is shipping. We have a roadmap.
<jcraig> https://
jcraig: Most use cases are already shipping.
… Reflecting ARIA attributes moved from incubator to shipping. They're aria properties on elements.
… Led to static element arrays.
… Lets elements to be referenced even if their IDs don't work. e.g. across shadow dom
… That's the first three elements of the explainer. User action events from AT are shipping.
… We'd thought about some new events which led to detectability, but now we're synthesizing events that look the same as standard events.
… Screen reader might have an 'increment' operation, but we synthesize the appropriate keyboard events.
… Same for dismiss.
tomayac: That's more detail than I wanted.
… I was told that v1 is dead-ish because the people it was supposed to address because they'd be detectable. That's concerning because you solved the problem for people and they said it wasn't good.
… Various sources told me.
jcraig: People think of the Virtual Accessiblity Nodes when they hear AOM. For now that's dead. Because of detectability.
… Didn't have a way to do it without making it detectable. There's a potential solution, but in response to some discussion, a variety of people said we never want to be detected. That makes it much more complicated to develop an API.
… Solving detectability is impossible. Every platform outside the web has custom views and relies on detectability.
… No precedent for canvas view in a way that doesn't expose the presence of AT.
… That particular portion is dead in the water.
… That's why we're proposing a test-only interface.
… To increase interoperability.
jcraig: Finishing the list. Full introspection of the AT: we're getting closer, but it's limited to devtools or WebDriver.
<Zakim> jyasskin, you wanted to discuss next steps
jyasskin: thinking about about next steps, seems like you don't need any more incubation, you're ready to go to WebDriver?
jcraig: this particular item, based on the discussion this week, I think that's true.
… AOM still has other topics, so don't close it.
Reillyg: ShapeDetection was originally built by my team in Chrome.
… Marcos Caceres from Apple has expressed some interest, so let's take a look.
… the original point of this API was to expose some underlying capabilites of the platform, so shape and bar code detection etc built in support could be exposed, and done more effectively than a JS implementation.
… Chromium has prototypes of these, but only barcode implementation is shipped on by default.
… There was too much variance in shape detection implementation for us to be confident in shipping.
… wasn't really something we could put on the web yet.
… all of these, barcode included, are still incubations. As Apple has commented in their standards position there seems to be some interest, I'm interested in whether Webkit is interested in implementation and what subset if so.
Shape Detection
Reilly Grant, Chrome
Myles MAxwell, Apple
Thomas Steiner, Google chrome
Matt reynolds, Chrome
Hongchan Choi, Chrome
Christian Liebel, Thinktecture
Jeffrey Yasskin, Google Chrome
Vincent Scheib, Google Chrome
reillyg: what is Apple's position on these?
myles: standards-position is our source of truth. informatively, the growth of LLM make some of this quite interesting.
… if the website wants to have barcodes, or text or shapes, this is the right kind of idea.
reillyg: exactly. The next question is what is our next steps? Do we start to address some of the questions Marcos started drafting in the draft position in WICG, or start a WG effort, or ??
myles: I think there's technical work to be done. As far as procedural next steps, we think this would make sense to do within the Web Machine Learning group. We are not members of that group right now.
jyasskin: scanning the ML charter, i don't think this is covered; we should get started on rechartering if we want to go that way
myles: there are two technical issues off the top of my head to bring up -
… 1) batch processing: turning the chip on and off can be "bad" (costly), so batch processing API would be helpful.
… 2) one of the most common use cases is video, and because this is Promise-based, by the time the promise comes back and the web site reacts to the data, the video has moved on.
… The data has become stale in the asynchronous delay. We should try to solve this.
Repository: wicg/shape-detection-api
myles: weak piece of feedback - we have many detectors on systems, for dozens of shapes; if we're going to have support for three we might ask well add more.
tomayac: why has the video moved on?
myles: because there's no synchronization between the video frames and the detection. It generally works, but there's no solid association. It would be better if we could synch (I think this is possible with WebCodecs).
<reillyg> => WICG/
<reillyg> => WICG/
scheib: I had a hallway conversation with riju at intel, whos working on an frame-to-frame aligning for software pan/tilt/zoom
… would benefit from the same thing
reilly: also related to another thing riju is working on, which is blur reduction. associating with frame
… not sure if there's an OS at the native level
myles: native apps don't get it for free, they have to do some amount of work.
tomayac: see https://
reilly: ideally we could get closer to the hardware, done at a platform level.
jyasskin: seems like someone should write an explainer and prototype for, and demonstrate the difference.
<tomayac> The demo is based on https://
reilly: I filed two issues on this
… where we can collaborate on it.
… I think the other issue Marcos raised in the stndarrds-position was test suite. I apologize for the test suite being Chromium-specific.
myles: Our philosophy is tests should be clearly cross-implementation, and clearly pass/fail.
… the bar shouldn't be high for exactly how to detect things
… that enables competition, and that's good.
reilly: we need to separate conformance testing from the underlying shape detection capability.
… we should try to get yours to work in Chromium-CI
myles: oh no, my picture will be permanently embedded.
… marcos and I had a long discussion about what's the right way to ensure we're representative, e.g. for face detection
jyasskin: what else needs to be covered today?
reilly: I wanted to be clear we touched on the venue; Tess seemed to imply we should work on this somewhere else?
myles: not sure if Apple wants to contribute to the spec.
reilly: it would be helpful to know if Apple wants to work on the spec.
myles: I decline to answer. We are WICG members, so I don't think that's a problem
jyasskin: or it could be in WeBML
reilly: seems like we should propose to WebML that we would like this in their charter
… what Apple might be interested in might be different that that, of course.
myles: we'd like to be compatible if possible, of course.
scheib: we're reluctant to go much further with these APIs without some support from another vendor
… I also don't think we need to propose it to a WG until we have another implementor who wants to work on it.
cwilso: WebML just charted fairly recently, didn't they?
jyasskin: yes.
reilly: I'd want to have support from another implementer before pushing to a WG.
cwilso: WebML charter lasts thru April 2025: that would be the first forcing function to consider unless we wanted to.
jyasskin: we'd want some indication of support in order to push for a recharter.
reilly: I think the prototype I saw was of all three APIs.
… barcode and shape detection is probably good enough to ship; for text, one or two platforms could not do OCR (could only tell you where text was; also, one platform only worked on iso-latin-1 character set
… I think the first of those was Android, the second Windows.
jyasskin: reilly, do you want to talk to the WebML chairs?
reilly: sure. Myles, could you talk to Marcos about completing the standards position?
myles: sure
<tomayac> Shape Detection API as implemented on Safari works on the main thread, but currently not yet in workers (demo: https://
<tomayac> As per @myles this is "for reasons", and a KI. The implementation isn't done.
<gb> @myles
<cwilso> s/Myles Maxwell/Myles Maxfield/