Meeting minutes
VY: we’ll start with introductions
please mention your name, pronouns, where you work, and what you’ve been passionate about lately
Valerie Young (Val), she/her
James Nurthen, he/him
Lucas Radaelli, he/him
Giacomo, he/him
Rémi Bétin, he/him
Shirisha Gubba (Siri), she/her
Zoë Bijl, she/her
James Teh (Jamie), he/him
James Craig, he/him
??, he/him
?? Rogers, he/him
Yamaha?
???, he/him
Arielle Gilmore (Ari), she/her
Jane Fulton, she/her
Daniel, he/him
Alan Page, he/him
Matt King, he/him
Sarah Higley, she/her
*fixing meeting settings*
Chris ??, he/him
Mike ??, he/him
Jacques Newman, he/him
Rahim, he/him
Joey Artner
Vladimir Levin (Vlad), he/him
OK, let’s go through the details for the week
at 9:30 we have our first session
Back to the day…
<jcraig> https://
we have a full agenda today
<Jamie> https://
we have an open half hour tomorrow
if there are breakout sessions that could be interesting for ARIA to join, feel free to edit the wiki
thursday there’s a Joint meeting with APA Joint meeting with APA for cross-spec review
and on friday there’s a Joint meeting with AGWG about how the automation projects may help with testing WCAG techniques
VY: the enter aria-at is a general session
SH: on wednesday at 5:30 there’s a session about IDRef\/CSS selectors
JT: they put all the accessibility ones in the same time slots
JN: there’s a whole list of sessions that day
VY: i’ll keep adding links to the wiki
feel free to add more
JN: mo-tu, th-fr are usually working group meetings and then wednesday are breakout sessions
link to breakout agenda: https://
<jcraig> TODAY AT 5 https://
AccName vs Content
JN: we’ll try and talk about label precedent and all sort of stuff
we have two main issues
first is whether `label` should override and set precedent
second clarify whether should split name from content
i think we probably want to come up with some presedence rules
the current algorithm doesn’t really work for anybody
we have a whole bunch of things where authors think they’re doing the right thing but they’re not
we should look what we can do to clarify this
Steve Faulkner and some other folks at ?? came up with a content model
<jamesn> https://
that’s probably worth a read
there are three different ways forward
1. do nothing
2. create some sort of presedence table
3. some sort of content model table
we’ve talked about coming up with those tables at various times
it could be v useful
but would be a huge lift
certainly something worth doing
for content models, Steve’s blog is probably the easiest way to look at this
those different kind of content models fall into different ???
SH: it looks like it’s focused on elements that make the element not be empty
jcraig: also certain types of content (SVG, USD via <model>) can have their accessibility defined in the remotely loaded "embedded" content... e.g. SVG title element in a remote file's sub-DOM
i’ll try to look it up before tomorrow’s meeting
JN: that’s the problem
looking at these different things
MK: you were saying about different browsers and AT
do we have any info about what they are doing?
in the APG we have a table that lists every role
this is what happens for every role
JN: it might be
to take an example that i think we all agree shouldn’t be named, a paragraph
but you still can
VY: the browser won’t complain
JN: but then a list item can be named and that could be useful
but it generally makes things worse
MK: do we have any instances where some SR+browser combo is overwriting content and others are not
JT: i believe so
<Zakim> Jamie, you wanted to clarify that name and content are not necessarily mutually exclusive, are fundamentally different and can have different requirements depending on type
but don’t have an example top of mind
JT: name is basically a flat string
as it’s currently defined
but content can contain semantics
and other objects
so if you flatten it you get just text
the second part is that they’re not mutually exclusive
an example is landmarks
the name is important
but you don’t want to overwrite the content
AT already know about these
with headings you can argue that the name should overwrite the content
ZB: for the heading example, a common pattern is links in headings, to create a permalink
JT: but in those situations people wouldn’t put a label on the heading
ZB: right
MK: we need to figure out the end goal
before we jump into solving
there’s either intentional masking
or unintential masking
because of how browsers and at work
i wonder if there’s agreement in the room
about intentional overriding
is encessary in cases and something we don’t want to break
JT: you’re saying there are use cases for that overriding?
MK: yes, and they’re supported by the spec
yea, so the ones where the spec currently allows it, and is being used… is there agreement that we don’t want to break that?
JT: i would agree with caviats
MK: it’ll be important to know about caviats
siri: for accordians ??
i don’t feel like heading with label didn’t break accessibility
for a paragraph i can understand
<Zakim> jcraig, you wanted to discuss the follow-on problem... it's not just the agreement of the browser engine... there re different expectations from users of different downstream AT
JC: clarifying question
if i recall correctly paragraph label is now prohibited
siri: yes
JC: what did you mean with the heading?
siri: so we have the heading, then the button inside
JC: this could be specific AAM problems
but the follow on, there are different user expectations based on which AT you use
VoiceOver on Mac might say “Accordian heading with two buttons”
then the user knows where to go from there
go into the heading to find the buttons for example
but the accname spec right now says the name should include the button labels
siri: ???
giacomo-petri: when we ?? the generic roles
at the time we also broke some of the ??
to provide visual presentation
we had normal price vs promotional price
we would use aria-label to indicate the different prices
JC: can you clarify the example?
GP: there was a strike through price and a discount price so the SRs were not adequately conveying the old content was stricken
and the aria-label was placed on the `div`
JN: we’re not too worried about things that never really worked
MK: for most AT label information just disappears in those situations
<Zakim> Jamie, you wanted to note that the spec doesn't actually specify whether names should or shouldn't override at all, plus answer the screen reader question
siri: i think it worked in JAWS at some point
JT: if the div was empty maybe
with the exception of name prohibited
what ATs are having to do is make assumptions
the spec doesn’t say anything about whether label should overwrite content
the answer to the heading questions
currently because headings are generally pretty straight forward in their content
if there’s an explicit label
aria-label for example
then the label overrides the content complely
if the heading contains an itneractive element it doesn’t do that
that kind of solves the problem
but it’s not defined
which is scary
<Zakim> jcraig, you wanted to react to Jamie to mention the logistics of queueing with a prompt
with the content model ???
?????
JN: drag this bag to problems and solutions
sounds to me that Jamie made it clear that label and content are different concepts
we should be clearer in the spec that these are different things
are there cases where label should overwrite content?
JT: yes
JN: do we have use cases for overriding where label overwrites content with focusable children?
JC: yes
there was ??
front-endian-jane: there are some data-viz examples
things that make visual sense and have some semantics
with trying to make it sr compatible
it would be a lot less code to be able to override content explicitly
we usually have to write it multiple times and hack with css
<Zakim> sarah, you wanted to answer use cases for nodes with child accessible nodes
JN: could you not use role=img on it?
front-endian-jane: we don’t necessarily want it to be treated as an image
SH: ???
so you overwrite that wiht an aria-label
another is ??
another is with an AI bot
if you’re arrowing through messages
it would be nice if you could hide the repeat commands (??)
<Zakim> jcraig, you wanted to respond to the charts/dataviz scenario
JC: to respont to the dataviz scenarios
which are kinda leading into aria actions…
there’s a session tomorrow where i talk about the ??
what are we doing with SVG
interactive SVG
relevant to this situation
do we want something like a leaf node
VO on the desktop has this ??
on iOS you can do it via the actions rotor
and it kinda sounds like that’s part of what we’re talking about
but you wouldn’t want that for the main content for a user
<Zakim> Jamie, you wanted to note that these are not override cases
JT: the examples are, as i understand, where you don}t want the label to override?
SH: no i do want the override
JT: when you cursor through the content you don’t want the child nodes to be read
but you want to be able to navigate into it and still read them
so that’s where the definition of override is important
same with list item
do you know how this works with the heading and link role?
various: no
JT: to answer your questions JN, are there any situations where we do want this
MK: the link naming one is a good example
you see a lot of links with an image inside of it
and then an aria-label on the link
that image is like it’s not there
so that’s what i meat with override
sounds like that’s what you meant too JT
JT: correct
Lucas: i had to rewrite parts of the accname stuff for chrome
when you have this list of things that override eachother
it feels like… as a human i’m trying to get it all in my head
it was complicated to get it to work
would the spec say you have a button, and a label
in the chrome implementation the label would override the content of the button
as a SR user you would only get the label
does it make sense to override here and there
if we do this
this list would grow to be unmangable
VY: too many categories…
SH: it’s not fully overriding, but not not fully overriding
with instances, with JAWS, it will duplicate the label and then the content (??)
moving through cells with arrow keys, you don’t want that duplication
JT: i genuinly thought grid and table cells were consistent
SH: with a feed list (articles in a feed)
???
the list should specifiy overrides in certain situations
semi-overridden?
MK: we kinda already have this
it’s not spelled out
JC: what are you thinking of?
MK: well if you look at regions and articles
you have name from content and name from author
this is more a matter of actually specifying of what SR behaviour is in sitations
we don’t want the content to be hidden
but we don’t want it to be read
JN: the content is never the name for a region
MK: no correct
we have name from content
name from author
name from either
or name prohibited
front-endian-jane: what SH said, having more control/specificity of what happens would be good
the authors are going to want to do different things
having a list of what should happen unless otherwise specified would be good
but having more control would be great
JN: so where do we move from here
SH: interactive controls that are also containers
like articles in a feed
and things that are neither of those things
<Zakim> Jamie, you wanted to note that this is an AT spec, the models, how it currently works
a region or a non-interactive list item
JT: i get this is kinda implicit
but explicitly, we are defining an AT spec
we are getting into AT speccing here
i don’t have a problem with that, just want to be clear about
browsers don’t like removing things
that way AT gets more ways to interpret/use the thing
and then how we define it
we have the name
i don’t want to talk about naming…
but we have:
- naming always overrides
- naming for container
- use name when focused
is there a fourth list?
fourth category is: name prohibited
the four categories:
1. name always overrides
2. name of container (append)
3. use name when focused
4. name prohibited
examples
1. button and link
2. landmarks
3. grid cell, list item, focusable article in feed
4. paragraph and generic
JN: ten minutes till break
<Zakim> jcraig, you wanted to discuss the pitfalls of "spec-ing" AT behavior, accidental hiding of content, and the VO web navigation changes about 8 or 10 years ago,
JC: there are some potential pitfalls what people are interpreting speccing AT behaviour
that’s broader than the W3C
?? to avoid accidental masking
this group as it should be is heavily represented by authors
i hinted we probably shouldn’t get into the changes we made to VO
but i think it’s apllicable here
VO on mac was developed for mac apps specifically
WebApps weren’t a big deal when it started
basically container elements had a label
so any container, in say mail, would have a label
and it would force the user to interact with the element
that particular behaviour…
the way people were developing applications at the time
this behaviour became problematic
it could be three quarters of the content on the page
you’d hear the name “group” and be like “oh this could be a massive thing”
we sat down to talk about this at the time
people kinda liked the navigation
the major shift we made was that VO would auto step in and out of things
people know VO+Shift+Up Arrow|Down Arrow to go out and in of containers
you can also do VO+Shift+Left|Right Arrow to skip around the tree
so what i’m hearing is not specifying AT behaviour
it’s ??
i’m always weary of speccing interaction behaviour
but there may be ?? for an aria flag
but maybe there’s a scenario like chart
where as long as it’s clear to users you can interact with it
you can have a override label and then still access (or interact with) the content inside?
maybe a switch user wants the content exposed but an AT user doesn’t(??)
Lucas: one of the ideas that it was going into
if it’s a leaf node
the idea i had
leaf node overrides, known leaf node appends
and a flag to override, append, or prohibit
JT: this isn’t really relative to browser ??
I’d argue the browser is, mostly, already doing what the spec requires
Lucas: i agree
i’m talking about the AT process
JT: ah, yes, understood
JN: not sure where we’re going with this
but it seems we agree this is something to work towards
??: were you suggesting a new feature?
JC: i think there’s an opportunity for it
what i’m weary about , it doesn’t matter how easy a feature is
some authors are going to missuse or overuse this
it’ll only affect their pages luckily
if everything’s a heading, nothing is
so i don’t know, but i think there’s an opportunity for it
VY: suggestion for next steps:
to start, i think it would be nice to have an explainer to the problem
i think we made great strides towards articulating that in this meeting
JT: how often have you heard this be a problem for users?
JN &SH: all the time
JT: ah
it’s like maybe, i’m not clear on where the lack of interop is
and where does it cause the most pain
MK: yea
certain places where browsers do overrides…
authors wanting to get a good experience
but not knowing how to get there
RESOLUTION: Jamie, Sarah, and Zoë will meet to discuss
*break time*
<Rahim> present_
Update on Open UI
<Jacques> reminder that the room is muted
JN: anyone new in the room?
<lucasradaelli> hello world. first message to IRC.
Keith Cirkel, he/him or they/them
??, he/him
Lola, she/her
<jcraig> https://
Penelope (Penny), she\/her
Jan Jaeschke, he/him
JN: Keith, you want to take it over?
KC: I would love to
still doing some login stuff
*elevator music*
<Penny> Penelope (Penny), she/her, PM @ Google for UI Capabilities (drawing pixels!), passionate about cats! 🐈
<jugglinmike> Adam_Page arigilmore re the new WebDriver locator strategy for "accessible name": w3c/
KC: we’re talking about a variety of Open UI features
wanted to give some details on popovers as a feature
`popover` is an attribute that can be added to any element
it adds elements to the top layer
which is a visiual thing
other elements on the top layer are full screen
an example is modal dialogs
?? ui
such as menus and floating
and usually that’s click
but there’s these different modes of popover
because we want to be able to dictate how to layer these UIs
when you use escap or a gesture, it will dismiss the popover
you can nest popovers
there’s also popover=hint
which is like a tooltip
but can also be a hover card
like issues on github or users on social media
popover=manual doesn’t have this light dismiss behaviour
it’s possible to make it work the same though
so the intent is that most of these modes are going to be auto
but they can used interchangably
popover’s have a source
this is usually a button
the popovertarget attribute
and the `commandfor` ??
`interestfor` uses a different model
`interestfor` is a device independent way
the short answer is hover on mouse, focus on keyboard, and context menu for touch
the point of the source is that it allows us to do interesting things with focus
like restore it to the source on close
when the button is focused ??
rather than looking at the next focuxsable thing in the dom
it will look at the ??
it skips the ???
in this way it creates this relationship where it skips focus into the popover
relationships
there are ways to open a popover
there are four actions
`popovertarget`, `commandfor`, `interestfor`, and imperative API
anchors
what’s an anchor?
CSS has created this new way to position elements
traditionalyl you would use `postition` and the inset properties
(top, right, bottom, etc)
this new way uses `anchor-name`
example: `heading { anchor-name: --foo }` `a { position-anchor: --foo }`
<jamesn> https://
<masonf> Also there's this cool tool: https://
the intent of anchors is that they present a ?? for visual design
but they can be decorative
they can just be for decoration
example: a tooltip
???
KC: elements can have implicit anchors
there are three things specified
- popovers with a source
- a `<select>`’s `::picker()` element
- any pseudo element’s “originating element”
MF: is that browser behaviour too?
KC: tbd on some of this
CSS is just vibes
*video of a decorative anchor*
shows a toolbar with icons, and a liquid glass blob moving over the icons as you hover different icons
this is purely decorative
JC: is that using a transform to animate the blob?
KC: yes
<jamesn> qw?
AP: so the source is dynamic?
KC: yes
JC: do you have a link to the demo?
KC: the disucssion around this
anchors are for spatial relationships
do anchors establish aria-details?
JT: maybe?
KC: that’s my opinion, that’s what we should discuss
and which relationship wins?
if you had an element with an anchor and a popover
what wins?
for the aria-details relationship
LR: for aria-details you can have a 1-2 mapping right?
so not understanding the problem
JT: you can do that
but some AT only look at the primary relationship
LR: when i was helping implement the feature
we talked with NVDA and JAWS teams
and how would they feel about having one to many lists
it’s not clear to anybody how aria-details is working in all ATs
i did a quick test among pro users
on how to lists aria-details
and nobody knew
JT: it’s new though
LR: yea
so my question is what is a proper way to solve it?
JT: KC, are you aware that
should anchor make the relationship
because there’s discussion around thatt
exposing the relationship could be useful
but not exposing hides it completely
we still don’t have a good place to document CSS mappings
we need such a document
ZB: isn’t there a CSS-AAM?
JT, MK: it has been proposed a bunch of times
JN: we decided to put parts of CSS-AAM into HTML-AAM for simplicity
SH: i wanted to add that this is about the default mapping
there’s discussion about how and when authors would override
scott proposed ??
what i think makes sense is that when you have ??
that would cause then the default mapping to not be there
so how would the author customise?
KC: it might be strange if users get different experiences
potential ways authors can override the default mappings?
if you provide `aria-details=""` or idl=null would override the default
labelledby, describedby, and owns
SH: iffy on owns
<Rahim> empty string would likely map to null in IDL
MF: popover and select
scribe spectranaut_
keith: the heuristics for whether something is decorate or meaningful is too hard
jamie: what if you can specific specifically
keith: but for pseudo elements, implicit anchor, what would you do? if you made tooltips that used :before element. pseudo elements cannot be controlled/specified role. e.g. ::scroll-marker
keith: (explains implicit anchors like in slides)
sarah: to mason's point about the association should come from popover, only popover is meaningful... if you call "showpopover" and have the anchor in css, that makes it meaningful?
sarah: if anchor is used with a popover, create the association???
keithamus: when do you resolve the anchor.....
keithamus: in firefox we have the implicit anchor, we look up the source
<Zakim> Jamie, you wanted to talk about current implementation in Gecko and Chromium
jamie: if I understand correctly, in chromepopover already creates implicit details, css-achoring, does it already set up description relationship?
answer: yes
jamie: i though we have a set of rules in firefox for what to do
jamie: if we don't have the relationship, users are lost in woods. if you have too many relations, too much verbosity
mattking: cautionary tale: flows-to and flow-from, people tried really hard, nothing actually works
mattking: is this a similar path but with larger consequences
jamie: this is a reason why we can't keep defining stuff like this out-of-band, what are firefox and chrome even doing? it needs to be in the public working group discussions
jamie: we can't have a useful confirmation, we barely know what our own implementations are doing
keith: I'm not sure we are resolve for commandfor and interestfor
lucas: chrome doesn't provide multiple details, I think we should, ATs are handling how to have multiple details relationships. even the 1-1 for details is hard, so keeping door open for multiple
RESOLUTION: we need to document what the browsers are doing, and should do, as spec, to investigate whether it is good or bad, in HTML-AAM (since it's CSS)
mattking: focus on 80% rule
Accessibility Testing: ARIA-AT and AT interop
Accessibility Testing ARIA-AT and AT interop
<sarah> Matt: Let's see if I can be coherent after the last 36 hours
<sarah> Matt_King: Context setting: this image is meant to represent what ARIA-AT people say is the formile(?) journey from the author to the user
<sarah> Matt_King: web, then browser, then OS, then SR
<sarah> Matt_King: after lunch we'll talk about all of testing
<sarah> Matt_King: was just working on these slides in the break. This is supported by the APG???, conformance testing should've been listed on here too, so authoring tools
<sarah> Matt_King
<daniel-mac> scribe+/scribe+
<sarah> Matt_King: next layer is WPT a11y tests, working on those for a long time in the group. Brought us a lot closer to making interop possible
<sarah> Matt_King: mile 3, OS level, Acacia AAM tests, after lunch
<sarah> Matt_King: mile 4, from the OS to the end user, this is where ARIA-AT CG has been occupying for the last 6 years, I think we're getting some place and are about to talk about what we've done
Matt_King: next steps in maturing this platform
Matt_King: in 2018 when we started, there were some places that did SR testing, but everyone was defining their own idea of what success or support looked like. Also different ideas about what you should test and how
Matt_King: so the first few years were just talking about how could we possibly approach SR interop through testing. We've developed an approach and consensus process. Also worked on defining a test structure and format, talking about that in depth this afternoon
Matt_King: that's a pretty robust format, there's room for improvement in scalability
Matt_King: also automatable
Matt_King: we've built a platform I'll show for managing all the tests and consensus process and delivering reports. Integrated in automation for JAWS, NVDA, and VO
Matt_King: that automation is based on a draft spec called AT driver
Matt_King: in something WG
Matt_King: will show you what we've built, but before that, macro-type questions?
Matt_King: this is the top-level reports page, this is dominated by test plans in the ARIA APG, another tab for aria features, another tab for HTML features
Matt_King: summary at the top, you can see we've done 65,000 assertion verdicts across more than 1000 testers, 9000 commands
Matt_King: over a thousand tests
Matt_King: before I dive into the test plans I wanted to give an overview of how the reporting works at an ARIA or HTML feature level, they're the same format
Matt_King: you can click on the % for any SR/Browser combo. There's only three now, JAWS + Chrome, NVDA + Chrome, Safari + macOS
Matt_King: we do have the ability to support Edge now, but we haven't yet added Chrome for mac
Matt_King: when you click on one of the %s, you get all the raw data for this. And the thing that I want to point out here is if you go down to the, there's a summary at the top, there's important info in the summary
Matt_King: when we calculate those %s, those are based on 2 levels of requirements. There are 3 levels of reqs in the system, MUST SHOULD and MAY, priority 1, 2, and 3
Matt_King: when we present high-level %s, those are based on MUST and SHOULD
<ZoeBijl> https://
Matt_King: and in this detailed table, if you look at any one of these tablse, this is a collection of -- I should've chosen a different one
Matt_King: Let's look at aria-activedescendant
Matt_King: I just wanted to show that when you look at a property like activedesc, if this report is based on data...
Matt_King: silence
Matt_King: is there a mistake here?
Matt_King: what happened to, uh oh
Matt_King: I was expecting this to show.. activedesc is used in a whole bunch of examples... this table should have like 100 rows
Matt_King: this table should have a whole bunch of rows and be showing data from many test plans for every single APG pattern that we've tested that uses aria-activedescendant. The live demo went wrong, we've recently made changes to the system
Matt_King: let's try aria-expanded, dammit I clicked the wrong link
Matt_King: we will ignore that for the moment, sorry folks
Matt_King: let's talk about how we make the sausage a little bit
Matt_King: I'm going to go to the data management page. OK, so on this page we have this table that essentially shows a high-level picture of our consensus process, or our whole dev process
Matt_King: there's currently 40 test plans in the system, and, uh
Matt_King: so I just filtered the table by R&D complete. The very first step in the process is to develop a test plan. When a test plan is merged int o the main branch, this platform picks it up and it will show in this filter of R&D Complete
Matt_King: we have... and then, next step is to go into draft revew. This is where the community group does most of its work. We have 5 plans in the draft process. In this phase of the working mode, the CG is essentially vetting the test plan itself. Looking for errors in the content of the plan, and the approach. Are the right things being tested
for this widget, ARIA feature, or HTML feature
Matt_King: what we do during draft review, when it's done it goes into candidate review. The AT devs ask the same type of questions but from the perspective of teh SR developer
Matt_King: when we're done with candidate review and have consensus from at least 2 SR develoeprs, it goes to recommended phase. What that means at this point in time is we'll automatically generate updated reports whenever a new version of a SR is released
jcraig: question: when you have not a passing test but positive review, then it's approved? So even when JAWS and NVDA approve, it'll be approved even if I have active objections from VO at the time?
jcraig: looking at an example that says "unsupported" ("Convey state of the radio button, 'unchecked'") for VO because it doesn't say unchecked but windows SRs do because they're more verbose. But that still goes into recommended even though there aren't passes from all 3?
Matt_King: we're esseentially trying to treat all the tests in the test plan the same way we do recommendations in a spec. The big thing I didn't mention earlier is we're trying to define interop without a spec. So the big diff between aria AT and everything upstream is there's a spec goverining it, but when we get to SRs, there isn't a spec
Matt_King: we're trying to back into the requirements with testability, then getting to a spec. I think it might be possible to back into a spec from the tests somewhere down the road
jcraig: sorry if my first question wasn't clear. First, thank you for making this review interface. As soon as the performance aspects of the system is manageable, it's high on my priorities to go through the tests
jcraig: but for the ones where I raised an issue, would that prevent that from being a rec?
Matt_King: yes
Matt_King: what we want to do is create a conversation there. So in that case, for toggle buttons, we got consensus there, and went back to Vispero and discussed that with them, and they were aligned with making that optional even though they speak some things they don't,
jcraig: in that scenario, I'm not saying it should be optional, I'm saying VO passes already but passes in a different way than Windows SRs do. But let's take that offline
Matt_King: so I have two more things I want to share in the app, regarding what's here and how the platform works
Matt_King: so this was the high level of the working mode. What does an actual test look like? And the work of the community group
Matt_King: the test queue page, this is where all the manual testing happens, this is where the CG is focused. When we approve for draft review, it goes into the test queue. Also talk about automated reports separately
Matt_King: two tabs at the top, one for manual one for automated reports
Matt_King: checkbox is in process right now. You can see, what we have here is a run of checkbox test plans with JAWS and Chrome, one NVDA + Chrome, one Safari + VO
Matt_King: in the CG, we have two people run each test plan for each combo. This could be at least three different people would be required here. Two distinct people for each combination
Matt_King: Normally the way this works is the bot runs the plan to collect the SR output so the person running the test plan doesn't have to manually copy it
Matt_King: open one of these as Joe Humbert
Matt_King: so in this case, the first test, navigate forward to an unchecked checkbox. That is the test. This afternoon we'll talk about what the different tests are, there's a pattern to all of these. In a test, navigate to a checkbox, we have a set of tests for the command X, F, and a series of three down arrws, tab with VC active, and tab with
VPC cursor active
Which each of these commands, we collect the output and you answer the questions that are assertions
Matt_King: show edit results form
Matt_King: here's the output for executing the X command. We have an option to mark it untestable, like X didn't even go, then you can't answer the questions
Matt_King: these are the assertions. Was the role checkbox conveyed, the name, the checkbox is in a list and if the boundary of the list conveyed
Matt_King: And the state is conveyed
Matt_King: and then we have a question of whether or not negative side effects occurred. We have a list of common side effects, some weren't as common as we thought. If they command was untestable we require you to choose one of those
Matt_King: if you check one of those, then you have to say if the impact was moderate or severe, and describe the impact. That's what a test plan looks like
Rahim: I just had a quick question about how we're taking into account forms, focus vs, browse mode, different types of SR interaction to James Craig's point. I guess when mobile SRs are added that may not have different modes. How do the testers document different modes and what the output should be in relation to that
Matt_King: I'll go back into one of the test plans.
jcraig: it's effectively a different test plan, they're all called out as different tests, they're doing well in that regard
Matt_King: for each test, you have a set of commands, and for each command, we specify if any settings are required for that command. So the assumption is every command is executed with default settings
Matt_King: so in this one here, this is JAWS, so it says F is the command, then in parens, it says VPC cursor active
Matt_King: then if you go down you'll see tab, we test it both with virtual cursor, and with PC cursor active
Matt_King: it's possible to specify other settings that have to in place when the command is activated. So far the only setting is just which cursor is active, or in VO whether quicknav is on or off
Matt_King: so candidate review, the SR developers can walk through the tests similar to teh test runner for draft review. Then at the end they have the opportunity to approve the plan, and they can raise issues at any point in the process, and those issues are taken up by the community group
Matt_King: there's 11mins left, I'll stop the demo. This is the essence of the platform, there's a lot more you can explore on your own. There's a lot of... you can dig deep here. Every single test plan... when we do find a problem, what do we do
Matt_King: we develop a new version of the test plan. As it makes its way through, if we found aproblem in draft review, we'll advance the test plan to draft review that deprecates the previous version
<Zakim> ZoeBijl, you wanted to ask about exposing baseline support like caniuse
ZoeBijl: about showing some level of baseline support for features. So on the first tab, the AT interop reports. There's a table with components, aria features, combos tested, support. if I take alert, that scores 100% in all combos tested and can see that in the table. But I can't see how long it's been supported, yesterday or years?
Matt_King: good question, if we have... we have a trend table but not trend charts yet. We have all the data, looking for an engineer who wants to built the trend charts
Matt_King: so we're tracking it by SR version, yes
ZoeBijl: so I can I check if something is supported for 30 months I think is baseline, or 18 months?
Matt_King: that q is really relevant for the earlier topic with HTML features. I'd love to see when we have. tehse new features, what do we want the SR behavior to be? We could write those tests, expose them on this platform, then the trend to support over time would be very relevant
ZoeBijl: should I open an issue?
Matt_King: there is an issue about trend charting
<Zakim> Jamie, you wanted to ask about the different test types on different screen readers
Jamie: wanted to clarify, there are these test, like for JAWS virtual cursor. Guessing there's tests defined separately for each AT? So VO might have 3, NVDA might have 3, but not the same tests?
<ZoeBijl> related github: w3c/
Matt_King: all the same tests, but different commands
Matt_King: will go through detailed structure
Jamie: but way to define that?
Matt_King: yes
lola: to answer Zoe's question as well about trend stuff. Later today, will discuss AT test, the project I'm working on tackles that directly. Also talking with web DX group about integrating that into baseline on MDN too
lola: Matt, was hoping you could give an update on AT driver and how that's going in terms of has it moved in the last 18 months? Are you seeing more use of it now, you mentioned NVDA, you're using automation for NVDA too?
Matt_King: in terms of implementation, we have a wrapper for VO that implements it on mac, NVDA plugin for NVDA and is integrated directly into JAWS core
Matt_King: since sept release this year
Matt_King: so we should have that consistently going forward. Mike is here
Mike: high level abstraction, spec is written in terms of raw keystrokes which presents problems for interop tests, you have to write different keystrokes. So intense, we're prototyping right now, if you look at the spec now it'll look strange, but the idea is this will be extended in the future with stuff like move to next heading in a general
sense. But what we're looking at right now, activate element, using that as a test bed because we want to support use cases where we can't do any sort of raw keyboard interaction
Mike: we want to enable interaction with web driver, so places that explicitly need keyboard interaction, you'd be using a combo of webdriver and ATdriver. One sends keys and respects implementation security driver, and AT driver for interaciton and traversal
jcraig: which is the one you're prototyping? in one implementation?
Mike: just in spec language
Mike: we're prototyping language in the spec, the spec is intended for all of them
siri: do you also track, for example NVDA, it doesn't say selected item but does say unselected. If someone says it is not working, but that's how NVDA works, how do you track that behavior?
siri: also if you find a bug but there's already a bug open, how do you track that?
<daniel-mac> AT Driver spec Editor's Draft
Matt_King: when an assertion fails, we now have the ability to associate a bug so we can track that. I don't know if I understood the first question
siri: if one of the list items in a select box, if you select one thing, it doesn't say selected, it says unselected
jcraig: i can respond, you can raise an issue in ARIA AT to say this expectation is invalid
Rahim: I can chat with Matt ofline
giacomo-petri: it was about the ACT rules, we have an a11y support section, we initially had it with WPT but were missing the AT part, would be interesting to work together
giacomo-petri: if we find something not working, we can reference your testing
<Rahim> I was curious about Matt mention of the tests being "automatable" (I'm assuming it's not just scraping the speech buffer text from NVDA/JAWS?)
Accessibility Testing: all (other) testing initiatives
<ZoeBijl> JC: let’s get started
<ZoeBijl> i’ll introduce myself, and then maybe new people in the room can introduce themselves
<ZoeBijl> with name, pronouns, employer, and something you’ve been passionate about
<ZoeBijl> Mike Jackson, he/him
<ZoeBijl> Jeffrey Yasskin, he/him
<ZoeBijl> Alan Stearns, he/they
<ZoeBijl> this morning we heard a bit about ARIA-AT
<ZoeBijl> AT is assistive technology
<ZoeBijl> the work boque and others are doing
<ZoeBijl> on the end to end process
<ZoeBijl> and some automated means
<ZoeBijl> there’s a fairly robust stack
<ZoeBijl> i’ll talk about the accessibility tech stack for the web
<ZoeBijl> a little overview first
<ZoeBijl> i’ll include things in high level first
<ZoeBijl> then we’ll back up because other memebers have ??
<ZoeBijl> we talked about the web stack…
<ZoeBijl> *zoom issues*
<ZoeBijl> within the web stack
<ZoeBijl> in general
<ZoeBijl> we kind of have to get from the author to the end user
<ZoeBijl> effectively it all turns into html and css
<ZoeBijl> that is then interpreted by browser rendering engines
<ZoeBijl> webkit, blink, gecko, etc
<ZoeBijl> that implementation handles all input and output
<ZoeBijl> things like mouse clicks focus events
<ZoeBijl> but also output such as rendering and the interaction portion
<ZoeBijl> for the web accessibility stack
<ZoeBijl> ??
<ZoeBijl> effectively what it does is build an internal model of what the accessibility tree of this page should look like
<ZoeBijl> in addition engines have knowledge of various platforms
<ZoeBijl> like how does a button map on windows vs on mac etc
<ZoeBijl> and next to that is AT
<ZoeBijl> some dip into the source
<ZoeBijl> but in general they talk to the accessibility tree
<ZoeBijl> what we’re going to talk about today
<ZoeBijl> is the web accessibility automated testing stack
<ZoeBijl> what most people are familiar with is client side validation
<ZoeBijl> like axe-core, powermapper, etc
<ZoeBijl> for the most part these run on the page
<ZoeBijl> in addition to that there are some in browser audits
<ZoeBijl> like, if an image has an alt attribute, what is it’s value
<ZoeBijl> so that’s reaching more into the browser implementation
<ZoeBijl> i think the largest automation platform in the world is WPT
<ZoeBijl> a few years ago we added to this with accessibility portions
<ZoeBijl> Jamie and Raheem are working on proptotypes for this as well
<ZoeBijl> and then Val and others have been working on what they call Acacia
<ZoeBijl> taking expectations from AAMs
<ZoeBijl> then tehre’s some platform specific tests
<ZoeBijl> each of these are specific portions that test different parts of the stack
<ZoeBijl> Matt talked about ARIA-AT before lunch
<ZoeBijl> it’s a combo of manual and automated testing
<ZoeBijl> they’re more detailed in their scope
<ZoeBijl> where as ?? is more end-to-end testing
<ZoeBijl> in addition, and what Lola is going to be talking about
<ZoeBijl> is Accessibility Compatibility Data
<ZoeBijl> taking data from all these different projects
<ZoeBijl> and exposing that in a single spot
<ZoeBijl> the three parts we’ll talk about are:
<ZoeBijl> 1. Web Platform Tests (WPT)
<ZoeBijl> 2. Acacia
<ZoeBijl> 3. Accessibility Compat Data ACD
<ZoeBijl> WPT now contains over 2 million tests
<ZoeBijl> tests different specs
<ZoeBijl> HTML, CSS, various DOM expectations
<ZoeBijl> not an official W3C project but it might as well be
<ZoeBijl> ??
<ZoeBijl> these were the two lowest denominators that we wanted to validate
<ZoeBijl> in 2023 someone suggested an accessibility focus area for WPT
<ZoeBijl> that led to discussions about WebDriver in WPT
<ZoeBijl> we wrote some 1000 tests that first year
<ZoeBijl> and we’ve continued with investigations beyond that
<ZoeBijl> there’s a whole lot more we need to get into
<ZoeBijl> Jamie will talk about some work that we’ve been discussing for years
<ZoeBijl> it’s now in prototype in both Gecko as well as WebKit
<ZoeBijl> JT: so far we can test `role` and `label`
<ZoeBijl> but there’s a lot more in HTML and ARIA we need to test
<ZoeBijl> there are properties like description
<ZoeBijl> but also tree relationships
<ZoeBijl> and also `aria-describedby` etc
<ZoeBijl> none of these are officially testable cross-browser
<ZoeBijl> quick background: there was an Accessibility Object Model (AOM)
<ZoeBijl> it had privacy issues
<ZoeBijl> another issue was that the accessibility tree is only running when AT is running
<ZoeBijl> that’s an unacceptable privacy issue
<ZoeBijl> JC: there are some browser extension that expose themselves as AT
<ZoeBijl> but those are few and far between
<ZoeBijl> so it would be a reasonable guess someone is running AT
<ZoeBijl> JT: right now there are two webdriver end points
<ZoeBijl> there’s computed label, computed role, and those are separate
<ZoeBijl> rather than adding another few thousand of these
<ZoeBijl> a property bag
<ZoeBijl> there are two ways of accessibing the property bag
<Rahim> Here's the WebDriver spec for "Get Computed Label": https://
<ZoeBijl> property bag is a object representation, like in JS
<ZoeBijl> so it’ll have keys and values
<ZoeBijl> JC: it’s a static copy of all the backing properties
<ZoeBijl> LR: ??
<ZoeBijl> JT: the requirement is that they need to be globally unique
<ZoeBijl> LR: ?2
<ZoeBijl> JT: that’s a good question
<ZoeBijl> but i think…
<ZoeBijl> JC: what we’ve done is we’ve agreed upon an approach
<ZoeBijl> there’s an RFC
<ZoeBijl> it’s like spec lite to get to this
<ZoeBijl> once we have two prototypes working we can turn that into an official spec
<ZoeBijl> but it should be globally unique for the context
<ZoeBijl> LR: that would be interesting to discuss
<ZoeBijl> ?? about processies
<ZoeBijl> making it unqiue, in chrome at least, is only in the browser
<ZoeBijl> JT: i have thought on how that’s solvable
<ZoeBijl> in terms of where this is at
<ZoeBijl> there’s a prototype for WebKit and gecko
<ZoeBijl> there are two PRs open against WPT
<ZoeBijl> there’s an RFC
<ZoeBijl> WPT technically requires a WD endpoinmt
<ZoeBijl> there’s debate about whether the RFC is required
<ZoeBijl> that’s something that needs to be resolved
<ZoeBijl> in terms of the implementation
<ZoeBijl> initial points
<ZoeBijl> there was another possible approach discussed to have a dump of the tree
<ZoeBijl> but then maybe the tree isn’t consistent across browsers
<ZoeBijl> JC: we know the tree won’t be consistent across browsers
<ZoeBijl> but giving us what the parent/child is
<ZoeBijl> we can still walk the tree
<ZoeBijl> if there are other key points
<ZoeBijl> JT: one more
<ZoeBijl> i defined in the PR
<ZoeBijl> there are these raw tests
<ZoeBijl> you can pass JSON
<ZoeBijl> JC: we have role, label, we have about 1500 accessibility tests
<ZoeBijl> these are basically run automatically for every major implementation
<ZoeBijl> which is amazing
<ZoeBijl> imagine we got 1500 tests for two properties
<ZoeBijl> there’re a lot more properties, we need a lot more tests
<ZoeBijl> this is going to make the web much more reliable
<ZoeBijl> LR: in terms of different accessibility nodes are backed by different… somethings like dumb node layout, and sometimes, inside Chrome, tehy’re completely invented by us
<ZoeBijl> they will not be consistent
<ZoeBijl> JC: correct
<ZoeBijl> LR: is this something that…
<ZoeBijl> JC: let’s talk at the end
<Zakim> lola, you wanted to ask what the timeline is to land the WPT work jamie & james are doing?
<ZoeBijl> lola: i wanted to know more about the timeline
<ZoeBijl> JC: i would love to have these testable behind a flag by the end of the year
<ZoeBijl> JT: that would be my hope too
<ZoeBijl> VY: WPT tests specs
<ZoeBijl> and we had to add to the spec
<ZoeBijl> what part of the spec are we testing with the property bag?
<ZoeBijl> JC: could be string representation
<ZoeBijl> …
<ZoeBijl> JT: i think mostly the AAMs
<ZoeBijl> JC: we still have to agree on things like what is the actual key for things
<ZoeBijl> JT: i would say all of HTML
<ZoeBijl> or like most of
<ZoeBijl> _not_ all
<ZoeBijl> jarhar: where is the explainer?
<ZoeBijl> JT: i’ll link it in the chat
<ZoeBijl> VY: update on the AAM tests, or Acacia
<ZoeBijl> AAMs are how the browser exposes information to ATs
<ZoeBijl> this is specific to platform APIs
<ZoeBijl> this is unusual for WPT
<ZoeBijl> because they’re noramlly noty platform specific
<jcraig> WPT/WebDriver accessibility extension incubation: WICG/
<jarhar> thanks!
<ZoeBijl> *shows Core-AAM*
<ZoeBijl> https://
<Jamie> Links for exposure of additional accessible properties for WPT. Explainer: WICG/
<ZoeBijl> the basic design for these tests is a testharness.js tests
<ZoeBijl> HTML file with the markup that you’re testing
<ZoeBijl> `<div id="test" role="blockquote">`
<ZoeBijl> then the JS runs the assertions
<ZoeBijl> the GetComputedROle and GetComputedLabel that we just talked about
<ZoeBijl> JT: i called it accessibile node
<ZoeBijl> JC: we talked about GetAccessibleProperties
<ZoeBijl> VY: ??
<ZoeBijl> but in this case the AAM tests on the backend
<ZoeBijl> the receiving of that API call
<ZoeBijl> is python, it queries the accessibility API of the browser
<ZoeBijl> it introduces into WPT executors for different platforms
<ZoeBijl> ??
<ZoeBijl> so last
<ZoeBijl> what i just described all existed last TPAC
<ZoeBijl> basically since then, there was a proof of concept that would run on Linux
<ZoeBijl> and we’ll discuss the test design
<ZoeBijl> not a lot has happened since
<ZoeBijl> at the end of summer we got some funding to work on this
<ZoeBijl> i opened a PR against WPT with these changes
<Jamie> web-platform-tests/
<ZoeBijl> and can succesfully run these tests in the CI
<ZoeBijl> first we want to make sure that the WPT maintainers are ok with these changes
<ZoeBijl> awaiting further review from them
<ZoeBijl> macOS and Windows implementation
<ZoeBijl> the basic infrastructure is there
<ZoeBijl> the big thing about this PR on WPT
<ZoeBijl> the tests only run when you enable the flag `--enable-accessibility-api`
<ZoeBijl> there are some dependencies that will automatically install when you enable the flag. Also enables accessibility in the browser
<ZoeBijl> when this PR is merged the test will fail by default
<ZoeBijl> long term goal: accessibility tests run in full runs of WPT, you don’t need to specify the flag
<ZoeBijl> one of the problems is that the flag turns on accessibility in the browser
<ZoeBijl> this should only happen on accessibility tests
<ZoeBijl> otherwise it slows down other WPT tests
<ZoeBijl> i have a suggested solution for this
<ZoeBijl> LR: where can i read more about this?
<ZoeBijl> VY: ??
<ZoeBijl> MF: there are already meta tags that enable/disable certain features
<ZoeBijl> VY: yea that’s a good point, we could look at that
<ZoeBijl> MF: it’s nice because it’s localt o the test
<ZoeBijl> VY: the other thing is that all tests in this PR are marked tentative
<ZoeBijl> i only want to force these to be run when they’re no longer tentative
<ZoeBijl> right now the linux python dependencies have a system dependencies
<ZoeBijl> WPT will currently crash if that system dependency isn’t there
<ZoeBijl> the PR on WPT is rather big
<ZoeBijl> because it has changes to wptrunner and also tests
<ZoeBijl> we need more approval on those changes
<Zakim> lola, you wanted to ask does this mean that OSs will/should start paying attention to WPT?
<ZoeBijl> lola: thank you James and Jamie for helping me find Because it's in Python, the tests could include Python and thus express any possible API test!
<masonf> s/find ??|/fight zakim
<ZoeBijl> does this mean that OSs will have to pay attention to WPT?
<ZoeBijl> if a test fails in chrome on linux, but passes the windows equivelant
<ZoeBijl> my assumption is that linux hasn’t implemented something
<ZoeBijl> VY: each of the browsers have to maintain an accessibility API for each paltform
<ZoeBijl> JT: it could be an API deficiency but then it’s up to us to take that to the paltforms
<ZoeBijl> VY: we discussed a few different methods last TPAC
<ZoeBijl> JC: let’s get the overview in
<ZoeBijl> VY: that makes sense
<ZoeBijl> the shape of the API is not decided, we can keep talking about it
<ZoeBijl> i really want feedback on this
<ZoeBijl> Jamie has some thoughts on what the test design should look like
<ZoeBijl> because we’re testing the API
<ZoeBijl> maybe the test can just be directly in python
<ZoeBijl> you can write infinite tests
<ZoeBijl> there’s no abstractions
<ZoeBijl> the current PR has a more abstracted approach
<ZoeBijl> if you care about test writing
<ZoeBijl> i would love for you to weigh in
<ZoeBijl> JC: if you’re a WPT contributor, not even specifically accessibility, this is the only platform specific one
<ZoeBijl> so more feedback on this architectual shift would be welcome
<ZoeBijl> JC: next Lola with ACD
<ZoeBijl> lola: i want to talk about a project i’m leading called Accessibility Compat Data
<ZoeBijl> working together with Shelly and others
<ZoeBijl> i gave a presentation on this earlier in the year
<ZoeBijl> we have a more complete overview now
<ZoeBijl> we have 4-5 main goals
<ZoeBijl> 1. ??
<ZoeBijl> 2. integrate that data into existing sources such as MDN and caniuse
<ZoeBijl> 3. to support developers and ?? from the start
<ZoeBijl> in the HTML 2025 survey accessibility was the key thing developers wanted to know more about
<ZoeBijl> so we need to get that data in front of them
<ZoeBijl> ACD is not an auditing tool
<ZoeBijl> it’s not like “we’ve checked all these boxes so it’s accessible”
<ZoeBijl> it’s more to see where gaps in support are and know where you need to pay attention to
<ZoeBijl> it does not define a baseline or mark features as generally available
<ZoeBijl> part of the reason is that we want baseline to be useable
<ZoeBijl> we currently have support from mozilla, and the a11y project, and some other indipendent people and organisations as well
<ZoeBijl> the impact will probably considerable because of the appetite for something like this
<ZoeBijl> the ultimate goal is to be integrated into multiple points of the chanin
<ZoeBijl> APA does horizontal review
<ZoeBijl> if we were part of APA we would have insight into those reviews
<ZoeBijl> while WPT has loads of tests, there are also lots of gaps
<ZoeBijl> i’ve writen a script to do some filtering
<ZoeBijl> but it can be tricky to filter on the user end
<ZoeBijl> there’s no way to add meta data to subtests
<ZoeBijl> for computed role for example
<ZoeBijl> we just want to know about aside, or
<ZoeBijl> i’d like to show a real world example
<ZoeBijl> earlier this year Adrian opened some issues against BCD
<ZoeBijl> Found a bug on MacOS with VoiceOver, wasn't sure if it was Safari or VoiceOver, thought it was Safari but turned out to be VoiceOver
<ZoeBijl> but at the moment, issues like that, there’s nowhere for devs to open issues like that
<ZoeBijl> adrian opened multiple different issues
<ZoeBijl> so that’s the overview for ACD
<ZoeBijl> JC: i’d like to give Mike the last five minutes to talk about the automation portion of AT Driver
<ZoeBijl> M?: we’re as part of the ARIA-AT project
<ZoeBijl> we’re developing a ATDriver protocoal
<ZoeBijl> for automating assistive tech testing
<ZoeBijl> this is a protocol that takes a lot from WebDriver and is included as part of the Browser Testing Tools work
<ZoeBijl> you can see it in action
<ZoeBijl> here it is in Safari with VoiceOver
<ZoeBijl> it’s running a test from ARIA-AT
<ZoeBijl> but it uses the WebSocket Protocol
<ZoeBijl> sending JSON
The future plans are to define a collection of "intents", such as activating elements
There are future plans to implement it for TalkBack on Android and VoiceOver on iOS
Currently there are some assumptions that the AT under test is a screen reader, but the intent is to extend to other ATs
There is an intent to extend beyond the web platform and to the a protocol that can work with non-web applications, and that has implications for the protocol, but that will be considered further down the line
Workshop: Screen reader interop test cases & test plans
<ChrisCuellar> w3c/
<jugglinmike> w3c/
<ChrisCuellar> Workshop doc: https://
<Jacques> Has the Tuesday meeting started yet? Assuming not since there isn't any chatter here.
<Jacques> A couple of us are in the zoom meeting but don't see the room yet.
Jacques: it starts at 9:45
(30 minutes from now)