Meeting minutes
Review agenda and next meeting dates
Matt_King: Next meeting: January 31
[no requests for additional topics]
Update from Vispero collaboration
Matt_King: I met with Vispero yesterday
Matt_King: They are engaging with us on implementing the AT Driver protocol in JAWS
Matt_King: What that means for us here in this project is that once they have completed that work (we don't know how long it will take), we will be able to run the automated results collection for JAWS as well as NVDA
Matt_King: That means collecting the JAWS speech automatically, and if the tests have already been run, then the system will compare the responses to the prior responses. If they match, then it will also re-assign the previously-reported verdicts
Matt_King: Separately, the refactored test plans entered the "candidate review" phase
Matt_King: Things are moving forward, and I'll be meeting with them again in two weeks
Hadi: Are they working on a scripting that collects the JAWS feedback and puts it in the form where we testers enter information
Matt_King: Actually, we've already built something along those lines. It's been built to integrate with any screen reader which implements the "AT Driver" protocol (which we are also developing)
Matt_King: Right now, the system supports NVDA. Bocoup is working on a project to integrate VoiceOver, and Vispero will be working on changing JAWS so it can be "slotted in" as well
Toggle button test issues
Issue 1031: Toggle Button Test plan, V23.12.14: Shouldn't role be 'Toggle Button' instead of 'button'
github: w3c/
Matt_King: This is a question about the "role" assertion
Matt_King: Right now the toggle button plan says that "role 'button'" is conveyed
Matt_King: Joe marked that as "passing," but when I looked at both JAWS and NVDA, the role that's coming across is "toggle button"
Matt_King: I would consider it a failure if I came to a toggle button and the screen reader just said "button"
Joe_Humbert: Apple doesn't have a non-selected state. I passed it because it said "button" and when it was selected, it had both a role and a state
Matt_King: I think this is a problem with a test. I think the assertion should be "toggle button"
Alyssa: but if it said "button pressed" or "button not pressed", then you could assume it was a toggle button
Alyssa: is the average consumer going to care if if it says "toggle button" as long as it also says the state?
Matt_King: The problem is that VoiceOver doesn't speak the "not pressed" state, so for a toggle button which is not pressed, it only says "button"
Matt_King: When I look at the ARIA specification, all of them say that the role that gets sent to the accessibility API is "toggle button" which is distinct from "button"
Joe_Humbert: I just verified: VoiceOver says "toggle button"
Matt_King: But what should we expect the screen reader to do?
Alyssa: It should say "toggle button"
Hadi: my interaction would be very different if I found a "toggle button" rather than a "button"
Joe_Humbert: most buttons conventionally don't have state
Matt_King: When I hear "button not pressed", I think "well, of course, all buttons are not pressed"
Joe_Humbert: There's another conversation about the difference between a button and a key
Joe_Humbert: A lot of buttons don't "bounce back." Keys, on the other hand...
Joe_Humbert: Also, keep in mind that iOS doesn't have radio buttons
Joe_Humbert: It sounds like we're all in agreement that the test needs to change. The expected role should be "toggle button", not "button"
Matt_King: I think the test should be about what the screen reader is supposed to do. It should communicate the role in the Core-AAM, which in this case is "toggle button"
Matt_King: I think this just hasn't been documented properly because there are only two roles in all of ARIA which are like this: "menu button" and "toggle button"
Matt_King: I'll assign this issue to myself, and I'll update the test (that means we'll have to re-run the tests)
Feedback: "Navigate backwards to a pressed toggle button" (Toggle Button, Test 12, V23.12.14)
github: w3c/
Matt_King: This is the one that's related to "shift J" when quicknav is on and focus didn't move
Joe_Humbert: Well, focus DID move when I pressed "shift J"--it just didn't move to the intended place
Matt_King: Did you check whether or not there was unexpected behavior?
Joe_Humbert: When I first went through the test, I passed one of the assertions (just one)--the assertion that said it was a button. That's because I did land on the button
Joe_Humbert: But it was the WRONG button, so I didn't know if it really passed
Joe_Humbert: The system considered it a "pass" when I passed two and failed one
Joe_Humbert: This happened when I looked at the conflicts. So maybe the wording for conflicts is specifically confusing
Joe_Humbert: The other part of that same issue is: the assertion says that focus lands on a button. Focus landed on a button, but it was the wrong button. Is that a failure?
[group reviews the test and the AT response]
Matt_King: It IS a little confusing
Matt_King: Maybe the "role" assertion should have passed
Matt_King: Are you suggesting that we should make a change to the test?
Joe_Humbert: I'm wondering if the tester knows the behavior is incorrect, but the assertion is satisfied, should the tester makes the assertion as "passing"?
Matt_King: I think it should pass
Joe_Humbert: Keep in mind that this is all dependent on how the test was constructed. If there happened to be some other element in that incorrect destination, then the "button" role would not have been spoken, and the assertion would have clearly failed
Matt_King: So it's clear that "B" is missing from the "button" test plan
Matt_King: Two things need to happen. I'm adding them to the issue itself as a to-do list
Matt_King: I'll assign this to me for now
Joe_Humbert: I'm glad I asked!
Feedback: "Navigate backwards to a pressed toggle button" (Toggle Button, Test 12, V23.12.14)
github: w3c/
Matt_King: The assertion states, "the 'pressed' state is conveyed"
Matt_King: Our guidelines say that we don't expect specific words
Joe_Humbert: I was mostly wondering about automated testing
Joe_Humbert: Would an automated test accept alternate words like this?
Matt_King: Yes, provided that a human tester had previously reviewed this response and marked it as acceptable
Matt_King: The automated system will only assign verdicts if it can match AT responses to some "historic" response to which a human has already assigned verdicts