Meeting minutes
Review Agenda and Next Meeting Dates
<Matt_King> View agenda at https://
<Matt_King> Next CG Meeting: July 27
<Matt_King> Next automation meeting: July 31
Navigation menu testing
Matt_King: NVDA, when you navigate by list item, it'll even read a nested list. That's surprising to me!
James_Scholes: I just tested on Wikipedia, and it does the same there
Matt_King: If they want to do that, that's their prerogative. They aren't mis-speaking, they aren't leaving anything out, either
mmoss: No, definitely not!
Matt_King: It doesn't seem incorrect to me, though
Isa_DC: I will change the results for test number 5
James_Scholes: The next conflict has the same root cause
Isa_DC: Then I'll resolve it in the same way
James_Scholes: Okay, the next conflict is Test 33, command "Up arrow"
James_Scholes: the AT responses match, but Murray reported "no output", while Isa_DC reported "correct output"
mmoss: I was confused because it didn't convey the list boundary in the same way from when you exit at the end of the list
James_Scholes: Got it. We think this is actually acceptable, so I'll edit the verdict to match Isa_DC
James_Scholes: The next four conflicts are fundamentally the same, just with a different command
James_Scholes: ...so I'm going to update mmoss's verdicts in the same way
James_Scholes: This test has assertions for both "list boundary" and the list role. Do we need both?
Isa_DC: I think we should keep the assertion about the list role and remove the assertion about the list boundary
mmoss: I agree
Matt_King: Vispero has already expressed some concerns about this Test Plan, so we'll probably have to make a new version
Matt_King: But everything is testing on this plan, and there are no longer any assertions, so I think we can promote it to Candidate. We should just expect to make that new version, soon
mmoss: I volunteer to test Disclosure navigation menu for VoiceOver and Safari
VoiceOver support for binary states
Matt_King: James_Scholes and Sam_Shaw and I had a conversation about how we want to negotiate with Apple around how binary "off" states are spoken
Matt_King: We thought the first best step to take would be to look across a variety of elements that have binary states associated with them and look at how VoiceOver treats them, both in native Mac apps and on the web
James_Scholes: Here's my research so far https://
James_Scholes: macOS toggle button--I could not find a native macOS toggle button
James_Scholes: There were some things reported as toggle buttons, like in the TV app, but that was only in their label
James_Scholes: I found others that actually behaved more like radio buttons
James_Scholes: That is: you couldn't turn them off by pressing them.
Matt_King: So both of those are situations which probably shouldn't be described as toggle buttons at all
James_Scholes: That's right. That's why I didn't include them
James_Scholes: I also don't have any examples of an iOS disclosure
James_Scholes: They definitely exist, but I need to do more searching
James_Scholes: The Mac is pretty consistent in that a lot of the time, it either has an "off" state explicitly, or even if it doesn't, it re-announces the control once you've actually activated.
James_Scholes: I believe the only control that exhibits the silence which precipitated this research is the toggle button
James_Scholes: I didn't test the unchecking of radio buttons since that would go against the way radio buttons are supposed to be used (even though you can do that with some radio buttons on the web)
Matt_King: Can you attach this report to a GitHub Issue? From there, we can talk a bit about how we want to discuss this with James Craig
Matt_King: It does seem that, at least for default verbosity, it could be a reasonable expectation that "off" states are spoken when reading
Matt_King: e.g. "not checked", "not pressed", "not selected"....
Matt_King: Okay. This is great--thank you, James_Scholes!
James_Scholes: I'll fill in the rest of the results as I can, and I'll write up a summary in a GitHub Issue
Testing of mode switching
github: w3c/
GitHub issue title: "Initial Mode Switching Exploration"
Matt_King: Ultimately, what we want to come out of this with is an approach to how we're going to test mode switching with JAWS and NVDA
Matt_King: One question: Are they separate tests?
Matt_King: And how do we build this into the plan?
James_Scholes: This is quite long and raised questions about mode switching
James_Scholes: If we have a test with, say, 5 commands, and we only want to include mode switching for one command, we can't do that without expanding the test into multiple tests
James_Scholes: That raises concerns about the number of tests in a plan. If we were to add every new test that I proposed (which is far from a given), we'd be growing already-large Test Plan (from 76 tests to 90 tests)
James_Scholes: And that makes it even more onerous for testers
James_Scholes: There are still more questions. What is the scope of the mode switching functionality that we want to test?
James_Scholes: Do we want to test only that mode switching occurs when we expect it to? Or do we also want to test that an unexpected mode switch does not occur?
Matt_King: We could add a mode switch to the list of "undesirable behaviors"
Matt_King: That way, we wouldn't have to constantly assert that it does not happen
Matt_King: But it does feel weird to have an assertion like, "the mode did not change"
James_Scholes: We're testing with defaults, but we feel that having the screen reader enter interaction mode in some cases could actually be quite harmful to screen reader users because it encourages accidents
Matt_King: I think if we told them to not change the mode when you press the "down arrow" key, that might be going a bit beyond the scope of ARIA-AT
James_Scholes: I'm not sure that there's a significant difference between asserting that something should happen and that something should NOT happen
Matt_King: I've thought of the distinction being this: if you're a web developer, and you've coded something in a particular way using ARIA, what you need is a predictable experience. I guess that kind of moves into the usability space
Matt_King: If people learn that there's some reading you can do that isn't reversible...
Matt_King: Well, the more I think about this, I'm not so sure.
James_Scholes: For me, it doesn't feel like we're crossing any line in the tests to require that the mode must be retained
James_Scholes: Let's say you manually switch to interaction mode, then you tab into this combobox, and then you leave the combobox. In that case, you remain in interaction mode
James_Scholes: I think asserting against *That* behavior would be going too far
Matt_King: I think that the logical next step is to explore what would be the consequences and requirements (both at the test-writing level and the system level) to be able to add those two capabilities into the Test Plans
Matt_King: I made a note to myself to add an issue and put it on next week's agenda
Matt_King: I think you (James_Scholes) should go ahead with exploring it on the test-writing side
Matt_King: But the work may go beyond what we can/should manage with a single issue
James_Scholes: I'll think about this, we'll talk about it, and we'll talk about it more
Matt_King: It may mean the abandonment of the CSV format
Matt_King: We put off developing a tool for test composition in favor of just using Excel. We may need to revisit that decision
James_Scholes: Perhaps