17:03:33 <RRSAgent> RRSAgent has joined #aria-at
17:03:38 <RRSAgent> logging to https://www.w3.org/2023/06/07-aria-at-irc
17:03:38 <Zakim> RRSAgent, make logs Public
17:03:39 <Zakim> please title this meeting ("meeting: ..."), jugglinmike
17:03:47 <jugglinmike> meeting:  ARIA and Assistive Technologies Community Group Weekly Teleconference
17:03:58 <Sam_Shaw> Sam_Shaw has joined #aria-at
17:04:14 <Sam_Shaw> present+
17:04:51 <jugglinmike> present+ jugglinmike
17:04:59 <jugglinmike> scribe+ jugglinmike
17:06:01 <jugglinmike> agenda: https://www.w3.org/events/meetings/ac8829ad-96bc-400b-b46b-81a62c6b12a1/20230607T100000
17:07:03 <jugglinmike> Topic: Update on test plan execution work
17:07:29 <jugglinmike> Matt_King: The primary goal is to make sure that there aren't any problems, and to make assignments if any volunteers have capacity
17:07:58 <jugglinmike> https://aria-at.w3.org/test-queue
17:08:32 <jugglinmike> James_Scholes: Color viewer slider with JAWS and Firefox - all testing is complete, but there are 7 conflicts
17:09:24 <jugglinmike> James_Scholes: Isa has been managing this work, but since she's not here today, I can't speak to the status of the conflicts
17:09:39 <jugglinmike> Matt_King: The site is loading very fast, by the way. Thanks to Alex and Paul from Bocoup!
17:10:09 <jugglinmike> James_Scholes: VoiceOver for macOS and Chrome - all testing is complete, but there are 4 conflicts
17:10:24 <mmoss> mmoss has joined #aria-at
17:11:50 <jugglinmike> James_Scholes: JAWS and Firefox - Navigation Menu Button - has been assigned to Alyssa, but not started yet
17:12:17 <jugglinmike> James_Scholes: With NVDA, testing is complete, but there are 7 conflicts. That seems quite high, so we'll look into that
17:12:47 <jugglinmike> James_Scholes: VoiceOver for macOS and Chrome, awaiting testing from Dylan
17:13:55 <jugglinmike> James_Scholes: Modal Dialog Example - JAWS and Firefox - testing is almost complete, but Dylan has only reported results for 17 of 18 tests, so I suspect that's a problem with the UI
17:14:23 <jugglinmike> James_Scholes: NVDA and Firefox - isn't present, so I presume we've published this already
17:14:59 <jugglinmike> James_Scholes: VoiceOver for macOS and Chrome - awaiting testing from Dylan
17:16:17 <jugglinmike> present+ Alyssa
17:16:51 <jugglinmike> Alyssa: I should have the Navigation Menu button with JAWS and Firefox tests done by next week
17:17:45 <jugglinmike> Matt_King: I think that's as far as we can go with this today
17:18:07 <jugglinmike> Matt_King: By the time we're ready to start the next Test Plan, we'll have even more available
17:18:22 <jugglinmike> present+ Justin_Yarbrough
17:18:44 <jugglinmike> Justin_Yarbrough: yeah, that's me! I'll be working with Sam_Shaw to get onboarded
17:18:47 <jugglinmike> present+ mmoss
17:18:50 <jugglinmike> mmoss: Me too
17:19:04 <jugglinmike> Topic: Improving new member orientation
17:19:34 <jugglinmike> Matt_King: Last week, we talked about wiki updates that I made, and we pointed out some obvious gaps and some not-so-obvious (to me) gaps, as well
17:20:09 <jugglinmike> Matt_King: I don't know if we have an issue to track that work.
17:20:28 <jugglinmike> Matt_King: I'd like to discuss next steps and figure out who is available to do what
17:23:05 <jugglinmike> Matt_King: gh-420 seems relevant https://github.com/w3c/aria-at/issues/420
17:23:51 <jugglinmike> present+ James_Scholes
17:24:51 <jugglinmike> James_Scholes: Assign it to Isa; their GitHub handle IsaDC
17:26:02 <jugglinmike> Matt_King: I have a "new member welcome kit" on the wiki. It has some links that are missing, but I'm wondering if this group here could advise on the most important next step
17:26:18 <Sam_Shaw> https://github.com/w3c/aria-at/wiki/New-Member-Welcome-Kit
17:27:24 <jugglinmike> [Matt_King reviews the contents of the page]
17:27:51 <jugglinmike> Matt_King: I'm a little worried about the currency of the page about running test plans
17:28:11 <jugglinmike> James_Scholes: Yes, I agree that we need to review that page for accuracy
17:28:39 <jugglinmike> Matt_King: I'll make a comment about that in gh-420
17:30:08 <jugglinmike> Matt_King: new members: are there other things that have been hard to learn? Or other roadblocks that you ran into? Anything that's missing from this list?
17:31:12 <jugglinmike> Alyssa: One thing that confused me when I first started running tests is the terms for "focus mode" and "interaction mode", or "reading mode" and "browse mode". Because the names are different in NVDA and JAWS
17:31:32 <jugglinmike> James_Scholes: I think that's documented, but the documentation isn't referenced in this welcome kit
17:32:19 <jugglinmike> mmoss: Also, making a stronger distinction about the use of the term "mode" - whether it's the "Working Mode" or the "screen reader mode"
17:33:04 <jugglinmike> Matt_King: Apple insists that VoiceOver is "modeless". They have quicknav, but they don't recognize it as a mode.
17:33:49 <jugglinmike> Matt_King: Nonetheless, I wonder if the test instructions should tell testers to, e.g. "turn on quicknav", where appropriate
17:35:27 <jugglinmike> Matt_King: This is a bit of a can of worms. I've been thinking. Right now, for we say things like "navigate in reading mode" and "navigate in interaction mode".
17:35:38 <jugglinmike> Matt_King: What if, instead, we used different verbs?
17:36:15 <jugglinmike> Matt_King: e.g. what if we say, "focus on" or "move focus to" (for "navigate in interaction mode"), and "browse to" (for "navigate in browse mode")?
17:36:38 <jugglinmike> James_Scholes: The majority of the reading mode tests that have, say, navigate forward to a button, include the Tab key
17:37:41 <jugglinmike> Matt_King: Maybe we say, "navigate" and "browse"
17:38:06 <jongund> jongund has joined #aria-at
17:38:06 <jugglinmike> Matt_King: Basically, using "browsing" for focus mode and something else for forms mode
17:38:40 <jugglinmike> Matt_King: If we do that, then we can use the same instructions for JAWS, NVDA, and VoiceOver
17:39:44 <jugglinmike> Matt_King: Because Testers using VoiceOver would interpret that instruction as a signal about whether or not to have "quick nav" enabled
17:40:23 <jugglinmike> James_Scholes: I'm not opposed, but I think we need to consider this in the context of specific tests because there always seems to be tests which don't fit the model
17:40:40 <jugglinmike> James_Scholes: As long as we can find words which don't inadvertently imply something we don't mean
17:41:30 <jugglinmike> Matt_King: I really like "browse to" for reading mode because it works so well in English. (NVDA happens to use the same term, but I don't mean to give that AT preferential treatment.
17:41:43 <jugglinmike> Matt_King: It's just an appropriate word which works as both a verb and a noun
17:42:46 <jugglinmike> s/noun/adjective/
17:43:42 <jugglinmike> Alyssa: People will come to the project with their own perspective, so I think that any words you choose will require some education
17:44:06 <jongund> jongund has joined #aria-at
17:48:32 <jugglinmike> Matt_King: It sounds like there's still apetite for refining our approach here, and that's a good thing for me to know
17:49:42 <jugglinmike> James_Scholes: I think the change to assertion verdicts that we discussed during the automation meeting on 2023-06-05 is potentially more impactful. I'd like to prioritize discussion and design on that, if possible
17:49:50 <jugglinmike> Topic: Plans for updating app
17:50:02 <jugglinmike> Matt_King: There are a lot of changes to the app being planned in the background
17:50:14 <jugglinmike> Matt_King: They will not affect how tests are run, generally speaking
17:51:00 <jugglinmike> Matt_King: right now, we have an early version of a page for managing the test plans. It's incomplete at the moment
17:51:59 <jugglinmike> Matt_King: When we're running tests, part of the process is to get feedback from the testers (is a command missing or incorrect? is a whole test missing that you think should be there? etc.)
17:52:11 <jugglinmike> Matt_King: In draft review, we could end up making changes to the test plan
17:52:23 <jugglinmike> Matt_King: That's a cue to James_Scholes to make a modification
17:52:43 <jugglinmike> Matt_King: Then, we'll have two versions of the test plan: the one that you ran, and the one that includes the changes that you requested
17:53:07 <jugglinmike> Matt_King: We're annotating Test Plans with dates, so it's possible to have multiple versions of the same plan going through the process at the same time
17:53:38 <jugglinmike> Matt_King: What we don't have in the app right now is the ability to manage these different versions
17:54:18 <jugglinmike> Matt_King: There's work going on right now; the whole team at Bocoup and PAC is working to make changes to the app in order to bring clarity to the work of managing multiple versions of a test plan
17:54:45 <jugglinmike> Matt_King: Meanwhile, while we've been working on automation, we've started to reconsider the verdicts of assertions
17:55:18 <jugglinmike> Matt_King: following a proposal for wording change from jugglinmike, we discussed whether we can/should simplify the states of an assertion
17:55:47 <jugglinmike> Matt_King: Right now, an assertion can be "supported", "not supported", or "incorrectly supported". We're thinking about reducing the results down to "supported" or "not supported"
17:56:42 <jugglinmike> Matt_King: One proposal is that the possible assertion verdicts are: "acceptable", "contradictory" and "omitted".
17:57:22 <jugglinmike> Matt_King: This would help with the kinds of conflicts where Testers have reported both "incorrect output" and "no output"
17:58:28 <jugglinmike> James_Scholes: When we were testing the "Switch" role, NVDA announced a checkbox. It's not so clear if that's "no output" or "incorrect output"
17:59:20 <jugglinmike> Matt_King: Or when VoiceOver said simply "toggle button" instead of "toggle button not pressed". Is that "no output" or "incorrect output"?
18:01:15 <jugglinmike> James_Scholes: neither the three-value or two-value design really provide enough information to understand the cause of failures. In either case, you need to read the AT response to understand what went wrong
18:01:38 <jugglinmike> mmoss: You've convinced me!
18:02:46 <jugglinmike> Alyssa: Having just two options like "pass"/"fail" would make using the radio buttons easier as a Tester.
18:02:54 <jugglinmike> Alyssa: There have been times where I've selected the wrong button and accidentally created conflicts
18:03:24 <jugglinmike> James_Scholes: let's plan to revisit this next week (we sometimes get more participants on Thursdays than we do on Wednesdays)
18:03:45 <jugglinmike> James_Scholes: Though there are already some strong arguments in favor of making this simplification
18:04:16 <jugglinmike> Zakim, end the meeting
18:04:16 <Zakim> As of this point the attendees have been Sam_Shaw, jugglinmike, Alyssa, Justin_Yarbrough, mmoss, James_Scholes
18:04:18 <Zakim> RRSAgent, please draft minutes
18:04:20 <RRSAgent> I have made the request to generate https://www.w3.org/2023/06/07-aria-at-minutes.html Zakim
18:04:27 <Zakim> I am happy to have been of service, jugglinmike; please remember to excuse RRSAgent.  Goodbye
18:04:27 <Zakim> Zakim has left #aria-at
18:09:53 <jugglinmike> jugglinmike has left #aria-at
18:11:55 <jongund> jongund has joined #aria-at
19:18:03 <jongund> jongund has joined #aria-at
21:03:49 <jongund> jongund has joined #aria-at
22:10:14 <jongund> jongund has joined #aria-at
23:32:04 <jongund> jongund has joined #aria-at