17:59:02 RRSAgent has joined #aria-at 17:59:06 logging to https://www.w3.org/2025/11/05-aria-at-irc 17:59:06 RRSAgent, make logs Public 17:59:07 please title this meeting ("meeting: ..."), Matt_King 17:59:32 MEETING: ARIA and Assistive Technologies Community Group 18:01:38 jugglinmike has joined #aria-at 18:05:38 spectranaut_ has joined #aria-at 18:06:50 Zakim, start the meeting 18:06:50 RRSAgent, make logs Public 18:06:51 please title this meeting ("meeting: ..."), jugglinmike 18:07:12 meeting: ARIA and Assistive Technologies Community Group Weekly Teleconference 18:07:26 present+ jugglinmike 18:07:29 scribe+ jugglinmike 18:07:36 present+ Matt_King 18:07:40 carmen has joined #aria-at 18:07:41 present+ Joe_Humbert 18:07:43 present+ carmen 18:07:53 present+ IsaDC 18:07:56 present+ james 18:08:03 present+ howard-e 18:08:12 topic: Review agenda and next meeting dates 18:08:15 https://github.com/w3c/aria-at/wiki/November-5%2C-2025-Agenda 18:08:20 Matt_King: Requests for changes to agenda? 18:08:43 carmen: I wanted to share an update: the 502 errors should no appear any longer. Please let us know if you experience them! 18:08:57 IsaDC: I have something to add about the quantity spin button 18:09:05 Matt_King: We can discuss that when we review the current status 18:09:23 Matt_King: Next week is TPAC, so this group won't meet. I'm tentatively planning for a meeting two weeks 18:09:32 Matt_King: That will be November 19 at this time 18:10:36 Matt_King: The date for the next automation subgroup meeting is TBD 18:10:49 Topic: Current status 18:11:01 Matt_King: There are a couple of changes from this past week 18:11:26 Matt_King: It was kind of small in some ways, but still a big milestone for the project: we advanced two more test plans to the "candidate" phase 18:11:48 Matt_King: Apple is in the process of reviewing more test plans. They've added three more people in addition to James Craig--they're all in the system, now 18:11:58 Matt_King: We'll be meeting with some of these people in-person next week at TPAC 18:12:17 Matt_King: We hope to move forward by the end of the year, and we'll discuss one piece of feedback today 18:12:59 Matt_King: We moved "tabs with manual activation" forward to candidate, and we're very close on "tabs with automatic activation" 18:13:20 IsaDC: Should we remove the "min" and "max" from the test plans? Because "home" and "end"... 18:13:28 Matt_King: Ah, yes, I responded that in an e-mail 18:13:49 James: This is specifically the tests for jumping to the minimum and maximum value, right? 18:14:01 IsaDC: That's right. Not the tests for the attributes with those names 18:14:38 Matt_King: The spin-button pattern in the APG suggested that you could use "home" and "end" keys to set values, but because it's in an edit field, those keys interfere with the behavior 18:15:01 Matt_King: Yesterday, the APG aligned on removing that guidance, and at the same time, they will make some adjustments on the guidance for "page up" and "page down" 18:15:30 Matt_King: We can just not test "home" and "end" for now. I think that's fine; the test plan will still be valid 18:15:40 IsaDC: As for "page up" and "page down"... 18:15:46 Matt_King: I don't remember if we have a test for those 18:15:52 IsaDC: I'm pretty sure we don't 18:16:08 Matt_King: We do have those tests for the slider incrementing those by 10 steps 18:16:27 Matt_King: We could make the test more generation by just saying something like "multiple steps" instead of "10 steps" 18:16:48 IsaDC: I'm going to replace the tests for "home" and "end" with "page up" and "page down" 18:16:51 Matt_King: That sounds like a good plan 18:17:28 Topic: Running test plan for Tabs with Automatic Activation 18:17:44 Matt_King: Last week, IsaDC reported that Hadi may be updating these things, but that hasn't happened, yet 18:18:03 IsaDC: Right. He didn't reply to my latest e-mail, so I may go in and address them, myself 18:18:20 Matt_King: I may get to it first; I'm going to try to advance this one before next week 18:18:35 Topic: Running tests for Switch Example Using HTML Checkbox Input 18:18:48 Matt_King: Joe_Humbert has completed all of the testing. We're waiting on results from dean and mmoss 18:18:59 IsaDC: Neither dean nor mmoss are present today 18:19:07 Matt_King: Okay, so I guess we can't get an update there 18:19:24 Matt_King: Since there aren't any conflicts right now, there isn't any progress we can make on this today 18:19:30 Topic: Running plan for Checkbox Example (Two State) 18:19:53 Matt_King: We're in a similar state here. All test plans are completed once (by Joe_Humbert), but we need a second test plan run for each of the three screen readers 18:20:08 IsaDC: I can take on JAWS 18:20:14 Matt_King: Okay, that would be helpful 18:20:51 IsaDC: If I have time, I'll try to accommodate one other one. I guess I'll just assign the bot's results to myself in that case 18:21:10 Matt_King: If there's any doubt, then just leave the run assigned to the bot so it's ready for others 18:21:12 IsaDC: Got it 18:21:22 Topic: Updating reports to latest screen reader versions 18:21:42 Joe_Humbert: Do I need to assign myself to this? Some of the tests with VoiceOver look incomplete 18:22:42 Joe_Humbert: Do I need to assign myself to this? 18:22:44 Matt_King: Yes 18:23:01 Joe_Humbert: It looks like there are two that I can do for VoiceOver; I'll get those done by next week 18:23:13 Joe_Humbert: If there is one JAWS test that is more important than the others, I can take that 18:23:44 Matt_King: I would say work through them in the order presented, if possible. I don't have a simple way of filtering just the ones you've worked on 18:24:05 Matt_King: You can do this through the "Manage Bot Run" dialog 18:25:46 Joe_Humbert: I was having trouble assigning multiple bot runs to myself 18:25:53 carmen: I can take an action item to look into this 18:27:15 Topic: Request for change to alert test plan 18:27:27 github: https://github.com/w3c/aria-at/issues/1316 18:28:08 Matt_King: This is feedback from James Craig at Apple 18:28:42 Matt_King: I wasn't able to test this prior to today's meeting, so we'll make the assumption that we can reproduce the behavior that James is reportng 18:29:18 Matt_King: It involves an earcon for the alert role 18:29:40 Matt_King: This is similar to test cases for JAWS and NVDA where we have a mode-switching test, and the mode switch is conveyed only via sound (with the default settings, anyway) 18:30:26 Matt_King: Both JAWS and NVDA have settings to convey that via speech (And thus capture it in a response), but we are not asking the tester to change their configuration for this specific test 18:31:14 Matt_King: We could treat this similar to how we test mode-switching tests. Those are currently not "bot testable" due to the sound. They could be made bot-testable if we changed the default settings. But that is a bit of a gap in our automatic verdict assignment capabilities 18:31:36 Matt_King: We can set aside that issue for now, but I want to focus on how we handle feedback for this one specific test 18:31:53 james: The assertion was classified as "MAY" due to feedback from Vispero and others 18:32:21 james: So, if we say that VoiceOver fails this assertion, that does not actually reduce their score 18:32:35 Matt_King: They are aligned with the assertion priority 18:32:43 james: They are saying that it could be changed... 18:32:57 Matt_King: I didn't read that as suggesting that they want the priority increased in order to approve it 18:34:13 Matt_King: We made a decision that's actually contrary to the intent of the ARIA specification and is more aligned to the real-world practice where "alert" is misused. We hypothesize that misuse is most of the time 18:34:32 Matt_King: It's not a great rationalization, to be honest, but it is a practical one that matches JAWS and NVDA's design 18:35:06 Matt_King: It was intended to do exactly what VoiceOver is doing--to call your attention 18:35:39 James: I feel like that is part of the problem. If a web app wants to draw your attention, it shouldn't be the responsibility of the screen reader. 18:36:15 Matt_King: Yeah, that's true. It could be a browser responsibility. That is, practically speaking, a valid approach and something that I'm almost motivated to raise an ARIA issue for 18:36:44 James: There becomes an issue for me when too much is placed on the screen reader. Then, web authors have a lot less control. And it implies that the only people who would benefit from sounds are screen reader users 18:37:07 james: But if the app implements its own sound, then it doesn't have a way to turn off VoiceOver's sound 18:37:10 Matt_King: Right 18:37:38 Matt_King: Tabling that for now, in the same way that JAWS and NVDA make unique sounds when their mode switches, should we be marking this assertion as "supported"? 18:38:00 james: The wording is out of date with our current practices 18:38:10 Matt_King: agreed, but we can address that separately 18:38:46 Matt_King: The VoiceOver help does include a feature that allows you to hear every sound, and it describes the meaning of the sound. We can validate that it is the appropriate sound 18:39:10 Joe_Humbert: I don't remember hearing a sound. I probably would have left a note regarding a sound 18:39:27 james: They could be alluding to a sound that was not present when the testing was conducted. 18:39:59 james: It's on them to be explicit about what they're pointing out. It's not clear what the sound is and when it was added. 18:40:06 james: We should verify those details 18:40:17 james: Does VoiceOver enable audio ducking by default? 18:40:22 IsaDC: Yes, it does 18:40:31 james: So it could be the case that the sound is subtle, and it was ducked 18:40:50 Joe_Humbert: So they're saying that this is a new sound or that it is a sound that people have to enable specific settings to hear the sound 18:41:12 Joe_Humbert: If that's the case, that seems pretty extreme. If we, as professionals, don't know about this, then I can't say that even power users would know to do that 18:41:49 Matt_King: I'm testing this, now. It does make a sound that I do recognize as distinct from other VoiceOver sounds. I do think that it's making an "alert" sound in this situation 18:42:59 Matt_King: I've heard this sound before. It's different from the sound you get if macOS is prompting you in the background for a password or something. It's definitely more subtle than that one 18:43:17 Matt_King: I am using macOS 15.6.1 and whatever version of Safari came with that (perhaps version 18) 18:43:30 Joe_Humbert: I just did the same thing, and I did not hear anything 18:43:42 Joe_Humbert: And I'm on 15.7.1 18:44:02 Joe_Humbert: When I trigger an alert on the APG example page, I see it visibly open up, but I hear no sound effects, and VoiceOver says nothing 18:44:23 Joe_Humbert: This is the "alert" example 18:44:48 Matt_King: The last time, I used "VO + space-bar" to trigger it 18:44:54 Joe_Humbert: I used "enter" 18:45:21 Matt_King: Relative to the volume of the voice, the sound is not nearly as subtle as the "activation" sound. It's present, but it's underwhelming 18:46:10 Matt_King: I just tried it another way, and it's very consistent for me. Though you have to reload the page if you want to trigger it a second time 18:46:14 james: My goodness 18:46:35 Matt_King: I don't know if that is a Safari-specific thing. You should be able to trigger the alert many times in a row 18:46:53 Matt_King: Doesn't the alert disappear visualy 18:47:39 Matt_King: This isn't the "click", this is a dissonant chord sound 18:48:03 james: The sound I'm hearing is probably the activation sound--two clicks with slightly different pitches 18:48:12 s/james/Joe_Humbert/ 18:48:44 Joe_Humbert: I'm on Safari 26 18:49:31 Matt_King: I want to return to the hypothetical: let's say that everybody was getting the same result that I and James Craig observe. A distinct sound is played. For people who get that experience, should we say that the assertion "may" convey... 18:49:55 james: Do we say that the sound played by NVDA satisfies the assertion? 18:49:57 Matt_King: We do 18:50:17 james: So the answer to me is that, if VoiceOver conveys the role via a sound, then it should pass the test 18:51:12 james: However, I can understand the objection in the group here today. Because it does explicitly mention "inexperienced testers", but the people here today have the most experience, and they are not observing it 18:51:31 Matt_King: Could you comment with your experience and to share your macOS and Safari versions when you do? 18:51:40 IsaDC: I can do that 18:54:04 IsaDC: I can test the braille behavior, as well 18:54:10 Matt_King: That would be great. Thank you 18:55:40 Matt_King: I have another sort of related question for us. If this were to change in the future (and they didn't play the sound), then a bot would not be able to detect that change. I can think of a couple ways of approaching this. In this particular case, since the non-audio case would not be detected by the bot (since the bot doesn't receive braille instructions, either), I think we need a way to designate that some tests always requi 18:55:40 re a human tester 18:55:47 james: I think so, too 18:56:12 james: They are kind of pointing out that we are mainly testing speech, and that the project is not currently taking non-auditory feedback into account 18:57:14 james: There is a world in which the project addresses that in a truly holistic way. That's a huge lift, so I think in the mean time, we could have a flag that designates some tests as being "untestable by a bot"/"always needs human verification" 18:57:29 Matt_King: When we do automated testing, those assertions would need to be left as not set 18:57:37 James: Would they be marked only for VoiceOver, though? 18:58:07 Matt_King: Yes. The flag would need to be set at the command-assertion level. That becomes more difficult--not insurmountable, but a little bit 18:58:26 Matt_King: We could set it at the assertion level or at the command-assertion level 18:58:37 Matt_King: In both cases, I think it would have to be a new column in the CSV 18:59:04 james: I think this is another assertion exception 18:59:42 Joe_Humbert: I did re-test quickly, and I did experience. The problem I found is that it plays it almost concurrently with the activation sound, so unless you are specifically trying to find it, then you will miss it 19:00:05 james: And that somewhat goes against the intent of the ARIA specification design on this 19:00:33 james: But regarding the CSV, the format could go after the ID of the assertion 19:00:43 Matt_King: Like a whole separate word 19:01:50 Matt_King: I'll raise an issue for this. It can hopefully make our bot reporting more accurate over time 19:02:40 Zakim, end the meeting 19:02:40 As of this point the attendees have been jugglinmike, Matt_King, Joe_Humbert, carmen, IsaDC, james, howard-e 19:02:43 RRSAgent, please draft minutes 19:02:44 I have made the request to generate https://www.w3.org/2025/11/05-aria-at-minutes.html Zakim 19:02:51 I am happy to have been of service, jugglinmike; please remember to excuse RRSAgent. Goodbye 19:02:51 Zakim has left #aria-at 23:55:08 jugglinmike1 has joined #aria-at