Meeting minutes
ARIA and Assistive Technologies Community Group Weekly Teleconference
Matt_King: Review agenda and next meeting dates
https://
Matt_King: Requests for changes to agenda?
Matt_King: Hearing none, we'll stick with the agenda as planned
Matt_King: Next AT Driver Subgroup meeting: Monday November 17?
Matt_King: Next CG meeting: Wednesday November 5
Current status
Matt_King: several test plans have newer versions, but they didn't require any new testing. I corrected some errors in which HTML features they are testing
Matt_King: Upcoming will be a quantity spin button test plan. IsaDC is still working on that
Matt_King: There will be a couple more which will advance to "recommended" soon, and we're getting three new people from Apple engaged in the testing process
App release 1.22
Matt_King: If you visit the "reports" page in the app right now, you will see two big differences
Matt_King: The first is highlighting the amazing work that you'll all doing. If you've ever wondered how much work you've actually done, at the top of the reports page and the home page, there is a new banner that shows stats about how many assertion verdicts we've generated, how many commands we've tested, how many tests there are, how many people have contributed, and how much of all that has been done in the past 90 days
Matt_King: We have 63138 assertion verdicts. That's a lot!
Matt_King: 9,493 in the last 90 days
jugglinmike: It's over 9,000!!!
Matt_King: This is motivation for us to see progress. It's also showing value to the community and helping them understand that this is fresh
Matt_King: The other change: every verdict is associated with an aria-html feature. There there are two new tabls: ARIA features and HTML features. There, you can see the breakdown of this data according to the assertions
Matt_King: If you click on THOSE, you can get the raw data for that. And all the data is available in CSV format
Matt_King: Which is pretty cool for analysis
Matt_King: There are still a few mistakes in that data. I'm working on getting that corrected before we highlight this at W3C TPAC during the week of November 10
ChrisCuellar: I'm excited to have this out!
Matt_King: I'm over the top about this one. This is going to be very impactful at TPAC; I'm positive of that
Matt_King: here are other changes in the app; you can find a link to the full change set in the release notes referenced from the agenda
Matt_King: Thank you to the whole Bocoup team for all the hard work on this release
Issue 1590: Failure to detect conflicts after editing previously generated reports
github: w3c/
Matt_King: I haven't seen any comments on this issue, yet. I wanted to check in because I added some test plans that are in the test queue right now. There are these hidden conflicts where I edited one of the runs but not the other, and so conflicts should be showing up in the test queue, but they're not there
Matt_King: I'm pretty sure I know where the conflicts are (because I know what changes I made), but I don't want to erase those until we know the root cause
howard-e: If you know where they are and you want to move forward, you can. They are indeed invisible conflicts. There's no worry about data loss.
howard-e: The update on what's happening here: it's a little trickier than we initially thought, but we're still hoping to have a pull request out for review by either today or early Monday
howard-e: When that change does land, there won't need to be any re-testing. It will be an in-place correction
Matt_King: Got it, thank you
Running test plan for Tabs with Automatic Activation
Matt_King: I asked IsaDC to de-prioritize conflict resolution, so that's up to Hadi or I
IsaDC: I heard back from Hadi--he hasn't been able to contribute recently, and it's not clear if he will be back soon. I've replied to learn if/when he thinks he can return
Running test plan for Tabs with Manual Activation
Matt_King: NVDA is done
dean: Joe_Humbert and I are still working out one last conflict on VoiceOver
Joe_Humbert: In that conflict, the AT moved to the wrong tab. Dean marked the assertion that it said "tab" as "pass", and I marked it as "fail"
Joe_Humbert: This is for "shift+j"
IsaDC: Should it be "untestable"?
Matt_King: Did "shift+j" move backwards or forwards?
Matt_King: We mark it as "untestable" when it goes to completely the wrong place
Matt_King: I've never heard VoiceOver say "first form element" before. Is that really what it's saying?
Joe_Humbert: I can double-check...
Matt_King: So it went to tab 2 of 4. I'm sure we didn't start on tab 3
Joe_Humbert: And yes, it says "first form element"
Matt_King: When you do the "run test setup", which tab does it focus on?
Joe_Humbert: "Carl Anderson"
Matt_King: Ah, so it doesn't work. In that case, I think you were right to mark this as "untestable"
Joe_Humbert: This is test number 10
Joe_Humbert: I'm assuming that this is probably the same thing for test 12
Matt_King: That's right
Matt_King: That is the weirdest VoiceOver output...
Joe_Humbert: On Test 13, it travels to some hidden tab panel
Matt_King: I observed this, too
dean: It's one of the tabs which is present but that is hidden, so it's not supposed to receive focus
Joe_Humbert: I tested that two times just now, and I'm seeing the behavior consistently
dean: Got it; I'll run the test again
james: How is the content hidden?
Joe_Humbert: It's a tab panel that is hidden with "display: none". And it's empty
Matt_King: Just use what Joe_Humbert wrote, dean
james: What does the setup script do?
IsaDC: It activates and focuses the fourth tab
dean: I'll duplicate whatever Joe_Humbert's got on test 13
Joe_Humbert: I was experiencing 502 errors quite frequently last night
dean: And I can't get it to load at all right now
james: If you carry on going, does it go to the other three?
Joe_Humbert: Yes
Matt_King: So that is VoiceOver conflicts
Matt_King: The Bocoup team is working on the 502 errors
carmen: We're working on a solution: we're increasing the RAM on the server and considering other options
Matt_King: Joe_Humbert and IsaDC have one conflict in the JAWS run
Joe_Humbert: It says "tab panel" but doesn't say anything about "tab list". Isn't a group of tab reported as "tab group"?
Matt_King: Yes
Joe_Humbert: It's test 13 for the "tab" keystroke
IsaDC: I think I made a mistake
Matt_King: This assertion is a "MAY" rather than a "SHOULD" or a "MUST"
Matt_King: I think Joe_Humbert got it right
IsaDC: Yes, I agree. I'm going to update my results
Matt_King: This is awesome. Both of the tab test plans are just down to catching up on a few conflicts whose resolution we all agree about
Matt_King: The next highest priority in the test queue (and where Joe_Humbert has already done a ton of work) is the switch with HTML checkbox
Matt_King: We're waiting on results from mmoss and dean
dean: That's next on my agenda
Matt_King: Fantastic. You're signed up for both VoiceOver and NVDA, dean. Mmoss is signed up for JAWS
IsaDC: mmoss is not present today
Matt_King: We won't bother talking about two-state checkbox today
Feedback on plan for tabs with manual activation
github: w3c/
Matt_King: I think this is feedback on the bot. I wanted to verify with IsaDC before moving it to ARIA-AT App
IsaDC: I think I tagged Carmen, but I'm not sure about that
Matt_King: You did tag Carmen, yes
Matt_King: I don't see the bad output in the issue
IsaDC: It should be recorded in the issue because I didn't modify it before
IsaDC: it should be in the result
Matt_King: the output isn't in the issue
Updating reports to latest screen reader versions
Matt_King: Joe_Humbert, if you have time to help with VoiceOver: if you go to the tab labeled "automated report updates" in the test queue...
Matt_King: Oh, my mistake. Not that tab
Matt_King: On the other tab, use the filter named "automated updates (21)"
Matt_King: I'm looking for VoiceOver ones that I want to prioritize
Joe_Humbert: You can send me a list asycnhronously
Joe_Humbert: I didn't want to update Safari to the new version 26, but I think it will be difficult to go back. I'm guessing the automation system is not using Safari 26, yet
IsaDC: we always recommend running the latest version
ChrisCuellar: I think updating is okay.
james: If the macOS version is 15, but the browser version is 26, that in itself could cause some differences
Joe_Humbert: Agreed!
james: But I also think that there could be a lot more differences when the Safari version matches the macOS version
Matt_King: I don't think there's any advantage to updating to Safari 26. If you stay where you are, you'll have the screen reader version that matches the bot right now
Joe_Humbert: Okay!
Joe_Humbert: If the output is wrong, do I correct the output, or do I file a bug?
Matt_King: Well, actually, it would be good to file a bug
Joe_Humbert: Okay, I'll go through and do all the VoiceOver ones and check the output compared to what my machine outputs, and if there are any discrepancies, I will file an issue
Request for change to alert test plan
Matt_King: We can't do this issue justice in the three minutes remaining in this meeting. I'll place it at the top of the agenda for next week's meeting (which will be the final meeting before TPAC)