15:57:13 <RRSAgent> RRSAgent has joined #aria-at
15:57:18 <RRSAgent> logging to https://www.w3.org/2026/08/26-aria-at-irc
15:57:18 <Zakim> RRSAgent, make logs Public
15:57:22 <Zakim> please title this meeting ("meeting: ..."), Matt_King
15:57:38 <Matt_King> MEETING: ARIA  and Assistive Technologies Community Group
15:57:55 <Matt_King> CHAIR: Matt King
15:57:56 <Matt_King> present+
16:02:35 <jugglinmike> jugglinmike has joined #aria-at
16:02:41 <mmoss> mmoss has joined #aria-at
16:04:23 <Matt_King> agenda: https://github.com/w3c-cg/aria-at/wiki/August-26%2C-2026-Agenda
16:04:24 <agendabot> clear agenda
16:04:24 <agendabot> agenda+ Review agenda and next meeting dates
16:04:24 <agendabot> agenda+ Current interop reporting status
16:04:24 <agendabot> agenda+ Tri-State Checkbox Test Plan
16:04:24 <agendabot> agenda+ App issue 1666: Can't advance tabs test plan
16:04:24 <elizabeth> elizabeth has joined #aria-at
16:04:31 <agendabot> agenda+ PR 1400: Add HTML Button Test Plan
16:04:31 <agendabot> agenda+ Revised how to test wiki page
16:04:44 <mmoss> present+
16:04:45 <Matt_King> agenda?
16:04:53 <jugglinmike> present+ jugglinmike
16:04:57 <jugglinmike> scribe+ jugglinmike
16:05:01 <jugglinmike> Zakim, next item
16:05:01 <Zakim> agendum 1 -- Review agenda and next meeting dates -- taken up [from agendabot]
16:05:01 <Joe_Humbert> Joe_Humbert has joined #aria-at
16:05:05 <jugglinmike> present+ Joe_Humbert
16:05:08 <jugglinmike> present+ Matt_King
16:05:12 <jugglinmike> present+ elizabeth
16:05:47 <jugglinmike> Matt_King: lola isn't present today, so we may need to skip her pull request for the HTML button tests
16:06:02 <jugglinmike> Matt_King: We'll see if she joins in the mean time
16:06:05 <jugglinmike> Matt_King: Requests for changes to agenda?
16:06:37 <jugglinmike> Matt_King: Hearing none, we'll stick with the agenda as published
16:06:42 <jugglinmike> Matt_King: Next CG meeting: Wednesday September 9
16:06:48 <jugglinmike> Zakim, next item
16:06:48 <Zakim> agendum 2 -- Current interop reporting status -- taken up [from agendabot]
16:07:34 <jugglinmike> Matt_King: There are no changes in the test plan phases. We're working on getting the test plans ready, so we still have 2 in draft review (one with active work ongoing, one stuck due to an application bug)
16:08:00 <jugglinmike> Matt_King: Lola's button test plans will probably be merged and in the system and advanced to "draft review" by the next meeting
16:08:39 <jugglinmike> Matt_King: I haven't made any progress in the stuff that's in "candidate review" in the last couple months with the screen reader developers.
16:08:48 <jugglinmike> Zakim, next item
16:08:48 <Zakim> agendum 3 -- Tri-State Checkbox Test Plan -- taken up [from agendabot]
16:09:21 <jugglinmike> Matt_King: Quick reminder (especially since I'm the only one who was working on the pull request): the more people who are paying attention to the details of the test plan, the better (to make sure there are no mistakes)
16:09:42 <jugglinmike> Matt_King: To that end, thanks to Joe_Humbert for surfacing some issues that I have since resolved
16:10:00 <jugglinmike> Matt_King: I've advanced the new version of the test plan to "draft review", and the system did a really good job of keeping all of the work that's been done in place
16:10:25 <jugglinmike> Matt_King: It looks as though in the JAWS run that Joe_Humbert completed, it only removed two verdicts, so Joe_Humbert only has to re-do those two verdicts
16:10:36 <jugglinmike> Matt_King: And then it's the same thing with the test plan run for NVDA
16:11:06 <jugglinmike> Joe_Humbert: I noticed that in the JAWS test, so I guess I glossed over it in the NVDA test
16:11:57 <jugglinmike> Joe_Humbert: In one of the earlier test, I believe it was "navigating backwards" with JAWS, it read out more information than there were assertions for. That was captured by the bot and that I confirmed through manual testing. The assertions were about the name and state of the checkbox, but when navigating backwards, it also gave the group label
16:12:13 <jugglinmike> Joe_Humbert: Is that expected or something new? (It was just for JAWS)
16:12:27 <jugglinmike> Matt_King: I'm taking a look at the test plan, now...
16:12:41 <jugglinmike> Joe_Humbert: Unfortunately, since it's just me assigned, I can't actually review the results
16:13:05 <jugglinmike> Matt_King: Ah, yes, "shift + x" announces the "all condiments" group
16:13:44 <jugglinmike> Matt_King: So that's not necessarily expected because the "navigate back from here" is outside of that group
16:14:26 <jugglinmike> Joe_Humbert: Right. It's just that generally, the assertions kind of match the shape of what you get because that's what you're expecting. I don't know if this has changed between versions of JAWS. I don't think it's a major issue; I just wanted to make note of it
16:15:10 <jugglinmike> Matt_King: Actually, the "mixed" checkbox is indeed inside the group. The label on the checkbox is "All condiments" and the label on the group is "sandwich condiments"
16:15:40 <jugglinmike> Matt_King: You wouldn't necessarily expect the group label because you aren't transiting the group boundary, I wouldn't mark this as excess verbosity because you are inside the group
16:15:45 <jugglinmike> Joe_Humbert: Agreed. I did not
16:16:09 <jugglinmike> Matt_King: Okay, I think this is just a case of acceptable extra information. A good call-out, though!
16:16:21 <jugglinmike> Matt_King: As for what to do about VoiceOver
16:16:42 <jugglinmike> Matt_King: Are you set up to capture the output though that VoiceOver capture program?
16:17:08 <jugglinmike> Joe_Humbert: I believe so. I have two different macs: one with 15 and the other with 26. Should I test with the latest version of macOS?
16:17:32 <jugglinmike> Matt_King: I think we should use the latest version because it will probably be difficult to find someone who is running the older verison
16:17:35 <jugglinmike> s/verison/version/
16:18:18 <jugglinmike> Matt_King: macOS 15 is so old now that I would be shocked if we could find two people to run it. macOS 26 will be old soon, itself.
16:22:14 <jugglinmike> Matt_King: mmoss, if you have the time to invest, it would be more beneficial for us to get the macOS bot upgraded
16:22:18 <jugglinmike> mmoss: Sounds good!
16:22:35 <jugglinmike> Zakim, next item
16:22:35 <Zakim> agendum 4 -- App issue 1666: Can't advance tabs test plan -- taken up [from agendabot]
16:22:43 <jugglinmike> github: https://github.com/w3c/aria-at-app/issues/1666
16:23:10 <jugglinmike> Matt_King: This issue is keeping the "tabs" test plan in "draft review" even though all the testing is done. We have some problem where the system errors when I try to advance it
16:23:25 <jugglinmike> Matt_King: elizabeth has volunteered to take a look!
16:23:41 <jugglinmike> elizabeth: I sent a message to the Slack channel on Monday. I received ChrisC's message
16:24:11 <jugglinmike> elizabeth: I was able to run it locally, but I noticed that on my side, it is in the "draft preview" but on production, it is in "r&d"
16:24:54 <jugglinmike> Matt_King: "r&d" is before draft. If it's in "draft", then the next step is "candidate"
16:25:30 <jugglinmike> Matt_King: In the production deployment of the app, on the "data management" page and the row for "tabs with automatic activation". It's at version 25.0.8.01
16:25:38 <jugglinmike> s/01/21/
16:26:08 <jugglinmike> Matt_King: I think the best thing to do here, without knowing much about how to do it, is to get a snapshot of the current production data
16:26:29 <jugglinmike> Matt_King: In order to go from R&D to draft to candidate: it can't advance to candidate until all of the results are available and the test plan run is marked "final"
16:27:13 <jugglinmike> Matt_King: If your version of that test plan is still in the "r&d" phase, that means that you don't have any of those test results in your local database. It would be a lot of work for you to regenerate all of those. It would be faster and easier if you were able to get a snapshot of the production data on your local  machine
16:27:33 <jugglinmike> elizabeth: I'm not sure how to do that. I will wait for chrisc's message to see if they are able to help me with that
16:27:44 <jugglinmike> Matt_King: That will be the next step. I would assume there is some documentation available
16:29:29 <jugglinmike> jugglinmike: I can look for documentation on that procedure. It's especially important to try that because the problem is due to a subtle problem with the production database itself. Recreating that exact state manually may be difficult
16:29:35 <jugglinmike> Zakim, next item
16:29:35 <Zakim> agendum 5 -- PR 1400: Add HTML Button Test Plan -- taken up [from agendabot]
16:29:50 <jugglinmike> Matt_King: We're skipping this topic because Lola is absent
16:29:55 <jugglinmike> Zakim, next item
16:29:55 <Zakim> agendum 5 was just opened, jugglinmike
16:30:07 <jugglinmike> Zakim, close item 5
16:30:07 <Zakim> agendum 5, PR 1400: Add HTML Button Test Plan, closed
16:30:08 <Zakim> I see 1 item remaining on the agenda:
16:30:08 <Zakim> 6. Revised how to test wiki page [from agendabot]
16:30:12 <jugglinmike> Zakim, next item
16:30:12 <Zakim> agendum 6 -- Revised how to test wiki page -- taken up [from agendabot]
16:30:30 <jugglinmike> Matt_King: I spent some time on a page that is linked in the agenda
16:30:34 <jugglinmike> Matt_King: https://github.com/w3c-cg/aria-at/wiki/Running-a-Test-Plan
16:30:41 <jugglinmike> Matt_King: The title is "Running a Test Plan"
16:31:06 <jugglinmike> present+ tugce
16:32:04 <jugglinmike> Matt_King: There is one section that have not finished writing, yet
16:32:24 <jugglinmike> Matt_King: I tried to start from the very beginning, assuming very little. I imagined a new tester who has just joined the group
16:33:33 <jugglinmike> Matt_King: I started with an overview of the testing process and tried to keep it as simple as possible. I reviewed the five steps that we have, trying to explain the basics of the language (test plan, test queue, etc), explaining the assignment of test plans to people, explaining the goals, explaining the process for correcting mistakes in the plan and differences between results, and finally getting it to the point where an admin can
16:33:33 <jugglinmike> finalize the plan
16:33:53 <jugglinmike> Matt_King: My goal for a new tester like tugce is to give an overall idea of what we're doing when we run these tests
16:34:04 <jugglinmike> Matt_King: The rest of the page is all about getting into the nitty-gritty of executing a test plan
16:34:33 <jugglinmike> Matt_King: I talk a little bit about the instructions and then go through dealing with the nuance of commands with multiple key presses, when to use the "no output" check box, the "untestable" check box
16:34:44 <jugglinmike> Matt_King: I haven't written about the "negative side effects" checkbox
16:35:01 <jugglinmike> Matt_King: And the details of recording screen reader output. I provided instructions and links
16:35:21 <jugglinmike> Matt_King: I'm intending for someone to walk through this and follow the instructions in a test plan to confidently execute a test plan for the first time
16:35:30 <jugglinmike> Matt_King: I'm looking for feedback on anything in here that isn't clear or confusing
16:35:56 <jugglinmike> Matt_King: In particular from Joe_Humbert or mmoss, if there's something you have encountered when you've been running test plans in the past, if there have been any problems that raise doubts
16:36:19 <jugglinmike> Matt_King: It occurs to me now that one thing I didn't talk about at all is the process of raising issues. I wonder if that should be included
16:36:27 <jugglinmike> Matt_King: I could provide some guidance about when and how to raise an issue...
16:36:45 <jugglinmike> elizabeth: I think so. I think it could also be helpful to discuss the bots
16:37:06 <jugglinmike> Matt_King: Ah, do you mean more explicit instructions about verifying the bot output?
16:37:08 <jugglinmike> elizabeth: Yes
16:37:49 <jugglinmike> Matt_King: I wrote a little about that [reads the current text]. In the second paragraph, under "recording screen reader output", I think I talk about that [reads the current text]
16:38:13 <jugglinmike> elizabeth: Yes, that's what I was talking about
16:38:50 <jugglinmike> Matt_King: I think, if someone discovers errors or omissions here, you could e-mail me directly or raise an issue in the repository. Either way is fine
16:39:11 <jugglinmike> Joe_Humbert: The vast majority looks good to me so far, but I haven't reviewed all of it, yet
16:39:32 <jugglinmike> Joe_Humbert: Somewhat related: do we know if the VoiceOver recorder application will work with macOS 26?
16:39:50 <jugglinmike> mmoss: It does work on macOS 26
16:40:03 <jugglinmike> Joe_Humbert: Good to know! The GitHub repository hasn't been updated in a very long time
16:40:23 <jugglinmike> mmoss: There is a bug where it will occasionally forcibly relaunch VoiceOver
16:40:29 <jugglinmike> Joe_Humbert: That's a bug in macOS 15, as well
16:40:58 <jugglinmike> Joe_Humbert: The biggest thing for novice users of that tool is in getting Homebrew set up.
16:41:21 <jugglinmike> Joe_Humbert: I've experienced strange installation errors with Homebrew, and I think those could trip up contributors without systems knowledge
16:41:51 <jugglinmike> Joe_Humbert: If it were instead packaged in a standard macOS installation procedure, I think that would be more robust
16:42:50 <jugglinmike> Joe_Humbert: I assume that is related to how strictly Apple polices third-party software installation--that it requires signing certificates for security purposes
16:43:20 <jugglinmike> mmoss: We ought to be able to generate a pkg file regardless of whether we sign it; it will just warn the end-user when they go to install it
16:43:41 <jugglinmike> Matt_King: What are the hurdles to securing a signing certificate?
16:44:12 <jugglinmike> Joe_Humbert: You have to join Apple's developer program, pay a yearly fee, and submit the application for review from Apple
16:44:59 <jugglinmike> mmoss: That's only needed if you don't want the user to have to go through any hoops to decrease security
16:45:53 <jugglinmike> Matt_King: Maybe the most practical solution is through an organization who has people who are already Apple developers. If they were to join ARIA-AT, they could help here. It could still be an open-source application, right? I wonder...
16:46:22 <jugglinmike> mmoss: You just need to be a member of the Apple Developer program. Technically, I am a member
16:46:38 <jugglinmike> mmoss: It does cost money, though, and there is identity verification
16:46:53 <jugglinmike> Matt_King: I wonder if Dequeue has built any software for macOS
16:47:27 <jugglinmike> Joe_Humbert: It would be lovely if we could find someone at Apple to help get us internal support
16:47:53 <jugglinmike> Matt_King: I can raise that with James Craig. Why not have a participant from Apple in ARIA-AT?
16:48:20 <jugglinmike> Matt_King: Meta had tons of open-source software (like React). I haven't noticed whether Apple does any open-source software work
16:48:35 <jugglinmike> mmoss: They do it. It's listed on https://github.com/apple
16:48:45 <jugglinmike> Matt_King: Oh, thank you! I'll research that
16:49:47 <lola> lola has joined #aria-at
16:49:52 <lola> present+
16:52:43 <jugglinmike> Zakim, end the meeting
16:52:43 <Zakim> As of this point the attendees have been Matt_King, mmoss, jugglinmike, Joe_Humbert, elizabeth, tugce, lola
16:52:43 <Zakim> RRSAgent, please draft minutes
16:52:45 <RRSAgent> I have made the request to generate https://www.w3.org/2026/08/26-aria-at-minutes.html Zakim
16:52:49 <Zakim> I am happy to have been of service, jugglinmike; please remember to excuse RRSAgent.  Goodbye
16:52:53 <Zakim> Zakim has left #aria-at
16:53:17 <elizabeth> elizabeth has left #aria-at