19:02:50 <RRSAgent> RRSAgent has joined #aria-at
19:02:54 <RRSAgent> logging to https://www.w3.org/2025/10/16-aria-at-irc
19:02:54 <Zakim> RRSAgent, make logs Public
19:02:55 <Zakim> please title this meeting ("meeting: ..."), jugglinmike
19:04:05 <jugglinmike> meeting: ARIA and Assistive Technologies Community Group Weekly Teleconference
19:04:09 <jugglinmike> present+ jugglinmike
19:04:11 <jugglinmike> scribe+ jugglinmike
19:04:14 <jugglinmike> present+ james
19:04:17 <jugglinmike> present+ IsaDC
19:04:21 <jugglinmike> present+ dean
19:04:27 <jugglinmike> present+ carmen
19:04:35 <jugglinmike> present+ ChrisCuellar
19:04:37 <carmen> carmen has joined #aria-at
19:07:52 <jugglinmike> present+ mfairchild
19:11:52 <jugglinmike> present+ elizabeth
19:12:06 <jugglinmike> present+ Matt_King
19:12:42 <jugglinmike> Topic: Review agenda and next meeting dates
19:12:50 <jugglinmike> Matt_King:
19:12:59 <jugglinmike> Matt_King: Next AT Driver Subgroup meeting: Monday October 20
19:13:07 <jugglinmike> Matt_King: Next CG meeting: Wednesday October 22
19:13:13 <jugglinmike> Matt_King: Requests for changes to agenda?
19:13:26 <jugglinmike> carmen: I'd like to discuss issue 1575
19:13:32 <jugglinmike> carmen: in ARIA-AT App
19:13:43 <jugglinmike> Matt_King: Got it, we'll do our best to get to that, too
19:13:49 <jugglinmike> Topic: Current status
19:14:11 <jugglinmike> Matt_King: Apple has approved at least one of the "link" test plans, "link example 1". I went ahead and moved it to "recommended"
19:14:33 <jugglinmike> Matt_King: I think they expect to approve a couple more test plans in short order
19:14:44 <jugglinmike> Matt_King: That's a big deal!
19:14:58 <jugglinmike> Matt_Kings: It's the simplest test plan, but having one is better than none
19:15:13 <jugglinmike> Matt_King: We also advanced the second of two "switch" test plans to the "candidate" phase
19:15:28 <jugglinmike> Matt_King: So we're really close to having all the "switch" test plans done
19:15:39 <jugglinmike> carmen: Should we promote this milestone publicly?
19:15:57 <jugglinmike> Matt_King: I don't want to make too big of a deal about it right now. Maybe at TPAC
19:16:21 <jugglinmike> Matt_King: Coming next in "draft review" is the "two-state checkbox"
19:16:47 <jugglinmike> IsaDC: I implemented the latest changes you requested
19:17:10 <jugglinmike> Matt_King: Okay, that's probably about ready to merge. That should be ready for the test queue next week
19:17:22 <jugglinmike> Matt_King: Then, after that is the "quantity spin button" test pla
19:17:26 <jugglinmike> s/pla/plan/
19:17:34 <jugglinmike> IsaDC: I've been working on that this past week
19:17:45 <jugglinmike> Topic: Question about Disclosure Navigation Menu Example
19:18:16 <jugglinmike> Matt_King: Did we raise an issue for this, and do we have an understanding of the root cause?
19:18:17 <ChrisCuellar> github: https://github.com/w3c/aria-at-app/issues/1574
19:18:28 <jugglinmike> ChrisCuellar: Yes, we did create an issue for this
19:18:50 <jugglinmike> ChrisCuellar: I think this is a test plan report that got hidden by our automated run filtering, initially
19:19:01 <jugglinmike> Matt_King: so maybe it is a valid run
19:19:05 <jugglinmike> ChrisCuellar: I think so, yes
19:19:21 <jugglinmike> ChrisCuellar: There were differences in the JAWS versions from the report that's already been available
19:19:49 <jugglinmike> ChrisCuellar: There was a JAWS/Chrome completed in March of 2025
19:20:25 <jugglinmike> ChrisCuellar: And there's another that is incomplete. I asked howard-e to check the internal timestamp on when it was created, and he reported that it was created in September of this year
19:20:45 <jugglinmike> Matt_King: I vaguely remember saying that we would re-run this when the new version came out
19:21:11 <jugglinmike> ChrisCuellar: I think there was another bot run that JoeHumbert took over
19:21:36 <jugglinmike> Matt_King: Okay, that means we actually have work to do here on this one
19:21:54 <jugglinmike> Matt_King: We'll take this up.
19:22:16 <jugglinmike> Matt_King: As I recall, JoeHumbert is out for a week or two
19:22:30 <jugglinmike> Matt_King: It looks like there is a second JAWS Bot run that we could assign to somebody
19:22:47 <jugglinmike> ChrisCuellar: Joe took one but just needs to assign verdicts
19:22:56 <jugglinmike> s/Joe /JoeHumbert /
19:23:20 <jugglinmike> Matt_King: You're saying that was done in September with the September release of JAWS
19:23:41 <jugglinmike> Matt_King: Do we have another JAWS tester who is willing to take on this second run?
19:23:54 <jugglinmike> IsaDC: It would have to be me, but I already have a lot of other test plans assigned to me
19:24:07 <jugglinmike> Matt_King: We need more JAWS testers in this group
19:24:29 <jugglinmike> carmen: I'll close the issue, then
19:24:50 <jugglinmike> topic: Issue 1576: Proposal to  open test runner in a new browser tab
19:24:55 <jugglinmike> github: https://github.com/w3c/aria-at-app/issues/1576
19:25:04 <jugglinmike> Matt_King: I raised this issue as a result of some of my frustration
19:25:13 <jugglinmike> Matt_King: But I wanted to check with people before decided to do anything
19:25:44 <jugglinmike> Matt_King: when I open the test plan in the runner and return to the queue, the state of the queue is completely refreshed--all the disclosures are collapsed, and the filters are reset
19:26:03 <jugglinmike> Matt_King: It would be very complicated to have the test queue remember the state that it was in and reload that state whenever you return
19:26:47 <jugglinmike> Matt_King: I was thinking that a simpler solution is that whenever you do "start testing" or "continue testing" or "run as", that those open the runner in a new browser tab, and that completing the testing just closes the new tab
19:27:34 <jugglinmike> james: Let's not open a new window
19:27:54 <jugglinmike> james: If it was a link, people could opt to do that themselves
19:28:28 <jugglinmike> james: I feel like a list of links for "run as" and a list of links for "start testing" would be fine
19:28:49 <jugglinmike> ChrisCuellar: We can look into this. I like the suggestion of opening in a new tab
19:29:13 <jugglinmike> ChrisCuellar: This is a complaint of mine, as well: I would like to add more relative links in the test queue so that it's easier to navigate to specific places
19:29:42 <jugglinmike> ChrisCuellar: I think we can also encode the state of the test queue in the URL. You're right that it would be a little more work, but I think the solution can be multi-faceted
19:30:41 <jugglinmike> Matt_King: My current state of mind is: "what is the absolute simplest solution, first?" You know--the 80%-20% rule
19:31:19 <jugglinmike> Matt_King: And you're right that the test queue performance is relevant because it does take a few seconds to load. Closing a browser tab is an instantaneous way to return to it
19:31:41 <jugglinmike> IsaDC: I think we could also use some breadcrumbs within the "conflicts" page
19:31:59 <jugglinmike> Matt_King: I always open the "conflicts" page in a new tab because it is a link
19:32:47 <jugglinmike> ChrisCuellar: In the issue, let's list the common scenarios that are most frustrating
19:33:03 <jugglinmike> james: The runner already has links, they just have role=menuitem
19:33:32 <jugglinmike> Matt_King: that means changing the UI element from a menu to a disclosure, and that takes away the typeahead...
19:33:50 <jugglinmike> Matt_King: Maybe ignore the "run as..." one for now and just focus on the "start testing" button
19:33:59 <jugglinmike> ChrisCuellar: This has all been helpful
19:34:37 <jugglinmike> Matt_King: We could consider changing "start testing" and "continue testing" from buttons to links
19:35:38 <jugglinmike> topic: Running test plan for Tabs with Automatic Activation
19:35:47 <jugglinmike> Matt_King: This is just a quick check-in
19:35:54 <jugglinmike> Matt_King: Have we heard from Hadi?
19:36:08 <jugglinmike> IsaDC: No, but I remember he said that we could change his results if necessary
19:36:21 <jugglinmike> Matt_King: Okay, let's prioritize your other work above this. Maybe I'll do it in the mean time
19:36:30 <jugglinmike> Topic: Running test plan for Tabs with Manual Activation
19:36:37 <jugglinmike> Matt_King: on JAWS, that's IsaDC
19:36:47 <jugglinmike> Matt_King: Thank  you very much for doing the work on this, dean
19:37:07 <jugglinmike> Matt_King: JoeHumbert is almost done; there's just one incomplete test at the very end
19:37:18 <jugglinmike> IsaDC: This is the one that we fixed
19:37:28 <jugglinmike> Matt_King: but we have 5 conflicting results
19:37:33 <jugglinmike> dean: They're pretty simple
19:37:41 <jugglinmike> dean: JoeHumbert had a side-effect that I didn't have
19:37:58 <jugglinmike> dean: I think the others are just checking the wrong box, judging from the output he reported
19:38:14 <jugglinmike> dean: I started another bot run and checked it this morning; I should be able to do those tomorrow
19:39:02 <jugglinmike> Matt_King: in test 4, you both had the same output, and in this one, JoeHumbert says it did not switch from "browse mode" to "focus mode", but dean says that it did switch
19:39:15 <jugglinmike> dean: But JoeHumber isn't here today, so we can't resolve that right now
19:39:26 <jugglinmike> s/umber /umbert/
19:40:03 <jugglinmike> Matt_King: The next conflict is in test 13. It looks like the same output to me.
19:40:26 <jugglinmike> dean: I think I was running 25.2, but I would have to double-check
19:40:38 <jugglinmike> Matt_King: This is another one where the negative side-effects are different
19:41:22 <jugglinmike> Matt_King: You both had all the same assertion verdicts, it appears
19:41:51 <jugglinmike> Matt_King: I don't remember if there was a "tab panel boundary" assertion, but if it failed, then it seems like this negative side-effect would actually be equivalent...
19:42:05 <jugglinmike> Matt_King: I guess we need JoeHumbert present to clarify
19:42:20 <jugglinmike> Matt_King: Ah, this is for "down arrow"
19:42:42 <jugglinmike> Matt_King: We'll wait until JoeHumbert returns to address this. But it looks like we're really close except for these conflicts!
19:42:52 <jugglinmike> Matt_King: As for VoiceOver...
19:43:09 <jugglinmike> dean: JoeHumbert is done with that, and I should be able to complete the work on my part tomorrow
19:43:16 <jugglinmike> Matt_King: Fantastic, that's great! Thank you, dean
19:43:43 <jugglinmike> Topic: Running tests for Switch Example Using HTML Checkbox Input
19:44:01 <jugglinmike> Matt_King: this has JAWS, NVDA, and VoiceOver all available, including bot runs ready to be re-assigned
19:44:16 <jugglinmike> Matt_King: But everyone here is busy, so maybe we will need to wait until next week
19:44:29 <jugglinmike> Matt_King: It is a relatively short test plan
19:44:35 <jugglinmike> Topic: Updating reports to latest screen reader versions
19:44:45 <jugglinmike> Matt_King: Right now, we have five VoiceOver automated runs
19:45:06 <jugglinmike> Matt_King: If you go to the filter on the test queue labeled "Automated updates", you will find five test plans listed
19:45:17 <jugglinmike> Matt_Kings: I completed one of these last week, but I can't remember which
19:45:58 <jugglinmike> Matt_King: I wanted to get a sense of the workload. The primary differences that I was seeing was the situations where the VoiceOver bot had collected the "hint text" information in the output but the human who previously ran the test plan omitted that information
19:46:28 <jugglinmike> Matt_King: I see that Elizabeth has raised an issue where they were getting output that differed from that of the bot
19:47:05 <jugglinmike> Elizabeth: I noticed for one of the tests that I was working on. Perhaps it was "down arrow with quick nav off". The response the bot recorded did not sound right, so I double-checked and found I was getting a different response
19:47:20 <jugglinmike> Matt_King: I wonder if this is a bot isssue
19:47:27 <jugglinmike> ChrisCuellar: It could be a bot issue!
19:48:33 <jugglinmike> Matt_King: This is kind of an interesting case in the GitHub issue functionality. If there's a conflict, then you can raise an issue, and it automatically captures the output. I guess in the test runner, you do have that information...
19:49:29 <jugglinmike> Matt_King: I guess if you wanted to see the difference, ChrisCuellar, you would have to use the "Run as" feature
19:49:35 <jugglinmike> Matt_King: This is test 4
19:51:39 <jugglinmike> Matt_King: This is strange output. It seems as though the focus was in the wrong place
19:51:43 <jugglinmike> Matt_King: when I
19:52:24 <jugglinmike> Matt_King: It looks like Elizabeth submitted the results. Elizabeth, you answered the assertions based on the VoiceOver output that was there, is that right?
19:53:04 <jugglinmike> Elizabeth: That's right. I was actually not sure what to do.
19:53:12 <jugglinmike> Matt_King: Thank you for raising this question
19:53:41 <jugglinmike> Matt_King: In this particular case--this is where the bot really produced an incorrect result. We actually do want to change the response and put in what VoiceOver is actually saying
19:54:03 <jugglinmike> Matt_King: It's really good that you raised this issue because we need to know that the bot is giving the wrong response
19:54:45 <jugglinmike> Matt_King: I think I might have seen something like this--where the bot was recording the VoiceOver response as if Quick Nav was off. Rather, it recorded an "arrow" response as if quick nav was on, but it should have been off.
19:55:00 <jugglinmike> ChrisCuellar: so there may have been an issue with the "quick nav" toggling
19:55:03 <jugglinmike> Matt_King: Right
19:55:48 <jugglinmike> Matt_King: I wonder if "quick nav" was on in the prior test--in test 3
19:56:02 <jugglinmike> Matt_King: So yes, it could just be the setting of "quick nav" on and off which is causing the problem here
19:56:32 <jugglinmike> Matt_King: Elizabeth, could you please change the value of the "output" field and put in the response that you are observing from VoiceOver?
19:56:45 <jugglinmike> Elizabeth: Sure. I just wasn't sure based on some prior discussions
19:57:40 <jugglinmike> Matt_King: If it looks like the bot response is correct, we don't want to change it at all--we want the response to be formatted in the way that the bot would recorded (just in case there's a material difference between how the bot would report it and how a human would record it). But if it looks incorrect, as it does in this case, then we should change it
19:58:26 <jugglinmike> Matt_King: We want to make sure that, as much as possible, we are recording responses in the way the bot would record them. In this case, we should just record the correct response as best as we can
19:58:47 <jugglinmike> Matt_King: We're out of time for this meeting, so we will include carmen's issue in the agenda for next week
19:59:07 <jugglinmike> Matt_King: Thank you everybody!
19:59:22 <jugglinmike> Zakim, end the meeting
19:59:22 <Zakim> As of this point the attendees have been jugglinmike, james, IsaDC, dean, carmen, ChrisCuellar, mfairchild, elizabeth, Matt_King
19:59:24 <Zakim> RRSAgent, please draft minutes
19:59:25 <RRSAgent> I have made the request to generate https://www.w3.org/2025/10/16-aria-at-minutes.html Zakim
19:59:32 <Zakim> I am happy to have been of service, jugglinmike; please remember to excuse RRSAgent.  Goodbye
19:59:32 <Zakim> Zakim has left #aria-at
19:59:41 <jugglinmike> rrsagent, leave
19:59:41 <RRSAgent> I see no action items