20:03:01 <RRSAgent> RRSAgent has joined #aria-at
20:03:05 <RRSAgent> logging to https://www.w3.org/2025/02/10-aria-at-irc
20:03:05 <Zakim> RRSAgent, make logs Public
20:03:06 <Zakim> please title this meeting ("meeting: ..."), jugglinmike
20:03:15 <jugglinmike> present+ jugglinmike
20:03:57 <jugglinmike> scribe+ jugglinmike
20:04:04 <jugglinmike> meeting: ARIA-AT AT Driver Subgroup Monthly Teleconference
20:05:37 <jugglinmike> Topic: Review agenda and next meeting dates
20:05:45 <jugglinmike> https://www.w3.org/events/meetings/47101a01-4abb-44d3-8a54-59653d6ed6bf/20250210T120000/
20:05:52 <ChrisCuellar> ChrisCuellar has joined #aria-at
20:06:07 <ChrisCuellar> present+
20:07:19 <jugglinmike> Matt_King: can we add something about BTT?
20:07:21 <jugglinmike> jugglinmike: Sure
20:07:34 <jugglinmike> jugglinmike: Next "automation subgroup" meeting: Monday, March 10, 2025
20:07:37 <Matt_King> Matt_King has joined #aria-at
20:07:46 <jugglinmike> jugglinmike: Next Browser Testing and Tools meeting: Wednesday, February 12, 2025
20:08:09 <jugglinmike> Topic: This week's BTT meeting
20:15:10 <jugglinmike> Matt_King: I think we don't want to indicate that we would use DOM terminology because that probably wouldn't work
20:15:26 <jugglinmike> Matt_King: We might use terms from CORE AAM, though they may need modification
20:17:07 <jugglinmike> jugglinmike: Yeah, if we focus on CORE AAM, we might not need to worry about the DOM or the browsing session etc
20:17:30 <jugglinmike> Matt_King: I think it just comes down to what is intelligible for the implementers...
20:18:24 <jugglinmike> present+ ChrisCuellar
20:18:26 <ChrisCuellar> scribe+
20:18:52 <ChrisCuellar> Topic: Updating AT-Driver in advance of new implementation
20:20:57 <ChrisCuellar> jugglinmike: There will be new folks coming onto AT-Driver. I'm concerned about correctness and stability of it. Specifically, I'm concerned about re-structuring "sendText" to be one of many future "userIntents".
20:22:00 <ChrisCuellar> That change in the specification would kick off implementation work for the AT vendors. We backed this changed out at the time because it didn't seem worth it at the time for implementers
20:22:36 <ChrisCuellar> Matt_King: If the change is only structural, what does that mean for the VO and NVDA bots?
20:23:15 <ChrisCuellar> jugglinmike: Right now those implementations are using "sendText" for all use cases
20:23:30 <ChrisCuellar> Matt_King: so it's the structure of the API calls
20:24:16 <ChrisCuellar> jugglinmike: Right, this is currently a very uninteresting structural change. But with more users potentially coming on, we might need to return to this.
20:24:50 <ChrisCuellar> Matt_King: Other implementors might be concerned about implementing code with lots of moving spec changes.
20:25:05 <ChrisCuellar> If we can prevent that, that would be helpful.
20:25:41 <ChrisCuellar> jugglinmike: It sounds like we need to move forward then with open PR, which will update AT-driver and then necessitate updates for implementors
20:26:19 <ChrisCuellar> Matt_King: I'm aligned with it. Are we confident that this is the right structure? This all comes from conversations at TPAC about how to deal with form input.
20:26:53 <ChrisCuellar> jugglinmike: Yes, and it also re-orients AT Driver to have an unbounded set of user intent that implementors may or may not decide to implement.
20:27:25 <ChrisCuellar> Matt_King: I've been wanting us to in that direction for a long time. It gives implementors a lot of flexibility and deals with Apple's security concerns.
20:28:11 <ChrisCuellar> jugglinmike: Ok, I will prioritize this this week. I can update the spec and work on making sure the implementations are caught up with the spec change
20:29:43 <ChrisCuellar> Matt_King: A question for Michael Fairchild, will this be suitable for UA? If an AT chooses to implement keyboard commands directly, that means they might not use all the different user intents, correct?
20:30:27 <ChrisCuellar> jugglinmike: Practically, that's correct. But implementations that only rely on keyboard commands, that wouldn't be cross-platform.
20:32:12 <ChrisCuellar> Matt_King: But if there are more user intents implemented across AT's, then it becomes easier to write cross-platform tests. We've always had a concern if screen readers behave the same way for a command like "Next Heading". There's always a chance that there may be differences between an AT API vs actual user experience. Most testers would care more about user experience.
20:32:49 <ChrisCuellar> jugglinmike: Since it's all a black box, there's still a certain amount of trust involved. We're using a protocol to simulate user behaviors at the end of the day.
20:35:01 <ChrisCuellar> Matt_King: How does all sound to you, Michael Fairchild?
20:36:07 <ChrisCuellar> Micheal_Fairchild: I don't have any strong opinions on any of this. I would feel more comfortable working with the keyboard commands. But I need to go back and review on this. Can you remind me about why we switched to a focus on User Intents?
20:36:39 <ChrisCuellar> Why does this need to be required for all ATs? Can we give preference for keyboard commands?
20:37:08 <ChrisCuellar> Matt_King: I don't think that will fly in BTT. They will favor approaches that are as cross-platform as possible.
20:38:05 <ChrisCuellar> jugglinmike: When we move toward this model of user intents, everything will be a user intent. "Send Keys" will be an Intent and it will be the first (and for now only) User Intent.
20:38:27 <ChrisCuellar> So there isn't really a difference between keyboard commands and user intents.
20:38:38 <ChrisCuellar> Michael Fairchild: Will there be any intents that will be required?
20:38:49 <ChrisCuellar> jugglinmike: All intents will be optional
20:39:26 <ChrisCuellar> Michael Fairchild: I'm ok with that. That does seem to give implementors a lot of flexibility.
20:40:15 <ChrisCuellar> Matt_King: If an implementor only wanted to use "Send Keys" that would be fine, tho again not great from a cross-platform perspective.
20:41:16 <ChrisCuellar> jugglinmike: Ok this helps me prioritize upcoming work.
20:41:50 <ChrisCuellar> Topic: Understanding the risks behind delays for MacOS releases
20:43:19 <ChrisCuellar> jugglinmike: In the ARIA-AT CG, we recognize that the Github automation system doesn't fully support MacOS 15. So there's some delay for us in running automated VO tests using 15.
20:44:56 <ChrisCuellar> I don't think this is that dire of a problem. We can still test the latest version of VO. We can also still use the VO Bot. The issue currently is that VO Testers have to be aware that they're working the automated output from an older version of VO. This isn't great, but it is better than nothing.
20:45:27 <ChrisCuellar> Most of the automated text that a tester will be provided will still be the same.
20:46:14 <ChrisCuellar> There may be differences in what they actually experience when running the tests and they may have to be manually editing the text, which is an inconvenience.
20:46:48 <ChrisCuellar> I think that even if this doesn't get resolved asap, I think we're still ok.
20:49:03 <ChrisCuellar> Matt_King: We have multiple use cases for automation. One use case is to help collect output from the screen reader so that a manual tester can make decisions about it (and they don't have to manually copy/paste). A bigger, more important use case is that we need to compare test plan results across versions of an AT when a new AT version comes out (regression testing). We can't do this manually.
20:49:47 <ChrisCuellar> I think this is a critical problem.
20:50:10 <jugglinmike> ChrisCuellar: Can we get more collaboration from Apple on this problem?
20:50:24 <jugglinmike> ChrisCuellar: A lot of the VO settings changed and are sandboxed now
20:50:43 <jugglinmike> ChrisCuellar: I wonder if there was a way that we had more of a heads up--that we knew that was coming--we might have been able to prepare
20:51:08 <jugglinmike> ChrisCuellar: The kinds of change we saw with macOS 15 are probably not of the same kind that we should expect to see with subsequent released of the OS
20:51:57 <jugglinmike> ChrisCuellar: For instance, part of the problem has to do with our ability to configure VoiceOver in the way that we need to. That's currently an unsolved problem because of where some of the preferences got moved in macOS. I'm still wrapping my head around the problem, but in general, the settings changes were significant
20:52:52 <jugglinmike> ChrisCuellar: macOS 15 is indeed available in GitHub Actions, but it isn't usable for use cases like ours our like GuidePup's
20:53:24 <jugglinmike> s/our like/and like/
20:53:57 <jugglinmike> Matt_King: Is this something GitHub can resolve, or are they blocked by Apple?
20:54:05 <jugglinmike> ChrisCuellar: That's unclear
20:54:21 <jugglinmike> Matt_King: It feels like, before we went to Apple, we'd have to get something definitive from GitHub
20:55:04 <jugglinmike> ChrisCuellar: In this situation, if we somehow knew ahead of time that Apple was going to change how user settings for  VoiceOver are made available for scripting--then, we would have been ahead of this problem
20:55:36 <jugglinmike> ChrisCuellar: I think the problem is less about how quickly GitHub can get macOS available in their CI, and more about support for VoiceOver automation
20:55:58 <jugglinmike> ChrisCuellar: It's a niche problem that GitHub may not have been aware of when they were first making macOS 15 available on GitHub Actions
20:57:16 <ChrisCuellar> scribe+
20:58:11 <ChrisCuellar> jugglinmike: I wasn't trying to de-prioritize this, but I do want to say that we should definitely address this before MacOS 16 comes out later this year. That's especially important for the second use case that Matt mentioned.
20:58:42 <ChrisCuellar> Matt_King: We already have that problem now. Most of our reports are out of date currently. We don't have a lot of results for MacOS 15
20:58:46 <ChrisCuellar> jugglinmike: Isee
20:59:17 <ChrisCuellar> s /Isee / I see/
20:59:47 <ChrisCuellar> jugglinmike: I understand this problem a lot better now so that's helpful. If there's no other questions, we'll close the meeting now. Thanks everyone!
21:00:50 <ChrisCuellar> Zakim, end the meeting
21:00:50 <Zakim> As of this point the attendees have been jugglinmike, ChrisCuellar
21:00:51 <Zakim> RRSAgent, please draft minutes
21:00:52 <RRSAgent> I have made the request to generate https://www.w3.org/2025/02/10-aria-at-minutes.html Zakim
21:00:59 <Zakim> I am happy to have been of service, ChrisCuellar; please remember to excuse RRSAgent.  Goodbye
21:01:00 <Zakim> Zakim has left #aria-at
21:01:27 <ChrisCuellar> RRSAgent, leave
21:01:27 <RRSAgent> I see no action items