20:00:28 RRSAgent has joined #aria-at 20:00:32 logging to https://www.w3.org/2023/11/06-aria-at-irc 20:00:41 RRSAgent, make logs public 20:00:49 Zakim, start the meeting 20:00:49 RRSAgent, make logs Public 20:00:51 please title this meeting ("meeting: ..."), jugglinmike 20:00:58 present+ jugglinmike 20:01:23 jugglinmike has changed the topic to: 2023-11-06 meeting 20:06:16 Matt_King has joined #aria-at 20:07:54 scribe+ 20:08:15 Sam_Shaw has joined #aria-at 20:08:30 TOPIC: Discussion of restrictions and keypress simulation on macOS 20:08:45 present+ 20:09:47 Mike: Two meetings ago we talked about the motivation behind Apple's reluctance to allow simulated arbetrary key presses. 20:10:02 James mentioned remote support as a precedent for allowing this. 20:10:15 So, wondering why our use case is different. 20:10:36 James: If Apple is building this into VO. 20:10:51 It is reasonable that they want to respect user privacy and security. 20:11:12 It should not be something that people can be tricked into overriding. 20:11:47 It is more reasonable for a 3rd party to expect users to turn off security and privacy. 20:12:15 For example, the 3rd parties the build remote access tools like Remote Incident Manager. 20:12:30 Michael: do you have to disable security for it to work? 20:13:00 James: There are a lot of permissions you have to allow in order for Remote Incident Manager to work. 20:13:20 scribe+ 20:14:29 MK: My question is since apple has been so firm about this, I'm wondering is our goal to figure out a way to move forward without allowing arbitrary key presses and still have what we need for web based product testingg? 20:15:17 MK: Or do we want to engage Apple in discussions that have the goal of getting some kind of modification to their stance such that the spec could provide a pathway for this. We would negotiate the obstacles in the pathway 20:15:34 JS: I would ask a higher level question of what is our goal? 20:16:33 JS: If the goal is to have apple build support or all of a subset of the spec into the platform, then naturally what they think of the security is paramount 20:16:48 JS: Or we move forward with the facilities that exist today 20:17:28 JS: What is the aim? Apple aren't saying the facilities exist to inject keypresses, but we also know that apple isn't willing to build something into apple that allows users to use them 20:19:11 MK: In the next year, since there are a lot of capabilities built into the platform and VO supports those, we could simulate AT driver by building a native MAC OS application that makes it look like there is integrated support 20:19:25 JS: If we did that I would recommend we not use apple script 20:20:19 JS: The support that apple are alluding to with controlling VO with apple script, do not include arbitrary key presses'. The problem is that support doesn't work very well 20:20:49 MK: So we would be providing interoperability based on what apple says it can do it 20:20:54 JS: Right 20:22:02 MK: We could build a native MAC app that makes it look like VO supports AT Driver, because remote incident manager exists, is our confidence level that we could build such an app very high? 20:22:13 JS: My confidence level is high 20:22:21 MF: Mine as well 20:23:24 MK: Building this for iOS would be very different 20:24:30 MK: If one of our goals is having AT Driver as a standard is to enable testing not just for ARIA AT but for other projects, the implementation details on how its implement like requiring a 3rd party library, creates some challenges in maintenance long term 20:25:05 MK: I think we need to find out what it takes to get more direct support for AT Driver, and not just for AT Driver which is where it gets complicated in the long term 20:25:47 JS: To go back to your pathways Matt, that the spec could endorse levels of support, like keyboard or headings, its up to an implementer to decide what they support? 20:25:50 MK: Yes 20:26:47 MK: I was thinking there is a feature flag that says the things you must support, and flags other things you are optional to support 20:27:34 JS: The Key press could be removed from the spec to support Screen reader automation. With browsers, with web driver there are certain things you cannot do 20:27:55 JS: The simple act of opening a browser, configuring the browser, closing the browser, that is seperate 20:28:09 MK: Mike can you explain that dividing line? 20:28:52 jugglinmike: Yep, Web driver classic has the concept of sessions, that have capabilities, from the local end you send a request to create a remote session, the remote end provides a browser 20:29:16 jugglinmike: This is to support testing farms, so the remote end might not be creating the session on the same machine 20:29:55 jugglinmike: you can put constraints like headless, or use firefox. Thats also the case in webdriver bi di, which is the new version built on top of sockets 20:30:05 JS: So the browser doesn't host the remote end? 20:30:09 jugglinmike: right 20:30:49 MK: So Who provides the code that sets up that remote? Individual developers like google or mozilla? Or with tools like celinium? 20:31:39 jugglinmike: The code that starts the process is written by Google and Mozilla for themselves. Like what PAC built with Go, it speaks a proprietary protocol, theres a facade 20:32:09 JS: So your're saying browser stack may have implemented their own facade that can spin up their own sessions 20:32:25 jugglinmike: Correct, but they would still call out to the gecko driver 20:33:33 MK: So if you want to be able to spin up chrome or web kit, or gecko, they each have their own tool to spin up webdriver, but to they have their own abstraction of the layer that allows you to spin up any of those 3, or is that outside the scope of the web driver? 20:34:11 jugglinmike: I'm not sure if I would say neither or both. When you spin up a session, you decide what kind of browser you spin up that would reject commands from other browsers 20:35:02 JS: But web driver the protocol only specifies how I must respond, with a session or saying I can't do that 20:35:15 jugglinmike: you mean like what it does to start a process? 20:35:21 JS: yes 20:35:31 jugglinmike: Thats correct 20:36:39 jugglinmike: I'm glad we are talking about this, because are use is a little distinct so far as what we are testing is a singleton, it will be an uphill battle to support multiple instances, like running JAWs twice on one hose 20:36:46 MK: Well I do it 20:36:55 JS: Okay well you shouldn't 20:37:21 jugglinmike: I'm just saying that this concept of a session is less relevant to us 20:37:53 jugglinmike: but we have this concept of intermediary nodes, and service providers like browserstack 20:38:45 MK: I would argue that its not a singleton because we had a case where users would use zoom and JAWS 20:39:29 MK: It does seem the session stuff in that use case 20:39:52 JS: Okay so I was wrong in suggesting the process management is outside the browser 20:41:28 MK: So Thinking about goals and pathways, in the near term, I'm not sure what the long term risks are, but in the near term, the ARIA AT taking responsibility that Apple is taking a position that if we want Automation to be able to AT Driver than we need t build a open source application that is a AT Driver provider 20:41:49 MK: Whether or not it could be scaled to other platforms is an open question 20:43:02 MK: In regards to our inclusion in BTT, We do need to figure out what language would get us around Apples objections 20:43:30 MK: But we've kicked this down the road as we are not on their roadmap in 2024 20:46:26 MK: I had envisioned in the spec that we would have user intents, not part of the spec but referenced by the spec, we have a intent mapping document that lists all possible user intents and the keyboard mappings 20:46:34 MK: That would bridge intent and keyboard 20:47:50 JS: I think that its important that not every AT maps its events to a key 20:50:38 JS: For some AT the primary way to use them isn't keyboard based, like for voice commands, the intent focus could allow developers to optional support intents and not keyboard, or keyboard and not intents 20:51:40 MK: Mike are we answering the questions that need to be answered? 20:52:05 jugglinmike: Yes I think so. We don't need apples support to move forward in a direction we are all comfortable with 20:52:48 jugglinmike: we have a clear vision of how this will evolve over time. I think this make sense, particularly in regards the the not keyboard based AT, it makes sense 20:53:20 JS: We are balancing the spec and the practical side of the project 20:55:27 MK: I think if we can understand the direction of AT Driver from the perspective of BTT, instead of ARIA AT 20:57:29 jugglinmike: I'm think this motivates more imediate work toward session work in the protocol, to signal more strongly how we expect to support other AT's over time 20:58:48 MK: It seems like some of the objections of having AT Driver in BTT was the platform independence, the way the tests are written, certain aspects of those cant be written in a platform independent way, I think moving toward intents move us in the right direction 20:59:32 MK: Mike can you test platform specific capabilities? 20:59:46 JS: What capabilities? 21:00:12 MK: you couldn't test anyting in window's that relies on a next item capability 21:00:27 JS: Wouldn't that be the case where some intents are supported and some are not? 21:01:47 JS: I would hope that we either have support for listed intents differ for AT, but secondarily having the session invocation be aware of specific intents 21:01:55 JS: and if can't do that it tells you 21:02:17 MK: If that is built into the spec, that theoretically platform independent testing is possible 21:02:26 MK: You're just limited in what you can test 21:02:55 MK: I see a fair amount of value in a web platform test that is supported by W3C and BTT is the best way to go there 21:03:30 MK: If we had the support of the chair and enough other people to make that a reality I hope their minds don't change but I still have that nagging concern 21:03:52 MK: If it doesn't go that way, we would have all the tech we need it just wouldn't be supported by a major platform 21:04:40 jugglinmike: The only other item on the agenda was a review of a design, so I encourage people to review that 21:05:42 https://github.com/w3c/aria-at-app/issues/732 21:06:17 rrsagent, make minutes 21:06:18 I have made the request to generate https://www.w3.org/2023/11/06-aria-at-minutes.html Sam_Shaw 21:37:58 Zakim, end the meeting 21:37:58 As of this point the attendees have been jugglinmike, Matt_King 21:37:59 RRSAgent, please draft minutes 21:38:01 I have made the request to generate https://www.w3.org/2023/11/06-aria-at-minutes.html Zakim 21:38:06 I am happy to have been of service, jugglinmike; please remember to excuse RRSAgent. Goodbye 21:38:08 Zakim has left #aria-at 21:38:08 jugglinmike has changed the topic to: 21:38:19 RRSAgent, leave 21:38:19 I see no action items