– DRAFT –
Bi-Weekly Meeting of Assistive Technology Automation Subgroup of ARIA-AT Community Group

<jugglinmike> https://www.w3.org/events/meetings/cff6ac42-bdc2-4a38-8993-7dd46e507741/20231023T120000/

field questions on AT Driver from JAWS maintainers

Folks from Vispero have begun reviewing the AT Driver proposal, and they have some questions they'd like to ask the group.

BL: I'm a software engineer at Vispero, we are interested in implementing a version of this driver for JAWS

GG: I've been working on JAWS since 1994, we have reviewed the Go Server and NVDA Add on

BL: First question, is there a newer version of the protocol we should be reviewing?

<jugglinmike> https://w3c.github.io/at-driver/

jugglinmike: I just pasted a link to the updated reademe

BL: Could you give us an overview of how this process would work, and how would we test?

GG: To test with NVDA today, whats missing from running a test today? What are the steps we would take?

JS: There is a split responsibility here, when PAC implemented the SPEC, there wasn't any additional tooling for testing, or simulation, or running ARIA AT Test. I will hand this to Mike

MK: I will interject, right now the driver doesn't make any assumptions about how you are going to construct your test

MK: Bocoup is developing the middleware that uses the AT Driver to run the test. If you want to recreate what we are doing with Bocoup thats great, but we aren't planning to standardize this.

GG: We want to know when we are done. If we have to create our own tooling, how will we know that this is working?

JS: In order to determine the implementation of the spec is correct, there needs to be a test suite, in terms of I sent this command and I got this response

JS: If that testing exists to put an implementation through its paces, you can be sure its working and complete. I assume you are looking for a some kind of test suite that makes sure all the actions are handled correctly

GG: We can send keystrokes and allow you to collect speech, but that doesn't check if you got all the speech, or the correct speech in the right amount of time?

jugglinmike: Its pretty agnostic about these things, we haven't specified behavior at that granularity. It would be hard to express normative behaviors

jugglinmike: We haven't yet developed conformance tests, but it may be early for that, this is just a proposal, its not on a standards track yet, we aren't there yet but are looking for implementer feedback

jugglinmike: The work that Bocoup is doing is open source

jugglinmike: We have created a harness that is a no JS tool, that consumes tests and speaks this protocol to run them. You could use ARIA AT to validate your test but that's not its intent

MK: The protocol we are talking about is AT Driver. Thats what you would implement in JAWS. Thats the part of the project we want to standardize. The reliability of that protocol is what Glen you are pointing toward. How do we know the speech is complete?

<jugglinmike> This is the "harness" that Bocoup is developing in service of ARIA-AT: w3c/aria-at-automation-harness

JS: We don't specify when a SR is finished speaking, because its never done

JS: The input you send in the form of commands, and the event stream are intentional decoupled.

JS: Its the responsibility of the consuming product to determine when it feels it has received the correct output

GG: At STG, what would you show off?

MK: What we are implementing is when you go to ARIA AT website, there is a page called the test queue, which we can give you access to, which contains runs that are executed by the bot

MK: We want to show, here's how a human runs the test, and how a bot runs the test. Compare the results

MK: The harness piece of it is the piece that runs the test and calls the AT Driver.

MK: Mike they don't have to have our whole server setup to run the ARIA AT Tests right?

jugglinmike: Right

BL: So we need this node.js project/app that generates the commands that are then sent on by the Go server to NVDA itself?

JS: Correct

BL: The the NVDA addon sends back the output

BL: The Go server is just a bridge

GG: If we were to implement something similar, would you be willing to make the Go server changes

JS: We could, but we will need to discuss with Matt first

GG: This kind of seems like we should implement this later. What is the goal of the presentation?

MK: I'm not expecting a terribly involved demo, I really want to show the capabilities and the progress of the project. The biggest value to us, if you have a new version of JAWS coming out in December, we don't have enough human power to rerun all the test for all the test plans and figure out the differences.

MK: We want to be able to re run these tests and get the data out as fast as possilb

BL: The speech output doesn't seem that complicated, we just don't know anything about Go. We don't know about the test harness either

MK: We wouldn't expect you to work with the harness. James how much is the Go server is NVDA specific?

JS: I think whats important to consider is that none of the communication between the GO host and client, is standardized or spec'd out, it only lives on the Go server and NVDA Add on

JS: The Go server is a black box

JS: Essentially you may find it easier to implement the spec into JAWS, rather than using the Go server

JS: Then you don't have to parse out the JAWS output

MK: The Go server lets you restart the Screenreader

JS: Correct, the SC may crash, all of that can communicated to the remote party

JS: It also allows the communication between the Go server and NVDA to change. If we wanted to formalize the communication we could make that change without the user knowing

GG: We want to build as much as we can on what you James have built

JS: When we made the decision to have an opensource server, we intended this extensibility

JS: If you choose to implement the communication between our binary server and your Screen reader, that process isn't documented

GG: Or complicated!

JS: Right, but i dont want to make any assumptions, we have to dicuss with Bocoup how much of our scope should be spent adapting the Go Server

JS: Currently Go Server work is outside our scope

MK: To have that conversation, we would need to know what the extent of changes and timeline

JS: Right, we could also make the server more flexibile, so that the user of the server could provide the commands so its not AT specific

BL: Move it towards a node application

JS: Exactly. That might be easier, it would make it more flexible

jugglinmike: There's also the alternative of forking the Go Server. A lot of the value of a standard is to not have the coordination overhead, like maintenance. Like James is saying there isn't much documentation because there isn't a need to create it.

BL: So you're arguing the SR should replace the Go server

jugglinmike: Sort of, I'm not saying we should get rid of the Go Server.

BL: But moving the onus of documentation to the SR is more plausible to maintain

jugglinmike: That also would imply that the SR maintains it and its more stable over time

jugglinmike: If we extend the Go server to support JAWS, we may have more maintenance long term

MK: If we fork it, then the maintenance is not longer dependent on the supporting other AT

JS: What I would to see, is that each AT that is involved, has its own implementation of the protocol. Each vendor is capable of deciding how to communicate and maintain security.

JS: So that each SR has its own binary that it maintains to make adjustments

JS: we are fully hopeful that in the future that NVAccess will make their own implementation of our ad on

JS: I'd love to see a suite of tools that you can download to use to test any AT

MK: When it comes to anything related to using the protocol, its within the scope of Bocoups work.

MK: All of that work is opensourced. We can discuss implementing modifications that would help you

BL: How would we go about testing our implementation? Do you have a set of cannonical commands?

MK: Mike do you have anything right now you can give them?

jugglinmike: I linked to the harness repo, I want to give a caveat though this has been an internal document so there may be rough edges to this

jugglinmike: Please file any issues you find

jugglinmike: Even with the harness, the determination of "reasonable" will still be on you, because of the generic nature of ARIA AT.

MK: Would it be possible to share the code of known good outputs?

jugglinmike: I don't think so, that more tied to the ARIA AT app

jugglinmike: One way to use ARIA AT outside the project, is a snapshot application to determine changes between versions

jugglinmike: Finally, I want to say because it hasn't come up, we are very much inspired by the Webdriver project, all its tooling and history

jugglinmike: Its where we get alot of our language, why we use web sockets, and alot of our decisions are based on Web driver

GG: Thank you!

MK: Okay we will find the best win win for you

MK: It would be great to present at STG, but would be amazing is if we had a tool to run test with when JAWS 2024 is released