Meeting minutes
<jongunderson> MK: Looking back at 2021 and looking forward to 2022
<jongunderson> MK: We have finished the year with about 30 test plans merge, about 67% of APG examples
<jongunderson> MK: Put these text plans JAWS, VO and NVDA
<jongunderson> MK: End of 2023 goal is all APG examples for JAWS, VO and NVDA, including iOS and Android devices
<jongunderson> MK: Two commitments: FaceBook funder and to the community of AT users
<jongunderson> Our current test plans are in draft form, byt 2023 we want to have a goal of making at least half of them final
<jongunderson> MK: I would like to develop more specific numbers
<jongunderson> MK: We are 2/3 the way drafting 3 out of the 5 ATs for the current APG examples
<jongunderson> MK: Is this clear?
<jongunderson> JG: Makes sense to me
<jongunderson> MF: I think it is pretty clear
<jongunderson> MK: All of our other goals are related to how we make this happen
<jongunderson> MK: We need to collect test results, resolve descrepencies and create reports to AT companies
<jongunderson> MK: If the tests are useful for along time, we need to be able to replicate the testing with auotmation, which requires developoing a standardize the API
<jongunderson> MK: There are alot of features of the website that are not public yet, these need to be worked out
<jongunderson> MK: The backend is pretty robust and we have a pretty good way to run tests
<jongunderson> MK: We probably have 90% what we need for testers, less than 50% for test plan developers and AT reporting
<jongunderson> MF: We have a prototype for NVDA for automated testing, and we need to expand that to be able to run our tests
<jongunderson> MK: On Windows we can feed it tests?
<jongunderson> ST: I think we have it working, we are still getting some errors that need to be resolved
<jongunderson> MK: IO will post an update from Simon
<jongunderson> MK: What does it mean about where we are?
<jongunderson> ST: The driver we built is generic and it should work across operating systems like macOS
<jongunderson> ST: As we add new OS there always new issues that come up, for example driving the keyboard
<jongunderson> JS: From a practical point of view we are getting the speech output from the process...
<jongunderson> MK: We have not done anything in the hareness to test the speech out to a test case, we need to work out what we want to do
<jongunderson> JS: That is issue 9, every thing is screen reader agnonstic, someone at Mozilla is already working on it
<jongunderson> JS: ... discussion of development issues for automation API speech and keyboard ..
<jongunderson> MK: There are a bunch of things to work out in the automation work stream
<jongunderson> MK: Big picture we have some prototypes and research that is very practical
<jongunderson> MK: Let's say we have some result in the DB and in the report, but as new AT versions come out we can reun the tests and identify the difference, I hope we can get here soon
<jongunderson> MK: Maybe just NVDA or maybe more
<jongunderson> MF: My minimal goal is NVDA
<jongunderson> ST: To satisify the goal you mentioned we will need an infrastructure for continous testing
<jongunderson> ST: We may need to look at some system level coordination
<jongunderson> MK: There can be people monitoring and triggering automation tasks
<jongunderson> MF: We would be looking for new versions of browsers and AT it would automatically run test and send reports
<jongunderson> ST: We need to expand the thinking to the system level of how things interact
<jongunderson> MK: We need to look at where we are with all the many activites
<jongunderson> MK: We need to explain where we would like to standardize, but we have not defined any new standards yet
<jongunderson> MF: Summary of some goals for Bocoup and Prime Access
<jongunderson> MK: A coordinator for testing
<jongunderson> MF: need to start getting numbers for the status of the tests plans
Working Mode
<Matt_King> https://
<jongunderson> MK: Test plan research and development
<jongunderson> MK: The part we spend the most time on is running tests
<jongunderson> MK: Then when we have a test we want to discuss with screen reader users, we can discuss it with them and then publish the results
<jongunderson> MK: If there are still problems, and that will happen, we already have over 5,000 assetions
<jongunderson> MK: This document descibes the human processes, it does not describe the automation processes
<jongunderson> MK: There are currently no non-human activities, we need to have a discussion on this process
<jongunderson> MK: This document is a guide, please read and give feedback, and it covers what we need to cover
<jongunderson> MK: Any thoughts?
<jongunderson> MK: I am not sure about adding automation, I understand why we do not include automation
<jongunderson> MK: Where is the best place to put in automation
<jongunderson> MK: We have done this whole process, but now we have a new version of a technology, we just say we will use automation
<jongunderson> MF: That is a good place to start
<jongunderson> MK: I think it fits in a road map
<jongunderson> MK: We need people to read this document, and this defines the activities with AT vendors
<jongunderson> AG: Where is the documents
<jongunderson> MK: It is also in the chat now
<jongunderson> MK: You can rasie issues on github or on the list related to the document
Where are we at running tests
<jongunderson> MK: Going into the app
<jongunderson> MK: Checking the test que
<jongunderson> MK: We got a JAWS report out for disclosure navigation, working on NVDA
<jongunderson> MK: ... discussion of a specific test problem ...
<jongunderson> ST: We are making changes to provide more information on when the commit was made
<jongunderson> MK: As admin I can run the test as someone else, so I have completed the test runs and fixed conflicts
<jongunderson> MK: I think disclosure and navigation menu, Joe and Jon, that one only shows three conflicts
<jongunderson> MK: We need to pick which of these plans we want to do next, I would like to prioritize the combobox
<jongunderson> JS: Which one?
<jongunderson> MK: The select only combobox, looks like we are pretty close to complete results
<jongunderson> MK: Jon you have 18 of 21 complete, Joe is done
<jongunderson> MK: Next week we could look at the conflicts for that one
<jongunderson> MK: We have a different combobox for JAWS and Chrome, Lewis is the only one done
<jongunderson> MK: We need to add auto complete and list to NVDA, since we have it for VO and Jawss
<jongunderson> MK: We currenly have Jemma and Hadi assigned for that one
<jongunderson> JS: I need to go to another meeting
<jongunderson> MKL Jemma, you were assigned to run VO on the macOS
<jongunderson> Jemma: When is it due?