19:59:57 <RRSAgent> RRSAgent has joined #aria-at
19:59:57 <RRSAgent> logging to https://www.w3.org/2022/01/20-aria-at-irc
20:00:09 <Matt_King> MEETING: ARIA and Assistive Technologies Community Group
20:00:20 <Matt_King> CHAIR: Matt King
20:00:30 <Matt_King> rrsagent, make log public
20:00:39 <Matt_King> rrsagent, make minutes
20:00:39 <RRSAgent> I have made the request to generate https://www.w3.org/2022/01/20-aria-at-minutes.html Matt_King
20:01:54 <Matt_King> present+
20:02:14 <michael_fairchild> michael_fairchild has joined #aria-at
20:02:23 <michael_fairchild> present+
20:02:41 <jongunderson> present+ jongund
20:08:25 <jongunderson> scribe: jongund
20:09:07 <jongunderson> MK: Looking back at 2021 and looking forward to 2022
20:10:03 <jongunderson> MK: We have finished the year with about 30 test plans merge, about 67% of APG examples
20:10:28 <jongunderson> MK: Put these text plans JAWS, VO and NVDA
20:11:10 <jongunderson> MK: End of 2023 goal is all APG examples for JAWS, VO and NVDA, including iOS and Android devices
20:12:26 <jongunderson> MK: Two commitments: FaceBook funder and to the community of AT users
20:13:41 <jongunderson> Our current test plans are in draft form, byt 2023 we want to have a goal of making at least half of them final
20:14:16 <jongunderson> MK: I would like to develop more specific numbers
20:14:52 <jongunderson> MK: We are 2/3 the way drafting 3 out of the 5 ATs for the current APG examples
20:14:58 <jongunderson> MK: Is this clear?
20:15:09 <jongunderson> JG: Makes sense to me
20:15:19 <jongunderson> MF: I think it is pretty clear
20:15:35 <jongunderson> MK: All of our other goals are related to how we make this happen
20:16:17 <jongunderson> MK: We need to collect test results, resolve descrepencies and create reports to AT companies
20:17:18 <jongunderson> MK: If the tests are useful for along time, we need to be able to replicate the testing with auotmation, which requires developoing a standardize the  API
20:19:21 <jongunderson> MK: There are alot of features of the website that are not public yet, these need to be worked out
20:19:49 <jongunderson> MK: The backend is pretty robust and we have a pretty good way to run tests
20:20:20 <jongunderson> MK: We probably have 90% what we need for testers, less than 50% for test plan developers and AT reporting
20:22:42 <jongunderson> MF: We have a prototype for NVDA for automated testing, and we need to expand that to be able to run our tests
20:23:03 <jongunderson> MK: On Windows we can feed it tests?
20:23:37 <jongunderson> ST: I think we have it working, we are still getting some errors that need to be resolved
20:23:47 <jongunderson> MK: IO will post an update from Simon
20:23:59 <jongunderson> MK: What does it mean about where we are?
20:24:26 <jongunderson> ST: The driver we built is generic and it should work across operating systems like macOS
20:24:52 <jongunderson> ST: As we add new OS there always new issues that come up, for example driving the keyboard
20:26:10 <jongunderson> JS: From a practical point of view we are getting the speech output from the process...
20:26:45 <jongunderson> MK: We have not done anything in the hareness to test the speech out to a test case, we need to work out what we want to do
20:27:36 <jongunderson> JS: That is issue 9, every thing is screen reader agnonstic, someone at Mozilla is already working on it
20:28:25 <jongunderson> JS: ... discussion of development issues  for automation API speech and keyboard ..
20:28:48 <jongunderson> MK: There are a bunch of things to work out in the automation work stream
20:29:19 <jongunderson> MK: Big picture we have some prototypes and research that is very practical
20:30:21 <jongunderson> MK: Let's say we have some result in the DB and in the report, but as new AT versions come out we can reun the tests and identify the difference, I hope we can get here soon
20:30:34 <jongunderson> MK: Maybe just NVDA or maybe more
20:30:54 <jongunderson> MF: My minimal goal is NVDA
20:31:27 <jongunderson> ST: To satisify the goal you mentioned we will need an infrastructure for continous testing
20:32:03 <jongunderson> ST: We may need to look at some system level coordination
20:32:32 <jongunderson> MK: There can be people monitoring and triggering automation tasks
20:33:06 <jongunderson> MF: We would be looking for new versions of browsers and AT it would automatically run test and send reports
20:33:33 <jongunderson> ST: We need to expand the thinking to the system level of how things interact
20:33:54 <jongunderson> MK: We need to look at where we are with all the many activites
20:34:33 <jongunderson> MK: We need to explain where we would like to standardize, but we have not defined any new standards yet
20:35:12 <jongunderson> MF: Summary of some goals for Bocoup and Prime Access
20:35:40 <jongunderson> MK: A coordinator for testing
20:36:21 <jongunderson> MF: need to start getting numbers for the status of the tests plans
20:39:24 <Matt_King> TOPIC: Working Mode
20:39:31 <Matt_King> https://github.com/w3c/aria-at/wiki/Working-Mode
20:41:42 <jongunderson> MK: Test plan research and development
20:41:57 <jongunderson> MK: The part we spend the most time on is running tests
20:42:48 <jongunderson> MK: Then when we have a test we want to discuss with screen reader users, we can discuss it with them and then publish the results
20:43:25 <jongunderson> MK: If there are still problems, and that will happen, we already have over 5,000 assetions
20:43:55 <jongunderson> MK: This document descibes the human processes, it does not describe the automation processes
20:44:27 <Jemma> present+
20:44:31 <jongunderson> MK: There are currently no non-human activities, we need to have a discussion on this process
20:45:33 <jongunderson> MK: This document is a guide, please read and give feedback, and it covers what we need to cover
20:47:15 <jongunderson> MK: Any thoughts?
20:47:43 <jongunderson> MK: I am not sure about adding automation, I understand why we do not include automation
20:48:07 <jongunderson> MK: Where is the best place to put in automation
20:48:46 <jongunderson> MK: We have done this whole process, but now we have a new version of a technology, we just say we will use automation
20:48:58 <jongunderson> MF: That is a good place to start
20:49:16 <jongunderson> MK: I think it fits in a road map
20:49:34 <AlyssaG> AlyssaG has joined #aria-at
20:51:00 <jongunderson> MK: We need people to read this document, and this defines the activities with AT vendors
20:51:15 <jongunderson> AG: Where is the documents
20:51:27 <jongunderson> MK: It is also in the chat now
20:51:59 <jongunderson> MK: You can rasie issues on github or on the list related to the document
20:52:17 <jongunderson> TOPIC: Where are we at running tests
20:52:24 <jongunderson> MK: Going into the app
20:52:39 <jongunderson> MK: Checking the test que
20:53:02 <jongunderson> MK: We got a JAWS report out for disclosure navigation, working on NVDA
20:53:07 <Jem> Jem has joined #aria-at
20:54:24 <jongunderson> MK: ... discussion of a specific test problem ...
20:54:53 <jongunderson> ST: We are making changes to provide more information on when the commit was made
20:56:02 <jongunderson> MK: As admin I can run the test as someone else, so I have completed the test runs and fixed conflicts
20:56:42 <jongunderson> MK: I think disclosure and navigation menu, Joe and Jon, that one only shows three conflicts
20:57:13 <jongunderson> MK: We need to pick which of these plans we want to do next, I would like to prioritize the combobox
20:57:19 <jongunderson> JS: Which one?
20:57:39 <jongunderson> MK: The select only combobox, looks like we are pretty close to complete results
20:58:03 <jongunderson> MK: Jon you have 18 of 21 complete, Joe is done
20:58:18 <jongunderson> MK: Next week we could look at the conflicts for that one
20:58:49 <jongunderson> MK: We have a different combobox for JAWS and Chrome, Lewis is the only one done
20:59:23 <jongunderson> MK: We need to add auto complete and list to NVDA, since we have it for VO and Jawss
20:59:45 <jongunderson> MK: We currenly have Jemma and Hadi assigned for that one
20:59:53 <jongunderson> JS: I need to go to another meeting
21:00:36 <jongunderson> MKL Jemma, you were assigned to run VO on the macOS
21:00:43 <jongunderson> Jemma: When is it due?
21:00:59 <Matt_King> rrsagent, make mintues
21:00:59 <RRSAgent> I'm logging. I don't understand 'make mintues', Matt_King.  Try /msg RRSAgent help
21:01:09 <jongunderson> rrsagent, draft minutes
21:01:09 <RRSAgent> I have made the request to generate https://www.w3.org/2022/01/20-aria-at-minutes.html jongunderson