Meeting minutes
Meeting schedule through September 22
Meeting schedule through September 22
Next Week Matt has a multi day event happening internally, and won't be able to attend this meeting. The TPAC conference is during the Week of September 12th
For Next Weeks meeting on August 25th, do we need to have a meeting?
James wants to discuss one topic sooner than later, but will need Matts input.
Decision - Cancel next weeks meeting on August 25th
Week of September 15th, there will be no meetings
Report on design workshop held August 17
Yesterday, Issac, Rich, Seth and Matt met to discuss fundamentals of the way that the status of the reports and data and test plans are reflected in the the UI of ARIA-AT App, and how the way things need to be in the future
Seth plans to post a summary of this conversation in GitHub
They arrived at a small list of changes to how to think about the working mode. The goal was to help separate out a few things a that are all currently mixed together
When talking about collecting results we are collecting SR output, and the Results which are the humans analysis of the output in answering a set of assertions
The working mode will need more terminology to define these points
In the future automation may run part of the plans, and humans will analyze the results
There will be one new page in the app, that will focus on the test plans and being able to dig back through the history the test plan, and compare to current status
You will also be able to see information about supported technologies as the relate to test plans
Check in on status of flight 2 of 16 patterns
These 16 test plans are going through our internal QA process
Our goal is to have these ready for Apple and Vispero to review by the end of the yera
There are 4 that are currently ready
Joe plans to have run the 4 tests by next meeting.
PAC and Louis from Bocoup have both been running the tests
James wanted to discuss an issue
There is a bit of confusion between incorrect output or no output
if youre expecting it to say button and it says something else, PAC interprets that as incorrect output. If the SR doesn't say anything PAC believes that is no output
the group agreed
James will create an issue to create some documentation around training related to the difference between no output and incorrect output
Michael Fairchild noted that documentation is good, but often unread. It may be better to make this explanation in the UI
Matt noted that eventually we will need a training program for humans to run tests
It wont be reasonable to assume everyone will come onboard and pick everything up based on osmosis, as is the case today
Michael suggested using a example test plan for training
Joe suggested that new users, or users who haven't tested in a while, be prompted to go through a training/refresher course on running tests
Seth suggested potentially creating a sandbox that users could see tests that have been run successfully. Matt noted that the APG is supposed to show people how these functions should work
Seth suggested potentially creating a sandbox that users could see tests that have been run successfully. Matt noted that the APG is supposed to show people how these functions should be coded
James made a note to make an issue to create a sample test plan for people to work through
They could go through the example and see the correct answers, or they could submit and have another person review their test runs
Next step on mode change testing
This is about when certain SR when you tab to an element, the SR changes their mode automatically
If you tab to any composite widget, the browser needs the key commands, so the SR switches mode to send those keystrokes.
We want to be able to capture issues around this mode change
this is currently difficult, because SR often use a sound to notate a mode switch which we can't capture
all SR's do have a setting that allows for instead of sound to use speech out put for mode changes
We decided that in order to due this kind of testing, we have to make an expectation and instruct testers to change the default mode of the SR
For JAWS, there is no longer an easy way to go back to the default settings
WE will need to figure out how to change instructions in test plans to instruct testers to make this change in settings
James - There is a list of 4 outcomes that are actionable
provide instruction to testers to change SR to announce mode changes
update 16 plans to include mode changing assertions
Amount the 16 test plans, we are going to prioritize getting the test plans ready that don't need these changes
Think through the new undesirable behavior with the App, like "unexpected mode switch occurred" Do we need one for each mode, or just a checkbox?
Bocoup noted that there are some changes in the works related to these features.
James will work on the new instructions for SR to make the changes to announce the mode change