Meeting minutes
<Sam_shaw> Agenda: 1. Discussion of new test writing language to eliminate use of terms "interaction mode" and "reading mode" 2. Data integrity assurance planning
Current Testing Status
MK: Right now, we dont have any NVDA testing in the queue, only JAWS and Voiceover. Two plans for VO
MK: For JAWS, Hadi has been assigned some test that aren't complete
MK: For VO, only Isabel has been assigned a test
MK: We don't have an additional person assigned for VO Test Plan Actions Action Menu Button Example Using aria-activedescendant
MK: Joe are you able to test that?
JoeHumbert: Yes I can
MK: Alyssa, have you completed everything you were assigned?
Alyssa: Yes, I don't think I've been assigned anything new
MK: Testing updates. All the test plans will be changing in the next couple of months
MK: Right now there are big changes related to the app, mostly test plan admin updates, that will be deployed in the coming weeks. These will help James and I view more clearly what is happening with the working mode process in the app
MK: As we've been working with the Screen reader devs, we've encountered two new issues. 1 as assertion is associated with a command, that the developer doesn't think should be associated with that command. Example navigating with quick keys with JAWS, they believe the output should be different than navigating with arrow keys
MK: In our internal meetings, we've decided we agree
MK: We also think we need a middle set of requirements, to notate assertions the SR should do, but not absolutely necessary Beauvoir
MK: To fix these things requires some big changes, and there are a some knock on problems that are a result
MK: Some of these plans, like the insert up arrow/tab problem, several of the test plans we have already completed, some we anticipate running into soon with the next plans we want to test.
MK: Also, for testing related to mode switching, we now know we will need the mode switch assertions only on some commands
MK: So those are the issues we are currently blocked on
JoeHumbert: Do you have a timeframe as to when the decision will be made?
MK: Yes, I will have a complete project plan proposal for James and I and anyone else to review, done by Monday. I'm spending all my time on this
MK: Then Bocoup will perform some assessments to prioritize the different changes a couple weeks after that. Then we will focus on the P1 issues, I aiming to have some real progress by september, wrapped up by the end of september
JoeHumbert: Okay, when should I aim to have this testing done for Action Menu with VO?
MK: in the next couple weeks would be great
Discussion of new test writing language to eliminate use of terms "interaction mode" and "reading mode"
MK: There are two problems here, both related to VO. Apple insists VO is modeless. Therefor we have all these tests we navigate in reading mode or interaction mode, then we have separate tests for VO that don't mention reading or interaction mode.
MK: This is a problem because essentially each SR has a different set of tests
MK: The other problem, with the current approach, we aren't really testing VO the way VO users use it, with is we don't turn on quick nav
MK: We also use a bunch of commands that aren't commonly used
MK: The bigger problem is having a different set of tests, which causes some confusion with reporting
MK: I've been thinking about how to solve this problem and I have an idea
MK: I'm thinking of ways we can get rid of that wording
MK: I don't have this written up in the issue yet.
MK: One this VO clearly has that seems analogous, is the voiceover curser and keyboard focus. JAWS talks about the virtual cursor and pc cursor, NVDA talks about application focus and browse mode but I think the use the word pointer
MK: NVDA doesn't refer to a cursor
Alyssa: NVDa has a review cursor and object navigation
MK: Thats right!
MK: The goal is to not use SR specific language
MK: So they all talk about cursors.
MK: Here is a proposal. I went through the discloser plan.
<Matt_King> Navigate forwards to a collapsed disclosure button in reading mode Move reading cursor forwards to a collapsed disclosure button
MK: That is my proposed wording change to the disclosure plan
JoeHumbert: I'm assuming there will then be less tests?
MK: No same amount of tests, just changing the wording. For VO actually we will end up with twice as many tests but they will be shorter with fewer commands
MK: Right now VO doesn't have a test for moving the curser
MK: If we have the navigate forward to x in reading mode test, that would become move reading cursor forward
MK: This would allow us to use the same test for JAWs and VO.
MK: If we refer to the javascript in in the browser as the application that works, and would work for native applications
MK: So that works well for navigation type commands
MK: I think that also works with any SR
MK: So we have four general categories of commands: Navigation, reading, operation, Those are 3 main ones, then some miscellaneous ones like dismiss
MK: Now for the reading ones
MK: These are trickier
MK: We currently have commands in two tests, read info about button in reading and read info about button in interaction mode
MK: Borrowing from the VO test, it has language that says
describe the element in the reading cursor, and describe element with keyboard focus
MK: We say read info about button in reading mode, or read info in interaction mode
MK: For reading mode
MK: Let me know what sounds good
MK: Read info about a button in reading cursor
MK: request info about a button in reading cursor
MK: Describe button in reading cursor
MK: Interrogate a button in a reading cursor
Alyssa: I feel like request would be fine, thats what you are doing
MK: Thats correct
MK: Yeah thats what I was wondering if someone would interpret that that way
MK: What do you think Joe?
JoeHumbert: I agree with Alyssa
Alyssa: Interrogate could be fun!
MK: You say insert tab is requesting info about it. If in a grid, JAWS has a command to give you row and column info, so it would have a different assertion but still asks the same request to info about where you are
MK: Then you don't need a separate test for row and column
MK: You could have one test with many assertions
MK: So when it came to the application cursor, I said request info about a button with application focus
MK: We are kinda saying request info about a button that has application focus, but instead to shorten it I has with application focus
MK: In VO the cursor is a box visually
MK: Alot of times in our documentation we talk about focus, but what we really mean is the cursor
MK: We could say with reading cursor focus instead of with cursor focus
<Matt_King> Request information about a collapsed disclosure button with reading cursor focus
MK: I think I will go with that for the proposal
MK: I think thats a good discussion about reading and requesting info in ways of working
MK: The next set of commands relates to operating
MK: We current have "operate a collapsed disclosure button in reading mode"
MK: My proposal is
MK: " Use reading cursor function to operate a collapse disclosure function"
Alyssa: That is very clear to me
MK: Then the interaction mode version " operate a collapse disclosure function in function mode" would become "use application functions to operate..."
MK: It seems when I apply that wording in operating, replacing operating mode with application function and reading mode with cursor functions it works well
MK: Here's one I haven't figured out yet "Dismiss dropdown in reading mode"
Alyssa "dismiss a dropdown with reading cursor active"
MK: If we do that, then what would be the application version? With VO the reading cursor is always active
MK: "Use reading cursor functions to dismiss a dropdown"
MK: I need to review the purpose of this test though I'm not sure this is the focus
MK: This might also be a case of examining "Do we want this test?"
MK: I will need to review this test
MK: "Use application function to activate a link in the drop down"
MK: The only that seems a little suspect to me is the dismiss one
MK: So this is some good feedback
MK: I'm actually hoping the tests become easier to read. We also need to increase the readability of the tests, so having clearly worded tests will effect how clear the reports are
MK: Okay this will be in an issue
MK: Anyone else have thoughts to what to watch out for?
JoeHumbert: Just be consistent and not choosing new terminology all over the place. So that its easy to jump between test for one AT and tests for another to clearly understand the direction
MK: I hoping that all desktop SR would have the exact same tests with the same words, just the list of commands and assertions may vary.
MK: We currently provide translation " How do you turn off the reading cursor"
MK: This will create some tests we need to rerun but I think we can handle it
MK: Thanks everyone, good discussion. Next weeks meeting is on Thursday
MK: We might be able to skip a meeting next week, I will discuss with James. I think with the state of the test queue and all the work on the application we may not need a meeting