Meeting minutes
Review agenda and next meeting date
<Matt_King> View agenda at https://
<Matt_King> Next meeting on Wednesday August 16
Current Test Work
Matt_King: Action Menu Button
Matt_King: Isa and Joe have completed the whole test plan, no conflicts, I think we can publish
Isa: I will publish it
Matt_King: Disclosure Navigation Menu Example
murray_moss: I will perform my testing tomorrow
Matt_King: Hadi you said you can finish soon as well
Matt_King: Currently for Disclosure Navigation Menu Example on JAWS and Chrome there are 21 conflicts
Matt_King: I wonder if we should review those conflicts before Hadi finishes up
Hadi: I think some plans are collecting too much info, For example interaction mode or reading mode. For reading mode, we drive little piece by piece, and forget our target. As we enter a list or our target, I see in our tests in virtual mode to satisfy our pass, we are requiring it read the parent item info and the item info.
Hadi: This would make a very verbose screen reader
Matt_King: I wonder if you are interpreting the test the same way, for this test when you are instructed to press the down arrow three times, when you record your output you need to record the output for each press of the key
Hadi: okay that makes sense, I did not do that, which may explain all the conflicts
Hadi: Another issue, for a test that uses the letter I to go to the next item, I think it reads to much
Matt_King: I think you're right, we had that discussion with Vispero and we decided to change the way we write our tests
Matt_King: We will be able to have one test for each of the commands, which allow the i key in this case to say less
Matt_King: I'm glad you're working on this test, when we apply the next round of changes to the app, we will want you to test it and let us know your feedback
Hadi: I have one more question. Isablel sometimes I retest something to make sure my observations are correct, sometimes I notice a change. When we are testing #25 in the Disclosure test, when I do that it says " the focus will be on the about disclousre, then in it should be in interaction mode and by default it is expanded. Sometimes when I rerun the test I see the about disclosure is expanded and sometimes not, do you see that?
Isa: Yes you need to refresh and press F5
Hadi: Yes I do that, I refresh the page and rerun the test
JS: It could be an more generic issue with the browser, or it could be an issue with a test. For that we need a specific example and an issue to filed, let us know if we can help with that.
Matt_King: I think I have observed something like that before as well Hadi, everytime thats happened, if I close the window and reopen it the issue is resolved.
Hadi: When I press insert tab it tells me all the correct info, when I press insert tab again, it doesn't announce the state, I think it might be a JAWS bug.
Matt_King: Before we move on, Isa I think we need Hadi re-record his test results right?
Isa: Yes! I think so
JS: He will need to rerun the tests
Matt_King: Hadi can you go back and rerun the test?
Matt_King: I think he just needs to rerun the test with conflicts
JS: It may not just be those, any test that includes multiple commands will need to be rerun
JS: I just reproduced the JAWS bug Hadi mentioned.
Isa: We're not catching that in our tests because we don't have you press the command twice
Matt_King: Correct, we don't want our tests to capture that, it would be a much different kind of test then
Hadi: This is a severe bug
Hadi: Though this is outside the scope of our tests, I thought this might be a good place to raise the issue
JS: I think this raises a good point, if someone collects an output that is outside the scope of our tests, Matt what should we do with those issues?
Matt_King: I think we can only report issues that are within the scope of our tests. For other issues I think Hadi you will need to report them outside this group. We are happy to help you and can pass it along, but it can't be reported in our app
Hadi: For the unexpected behavior's, should I remove those?
JS: Yes please, if its not related to the test please remove those
Matt_King: We only have 18 minuets left, we wont be able to discuss everything
Matt_King: Lets skip Update on coming app changes
New Test Format
Matt_King: These changes are moving quickly
Matt_King: First what is happening. Right now we have some csv files that are in the test repo and they define what you see when you run a test. We are going to change those files and how they are built, which will change what you see on the webpage
Matt_King: This will mean we will need to rerun every test
Matt_King: This doesn't mean that the data we have isn't valuable, it has gotten us to where we are with screen readers and getting feedback from them
Matt_King: Why are we doing this? We'll one reason is the one you raised Hadi.
Matt_King: Right now we can't have one test with command within the test have different assertions. This problem shows up in many different ways
Matt_King: Another Challenge we have, we only have required and optional for assertions. Many W3c standards have must, should and may. I think we will be in a better negotiating position if we make a distinction before must and should
Matt_King: We also go some feedback about the reports, that it wasn't clear what a partial support like 80% means. Are there parts of that pattern that shouldn't be used, but are? Our report should detail this, users of our reports would find this clarification helpful when deciding whether to use something or not at all
Matt_King: I think those are the two biggest changes
Matt_King: There are some other smaller changes
Hadi: For the must and should discussion, have people already decided which assertions should be must and which should be should?
Matt_King: Not yet, those will be ready for you to review in a couple weeks
Hadi: I'm concerned if you are going to much into telling the screen reader how to behave. If we tell one SR to do things this way, what's the difference between that and apple telling Jaws what to do?
Hadi: I think it should be the users choice, to display the order of the list for example. I don't have to order the list in the way the screen reader tells me
Matt_King: I think thats getting a little into the wild west of what Screen readers have been forever
Matt_King: There has to be some amount of agreement for the Must items, for Should there will be more room for options and room for the users choice
Matt_King: That is the essential problem we are solving with this project, what does everyone agree on?
michael_fairchild: I'm going to open a can of worms. I know our initial goal was not to follow standards because we thought that terminology may turn people away. Have the tides turned in that regard that we are turning more toward standards and their language like must and should?
Matt_King: I think there are more and more people clamoring for a standard
Matt_King: At the conference I went to in Seattle, there were three Screen reader vendor JAWS there, and they we're being dissenters. For a while there was a movement to remove should or may, but that was recently reversed. So yes the tides are changing
JS: I also want to add, that when we say something is required, we are saying the screen reader Must have something to pass a test. We are actually making more exceptions for Screen readers to create behavior's the users wants that are May have, like Hadi's example of where an Item is in a list, some people will find this helpful, some will find it verbose. We are now able to say, we think this is something you should offer, but you don't need to
JS: The aim isn't to force screen readers to all act the same, but to provide more feedback to vendors, which is based on feedback from them
Matt_King: I'm so excited about these changes, James I made a bunch of changes and comments yesterday.
Matt_King: This is coming, we can talk more about it next week, there are links in the agenda with more info to issues
Matt_King: Okay thats as far as we can get today, good discussion
Matt_King: Thank you everybody
Matt_King: James I'm trying to wrap up this spec by Monday to send it to Bocoup