<scribe> scribe:shimizuyohta

Test plan report format

<Matt_King> Link to issue: https://github.com/w3c/aria-at/issues/37

Matt: Posted an issue 36 about test plan report format
... I described the purpose of this. After somebody done all the tasks of encoding in WPT format, then we want people to review to make sure what got into that plan is correct.i.e. peer review.
... To explain what is being tested for stakeholders.
... To support this last one, it's important to filter the report to focus on one AT technology.
... If you go down to first comment, I have generic format layed out.
... Assumptions is one report per pattern for now.
... Each test has title. See my second comment on the issue for example.
... Currently we only have SR now, but we can target mobile SR in the future.
... Or screen magnifier
... Anybody has thought on this?

Michael: Looks good, the only thing missing now would be adding title to command.

Matt: We can add metadata for command.People who would use this might be fine without this. But another way to think of this document is this is the spec of JAWS does for checkbox.
... That's why we want buy-in from SR developers.
... Having a broader audience for people who wanna learn. For that audience, we need more information as you mentioned. Command name would be helpful for that point.

Michael: Right, especially as we test more, I found that remembering all the key strokes will be helpful.
... I'm gonna create a issue for the thoughts I proposed.

Review of the checkbox assertion

Michael: One thing I noticed was in issue 36

https://github.com/w3c/aria-at/issues/36

Michael: First comment is We aren't saying how we navigate to the checkbox.
... I think the path to navigate to the checkbox would affect the test result.

Matt: List of command that we provide is the command that we expect users to navigate to it. So in those cases, those are the commands navigate to it.

Michael: We have tests to/through the checkbox. Test instruction says "1.Put JAWS into Virtual Cursor Mode using Insert+Z" "2.Then, perfom the task~"
... So we want to clarify the second step that tester's focus is in right place.
... Reading and interactive test has currently the same prompt.
... How you got there does matters.
... I think it's not reasonable to test every combinations of users how you got there. But at least we want to add instruction for that.

Matt: Could you add pull-request about that insight Michael?

Michael: Yes
... My second comment was about "Reading mode commands are listed for "Navigating to checkbox in interaction mode" So I think the title just got swapped.
... I can fix that in my pull request.

https://github.com/w3c/aria-at/issues/34

issue 34

<michael_fairchild> Yohta: Made some progress in the last week. A spreadsheet of assertions based on the checkbox assertions.

<michael_fairchild> Yohta: I'll review these with Jon later this week

<michael_fairchild> Yohta: Jon also made a branch so that we can start to contribute this

<michael_fairchild> Yohta: I do have a few questions

<michael_fairchild> Yohta: Q1: jaws automatically switches to interaction mode with interacting with from controls and menu items. Should I write my tests in interaction mode?

<michael_fairchild> Matt: We have a test for checkbox where we test switching modes

<michael_fairchild> Matt: It would be good to model your tests on the checkbox tests, where we have tests for each combination of task and mode

<michael_fairchild> Matt: In some cases, screen readers switch modes when clicking a menu item, and some switch modes when it receives focus.

<michael_fairchild> Matt: I think menubar will switch modes and open the dropdown when it is activated

<michael_fairchild> Matt: maybe to make the tests easier to read we should say 'menu bar item'

<michael_fairchild> Yohta: once it is open, will all tests for sub menu items be in interaction mode then?

<michael_fairchild> Matt: no, because screen reader users can change modes explicitly

<michael_fairchild> Yohta: Q2: for the test about navigating through a checkbox, is a similar test and assertion needed for menubar?

<michael_fairchild> Matt: the one you want is called 'navigate to checkbox in interaction mode' - where the assertions are that the menu bar name, as well as the menu item role and name.

<michael_fairchild> Matt: so the menubar will have an extra assertion for the menu bar name

<michael_fairchild> Matt: there will be another assertion for the haspopup property

<michael_fairchild> Matt: so there will be 5 assertions on it

<michael_fairchild> Yohta: thank you, I think I'm good. I'll continue to work on it.

<michael_fairchild> Jon: for the combobox reference, should we update that with the latest code?

<michael_fairchild> Yohta: the example code that is.

<michael_fairchild> Matt: yes, but let's wait until after we have merged that update in the apg

<michael_fairchild> Matt: It looks like we have some inconsistencies in wording in our tests.

<michael_fairchild> Matt: before we get too far, we should do something to make sure that we are being consistent

<michael_fairchild> Yohta: what is the expected timeline for finishing the menubar test plan?

<michael_fairchild> Matt: we hope to have it ready by the 3rd week of Feb

<michael_fairchild> Jon: we can work on it this week

This is scribe.perl Revision: 1.154  of Date: 2018/09/25 16:35:56  
Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/

Guessing input format: Irssi_ISO8601_Log_Text_Format (score 1.00)

Present: Matt_King zcorpan shimizuyohta jongnud
Found Scribe: shimizuyohta
Inferring ScribeNick: shimizuyohta

WARNING: No date found!  Assuming today.  (Hint: Specify
the W3C IRC log URL, and the date will be determined from that.)
Or specify the date like this:
<dbooth> Date: 12 Sep 2002

People with action items: 

WARNING: IRC log location not specified!  (You can ignore this 
warning if you do not want the generated minutes to contain 
a link to the original IRC log.)