<scribe> scribe:shimizuyohta

Issue 162: Need volunteers for Pilot test - May 11-15

Simon: We need to reschedule the test since runner isn't ready to report new result.
... We need to the week of May 25.

Matt: The original plan was week of May 11, so we'll be 2 weeks behind.
... But we can keep doing usabiliy test.

Simon: We also need instruction to how to do pilot test.

Matt: For anybody who hasn't been regular participants, we should think about mini-training session.

Simon: Yes, plus synchronous help opportunity if helpers get stuck

Matt: How are we doing for volunteer?

Simon: We have 3 people now.

Matt: 3 people except these regular members?
... Michael, will you be able to volunteer?

Michael: Yes

Jemma: What kind of plan are you thinking of for training?

Matt: We need to discuss, but I'm not worried about Jemma.

Michael: I'm curious how new test runner will live in github.
... There's some prototype needs to be fixed for new runner.

<zcorpan> https://github.com/bocoup/aria-at-report

Simon: New runner is on github. It's not based on prototype, but new implementation from scratch.But the test data is in ARIA repo.

Matt: New runner will be in W3C website.
... The harness and runner are separate thing.
... All the test cases and harness is on ARIA-AT repo.

Issue 127: Recurring agenda item: Status on test plans

Matt: Checkbox and menubar, any high-level readyness?
... For Usability test, checkbox and menubar should be fine.
... By the time we get pilot, we need at least 3 plan, adding combobox
... I was wondering if there is anything we can do to speed up test plan.
... We know we'll be tied up with combobox. Those will take at least till next week to 20th

<shimizuy_> Jon:Trying to harmonize the assertions.

<shimizuy_> Matt:We have one checkbox related changes.

<shimizuy_> Jon:We were talking about how much setup script would do stuff.

<shimizuy_> scribe:shimizuyohta

<shimizuy_> Jon:Do we wanna make less sork on script and add more instructions?

<shimizuy_> Michael:I think we agreed to the direction of adding instructions, instead of code more.

<shimizuy_> Jon:Right now, we only have one liner.

<shimizuy_> Jon:If we are to ask people to ask multiple steps, that could be tricky.

<shimizuy_> Matt:We could use string like step colon to begin each step

<shimizuy_> Jon:If that's the case, harness needs to be updated.

<shimizuy_> Matt:Actually, I think Jon's script is something needs to be updated.

<shimizuy_> Jon:New test is gonna be json object?

<shimizuy_> Michael:HTML file are more or less headers and footer.

<shimizuy_> Matt:Should we leave it in the json as it is in HTML?

<shimizuy_> Matt:That means we'll use more conventional approach for writing.

<shimizuy_> Matt:So if we use something like string step colon, that would be separator of array and steps.

<shimizuy_> Matt:I'm thinking string-like back slash is good, since it's more simple.

<shimizuy_> Matt:One problem with back slash would be end will become part of next instructions, as a SR user.

<shimizuy_> Matt:Would either of pipe or backslash appear in instructions?

<shimizuy_> Jon:I wouldn't use that.

<shimizuy_> Simon:I think we shouldn't tweak the data format.

<shimizuy_> Jon:Runner needs to consume that data,

<shimizuy_> Simon:We don't need to split the string in json.

<shimizuy_> Simon:Tweaking the data format will negatively affect the engineering of the runner.

<shimizuy_> Michael:We were doing research on native menubar. I did JAWS, and keep working on it.

<shimizuy_> Matt:We'll keep pipe, but tell the runner use pipe as a separator.

<shimizuy_> Simon:I can consult engineering team, and report that back.

<shimizuy_> Jon:Second question is where should setup script start then.

<shimizuy_> Matt:We should do what we can interms of focus and state.

<shimizuy_> Jon:In general, if we were giving people instructions, we want to convey if there is specific spot people start test.

<shimizuy_> Matt:If it's a reading mode test and menu needs to be visible, we wanna make it visible.

<Jemma> let testData = {

<Jemma> setup_script_description: getSetupScriptDescription(test.setupScriptDescription),

<Jemma> setupTestPage: test.setupScript,

<Jemma> applies_to: getAppliesToValues(test.appliesTo),

<Jemma> mode: getModeValue(test.mode),

<Jemma> task: getTask(test.task),

<Jemma> specific_user_instruction: test.instructions,

<Jemma> output_assertions: assertions

<Jemma> };

<shimizuy_> Jon:Goal will be put focus as close as possible to carry out the following test?

<Jemma> the mode, getModeValue isn't it about test mode?

<Jemma> so that the mode can be switched accordingly?

<Jemma> Jon's dicussion is more about focus management, I guess.

<shimizuy_> Matt:Yes, I think you have the right understanding. But we need actual implementation to make sure we are on the same page.

<shimizuy_> Simon:I'd like to ask review for test plan we have for AT developers, which was due few weeks ago.

<shimizuy_> Matt:We wanna make sure CG is good for that before we ask AT developers.

<shimizuy_> Matt: Menubar is not ready.

Issue 162: Need volunteers for Pilot test - May 11-15

Issue 52: Create tests for APG design pattern: checkbox

<shimizuy_> Matt:For checkbox, we need to talk about is the feedback we got from Apple, that VO doesn't have a mode.

<shimizuy_> Matt:I don't think that's how we wanna approach it. If you're VO user and saw mode on title, that's confusing.

Michael: Would this be addressed in the production runner?

<Jemma> mck: I am not sure wha the best way forward here.

Matt: In the prototype, the harness doesn't use to apply the field which tests are presented.
... We'll have production runner for test.
... If you filter out a SR, that filter would work in the report.
... And the prototype runner will be used mainly for us.
... If SR has a mode prerequisite, are we okay with mode-less ver. of test, primarily for mobile test in mind?
... Plan A: Not having mode in the title, but if mode applies, harness will adjust the title accordingly.
... So in the case of VO, the mode will always be N/A

Simon: I think that's test creation scripts that write the title.

Jon: I think having different assetions for VO will be much simpler.

Matt: Every interaction mode row is an identical copy. The only difference is applies to column.
... That approach put more onus on test writers.
... So this means making adjustment in menubar
... There's more test, since I separated the reading through key with arrow key.
... I think that is probably easier thing for people to reviw, so plz merge PR Michael.
... We need people to scrutinize this.
... I'll ask CG members to investigate checkboxm, and hoping to release this on AT developers next week.
... The Applies to coulumn says "VoiceOver" so it needs to say"VoiceOver_Mac"

Michael: Yes, this will be addressed in the PR.

Simon: I have update on json insructions. It won't affect runner, but only for harness.

This is scribe.perl Revision: 1.154  of Date: 2018/09/25 16:35:56  
Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/

Guessing input format: Irssi_ISO8601_Log_Text_Format (score 1.00)

Present: Matt_King michael_fairchild shimizuyohta isaacdurazo Jemma
Found Scribe: shimizuyohta
Inferring ScribeNick: shimizuyohta
Found Scribe: shimizuyohta
Inferring ScribeNick: shimizuyohta

WARNING: No date found!  Assuming today.  (Hint: Specify
the W3C IRC log URL, and the date will be determined from that.)
Or specify the date like this:
<dbooth> Date: 12 Sep 2002

People with action items: 

WARNING: IRC log location not specified!  (You can ignore this 
warning if you do not want the generated minutes to contain 
a link to the original IRC log.)