<michael_fairchild> agenda item: seek consensus in what to include in assertions

<michael_fairchild> agenda item: discuss short term deliverables and how to accelerate the project

<michael_fairchild> scribe: mf

Definition of Assertion deliverables

<shimizuyohta> https://a11ysupport.io/

Michael's email on this topic:

https://lists.w3.org/Archives/Public/public-aria-at/2019May/0007.html

MF: Updated a11y.io support project. Looking for feedback.

Considered aria-details.

Not current with all our recent discussions, e.g., doesn't have all preconditions and post conditions.

This is just an experimental project.

There are must, should, and may assertions.

Is wording understandable? Is must/may terminology used appropriately?

Is addition of rationale for each assertion helpful?

Also updated architecture. E.g., can define assertions at the feature level.

For instance, aria-details is a feature, html elements can be a feature in this model too.

Can reference a feature from ARIA, HTML, and CSS for one assertion.

<michael_fairchild> https://a11ysupport.io/tech/aria/aria-details_attribute

Info about the feature and related tests and summary of assertions.

And summary of support, which is a result of the tests bubbled up to the feature level.

<michael_fairchild> https://a11ysupport.io/tests/tech__aria__aria-details

Link to a specific test

Summarized assertions.

The output is a list all the commands used to navigate to target element that has aria-details.

Reviewed structure of output.

Described run test flow, start by describing what tech you are using to test.

Yota: When you ran this with Narraotor or NVDA, there was a fail b/c aria-details was not conveyed.

MF: correct

Yota: where did the expected result come from?

mf: SR must convey presence of aria-details, for instance, is what I thought it should do.

there is no standard that describes what that expectation is.

that is one of the outcomes of this group/project.

MF: when form is submitted, it creates a github issue with the data.

curious what you think of the assertions?

Is this direction we watn to take?

<michael_fairchild> https://a11ysupport.io/tests/tech__checkbox-two-state

<michael_fairchild> https://github.com/A11yance/aria-query

<michael_fairchild> mck: we could include some information of the aria feature in the feature description, such as where the name is sourced from, etc

<michael_fairchild> mck: i like the inclusion of the rationale for assertions

<michael_fairchild> mck: the rationale in some sense should support the decision of "MUST", "MAY" or "SHOULD"

<michael_fairchild> mck: I like the lens that you have here, because you can take the same assertions and apply them to many different test cases

Accelerating pilot project phase -- next steps

<michael_fairchild> mck: we have had a lot of discussions of what this might look like. I wonder if we are in the position to be able to create a solution

<michael_fairchild> mck: I want to do this fast, but this is a complex problem

<michael_fairchild> mck: what should our next deliverable be?

<michael_fairchild> mck: should we try to do more manual testing in spreadsheets before we dive into design?

<michael_fairchild> Yohta: what are you envisioning and what do you mean by efficiently?

<michael_fairchild> mck: something like what Michael put together, but we will needs to create assertions and collect results

<michael_fairchild> mck: I was previously thinking that an assertion would be associated to a specific command defined in APG (the command created by the author as opposed to the assistive technology)

mf: If you are putting priority on getting a deliverable where we are getting test results, we definitely need the assertions, test results, procedures for getting test results as well as scoring.

If we need a place to start and we want to start quickly, I'm OK with this group adopting my project.

We could hire a vendor to make it better match.

One approach beyond that could be to focus on one pattern at a time.

someone could be repsonsible for defining assertions for one example in the APg.

Another person could test, then rinse and repeat and learn as we go.

<michael_fairchild> mck: there might be value in testing the two state checkbox example with Michael's method vs Yohta's method

<michael_fairchild> mck: we might not want to test the name computation, or create a single assertion for the computation and provide what we think is expected

This is scribe.perl Revision: 1.154  of Date: 2018/09/25 16:35:56  
Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/

Guessing input format: Irssi_ISO8601_Log_Text_Format (score 1.00)

Present: shimizuyohta Matt-King

WARNING: Fewer than 3 people found for Present list!

No ScribeNick specified.  Guessing ScribeNick: mck
Found Scribe: mf

WARNING: No date found!  Assuming today.  (Hint: Specify
the W3C IRC log URL, and the date will be determined from that.)
Or specify the date like this:
<dbooth> Date: 12 Sep 2002

People with action items: 

WARNING: Input appears to use implicit continuation lines.
You may need the "-implicitContinuations" option.


WARNING: IRC log location not specified!  (You can ignore this 
warning if you do not want the generated minutes to contain 
a link to the original IRC log.)