Silver COnformance Subgroup meeting -- 06 Aug 2019

<jeanne> https://docs.google.com/document/d/13A8mGMnQujfEVqcw_LmAUYT8DDq_qW0TNcHxmCHd0io/edit#heading=h.4c2v4xktzeh

<Chuck> scribe: Chuck

Jeanne: I'll open requirements doc and we'll read the whole thing.

<jeanne> https://w3c.github.io/silver/requirements/index.html#multiple-ways-to-measure

Jeanne: <reads from doc>
... Essentially we are writing a sc to determine if we met multiple ways to measure.

JF: Do we have multiple ways of measuring?

Jeanne: Not yet, no content yet. But let's assume that we have them. We have several drafts that have rough versions of other ways of measuring.
... We know it's possible, let's assume we have it, how do we test for the existance.
... Then we need to test 2nd clause, that more needs of people are met.
... 2nd part might be easier to start with. In some ways... first part I think is that we need to test it with other ways of measuring.

JF: I don't have a screen in front of me, listening and paraphrasing. One of the points Charles has made is rather than look at it from disability types
... He wants to look at it from functional perspective. We go beyond pass/fail that WCAG gives today.

Jeanne: It's true/false.

JF: What I heard is increasing disabilities or multiple disabilities is focusing on the disabilities and not the functional outcomes.
... Maybe we measure functional outcomes and how successful are we at that?

Angela: What will we base those outcomes on then? How will we address that if we aren't looking at the disability?

JF: Myself... it seems that it's a combination of both. We noted that WCAG is only looking at a subset of all the different types of disabilities.
... We started to expand that list to be more granular. For example - blindness, do we make a distinction between blind from birth vs acquired
... Harder to describe a sunset to someone who has never seen vs someone who had site for first 20 years of life.
... Functional outcome would be if it was adequately described to users who would require an adequate description.
... In spreadsheet I sent out there's...

Jeanne: Let's stick with multiple ways to measure. Does this relate to multiple ways to measure?

Angela: I think I'm understanding the approach John is discussing and agree with granularity. A hybrid method would help us define those test methods.

Jeanne: I've been thinking about as you were asking that Angela, the functional needs that Charles Hall were related to identifying the user needs.
... The multiple ways to measure, and that could be specific to the disability. We don't need to stay with functional need for this part, where we are testing what's in the content.

Angela: I think I understand the difference for what you are saying now. We aren't looking at the disability types to test the guidance we are giving. Or are we?

Jeanne: That depends on the specific method. Trying to think of example....
... We were looking at content meeting: pause/stop/hide. That has an impact for people with vestibular disabilities.
... Also has an impact on people with some cognative disabilities. We have that identified in the user need.
... The next part of it is how do we test if it met the user needs. The way we test is multiple ways to measure.

<jeanne> https://docs.google.com/document/d/1JKn_SvyHwT6cpglkANJ0Lty74KtWLY7T2rmKyWUPtYM/edit

Jeanne: Draft of pause/stop/hide (early draft) does have methods in it with potentially new tests.
... Take a look at that...

Chuck: So a question... we are discussing how to test for multiple tests in abstract?

Jeanne: Trying to answer Angela's q. I think that the answer to it is we identify the functional needs, we'll have various ways of testing whether the content meets those functional needs.
... Doesn't matter if it meets the needs of the specific disability. Chuck is right. How do we test for multiple tests.
... I think we want to say we have content written with different types of test and that fits into the conformance model. Or we have new types of tests written.
... And that works in the conformance prototype.

JF: I don't see how that's applicable. There are some things that are going to be pure... like 1.3.1.
... There are multiple ways of inspecting, automated testing tools, dom, screen reader. when we get into higher level functional requirements
... Can you perform the functional outcome. I don't know if there are multiple tests to determine that.
... Other than does it meet it (yes or no)? Have you performed the functional...
... The end of the day the q is "did you do that"? The answer is "I did or I didn't".

Jeanne: That's one type of test. Are there other types of tests?

Chuck:

JF: When the higher level test is "did you do a higher level test to see if the functional need was met", the cognative walk through will have multiple steps.
... If we get that point, we have a checklist of steps to complete larger test.
... I keep tripping up at... there are test that ultimately come to true/false. That's what the ACT task force is doing.
... They have a framework that allows us to express tests in a uniform way that boil down to yes/no on success.
... When we talk about if there are multiple tests, the answer is is there more than one way to determine if the functional outcome is met.

Jeanne: We are talking about measuring the prototype. You mentioned the coga test, the functional outcome test. Rubriks, sliding scale...
... We are going to need examples of all of these to evaluate the conformance model. What are some others that we know people with cognative issues and vision issues will need?
... That couldn't get into WCAG? They needed a more complex test?
... I'm thinking is we'll have samples of tests that can be used to show conformance to silver. We'll have these samples and figure that out.

JF: I question this requirement at a higher level. This requires "multiple ways to test". But can we have multiple ways to test multiple ways of testing?

Jeanne: No, we just need to show that we have multiple ways to test.
... I'm proposing we have samples of these x types of tests, we will show that we have samples of these tests.
... Do you think that adequately covers that we have proven we have multiple ways to measure?

JF: For any given requirement...

Jeanne: This is for the requirements doc. How do we show we met the requirements in the requirements doc.
... We just need to show functional outcomes work, rubrik works, ...

<someone> can we stop at those three?

Jeanne: I'd like to get more of them janina.
... How can we test that we met the requirement that we served more people with disabilities?'

Jennison: More needs than what?

Jeanne: WCAG.
... We could have sample sc in new things...
... Low vision and coga sent us sample sc that couldn't get into wcag. We could show that the new ones that we are writing as part of content
... That those can be included in silver.
... Do you think those 4 sc would be enough, or do we need more?

Angela: Good start.

Jeanne: Another part of this. If we could ask low vision and coga to give us feedback and find out if they feel it's meeting more of their needs.
... Asked task forces for examples of things they wanted to get into WCAG but couldn't because of structural issues.

JF: Do we have concrete examples?

<Chuck_> jeanne: Yes, we have 4 specific examples from COGA and LV. THey are linked on the wiki.

<Chuck_> scribe: Chuck_

Jeanne: Let's make a todo item that as soon as we finish this list we eval one of the four sc.

Chuck: Functional outcome, coga walkthrough, rubrik.

<jeanne> Multiple ways to measure

<jeanne> That we have more ways to measure than just true false statements. We will have samples of the types of tests that will be used to measure

<jeanne> Functional outcomes

<jeanne> Cognitive Walkthrough

<jeanne> Rubric

<jeanne> That we serve more needs of people with disabilities

<jeanne> Test that the 4 success criteria proposed by Low Vision TF and Coga TF can work in Silver

<jeanne> Ask for feedback from COGA and LVTF

Chuck: So if we can do 2 of those three?

JF: What does "it" mean that we need multiple ways of measuring.

Jeanne: The it is are we meeting the requirement doc.

JF: Not trying to be beligerent. Multiple ways to measure what...
... ARe we saying we need multiple ways of measuring multiple requirements, or we need to measure conformance...
... some of the things may not have multiple ways of measuring compliance, others will.

<Chuck__> jeanne: Not every success criterion needs multiple ways, but multiple ways need to be available.

<Chuck__> scribe: Chuck__

Jeanne: Want to start next one?

<jeanne> Create a maintenance and extensibility model for guidelines that can better meet the needs of people with disabilities using emerging technologies and interactions. The process of developing the guidance includes experts in the technology.

Jeanne: How do we test this?
... That might be one way to demonstrate that we met this requirement (some sc for VR)

Chuck: But something like that all the way through...

Jeanne: Pick an emerging technology and prove the model for silver works for new technologies.

Chuck: How about VR as one and voice interface as another?

<jeanne> Janina: There are meetings scheduled at TPAC to discuss more of how semantic data that can be captured. Currently, there isn't a way to capture semantic information from 3D VR/XR.

<Chuck> Chuck: So a way to measure this would be...

<Chuck> Chuck: to pick at least 2 emerging technologies, take sc for those through the entire cycle, demonstrate or work product with

<Chuck> Chuck: SMEs in the area of that emerging technology... If our work product meets with their satisfaction, and we have more than one, we would call that a success.

<jeanne> JF: With emerging technologies, we may not be able to apply WCAG to emerging technologies.

<jeanne> ... we may not have multiple ways to test.

<Chuck> time check

<Chuck> over the hour

<jeanne> Janina: We can choose ones that work well with existing SC.

<jeanne> Chuck: We will have the "tools in our toolbox" to test a success criterion in the best or most appropriate way for that success criterion. Some will have a true/false test, some will have one new type of test, others could have multiple tests. We can write tests that are best for that SC, instead of being limited to a true/false test.

- DRAFT -

Silver COnformance Subgroup meeting

06 Aug 2019

Attendees

Contents

Summary of Action Items

Summary of Resolutions

Scribe.perl diagnostic output