– DRAFT –
Silver Task Force & Community Group

Review drafted changes to reflect new direction for tests

Rachael: I can provide an overview

Rachael: We have discussed in subgroups and joint meetings how different types of tests can help expand on WCAG 2
… we have a description of this as something to explore

<Rachael> rawgit: https://github.com/w3c/silver/pull/624/files

Rachael: on the left you see what is being removed on the right what is being put in
… next is the actual resulting document

<Rachael> https://rawgit.com/w3c/silver/update_test_section/guidelines/index.html

Shawn: lets go through types of changes at a high level

Removal of scoring related content

<Lauriat> rawgit: https://github.com/w3c/silver/pull/624/files

<Lauriat> https://rawgit.com/w3c/silver/update_test_section/guidelines/index.html

Rachael: The first of the challenges is the removal of content that did not seem to be working
… the red on the left of the raw git shows what was removed
… I will walk through the proposed removals
… a lot of this is about the critical errors
… anything around actual scoring is being proposed as a removal
… down in the conformance section, we are proposing removing anything about scoring there, to reset it and include later after the scoring work

Rachael: The next groups of changes is related to adding to the test section

Jeanne: In section 1.2 Status Levels, the names have changes as we have worked with AGWG COGA etc
… the first is placeholder content, which is expected to be replaced and hidden by default, but alot of people have asked for placeholder content in order to help provide context and mental framework to understand those items we are currently working on

<Lauriat> rawgit rendering of the section: https://rawgit.com/w3c/silver/update_test_section/guidelines/index.html#section-status-levels

Jeanne: the next is exploratory and that is mainly what we have so far, and the feedback should be about the proposed direction
… exploratory is also hidden by default
… there is a control to expose placeholder and exploratory content
… there is a refining section and a mature section, with advice about the type of feedback that is appropriate

Janina: This sounds similar to W3C recommendation levels, is that an issue?

Jeanne: This is about a granular paragraph level, more so than full documents like the W3C levels

Jeanne: It gives people an idea of the real status of the document

Janina: That is helpful

<jeanne> update_test_section/guidelines/index.html#testing

<Lauriat> Testing section: https://rawgit.com/w3c/silver/update_test_section/guidelines/index.html#testing

Jeanne: In section 4, Testing (be sure to press the button to reveal placeholder and exploratory content if you don't see it)
… <goes through content of that section>
… right upfront that this is exploratory
… <goes through section about testing items, views, processes, full site, etc>
… there are definitions and examples of each of these

Sarah: I had a question about process
… what is part of a process, the full view or just the items needed to complete the task

Rachael: great question, but we had been talking about each of these separately
… since it exploratory, this will be part of further exploration

Sarah: I was thinking more that there would be items that are not part of the process

<JF> Is/are process evaluations intended to be part of conformance evaluations?

Rachael: No, items in a view that are part of a process but not needed for the process would not be part of the process in our current thinking

Janina: One concern is when users want to go back and forth within a process to make decisions

<JF> Other actions would be to increase/decrease quantities of item in shopping cart, modify the order in other ways (color, size, style, etc.)

Jeanne: We might need a meta level of processes, what is the function of the aggregate, and what needs to be included at this higher level (purchase/return/etc)

Janina: Common patterns might be helpful

Rachael: In the scoping group, we thought we might want to collect patterns and subpatterns. will bring back idea of aggregate level

Wilco: What type of feedback are you currently looking for. Why would we want to define these now, before we have examples?

Rachael: In order to show the direction (this is exploratory) and to document what we don't know yet

Rachael: The feedback we are looking for goes from word-level through over all ideas

Rachael: I'm editing as we go along, let me know any thoughts on the changes

JF: Is the intention of the small process or component tests to contribute the score, or are they just part of how to test?

Rachael: The intention is to help support multiple ways to test and yes, they are expected to be part of scoring and conformance

Maryjom: We might want to look into what designers call these things

<janina> +1 to Mary Jo

<janina> I recall reading once that a computer picsel has a different name in television

Rachael: will capture this as "feedback from communities on terminology"

<jeanne> types of tests

Jeanne: Next we will move into discussion the types of tests

<maryjom> For example "user scenario" or "user workflow" may be more understandable than "process".

Jeanne: there are 4 types, from very objective to most subjective

< reads the types >

<janina> suggest /address/are intended to address/ ?

Wilco: I think the word "objective" might be controversal
… this is how ATC uses the word
… in in WCAG 2, this word is used for inter-tester-reliability
… so we might want to use a different word

<JF> +1

Wilco: Also, I don't think we can have objective things in normative requirements in WCAG 3

Jeanne: Will the tests be normative?

Rachael: currently they are not normative, but that can be discussed

Wilco: could use quantitative vs qualitative

Shadi: quantitative might create expectations that the tests are not manual

<janina> +1 to asking for feedback on naming things

Shawn: Let's create a note about naming this concept

<Wilco> +1

Rachael: Could add this a survey question for Tuesday

Jeanne: <continues through the definitions and examples>
… methods are technology-specific

<Lauriat> Objective tests

<Lauriat> Conditional tests

Jeanne: Silver research and design sprint indicated that conditional tests were needed

<Zakim> Rachael, you wanted to point out that WCAG 2 has these conditional tests for example alt text

Rachael: Both of the first tests are already in WCAG 2.2 - is the alt-text there, does it describe the image

JF: We also received feedback asking for less subjectivity in conformance testing

<Zakim> Wilco, you wanted to respond

Wilco: This comes into issue where different people define "objective" differently

<Jem> +1 wilco

<Zakim> Rachael, you wanted to point out the purpose of this approach is to address the feedback and solve the point of view

Wilco: the intent is good, but wording needs to make it clear that we are not reducing inter-tester-reliability

<Jem> I think the core concepts of each testing is solid.

Rachael: Some ways to ensure inter-tester-reliability are outlined here, such making sure a process was indeed followed

<Jem> +1 Rachael re: make it clear that we are not reducing inter-tester reliability

Makoto: concrete examples of each test type would make this easier to understand

<Wilco> +1 I too would prefer an example of each before we add this as exploratory. Can live without though.

Makoto: this section highly on English vocabulary

Shawn: Could we add an editor's note linking to one or more of the examples we have been working on?

<JF> @shawn - yes

<Rachael> +1 to adding an editor's note with examples.

Shawn: we are working on these tests for existing SC

<Zakim> Lauriat, you wanted to suggest linking to some working doc or docs

<Jem> +1 with solid examples

<Lauriat> Convention tests

Jeanne: Next is convention tests, and this is the first type of test that moves beyond the types of tests regularly used in WCAG 2
… < reads section on "Convention Tests" >
… an example might be a organization with a design pattern that includes a few accessibility requirements (contrast/focus visible and their own style guide on plain language)
… < reads description of methods for Convention Tests>

<Zakim> Rachael, you wanted to add reading level and visible controls examples

Rachael: An example could be that a company chooses an appropriate reading level, then meets that
… another might be a convention for how invisible controls can be found (such as a company always uses a specific arrow)

JF: What safeguards will be in place to ensure these rules are going in the right direction and not the wrong direction?
… in addition, how do we have consistent measure of conformance?

Rachael: We would dictate a requirement, such as You must test against a reading level

JF: What if a company chooses, for example, 7 point footer text?

Jeanne: This is not intended to allow product teams to over-write the first 2 types of tests, it is to allow, for example, a different reading level for emergency info versus a university professor's blog

JF: What do we have in place to ensure these conventions are correct?

Shawn: The idea is create a structure to support organizations in creating what they need to go further, but not to replace the WCAG 2-style tests

<Zakim> Lauriat, you wanted to add an industry convention example of link rendering

Shawn: there is still much to work needed for how this would work with conformance

<JF> +1 Shadi

<Lauriat> +1 to Shadi

<JF> A better 'convention' example would be around multipole h1's on the same page

Jemma: My understanding is that these types of tests go together, they are not separate islands, and the new tests add transparency
… also +1 to Makoto

<Jem> These are great discussions.

JF: to me a convention would be around the conversation about whether or not to have multiple H1s on a page - it would be nice, but we need to limit the types of conventions that are allowed