– DRAFT –
Silver Task Force & Community Group

Timeline review with Virtual Face to Face and publication

ShawnL: We wanted to review timeline. We want to keep a good cadence of publishing drafts.

We want to publish end of May. Working backward, need CfC week before then and a week before that for the changes. Then a week of May 4th would be discussion of changes.

We need to make sure the Face to Face changes we want to be before April 30th.

Chuck: That would be in the form of discussion and calls.

Jeanne: Subgroups who want changes in response to their comments need to have their work finalized in pull requests by April 30th. If you are unable to due pull requests, please let us know by April 28th.

Wilco: When AG approved the last draft, I believe AG would review the next draft before we published next draft. Is there enough time for us to do that?

Chuck: On review, I don't have the details. I believe AGWG has been involved and had ability to express feedback as everyone else.

What is your definition of a throughough review?

Wilco: We can have this discussion in AG. A lot of parts of WCAG 3 was not in survey and I thought we were going to have the ability to review that way.

ShawnL: A lot of work will be incremental work and changes. There should be plenty of time to review on what is needed to review.

<bruce_bailey> Shawsn just answered my question.

Rachael: My understanding is that we'd bring in AG more in more into the conversation, example the AG April 29th meeting. Bringing content more to AG for review.

<Wilco> +1, sounds good

Let us bring that back to AG for discussion for mutual understanding.

Chuck: On CfC it went through a process when we were getting objections. We can review on how we can improve upon that.

<Wilco> raise hand to volunteer

ShawnL: On the working session on April 29th, we want to identify github knowledgeable folks so we can get responses in place on call or thereafter.

ShawnL: Action item to make a list with Jeanne to identify people for github.

Bronze Silver Gold options

<Chuck> https://docs.google.com/document/d/1BjH_9iEr_JL8d7sE7BoQckmkpaDksKZiH7Q-RdDide4/

<Lauriat> https://docs.google.com/document/d/1BjH_9iEr_JL8d7sE7BoQckmkpaDksKZiH7Q-RdDide4/edit#heading=h.juk4rs86lt47

ShawnL: Talks to option 12 , as we had talked to option 13 last week.

<Chuck> Option 12: If an automated test can test for it, it is required at this level (caveat: where there are levels e.g. 4.5:1 contrast vs. 7:1, though an automated text may be able to test for both 4.5:1 and 7:1, 7:1 wouldn’t be required for the “Automated level”).

<Chuck> Expands as automation becomes more robust - as we develop new automated techniques for testing things, those test become part of the Automated level within a not unreasonable amount of time after discover, allowing for broad dissemination of the technique and incorporation into website test automation

<Chuck> Not frozen to the current technology. Includes AI (unless extremely expensive computing costs). Updated at least once a year

<Chuck> Pros for option 12

<Chuck> Speaks to GitHub #451

<Chuck> Addresses some of the key challenges faced by highly dynamic websites

ShawnL: I think the caveat is for minimum required.

<Chuck> Organizations do this (limit efforts to what a tool reports) anyway. Making an official level for it: Provides clarity about what it is and what it is not. Unlike option 11, option 12 allows this level to become more and more robust overtime.

<Fazio> my concern is if one tool has automated tests that another does not

<Chuck> No cons

<Fazio> what if it costs more?

<Chuck> Issues to Work Through

Sheri: Who's automated tests? In house, purchased?

<Chuck> What if one tool has a proprietary test or far excels at AI testing?

<Chuck> Is there a higher level which defines a cap for the automated level? (e.g. Imagine a future where all of WCAG 3.0 can be tested through automation, what is included then in the Automated Level?)

ShawnL: Automated tests that we identified could work.

ShawnL: This could be a con, in terms of who owns the automation tests.

Janina: Threshold conformance topic and necessary and insufficient. It goes into detail of testing and versions. I think there interest in basic threshold and automated is part of that. I.e. this much is able to be automated.

Janina: This is coming up in conformance subgroup , I hope we get there for the 29th of April. Third party content is a part of discussion. Legal prohibitions and use cases talk to that in our work.

<JustineP> How can "extremely expensive" be quantified?

<Wilco> https://act-rules.github.io/pages/implementations/overview/

Wilco: Possible solution to Sheri's challenge. ACT Community group and what can be done automatically , Wilco provides link to implementations.

<Fazio> Not all tools are created eequal. Costs are a factor. I don't think we should require automated testing

Wilco: Rather than W3C stating this can be automated this can't be, we can rely on the automaters to report that information to us.

ShawnL: No matter what we do, this is extremely helpful.

JF: On Sheri's question, I think the answer is the spec owns the test. The content owners need to reference the spec.

The steps are within the spec. We need to provide the ability to the content author the ability to understand if they've met the test criteria or condition. We own the test in writing the spec.

<Fazio> +1 same question

Sheri: My concern is say that Level Access can evaluate a test in one way that others don't have ability to yet. Then people testing would have to purchase "X" to evaluate "Y". I'm just concerned with for profit vs. non profit and construction of tests.

JF: Tools are used to prove whether or not text alternatives are in place. The wording of tests is editorial, in that the ACT task force was created to unify that language. If this then else statements ultimately get you to succeed or failures.
… The outcome is in the spec, here is where we are , here is where you need to be and this is how you do it should be in spec.

<JenniferC> +1 for John Foliot comment on ACT Rules harmonization and reason for Siteimprove's involvement.

<Zakim> jeanne, you wanted to mention that it proposes a minimum bar for passing any guideline

Jeanne: One interesting idea was that automation be a minimum for all guidelines not just this test for this guideline. Peter mentioned this in conformance subgroup.

I.e. first step, run automation tests and fix first before doing any other individual guideline levels.

We don't want to undermine advances of WCAG 2

<Fazio> I'm against telling people you HAVE to run automated tests

<Fazio> Having it as an option, sure, but mandate no

Jeanne: We have to be careful it a level, but it is an interesting piece of the process.

<JustineP> Agree that automated testing should be an option, not a mandate.

Janina: Thank you, Jeanne. I wanted to say the use case is coming out of conformance. The examples are there in regard to alt text and structures to contain it. The third party may need a CMS , etc. Sheri, I think we are talking to what we commonly agree on and how we apply it.

<Zakim> Rachael, you wanted to add a con that standard will always be behind the technology

Rachael: On making a level around automated testing, I think automated will always be advanced so tying it to a level is concerning.

<Zakim> JF, you wanted to note that mandated automated tests may actually be a race to a higher level than where we are today...

JF: Mandating automated tests would be a race to the bottom topic. It appears that that is being misconstrued. Framing of automated tests can be positive and negative. It is not universal applicable that it is a race to the bottom.

<Zakim> Lauriat, you wanted to raise the issue of conformance level determined by the ease of creating automation tooling rather than a level that reflects actual accessibility for people with disabilities

<Fazio> JF made a good point

<Wilco> +1

<Fazio> +1SL

ShawnL: Creating a level of conformance on ease of creating tooling vs. level of accessibility with people with disabilities may be tying back to the incorrect main interest. We should better support automation but tying it conformance level is a good idea.

<JF> what if we tied automated tests to a minimal level of "Score", not conformance

Chuck: We cover at Oracle, the importance of automation to catch bugs. It is beneficial to include it , but perhaps not at a conformance level.

<jeanne> +1 to including it at scoring level

JF: 10:01 AM <JF> what if we tied automated tests to a minimal level of "Score", not conformance

<Lauriat> +1 to JF's suggestion

<JF> +1 to Janina. Tools to get better, not to get "caught"

Janina: Anything normative is hard to version. If we are encourage competition and delivery for users, having a way to version up the threshold testing and keeping it out of conformance is a good idea. Tying to scoring is interesting idea.

JF: I think you hit on the subject. We are talking to conformance but talking to different level of conformance. The scoring is key and I've been stressing about it for a long time. The score encourages the vendors to compete. I.e. measurable increase.

Goal is to make things better and not driving toward perfection.

<sajkaj> +1 to SL

<Rachael> +1 to SL

<JF> +1 Automated tests simplify and "speed-up" testing, but the tests are the tests...

<Zakim> Lauriat, you wanted to distinguish action from result (using a tool vs. getting test results)

ShawnL: On scoring and conformance . We should set the levels and scoring to reward to automation but it doesn't matter if they used automation or manual approach to arrive at scoring. The actual use of automation is more related to a maturity model aspect rather than conformance.

Wilco: The conformance model is tied to scoring. I believe 3.5 score average on functional . So if we need to tie that in , we'd need to have normative in both places. As long as scoring is tied to conformance, we don't have flexibility in automation that we are talking to.

Chuck: To Wilco, in that we tied scoring to conformance, do we have to include automation?

Wilco: Everything you need to come to a conclusion needs to be normative, the way it is built right now.

If we want to include automation right now in a level, the automation piece should be normative based on how scoring is set at moment.

Chuck: Because scoring is normative, automation would have to be part of that .

Wilco: Yes.

Janina: I would trust ACT to develop the rules around automation and think it would be a trailing edge. Versioning would be important.

<Wilco> living standard please

<Sheri_B-H> +1 Janina

<Lauriat> +1 to Wilco's "living standard please"

Jeanne: On Janina's thought, I think we should be doing dot x development on guidelines.

<Zakim> jeanne, you wanted to say that WCAG2 tests are not normative, so I don't think Wilco's argument holds

<Wilco> suggest we have that discussion separately. Off topic, yet important

Jeanne: On what has to be normative, on the WCAG 2 examples, the tests in WCAG 2 are not normative, the SC are. In the scoring of WCAG 3 , scoring is normative . We have to be careful on to what is normative and what is not regarding testing and maintaining documents.

ShawnL: Plus one to Wilco having this as a discussion on issues to work through.

ShawnL: On pro and cons, Pro: Links to github 451 , broader interest in this sort of a level.

pro - dynamic websites applicability.

pro - clarity on what it is and what is not. More robust over time.

ShawnL: On cons - maintenance concerns, creating a lower level i.e. race to bottom, automation being ahead of standard, and automation piece in prior discussion thread above.

Issues to work through -

What if one tool has a proprietary test or far excels at AI testing?

Is there a higher level which defines a cap for the automated level? (e.g. Imagine a future where all of WCAG 3.0 can be tested through automation, what is included then in the Automated Level?)

JF: On race to the bottom, I don't think we should use that terminology anymore.

<Wilco> +1

<SuzanneTaylor> +1 to agree that such phrases leave too much open to interpretation, as well

ShawnL: Jeanne is re-wording document as we speak. Note taken.

ShawnL: Any other comments on any of these?

<Zakim> JF, you wanted to object to the phrase "race to the bottom"

Jeanne: It would be helpful on options 1 , 2 and 3 . That is where we started . I think talking about 1 and 2 in detail would be important.

<Chuck> +1

<Rachael> I think we need to go through option 5

<Chuck> +1 to review again. In future meeting? Friday?

ShawnL: Do you want to revisit or do you want to talk to those as a starting point for pros and cons ?

Jeanne: I think doing so as we have for other options would be worthwhile on Friday.

Jeanne: Rachael states we also need to talk to option 5.

<SuzanneTaylor> https://docs.google.com/document/d/1lonY6dmgfqNYglJ7OhZVusc16q1Jv8i_mGDZtFi_bkc/edit#

Jeanne: Suzanne was talking to options and what topics were addressing "X". Topics were talked to in questions.

Suzanne: I don't know if it makes in simpler or more complex but a different way of looking at data. I will complete adding the options that aren't included here.

Rachael: I think there are more questions than these. I really appreciate and like the approach here.

<Lauriat> +1, I think this can help immensely to work through this!

ShawnL: I wanted to highlight that Makoto added comments on option 13 and answered questions.

<Zakim> Chuck, you wanted to ask about first 30 minutes of AGWG

Chuck: Reminder , the AGWG , first 30 minutes talk toward FPWD issues raised. I encourage participation.
… this helps unify the two groups work together.

<bruce_bailey> Thanks @SuzanneTaylor for that Google Doc

<Chuck> https://www.w3.org/2002/09/wbs/35422/wcag3-weekly-responses-survey/

Janina: Do we have a link to survey? Thank you.