Substantial Conformance

29 Oct 2020


PeterKorn, Wilco_, sajkaj, sarahhorton, John_Northup, bruce_bailey
PeterKorn, wilco_


Agenda Review

<PeterKorn> scribe:PeterKorn

<scribe> scribe: PeterKorn

sajka: continuing from last week. Start with process conversation, then most of time on substance

Google Docs Procedures

<Wilco_> scribe: wilco_

sajka: Asking for some patience with Google docs. Feel free to mark sections of the document with comments.
... Would like to control those edits. Don't know how well this works in Google docs.
... Took a few days to get the document up, making Jeanne the owner and getting it as an official Silver document.
... It now is and everyone in this group should have edit access.

<PeterKorn> scribe: PeterKorn

John: Does not have edit access

Problem Statement & Principles Discussion

sajkaj: Where to start?

PeterKorn: Suggests that we focus first on Key Principles

sajkaj: Problem Statement may be the first outcome, once we have key principles. Objections?
... Hearing no objection, look at proposed principles
... look at principle #1...

Wilco: want to be clear that "Substantial Conformance" in these principles is a placeholder term

sajka: open to that term being provisional. Notes it is in use in the wild (US at least)

sarah: wondering if we could describe how this would used out in the world, in terms of designating a resource as "substantially conferment"

sajkaj: presumes it is the same as in the model of FPWD. Up to whoever wants to make a conformance claim, to make the claim they feel comfortable making.

PeterKorn: suggests we keep "Substantial Conformance" as the working term, given history from Silver research & what is in FPWD. Suggests we also track open questions that arise

Bryan: looked through principles. Where it has really resonated, when going through the exercise of design principles, they help quickly align individuals on where the guidance is.
... good principles help you say "no". Help you understand "is something substantially conferment". Shouldn't be broadly stated truisms, but can be applied broadly.
... Feels a good sent of principles will help us get along way down the path.

sarah: agree that we move through principles; can feel OK as long as we put the concept of "substantial conformance" within quotes - so we will revisit it as a descriptor of what we are talking about.


sajka: feel free folks with edit access to make typo/etc. edits. But for substantive changes, please work first through comments

PeterKorn: just because a Key Principle is in the doc for "Substantial Conformance" doesn't necessarily mean that the principle isn't already covered by the conformance model in FPWD.

sajka: hearing no objection to text of principle 1, move to #2

Wilco: WCAG 3 doesn't currently scope to websites. Rather, to views & processes.

John: suggests replacing website visitors with users

sajka: scoping question. Does substantial conformance require you to define a scope?

sarah: think it is a great question; one we will need to explore
... typically an evaluation isn't of a website; rather it is of a few flows within a website or app. If I were to evaluate a website, it would probably be ~30 pages out of 10k, and come up with a result that

<bruce_bailey> I would love to have Janina's opinion on the functionality of Google doc Suggesting mode (inserts and deletions visually marked up as underlines and strikethrough).

sarah: ... that would have some level of conformance. But to be able to say something about that in a credible way, don't think I could say "my website is substantially conferment"
... ... could say "these features" or "these task flows" are substantially conferment.
... ... we are never evaluating
... so if I were an owner of a resource and I want to get to a "substantial conformance claim", I'd need to scope the claim so I could say something meaningful. E.g. "of the pages"

<Wilco_> Peter: I suggest we add a new section in which to capture key questions to discuss.

<Wilco_> Sajkaj: agreed

Wilco: Feels the question is phrased oddly. It is always scoped - even if the scope is the entire product. So the question is perhaps better stated as "can the scoping be different than WCAG 3 conformance scopgin"
... ... the way I think about site conformance - agree with Sarah. If you aren't evaluating every single page, then you can't claim conformance for the entire thing.
... what is missing in WCAG is some way to say "we believe it is conformance because we've done x,y,z, and found q". "Based on our evaluation, we are concluding it is likely conferment"

John: following onto the conversation about certifying an entire site - take a representative sample of entire content, and try to hit on every different kind of layout. Then when they certify,
... ... they certify with text "based on what we sampled". Then look at user impact. Missing ALT text is a key failure, but if in a part of page that isn't important, then may hit "substantial conformance"

sajkaj: : another Q we might need to address. When you make a claim, who do you make it for? Who is the consumer? A better understand of who that is might give us comfort as to different

sajka: ways we might make claim. Point #2: important point from Wilco on what an entity understands its procedures. Have multiple methods in WCAG 3, and will have more. Some new method
... ... may allow it. Most important point: someone who makes this kind of claim, there is a way to go through item by item and find issues.

sajkaj: ... you have to be wiling to defend when it comes to specific cases. Is it going to be defensible? That is what I think we are talking about. Not so much a different way of evaluating,

sajka: ... but a way to agree to the claim.

Wilco: thought, which probably applies to entire conformance model. What if we had an option in conformance for samples? You can say "we've used this sampling method"?
... that sampling method includes some level of accuracy. But based on that, you might be able to claim conformance over entire website. Was involved in a study on sampling of web a11y
... ... and what jumped out - already the first page gives you a lot of accuracy. Within first 5, you almost always know if the site will be conformant. If you find nothing in first 5, likely you won't
... ... find in next 15.

PeterKorn: suggests we stick with process for now of going through Key Principles. Find out if we are missing any. See if we can agree on all. And only then look at solutions.

sajkaj: we may have a problem we need to solve. If "Substantial Conformance" is an alternative to bronze/silver/gold, then it needs a name.

sajka: so we may wind up with WCAG 3 having an alternative approach, and need a more clearly name.

<Wilco_> peter: I suggest we do that by adding as our first key question: is substantial conformance the right term, and does the term in WCAG 3 need to change?

<Wilco_> Sarah: suggest certification model, because it is based on the LEED model

<Wilco_> Sarah: Names help a lot to keep things separate

<Wilco_> Sajkaj: for us I think that is enough

<Wilco_> Peter: Because Silver research identified folks wanted the term Substantial conformance and we have it in a note, I think it's worth keeping the term for now.

Summary of Action Items

Summary of Resolutions

[End of minutes]

Minutes manually created (not a transcript), formatted by David Booth's scribe.perl version (CVS log)
$Date: 2020/10/29 17:05:16 $

Scribe.perl diagnostic output

[Delete this section before finalizing the minutes.]
This is scribe.perl Revision of Date 
Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/

Guessing input format: Irssi_ISO8601_Log_Text_Format (score 1.00)

Succeeded: s/leads/LEED/
Default Present: PeterKorn, Wilco_, sajkaj, sarahhorton, John_Northup, bruce_bailey
Present: PeterKorn Wilco_ sajkaj sarahhorton John_Northup bruce_bailey
Regrets: Rachael
Found Scribe: PeterKorn
Inferring ScribeNick: PeterKorn
Found Scribe: PeterKorn
Inferring ScribeNick: PeterKorn
Found Scribe: wilco_
Inferring ScribeNick: Wilco_
Found Scribe: PeterKorn
Inferring ScribeNick: PeterKorn
Scribes: PeterKorn, wilco_
ScribeNicks: PeterKorn, Wilco_
Found Date: 29 Oct 2020
People with action items: 

WARNING: IRC log location not specified!  (You can ignore this 
warning if you do not want the generated minutes to contain 
a link to the original IRC log.)

[End of scribe.perl diagnostic output]