<Lauriat> Thank you for signing up to scribe today, Jennifer! https://www.w3.org/WAI/GL/task-forces/silver/wiki/Scribe_List
Jennifer: scribe
<scribe> scribe: Jennifer
testing
Thursday and Friday meetings are cancelled next week
due to the U.S. holidays
Please sign up to scribe in December! :D
<Lauriat> https://docs.google.com/document/d/1sugAtqie_x1XqHDZo1Im7ftDNllWeRV_ty4PULeoTV0/
Sign up to scribe: It's on the main page of the wiki near the bottom with the meeting information. https://www.w3.org/WAI/GL/task-forces/silver/wiki/Scribe_List
Wilco discussing Writing Testable WCAG 3.0 Outcomes
Expect to survey in AG.
Chuck said it is undecided.
Highlights…
Document starts with brief definition or explanation of testable outcome.
<Chuck_> I think Jeanne wanted to chime in.
Four qualities:
address a clear user need
little to no room for interpretation
quick & inexpensive to test (possibly using tools) can't take hours
must be easy to learn
brief section on why testability matters
then defined a process to creating these outcomes, in 5 steps
1. come up with examples & edge cases
2. create a rough outline of the outcome
3. simplify where you can
oops!
3. define the terms used in the rough outline
4. simplify where you can
5. complete the how to & methods
6. get feedback and iterate
Hey, that's 6 steps!
In the document, the material is expanded.
<Lauriat> qv?
Testing qualitative aspects
Rather than 'legible' WCAG uses color contrast to evaluate legibility.
To summarize, if you can read more than one meaning into a word, then that's an ambiguous thing.
Series of definitions in the Google doc.
<Fazio> sample sizw?
Next section is on the soundness of outcomes.
David Fazio question: For subjective and qualitative stuff, have you considered sample size requirements?
Wilco: there's no "this many people need to agree". We do it by discussion. There are no shortcuts.
A process we go through over and over until enough people have no objections. It's building consensus.
How Nuanced: a trade-off between accuracy and complexity. No clear way to do that, yet.
Shadi: one of the aspects that addresses subjectivity is to reduce the decision space. i.e., text alternatives.
separate text alts for buttons versus others, then the subjectivity would be reduced.
Wilco: you want to break it down, have a narrower definition.
But we can definitely break it up to do a better job of expanding on that.
<JF> https://www.w3.org/WAI/tutorials/images/decision-tree/
JF: might be out of scope for this conversation, such as the subjectivity of text alts
we have the EO's decision tree
there's subjectivity in determining a text alternative
using that guidance, then that helps contribute to getting consistency
you'll never have 100% agreement, but if we can evaluate that text alt is there and the decision tree was used, it could be worked into conformance somehow
Wilco: the last thing I wanted to highlight was subjectivity vs objectivity. we use those terms somewhat inconsistently.
when ACT uses objective, it basically means programmatic.
"subjective" does not mean vague or poorly defined. things can be subjective and still be tested consistently.
<SuzanneTaylor> +1 in general to this guidance - I think it will be very helpful toward moving forward with writing outcomes
Jeanne: I think you covered it really well.
Chuck: any comments or questions before moving on?
David Fazio: I've been receiving complaints lately that if we need a glossary then it isn't plain language.
Jennifer: we are working cross-culturally so the glossary helps with that.
Wilco: in ACT we try to use only definitions of things that are fairly intuitive. if you can't guess, then it's probably something that should be defined.
JF: I disagree with that. I think glossaries are an important part of what we do. Standards are intended to remove ambiguity.
<Fazio> I'm just bringing awareness of the public's comments
Understanding specific ways of using terms, glossaries are likely an important part of what we're doing.
Shawn: we can review later.
<Lauriat> https://lists.w3.org/Archives/Public/public-silver/2021Nov/0025.html
picking up from last Friday, issue 304
<Lauriat> 304 - conformance claims need feedback mechanisms
<Lauriat> https://github.com/w3c/silver/issues/304
Can someone pick up where that left off?
Jeanne: I took an action item to check with maturity model to see if a user feedback was included.
I checked with David, eh said they did.
David joined this call for that.
Would someone please write that answer so we can document?
David: we orig had it in one
place
... so people could voice commentary/complaints, so feedback
could be actionable
<Fazio> ohttps://docs.google.com/document/d/1Y5EO6zkOMrbyePw5-Crq8ojmhn9OCTRQ6TlgB0cE6YE/edit#
David: so we put it in a
different place. The maturity model is public so you can check
https://docs.google.com/document/d/1Y5EO6zkOMrbyePw5-Crq8ojmhn9OCTRQ6TlgB0cE6YE/edit#
... we're still editing the Github version.
<Lauriat> https://github.com/w3c/silver/issues/352 - multiple conformance models vs simplicity
Jeanne will write the answer to that for next week./
Chuck: Encourage others to take ownership, so Jeanne isn't doing 99% of the work.
Suzanne: I can write the response, I'll write it now.
Jeanne: Awesome! Thank you, Suzanne.
<Lauriat> https://github.com/w3c/silver/issues/352 - multiple conformance models vs simplicity
Lauriat: have we been reading through and discussing?
<Lauriat> "What do you see as the possible tradeoffs in using a flexible conformance approach? Would you want to see multiple conformance models? The tradeoffs are that you’ll have some loss of quality in areas. I wouldn’t want multiple conformance models. I would encourage simplicity."
Jeanne: yes, that's what we've been doing, unless really complex. opening up for discussion.
<Chuck_> +1
Lauriat: We want to compare & contrast, and get down to one conformance model.
Jeanne: I think the person who
contributed that interpreted that as multiple, so I think it's
just a clarification.
... the flexibility is in the types of tests that we are
writing and in the moving away from 100% or failure.
... so that's more the principle of what we meant by flexible,
rather than multiple conformance models.
<Zakim> JF, you wanted to note that we will still have 100% or failure
JF: note that we, even with
flexible conformance or scoring model, we will still have 100%
or failure. In my mind, we do have some tolerance for some
failure (silver vs bronze)…
... this is something that has concerned me going forward, as
we move to bronze, silver, gold… bronze will be less than
perfect.
Jeanne: that isn't what went in the FPWD, I think we should stick to that, this is the direction we're going, and that may change.
<Wilco> +1 to Jeanne
Jeanne: plz put that in, that we do have disagreement, and that may change.
Chuck: we will have one conformance model, rather than this will be the conformance model.
JF: the question in my mind is still if gold is 100%, then silver & bronze are less than 100%.
Jeanne: bronze was regulatory, silver was usability, gold was [something]
Chuck: I don't think the answer
to this needs to address that to provide a satisfactory answer
at this time.
... I'll write something, and we can tweak if as necessary.
<Lauriat> https://github.com/w3c/silver/issues/386 - if we allow 95% to pass, people will stop at 95%. Proposal for addressing minor errors
Lauriat: this comment: I do think that organizations will stop at 95%. call me cynical, but I think that's true. I'm not a technical person in this area. is there a way to weight this that things have to be 100%. is there a way things can be missed without failing… details in link.
<JF> I won't comment further here, but again, issue #386 is asking for less than 100% - but 100% at Bronze, or Silver, or Gold?
Jeanne: it's an interesting idea.
Not exactly what he said… trying to identify what the images…
what I think is an interesting idea is the idea of looking…
instead of defining what's the task… which we've struggled
with… it might be easier to say,
... something that's used only once or not common, it might be
easier to… it's much easier to count the negatives instead of
the positives. might be interesting idea to explore for the
group working on tasks, defining processes.
... I know Rachael has worked on that.
Rachael: we've only just started
to discuss.
... we've brought up the difference between critical error and
the concept of priority against any given item within a
set.
<JF> who decides the priority?
Rachael: instead of top down,
think you have an image, in the order of things it may have a
different priority… I have it noted as something to discuss
with group sometime after Thanksgiving.
... I'll take it to respond.
<Rachael> @JF, priority would be set by working group.
JF: who decides priority
of…
... it's dependent on how it's used, so subjective/contextual
discussion. Industry wants less subjectivity.
Rachael: I understand, John.
Jeanne: I don't think we can
close it, I think we can transfer it to the group working on
it.
... transfer to the process group, but not close it.
... you can take this off your to do list
<Lauriat> https://github.com/w3c/silver/issues/394 - the effort of proving conformance can overshadow development.
Rachael: I run that subgroup, it's still on my to do list.
<Lauriat> We are concerned that the effort of proving conformance would overshadow the effort of conformant development. Will you provide guidance for large organizations for testing and showing results?
Lauriat: my sense is the answer
to this is yes, we will have some amount of this, here's how
you can put together a statement of conformance.
... this is something we haven't completely solidified,
yet.
... so a response to this, so far, the way we've approached
this convern
... it's certainly not the first time we've heard that this
might be a lot of work
... as we start, things will start off complicated — what can
work — then we can refine it from there.
<jeanne2> +1
Lauriat: volunteer to draft that response?
<Fazio> we are conscientious of scalability
Lauriat: 30 seconds to find a volunteer
<Fazio> in our Maturity Model at least
Lauriat: I will draft it.
This is scribe.perl Revision VERSION of 2020-12-31 Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/ Guessing input format: Irssi_ISO8601_Log_Text_Format (score 1.00) Succeeded: s/ @stevelee - I think you may be in another call.// Default Present: jeanne, JF, jenniferS, shadi, Wilco, SuzanneTaylor, Chuck_, Makoto, sajkaj, GreggVan, kirkwood, Rachael, Jemma, Lauriat, Jennifer, ToddLibby Present: jeanne, JF, jenniferS, shadi, Wilco, SuzanneTaylor, Chuck_, Makoto, sajkaj, GreggVan, kirkwood, Rachael, Jemma, Lauriat, Jennifer, ToddLibby, jeanne2 Found Scribe: Jennifer Inferring ScribeNick: Jennifer WARNING: No meeting chair found! You should specify the meeting chair like this: <dbooth> Chair: dbooth WARNING: No date found! Assuming today. (Hint: Specify the W3C IRC log URL, and the date will be determined from that.) Or specify the date like this: <dbooth> Date: 12 Sep 2002 People with action items: WARNING: Input appears to use implicit continuation lines. You may need the "-implicitContinuations" option. WARNING: IRC log location not specified! (You can ignore this warning if you do not want the generated minutes to contain a link to the original IRC log.)[End of scribe.perl diagnostic output]