<jeanne2> agenda order is 1, 4, 3
<chrisloiselle> Unable to scribe, apologies.
<JF> scribe: JF
<Lauriat> https://www.w3.org/WAI/GL/task-forces/silver/wiki/Issue_Processing_Report
jeanne: start with reports
looking to make this a regualr starting action
finaly have all email issues in github - can now identify all issues posted
currently 283 issues received - so far closed 12
now have all email issues entered into github, all issues have been triiaged
14 editorial issues awaiting closing - need to wlrk out HOW to do that
referred 220 issues to sub-groups
[jeanne lists numbers by sub-group]
in effort to keep this brief, this has been labeled by sections -
Jeanne has also noted some items that need to be discussed on calls
asking subgroups to start drafting responses, and prepare for internal survey
<jeanne> https://drive.google.com/drive/folders/1WQ5wfAQ30cfS3VfwspmvyN-mzhhyY9l-
[jeanne provides 'folder' for draft respopnses
If done in GitHub, label "Ready for Survey"
(If you do not have write access, contact Jeanne)
MC: may be easier to ask for
write access: email M. Cooper or Jeanne with email address and
request
... it's not a lot of work to add people
<jeanne> https://github.com/w3c/silver/issues/458
jeanne: the issue I saw today that we can discuss and perhaps close is comment from ITI
they have provided a detailed email response
"Types of tests: We are looking for more appropriate terms to distinguish between these two types of tests and welcome suggestions."
we are using atomic and holistic - issues there
suggest replacing with module testing and end-to-end testing
<Fazio> doesn't seem like plain laanguage
JS: we've also been looking at "component" as well
+1 to JS
JS: atomic is essential the things we do today, 'holistic' was more 'usability testing'
SL: not a fan of using "component", as that starts to overlap with other concepts. Atomic also came from ACT efforts (language)
<Zakim> Lauriat, you wanted to say I don't like component but don't have a great idea.
SL: like idea of 'end-to-end" testing
<sajkaj> +1 to "end to end"
but don't have a great idea for replacing component or atomic
<johnkirkwood> +1 to end to end
<Zakim> JF, you wanted to note we need to test all types of fragment code
<Chuck> jf: Contradicting what Shawn said, we need to test content free material. Component library, design system. There are just primitives used to create end to end.
<Chuck> jf: If a company wants to put together a design system that when used properly to generate compliant code, we don't have a way for them to do that.
<Chuck> jf: While testing end to end is useful, testing beginning to middle is useful.
<Chuck> sl: We need a way to support that kind of testing, as it is equally important.
<Chuck> jf: component or atomic may be difficult, but I believe we need to have unit tests that do not encapsulate the full end to end.
<Chuck> sl: I agree, terminology is difficult.
<Chuck> jeanne: they are recommending "module" and "end to end".
<Fazio> module sounds like online training
JS: want to remind that the issue is around module and end-to-end testing... terminology
WF: speaking of ACT... they use atomic in the context of "small"
it has to be a small, specific test: i.e. "Title describes topic of page" breaks down to many smaller questions - multiple small steps to test
<Zakim> Chuck, you wanted to ask that "atomic" may be purposely like ACT "atomic"
so how ACT uses "atomic test" likely different
CA: was going to suggest something else, but +1's Wilco
JS: thinks 'atomic' may be a good choice (root of term) - thinks maybe the 2 terms together is what is causing the concern
<Fazio> I like holistic
JS: notes that end-to-end may be a concern as well.
but it would need to be scoped: 'end-to-end in a process' [sic]
JS: thoughts about module?
<Fazio> It sounds akward to me
Jeanne: [something about a module] - narrowly scoped test
<jeanne> ITI recommends "module"
<Lauriat> +1 to sajkaj
<Fazio> do we want to use functional tests?
WF: wondering if this is the conversation to be having, since we don't have any examples of holistic tests today
<Fazio> +100
<Wilco> +1
<Chuck> +1+1+1
<sajkaj> +1 to SL, and could ask for more, too
SL: +1 - we should respond with an interim response, noting that we're still fleshing this out
<Fazio> lol
<KimD> +1
<laura> +1
<johnkirkwood> +1
jeanne: Will put this in minutes: looking for a volunteer to draft response
<scribe> ACTION: Chuck Adams will draft response for issue Issue #458
<Lauriat> Thanks, Chuck!
<Lauriat> https://docs.google.com/document/d/1BjH_9iEr_JL8d7sE7BoQckmkpaDksKZiH7Q-RdDide4/
SL: we left off at Option 6 or
7...
... will addressOption 7
<Lauriat> Direct link to option 7: https://docs.google.com/document/d/1BjH_9iEr_JL8d7sE7BoQckmkpaDksKZiH7Q-RdDide4/edit#heading=h.arrgu8j4ygno
JF: notes we already discussed Option 7 on last call
jeanne: we had stopped at questions/pros & Cons on Option 7
notes there are no issues to work through
<Zakim> Chuck, you wanted to ask I've put together a draft, do we want to review here? in list?
CA: In AGWG we usually put dr4aft response in Issue directly, but we may not want to do that for all issues. For this however I think we'd be OK
<Chuck> proposed response: Thank you for your comment. This terminology is intended to express concepts that have yet to be fully explored. The working group will maintain a list of different possible terms for these concepts, which will guide us as we explore these concepts in greater detail.
jeanne: suggest to wait and we will revisit
adds proposed response
<Chuck> proposed response to issue 458: Thank you for your comment. This terminology is intended to express concepts that have yet to be fully explored. The working group will maintain a list of different possible terms for these concepts, which will guide us as we explore these concepts in greater detail.
<Lauriat> +1
<laura> +1
<sajkaj> +1 to CA
<Fazio> vey polished
<KimD> +1
<Jemma> +1
Jemma: we can move on to next option: if anyone notes something please comment
<Lauriat> Direct link to option 8: https://docs.google.com/document/d/1BjH_9iEr_JL8d7sE7BoQckmkpaDksKZiH7Q-RdDide4/edit#heading=h.3d3oai30qz26
SL: yes, and/or a "combo" - a
little from this, a little from that
... reads option aloud
<Chuck> Description: Bronze (Low subjectivity) Guidelines that are not subjective Subjective guidelines on a rudimentary (“everyone can agree”) level
<Chuck> Silver (Slight improvement in non-subjective. Bulk of subjective work falls here.) Lower margins of error for large sites on Guidelines that are not subjective Subjective guidelines require evidence: a written explanation including some evidence such as: User testing/Remediation/Re-testing report User testimonies ???
<Wilco> "Everyone"?
<Chuck> Gold (Significant improvement in non-subjective.) No margins of errors on Guidelines that are not subjective Non-subjective guidelines require efficacy evidence: a written explanation including some evidence such as...
<Chuck> Successful user testing User testimonies A published schedule of AT-testing by people with AT-relevant disabilities and publicly-available issue triage
<Chuck> Cons: (same as Option 6) It does not provide a structure that does more to provide fair treatment of all disability needs. It is too easy in this structure to assign the hard-to-measure needs of people with low vision and cognitive disabilities to Silver and Gold levels, where they will be considered optional and not be met.
<Chuck> This is a civil rights issue. Unless we can put in structure to ensure that the hard-to-measure needs of people with disabilities will be addressed at bronze level, then we should not go forward with this proposal.
<Chuck> scribe: Chuck
<Zakim> JF, you wanted to ask where all of that documentation would be stored? (Must be public "Read")
jf: Published testimonies
schedule of at testing, where would it be published, how, what
format, accessible to who? Does it need to also be WCAG
compliant?
... There's a lot of questions. We have conformance statements
that are adhoc, VPATs. It sounds like you need to publish a
book, and where would that live?
shawn: Yes, by the time you put
together all of this, it would be a large conformance
statement. We could note this as an issue, what kind of options
would be acceptable?
... Or is it something we have to figure out here?
... But a good aspect of this to note.
wilco: I'm surprised by this proposal. Why would we tie conformance level, which seems how it would indicate how well you are doing, to how subjective a requirement is? What's the relationship to how accessible a website is?
<JF> WF: surprised by this proposal. Why would we tie the conformance level (how well you are doing) to a subjective requirement. Seems unrelated
<Lauriat> +1 to Wilco
<JF> +1 to wilco
<jeanne> +1
shawn: To elaborate, this turns bronze, silver and gold into a redux of A, AA, and AAA. In terms of difficulty and complexity of the test, rather than measuring accessibility.
<JF> +1 to Janina - opinons can be challenged
sajkaj: Blew me away is that issue tracking has to be publicly available? That's fine for open source, but I don't think conformance should be tied to making ticket system public.
<Lauriat> +1 to sajkaj with bells on
+1 to sajkaj
<JF> +1 to JS
<Zakim> jeanne, you wanted to address civil rights issue
<JF> +1 to Jeanne
<chuck_> scribe: chuck_
<Fazio> +100
<Lauriat> +1 to jeanne
jeanne: tying to bronze, silver,
gold will not be good for regulators as we have to constantly
change.
... It is a civil rights issue that disabilities need to be
treated equally and fairly. Subjective conformance perpetuates
that civil rights issue. Now that we know it exists, we have to
fix it..
shawn: Anything to discuss in the pro section?
<Zakim> JF, you wanted to respond to janina
<KimD> +1 to Jeanne
jf: I want to address Janina's point. The problem we have here is the moment we have subjective evaluations we have to make an assertion.
<KimD> +1 to option 8 returning to an A/AA/AAA kind of structure
jf: The technology to doing that is secondary. The point remains, if you are making an assertion or a claim, and part is subjective...
<Fazio> +1
jf: All that information needs to
be made public. Otherwise it's "trust us, we can't show you."
That's a brick wall, and that can't be challenged.
... anytime we use subjective in scoring, we need a publicly
available document, a way to validate those assertions.
... I need to be able to see what the assertion is. At least
I've got data. "trust us" will break.
janina: Courts are full of cases
which have to pull out proprietary data before they move
forward. Such as encryption protection.
... that's not defensible.
wilco: My comments on another topic. Others should go first.
shawn: Respond to the assertion
that put of lack of transparency would put up a wall. I don't
think it does. You can put forth evidence...
... "based on our evidence"... we find these issues. Your
assertion doesn't look credible. Pressure would be on person
making assertion with no evidence.
<sajkaj> +1 to SL
shawn: I don't think lack of public testing details builds up that wall. Janina's point, the assertion could be called into question with evidence.
<Zakim> Lauriat, you wanted to respond to the wall I don't see in the way of validating assertions
<jeanne> +1 to SL - when taken to court, they have to present evidence. We don't need to require a public report.
david: I wouldn't call it
subjective, we back it up with evidence. We create anonymous
profiles. You can still keep information private.
... Not subjective, based on actual research and data
available. I see no problem what-so-ever. You anonymize your
participants.
<Zakim> JF, you wanted to ask whether we want to increase legal challenges/discussions
<Jemma> I still see the burden of people to prove that it is not accessible
jf: My concern, Janina to your
point, we start to build a mechanism that people need to go to
court more often, we are creating a nightmare.
... I still want to add on... recognize this is not the topic,
if we are going to have assertion statements, we need to
standardize how they are made and recorded.
... I'll leave as an open question. If we use public
assertions, what format will they be recorded in? txt, pdf,
excel?
... Needs to be standardized.
<Jemma> + 1 for recorded assertion to be opn
shawn: +1 to your proposal that
we log this as a larger question.
... related to us working through the options. We'll need to
discuss this regardless of what we call the conformance.
sajkaj: Agree we need to formalize what we expect in an assertion, and also that we are not in the business of enforcement.
shawn: I look forward to the conversation at a later point.
<JF> we may not be responsible for enforcement, but out standard needs to support that
wilco: Subjectivity... I have an
idea that ACT came up with. Distinction between subjective and
ambiguous. Lot's of things in accessibility are
subjective.
... does an alt attribute accurate describe an object? To what
extent? Does the title describe the page? That's
subjective.
<Lauriat> +1 to Wilco, thank you for clearly articulating that important difference!
<Jemma> I think it is more about level of accessiblity knowlege and training rather than subjectivity.
wilco: Where it gets difficult is
where things get ambiguous. If there is more than one way to
understand what is written, or what the requirement is, that's
an issue.
... That is often times one of the challenges with the new
requirements. The language isn't specific enough on what needs
to be tested.
... If there's more than one way to understand, that's where
these differences come from.
<Fazio> good point wilco
wilco: subjectivity is fine and it works. It's where wcag is ambiguous that causes issues.
+1 I am now smarter!
<Jemma> +1
<jeanne> +1
shawn: jeanne is noting in the issues to work through.
<Jemma> sorry. I
<Jemma> can someone unmute me?
<working on remote unmuting>
Jemma: Clarification queestion on
what wilco said. What do you mean by...
... Not sure "if there's more than one way to understand what
is written". How is that requirement relevant to this
discussion?
... What do you mean by requirement?
wilco: Like a sc is a requirement, or an outcome is a requirement. The thing you have to meet.
Jemma: Thx.
shawn: any more q or comments on option 8? Not enough time to introduce option 9.
+1 to returning 5 minutes.
david: Turning back to the standardizing. I'm thinking in terms of automated testing. Do we have a standard for recording automated testing? If not, why push here?
shawn: Let's hold until we have
the larger conversation here?
... There's a few q that we would need to work out before we
standardize test results as part of it.
David: Just thought there may be something in existence.
wilco: I have an idea of another option for conformance model levelling.
<many want to hear about it>
wilco: Can I get access... I already have it!
jeanne: Everybody who's active
gets access.
... this discussion gets public comment access.
sajkaj: Is there a way to track changes in a folder? Or should I ask for a chaser email anytime a doc is added?
shawn: Best if we
over-communicate. Good practice to send email to list.
... There are some ways of tracking changes, but not super
efficiently, rather manual.
<Zakim> JF, you wanted to ask if weekly agendas note/track addtions
jf: I like, the minutes have a summary. Going into meeting... trackable items are included in the agenda. "Look at folder... xxx... 3 new additions"
<Lauriat> +1 to JF, definitely a good idea to have these tied to agenda items, I try to include links on them whenever possible.
jf: Put's more work on the chairs, but capturing summary of things that have changed since last time we met will address those concerns.
sajkaj: If I have to find 3 new docs, that hasn't met my concern. I'm still doing a diff.
jeanne: shawn and I do the summaries differently. Do you see a need for us to indicate more docs?
sajkaj: I do want to read wilco's.
jeanne: hopefully he adds to this doc.
This is scribe.perl Revision VERSION of 2020-12-31 Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/ Guessing input format: Irssi_ISO8601_Log_Text_Format (score 1.00) Succeeded: s/JS [/ Jeanne: [/ Succeeded: s/addres /address/ Succeeded: s/And it's becoming a civil rights issue/It is a civil rights issue that disabilities need to be treated equally and fairly. Subjective conformance perpetuates that civil rights issue. Now that we know it exists, we have to fix it./ Default Present: jeanne, sajkaj, ToddLibby, JF, Chuck, MichaelC, Francis_Storr, Lauriat, ChrisLoiselle, sarahhorton, KimD, jennifer_strickland, CharlesHall, AngelaAccessForAll, Laura_Carlson, shari, Rachael, SuzanneTaylor, johnkirkwood, JakeAbma, mgarrish, Fazio, Jemma, joconnor Present: jeanne, sajkaj, ToddLibby, JF, Chuck, MichaelC, Francis_Storr, Lauriat, ChrisLoiselle, sarahhorton, KimD, jennifer_strickland, CharlesHall, AngelaAccessForAll, Laura_Carlson, shari, Rachael, SuzanneTaylor, johnkirkwood, JakeAbma, mgarrish, Fazio, Jemma, joconnor Found Scribe: JF Inferring ScribeNick: JF Found Scribe: Chuck Inferring ScribeNick: Chuck Found Scribe: chuck_ Inferring ScribeNick: chuck_ Scribes: JF, Chuck, chuck_ ScribeNicks: JF, Chuck, chuck_ WARNING: No meeting chair found! You should specify the meeting chair like this: <dbooth> Chair: dbooth WARNING: No date found! Assuming today. (Hint: Specify the W3C IRC log URL, and the date will be determined from that.) Or specify the date like this: <dbooth> Date: 12 Sep 2002 People with action items: chuck WARNING: Input appears to use implicit continuation lines. You may need the "-implicitContinuations" option. WARNING: IRC log location not specified! (You can ignore this warning if you do not want the generated minutes to contain a link to the original IRC log.)[End of scribe.perl diagnostic output]