<jeanne2> agenda order is 1, 4, 3

<chrisloiselle> Unable to scribe, apologies.

<JF> scribe: JF

GitHub issues

<Lauriat> https://www.w3.org/WAI/GL/task-forces/silver/wiki/Issue_Processing_Report

jeanne: start with reports

looking to make this a regualr starting action

finaly have all email issues in github - can now identify all issues posted

currently 283 issues received - so far closed 12

now have all email issues entered into github, all issues have been triiaged

14 editorial issues awaiting closing - need to wlrk out HOW to do that

referred 220 issues to sub-groups

[jeanne lists numbers by sub-group]

in effort to keep this brief, this has been labeled by sections -

Jeanne has also noted some items that need to be discussed on calls

asking subgroups to start drafting responses, and prepare for internal survey

<jeanne> https://drive.google.com/drive/folders/1WQ5wfAQ30cfS3VfwspmvyN-mzhhyY9l-

[jeanne provides 'folder' for draft respopnses

If done in GitHub, label "Ready for Survey"

(If you do not have write access, contact Jeanne)

MC: may be easier to ask for write access: email M. Cooper or Jeanne with email address and request
... it's not a lot of work to add people

ITI issue 458

<jeanne> https://github.com/w3c/silver/issues/458

jeanne: the issue I saw today that we can discuss and perhaps close is comment from ITI

they have provided a detailed email response

"Types of tests: We are looking for more appropriate terms to distinguish between these two types of tests and welcome suggestions."

we are using atomic and holistic - issues there

suggest replacing with module testing and end-to-end testing

<Fazio> doesn't seem like plain laanguage

JS: we've also been looking at "component" as well

+1 to JS

JS: atomic is essential the things we do today, 'holistic' was more 'usability testing'

SL: not a fan of using "component", as that starts to overlap with other concepts. Atomic also came from ACT efforts (language)

<Zakim> Lauriat, you wanted to say I don't like component but don't have a great idea.

SL: like idea of 'end-to-end" testing

<sajkaj> +1 to "end to end"

but don't have a great idea for replacing component or atomic

<johnkirkwood> +1 to end to end

<Zakim> JF, you wanted to note we need to test all types of fragment code

<Chuck> jf: Contradicting what Shawn said, we need to test content free material. Component library, design system. There are just primitives used to create end to end.

<Chuck> jf: If a company wants to put together a design system that when used properly to generate compliant code, we don't have a way for them to do that.

<Chuck> jf: While testing end to end is useful, testing beginning to middle is useful.

<Chuck> sl: We need a way to support that kind of testing, as it is equally important.

<Chuck> jf: component or atomic may be difficult, but I believe we need to have unit tests that do not encapsulate the full end to end.

<Chuck> sl: I agree, terminology is difficult.

<Chuck> jeanne: they are recommending "module" and "end to end".

<Fazio> module sounds like online training

JS: want to remind that the issue is around module and end-to-end testing... terminology

WF: speaking of ACT... they use atomic in the context of "small"

it has to be a small, specific test: i.e. "Title describes topic of page" breaks down to many smaller questions - multiple small steps to test

<Zakim> Chuck, you wanted to ask that "atomic" may be purposely like ACT "atomic"

so how ACT uses "atomic test" likely different

CA: was going to suggest something else, but +1's Wilco

JS: thinks 'atomic' may be a good choice (root of term) - thinks maybe the 2 terms together is what is causing the concern

<Fazio> I like holistic

JS: notes that end-to-end may be a concern as well.

but it would need to be scoped: 'end-to-end in a process' [sic]

JS: thoughts about module?

<Fazio> It sounds akward to me

Jeanne: [something about a module] - narrowly scoped test

<jeanne> ITI recommends "module"

<Lauriat> +1 to sajkaj

<Fazio> do we want to use functional tests?

WF: wondering if this is the conversation to be having, since we don't have any examples of holistic tests today

<Fazio> +100

<Wilco> +1

<Chuck> +1+1+1

<sajkaj> +1 to SL, and could ask for more, too

SL: +1 - we should respond with an interim response, noting that we're still fleshing this out

<Fazio> lol

<KimD> +1

<laura> +1

<johnkirkwood> +1

jeanne: Will put this in minutes: looking for a volunteer to draft response

<scribe> ACTION: Chuck Adams will draft response for issue Issue #458

<Lauriat> Thanks, Chuck!

Options for Bronze, Silver, Gold

<Lauriat> https://docs.google.com/document/d/1BjH_9iEr_JL8d7sE7BoQckmkpaDksKZiH7Q-RdDide4/

SL: we left off at Option 6 or 7...
... will addressOption 7

<Lauriat> Direct link to option 7: https://docs.google.com/document/d/1BjH_9iEr_JL8d7sE7BoQckmkpaDksKZiH7Q-RdDide4/edit#heading=h.arrgu8j4ygno

JF: notes we already discussed Option 7 on last call

jeanne: we had stopped at questions/pros & Cons on Option 7

notes there are no issues to work through

<Zakim> Chuck, you wanted to ask I've put together a draft, do we want to review here? in list?

CA: In AGWG we usually put dr4aft response in Issue directly, but we may not want to do that for all issues. For this however I think we'd be OK

<Chuck> proposed response: Thank you for your comment. This terminology is intended to express concepts that have yet to be fully explored. The working group will maintain a list of different possible terms for these concepts, which will guide us as we explore these concepts in greater detail.

jeanne: suggest to wait and we will revisit

ITI issue 458

adds proposed response

<Chuck> proposed response to issue 458: Thank you for your comment. This terminology is intended to express concepts that have yet to be fully explored. The working group will maintain a list of different possible terms for these concepts, which will guide us as we explore these concepts in greater detail.

<Lauriat> +1

<laura> +1

<sajkaj> +1 to CA

<Fazio> vey polished

<KimD> +1

<Jemma> +1

Options for Bronze, Silver, Gold

Jemma: we can move on to next option: if anyone notes something please comment

<Lauriat> Direct link to option 8: https://docs.google.com/document/d/1BjH_9iEr_JL8d7sE7BoQckmkpaDksKZiH7Q-RdDide4/edit#heading=h.3d3oai30qz26

SL: yes, and/or a "combo" - a little from this, a little from that
... reads option aloud

<Chuck> Description: Bronze (Low subjectivity) Guidelines that are not subjective Subjective guidelines on a rudimentary (“everyone can agree”) level

<Chuck> Silver (Slight improvement in non-subjective. Bulk of subjective work falls here.) Lower margins of error for large sites on Guidelines that are not subjective Subjective guidelines require evidence: a written explanation including some evidence such as: User testing/Remediation/Re-testing report User testimonies ???

<Wilco> "Everyone"?

<Chuck> Gold (Significant improvement in non-subjective.) No margins of errors on Guidelines that are not subjective Non-subjective guidelines require efficacy evidence: a written explanation including some evidence such as...

<Chuck> Successful user testing User testimonies A published schedule of AT-testing by people with AT-relevant disabilities and publicly-available issue triage

<Chuck> Cons: (same as Option 6) It does not provide a structure that does more to provide fair treatment of all disability needs. It is too easy in this structure to assign the hard-to-measure needs of people with low vision and cognitive disabilities to Silver and Gold levels, where they will be considered optional and not be met.

<Chuck> This is a civil rights issue. Unless we can put in structure to ensure that the hard-to-measure needs of people with disabilities will be addressed at bronze level, then we should not go forward with this proposal.

<Chuck> scribe: Chuck

<Zakim> JF, you wanted to ask where all of that documentation would be stored? (Must be public "Read")

jf: Published testimonies schedule of at testing, where would it be published, how, what format, accessible to who? Does it need to also be WCAG compliant?
... There's a lot of questions. We have conformance statements that are adhoc, VPATs. It sounds like you need to publish a book, and where would that live?

shawn: Yes, by the time you put together all of this, it would be a large conformance statement. We could note this as an issue, what kind of options would be acceptable?
... Or is it something we have to figure out here?
... But a good aspect of this to note.

wilco: I'm surprised by this proposal. Why would we tie conformance level, which seems how it would indicate how well you are doing, to how subjective a requirement is? What's the relationship to how accessible a website is?

<JF> WF: surprised by this proposal. Why would we tie the conformance level (how well you are doing) to a subjective requirement. Seems unrelated

<Lauriat> +1 to Wilco

<JF> +1 to wilco

<jeanne> +1

shawn: To elaborate, this turns bronze, silver and gold into a redux of A, AA, and AAA. In terms of difficulty and complexity of the test, rather than measuring accessibility.

<JF> +1 to Janina - opinons can be challenged

sajkaj: Blew me away is that issue tracking has to be publicly available? That's fine for open source, but I don't think conformance should be tied to making ticket system public.

<Lauriat> +1 to sajkaj with bells on

+1 to sajkaj

<JF> +1 to JS

<Zakim> jeanne, you wanted to address civil rights issue

<JF> +1 to Jeanne

<chuck_> scribe: chuck_

<Fazio> +100

<Lauriat> +1 to jeanne

jeanne: tying to bronze, silver, gold will not be good for regulators as we have to constantly change.
... It is a civil rights issue that disabilities need to be treated equally and fairly. Subjective conformance perpetuates that civil rights issue. Now that we know it exists, we have to fix it..

shawn: Anything to discuss in the pro section?

<Zakim> JF, you wanted to respond to janina

<KimD> +1 to Jeanne

jf: I want to address Janina's point. The problem we have here is the moment we have subjective evaluations we have to make an assertion.

<KimD> +1 to option 8 returning to an A/AA/AAA kind of structure

jf: The technology to doing that is secondary. The point remains, if you are making an assertion or a claim, and part is subjective...

<Fazio> +1

jf: All that information needs to be made public. Otherwise it's "trust us, we can't show you." That's a brick wall, and that can't be challenged.
... anytime we use subjective in scoring, we need a publicly available document, a way to validate those assertions.
... I need to be able to see what the assertion is. At least I've got data. "trust us" will break.

janina: Courts are full of cases which have to pull out proprietary data before they move forward. Such as encryption protection.
... that's not defensible.

wilco: My comments on another topic. Others should go first.

shawn: Respond to the assertion that put of lack of transparency would put up a wall. I don't think it does. You can put forth evidence...
... "based on our evidence"... we find these issues. Your assertion doesn't look credible. Pressure would be on person making assertion with no evidence.

<sajkaj> +1 to SL

shawn: I don't think lack of public testing details builds up that wall. Janina's point, the assertion could be called into question with evidence.

<Zakim> Lauriat, you wanted to respond to the wall I don't see in the way of validating assertions

<jeanne> +1 to SL - when taken to court, they have to present evidence. We don't need to require a public report.

david: I wouldn't call it subjective, we back it up with evidence. We create anonymous profiles. You can still keep information private.
... Not subjective, based on actual research and data available. I see no problem what-so-ever. You anonymize your participants.

<Zakim> JF, you wanted to ask whether we want to increase legal challenges/discussions

<Jemma> I still see the burden of people to prove that it is not accessible

jf: My concern, Janina to your point, we start to build a mechanism that people need to go to court more often, we are creating a nightmare.
... I still want to add on... recognize this is not the topic, if we are going to have assertion statements, we need to standardize how they are made and recorded.
... I'll leave as an open question. If we use public assertions, what format will they be recorded in? txt, pdf, excel?
... Needs to be standardized.

<Jemma> + 1 for recorded assertion to be opn

shawn: +1 to your proposal that we log this as a larger question.
... related to us working through the options. We'll need to discuss this regardless of what we call the conformance.

sajkaj: Agree we need to formalize what we expect in an assertion, and also that we are not in the business of enforcement.

shawn: I look forward to the conversation at a later point.

<JF> we may not be responsible for enforcement, but out standard needs to support that

wilco: Subjectivity... I have an idea that ACT came up with. Distinction between subjective and ambiguous. Lot's of things in accessibility are subjective.
... does an alt attribute accurate describe an object? To what extent? Does the title describe the page? That's subjective.

<Lauriat> +1 to Wilco, thank you for clearly articulating that important difference!

<Jemma> I think it is more about level of accessiblity knowlege and training rather than subjectivity.

wilco: Where it gets difficult is where things get ambiguous. If there is more than one way to understand what is written, or what the requirement is, that's an issue.
... That is often times one of the challenges with the new requirements. The language isn't specific enough on what needs to be tested.
... If there's more than one way to understand, that's where these differences come from.

<Fazio> good point wilco

wilco: subjectivity is fine and it works. It's where wcag is ambiguous that causes issues.

+1 I am now smarter!

<Jemma> +1

<jeanne> +1

shawn: jeanne is noting in the issues to work through.

<Jemma> sorry. I

<Jemma> can someone unmute me?

<working on remote unmuting>

Jemma: Clarification queestion on what wilco said. What do you mean by...
... Not sure "if there's more than one way to understand what is written". How is that requirement relevant to this discussion?
... What do you mean by requirement?

wilco: Like a sc is a requirement, or an outcome is a requirement. The thing you have to meet.

Jemma: Thx.

shawn: any more q or comments on option 8? Not enough time to introduce option 9.

+1 to returning 5 minutes.

david: Turning back to the standardizing. I'm thinking in terms of automated testing. Do we have a standard for recording automated testing? If not, why push here?

shawn: Let's hold until we have the larger conversation here?
... There's a few q that we would need to work out before we standardize test results as part of it.

David: Just thought there may be something in existence.

wilco: I have an idea of another option for conformance model levelling.

<many want to hear about it>

wilco: Can I get access... I already have it!

jeanne: Everybody who's active gets access.
... this discussion gets public comment access.

sajkaj: Is there a way to track changes in a folder? Or should I ask for a chaser email anytime a doc is added?

shawn: Best if we over-communicate. Good practice to send email to list.
... There are some ways of tracking changes, but not super efficiently, rather manual.

<Zakim> JF, you wanted to ask if weekly agendas note/track addtions

jf: I like, the minutes have a summary. Going into meeting... trackable items are included in the agenda. "Look at folder... xxx... 3 new additions"

<Lauriat> +1 to JF, definitely a good idea to have these tied to agenda items, I try to include links on them whenever possible.

jf: Put's more work on the chairs, but capturing summary of things that have changed since last time we met will address those concerns.

sajkaj: If I have to find 3 new docs, that hasn't met my concern. I'm still doing a diff.

jeanne: shawn and I do the summaries differently. Do you see a need for us to indicate more docs?

sajkaj: I do want to read wilco's.

jeanne: hopefully he adds to this doc.

This is scribe.perl Revision VERSION of 2020-12-31
Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/

Guessing input format: Irssi_ISO8601_Log_Text_Format (score 1.00)

Succeeded: s/JS [/ Jeanne: [/
Succeeded: s/addres /address/
Succeeded: s/And it's becoming a civil rights issue/It is a civil rights issue that disabilities need to be treated equally and fairly. Subjective conformance perpetuates that civil rights issue.  Now that we know it exists, we have to fix it./
Default Present: jeanne, sajkaj, ToddLibby, JF, Chuck, MichaelC, Francis_Storr, Lauriat, ChrisLoiselle, sarahhorton, KimD, jennifer_strickland, CharlesHall, AngelaAccessForAll, Laura_Carlson, shari, Rachael, SuzanneTaylor, johnkirkwood, JakeAbma, mgarrish, Fazio, Jemma, joconnor
Present: jeanne, sajkaj, ToddLibby, JF, Chuck, MichaelC, Francis_Storr, Lauriat, ChrisLoiselle, sarahhorton, KimD, jennifer_strickland, CharlesHall, AngelaAccessForAll, Laura_Carlson, shari, Rachael, SuzanneTaylor, johnkirkwood, JakeAbma, mgarrish, Fazio, Jemma, joconnor
Found Scribe: JF
Inferring ScribeNick: JF
Found Scribe: Chuck
Inferring ScribeNick: Chuck
Found Scribe: chuck_
Inferring ScribeNick: chuck_
Scribes: JF, Chuck, chuck_
ScribeNicks: JF, Chuck, chuck_

WARNING: No meeting chair found!
You should specify the meeting chair like this:
<dbooth> Chair: dbooth


WARNING: No date found!  Assuming today.  (Hint: Specify
the W3C IRC log URL, and the date will be determined from that.)
Or specify the date like this:
<dbooth> Date: 12 Sep 2002

People with action items: chuck

WARNING: Input appears to use implicit continuation lines.
You may need the "-implicitContinuations" option.


WARNING: IRC log location not specified!  (You can ignore this 
warning if you do not want the generated minutes to contain 
a link to the original IRC log.)