<jeanne> Sampling proposal

<scribe> scribe: sajkaj

Wiki is reorganized. Each subgroup has their own page and can create new pages.

js: Notes a reorganized wiki
... Now each subgroup has their individual wiki page
... Simplified each page to make finding things easier

<Lauriat> Thank you, Jeanne!

<bruce_bailey> https://www.w3.org/WAI/GL/task-forces/silver/wiki/Main_Page

Updates to Jake's proposal

js: Jake has updated his proposal following mtg feedback

<jeanne> https://docs.google.com/spreadsheets/d/1iCJfyMtcsSq7GHmwnc4aTNguadRfGDa0H8FBZMaJpcQ/edit#gid=1825600441

ja: Started a conformance chart with scoping and scaling but want to focus on headings for now
... Considering how to note for whom we've looked at accessibility in a final number
... Notes we have a flatter structure than older wcag
... Using jf's examples still
... biggest change -- Questions now based on core principles
... 5 ver difficult; 1 very easy
... Notes these are based on the howto guideline
... So based on why and how of each guideline
... So the evaluation questions are directly based on howto content
... Trying to get to reliable and consistent results despite who's auditing
... reviews a table showing how to develop one's judgement easy/hard

[ja walks through an example]

ja: notes how the calculus works
... suggests possibility of another appropriate where minimums are required

pk: Finding this interesting and lots of valuable potential
... Asking whether this means far fewer top level items

ja: wasn't the aim in this exercise
... We weanted to look at login and search tasks, so broke out sub tasks from that

pk: maybe I'm guetting excited about the wrong thing?

ja: perhaps -- it might come out that way

pk: Feels much more understandable and workable especially for newbies

<Fazio> +1 to Peter

pk: suggests calling it structure rather than headings ...
... Wondering how to also eval importance to the task
... how to bring in the importance of the specific subtask -- importance to the overall task
... Concerned that too much granularity will end up being way too voluminous

ja: believe this is close to how audits currently work
... Notes still working on overall scoping approach; this is not that
... Notes the value of working through examples
... Recalls past challenge of wcag conformant pages that weren't very usable
... Trying to avoid that here

jf: likes this; concerned calculus gets a bit complex
... worried about supporting regulatory with this
... the usable with x functional reqs -- not seeing that here

ja: By evaluating functional needs; these the first step in functional outcome that end up with responses to those
... notes this isn't yet complete

jf: wants to see regulatory needs more integrated into scoring

ja: notes the analysis here are contributory to making those kinds of assessments

<JF> +1 to the second exercise

ja: Trying to avoid making the evaluation too complex

df: likes how this is coming along and incorporating core principles
... suggests we could analyze core principles for specific functional needs

ja: yes
... re-emphasis the questions must be based on the how and why
... if that isn't working out, it might actually mean our howtos need more work

df: So we're going to create the questions/ not unknown third parties?

ja: yes, it's our job

df: agrees this has lots of potential

jc: asks whether automated results will feed into this?

ja: yes, but not part of the current exercise
... There is more not showing here still in prep
... Definitely looking at automating all we can via ACT; even extending those some
... Most ACT rules still require some manual

jc: So, how to keep people from gaming by fixing some kinds of issues and not others

ja: looking at threshold requirements
... still working out how to build in required minimum thresholds so that without certain things none of the rest matters
... need to look at what those will be

js: like that the testing is setup and driven by the functional outcomes
... also usability comeing out of the core principles
... believes this is a very adaptable approach

<Fazio> +1 to Jeanne

js: want to hear how well this might scale for large, dynamic, complex ...

pk: biggest concern so far is how to catch the truly important parts of a site/task and not spend too much time on technical violations that don't really impede the task

<JoeCronin> +1 to Peter

pk: not clear how that fits

js: Yes, probably because this is is down in the detail weeds; but agrees the overall is important and is determined by site owner, not agwg

<Fazio> Doesn't EM Conformance doc have some info on that?

pk: Yes, I looked ahead and saw the benchmarking -- so just wanting to see more of how all that flows through
... not a criticism, just a concern about where this needs to go

ja: after this highly focussed scope we need to look at that big/overall task to look at how it all fits together

pk: Second concern -- sheer volume of detail that could explode to become too voluminous

js: agree

jc: suggesting it might be helpful to have non experts try to walk through these exercies

ja: I have those resources actually

pk: believe this approach does help newbies

ja: appreciate that because i've been working toward that kind of presentatin

pk: wonders whether another yardstick might be the web aim one million -- how does this help the world who don't understand a11y understand what needs attention better
... think we're out to broadly make the web more accessible
... so would be interesting to assess whether we're helping non expert developers/authors o a better job

s: agrees

mc: notes his hci schooling has students do these things and offers to put students on this approach to see how it works out

pk: agrees, but hard to know until we put it all together

bb: finds this fantastic
... believe this is a great implementation of conformance; but concerned this shouldn't be the bare minimum
... was hoping verbal scoring would be more approachable
... likes the idea of seeing how well students do

mc: notes a11y integration into their hci curriculum; so would expects important info from running this with them

bb: you would give this to your students?

mc: yes
... could probably recruit over the summer

js: haven't thought about giving people these spread sheets per se, it's for us so we can start to see it all fitting together
... what we give real people would be simpler

<Jan> +1 to Jeanne ... it will be simplified

<Jan> It has to get complex while you're working through the detail and then you can simplify it

js: wonders whether we should try the simplification for ordinary users fairly soon? would that make sense

<Jan> I would be interested in working on this.

ja: discusses how this works with large teams, each of whom may not become all that expert in a11y

<PeterKorn> Apologies - I need to drop.

ja: if we take into account all functional needs -- it will never be easy to be an wcag 3 expert either -- but we might walk nonexperts through the process
... notes this all goes back to the howto doc

<bruce_bailey> Basic idea from WCAG 2x: WCAG 2.1 success criteria are written as testable statements that are not technology-specific.

bb: agrees; seeing this as a real world implementation of a11y testing

<michaelcrabb> (I have to run - reminder that XR group meeting is Monday @ 9am)

bb: making silver attractive for policy makers -- this is best practices, not what it means to be wcag conformant

<Jan> There are plenty of spreadsheets behind the scenes in WCAG 2.x right now ... there's plenty of complexity with what we are currently doing.

ja: mulls on how minimum required thresholds -- this is critical, etc -- that will help with objective weighting
... explaining that will help

js: also a lot of room for tooling, automation, etc

ja: yes, but this is what makes that possible

<Jan> I have to drop for another call.

<Jan> Great job, Jake!

<Zakim> bruce_bailey, you wanted to just say my concern is about what we mean by quote wcag conformant unquote

<Jan> +1 Bruce's concern - it's an important point.

js: next steps?

ja: going through jf's pages to see impact on total scoring
... next adding a few more guidelines to see whether similarities hold up

ja; then third into a task and evaluate what is conformance, partial conformance, etc

ja; this all takes time and good examples really help

Wiki is reorganized. Each subgroup has their own page and can create new pages.

starting a subgroup to work on drafting a Representative

js: looks for volunteers to help draft a representative sample task
... points to links in today's agenda

<jeanne> https://w3c.github.io/silver/guidelines/#sampling

js: notes concern of what's still needed to get to fpwd
... we need to do that with people who've spent time becoming familiar with this approach
... email Jeanne if interested

jf: notes almost any eval service ultimately comes out to a single score
... believes single score is something site owners will want and should help regulators

<Fazio> Can't we have both a total score and score by POUR, and/or disability group with this new spreadsheet Jake made?

<Fazio> I think that would be better

ja: notes various tools from various vendors and the scores just don't tell anything
... it just has no value

<bruce_bailey> +1 that score are pretty arbitrary

ja: sees no value

jf: agrees it's not helpful to the devs, but thinks the ceo would like to see improvement in the final number
... suggests it's a way to show and encourage progress -- competitive analysis

<JF> https://docs.google.com/document/d/1PgmVS0s8_klxvV2ImZS1GRXHwUgKkoXQ1_y6RBMIZQw/edit

ja: suggests the scores aren't correct; different dashboards come out with different scores

<Zakim> bruce_bailey, you wanted to say multiple currencies may not be needed with the approaches we are working on

ja: ok with having scores, just rolling into a single number doesn't tell anyone much

bb: agrees with ja
... believe we're moving that way especially tracking toward functional needs
... don't believe we need worry about that much right now

df: see both points and think we can support both
... notes legislation doesn't have this kind of detail; just talks in general terms of what's required

js: asks the group to consider what's W3C's responsibility vis a vis what other players bring to the party

<bruce_bailey> +1 for focus on our role as standards org

jf: notes every vendor dashboard supports a final score
... it's what the customers want
... wants w3c to provide what the tooling vendors need

ja: not disagreeing; notes it's supported by the sheets; just don't think it means that much
... don't think there's much gap here

js: asks jf whether w3c based data; is that sufficient?
... or does w3c need to tell tool vendors how to put it all together

jf: points to html5

<bruce_bailey> @jf wcag2x does not lend itself to percental score, and that has been okay

jf: gaps in how data is measured or caluclated would be a problem
... notes tool vendors all using ACT
... whose ever tool should all come back with approx the same score
... looking for sufficient guidance to do proper eval; then we can differentiate from there
... notes html5 heavy on error handling

This is scribe.perl Revision of Date 
Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/

Guessing input format: Irssi_ISO8601_Log_Text_Format (score 1.00)

Present: jeanne sajkaj CharlesHall JakeAbma bruce_bailey MichaelC Lauriat Fazio PeterKorn JF OmarBonilla Rachael Chuck KimD kirkwood AngelaAccessForAll JoeCronin michaelcrabb Jan
Found Scribe: sajkaj
Inferring ScribeNick: sajkaj

WARNING: No date found!  Assuming today.  (Hint: Specify
the W3C IRC log URL, and the date will be determined from that.)
Or specify the date like this:
<dbooth> Date: 12 Sep 2002

People with action items: 

WARNING: Input appears to use implicit continuation lines.
You may need the "-implicitContinuations" option.


WARNING: IRC log location not specified!  (You can ignore this 
warning if you do not want the generated minutes to contain 
a link to the original IRC log.)