I can scribe

comments from AGWG today

janina: we got a little into the weeds of known issues
... particularly scoring and it kind of subverted the agenda
... it's pretty easy when questions start coming in to dive into details and you're trying to flesh them out in real time
... it was more positive than not and it seems the overall approach has agreement

kim: there are three people on our team and we all attended; the others thought it was a little confusing
... I told them it was a little high level; once we got clarity to look at the ideas, not the exact content, they were on board

fazio: it's a good framework; a great starting point for where we're at

jeanne: great, thanks

Questions for AGWG

jeanne: let's get back to work. I want a points system proposal soon. it's starting to feel really close

Point system proposal

jeanne: things are starting to come together in a way that can work really strongly

<jeanne2> https://docs.google.com/document/d/1y_HOyuMKltOQoZr0Gk7hMQXi3Jd8Mc5fyr-XkH7kZQY/edit#heading=h.bex9z1k5ual2

<Fazio> btw is this modeled on VPAT?

jeanne: last week we were working on the points system proposal in the sampling document

<Fazio> it seems similar

jeanne: I wouldn't say it's modeled on VPAT, I wouldn't want to be far away from VPAT, but we're not taking it as a starting point

<Fazio> agreed

jeanne: we're looking similar to VPAT by taking a similar route
... we're looking at the needs and the problems that need to be solved; maybe they did something similar

janina: I think saying "taking a different route" is accurate

<Fazio> thanks

jeanne: we didn't get into the point system proposal last week right?

janina: we touched on it some

jeanne: it's additive and percentage based
... you're going to need a minimum in each disability category; I think we're all good with that...at least for a while

<KimD> starting with "There is no weighting for importance, severity, difficulty, or other reason."

jeanne: no weighting for any reason is the next bullet point

fazio: what do you mean "no weighting?"

jeanne: part of the reason people with cognitive disabilities are poorly represented in WCAG is in part because they're at the AAA leve
... and success criteria for other disabilities are at the A level
... one of the factors for determining levels was whether they were blocking issues for someone
... this ends up saying "people with physical disabilities have a higher priority"
... because it could just be "harder, but doable" for someone with a cognitive disability

fazio: it kind of demeans what it means to have a cognitive disability

jeanne: we've been working on what we can do to get extra weight for the score, but none have really worked
... I think we have the evidence to hold firm when we get any push back on it

kimd: when you say "difficulty" do you mean for someone to get around an issue or for something to be made accessible?

jeanne: both, actually
... difficulty in implementing wasn't very consistent over platforms. it might be easy in one, but difficulty in another. a simple technological advance could change that
... difficulty also when building it the first time versus when remediating an issue

janina: it will also be difficult selling the points system...and it's hard to assign any kind of points for plain language

fazio: there might be a way if you hook someone up to an EEG machine and measure some cognitive difficulty while someone is going through a task

jeanne: if you can say "this is how many active/passive verbs" you can come up with a percentage, which leads to the next bullet point

<KimD> "For automated tests, the total instances of the condition are divided by the total passes for that condition. "

thanks!

scribe: we'll have to develop tests that do that kind of counting
... but certainly something that can be done by automated testing
... and then we have the next one which is really hard

<KimD> Next bullet: "For advanced Silver tests, there will be a mathematical condition that will be a part of the test. For example, a rubric test would have the total points scored divided by the number of factors. (We need examples of other tests.) "

scribe: this came up in the call today; we were looking at the rubric for clear words

<jeanne2> https://raw.githack.com/w3c/silver/conformance-js-dec/guidelines/methods/Method-plain-language-principles.html

scribe: if you go to the test tab we have three columns, substantially is 100%, partially is 50%, and limited is 0%
... you can normalize it by dividing by the number of actions we're testing for
... so you always end up with a score between 0 and 1 and will be a percentage
... it makes each one equal since some things are important to some and others to other groups
... it makes a difference on the individual user need scores
... so there might be less "actions" for some types of disabilities and more for others. by dividing it within each category, it makes it equal
... there was no way of making it fair, but this way does!

janina: there is at least one more W3C specification not related to accessibility; authentication does something similar
... they weight a bunch of factors like that...can you really trust something in a login attempt
... it's based on similar thinking

jeanne: the scoring would have to be by Method...and using this method, it actually works by Guideline. This was something I had gotten exciting about, because we're also worried about folks gaming the sytem
... they couldn't score points with a lot of methods
... if it's by guideline

kimd: comment about language...some languages are passive, so we might need some disclaimer or generalization for language consideration

jeanne: thank you. it's in the Google Doc, but I forgot to put it in the HTML version
... the beauty of this, is because it's in a method, people can change it in a way appropriate for their language without having to change the normative guideline since it's all in the non-normative side
... you just change how you normalize it; if there's a language that doesn't use passive voice...you're just dividing by one less action so you're not being penalized or getting an unfair advantage
... the next bullet point

<jeanne2> Once the minimum level is achieved in each category, all of the guidelines that are met to whatever percentage they are met increase the score.

<jeanne2> The total score is divided by the number of guidelines to give a final percentage.

jeanne: then all the guidelines that are met, whichever percentage that are met increase the score
... there are guidelines that apply in certain circumstances..if a site doesn't have media, how do you count those? do you just omit them

janina: if you're doing audio, higher bitrates should score higher. and an ability to equalize frequencies that are heard better...that's a whole way of writing a guideline for a group that's not been thought of yet
... but something like that will matter with real time communications; we're looking at use cases we want support from WebRTC for

jeanne: maybe we could use the model of visual contrast where we're giving a range of...where you can say "here's the minimum at this level, and this level, etc."

janina: something similar is likely to be quantifiable and defensible

jeanne: do we want to say...getting back to that bullet points..do we want to say "guidelines that apply"
... I think it's how we do the tests and how they're scored. we have the flexibility to be situational

janina: having it be non-normative makes it possible to be updated more quickly
... and experience can improve the guidance
... more quickly than if it was normative

jeanne: and then the last bullet point...

<jeanne2> Bonus points to reach higher levels can be earned by task completion testing with people with disabilities or user research with people with disabilities. (Should we have this? On one hand, it could incentivize doing more substantive testing, on the other, it gives an unfair advantage to large organizations who have a larger budget for testing. For example, LFlegal.com should always be

<jeanne2> able to achieve Gold rating.)

jeanne: jeanne maybe we say that gold level is 90% or higher and silver is lower, etc.
... we would need to test sites to know where we should draw those lines for the levels
... we could include it like a guideline

fazio: I think the VPAT and WCAG documentation says something about usability testing

jeanne: I don't think I've seen it since it's a problem we're trying to solve

fazio: I saw it earlier today, I'll find it for you

jeanne: great!
... I propose we take this bullet point out for the moment since I don't think it's working out how we thought it would a while ago
... is there anyone that can work on turning this into a clear explanation for the conformance section of the guidelines

AngelaAccessForAll: I can work on that

jeanne: do you have time this week?

AngelaAccessForAll: no, what's our timeline otherwise? is it this week or bust?

jeanne: if we can get it in this week, we can get comments from AGWG..can you do two weeks?

AngelaAccessForAll: definitely

jeanne: and I can help you with it as well
... okay, what's left on the agenda now
... what things do we want to ask them about what we're thinking for conformance...this includes sampling and the points scoring proposal

janina: I'm not sure it's mature enough to ask questions right now. in general, for now maybe stick to things that are more mature for them

jeanne: maybe "does the sampling proposal work across a range of products?" like small businesses, very large, etc. are there things this doesn't work for

kimd: maybe some things that are in the weeds...like 3 categories in the rubric...is that sufficient? if we ask high level, it helps keep them focused on high level answers

jeanne: well for some guidelines, we might have more columns; but maybe something like "we can all agree on the perfect score and the 0 score. how do we figure out what the middle scores should be?"

fazio: that's where we need to do user research

janina: how do you make sure your ".5" score is consistent across different orgs

jeanne: this is a good place to stop. if you think of any questions, send them to me, I'll be able to get the survey out tomorrow

This is scribe.perl Revision: 1.154  of Date: 2018/09/25 16:35:56  
Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/

Guessing input format: Irssi_ISO8601_Log_Text_Format (score 1.00)

Present: jeanne KimD janina joecronin Makoto LuisG Fazio AngelaAccessForAll
No ScribeNick specified.  Guessing ScribeNick: LuisG
Inferring Scribes: LuisG

WARNING: No date found!  Assuming today.  (Hint: Specify
the W3C IRC log URL, and the date will be determined from that.)
Or specify the date like this:
<dbooth> Date: 12 Sep 2002

People with action items: 

WARNING: IRC log location not specified!  (You can ignore this 
warning if you do not want the generated minutes to contain 
a link to the original IRC log.)