Silver Community Group Teleconference -- 03 May 2019

<Cyborg> where are we now?

Design Principle 9 (data-informed)

This is just a pre-meeting conversation, started before the top of the hour.

<jeanne> https://w3c.github.io/silver/requirements/index.html

<scribe> scribe: Chuck

jeanne: I updated github doc after meeting. We changed it to... glad Shari is here... we have ... <reads>

<Lauriat> 9. Be data-informed and evidence-based where possible. We recognize that research and evidence are influenced by the number of people with a particular disability, by the size of the body of research, and the difficulty in capturing data regarding some disabilities. The intent is to make informed decisions wherever possible to ensure that needs of all people with disabilities will be prioritized equally.

<Lauriat> …In situations where there is no evidence or research, valid data-gathering methods can be used to obtain information from advocacy groups, people with lived experience, and other subject matter experts.

jeanne: We received comment q the "valid data gathering methods", is there a standard definition? A chain of discussion about the proposal to add a sentence to "quality of research". Shari, you are on the spot.
... Shari, how good is this?

Shari: Caught last part... for what we are currently using... "valid data gathering" is that the question?

jeanne: No, overall. Any problems in the statement?

Shari: Not last time I looked, I'll take a more thorough review and get back to you.

jeanne: It is #9 in req doc.
... Should we address "valid data gathering" first?

Jan: I think that there are some other thoughts lost in the thread. I have some concerns with labeling people on the edge or edge cases. I reworded the language in my proposed text shortly after meeting ended.

<dboudreau> I would also strongly oppose to using words like edge cases when we talk about people

Jan: I think there's a few things to discuss. We could identify some examples of "valid data gathering", like surveys... list a few things if we want to. I think what we are trying to avoid is a single point of opinion as research or fact.
... I'm pasting what I put in here.

<Jan> Part 1: Be data-informed and evidence-based where possible. We recognize that research and evidence are influenced by the number of people with a particular disability, by the size of the body of research, and by the difficulty in capturing data regarding some disabilities or combination of disabilities.

<Jan> Part 2: The intent is to make informed decisions wherever possible to ensure that the needs of all people with disabilities are prioritized equally, including individualized needs that differ from the majority. In situations where there is no evidence or research, valid data-gathering methods should be used to obtain information from advocacy groups, people with lived experience and other subject matter experts.

dboudreau: Yes, I like extreme cases, not edge cases.

Jan: I think we missed last Tuesday that... culmination of disabilities. I put that in part two of the statement here. We'll talk about additional language that was suggested.

Cybel: Seems like we are jumping ahead, should I talk about this or valid data gathering methods?

JF: I'm in q for that as well.

Cybel: Not to defend "edge cases", that is a term that they frequently use for data points furthest outlying. One of my concerns: in addition to input that John brought up, I'm not only concerned about input, but how research is interpreted.
... Create's a "majority impression". Ultimately I hope that silver gets to research that informs silver has interpretations about those who don't fit in and what their needs are as well.
... May not be John's edit, may be something that wasn't clear in what I proposed. Some clarification be that research not be excluded when they fall outside of the majority.

JF: I'm concerned about politicizing of "majority/minority". We are trying to ensure we are as inclusive as possible. If research shows 99 of 100 xyz... what we are constructing here is not the roadmap to perfectoin.
... Individuals are individual. No matter what we do we won't be able to address everybody on everything. I'm sympathetic to weighted responses, at some point we have to draw a line and say "in most use cases this is the best path forward".
... There are times when... we haven't talked about conflict resolution. Technology may not be in a place to address this. Yes we need to be mindful of not majority rule, but we can't ignore the fact that we may be informed
... by broader body of users.

Cybel: The only part I have a direct disagreement is where we draw the line, and we haven't established the line. Where is the line? 51/49? That being clear and transparent on where the lines are is important.
... Canada line is extremely high. If we use legal standard, it might be closer to 99. Do we establish that now?
... In regards to conflict resolution, may merit extra design work. Research is not how we implement it, but what do we define as good research for disabilities? I propose one that doesn't just look at #'s by majority/minority.

<CharlesHall> the intent of the principle is that guidance is informed by evidence. the challenge is then what evidence and how was it interpreted. I don’t think we can explicitly define these at a principles level.

<Zakim> jeanne, you wanted to say my concern with how much of a research data of a paper we can see.

Cybel: I support that in the minority space, what the potential is to make things better for most users.

<Jan> Does this phrase not address the concerns being raised about numbers: The intent is to make informed decisions wherever possible to ensure that the needs of all people with disabilities are prioritized equally, including individualized needs that differ from the majority.

jeanne: From a practical viewpoint we aren't going to see raw research data. We won't be able to do 49/51 evaluation. Most research will be behind a paywall. Only a few of us will see it.

<Jan> This speaks to individualized needs...

<Lauriat> +1 to Charles

<Lauriat> +1 to Jeanne

jeanne: Secondly we won't see the data. The time we spend trying to draw a line is moot. We won't be able to do this stuff.

Cybel: That's why I propose is an "encourage" sentence. That research is done in that spirit.

JF: Are you trying to use that as the definition of "valid"? What does "valid" mean? Current statement talks about "valid data gathering". Mikoto asks what that means.
... Could be any kind of research activity that has any kind of rigor. Ackidemic, corporate.

<CharlesHall> we also have to not imply that anything that informed WCAG is invalid

JF: The methodology is hopefully open so we can evaluate. At some point we are going to have to make value judgements as a task force/working group. I don't know how to draw that line.
... Being asked if it's 80/20, 90/10... I don't think we can fix that number. We'll have to make a subjective group consensus decision on the data. Based on gut feel, source, what-have-you.
... At some point the task force will have to decide as a group.

jeanne: Could we drop the word "valid"?

JF: We could replace it with "traditional" or "alternative".

Shari: I disagree with that. What Jeanne last wrote... <quotes/reads> The importance of using word valid indicates that the collection method used represents that it does gather data for what is being tried to research.

<Cyborg> Cybele's suggestion from email thread:..."In situations where there is no evidence or research, valid data-gathering methods can be used to obtain and evaluate information from advocacy groups, people with lived experience and other subject matter experts. We aim for and encourage research that is rigorous and includes those at the edges, rather than only addressing the needs of the majority. "

Shari: <offers example>
... I personally we need to keep "valid" in there. It means we are collecting evidence of what we want to measure. Or we leave the door open to get research and data that is not in fact representative of the group.

JF: What I heard is that you want to see traditional data gathering methodology.
... As a research scientiest there are traditional methodologies that are valid or not. Are we boiling the ocean? This is a principal. Can we find another word that summarizes what "validity" means to us.

jeanne: If it's accepted by researchers, why can't we use this?

<KimD> I'm fine with "valid"

Shari: These are the appropriate terms. "Traditional" is not great word, because there are new research methods that aren't traditional. I'm concerned about trying to find a word to replace a word that is common and accepted.
... It means what we are wanting to say.

JF: You may have said the word, "common".

jeanne: Let's use "valid", and define it per Shari.

<Jan> +1 to using the term "valid."

jeanne: Let's define it and move on.

Shari: In req document, will we have a glossary?
... I'll work on what Jan wrote and put it in, just a moment.

<CharlesHall> Validity in data collection means that your findings truly represent the subject you are claiming to measure.

Shari: To define "validity" seems silly. But if that's what the group wants....

JF: Because Makoto asked what it meant. English is not his native language.
... He was unsure what it meant. We should be clear about what we mean by that term.

jeanne: If we fix this problem, we get back to whether or not we need something to support edge cases or whatever we want to call it.

<shari> Part 2: The intent is to make informed decisions wherever possible to ensure that the needs of all people with disabilities are prioritized equally, including individualized needs that differ from the majority. In situations where there is no evidence or research, valid data-gathering methods should be used to obtain information from advocacy groups, people with lived experience and other subject matter experts.

jeanne: Jan, your proposal, you ended it where I did. <pasting it back in>

Cybel: I see it in Jan's sentence, you added a comma and an additional clause.

Jan: I put in <reads>
... I added "or combination of disabilities".
... <continues to read>
... I didn't comment on Cybele's final sentence.

jeanne: Cybele could you live with that>

s />/?

<CharlesHall> for ‘prioritized equally’, I would prefer something like ‘no individual need is prioritized over another’ in order to avoid the risk of being challenged on ‘equally’

Cybele: I can live with that if we don't want to use "edge", the only thing it doesn't address is the beginning the sentence about encouraging more research (beyond research that currently exists).

jeanne: Do we have a way to encourage research anymore?

jan: If you want to encourage them to use our point system....

jeanne: That's more orgs evaluating their own products. I'm talking about like what we did when we started this project. We partnered with acedemics... I don't know that we can do that anymore.

Cybele: Maybe I can put that question to people in acedemics.
... Is there more ...

Seanne: I don't think we need to include that in the requirements. Doesn't make sense. We can encourge, but need not be in requirements.

s /Seanne/sean/

sean: I don't think it belongs in req doc.

<CharlesHall> this principle for silver is about the guidelines themselves not conforming to them

<shari> validity in data collection means that your findings are representative of what you are claiming to measure.

Cybele: In that case I'm fine with Jan's phrasing. If we could add "evaluate" or discuss "evaluate".

jeanne: Before we talk about evaluate, JF are you ok with Jan's phrasing?

JF: Still trying to find the phrasing we are looking at.

jeanne: Jan can you past it in again?

<Jan> Be data-informed and evidence-based where possible. We recognize that research and evidence are influenced by the number of people with a particular disability, by the size of the body of research, and by the difficulty in capturing data regarding some disabilities or combination of disabilities. The intent is to make informed decisions wherever possible to ensure that the needs of all people with disabilities are prioritized equally, including individualized need

JF: There's been a lot of changes.

<Jan> individualized needs that differ from the majority. In situations where there is no evidence or research, valid data-gathering methods should be used to obtain information from advocacy groups, people with lived experience and other subject matter experts.

Cybele: While she's doing that... harping on the point: I hear you jeanne that we don't have control over research or access, but I hope that... maybe it's not a requirement thing, but I hope that the research tries to identify needs being excluded
... For those that don't fit in the majority.
... I feel like it's covered for requirements. I can live with it.

<johnkirkwood> "prioritized equally" ?

<CharlesHall> -1 to ‘prioritized equally’

JF: Struggling with "individualized". Indicates that it goes down the individual. We'll have to make some judgement calls. Whatever we do has to be scalable by large organizations. I struggle with "individualized".
... <reads aloud>... including needs that differ from majority.
... My concern is that someone will say "I'm an indvidual and my opinion equals the large research".

jeanne: Interesting. I read that as the work you are doing in personalization.

<Jan> How about "personalizable needs that differ from the majority?"

JF: The whole goal is to try and create content that meets the needs of individuals. Even the work there is nacent, we are working towards that. We are making broad generalizations and narrow it down from there.
... We are saying that we are trying to create an environment where that kind of personalization can be met.
... I don't see "individualized" being a realistic end state.

Cybele: Maybe what's missing is those outlying/edge cases individuals, sometimes there are needs in that space that aren't at the majority space, but are at the benefit level.

JF: I don't disagree. I'm struggling with a personal experience I had in a client. Web site under legal distress. At one point there was a subjective call.

jeanne: We need to come to a conclusion.

JF: it was in regards to the nature of an image. 9/10 people said alt="" was sufficient, 1 person thought otherwise. I want to avoid this scenario. We don't want tail wagging dog.

jeanne: Cybele, is there a place where you want to put "evaluate"?

<jeanne> Be data-informed and evidence-based where possible. We recognize that research and evidence are influenced by the number of people with a particular disability, by the size of the body of research, and by the difficulty in capturing data regarding some disabilities or combination of disabilities. The intent is to make informed decisions wherever possible to ensure that the needs of all people

<jeanne> with disabilities are prioritized equally, including personalizable needs that differ from the majority. In situations where there is no evidence or research, valid data-gathering methods should be used to obtain information from advocacy groups, people with lived experience and other subject matter experts.

Cybele: It was after....

jeanne: Are there other people who can live with "personalizable needs"?

sean: "personalizable" is too specific.

Jan: Ok to drop "individualized". Rather than hash over repreatedly.

Cybele: I can live with it too.

<CharlesHall> I still prefer to avoid “prioritized equally”

<KimD> +1 to removing "individualized" although I think it's fine to include.

Cybele: Where I put evaluate is in the last sentence.

<jeanne> with disabilities are prioritized equally, including needs that differ from the majority. In situations where there is no evidence or research, valid data-gathering methods should be used to obtain and evaluate information from advocacy groups, people with lived experience and other subject matter experts.

<Jan> +1 to inserting evaluate

<Lauriat> +1

jeanne & JF: I can live with that.

<KimD> +1

JF: Saying at end of the day we will take all the data we can get our hands on, but we will not be told what to do with the data, we will be informed by the data.

jeanne: Paste it in one more time.

<jeanne> +1

<johnkirkwood> +1

<AngelaAccessForAll> +1

<KimD> +100

<Lauriat> +1

<JF> +1

<CharlesHall> -1 to “prioritized equally”

<Cyborg> +1

<Jan> +1

<shari> +1

Charles: Fine with all rest of wording. But need to replace that phrase. Because we will be challenged on "equally". Tail will wag the dog. The one need was equal to the 99.
... If we instead that one need is not prioritized over another, or some such would be fine.

Sean: I'm good with that rewording also.

<Jan> +1 to dropping the word "equally"

<Cyborg> Jeanne - no need is prioritized over another?

<johnkirkwood> treated equally

all needs are treated equally and no need is prioritized over another?

<Jan> just use "prioritized" without the word equally

<jeanne> isn't prioritized over another, including needs that differ from the majority. In situations where there is no evidence or research, valid data-gathering methods should be used to obtain and evaluate information from advocacy groups, people with lived experience and other subject matter experts.

++++1

<KimD> +1

<CharlesHall> +1

<jeanne> +1

<Lauriat> +1

<Jan> +1

<johnkirkwood> +1

<AngelaAccessForAll> +1

RESOLUTION: Update the requirements as stated above.

<Jan> How about this: The intent is to make informed decisions wherever possible to ensure that the needs of all people with disabilities are prioritized, including individualized needs that differ from the majority.

<Cyborg> +1 to Jan

<Jan> Take out Individualized

<Jan> That was a mistake

<shari> +1

<Cyborg> +1

<AngelaAccessForAll> +1

<CharlesHall> +1

<Lauriat> +1

<KimD> +1

RERESOLUTION? nah...

<jeanne> +1

RESOLUTION: as amended.

jeanne: I want to let people know that I did a rewrite of conformance prototype. Sending out invitation to select people who are mostly acedemic and experts in measurement and conformance.
... Also had a productive conversation with a group in AU who are interested in helping us write content. We won't be ready for a while, we will practice first, and update template.
... But we do have the offer.
... I would like to ask everybody to think about this: Take a look at what we've done with migration document (linked in agenda), think about the success critieria we should start with.
... People will meet at AccessU. Which ones should we start with?
... That will take us to next week.

<Jan> What day(s) did you land on for meeting at AccessU?

Cybele: Quick q? new sc, or sc to migrate?

jeanne: sc to migrate.

JF: They want to see a draft. Not expecting perfect.
... That's important that we set our own expectations. AG working group wants to see what we have and may provide feedback and comments.
... We should recognize that we won't have end result, but close.

jeanne: The other piece of this is that the reason it's urgent, not only because ag wants to see it, but also ... <audio problems> with some sample guidance in it that can be looked at in order so that ag recharter and includes us.
... Will go more smoothly if there's an editor's draft (even if incomplete).
... Not to have comments from ag, but to have an editor's draft.

jf: fair point.

Jan: Did you send out a note on meeting days?

jeanne: I announced it. Sharon will give us a room for wed-fri, and people can come and go as needed. I'll send out notification.
... I will be there on Monday. If people want to work and meet, we can do that.

- DRAFT -

Silver Community Group Teleconference

03 May 2019

Attendees

Contents

Design Principle 9 (data-informed)

Summary of Action Items

Summary of Resolutions

Scribe.perl diagnostic output