Challenges document

<jeanne> https://w3c.github.io/wcag/conformance-challenges/

<jeanne> Janina: We will be proposing the Challenges to AGWG next week as a Note-track document

<jeanne> ... First Publishing Working Draft (FPWD)

<jeanne> ... Make the biggest splash as we can can , to get as much as publicity as we can so get as much feedback as we can/?

https://raw.githack.com/w3c/wcag/conformance-challenges-working-draft/conformance-challenges/

<jeanne> ... and have the broadest input and create the best possible document

https://raw.githack.com/w3c/wcag/conformance-challenges-working-draft_Detlev1/conformance-challenges/

<jeanne> ... there is a new branch with input from Detlev Fischer who gave us a lot of information.

<jeanne> ... we will try to get that into the document that AGWG will review

<jeanne> ... Jeanne asked that we credit the Silver Task Force and Silver Community Group in the Acknowledgements. We want to do that.

<jeanne> ... we will keep working on the text.

<jeanne> ... there may be more success criteria that we say something about.

<jeanne> ... the order may change. There will be more work on the document.

<Zakim> Lauriat, you wanted to ask about the issues I brought up before around testability vs. conformance and "under construction" that I still see.

<jeanne> Shawn: I raised some comments at TPAC and then in the document. Testability and conformance

<jeanne> ... there is nothing inherently involved in Silver that will address testability. What we are looking at is testing that will have more testing that involves people.

<jeanne> ... Under construction @@

<jeanne> Janina: Does this need to be addressed before FPWD, or could we address them as an editor note or question in the document that would have wider review.

<jeanne> ... W3M always asked for "What questions do you have?"

<jeanne> Shawn: I see this document as "here are things we should address in Silver" and if I were going to address that in the document, I would remove Challenge #1 and remove "Under construction" and replace it as "always dynamic, always changing". Under construction implies shipping non-tested code.

<jeanne> Peter: When I was working on Challenge #1, we have the language of WCAG 2.0 conformance, which is "this is a page". It feels like that is the original normative definition that is challenged in today's technology.

<jeanne> ... If you want to address a page, you take a human and look at a page. When you have a site that is large, you can't address it with a human looking at every page.

<jeanne> ... It is trying to apply a concept to technology that it wasn't designed for.

<jeanne> ... a statement that requires human judgement about a site with millions or even billions of page, doesn't address it.

<jeanne> Shawn: The WCAG-EM document talked about how that could be done using sampling. "Based on this sampling, this is how we do it." Makoto talked about how sampling is done in Japan.

<jeanne> ... is this document about "what we want Silver to address" or is it something differnt"

<jeanne> Peter: Silver may take a long time to address these issues, and this document could be helfpul.

<jeanne> Janina: We could include templates and we need to ask how well this works.

<jeanne> ... it was included in comments on the Google doc.

<jeanne> ... I will be working on the Issues over the weekend so that people will be able to see the issues.

<jeanne> ... we are publishing this document to get input on the approach.

<jeanne> Peter: With Challenge#2, every piece of sofltware I have ever seen has had bugs. You work to fix them and balance not bringing them to market. I think about the fact that differnt parts of a website may be pushed at different points in time. At any given moment in time, things may not be 100%. When do you rerun your tests? How coherent does it have to be to ship?

<jeanne> ... we thought that getting the document out now will allow comment and allow Silver FPWD to reference that.

<jeanne> Shawn: Given that, I would remove Challenge 1 entirely.

<jeanne> ... it is an important point to make, that all software has bugs. That is a conformance model challenge and it is an important challenge to WCAG today. But the under-construction phrase implies that software is shipped without testing and that is not true.

<jeanne> Janina: We should take more work on "under construction" metaphor. I would agree that we can say more. I'm concerned at moving Challenge #1 because that is where we have put most of the work to date.

<jeanne> ... it's important to inform the construction of a conformance model to allow automated testing, define manual testing, and look for solutions to how we can open up the binary pass/fail to more nuance.

<jeanne> Shawn: My thinking around Challenge #1, I agree that it is a challenge for large and small organizations, but I don't see that it can solved in a conformance model.

<jeanne> ... the conformance model doesn't care how you get the results, but it is for how you express that results.,

<jeanne> Charles: The challenge is valid and we can appreciate the work t hat went into articulate it, but it doesn't mean that Silver can solve this challenge.

<jeanne> Peter: Fair enough. WHen you have missing alt text on 1 page out of billions, then you fail. I don't see how a model that involves humans is achievable. Even if it is bounded to a number of pages, the potential of errors is boundless. This is a problem with WCAG overall.

<jeanne> Peter: Maybe if Silver can't solve it, maybe it is better language

<jeanne> Janina: "reasonable likelihood based on [whatever]

<jeanne> Shawn: It's a reasonable way to look through things. I want to be careful not to set expectations that Silver cannot meet. Because of the need for human involvement, we can say "here are the things we can automate, here are the things you can write a script for" that will change even if WCAG stays the same.

<jeanne> Peter: Hopefully getting better and better.

<jeanne> ... is there potentially a badge that could be associated with a snapshot of how things are today?

<jeanne> ... every conversation I have been in, quickly moves to solutions.

<jeanne> ... maybe if we capture all the challenges and

<jeanne> ... think of broader

<scribe> scribe: janina

jeanne: We need to make this broader? Perhaps include some of the challenges that affect small and medium orgs?

<Lauriat> +1

jeanne: Suggests consolidating some of the related work

peter: Change the title? Then restate the focus
... Worries that refocus will take more than a few weeks

jeanne: We do have a lot that's done, just not published

<jeanne> Peter: I'm thinking about a few ways to address this. Change the title, and say that the first draft is focusing on the large organizations

janina: Offers recent pointers combined

<jeanne> Janina: I can combine the issues that came in the last few weeks.

peter: But I think Jeanne means even more in the archives
... So is our timeline changed?

<jeanne> Peter: Part of our timing was the first public working draft of SIlver timing.

jeanne: we don't have the content ready that we would have ready by now

jeanne; Without that, I don't know whether publishing makes sense

shawn: That's a good question we don't know!
... publishminimal in order to start the process, or wait for something more substantive later?

jeanne: And Charter is still not resolved as of today
... Actually hoping for a little more time for more robust content

shawn: I think the question of Challenges publication is the question of the goal

<jeanne> Shawn: For this document of the Challenges, to have a draft published in a format where there is broad public input.

shawn: Who's the audience/

<jeanne> ... who is the audience for this document

peter: i believe to focus some time around better understanding of what all the issues and concerns are, so we are better informed moving toward solutions

shawn: Wants to suggest Challenges for Anybody, not just Large/Complex/Dynamic

peter; How about we communicate that in text?

shawn: if the purpose is to get it in front of people for more contributions, it makes sense to publish

peter: Even as much as we've already pulled attention, we've gotten great input
... Believe wider review will be very helpful that way

jeanne: Offers to create a branch that pulls in some of our other content into this doc format

peter: asks whether accordian markup would help

jeanne: whatever makes it more accessible, including coga

<jeanne> Shawn: I would put something in the Abstract that describes the purpose of the document so that we can solve these problems more holistically.

<jeanne> Janina: Never thought of an Explainer for a NOte, but maybe we need that

<jeanne> Shawn: I don't think you need an explainer, but I put it in the Introduction or Abstract to help us understand the problems more holistically so that we can build better solutions

<jeanne> Janina: [also contributed to the language above]

<jeanne> Peter: 1) change the title to challenges we are wrestling generally

<jeanne> ... 2) modify the "under construction"

<jeanne> ... 3) Abstract or Introduction changes

<jeanne> ... 4) Explain that while we have focused on large and dynamic, the intention is to include more issues than just large and dynamic

<jeanne> Shawn: Another problem is that when testing that two testers do not agree

<jeanne> Charles: In the Silver Research, we discovered that at BEST 80% of testers agreed on the evaluation of the same page.

<CharlesHall> note: we are at time :/

<jeanne> Peter: There are other external studies, I have been wondering how to reference, in addition to referencing the the Silver Research 2017-2018

<jeanne> ... Like the WebAim research of 1M sites let than 1% conformed of home page with automated testing, while another study had very different results

This is scribe.perl Revision: 1.154  of Date: 2018/09/25 16:35:56  
Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/

Guessing input format: Irssi_ISO8601_Log_Text_Format (score 1.00)

Succeeded: s/\//?/
Present: jeanne janina CharlesHall PeterKorn AngelaAccessForAll Jan Shawn Shari
Found Scribe: janina
Inferring ScribeNick: janina
Found Date: 15 Nov 2019
People with action items: 

WARNING: Input appears to use implicit continuation lines.
You may need the "-implicitContinuations" option.


WARNING: IRC log location not specified!  (You can ignore this 
warning if you do not want the generated minutes to contain 
a link to the original IRC log.)