http://www.lflegal.com/2018/05/wai-eowg/ - write-up on the EOWG

<jeanne> https://w3c.github.io/silver/prototypes/ConformancePrototype/index.html

Continue discussion on the Conformance prototype

Jeanne: We were talking about the conformance model on Tuesday and I think we were talking about the point system

Luis: We were talking about different point systems for different types of web sites ....

<jemma> https://www.w3.org/2018/07/31-silver-minutes.html

Luis: brought up heuristic evaluation

<jemma> luis is sharing the meeting minutes from this Tuesday.

<Zakim> MichaelC, you wanted to ask if points are assigned to types of tests, specific SC, context in site, etc. and to ask how points mapped to levels

Michael: How do we decide what tests give points - can you get double points if you test one thing in two ways? How is it mapped out?

Jeanne: We haven't mapped out how the points will be awarded yet
... at AccessU, people were suggesting that you would have blocks so that you had to have points in each block that were roughly equivalent to the major guidelines or principles.
... this was to keep people from stacking up points in one area to the detriment of other areas
... we have not talked about people testing things in two different ways to get points

Michael: One scenario as a thought experiment, what if you had a site that had a lot of images and you get points for alt text, but you still can't complete the form, but you have enough points that the site passes without actually being accessible. Does the point take into account context like that, and since it's so site specific, how would that work?

Luis: We don't have all of that worked out yet. Where I work, we have internal score cards that separate out the responsibilities by group. For example, we give you points for the first 5 alt text - we assume that you have the points and then we take away from those points. There are different ways of dealing with this.

Michael: Some of the answers you are giving, but it would be nice to have it written down. My next question is how to map the numbers?

Luis: We have not really gotten into the details of what the scoring system will be.

Jeanne: We would like to do a full example to show how the points would work and how things would map to a different level.
... I need some help on this to devote to the document.

<kirkwood> OpenAir

Luis: I am willing to help and just need to set up some time to work on it.

Jan: Maybe include Sharron Rush because of her Accessible Internet Rally

<kirkwood> Can u forward on their rating sys?

Isabel: I would like to be in the loop on this as well - particularly around the potential risk of scoring and its misuse. The worry I have is that if someone scores well and then gets complaints about a11y and then they use their score as an excuse not to address the complaints.

Luis: My experience has been that scores are good for communicating to teams. The way you use the scores is important - it gives you a way to demonstrate that there is still work to be done. In terms of compliance, I would be concerned about using a score for compliance.

Jeanne: Why?

Luis: Because of some of the concerns that we have raised here. One score is not necessarily going to tell you how accessible something is. In Open AIR, you can get bonus points for doing video and audio. So, while that's good, it might not be relevant to the accessibility of the core content and purpose of the page.

Jeanne: What if we only had bonus points apply to upper levels and say that the base, bronze level would require basic a11y

Luis: Yes, but it can still get complicated; we have things weighted in our system (e.g. active image vs. image that does not convey meaning)

Michael: Two additional thoughts: Aligning scoring with conformance (they don't necessarily map); Subjectivity, which I see in the prototype (human testable) - I am not sure that there is a completely objective reality, so how can we avoid subjectivity ... especially with a point system?

Jeanne: I thought that table 5 dealt with that well. They had ideas on how to put subjectivity in the block so that you could still measure without it necessarily being precise. That would work at the individual test level, but not necessarily at the scoring level.

Michael: I am also concerned about incomplete knowledge.

Luis: Yes, people might disagree on how many points something is worth.

Isabel: Two examples: One ... sites by transportation companies have root planners that conform to WCAG, but the root planners are really hard to understand and I have been told that it's a usability issue and not an accessibility issue; Second - LinkedIn is linear and can create barriers for people with disabilities - it highlights gaps in employment and none of that gets addressed; they can get a great score, but their design is affecting PWD
... how many SCs are evidence based and how many are based on the needs of people with disabilities (PWD)
... scoring can misrepresent the reality on the ground
... how are we going to get feedback from PWD and how to quantify that?

Jeanne: Any ideas on how to solve these?

Isabel: I don't have any ideas for solutions right now. The first thing that comes to mind is that it would be good to consult PWD and do co:design on the conformance model with them so that it's not imposed from the outside. Also, just having more discussions of what evidence-based means and having conversations with people who have had these issues in other sectors.

Jeanne: Do you think we need to involve PWD at this phase more? We have PWD as part of this group. I originally pictured that we would refine things a bit more before we sought more feedback.
... generally, people do better when they have something they can comment on, rather than coming up with something from scratch.

Isabel: I have one more suggestion: If there are edge cases of things that we want to have run through the conformance point model that are difficult in terms of risk - perhaps we could create a prototype around those issues so that when we go to PWDs for feedback, we get feedback on the risks as well as the conformance solution and scoring system.

Jeanne: I would rather put our focus on how to do good, rather than how to prevent bad. We have to look at the unintended consequences of what we do, but we should approach this about serving the intended audiences better, rather than making this system airtight to prevent cheating.
... I think the point system gives us flexibility to change things in response to problems that get identified.
... One of the problems that came up at the design sprint was the concern of never meeting WCAG due to how frequently the site updates and the model is too static web-oriented. We can consider having a static web point system and a different system for dynamic, complex sites that have unique needs.

Luis: One quick note about the conformance being more fluid ... in WCAG, the conformance is in the main WCAG document. Do we need to have a normative scoring system in the main spec?

Michael: My recollection is that, at the time, there was an emphasis on specification quality and one things that had to be included was a conformance section.
... that then led to checklists that were primarily focused on technical specifications.

Jeanne: I do think that we want to have a conformance section because we want WCAG to be referenced in policy, but hopefully we can determine how some of it can be normative and some of it could be adaptable to changes that need to be made.

Michael: We do have the requirement that you can prove that something is implementable and you have to have a conformance section for that.

Luis: One of the goals for Silver was for it to be more evergreen and more easily updateable. Do we want the nonnormative parts to be easily updatable, or do we also want the normative parts to be updateable?

Jeanne: We don't have an answer to that yet.

Michael: Although I said that conformance was included in large part for procedural reasons, it has been used for integrating into policy, so it may be difficult to move people away from using that.

Jeanne: We want to have a conformance section.

Michael: If we try to make it less overbearing, people may complain that it is not specific enough.

Looking for outside prototypes or more prototypes

<jeanne> https://w3c.github.io/silver/prototypes/index.html

Jeanne: I did an update the silver prototypes list
... I added a new section called the "external accessibility guidance or prototypes"

<jeanne> http://intopia.digital/pdf/WCAG2_1Map.pdf

Jeanne: I added the WCAG Techniques Finder and the BBC guidelines and Intopia's WCAG 2.1 map

<Zakim> MichaelC, you wanted to ask about smoothing subjectivity and to align scoring with conformance

Continue discussion on the Conformance prototype

<jeanne> https://w3c.github.io/silver/prototypes/FlavorPrototype/site/index.html

<jeanne> https://docs.google.com/forms/d/e/1FAIpQLSfmXR8mR7-XZ9b0s9bWJgjEvSbCyIgv19KYEw8DdRnLfQAVpQ/viewform

Jeanne: I was asked to address the Survey Questions of the Flavor's Prototyp during Tuesday's meeting - I did this in a Google Form; there are not a lot of questions on this, but please look at the questions.

Isabel: The tabs themselves - there seem to be a lot of tabs.

Jeanne: Yes, we need to get some testers in the Flavors prototype

<jemma> +q regarding "Should you be able to see the tabs that have no information? Should they be disabled, or hidden?"

<jemma> +q

Luis: The page I get to when I choose to "access the prototype" doesn't have any tabs

Jeanne: So, we need more instructions for how to get there.

Luis: What counts as "core information" and what counts as "technical information"
... I don't see anything on the prototype that addresses those terms.

Jeanne: The general information tab was the "core information" but I will fix that. Tab under the developer was the static web, dynamic web, aria, flash, etc.

Luis: Technical is the development stuff and everything else is core?

Jeanne: Yes

<jemma> my question is about understaning the context of "no information". why is there a tab in the first place if there is no info? or is this just a question about the micro level user interface design, diabled or hidden?

Jeanne: what we are going is putting general information at the same level as the technical information so that you don't have to drill down to the techniques.

Luis: so is the question about separating core from technical?

Jeanne: Yes, we could compare this design to the current WCAG design where the techniques are listed as a link.
... I could use the Silverlight example.

This is scribe.perl Revision: 1.152  of Date: 2017/02/06 11:04:15  
Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/

Guessing input format: Irssi_ISO8601_Log_Text_Format (score 1.00)

Succeeded: s/meeting/minutes/
Present: jeanne KimD Jan jemma LuisG MichaelC AngelaAccessForAll shari kirkwood
Regrets: shawn
No ScribeNick specified.  Guessing ScribeNick: Jan
Inferring Scribes: Jan
Found Date: 03 Aug 2018
People with action items: 

WARNING: IRC log location not specified!  (You can ignore this 
warning if you do not want the generated minutes to contain 
a link to the original IRC log.)