Silver Community Group Teleconference -- 16 Sep 2019

<AWK> +AWK

<Chuck> I see Janina's test.

<Chuck> The meeting has not started

<Lauriat> FYI to all, I just updated the schedule for today to reflect what we got through and didn't yesterday: https://www.w3.org/WAI/GL/task-forces/silver/wiki/2019_September_16-20_in_Fukuoka,_Japan#Agenda

<Chuck> scribe: Chuck

Shawn: Waiting a few minutes.
... Quick update for today, intro to Tuesday
... Agenda has changed a bit based on how things went yesterday and what we want to complete today.
... We'll start with how to assess conformance model prototypes. One more thing to get through.
... There was a lack of consensus on how to assess the motivation one, but we want to talk through scope.
... Jeanne has some things drafted.
... Second part of the morning, we'll talk through conformance issues and exceptions.
... Afternoon we'll switch to project plans needs. We'll be charged up and fresh to talk through project plan.
... Then talk about migration of WCAG to Silver.

The Michael Cooper arm-wrestling showdown

How to assess conformance model prototypes: Scope

Shawn: how to assess conformance model prototypes.

<Lauriat> 3.8 Scope: https://docs.google.com/document/d/13A8mGMnQujfEVqcw_LmAUYT8DDq_qW0TNcHxmCHd0io/edit#heading=h.pxm73yeoktv9

Shawn: <pasting in text>

<Lauriat> The guidelines provide guidance for people and organizations that produce digital assets and technology of varying size and complexity. Our intent is to provide guidance for a diverse group of stakeholders including content creators, browsers, authoring tools, assistive technologies, and more.

Shawn: Then will turn over to Jeanne

Jeanne: We were working on this in the task force the same day that AGWG was discussing charter.
... Discussion got sideways into discussing the charter.
... Several weeks old, may have changed.

<Lauriat> Develop a mix of SIlver guidelines and Methods that would include at least 2 methods each for: content creators, browsers, authoring tools, and assistive technologies. Show the methods to AGWG members (content creator experts), accessibility experts from 2-3 major browsers, 2-3 assistive technology vendors (including at least 1 non-English language assistive technology), and 2-3 accessibility experts from authoring tool vendors.

Shawn: <pasting in>

Jeanne: We can show methods to AGWG members who hopefully have expertise, show it to accessibility experts in 2 browser, 2 vendors including one non english language AT.
... 2-3 accessibility experts from authoring tool vendors. They can see methods we are planning to create and we can get feedback from them.

Charles: Are we going to queue up response to that or jump in?

Shawn: You are first in queue.

Charles: the quantifiability of 2 methods each puts a large number and pressure before we can evaluate the conformance model.
... That's a great deal of effort before we can do the evaluation.

Shawn: True, we talked through a few. We would have to have a not tiny amount of content already created to test that.

Jeanne: I need to be more specific about each, I meant each group. 2 specific to browser venders, 2 for AT.

Charles: That's 8 that need to be created before we can measure.

Shawn: Would have to be from guidance that's relevant for each of those.

Charles: Doesn't need to be the same for each.

<jeanne> We have almost that many already done

<Makoto> I can invite "PC-Talker" as one of the non-English screen reader for sure. (In Japanese: http://www.pctalker.net/")

Shawn: No, probably wouldn't be.
... Any comments, q? Any ideas on how to test?

Makoto: Japanese leader to the issue... if it is needed ask them.
... Pc-talker.

<jeanne> Thanks, Makoto

Makoto: We have JAWS and NVDA.

Shawn: Added comment that Makoto can get us in touch.

AWK: What do we anticipate are methods for browsers that require them to do anything, a technique for someone, you have to put alt text on image to be correctly read.
... What's the method for the browser?

Katie: Browser needs to read the alt text.

AWK: I can write that such that every browser already conforms.
... I don't know what are the scenarios where the browser vendor declines becaue they don't want to do that?

Shawn: The one that will come up more readily is emerging technologies, where the browser doesn't do the thing. For the alt text example, we have an example.
... In chrome it will render the alt text for you. We do have more examples of "you can fill this gap browser vendor".
... Yesterday's example for virtual reality platform maybe it doesn't allow for specifying the language. It would be applicable for virtual reality but no way to declare it
... Or maybe it can be declared but AT doesn't check it.

Katie: Or default focus ring. If we have a suggestion about the visible indicators being a certain depth or lightness of color, they would be doing that.

AWK: Browser could do it or authors would take it up.

<Zakim> jeanne, you wanted to give context to AWK

Jeanne: One of the things we learned with user agent accessibility guidelines 2.0, it's not a good idea to force companies to do things.
... If we give the guidance and ideas they are much more likely to implement them.
... A browser which will go nameless did implement most suggestions from a list, they were happy to improve accessibility.
... We want to give guidance and not force them to do things. That doesn't apply to content. But for browser manufactures, there is no benefit to trying and force them to do things.
... ...here's a filtered list of things you could do to improve things for low vision, personalization. A convenient list and how it should support the content, which would be normative.
... We aren't looking at forcing browers into doing anything. That's not our intention.

<Zakim> janina, you wanted to ask whether we've considered any value in splitting content authoring from content hosting?

Janina: New kid on the block, has it been considered to add a category for content hosting providors? Facebook, Amazon, Ebay.
... Wouldn't that simplify our ask? You are making it difficult for what a host should provide vs what is directly responsible for.
... I'm wondering how it plays out and how we handle that.

Shawn: Language of environment example, for it when we worked through the tests for different platforms, different methods...
... Rather than split up by type or thing... we list the ways to declare the language. One of those ways is... an authoring tool to specifify the language.
... That's along the line of what we were thinking. Not along the different types of sites, too deep too quick.
... Amazon crosses may borders, content creation, authoring, categorizing it as one site is an exercise in futility, like many other sites.
... We listed technical and mechanical things that can be done.

Janina: OK.

<Zakim> MichaelC, you wanted to say we should leavve door open to different ua implementation expecations

Michael: Jeanne was saying that we want to avoid normative user agent requirements. I would like us to structure things such that the guidance is agnostic to the normativity.
... We are limited to what methods we can list based on what user agents support. If we can get user agents on board even if we don't make a way for them to claim conformance..
... we work with them.

<jeanne> +1 Michael

Shawn: We don't want to claim what accessibility is for user agents.

<Zakim> alastairc, you wanted to ask if our scope is authoring oriented

Alastair: Going back to AWK's example, would we be writing user methods for user agents to handle alt text, we wouldn't do that.
... We would take approach of content authoring.... automatic creation of alt text would be a low scoring method.

Shawn: If we identify major gaps in a browser's implementation, we look at the possible ways to bridge that gap for the users. For people writing html and javascript...

<jeanne> I don't think we should have to write something for every WCAG SC

Shawn: Pretty fragile, we can write something that suggests how you work around it, but we suggest to user agents that they can bridge the gap for everyone.

Alastair: No point in documenting something that already works.

<jeanne> We should only write methods where it is appropriate

Shawn: Any other q around this? If not we can move on.

<jeanne> Low vision is a low hanging fruit for browser guidance

RESOLUTION: The thing that Jeanne drafted is accepted.

Charles: Quick note. It's not worded the same as an evaluation method.
... It's not written as validate this.

Shawn: I'll work on that before Thursday.

Conformance Issues and Exceptions discussion

Shawn: Since Jeanne has done most of work, can you introduce?

<Lauriat> Conformance Issues and Exceptions: https://docs.google.com/document/d/1II0MP6l_Xn8GaRhxGSJPIbCto7xTLww3zpSjr6k57g4/edit

Jeanne: First thing I'll start with is to give people an opportunity, a link to read for those who haven't read this in a while.
... This is the conformance plan as of last May.

<jeanne> Conformance Plan as of last May: https://docs.google.com/document/d/1wklZRJAIPzdp2RmRKZcVsyRdXpgFbqFF6i7gzCRqldc/edit#heading=h.sevi88jq0fiq

Jeanne: Lot's of background and goals we've been working towards. As we started evaluating some new conformance proposals we brought up in July...
... We realized a number of issues identified that needed to be brought out and discussed before we can address ...
... We pulled in separate document and started working through it.
... Again, our goal is how do we make conformance better align....
... We made list of issues, we recently got more additions to that.
... First one is very important but we can't address, does the model of the architecture address the needs identified.
... We are working on #3, what measurements we should explore and what should get points.
... We can go into this if desired, it would be better use of this group to work on some of the others, because the measurements one has lots of details and moving parts that require solving the others.
... We keep spiraling around on it, but we are really making progress.
... The other part of this document is the exceptions needing solutions. We recently did a lot of work on. Better to start here rather than the issues, but here are the issues...
... <lists issues>

<Lauriat> 4. "Conformance in WCAG 2.x is defined at the page & collection of pages level, and is difficult to apply to large, complex, and/or dynamic websites. Such sites update so rapidly they are impossible to guarantee they are 100% conformant - e.g. Facebook that updates tens of thousands of times a second, and so is impossible to test "

Jeanne: example from silver design sprint is that facebook updates multiple times per second. Impossible to conform to WCAG because they can't test collection of pages because of strict conformance requirements.

<Lauriat> 5. How do we set up a point scoring system that will be transparent, fair, and motivate or reward organizations to do more? There is an experiment with a point scoring spreadsheet. That is not intended to be used by regular users, only accessibility policy experts, regulators, and lawyers. (Bruce recommends a proof of concept that is more exaggerated (order of magnitude) to develop the concept, then refine it later. )

Jeanne: That's an area where we had a lot of discussion in July.

<Lauriat> 6. How do we maintain a point system so it stays current, but is protected from "gaming"?

Jeanne: How do we maintain a point system that stays current but is protected from gaming?

<Lauriat> 7. How do we set up methodologies for task-based assessment that can be used across a breadth of websites and products? The nuance of defining a task (granularity, paths, whether different multiple paths are more accessible to certain disabilities)

Shawn: <pastes in issues>

<Lauriat> 8. How do we migrate people from WCAG 2.x to Silver from a compliance viewpoint? (for example, should Bronze level equal WCAG 2.0 or WCAG 2.1?

<Lauriat> 9. How do we decide what are minimums that organizations must meet? Should that just be the non-interference success criteria of WCAG 2.x or are there more?

Jeanne: Alastair made a very attractive suggestion... there's a minimum score you need to get in each category of user need, so that all of effort doesn't go into a single category.

<Lauriat> 10. Should we require points be spread over categories of user needs? What list of user needs should we use?

<Lauriat> 11. How do we draw a line between "equivalent experience" and not identical experience? The example is a Map application where the complexity of the visual experience is too overwhelming to express in text equivalent.

Jeanne: Another really difficult one, where example is map application that's too much to express in text. How do we want to handle that?

<Lauriat> "Substantially conforms" came out of the Silver research where companies had a generally accessible site, but it was so large or updated so quickly that it wasn't possible to guarantee that it was 100% conformant. Facebook was an example of a site that was literally impossible to test because it was updated tens of thousands of times per second.

Jeanne: Exceptions needing solutions may be a better place to start.
... Facebook was saying that their site... you have a website that is generally accessible, you can't prove it's 100% conformant, can a company show that they are substantially conformant and wouldn't have to prove 100%

<Lauriat> "Tolerance" is a different concept of a less-than-ideal implementation but no serious barriers. I think we could collect those "less than ideal" examples when we write the tests for the user need. How we would we flag them as "less than ideal" and refer people to better methods seems like a solvable problem.

Jeanne: Another is tolerance. Less than ideal implementation but has no serious barriers. This is one the example Shawn gives of a google product where they have an iframe a user would never access, but didn't have a title.
... we want to identify these things that may not conform but don't introduce a barrier.

<Lauriat> User Generated content: For sites like social media sites where users generate the content. The site can provide the tools to make it accessible, but should not be held responsible for the content generated by users.

Jeanne: A big one amazon brough up is user generated content. Social media sites, the host can provide tools to help, but how much should the host be responsible?
... Third party content, with a contractual relationship on one end. The other extreme is the user content where there's very little control over what is posted.
... These should be handled separately. Shouldn't come up with a static way to handle 3rd party content. These cases may need to be handled differently.

Shawn: Do you want to start talking through one of these?
... I'm forgetting preceding list items.

Katie: User generated content, contracts, that is getting into contract law. That's the thing you do, our govt in the US that whoever hosts is responsible. There are interagency agreements.
... I'm not sure that this is in the purview of our discussion.
... I think it gets a lot more complicated when you get into user generated content where there is no contract.
... Unless they sign their life away. I'd like to hear other peoples opinions, maybe later.

Third-party content

Katie: Orgs end up doing this. This is less about compliance with a standard and more about behavior of orgs.

<Zakim> AWK, you wanted to say that this area seems like a good candidate for higher level conformance (social content submissions)

AWK: I agree with Katie, we wouldn't... it would be difficult to actively differentiate with our advice on 3rd party content and legal relationships or not.
... Maybe we can address it simply... content not controlled by site owner. If i'm running facebook and I set up something that does additional things so that...
... videos are captioned and content will have structure and headings if I do more, that promotes my site.

Charles: Trying to avoid a can of worms and derailing.. I want to point out that user generated content can have a very big impact on other users. For example.
... Codepen, user generated content... I as a user an create something malicious that strobes. Codepen can't be held responsible unless they had a mechanism to prevent.
... Does codepen have a policy to moderate?

Shawn: Someone twittered flashing content to an individual with epilepsy.

Charles: The content providor has a policy to remove such content, but it takes time.

AWK: There's policies to prevent things such as inappropriate images of children, but there's an alert process that identifies the violations.
... There's some moderation of sorts. But those things have to be... many thousands of transactions a day. If codepen can do something more...
... "We've set up moderation to help prevent flashing content". "We are doing more than the mininum", and the minimum is warning users to not do those bad things.

Shawn: That's backing up the things that are in ATAG. Two sides of allowing and promoting users to create accessible content, to identify the gaps in that content.
... There are some bits around remediation, but not quite the thing we are talking about, where users publish an awful thing, and it's being addressed AFTER the fact.
... Context there is more in the authoring action.

AWK: As Charles says there is all sorts of stuff. Facebook can prompt for alternative text for example, but a blackboard with 19 documents and presentations may have accessibility issues.

Katie: There's also the difference between the prompts are great. Aligning with the kinds of things we think will go into legislation such as what did happen.
... Makes sense. Prompts and so on and so forth. But if you are going to allow authors to create content on your site, provide an accessible editing tool.
... We can do things like that. We have to do things that address the technical specification and has a technical thing to be able to say we did that.
... Not just remind what's helpful for users. If you add a widget a user can use, you get one that meets the requirements for the users.
... What an individual provides, I don't know how far that can go. Start out prompting the user.

Shawn: In the doc Alistair commented, the site should provide the tools but can't be held liable...
... <pasting>

<Lauriat> From Alastair: Probably not a decision for standards, but legislators. Also, a platform can 'tilt' users in a way that is more or less accessible. E.g. the character limit in Twitter encouraged people to post images of text. Twitter could auto-generate alt text for those, or provide an extended-tweet feature.

Alastair: General thought, we are helping people measure accessibility, but we don't get way to complicated.

Shawn: If we stay on Katie's point on the technical side, point out the issues.

Katie: A couple of techniques (a method) make sure that we provide a place to author content that is accessible, and possibly allow a user to skip the content.
... A site gives the user the opportunity to interact with the content or not.

Janina: That's important, you are giving tools to provide indicators, such that someone CHOSE to ignore it.
... As opposed to "my hands were tied and nobody helped".

<Zakim> alastairc, you wanted to ask whether there could be a platform-oreinted conformance

Alastair: Platform accessibility, I'm use to a sampling approach. In facebook example, you take a sample and look at a user's version.
... Codepen is one extreme where anything goes, twitter is on the other end, restrictive. Instead of a sampling approach, maybe there's a method of allowing things that can be done on platform.
... Twitter plain text maybe.
... There will be a surface of accessibility issues where the users is limited to what they are allowed to do, that can only be accessible and what guidelines would be triggered.

Shawn: That allows somebody to say "this is what is possible to post and create" and how accessible it would be by default, but not necessarly how accessible it is.
... You can't prevent someone from creating inaccessible content.

Alastair: But you can tilt it towards accessibility.
... ...if they were better at encouraging alt text. You could work out that surface area and score on how well you are enforcing accessibility.

Janina: Looking for patterns of where things fail. Who is failing more often.

Katie: The AIPs will be there with privacy.. it's their...

janina: If privacy, why not accessibility?
... I like Alastair's suggestion. It's a good approach to develope further, better than sampling.

Shawn: Is there another aspect Jeanne? I think we are coming to a solid resolution.

Jeanne: Chuck is great at scribing!

Shawn: Did we cover the user generated content well enough?
... Is there another one you want to go to Jeanne?

Jeanne: Let's go tackle "substantially conforms". That's huge. People in AGWG have a lot of experience.

Substantially Conforms

<Lauriat> Review: "Substantially conforms" came out of the Silver research where companies had a generally accessible site, but it was so large or updated so quickly that it wasn't possible to guarantee that it was 100% conformant. Facebook was an example of a site that was literally impossible to test because it was updated tens of thousands of times per second.

<alastairc> scribe:alastairc

SL: <reads pasted text>

<CharlesHall> relates to: https://docs.google.com/document/d/17ByXEqqXuqWtDzxq2J6Vch27n4RWMZig7P426aPiIto/edit?usp=sharing

SL: Some overlaps with what we talked about, but put user-generated to one side. There are other very large sites out there.

<janina> q_ to say that some sie functions cannot be exceptions, eg. login

Katie: The only way I see, without gaming it, is to either: have a section of the site to test, or ... [missed second way]. Everyone's site may be rapidly changing in future, can't just let that go.

<Zakim> AWK, you wanted to say that the threshold for full conformance claims is substantially lower than facebook-size sites

AWK: Don't know if you can entirely prevent gaming, that depends on what companies are included to do. Main point: The size of site, where making a 'full' conformance claim gets difficult, is well below the size of Amazon.com. In small sites we ID key workflows, may represent <25%, but still very big expense.

SL: I've been thinking about it as getting a perspective on everything, and looking for other ways of slicing it. E.g. NY times. Large site, perhaps not as varied as others, but still have news, comics, crossword puzzle, probably done with different teams, management structures, cost centres etc.
... could they declare their own scope of conformance? E.g. our classified section meets silver...

Charles: Different ways to slice, we've previously discussed task based. Mostly happy with throwing away 'page based', but substantially conforms could be task associated. E.g. site has authentication requirements, so auth is a task. Then 'get a document', 'make a thing', whatever the task is becomes the task.
... task-based assessment idea, should lead to task based conformance.

<Zakim> janina, you wanted to suggest some functions are too critical to be exceptions e.g. authentication

Janina: Would want a stricter conformance for login, payment tasks. Other aspects may not be as critical.

Charles: I'm suggesting that if you have a list of tasks, e.g. top 100. If they all pass conformance, the rest of the site might not.

AWK: Our guidance around conformance claim could specify that certain tasks (legal/financial) those must be part of conformance claim. But could exclude other areas (esp legacy content)
... like the idea of the site defining it's own scope. Doesn't change what happens when end-user makes a complaint, that can still happen, but at least the conformance claim says what was tested/conforms.

<Zakim> MichaelC, you wanted to say partial conformance, minimal requirements, procedural conformance and to separate hosting from posting and to mention critical functions

MichaelC: For task-based, was thinking about critical functions, and minimal requirements. e.g. any site that has the least bit of content shouldn't have flashing content, even from user-posting. Could have a below-bronze level of 'don't kill people'? In WCAG 2.0 the answer was 'partial conformance', want to avoid straying into that approach.
... one reason we had that was due to content we don't control, but talking ATAG-iness, e.g. wordpress have both content and hosting functions. Some they can make conform, other areas they have to provide the tools for users to conform.

<Fazio_> Making the tools discoverable is important

MichaelC: a user (including commercial entities), could conform for their facebook presence, and facebook would have to provide the tools for them.

<Fazio_> Often times a11y tools are hidden

<Zakim> jeanne, you wanted to talk about Large Websites paper

<jeanne> Challenges with COnformance for Large Dynamic https://docs.google.com/document/d/17ByXEqqXuqWtDzxq2J6Vch27n4RWMZig7P426aPiIto/

Jeanne: I should have said - Peter et al wrote up a paper on challenges of conformance for large / complex dynamic web sites.

<Lauriat> +1 to Fazio_, and ATAG talks about prominence of these things that we should definitely look at.

<jeanne> (1) numerous WCAG Guidelines and Success Criteria expect human involvement to verify conformance, which is challenging to scale to large sites and to dynamic sites; (2) large and dynamic sites have portions that are always under construction, which is challenging to address with the WCAG 2.x conformance model which makes no allowance for that; and (3) third parties frequently add and change content on large and dynamic sites, and related non-conformities in[CUT]

Jeanne: <pastes text and talks through>
... e.g. this part is being worked on and may not be conformant. Also, the 3rd party issues.

<AWK> Thinking: "Conformance = a level determined by an evaluation for a set of tasks on a page or collection of pages that is defined by the provider, and does not necessarily include 3rd party content. Conformance is based on the tasks and may not include all aspects of every page that is part of the tasks"

Jeanne: I assume there are more details, but people can read the doc.

Janina: Good summary, and Peter and I have been back and forth on this draft. Would like others to contribute thinking. Only other thing - in a complex & dynamic site, different depts might have things under construction, but will be responsible for different things. Some developer somewhere else might have changed things.
... there's a large urban env analogy, things happen that you find out through some mechanism, then people show up in a truck to fix things.

<Zakim> alastairc, you wanted to say defining task will be tricky

<AWK> AC: Defining tasks will be tricky, although if the org defines it then they own it

<AWK> ... as a 3rd party to define them is harder

Charles: Just to add, there's another type of dynamic condition, e.g. AB testing, with 12 variations of a page, and conformance may have tested one of them.

SL: Comes back to non-page based conformance, and you'd need to define that as part of your claim. You'd probably wait until after it's done.

<Zakim> Makoto, you wanted to share my document "Conformance model of JIS X 8341-3:2016"

<Makoto> Conformance model of JIS X 8341-3:2016 https://docs.google.com/document/d/1SVDWemejSSBPPqJl4t_KBGeXWsFNWjHv0JW1y6RWgdg/edit#heading=h.ltnf1nkl7yg0

Makoto: In Japan we have national standard, adpoted WCAG 2.0, published 2010. At that time the working group, we found issues for larger websites, so this doc
... for large websites, we took random sampling, for dynamic websites we took screenshots to snapshot the claim.

<Lauriat> ack 3d

Kim: Sometimes, e.g. NY times, you may have content that is copyrighted, and may not be able to change it. Or if a comment comes in as an image, could put in alt text, but what about cartoon part? Sometimes you can't change it as it is, and with large volume you couldn't feasibly tackle that.

SL: For things like comics, e.g. someone used non-contrasting colors, not sure we have solid solutions.

AWK: For 3rd party stuff, if they hire people, I'd hope that's in scope.

Charles: Wouldn't we need to make exceptions in the guidelines itself?

SL: Noting 3rd party stuff as case.
... E.g. you want to claim conformance, here is how to express some of the things you might hit.

Kim: We have a series of books we publish & host, we can't change those. It belongs to someone else.

Katie: Remind us that the task based things may include permissions as another path.

<Zakim> alastairc, you wanted to say that users will encounter these things

<AWK> AC: exp around dynamic content, users will be encountering things but we want to measure accessibility and let the legislators make some scoping decisions

Chuck: Could throw in simple statement around not requiring things that would require people to break laws...

<Cyborg> re: what Chuck wrote -- except what if law works the other way, and ends up enforcing Silver?

SL: 15 min break, back at 10:45 (Japanese time)

<Cyborg> could be circular argument

<mbgower> rrsagent make minutes

<scribe> scribe:Jake

Jake: Were we still talking about conformance challenges, and the 1-3 from amazon?

<scribe> scribe:alastairc

SL: Covered quite a lot, but there is a lot of complexity. If there's something you'd like to pick up from?

Jake: My thoughts, are we agreeing? Those 3 points, I just thought they were disappointing, and not what we are looking for when looking for challenges with conformance. Reasons: The 3 issues are in general for all companies, tend to be the excuses we all hear, e.g. too much human involvement needed.
... 'there are too many user needs', but that's why we are there in the first place. If we want to fill gaps in user-needs, and create guidelines, it seems like a weak excuse. What's the result, do we skip some user needs? That's not why we're here.
... those are the facts to deal with, and then ask what the challenges within the company as to why you can't comply.
... E.g. if you don't fulfill the criteria, don't go live.

SL: In a team I was in, we were launching 12 times a week. We aren't ignoring it, it is about how to not make it more difficult.
... there are many ways to do it, and many ways to alleviate it, e.g. with a focus on scalable testing. Even if there is still a human to review the results, we could alleviate the need to do lots of human testing.

Jake: Seems like there is a need to find solutions.

<mbgower> +1 to SL's comments on scalable testing

Jake: the large dynamic sites & under construction, that's a fact, the challenge is how can we still make sure... like with AB testing... I just hope that there's a lot, and do AB testing, and simple example: We didn't use heading element because it's under construction, the core basics can still be there when under construction.
... that's what we have to deal with, not the challenge.

SL: On a given page, there could be many tests going on, so page level conformance doesn't work. Other approaches could, e.g. task-based.
... agree with things like AB test, should be tested before going live. But, the declaration of conformance is a concern here.

Jake: Same with 3rd parties, could have procurement that includes WCAG.
... the challenges tend to be that the teams going live with a feature don't have the knowledge, not about 50 criteria.

<Zakim> jeanne, you wanted to say that we want people to be able to conform

Jeanne: Some context - when we look at basic principle, improving accessibility for disabilities, some of the biggest sites are the most important for those people.
... if the big sites improve, makes a big difference. The companies are making improvements, even though they want to improve.
... want to provide a path to improve that is recognized.
... this discussion is more about adjusting the things that are structural so big orgs can claim conformance.

Katie: Are we still talking about substantially conforms?

SL: Bit of tangent, but mostly.

Katie: Understand that point, but those are not most websites. If we go in direction of path-based, still need a concept of rendered /screen-based for many other websites.
... have to be careful about trying to give big orgs a break.

SL: +1 to not ignoring other sites. E.g. for local people shop, the tasks may be view contact info and view menu.

<Zakim> AWK, you wanted to talk about hurricane ties

<Rachael_> +1 to covering the smaller sites as well

<jeanne> Katie, this is not about giving the big orgs a break. We have a lot to support medium-size sites, but the large sites have needs that it will benefit people with disabilities if we can address them.

<mbgower> +1 to SL. Identifying functional processes and measuring those is an effective way of assessing a site.

AWK: Agree we don't want to ignore the other sites. Comes down to what you can practically look at. E.g. a housing project, you have something built. There are building codes, e.g. for hurricane ties. They don't check every tie. But they do check the Ground fault circuit interrupter, as it is critical. They don't look at the morter between each brick.
... on websites, you might not check every heading, but you can't practically watch every time they do that. So IDing tasks/flows, or page by page. E.g. 3 page site you test every page. For larger sites you ID tasks and use that to dig in further.

<mbgower> AWK, is it also possible to allow the person doing the 'hurricane ties' to report an accomplishment and roll that up into a site report/measurement?

AWK: Many sites are too large to evaluate in that way.

Chuck: I don't look at it as giving big orgs a break, but at the moment they can't conform due to scale.
... we should be taking away an excuse.

Katie: Does that take away the responsibility for individual cost-centres to be responsible. Want to cover all types of websites, inc XR stuff. Need a name for 'current screen', not viewport.

SL: Couple of aspects, 1. Clarity for those making things accessible. 2. Transparency about whether something was done incorrectly, and they know reason for guidance, they can make better judgement calls.

<Zakim> alastairc, you wanted to say about ISO 27001 approach.

<Chuck> alastairc: This reminded me of previous thought around the approach ISO # takes. Not in detail, someone else took us through this. You start with a benchmark of where you are now (not passing everything)

<Chuck> alastairc: And you put together a plan on how you improve the deficiencies. Its not a back and white approach. "we don't have badges now, we will have badges next year".

<Chuck> Katie: That's the remediation plan.

<Chuck> alastairc: Org wise it's nice to not get a big fat fail.

<Zakim> mbgower, you wanted to say there may be ways we can construct measures which would allow us encourage conformance at a more granular and earlier stage in addition to a 'final

MichaelG: Wonder if there's a way to roll up, to apply towards conformance. Still have to go through list, but moving to an agile process. No longer waterfall with big test at end. As you go along. As a webdev, do these things.
... e.g. on the component you're working on now. Proof of actions take, rather than final proof.

SL: That kind of dev practice, doesn't negate an end result at the end.

MichaelG: It's hard to have that stuff rolled up into an SC. A lot would have to be a different way of working.

Jake: Thinking of more holistic approach, what are you specifically looking for in new conformance model? If it's holistic, we devine that 'this' should be done, and you have done it or not.

<Ryladog> <scribe> Ryladog

<scribe> scribe: Ryladog

Jake: This isnot a lot of grey area? Is between 80% and 100% is OK?
... Part of the Dev Cycle, is there challenge. That is not these are the user needs and these are the methods?

Shawn: No, a CM that goes beyond flat websites

SL: a CM that more accurate reflects the experience for user with disabilities

<alastairc> Katie: Michael said we could include things from current model that are blockers, those could be part of it, ensure not destroying the user-experience.

AWK: That is what we have in the current NM

SL: Yes

I thinkwe can ensure that we have non-interference SCs in the new CM

Jake: Orgs have depenndencies

SL: That is the companies proble,

<alastairc> Katie: User-experience is what the user gets, regardless of the where things come from

Jake: The end user is only interacting with one thing at a time if 500 things are going on in the background

Alastair: If can components at the top, scrollable areas

SL: Yes

<AWK> FYI - the example about a page with an iframe that is used for some control function and not meant for user consumption and it fails because that iframe lacks a title - "web page" is defined as "a non-embedded resource obtained from a single URI". The iframe is embedded so "page titled" doesn't apply.

SL: (team looking at a google ui) taking the users that is right in front of me
... Docs is good but Calendar is not -

Jake: but the CR allow that today, you can scope out an area

SL: My point in not that you can excuse this because this is not at the page level
... In docs there are tons of UIs that are in the Docs realms

Jake: that is my experience for maybe proper reason, this is not a webpage. If you ask 100 people and they will give you a 100 names

SL: My point is application are going to create there own concept of a webpage
... Come of with a concept of scrope
... Give examples
... w3cwould not definethe scope, the organization would

Chuck: The way that I have been thinking about this Schrödinger's page

SL: We are moving on

we need a NAME of currently rendered view

Should there be minimum requirements?

JS: Jake raised, shoud

ld we have minimum reqs?

scribe: we have been talking about a point system
... people would nt have to do every singleGL, but they would have to have a certain number of points they would have an overall score
... one thing we kee coming back to, like the non-interference SC.
... if you make a more basic than that

<Zakim> mbgower, you wanted to say there are conformance claims and conformance claims

MG: we express conformance through some kind of VPAT, my expereince is that lots of sites dont comply with various levels of accuracy
... I am worried about people doing less
... captioing hasnt really been that strong
... it is rare to see videos properly captioned and described for both deaf and blind users
... it would be good for sites to be accurate with what they achieve, and for those who go beyond the basics to be able to claim it

SL: they have a way express this is a VPAT, It can be a report of hw they dont comply

AWK: I think there will always be a e=need for that. The complaints wehear are that ACR/VPATs are very lengthy

,,,,,in order to reach bronze, you should e

be able to say we have reach 80 of 90 things

<Zakim> alastairc, you wanted to say that without page model, still need non-interferance (and more) for tasks

scribe: you wind up with site that dont claim anything

<mbgower> Right, we want sites to be able to report on what they've done, even if it doesn't fully comply or meet a target

Alastair: if we are not using a webpage model, and a page includes a calendar, I would want the entire rendered view - we would want conformance to cover that

SL: Non-interference has to remain at the top, beyond that if the browser just plashes at a crazy rate - there are all kinds of lelves of Non-interference

Jake: Beyond non-interference, what do you do?

DF: I am sceptical that people will want to talk about what they are NOT conform
... maybe adding a Readily Achievable
... you are mapping out a lawsuit

SL: I didnt see it as that, For these cases, what we have done

CharlesHall: If the CC partial without identifying the gap?

DF: Whether or not you say what you have met is not

SL: We should have a mechanism to do what they have done

<Zakim> AWK, you wanted to talk about non-interference challenges

AWK: I have seen many sites where they do say that we are working towards this X - that is the companiesdecision

<alastairc> Having a graduated approach which does identify areas of non-compliance would fit nicely with the EU public sector bodies approach.

AWK: Part of what we talked about before, if a company is defining a task and a flow - it is going to be hard to get into a page without a title
... if you open a page
... if you have a page with 4 buttons, my task is to run query

<alastairc> Katie: If we still require some things may force AT to continue to support. E.g. in VR, a battlefield, the title of the page should be battlefield, however it is done.

<Chuck> katie: If we still require them may force technologies to implement things, say XR experience (the battlefield). The title of the page should be "battlefield" however it's done in that tech.

<Chuck> Katie: We want to keep doing the things that have been done. What is the tech capable of, and how do we push the technology further?

<Chuck> Katie: They are providing the same functionality in a way we know that AT can pick up relative to AT. Not specific to AT, but like that.

<Zakim> bruce_bailey, you wanted to say that advertising partial conformance is probably not inviting lawsuit

<Fazio> My Fortune 200 clients have

BB: I want to reply to that having statementishaving a lawsuit, I think that is legitimateto fear. I think you are more likely to havefear ifyou dont do anything at all

<Fazio> Which is why all our testers were required to NDA

BB: I have heard her speak, if the university has a plan and they go after

<AWK> (her=Eve Hill)

SL: This is also overlapping with the clear and transparent

<JakeAbma> scribe: JakeAbma

Chuck: MC mentioned a less than bronze possibility

MC: we might want to say, not complying with bronze is not killing, but at least you can do "this...""

SL: would like to see as an example that we can say "docs" is conformant, but the calendar is excluded / has it's own conformance model

<alastairc> AC: What if the tasks don't cover the site/application very well?

<Zakim> AWK, you wanted to ask about ACT tie-in

AWK: I'm a little wary if we introduce a "level" lower than bronze
... like you see with the fast pass of MS Insights

SL: automation doesn't tell you what's right, but at least what's not wrong

AWK: ACT gives you possibilities on what may be checked automatically
... in 2, 5, 10 years automation will improve

<mbgower> agree that a clean automated tool should be a baseline on which other scoring takes place. Say you get 40/100 for a clean baseline? And as ACT grows, the baseline amount increases.

<Zakim> MichaelC, you wanted to ask about web page unit of conformance, coformance scope, task based scope, function scoope and to say scope / tasks correspond to users and to say the

MC: in WCAG 2 in conformance we talk about web based, where some pages are more important, not ideal...
... essential functions might be achievable for conformance, not all 10.000+ pages, to make it more feasable
... we should ask people to not make websites which kill people

<AWK> (we are willing to ask nicely for that)

<Cyborg> just mentioning that killing can happen fast or slowly

MC: we should have statements added to not do that

<Zakim> mbgower, you wanted to say has silver TF come up with a list of common functions?

MG: did the TF come up with a list of functions or tasks?
... we might come up with common functions

SL: a organisation could say, the top x functions are conformant

<Zakim> alastairc, you wanted to discuss sampling tasks?

CH: it's up to the organisation to define their tasks, we need to provide them guidance

<mbgower> In my experience, most testers create a sampling.

<mbgower> At IBM, I tend to recommend that testers identify the critical end-to-end tests to run to confirm the entire process is accessible.

<Zakim> AWK, you wanted to speak to the onus of responsibility to define tasks accurately

AWK: companies may choose specific teams / persons work to make a conformance claim, because they know they did a proper job
... there are part of sites we companies know are not conforming
... it doesn't take away the responsibility

<Zakim> MichaelC, you wanted to say task based conf claim nearly as easy a page based

MC: agree with AWK
... task base conformance claim can be as easy as page based claims
... in favour of task based

<Lauriat> RSSAgent, make minutes

Silver Project Plan Needs

<Lauriat> https://docs.google.com/document/d/1zFgVcDUMSOrZ5nnGRocs2pZYkqOhwdyMU_Z62_CedbQ/edit

<scribe> scribe: JakeAbma

SL: we will talk about Silver process
... drafted some categories: Process, Structure, Content,
... nothing to do around conformance model / testing
... it's about how we work , the needs of the people involved creating Silver

<Lauriat> Reposting link to the Silver Project Plan Needs: https://docs.google.com/document/d/1zFgVcDUMSOrZ5nnGRocs2pZYkqOhwdyMU_Z62_CedbQ/edit

SL: like involved groups, community groups, testers etc. on how we want to do it
... also the WCAG WG itself

MC: seems 2 questions: are these the ones to address?
... next is how to execute the plan... ?!

SL: my goal and expectation is go over the outline and high level thing on "this might be our process"
... how do we handle some new shining technology / platform
... we cannot provide guidance for all new emerging technologies, but we need a structure
... and how do we maintain these

MC: we need to plan this AFTER the 1.0 release

SL: can be moved lower, but maintenance is one of the hot topics for SIlver

MC: want to split Communication paths to internal and external

SL: how do we manage updates?
... want to avoid, "if my thing doesn't make it, it will be postpones for the next 10 years..."
... need to establish a clear path for this

MC: want a clear separation for different parts of iterating

AWK: WG indicated 18/24 months for updating ( 2.x) and now we might have capability to continuous updates
... governments are not able to update dot releases / versions as much as we publish releases

MG: discussions are going on for providing "living standards"
... process features might be provided
... no clear ideas yet, but talk are there

CH: will there be a deviation between normative and informative?

MG: yeh, there will be a determination of some sort, probably

<Fazio> Governments won't like standards that are continuously updated

SL: we haven't decided what will be normative and what not, take the methods as an example

<Fazio> It was 20 years between ADA technical guidelines update

<Fazio> even that was contentious

SL: was there a process change for 2.2?

AWK: not really, only the acceptance requirements, and also have talked about and promoted some changes on how work is done
... specs / editors draft will be published in Github, but work can be developed in environments of choice

<alastairc> Working process for WCAG 2.2 SC creation: https://www.w3.org/WAI/GL/wiki/WCAG_2.2_working_process

SL: content migration is pretty straight forward, other parts not yet established, like conformance model
... how does the WG see this working best (in larger projects)
... 1 person maintaining all, or phone calls not always work fine, like people in other countries and all work done in phone calls they can't attend

AWK: critical parts do require maintenance, high level of organisation is critical
... it's very hard if information is scattered around the place and hard to find

SL: tools are hard because they're not perfect at all, never came across a proper tool

MC: we need to explore options for good tools

<AWK> Scribe: AWK

SL: The PM tasks would be pretty deep

MC: Maybe 2 days a week

AWK: That's 4x the advertised chair committment

MC: May need to borrow from the 2.x acceptance criteria process for SIlver
... May be outcome goals - "does SIlver include X number of 2.x SC" etc
... expectation is that for SIlver 1.0 we will have ported everything that we want to and added some things

SL: Version 1 of Silver - (see page 4)
... option 1: silver=wcag 2.x
... option 2: silver=Wcag 2.x + something

(general agreement that silver uses option 2)

SL: What does "some new stuff" mean?
... JSp and SL thinking that is a modest amount

MC: effective requirements include some emerging technologies

SL: if Silver has no more than WCAG 2.x, why would people take on Silver?

JA: perhaps for the new conformance model?

CA: idea of adding some stuff is of interest to valid the extensibility bit

<alastairc> AWK: Part of the challenge is that until the conformance model is established, hard to scope what is scoped in/out.

<alastairc> ... if that's missing requirements, don't know yet.

<alastairc> SL: To validate we're using prototypes, but agree that we don't know where that gets to yet.

DF: don't want to dilute the integrity
... can't make a new version just because - it needs to do something extra

SL: Management of iterations?

AC: Should the deliverables include the list of things that are needed to get new requirements into Silver?

SL: Added to list

KHS: JSa suggested about an "explainer"

JSa: We can't expect the AGWG to fully understand all of the documentation in the wiki - something simple to help new people and visitors understand

MC: Having an explainer may become a requirement for specs in the future

SL: Lots of items around the info architecture
... we are going to need a lot to accept new content
... e.g. tagging means need a tagging database system

AWK: We have done something like that with text files and exported content

MC: Not sure that there is a relational DB that is needed

SL: support for delivery of Silver in an API is in the plans that have been conceived of

MC: technical requests go through michael

AWK: For 2.0 it was Ben Caldwell that provided that technical support services

CH: would an API be used for translations also?

MC: maybe?

AWK: Would probably help?

SL: Question about how and when we rollover silver work to AGWG work.
... we've had some more people come over to help with Silver but haven't wanted to have 50+ people working on everything

MC: hearing two questions
... 1) when does the WG want to start on 2.x and switch. Right now thinking is 2.2, then Silver
... 2) question about when is an appropriate time to accept new help

SL: important to get timings worked out

<Zakim> bruce_bailey, you wanted to say i am one of the people hoping for a 2.5 or 2.9

BB: Want to agree with SL that Silver would need to have more req to be appealing
... also that the USAB is looking for a stable version before incorporating into regs

KHS: other things in project plan are implementation plan and more - once that stuff is in place the WG will be able to work with it
... New thing needs to be good, not half-@55ed

<Lauriat> +1

KHS: expect Silver will have holes that weren't anticipated

<Zakim> AWK, you wanted to say ha ha katie and to say WG needs to start working before 2.2

<Chuck> awk: One thing that I feel strongly about is that I don't think there is a grand switch-over. Not 2.2 milestone then immediate move to Silver.

<Chuck> awk: We need to do it partially, get less done in 2.2, and that's ok.

SL: yes, this is what we talked about when we needed more help around conformance models.
... getting more people involved
... with the project plan defining work that is needed and finding people to help

<Zakim> alastairc, you wanted to talk about trigger point for when silver is ready for all-in

KHS: Conformance model discussions in 2.0 were hard, long, and everyone was part of it

AC: avoid having everyone all in on conformance model discussion
... somewhat before writing requirements

<Zakim> AWK, you wanted to talk about conformance revieew

<Chuck> awk: even within 2.0 there are some people doing more of that work in the writing. 2.0 the conformance model was the main thing and everyone was contributing.

<Chuck> awk: It's ... there are people doing the heavy lifting. We have to make sure that people in the wg are aware, and that if they want to be a heavy lifter then need to pick up some weight.

<Chuck> awk: The wg is going to expect that they have an opportunity to provide comment and thoughts on what is in it. And I expect that we expect that to be the case.

<Chuck> awk: It's best to come up with a conformance model note so that the main wg engagement is around "are we behind this note"? Then that piece is locked into place.

KHS: so not a couple of options?

<Chuck> awk: I think that providing updates (monthly)... here's what's going on in the conformance model, giving a status, and convey our next steps.

<Chuck> awk: If you see something before then... march forward. At some point it's "here's the pros and cons", and then the wg has the big debate.

<Chuck> awk: There's going to be something good and not as good in the conformance models, the wg will want to be involved.

<Chuck> awk: ...in helping assess where the weight is placed based on pros and cons.

SL: yes, even as we get to the prototypes there may be ways to combine aspects of different prototypes

KHS: Concerned about a conformance model not being able to handle some aspect
... e.g.. media

SL: That's why we are defining how to test conformance models
... if there are known gaps we can address now

AC: We do have the existing WCAG SC and things that haven't gotten into WCAG and some ideas around emerging tech to help test the conformance models

<KimD_> https://www.w3.org/WAI/GL/task-forces/silver/wiki/Welcome_to_Silver

Example explainer doc: https://www.w3.org/TR/personalization-semantics-1.0/

https://www.w3.org/TR/motion-sensors/

<Zakim> bruce_bailey, you wanted to say i am hearing MC say no 2.3, so that means USAB will be waiting for Silver 1.1 at the earliest

<Zakim> AWK, you wanted to ask about FPWD

For reference, the WCAG 2.0 FPWD: https://www.w3.org/TR/2001/WD-WCAG20-20010125/

<Fazio> scribe: Fazio

<MichaelC> mc: our charter under review indicates FPWD soon, think we need to honor that commitment. It can be minimal, an intro and a few ported SC and description of the structure

MC: SL: concerned about structure of guidelines
... need 1st working draft in November
... early 2020 for focus to shift to silver

<Lauriat> SL: Mostly concerned about the assumptions of usage of structure, rather than the structure itself.

AK: AGWG should shift to silver focus when silver structure and model is ready

<Zakim> bruce_bailey, you wanted to ask if 2.3 might be a small tf

Focus on silver more important than developing WCAG 2.3

MS: Focus on silver more important than developing WCAG 2.3

MC: Focus on silver more important than developing WCAG 2.3

SL: need different name for silver for 1st public working draft

Migration of existing WCAG content to Silver: building up tests

<Lauriat> SC 1.3.4 Orientation, SC 1.4.10 Reflow: https://docs.google.com/document/d/1aCRXrtmnSSTso-6S_IO9GQ3AKTB4FYt9k92eT_1PWX4/edit#heading=h.hirzywsbl4d

<Lauriat> 1. Users who cannot change device orientation.

<Lauriat> 2. Users who need the ability to view content in the orientation that works best for them.

<Lauriat> 3. One column view in responsive design: A site uses responsive design. When a person zooms in to over 300%, the layout is reflowed to one column. The user can read the content easily and does not have to scroll sideways to read. PDF offering reflow: In a PDF created to conform to PDF/Universal Accessibility (ISO 14289), the content can be reflowed and zoomed in to make reading possible for someone with low-vision.

<Zakim> MichaelC, you wanted to say user needs should describe functional need not user characteristic

MC: user needs look like characteristics
... user needs need to be reframed

AK: you can combine reflow with resizing text
... reflow building on text sizing that was insufficient in WCAG 2.0

Jake: reflow part is what is happening in viewport
... wants to split up parts of reflow

Charles: Usage with or without orientation - suggested rewrite

SL: can be brief with user needs for now
... trying not to list characteristics

Charles: better to say single axis instead of column

<Lauriat> Understanding documentation for Reflow, for context and reference: https://www.w3.org/WAI/WCAG21/Understanding/reflow.html

Charles: cognition needs missing

AK: valuable to list types of users that benefit from each SC not sure if that's the right level of where we need to go

<CharlesHall> the proposed format: People may have [with/without/who can/cannot] [action or function] due to: human factors (like [example of disability]); technical factors (like [example]); or contextual factors (like [example]).

Jake: situational needs are most powerful for listing user needs

SL: should we be framing in usage of mobile device?

MC: Would like to be device agnostic

Charles: examples cases fill in gaps

<KimD_> Do we need something about scrolling and limited mobility?

I wonder if the cognition is different for Japanese Chinese etc characters

Charles: user need is ability to use without losing place

Jake: sticky headers/footers with text are an issue

<janina> .

Kim: limited manipulation to only scroll in one direction

<CharlesHall> https://docs.google.com/spreadsheets/d/1W5CSvU4XxWXNneu4ibokjcYUCsG386xL1rGOiTrDvt8/edit?usp=sharing

<Lauriat> Direct link to the tab in question: https://docs.google.com/spreadsheets/d/1W5CSvU4XxWXNneu4ibokjcYUCsG386xL1rGOiTrDvt8/edit#gid=1834752082

<Lauriat> trackbot, end meeting

- DRAFT -

Silver Community Group Teleconference

16 Sep 2019

Attendees

Contents