<Cyborg> Presen+

<mikeCrabb> Jeanne: Dave is one of the founders of the silver project, has stepped back recently but is very familiar and passionate about the work. Heavily involved in the research and has conducted reserach studies to look at previous WCAG and how easy they are to understand. Has done a lot and is here to help with having PwD more involved with silver. We didnt have anyone in the group that was active in this area so I've invited Dave to the meeting today.

<mikeCrabb> Dave: Really interested in finding out the questions that focus on the discussion. Lets start with that. There are a number of different ways that we can involve PwD, so if we are looking at trying to figure out how we can include soem form of commitement to see how we can include PwD in conformance with silver then that would be an interesting start

<mikeCrabb> Jeanne: thats quite new stuff that @cybele has been doing, really closly related to silver conformance but it is important to look at both. When we wrote the Ag requirement document we got a deal of pushback when it was presented, people were saying that it wasnt realistic to think that testing with PwD did not produce measureable/testable responses where you coudl say to a regulator this passes and this doesnt. We have had some discussion about air

<mikeCrabb> line regulations where they have to show that something is usable and we started to see limitation there.

<mikeCrabb> ... that was when we thought we would bring @DaveSloan back to help out with this area.

<mikeCrabb> Dave: Thats helpful, are there any particular questions or discussion topics?

<mikeCrabb> JF: Last week there was a thread about the cost of testing, I was adiment that we have to involve human testing. I want to involve PwD in terms of testing but from the regulatory perseoective there has to be a measurable metric. The issue is that "at the end of 1 hour you will konw the experience of 1 user". Part of the testability that we need to look at is the constant measuarable measure

<mikeCrabb> ... when we start injecting single users we start to throw it off, we start to test the user and not the content

<mikeCrabb> Dave: There are all sort of challenges to setting quantitative requirements. Disabilities represented, number of people. All of these quantitative measures have some shortcomings. It is easy to talk about limitations of trying to achieve documented definitions of what is acceptable but the discussion on cost was an important one in terms of the cost of testing.

<mikeCrabb> ... doubts were expressed about the cost of testing WCAG2 and implications that this has on accessibility for organisation that have limited budgets. It seems like one possible angle to explore is this tiered level of conformance where the gold standard is one where you have evidnence that you have included PwD in design/development/evaluation process. At this stage its just one attribute of the highrst level of conformance

<mikeCrabb> ... increasing the nubmer of people with each level would not be where we want to go, and also do not want to have token involvement.

<mikeCrabb> ... same people would be used repeatedly. Brings particular bias etc.

<mikeCrabb> ... One thing that I think might be helpful is to look at existing standards and legistlation that require evidence of involvement of PwD and find out what that looks like

<mikeCrabb> ... CVAA legislation, any a11y standard or legistlation that requires evidence of involvement of PwD. Need to look for normative guidance on how many, when who?

<mikeCrabb> ... I dont know if there is a defined level of what is enough, you need to do something but what?

<mikeCrabb> JF: Do you know if there **IS** anything that exists like this? I'm not aware of anything that does this. One of the problems with standards is that they need to be measureable. This introduces a random factor seat. As soon as we bring in real users we get the bias attached to an individual user or group

<mikeCrabb> ... users from different areas will have a different idea about access

<mikeCrabb> Jeanne: We're not at that stage yet, we are still at the stage of **COULD** be measurable or **COULD** be standardised

<mikeCrabb> JF: Are there any standardised usability metrics out there, formalised metrics that can be used?

<mikeCrabb> David Sloan: I'm not aware of anything obvious, we can look at this in a few differnet ways. What type of activites do you imagine, not looking at quantitative (the how and who) but we are looking at the types of activities and evidence of how that influences design

<mikeCrabb> ...this is a problem in a more meaningful area to look at

<mikeCrabb> ...if an organisation can show that they did something and this changed their decision methods in a meaningful way then they can show that it isnt a box ticking exercies. One way might be to look at the types of activiteis that were underating with PwD and then look at how higher level conformance could apply to the stages that PwD were involved. The metric is just "did you do this"

<mikeCrabb> ... the concern is looking at organisation with small budgets and low complexity digital resources that are being created

<mikeCrabb> Jeanne: one thing that I didnt talk about was to look at the conformance model. We are looking at using the leeds inspired Gold/Silver/Bronze. They have a different point system for different types of buildings. We could have a very different process for how a small organsiation with a limit budget would compare agianst a large organistaion with unlimited resources. We dont need to worry about that today but we can look at how to set up a process th

<mikeCrabb> at would be considered vali

<mikeCrabb> d

<mikeCrabb> Dave: Thats a big quesiton and we need to start by defining the different dimentions and definitions of involvement that PwD would have in the design process. What is the objective and is this to help ensure that whatever is produced is as accessible and usable as possible. Not to saythat you have just done it. Is what you are trying to design is something that is pre-validated off the shelf, if so its it less imperiteve to have involvement to have

<mikeCrabb> people throughout the process because what you are doing is already well understood.

<mikeCrabb> ... involvign PwD in these projects could less important than in complex projects and interactions

<mikeCrabb> Cybele: I want to address a couple of the things that have come up, it starts with me wanting to share the slides from last week this might help with where I'm coming from

+1 to looking at the contrast between testing a new product and a product with well used components

<mikeCrabb> Couple of points that are useful, 1 is the concept that pwd would be set out to replace auto testers, I dont think that is in anyones interest

<mikeCrabb> ... https://docs.google.com/presentation/d/1dsgjIBp7uEwRr63Z3ZFZ6tJ78eRG_qKDS-mjBGqaJwY/edit#slide=id.p

<mikeCrabb> Cybele: it could be seen as 1 person, 1 persona but also as an employment oportunity to skill up PwD to be come manual testers

<mikeCrabb> ... 3rd, gold standard being the place to start, I think this is what we want to aspire to, but if we want to be good at gold then you really nede to start early

<mikeCrabb> Cybele : start at the bronze level and look at methodology

<mikeCrabb> ... canadian legislation was to take people in that have not been overly involved and to then get them engaged

<mikeCrabb> ... the third thing that was raised was about the different sizes of otganisations. I think this is one of the larger bariers to address. There has been a reference to leeds and how to address this

<mikeCrabb> Dave: I guess one thing that would help me would be to go back and read more about where things are in terms of current proposed conformance model. What I'm struggling with is to definitvely say if we have answered the question "why would be integrate pwd within the design and devlepoment proces within silver?" Whatever the answer is will help us define how we move ahead in terms of defining what we recommend and how we measure

<mikeCrabb> ... its a major shift from a standard that meausre the outcome of a process towards one that also looks at the components of a process as well

<mikeCrabb> ... not only are you saying what you should end up with but also how they should get there. I'd like to know more about the definition of why this is being brought into the conformance discussion. I'm not quesitonining or skeptical about this but I think it will help the conversation more

<mikeCrabb> Cybele: There have been a number of (anicdotal) incidents where PwD will approach an organisation and are told that it is wcag compliant.

<mikeCrabb> ... second is the issue of confict between the needs of people with different disabilities.

<mikeCrabb> ... third there is an opportunity for innovation in a way where there are a lot of complaint about WCAG being compliance driven and only see it as a cost.

<mikeCrabb> ...early use and involvmeent is a way to promote this

<mikeCrabb> ... also getting ideas from people that live with disabilities to help organisation improve accessibility

<mikeCrabb> ... this move from culture of compliance to culture of innovation.

<mikeCrabb> ... culture change inside organisation on how to work with PwD

<mikeCrabb> Dave: thats a really good list and looking through each of the items, I think it would be an interesting exercise to look at where does W3Cs responsibility begin and end

<JF> +1

<mikeCrabb> ... what is the responsibility of a W3C standard?

<mikeCrabb> ... when we look at organisations helping people with disabilities in a number of different ways. I think there has to be a line drawn somewhere. A standard has a role to play in a cometitive marketplace but there is goign to be resistance if it is percieved that it is telling everyone to do everything the same

<mikeCrabb> ... the list is good for reasons why involving PwD might be worthwhile in silver but is this appropriate to influence a technical standard. Is silver going to be culture change within an organisation, is this part of a W3C standard,, is it overreaching or appropriate

<mikeCrabb> ... next step woudl be to look at all of these arguments and figure out what are valid reasons to extend silver and how to do this

<mikeCrabb> Jeanne: This is a chicken and egg problem. Its very difficult for people to visualise how different what we are doing is

<mikeCrabb> ...we are trying to set up examples that we can point people towards and then have the discussion.

<mikeCrabb> ... what could we do and what could it look like if we did it. Then we have something solid to point people to to have the debate

<mikeCrabb> ...reminder that W3C is a technical standard organisation but WCAG is a guideline and has different standards. We shoudlnt assume that becaues it isnt technical we cant do it

<mikeCrabb> Dave: If we look at the ISO that WCAG is part of, can we have a debate along those lines?

<mikeCrabb> JF: it is important to look at the regulatory environment and how this impacts, it is up to regulatory agencies to adopt/modify these. W3C has taken a lot of time to look at common/global standard but I'm always wary when we look at regulatory environments as a mejor driver. It has its place but it is not what we do

<mikeCrabb> Dave: Good point, one of the list items there was the gap between wcag conformance and successful task completion with PwD, if you can/can't be successful is it unlawful discrimination?

<mikeCrabb> ... if the technical standard falls short there is an issue. Silver helps to close that gap and we are not looking at confromance in terms of technical pages but instead of tasks, thats antoher way to look at this

<mikeCrabb> JF: @Jeanne had mentioned ACAA, there was user testing on kiosks, but to say you had met ACAA the legal standard had a list of tasks that you had to be able to accomplish. The measurability looked at tasks and how these could be completed by a variety of PwD

<mikeCrabb> ...not a specific function on web, there are a large number of tasks

<mikeCrabb> Dave: So the quesiton then is "is it feasible to include a number of key tasks and provide evidence for PwD to successfully complete those tasks?"

<mikeCrabb> Jeanne: Could we have very generic tasks and get organisation that want to conform to set up their own tasks for what people must accomplish

<mikeCrabb> ...basic ones, log in, check out on ecommerce but we dont need to make the full list

<Lauriat> +1, we prototyped this in the design sprint not just in order to take usability into account, but to get away from element-based accessibility and focus on task-based accessibility.

<mikeCrabb> Dave: Maybe tasks are more specific and are somewhat more than what we have at the minute. Less fine grained than WCAG2 (think form interaction, labeling elements and conveying error conditions) but is there somewhere inbetween where forms can be used by a range of PwD?

<mikeCrabb> Dave: looking at interactions in a broader level, there might be something in there

<mikeCrabb> Shawn: looked at this in design sprint

<mikeCrabb> ...for a given app and given the app authors there is accessibility based on task success

<mikeCrabb> ...how can testers look at accessiblity of different tasks

<mikeCrabb> Dave: theres something in there if we can get to a point where we require that confromacne claims include some evidence to say that it is possible to do these key things.

<mikeCrabb> ...maybe the way that this claim is made includes doing a usability study and we improve the site or we do the study to validate while ideally a usability study is confimrming that a desgn works but it can also show that there are bits that dont work.

<mikeCrabb> Cybele: I was puzzled with task based approach, thought that it was similar to a journey map and identify pain points there.

<mikeCrabb> ... how are these dealt with on the map.

<mikeCrabb> ... putting in a human criteria about expections for when pain points are identified may be worthwhile

<mikeCrabb> Dave: yes, pointing to something that defines where silver should focus and what should be beyond the scope.

<mikeCrabb> ... the reaction to a complaint (in my view) is beyond the process of what silver should deal with, but this is just a hunch. With redefining the scope of silver, maybe it should be

<mikeCrabb> ...anything that defines successful interaction with a resrouces for PwD is worth exploring more

<mikeCrabb> Jeanne: Part of the IA work that mikeCrabb has been looking is flattening existing content so that we have guidelines and methods. Not having criteria.

<mikeCrabb> ...lots of different methods, people can chose the ones that are most appropriate for them. WCAG is very compoent oriented, we want to have part of silver to be product wide, a lot of what we are talking about sits in the product wide guidance. Thes would have methods for accomplishing this

<mikeCrabb> ...methods could have different amounts of points, overall score instead of A/AA/AAA and these points can add up to a total score to give a level.

<mikeCrabb> ...we have a lot of flexibiilty here to incorporate guidance and methods for what will work the best for them

<mikeCrabb> ...even have an open ended one where people can create their own

MikeCrabb: One of the ideas we are thinking about is allowing people to create their own methods.
... There is a tagging system that sits between the guidelines and the methods, so it makes it more flexible for people to discover methods that are appropriate.
... there may be some of the Techniques. We want to get around the problem around of new technology that doesn't have accessibility guidance.
... it future-proofs Silver more

<mikeCrabb> Dave: if a method has evidence that it was user tested, you know that its been validated, you know that it has been doen with relevant PwD for the group that it is intended to support.

+1 to use research with PwD to validate new methods.

<Lauriat> +1

MikeCrabb: We have to be careful that we don't penalize developers for creating innovative methods

<mikeCrabb> Cybele: question of peer review component to validation and would this be possible

<mikeCrabb> ... would engaging in evaluation be part of criteria?

<mikeCrabb> Dave: this part of the conversation, there is an opportunity to balance desire to encourage and focus more on accessible ux

<mikeCrabb> ...not placing undue burden on PwD if you can show that you are using methods that have been validdated. As a small development shop you are picking this form design / date picker because there is evidene that shows that it works

<mikeCrabb> ...its a way to say that you have taken care of this with a limited budget. You can select code based on how successful it is

<mikeCrabb> Cybele: could be problematic where people then cherry pick ones that they find easier to implement that dont require them to think outside the box

<mikeCrabb> Dave: there will be shortcomings but if you are devolving this to components, but where do we address the challenge of an organisation that is trying to achieve silver with facilities to recurit or run budget with PwD

<mikeCrabb> JF: also need to be mindful of scale. World of difference between 5 and 5000 page webiste. Human testing at scale, we need to think about impact on organistaion

<mikeCrabb> Dave: Yes, when we are talking about tasks there are large applications and large tasks. Some are common, some are less frequenrtly done

<mikeCrabb> ... how do we guide poeple to say which tasks and how many?

<mikeCrabb> ...scalability is a major one in a number of different ways

<mikeCrabb> ...should be talked about more

<mikeCrabb> JF: key thing - is there other things out there measuring usability that we can look at?

<mikeCrabb> Dave: there are metrics but are they performance or process metrics?

<mikeCrabb> ...I think these work for an organisation. When its becomes a standard it makes it more complex. Its maybe overreaching for a standard to say that it should be possible for all screen readers to fill in a form within 50 seconds

<mikeCrabb> I'm going to have to run...can someone else take over scribing?

<scribe> scribe: jeanne

JF: COGA had issues with readability score. There are many different ways to test readability (list of readability testing methods)
... a possible solution could be that Silver require that they show evidence of testing, not how to test.

Cyborg: That could be problematic, because it is possible tokenism. We have to give some boundaries.

KimD: I am an attorney and not a legislator. I don't think we should be worrying about that. We have to decide the scope and be transparent, so that a legislative or regulatory body can review our work.

Cyborg: There are processes to look at.

KimD: I don't think we should look at how difficult it is to make it into law. If we are transparent and how to scope it, then we have done our job and we don't have to worry about how a country would go about adopting it.
... If we are clear on our scope and how WE think about it, that is enough as long as we are clear and transparent.

DaveS: I think we could have a part of Silver that is about process and documenting process. It doesn't have to say how to do things in a particular way, but it could have a discrete part that had process requirements. It could be optional for organizations that don't need that guidance, but it could be required for organizations that are required to not discriminate against people with

disabilies.

[discussion of next steps]

jeanne: Anyone who wants to continue this discussion send Jeanne an email and Jeanne will set up a group to work on it.

Cyborg: We need to iterate in a positive way - a macro to micro approach -- broad and big, then work out the details.
... how do we create these approaches.

<Jennifer_Chadwick> Thanks, all. I'm in support of creating guides for testing without being too prescriptive. I would like to include end user testing with persons with disabilities but wary of tokenism that could occur. I like Cybele's suggestion of peer review. Thanks!

Cyborg: I don't want to be too detailed oriented or prescriptive so we end up entrenching a new set of problems.

This is scribe.perl Revision: 1.152  of Date: 2017/02/06 11:04:15  
Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/

Guessing input format: Irssi_ISO8601_Log_Text_Format (score 1.00)

Succeeded: s/Could I get the password for WebEx again please?//
Succeeded: s/Thanks, Jeanne//
Succeeded: s/vali/valid/
Present: Cyborg sloandr Lauriat mikeCrabb JF jeanne kirkwood KimD
Found Scribe: jeanne
Inferring ScribeNick: jeanne

WARNING: No "Topic:" lines found.

Found Date: 11 Sep 2018
People with action items: 

WARNING: No "Topic: ..." lines found!  
Resulting HTML may have an empty (invalid) <ol>...</ol>.

Explanation: "Topic: ..." lines are used to indicate the start of 
new discussion topics or agenda items, such as:
<dbooth> Topic: Review of Amy's report


WARNING: IRC log location not specified!  (You can ignore this 
warning if you do not want the generated minutes to contain 
a link to the original IRC log.)