15:47:19 RRSAgent has joined #ag 15:47:19 logging to http://www.w3.org/2017/02/07-ag-irc 15:47:21 RRSAgent, make logs public 15:47:21 Zakim has joined #ag 15:47:23 Zakim, this will be WAI_WCAG 15:47:23 ok, trackbot 15:47:24 Meeting: Accessibility Guidelines Working Group Teleconference 15:47:24 Date: 07 February 2017 15:47:28 Chair: AWK 15:47:32 Zakim, agenda? 15:47:32 I see nothing on the agenda 15:47:47 agenda+ Joining AGWG – WCAG members need to rejoin. 15:47:54 scribe: Wayne 15:48:04 Agenda+ Testing Survey: https://www.w3.org/2002/09/wbs/35422/testing20170207/results 15:48:19 agenda+ New SC Survey: https://www.w3.org/2002/09/wbs/35422/SC_20170207/results 15:49:50 regrets+ Jim_Smith, Srini, Neil_Milliken, Bruce_Bailey, EA_Draffan 15:50:22 interaccess has joined #ag 15:50:32 trackbot, start meeting 15:50:35 RRSAgent, make logs public 15:50:38 Zakim, this will be WAI_WCAG 15:50:38 Meeting: Accessibility Guidelines Working Group Teleconference 15:50:38 Date: 07 February 2017 15:50:38 ok, trackbot 15:50:47 Chair: AWK 15:51:06 JohnRochford has joined #ag 15:51:06 zakim, agenda? 15:51:06 I see 3 items remaining on the agenda: 15:51:07 1. Joining AGWG – WCAG members need to rejoin. [from AWK] 15:51:07 2. Testing Survey: https://www.w3.org/2002/09/wbs/35422/testing20170207/results [from AWK] 15:51:07 3. New SC Survey: https://www.w3.org/2002/09/wbs/35422/SC_20170207/results [from AWK] 15:51:19 rrsagent, draft minutes 15:51:19 I have made the request to generate http://www.w3.org/2017/02/07-ag-minutes.html AWK 15:51:38 rrsagent, set logs public 15:55:39 Greg has joined #ag 15:58:16 kirkwood_ has joined #AG 15:58:35 alastairc has joined #ag 15:58:56 zakim, take up next item 15:58:56 agendum 1. "Joining AGWG – WCAG members need to rejoin." taken up [from AWK] 15:58:59 +AWK 15:59:14 adam-lund-tr has joined #ag 15:59:41 David_MacDonald has joined #ag 16:00:09 jeanne has joined #ag 16:00:10 present+ mattg 16:00:37 present+ jeanne 16:00:42 present+ Greg_Lowney 16:00:49 Present+ David_MacDonald 16:00:54 JF has joined #ag 16:00:57 present+ JohnRochford 16:01:01 Present+ JF 16:01:12 present+ Adam Lund 16:01:18 present+ Wayne 16:01:27 present+ alastairc 16:01:35 marcjohlic has joined #ag 16:01:36 allanj has joined #ag 16:01:39 Makoto has joined #ag 16:01:44 Lauriat has joined #ag 16:01:48 present+ marcjohlic 16:01:50 Present+ Lauriat 16:02:47 present +shwetank 16:02:54 present+ Makoto 16:02:59 present+ Joshue108 16:03:50 present+ 16:03:54 present+ KimD 16:04:19 Lisa_Seeman has joined #ag 16:04:31 yup 16:04:33 hi 16:04:48 https://www.w3.org/2004/01/pp-impl/35422/join 16:05:12 present+ lisa seeman-kestenbaum 16:05:15 AWK: If you have not asked your AC rep to sign you up then ask your rep. 16:05:20 laura has joined #ag 16:05:28 ... You must rejoin. 16:06:00 present+ Laura 16:06:05 Wilco has joined #ag 16:06:10 https://lists.w3.org/Archives/Public/w3c-wai-gl/2017JanMar/0458.html 16:06:14 MichaelC: I sent an email on the mailing list, there are a few boxes to check. 16:06:39 Kathy_ has joined #ag 16:06:44 present+ Kathy 16:06:56 Glenda has joined #ag 16:07:09 zakim, take up next item 16:07:09 agendum 2. "Testing Survey: https://www.w3.org/2002/09/wbs/35422/testing20170207/results" taken up [from AWK] 16:07:16 gowerm has joined #ag 16:07:24 present+ MikeGower 16:08:06 that is a good way to chair - two screens 16:08:12 will do that 16:08:14 present+ kirkwood 16:08:32 Mike_Pluke has joined #ag 16:08:54 Present+ Mike_Pluke 16:09:01 erich has joined #ag 16:09:05 q? 16:10:30 present +Glenda 16:10:33 AWK: Success criteria acceptance criteria. One big topic what types of testing? That is one of the keys. What does it mean to be testable? Automation is not required. There is human judgement. Confirm a filure automatically for language ID, you need a person to ID it as French. Is Usability testing sufficuent. 16:10:54 present+ 16:10:56 ... Manual testing must incude user testing, and must not. 16:11:06 -1 16:11:37 q? 16:11:42 JohnR: I have a hard time with not including users. It is weird. 16:11:42 q+ 16:11:45 Q+ 16:12:46 Q+ 16:13:10 Josh: I have experience with usability and users with disabilities. I have to push back on use testing on as a requirement. Avaialblity is the problem. No I cannot require user testing. 16:13:13 ack jos 16:13:52 Josh: There is a big overhead and extreme cost. 16:14:46 q+ 16:14:52 JohnR: There are cloud based testing. When do we require user testing. If you want your product you must test with users. 16:15:09 https://www.w3.org/TR/UNDERSTANDING-WCAG20/conformance.html 16:15:16 Josh: There may be new ways to reduce the cognitive load. 16:15:30 ack JF 16:15:39 JohnR: There are cloud based testing sites. 16:15:52 WCAG does this as a SHOULD, not a MUST "The content should be tested by those who understand how people with different types of disabilities use the Web." 16:16:00 I also support recommendation, but not a 'must' requirement 16:16:34 +1 user testing is very important and yet the data is very subjective 16:16:38 Text says "Although content may satisfy all Success Criteria, the content may not always be usable by people with a wide variety of disabilities. Therefore, usability testing is recommended, in addition to the required functional testing. Usability testing aims to determine how well people can use the content for its intended purpose. It is recommended that users with disabilities be included in test groups when performing usability testing." 16:16:43 s/Avaialblity/Availability 16:16:52 q+ 16:16:56 JohnF: There will be inconsistency. UT to ensure accessible. The ability to have consistent testing. 16:16:58 ack lisa 16:17:00 +1 to Josh's point on burden, and JF's point on mixed results point. WCAG can't mandate people's process. 16:17:03 q+ 16:17:06 + 1 to John and thats why a good test facilitator is needed to read between the lines. 16:17:07 +1 to concerns of inconsistent results 16:17:33 q+ to say we want good input from users in setting the guidelines, but guidelines are too abstract and generic to mandate user testing as part of conformance (though I stick with recommending it) 16:18:05 Mike_Elledge_ has joined #ag 16:18:45 Lisa: There are things that are not subjective like can they finish this task. They managed to access help. These are reliable. Add and A+ not just A level. 16:18:57 I like the idea of a WCAG 'plus' for public / larger companies, but that's Silver thing in my mind. 16:19:27 q+ 16:19:37 lisa: If you ask can you get a product. We can get quantitiative. 16:19:40 ack kathy 16:21:07 q+ to say we have to support inclusive design methodologies and good usability 16:21:38 q+ 16:21:40 I agree with everything Kathy is saying. 16:21:42 Kathy: We do not require accessibility support method. So user testing may not control for AT. We need to include more into how to make things to work for disability. User's knowledte of the AT, or Browser bugs. 16:21:44 q- later 16:22:19 ... Benchmark that user cannot do this, is not a test because of other factors. 16:22:32 Kathy has convinced me. I'm chaning my vote. 16:22:43 q+ 16:22:44 q? 16:22:52 ack gowerm 16:23:40 MikeG: We are talking about #2 be testable through manual or automated processes? Are we talking about that. 16:24:42 ack wayne 16:24:48 q+ 16:24:49 ... Nobody has defined user testing. Someone is doing testing. It is should be incorporated if you say you must 16:25:24 Wayne: wondering if there are things at this time that we can't really measure with existing tests 16:25:26 WD: I'm wondering if there are things that we can't measure by the tests we originally envisioned. 16:25:50 WD: For some esp cognitive issues - this could be the only way to capture them 16:25:53 q+ to say what if we changed the “must” to “may”? 16:26:18 ack gl 16:26:23 WD: There are things where we dont always have agreement, and testing isn't always perfect anyway. 16:26:37 ACAA - Requires carriers to test the usability of their accessible primary Web sites in consultation with individuals or organizations representing visual, auditory, tactile, and cognitive disabilities. 16:27:09 correct 16:27:28 jon_avila has joined #ag 16:27:34 present+jon_avila 16:27:36 q+ 16:27:43 the problem is cost glenda 16:27:47 Glenda: (see Glenda's inclusion) What if we require user testing, and later add what to do with it. 16:27:47 my q+ is "Why are we saying manual processes for testing *must not* include user testing?" 16:27:51 i want my ho,me page to conform 16:27:51 ack josh 16:27:51 Joshue, you wanted to say we have to support inclusive design methodologies and good usability 16:27:52 ack me 16:27:55 +1 16:28:46 are there enough blind people for all the user testing of every website? 16:29:01 Jossh: That exciting to do this, It takes it off the dry domain. I wonder you could strongly suggest that user testing could be don regardless of the outcome. 16:29:02 Hands down, my fav part of ACAA usability testing is with people with cognitive disabilities. It has been a gold mine of data for improving the user experience for all. 16:29:05 (there are enough coga users) 16:30:21 ... User is a good way to get people in the room. If there is a way to get this in the room. But not require. We are trying to use techniques encourage user testing. There is a halo effect. 16:30:28 q? 16:31:34 ack david 16:31:39 q- 16:31:39 AWK: If we consider a user test that says if it passes the chriterion passes. 16:32:42 q- 16:33:34 q+ AWK to point out that policies that adopt WCAG can require user testing 16:33:50 ack michaelc 16:33:50 MichaelC, you wanted to say we want good input from users in setting the guidelines, but guidelines are too abstract and generic to mandate user testing as part of conformance 16:33:53 ... (though I stick with recommending it) 16:33:56 DavidM: User testing is included as a recommended. We cannot change from a should to must in 2.1. I think it is required it is not. It is strongly recommended. 16:34:38 jnurthen has joined #ag 16:34:59 q? 16:35:04 MichaelC: A major purpose of guidelines is to give a start, but uses testing is strongly recommended. I don't see it can be part of a conformance model. 16:35:18 ack mike_pluke 16:35:27 +1 to Michael's comments 16:35:30 ... User testing is impractical for guidelines. 16:35:53 q+ 16:36:02 q- 16:36:09 q+ to say that an SC can't require usability guidelines, or in fact testing across UAs. The trick is translation from user-requirement to content requirement. 16:36:35 q+ to say there are complex ethics issues around user testing with people with cognitive issues 16:36:54 ack mike 16:36:58 MichealP: We are not talking about accessibility. ... The people who might benefit might be for a well organized testing lab. I don't see it passing. 16:37:45 rrsagent, make minutes 16:37:45 I have made the request to generate http://www.w3.org/2017/02/07-ag-minutes.html jnurthen 16:38:22 MikeE: Practicality. There are still many small companies who do not testing. It would cause people to back away. It would be an impediment. Are we trying to make 2.0 compatible with 2.1. 16:38:38 ack AWK 16:38:38 AWK, you wanted to point out that policies that adopt WCAG can require user testing 16:39:26 ack al 16:39:26 alastairc, you wanted to say that an SC can't require usability guidelines, or in fact testing across UAs. The trick is translation from user-requirement to content requirement. 16:39:28 Q+ 16:39:32 AWK: Is certainly can be a policy requirement, like the air carriers act. 16:39:52 CVAA also includes some requirements for consulting with people that have disabilities. Revised ICT Standards and Guidelines from Access Board dropped the requirement 16:39:53 AWK: Any policy uptake process might add user testing as a requirement 16:40:14 ack josj 16:40:17 ack josh 16:40:17 Joshue, you wanted to say there are complex ethics issues around user testing with people with cognitive issues 16:40:17 ack me 16:40:24 Alastair: There are difficulties with testing usability and testing across user agents. 16:41:07 we have a issue paper coming that gives direction how to deal with this josh 16:41:22 ack JF 16:41:29 Alastair: The trick is translation from user-requirement to content requirement, it is the content that has to pass, not the interpretation of that content. 16:41:31 it is a must for good coga inclusion 16:41:46 Josh: There are problems with cognitive disabilities have special requirements. The impact on the individual may be harmful. 16:43:04 JohnF: Testing is repeatable. Not opposed to UT. 16:44:03 have to go...bye all 16:44:05 NB: The survey results are now 11:0 16:44:27 AWK: User test is strongly recommended in addition to conforming to WCAG material. We cannot have UT as a requirement. Reparable, Cost and you can do it well. 16:45:07 +1 to AWK 16:45:14 +1 16:45:56 RESOLUTION: User testing is not part of a manual testing for WCAG test criteria. 16:46:00 +1 16:46:05 +1 16:46:11 +1 16:46:22 +1 16:46:25 +1 16:46:35 I'd suggest: Usability testing is not required for manual testing of WCAG test criteria. 16:46:38 "User testing is not a *required* part of manual testing for WCAG test criteria." 16:46:45 yet 16:46:50 +1 16:46:52 in the 2.x series. 16:46:56 +1 16:46:57 +1 16:47:09 user acceptance testing? 16:47:25 s/User testing is not part of a manual testing for WCAG test criteria./User acceptance testing is not a required part of a manual testing process for WCAG test criteria. 16:47:30 +1 16:47:41 wow - UAT is different from usability/user testing 16:48:13 s/WCAG test criteria/WCAG Success criteria 16:49:04 usability is different from user testing IMO 16:49:42 s/User acceptance testing/User testing 16:49:49 +1 16:50:12 +1 16:50:20 +1 16:50:24 q? 16:52:07 Jeanne: I have no objection for internal use. 16:53:32 q+ 16:54:23 ack ala 16:54:25 LisaS: In the wording experts may not be an issue with defining expert. 8 out of 10 with a high level of content. Not 100% sure. 16:55:38 Alastair: The bar is that accessibility experts can do the testing and they must have a high level of confidence. 16:55:53 steverep has joined #ag 16:56:01 present+steverep 16:56:14 present+ JamesNurthen 16:56:38 Alastair: we should expect current experts (e.g. this group) to be able to read the SC & description, and then have a high degree of agreement for any particular page. 16:56:56 AWK: This aligns with 2.0 it is subjective. 16:57:17 it might be more then just reading 16:57:21 q? 16:57:29 one discription. you might need to follow some links etc 16:58:29 I have nothing 16:58:30 WG agrees that SC should have a high degree of confidence of inter-rater reliability, approximated by the expectation that 8 of 10 experts in a area would agree. 16:59:04 +1 16:59:20 yes but this is the expectation 16:59:24 +1 (I like 8 out of 10) 16:59:30 me/ 8 out of 10 doesn't work in practice: http://dl.acm.org/citation.cfm?id=2141946 16:59:31 8 out of 10 is a standard measure of reliability 16:59:48 +1 17:00:17 +1 17:01:13 +1 17:01:18 +1 17:01:22 +1 17:01:23 +1 17:01:30 ÍDavid: I can live with it 8 out of 10 17:01:35 can live with it 17:01:41 can live with it 17:01:49 Not happy, but will not oppose 17:01:56 q+ 17:01:57 RESOLUTION: WG agrees that SC should have a high degree of confidence of inter-rater reliability, approximated by the expectation that 8 of 10 experts in a area would agree. 17:02:24 +1 8 out of 10 17:03:06 Zakim, take up item 3 17:03:06 agendum 3. "New SC Survey: https://www.w3.org/2002/09/wbs/35422/SC_20170207/results" taken up [from AWK] 17:04:06 WD: I worry about setting ourselves up for not agreeing. 17:04:13 WD: On basic stuff. 17:04:50 WD: I like the wording, using confidence etc hitting 80 % 17:04:54 WD: We don't always 17:05:02 AWK: You happy with the language? 17:05:14 WD: On average 17:06:10 TOPIC: SC Proposal: Graphics Contrast (Issue 9) 17:06:36 subtopic: Graphics Contrast 17:07:16 q+ 17:08:47 Q? 17:08:52 ack w 17:08:52 Q+ 17:08:54 ack way 17:08:57 ack john 17:09:04 AWK: I what will happen with these SCs on the list we will do a survey we can put it into the editors draft. We don't need a call for consensus for editors draft. We do need a CfC for publish. If we decides it is good then we will but it on the 17:09:43 JohnR: How do I revise my criterion so that we can discuss it. 17:10:08 JohnR: I put the text in myself and it is a link to it. 17:10:25 AWK: Let us talk about this off line. 17:10:42 ack lisa 17:10:47 q- 17:11:02 q+ 17:11:49 LisaS: I starting if 8/10 it goes in the editors draft. What about those that do not have enough support technology, what do we do. 17:11:51 q- 17:14:07 AWK: There are easier the WG on call says this sounds reasonable it can be done with existing tools. We will get good feedback when we put it out. We cannot agree on this wo we don't want it to be in the editors working draft. There are those that are in between. We put it in the Ed draft and put in a note and we need feedback. The group needs to agree on if it cannot be done. 17:14:19 q+ 17:14:55 Q+ 17:15:01 ... The goal is to have a chritirion that is achievable. 17:15:02 ack w 17:15:20 Q+ 17:16:14 LisaS: There are issues. 17:17:00 AWK: if it does not meet the requirements. Things can be deleted. 17:17:25 ack jf 17:17:29 ack lisa 17:20:34 JohnF: As we advance our FPWD we need to have SCs then add a Note with user needs. 17:21:45 yup 17:21:54 to wayne 17:22:04 John's idea is to publish a note that lists out the user requirements that we are not able to directly meet with WCAG 2.1, as a tool to help guide future development by authors/tool makers/etc 17:22:19 -1 to john 17:23:14 we will discuss John's idea later, but it seems like a great idea since we will not be able to address everything that is a requirement today and we can update it with new requirements as they are identified. 17:23:38 q? 17:25:09 LisaS: (I missed most of this) If it is not in the draft it won't happen. .. I don't see it happening. 17:25:53 AWK: There is an onus that there need to be on the proposer to support them. 17:26:53 q? 17:27:14 ... There is an expectation that there is a high quality draft. We are proposing a refinement. We are reluctant to put out something that the commenters say this is more wrong than right 17:27:53 +1 17:27:54 ... There might want to mark for comment 17:27:59 +1 17:28:02 +1 onus is one task forces to improve the lanugage based on feedback 17:28:04 q+ 17:28:07 +1 17:28:12 s/one/on 17:31:26 Q+ to say that 17:31:39 ack JF 17:31:39 JF, you wanted to say that 17:31:46 The main issues seem to be translating from user-requirement to content requirement. It is impossible to test as a user-requirement, it needs to be something that is identifiable about the the page. 17:33:04 q? 17:33:06 ack way 17:34:01 present+ erich 17:34:08 +Rachael 17:34:12 Glenda has left #ag 17:34:15 bye all 17:34:20 trackbot, end meeting 17:34:20 Zakim, list attendees 17:34:20 As of this point the attendees have been AWK, mattg, jeanne, Greg_Lowney, David_MacDonald, JohnRochford, JF, Adam, Lund, Wayne, alastairc, marcjohlic, Lauriat, Makoto, Joshue108, 17:34:23 ... MichaelC, KimD, lisa, seeman-kestenbaum, Laura, Kathy, MikeGower, kirkwood, Mike_Pluke, allanj, jon_avila, steverep, JamesNurthen, erich, Rachael 17:34:27 laura has left #ag 17:34:28 RRSAgent, please draft minutes 17:34:28 I have made the request to generate http://www.w3.org/2017/02/07-ag-minutes.html trackbot 17:34:29 RRSAgent, bye 17:34:29 I see no action items