08:02:04 RRSAgent has joined #wai-act 08:02:04 logging to http://www.w3.org/2016/09/22-wai-act-irc 08:02:26 meeting: WCAG ACT TF Face-to-Face 08:02:31 chair: Wilco 08:02:36 scribe: shadi 08:03:05 Topic: Introductions 08:03:13 present+ Kathy 08:03:29 present+ JohnJansen 08:03:32 Wilco: from Deque, co-facilitator of this TF 08:03:56 present+ Romain_Deltour 08:04:34 Kathy: from Interactive Accessibility, co-facilitator of Mobile A11Y TF 08:04:45 Alice: from Google, working on UI 08:05:03 Shadi: W3C staff contact for this TF 08:05:07 aboxhall has joined #wai-act 08:05:10 agenda: https://www.w3.org/WAI/GL/task-forces/conformance-testing/wiki/Face-to-Face_Meetings#Thursday_22_September 08:05:23 Song: from NIA, Korea 08:06:05 John: from Microsoft 08:06:17 Romain: from Daisy 08:06:38 Michael: W3C staff contact for parent WCAG WG 08:06:48 ...focus on interfacing with WCAG WG 08:07:03 present+ shadi 08:07:11 present+ MichaelC 08:07:17 present+ aboxhall 08:07:30 ACT Overview: https://www.w3.org/WAI/GL/task-forces/conformance-testing/wiki/ACT_Overview_-_What_is_ACT 08:07:48 Wilco: many tools out there, each slightly different, conflicting results 08:07:58 ...that's a problem 08:08:30 ...want to figure out, how to get to a core set of rules that we can all agree on 08:08:44 Bong has joined #WAI-ACT 08:09:48 Michael: often get questions, can't provide support requests 08:10:11 ...sometimes authoritative review needed 08:10:30 JohnJansen has joined #wai-act 08:10:33 Kathy: sometimes also questions on coverage of WCAG 08:10:55 Wilco: impression that previously such discussions were avoided 08:11:30 Michael: when we get a request for interpretation, often it is one that the group agrees on 08:11:37 ...when not, we look into why 08:12:01 ...sometimes unclear with different history of how requirements were created 08:12:14 ...often documentation not available anymore 08:13:56 Wilco: would try to interpret to take the load off the WG 08:14:18 Shadi: so we bring results of our discussion to the WG? 08:14:41 Michael: only in areas of dispute, if existing supporting resources are not sufficient 08:15:19 Wilco: difference between success and failure? 08:15:46 Michael: failure is direct result - if there is a fail, requirement is not met 08:16:13 ...to prove success, not meeting a technique is not exhaustive 08:16:21 ...could meet it in a different way 08:16:42 Wilco: we assume accessible unless a failure is found 08:16:58 Michael: might not be a common approach 08:17:50 Wilco: accessibility support is also an open issue 08:18:42 ...in aXe we check if implementation is viable 08:18:50 ...and if there are alternatives 08:19:01 Kathy: how do you keep that up-to-date? 08:19:38 Wilco: this is exactly the problem description 08:21:48 [Wilco walks through "Goals" from ACT Overview page] 08:22:40 Wilco: rules will not be exhaustive 08:24:08 Michael: but developed tests should be authoritative 08:24:18 s/Michael/Shadi 08:24:45 Kathy: warning about calling it conformance testing 08:25:05 ...can test requirements partially but often not fully 08:25:29 ...difficult to prove conformance 08:25:42 q+ joanie 08:25:46 q- 08:26:18 ...already confusion out there, need to be careful about wording 08:28:19 Romain: need to be careful with the word authoritative as well 08:28:33 ...not exclusive, tool vendors can develop any rules too 08:28:45 Shadi: agree, was looking for the issue of validation 08:29:13 ...somehow recognizing the rules that have been validated for interpretation of WCAG 08:31:00 Kathy: conformance has very specific meaning in WCAG 08:31:44 Shadi: just wanted to make sure there is the tie-in to WCAG as a specification 08:31:58 ...rather than accessibility testing in the wild 08:33:14 John: layer of conformance - browser, platform, content? 08:33:29 Wilco: it depends 08:35:58 Kathy: even absolut failures are fairly rare 08:36:08 Wilco: not sure I agree with that 08:36:20 ...there are always exceptions 08:36:31 ...like conforming alternative aspect 08:36:50 ...but that is caught in semi-automated 08:37:56 Kathy: ok, as long as semi-automated 08:38:45 Shadi: there is this perception that it is all about automated only 08:39:09 ...but in fact, even most auto-WCAG tests are actually semi-automated 08:40:08 Kathy: confusion out there about what is conformance really 08:40:50 ...most focus is on the success criteria, rather than the actual conformance requirements 08:41:12 q+ 08:41:23 ...we propagate that by calling it conformance testing or automated testing 08:41:42 ...maybe one goal of this group is to help define what is needed for conformance testing 08:41:46 q+ to note conformance testing only needed for conformance claims, which aren´t required 08:41:54 q- joanie 08:42:03 ack me 08:42:03 MichaelC, you wanted to note conformance testing only needed for conformance claims, which aren´t required 08:42:13 ack joanie 08:42:28 Wilco: one of the discussions with WCAG WG, also in light of Silver 08:42:58 Michael: one of the requirements for WCAG was to have conformance claims 08:43:28 Kathy: conformance claims becoming very important in the US, for example 08:43:35 ...seeing more and more the need 08:43:44 boazsender has joined #wai-act 08:44:47 Wilco: conformance testing important in the Netherlands 08:46:30 q? 08:47:34 Shadi: take away is that we have an issue with the naming 08:47:49 ...suggest to take that back for later discussion in the group 08:48:00 Wilco: has previous discussions on that 08:49:11 s/has/had 08:49:38 Kathy: agree, difficult problem but needs to be discussed 08:50:42 [Goal #4] 08:51:07 shadi has changed the topic to: https://www.w3.org/WAI/GL/task-forces/conformance-testing/wiki/ACT_Overview_-_What_is_ACT 08:51:58 Wilco: not limited to HTML 08:52:20 q+ 08:52:24 q- 08:55:14 ...also CSS, etc 08:55:19 ...even UAAG 08:56:34 Shadi: is it a scope creep to include browser testing beyond content testing? 08:56:48 Kathy: think it would be important to keep in mind 08:57:24 Wilco: primary focus HTML, CSS, etc 08:58:41 Boaz: is the framework exclusive to accessibility testing at all? 08:59:12 ...possibly shareable beyond accessibility alone 08:59:39 MichaelC_ has joined #wai-act 09:00:06 Kathy: agree with that comment 09:00:16 jnurthen has joined #wai-act 09:03:32 Wilco: would not want to put a hard boundary to UAAG 09:03:39 jamesn has joined #wai-act 09:04:01 ...everything converging 09:04:24 Kathy: agreed, also in light of Silver 09:04:40 Shadi: convinced, just worried about scope creep 09:05:30 JohnJansen has joined #wai-act 09:06:56 Romain: digital publishing technologies not separate from web technologies 09:07:07 ...ePub uses web technologies 09:07:38 ...our selectors are pretty much focused on web technologies 09:09:15 [Target Audience] 09:13:15 Kathy: how to treat false positives? 09:13:47 ...only rules that don't create false positives? 09:13:57 Romain: think so 09:14:03 Wilco: would say TBD 09:14:22 Romain: could be goal to reduce false positives 09:19:59 Shadi: do we want to differentiate false positives and false negatives, or just talk about accuracy? 09:20:20 Romain: false negatives can often block from publication for no reason 09:20:40 Kathy: some tools will suggest additional code to fix a false positive 09:20:51 s/false negatives/false positives 09:23:21 Wilco: agree need to look at accuracy 09:23:46 Kathy: can't eliminate false negatives in semi-automated 09:24:04 Shadi: depends on atomicity of tests 09:27:31 Romain: initial rules exclusive to HTML, CSS, and ARIA? 09:27:46 ...what does that specifically mean? 10:02:58 rdeltour has joined #wai-act 10:06:35 yatil has joined #wai-act 10:06:43 scribenick: rdeltour 10:06:45 Kathy has joined #wai-act 10:07:13 Wilco: we're on the Initial Rules section of the Overview https://www.w3.org/WAI/GL/task-forces/conformance-testing/wiki/ACT_Overview_-_What_is_ACT 10:07:39 ... Romain said it sounds like the Rules development is in scope of ACT 10:07:47 ... let's change that 10:07:54 Sharron has joined #wai-act 10:08:02 present +sharron 10:08:29 Romain: what's the ACT role? Overseeing or coordination? 10:08:37 Wilco: we should coordinate with auto-wcag 10:09:15 ... **changes the section accordingly** 10:09:46 scribe: rdeltour 10:10:46 Kathy: we wanted to talk about whether we need to further define "HTML, CSS and ARIA" 10:10:58 Wilco: I think we don't need to be more specific 10:11:17 ... we don't want to promise more that what we're doing to do 10:12:14 Romain: do we want to mention the EPUB rules 10:12:44 MichaelC_ has joined #wai-act 10:13:08 ... we will develop a rule set specific to EPUB, ultimately based on the ACT fwk 10:13:31 ... I'm not pushing for mentioning it here, but it might help with the 2-implementation req for the CR 10:13:57 shadi: we don't want to boil the ocen right away 10:14:20 ... but I'm wondering if we should rephrase the second part "These rules will focus on..." to be more inclusive 10:15:12 Romain: what about saying sth along the lines of "and coordinate with any other community interested in developing rules for the ACT fwk"? 10:15:28 jnurthen has joined #wai-act 10:15:41 Wilco: I want to be explicit about Auto-WCAG 10:15:44 Romain: +1 10:17:11 Shadi: ok for Romain's proposal, with s/any other communry/other groups/ 10:17:30 +1 10:18:06 Wilco: "Use Existing Rulesets" 10:18:32 ... we want to coordinate with orgs that have rulesets, and help them transition to ACT 10:19:03 kathy: we should also look at orgs checklists, not only automated tools 10:19:30 q+ to ask about "will work with" 10:19:55 Wilco: **opens the editor to improve the language** 10:20:22 ... add "or test procedures" 10:20:26 ack shadi 10:20:28 shadi, you wanted to ask about "will work with" 10:21:14 shadi: "will work with" means we'll support organizations? 10:21:35 kathy: what about we change that to say we're going to use their rulesets 10:21:48 shadi: there's the aspect that we're going to look at their stuf 10:21:53 s/stuf/stuff 10:22:12 ... then there's the question of how to assist people who want to migrate 10:22:26 ... provide some kind of support or guidance in getting their rules in the right format 10:22:35 kathy: then let's say that explicitly 10:24:06 ... "The ACT TF will collect the rulesets from organizations interested in contributing to the report, for the purpose of forming and evaluating the framework" 10:25:41 agarrison has joined #wai-act 10:26:00 Romain: do we need the agreement from these organizations? we can just pick the rules 10:26:30 kathy: right, but some rules are not open, so those organizations need to explicitly contribute them 10:27:24 Wilco: we also want to say that the orgs can contribute their rules to Auto-WCAG 10:28:45 kathy: the second thing is to talk about the "specific acceptance requirements" 10:29:15 ... do we need to mention the orgs that do not want to contribute? 10:29:31 Wilco: it was there to address people asking "why should I contribute?" 10:29:44 kathy: ok, we might need to rephrase that 10:30:25 shadi: "ACT TF will provided guidance to help organization migrate their test rules and procedures" 10:30:34 Wilco: we're supporting auto-wcag 10:31:29 shadi: auto-wcag is not a body that can help organizations, it's the other way around: organizations who want to contribute can join auto-wcag 10:32:25 wilco: **rewords the paragraph accordingly** 10:34:02 eric: we should use the present tense 10:35:58 sharron: you need to be persuasive 10:36:11 eric: we can list the benefits of contributing 10:36:56 shadi: I'm wondering if the promotion part should be in another document 10:38:19 shadi: we use the words "ruleset" "test rules" "test procedures" 10:38:44 ... are we clear about what these are? 10:39:04 sharron: do you say we need a glossary? 10:39:13 shadi: we might be able to use one term 10:41:00 eric: for me, a ruleset is scoped to the thing to test, like "ruleset for WCAG AA" 10:41:23 wilco: in our terminology, all of these are rules, whether automated or not 10:41:57 shadi: for me, rules almost means automatated, test procedure is more vague 10:42:21 wilco: a test procedure is part of rule 10:42:38 kathy: for our existing ruleset it doesn't really matter 10:42:54 ... we're giving examples, which kind of defines what we mean 10:43:32 romain: it's an overview document, we may not need to be super-specific 10:44:25 [[Rules is the overall term, Test Procedures is part of a Rule]] 10:44:47 eric: as someone from the outside, it might be good to define the terms 10:45:15 shadi: we should revisit the wiki pages and make sure that we're consistent 10:46:14 wilco: do we want to list the deliverables in this document? 10:46:27 shadi: yes, it fits well 10:47:25 eric: I like being able to learn this info, but it can be hard to keep in sync with the formal deliverables 10:47:35 ... maybe we can just link to the deliverables doc? 10:47:44 shadi: the deliverables are quite stable 10:48:01 sharron: why are there 2 groups? 10:48:09 shadi: to maximise the likelihood of contributions 10:48:52 romain: you don't need to be a w3c member to join a CG 10:49:31 shadi: it's also a source of confusion, people do not always know which group they should join 10:50:00 ... people are sometimes confused by the whole w3c structure: TF, WG, CG, etc 10:51:18 romain: the description of "Benchmark method" sounds confusing 10:51:36 ... it's a description on how to test for accuracy 10:51:50 shadi: there's accuracy and validity, two different things 10:52:07 wilco: validation should be part of the ACT fwk 10:52:23 ... when you write a rule you need to provide enough background 10:53:12 ... how do we validate the rules that can go in the ACT rule suit is still an open question 10:53:26 shadi: I always thought that would be part of the benchmark method 10:53:52 ... what else can go in the benchmark method? 10:54:12 ... do we provided a litmus test for a specfic rule 10:54:16 wilco: yes 10:54:31 shadi: so it's only the accuracy check, not the validity check 10:54:48 ... the validation needs to happen on a rule basis (or classes of rules) 10:55:20 [[validity vs accuracy checking]] 10:55:31 romain: in my understanding the validity check is done manually when writing the rule. 10:55:44 ... the benchmark tool tests only the accuracy 10:56:08 wilco: what if the validity changes later 10:56:14 shadi: yes, the AT can change, etc 10:56:27 wilco: rules will need to be updated as technology change 10:56:35 ... it doesn't mean the rule was invalid 10:56:45 ... but new technologies will need different rules 10:56:57 ... a rule is valid given its scope 10:57:18 ... this doesn't change over time 10:57:30 ... whereas the accuracy may change depending on how AT or UA evolves 10:57:37 s/evolves/evolve/ 11:02:04 jamesn has joined #wai-act 11:03:25 shadi: we get the question of validation very often 11:03:36 ... how does it move to a contributed test to become approved 11:03:43 ... we should describe it somewhere 11:03:53 ... if one of the thing we need to think of 11:04:38 romain: we're talking about incubation 11:04:55 ... do we need to formalize the process? 11:05:21 shadi: at least some description to give the rough idea 11:05:37 ... we envision there will be a process for approving test rules 11:05:46 ... it is kind of missing in the overal description 11:07:08 romain: if the benchmark is to test the accuracy, does it mean we lack a deliverable to describe the validity process? 11:07:25 shadi: not necessarily a deliverable, it can be another section in this list 11:08:34 kathy: WCAG has to review the work for the TF 11:08:48 ... the TF has to define what the rules are, then the WCAG reviews the TF work 11:09:23 wilco: the ACT TF can get consensus on rules 11:09:47 kathy: there is a document on the WCAG wiki that's institution knowledge 11:10:04 ... do we have WCAG people in the TF? 11:10:18 shadi: we didn't want to pull for the WG, but bring new people 11:10:36 kathy: having somebody in the TF will help with the amount of back and forth 11:11:40 wilco: I think it will be straightforward enough 11:11:55 shadi: we need to have bridges 11:12:10 kathy: I may be willing to help out 11:12:36 ... it will make things faster within the TF, at least to identify the things that need to go back to the WG 11:18:29 wilco: **edits the deliverables overview** 11:21:43 romain: "results sufficiently accurate for (non) conformance testing to WCAG" sounds like a bold statement 11:22:11 shadi: reword into "collection of rules that have passed the validation and benchmarking requirements" 11:23:41 kathy: in the first bullet, I would remove "conformance". we're talking about test rules for accessibility testing. 11:24:42 jnurthen has joined #wai-act 11:57:29 MichaelC has joined #wai-act 12:13:33 rdeltour has joined #wai-act 12:17:03 shadi has joined #wai-act 12:22:19 Wilco has joined #wai-act 12:25:02 jamesn has joined #wai-act 12:34:49 Kathy has joined #wai-act 12:43:07 Zakim has left #wai-act 12:45:31 Zakim has joined #wai-act 12:46:01 jcraig has joined #wai-act 12:46:06 https://webkit.org/blog/3302/aria-and-accessibility-inspector/ 12:47:24 agarrison has joined #wai-act 12:50:06 aboxhall: demo similar to jsbin layout 12:50:26 JohnJansen has joined #wai-act 12:50:28 html and js visible, rending and run button with console output on the other side 12:50:44 s/html/aboxhall: html/ 12:51:20 aboxhall: describes code sample that will changes the label property of an accessibility backing node for a DOM elements 12:51:28 s/elements/element/ 12:52:11 aboxhall: runs voiceover with standard labels.... clicks run... console prints done. voiceover reads new label on the button. 12:53:46 aboxhall: describes similar layout of a slider example with an "onaccessiblesetvalue" event handling silder increment/decrement events 12:57:19 aboxhall: js fetches the node, adds event listener, receives the event (when VO increments the slider) and changes the aria-value/min/max, etc 13:00:03 MichaelC_ has joined #wai-act 13:21:09 One more time with feeling: https://discourse.wicg.io/t/contributing-the-accessibility-object-model-specification/1702 13:22:39 https://github.com/a11y-api/a11y-api/issues 13:23:25 aboxhall: inspecting the lang drop down 13:23:52 scribe: jcraig 13:24:02 aboxhall: ... 13:24:26 partial ax tree in chrome ax extension 13:25:10 showing both unknown role and aria-equivalent role in the inspector 13:25:32 chrome://flags/#enable-experimental-web-platform-features 13:25:42 set the experimental flag in chrome settings for this tool 13:25:58 then you can request computed role and name (label) 13:26:42 aboxhall@chromium.org 13:29:07 scribe: Kathy 13:29:49 wilco: feedback on the requirements was pretty good 13:30:37 ... Charu provided feedback 13:30:53 https://lists.w3.org/Archives/Public/public-wcag-act/2016Sep/0003.html 13:31:27 ... if rules are pseudo code then not a rule - we have gone beyond this discussion and will not be write pseudo code 13:32:12 ... discussion of the comments 13:33:51 ... defining standard common standard - we will make that change 13:35:05 ... maps to an underlying requirement - when you write a rule you should be aware of validation 13:36:17 Romain - sounds like requirement for the rules not the framework 13:36:58 Wilco: yes, it is the ACT framework 13:37:32 Romain: the TOC should be updated to reflect this 13:37:44 Shadi: agree with Romain 13:37:58 Romain: i proposed changes in my email 13:38:25 Wilco: we will get to that when we review the feedback 13:38:58 Romain: we may not want to spend too much time on this until we look at the proposed changes 13:39:22 Wilco: skip over some of the comments from Charu 13:39:53 Move to Romain's comments 13:40:01 Romain: many editorial changes 13:40:24 ... see the email for details 13:40:33 https://lists.w3.org/Archives/Public/public-wcag-act/2016Sep/0004.html 13:41:52 Wilco: want to be careful with scope 13:43:04 James: how about discovering failures 13:44:48 Romain: next comment 13:45:09 rule will not test the policy itself 13:46:58 Shadi: I don't think we should promote other standards and policies. 13:47:13 James: we would want checks for example for ARIA 13:47:31 Wilco: good for organizations to go beyond WCAG 13:47:43 James: we insist that all tables have captions 13:48:48 Romain: instead of standards, how about guidelines 13:49:08 Shadi: test rules for WCAG 2 and other standards 13:49:26 ... says the same thing in the paragraph 13:50:27 Wilco: I will do some wordsmithing on this 13:50:34 Romain: next comment 13:51:21 Shadi: agrees with the direction 13:52:02 Romain: might be a bold statement. How about "The ACT TF aims to increase the adoption of sane accessibility testing principles." 13:52:46 Wilco: how about promote 13:53:23 ... we want to make it easier for developers to come to a shared approach 13:53:59 James: change should to can 13:54:51 "similar to how the HTML standard is a standard on how to write HTML documents" 13:54:51 Romain: I see what you mean, but I would avoid the comparison given the very different nature of the two specs 13:55:17 ... I would remove it 13:55:49 Wilco removes the part of the paragraph 13:56:45 Romain: /ACT Rules/The rules written in conformance to the ACT Framework specification (later referred to as ACT Rules)/ 13:57:21 Wilco makes the change 13:59:17 Romain: s/, that can be used in ATTs or for QA testing of accessibility./. They can be implemented by ATT, or used as a reference when performing manual accessibility QA / 14:00:38 Wilco: earlier we said this was not for QA 14:01:28 ... removed the word QA 14:01:35 JohnJansen has joined #wai-act 14:01:44 BREAK 14:02:00 >> Back in 30 minutes 14:12:03 jcraig has joined #wai-act 14:23:18 rdeltour has joined #wai-act 14:31:37 jcraig has joined #wai-act 14:51:06 jamesn has joined #wai-act 14:57:07 jcraig has joined #wai-act 14:57:31 jcraig has left #wai-act 15:03:21 Webex has stopped! 15:03:42 Wilco has joined #wai-act 15:04:29 Thanks. 15:05:19 action: Wilco adds techniques proposal to the agenda for next meeting 15:08:09 scribenick: rdeltour 15:09:07 shadi: I think someone should work on a proposal, then discuss it in ACT before involving WCAG 15:09:22 wilco: maybe setting up a couple paragraph to summarize our thoughts, I'll do that 15:09:41 ... for the last 15 minutes, let's switch back to Romain's comments 15:10:09 https://lists.w3.org/Archives/Public/public-wcag-act/2016Sep/0004.html 15:10:29 Romain: let's look at the new titles proposal for section 2 "Requirements" 15:12:02 ... current titles sound like reqs on the Rules, but this doc is on the framework 15:12:12 ... I propose to reword the headings 15:12:52 ... "1. ACT Rules should be clear" could be "Ensure Rules readability" 15:13:10 wilco: not sure about readability. what's readability vs clarity? 15:13:19 Zakim has left #wai-act 15:13:50 shadi: these are more for tool implementors, not for the general public 15:14:57 romain: proposed heading 2. "Ensure soundness" 15:15:18 wilco: what I want for the rules is to be valid 15:15:34 ... I want the rules to get valid results for a requirement 15:16:01 shadi_ has joined #wai-act 15:16:51 romain: I thout req #2 was more about mapping a rule to a success criteria, not about correctness itself 15:17:45 wilco: what's the difference between validity and soundness? 15:18:03 alistair: I don't think you can "ensure soundness" without context 15:18:26 ... you're trying to say sth along the lines of "non interpretational" 15:18:41 shadi__ has joined #wai-act 15:19:11 wilco: what I try to mean is "if you ask a11y experts, they would agree this covers the requirement" 15:19:27 shadi: in my mind you mean validity 15:20:52 ... I'm wondering if we're not too high level 15:21:14 ... the framework needs to have a description for assumptions 15:21:18 ... it needs test procedures 15:21:25 ... the outcomes needs to be such and such 15:21:53 ... this is what the requirements should output 15:24:09 romain: I think you're too specific, we're not defining rules but the requirements for the fwk 15:25:15 ... we need sth along the lines of "the rules should reference some accesiibility guidelines" 15:25:51 shadi: we're talking about headings, there is content in the sections 15:26:35 alistair: we're not developing sth new, other industries that do testing have similiar framework 15:26:57 ... we should do a little research, find what's interesting in them, and start from there 15:27:08