See also: IRC log
<MoeKraft> Latest draft: https://w3c.github.io/wcag-act/act-rules-format.html
Wilco: Publication requirements...any comments?
Shadi: Stein-Erik will follow up with me next week
<Wilco> https://github.com/w3c/wcag-act/issues/82
Wilco: seems to be strong emphasis on automation
what are people's comments on this?
MoeKraft: There is no specific call out on automation. Ruleset talks to it. Explain in scope.
<shadi> +1 to Romain
Romain D : QA testing and distinguishing between automatic testing and semi-automated , test tools in broader sense
<MoeKraft> https://w3c.github.io/wcag-act/act-rules-format.html#abstract
Shadi: I agree with Romain's thought. Testing tools in the broader sense
Wilco: May not even have to say for "tools" , rather for evaluation of accessibility
<MoeKraft> Currently the first statement reads: The Accessibility Conformance Testing (ACT) Rules Format 1.0 is a specification designed to harmonize how accessibility rules are described for automated test tools, and how test procedures are written for quality assurance testing.
Shadi: good to mention test tools, but
could mention later on. Primary goal is to harmonize accessibility
rules...."including" automated and semi-automated tools.
... Will take as a pull request and work on it.
MoeKraft: will open item on that for Shadi
Wilco: Do we want to include partial test results text into introduction as well? Combining different test results...abstract...
Shadi: A rule doesn't necessarily test one success criteria, could include many within one test
Wilco: Rule aggregation already covers that.
Shadi: this should be a heading of its own. Relationship between test rules and WCAG SC's. Or broaden criteria. Mapping between test rules and SCs. Needs own heading.
Wilco: Mentions we can talk to this in section 3.3.
Shadi: either in introduction or scope
would be better, more general and not inside test rule
... use "Mapping" vs. "relate".
MoeKraft: will create ticket on creating
section within scope on mapping between rules and success criteria
... will assign that to Wilco
Wilco: Talking towards the comment 2: Outcome "Cannot tell". Confusing on what that actually means. Might consider moving away from that outcome in regard to testing / auditing. Was part of EARL 1.0 , why did it happen that way?
Shadi: Cannot tell, in EARL 1.0 was more general. WCAG SCs are designed that they are always applicable.
If no video, there is no applicable content, so SC is met when no applicable content
heuristic tests and human input is needed...
Shadi: in 2.0 there is no such thing as Not Applicable
Wilco: when it comes down to it, human input is needed.
Shadi: several tests may be needed to accomplish the rule, and there may be cases where the person completed 1 of 2 tests and can't meet.
Wilco: ACT rules are supposed to be independent of each other.
Shadi: Is result always pass or fail ? Or are other results possible?
Wilco: yes, there are others than pass / fail.
Charu P: in IBM rules, it is a violation, clear cut pass / fail. Usually fail. Then possible violation, if more than one way to meet SC. If rule is looking for one technique....manual tests form should have submit button as an example
manually testing: video needs testing manually for captions
Wilco: needs review auditing issue
type...these all somewhat relate to the cannot tell type.
... Do we want to use a different wording than "cannot tell"?
Charu: cannot tell should point to human decision required.
Wilco: more thought would need to be put into this.
<MoeKraft> I like "potential violation"
Shadi: example situations could be introduced in terms of "cannot tell".
<MoeKraft> or "requires review"
Wilco: What happens if we don't go with EARL terminology?
Shadi: subset can be put into place. But what is need? I.e. is outcome undetermined, neither pass or fail.
<MoeKraft> +1 Shadi's term "undetermined"
<Wilco> https://w3c.github.io/wcag-act/act-rules-format.html#output-outcome
General question from me: I.e. is undetermined not present and therefore not testable?
<MoeKraft> Sorry, my phone batter died. Just rejoined.
<shadi> "The result of the test is undetermined. This may be because human intervention or further testing is needed."
Wilco: Will ask Raph to clarify comments for 6.2. Will talk to variations of this.
<Wilco> https://www.w3.org/WAI/GL/task-forces/conformance-testing/wiki/Testing_Resources
<Wilco> http://lists.w3.org/Archives/Public/public-wcag-act/2017Apr/0020.html
Wilco: Would like to start doing is have a
look at Deque's repository
... can we update the test case repositories to work with this file
format as well?
Charu: Take each test case and put this in this format?
Wilco: Put JSON file , pull into large repository
MoeKraft: to Charu: Karma test run...instead of JSON that takes rules to scan content. JSON for each test case. JSON used to describe rule format i.e. meta data
expected outcomes
Wilco: Moe and Charu will do that at IBM and repositories will be talking to each other soon hopefully.
<Wilco> https://www.w3.org/2017/11/TPAC/
Wilco: to book hotel with group code
... feel free to join the AG working group call tomorrow if you want, as
we have a slot during that call
... talk to you in two weeks