<Wilco> https://github.com/w3c/wcag-act/issues/121
Wilco: Would it possible to have rules that rely on other rules? Either super rules or meta rules. Similar to rule groups in the current draft. Next step from simple aggregates where rules directly use the results from other rules.
rdeltour: Is there any difference between this and the current rule groups?
Wilco: Rule groups are more simple. They
allow you to pass a rule of groups by simple passing any one rule within
that group.
... Meta rules can do more. It gives you a hierarchy of rules as Kathy
talked about.
rdeltour: It does make sense, though an example would be nice.
Anne: We don't think it would be a good as
it breaks the atomic nature of rules.
... Changes to results in one rule might affect the results of another
if we go this way.
Mads: From an implementation standpoint,
shared functionality would make sense. Have it be an implementation
question rather than a rule question.
... The rule definitions should be atomic, though optimizations at the
implementation level can be made.
Wilco: This would lean towards a specific
implementation of a rule where we're looking to optimise things.
... Would be too descriptive in a way we don't want.
Anne: Color contrast checks for level AA and AAA would make sense when talking about shared definitions. Same goes for definitions of visibility. These definitions could be shared across rules, but the rules themselves need to be atomic.
Wilco: I like the glossary idea which makes sense for a lot of stuff. These definitions could be a lot more technical, which we don't need for rules.
Mads: What super tests do allow you to do is to allow having a single rule fail which would fail the depending rules.
Anne: Couldn't we build logic into rule groups?
Mads: Not the way it's worded now.
Wilco: Do we have examples of rules where
when one of them fails the whole group should fail? Sounds like an
implementation detail.
... If you like at image rules, a first rule would be to make sure the
image has an alt attribute. If it doesn't, it's always a violation. A
second rule in a similar group would be checking if the alt attribute is
descriptive.
... If we're looking for more atomic rules, then separating them out
into groups has a certain logic to it.
... Would be interesting to see what makes sense for manual tests.
Anne: I think that would be interesting to look into. From the manual perspective, tests are often reversed versions of the automated rules.
Wilco: The question we need to answer is
which is preferred for documentation?
... If you want to rate how a tool should do image testing, would it
make more sense for it to be more documents or just one?
Anne: What are talking about splitting out?
Wilco: Any violation meaning that a whole group would fail would allow to have less steps in a rule.
Michael: Both examples are atomic rules, so both need to pass. It sounds crazy to have them in different documents in that case. We might also be able to reuse them.
Wilco: Different documents would mean different rules.
Anne: It would make sense to implement only
automated rules if needed and having atomic rules would allow this.
... The rule group description would then link to the associated rules.
Wilco: Makes sense.
... If think what we're missing is allowing a rule group to fail if any
single rule within them fails. Meta-rules would lean more towards
implementation.
<rdeltour> +1
Wilco: What we right now allow is for
different rules within a group to have multiple selectors.
... Does that still make sense to us?
Mads: It might be necessary.
... Different input might be required for rules within a rule group.
<rdeltour> IMO it still makes sense to have different selectors
Mads: Two separate pieces of data might need to be passed to a rule.
Wilco: The point where this confuses me is
that you've got the different selectors, now an element is only tested
by one rule in a rule group. What does that mean for those other rules?
... You've might have a conflict of applicability if only a single rule
within a group says that an element passes.
rdeltour: If you're looking at
applicability for an element within a rule group, it doesn't matter for
the individual rule. Applicability is aggregated for the rule group.
... There shouldn't be a conflict, but the rule should explicitly define
the applicability. I don't think it's an issue.
Wilco: What happens if you haven't implemented everything in a rule group?
Michael: Can you do that?
Wilco: I think so.
... We need to list out the different scenarios.
... It simplifies things if you can't claim anything about a group
unless you've implemented everything.
rdeltour: Maybe another approach is to assume that we can use different selector and then come up with counter examples. If we don't come up with any then it should work.
Wilco: We went from something straight
forward with an inverted aggregation where a single rule passing lead to
everything else passing.
... We want to make that we thought through the whole thing. What
happens with elements that only apply to one rule within a group?
Mads: Some rules within a group might be automated, some might be manual.
rdeltour: It might be a bit overkill to
restrict things by design.
... We have a process in place for approving the rules.
Wilco: I'm worrited about getting
inconsistent results.
... Are we getting back to the problem we're trying to solve with rules
format in the first place?
Martin: Isn't it just a question of
transparency?
... If we're good at showing how we got certain results is it really a
problem?
Shadi: Some tools might support more test
cases than the others.
... That brings back the question of when to adopt a rule.
... What if tools support different parts of the test cases?
... At what point do we say that a rule is supported sufficiently?
Wilco: Unless you pass every test case, I don't think we should say that you support a rule.
Shadi: What if someone brings in new test
cases to a rule?
... Some vendors might then adapt their tools to those cases.
Wilco: That would be a new version of the rule then.
Shadi: That reduces complexity. Where would the differences then be?
Wilco: For rules that require X number of
passes, you can then only claim the rule as a violation if you've
implemented everything.
... Reverse is true for a rule that require X number of violations.
... That's the thing that is missing.
Shadi: If we do at a later point add a rule to a group, that would be a new version of the group.
Wilco: Groups have to versioned, yes.
Anne: Do groups have test cases?
Wilco: Groups don't have logic, so they don't need to.
Shadi: Could there be interaction between the pieces that require unit testing?
Wilco: I would say not.
Anne: If the test cases indicate how much
you support, don't we also need test cases for rule groups?
... Also in relation to the matrix we talked about.
Wilco: You would have to aggregate the data
for the success criteria, though we have to define that.
... Anyone who doesn't think that's the case?
... Also means we have two types of rule groups. Do we need definitions
for them?
Anne: What are two types?
Wilco: A group that requires certain number
of passes vs. a group that requires certain number of failures.
... Anybody have ideas for what to call them?
Martin: Wouldn't it make sense to introduce logic for rule groups?
Wilco: That was the proposal for the meta
rules.
... It's overly prescriptive of the implementation.
Mads: What we opposed with meta rules was
the dependencies between rules.
... Rule groups seem more like an aggregation tool.
... We could have more conditions for rule groups.
Martin: In the current draft, we're still saying the standard should be technology agnostic. If we want an example for everything, we might limit ourselves.
Wilco: Rules are not technology agnostic. A rule is written for a specific technology.
Martin: We're trying to find examples to back up our decisions, which might be dangerous. The examples we might think of in a web environment might not be enough for the standard to support.
Wilco: Let's talk about next time how we can be more abstract and get away from implementation details.