<maryjom> https://github.com/w3c/wcag-act/issues/290
<maryjom> https://github.com/w3c/wcag-act/issues/282
maryjom: going round back to related issues, 290 and 282
... we're tying it requirement mapping and how to best handle things like non-normative tests
... how to have that marked more clearly in the rules
... so that non-normative tasks aren't interpreted as being required
... this is a coding problem
... we don't want to incorporate something that someone would misconstrue as a requirement
... I want to go back to #290. At TPAC we talked about use cases
... [see Anne's notes taken during TPAC as issue comments]
... we have several use cases like "not required", "partial test" (as composite rule, composite-composite rule, atomic)
... has anybody thought of possible additional use cases? or questions?
Wilco: I just want to make sure we're clear about the language
... when we're saying "normative", are we saying "W3C normative" or whatever is established in the requirements spec?
... also, are we looking at WCAG or any kind of requirements document?
... for instance, some website could say that this particular WCAG SC is not a requirement to them, only a best practice
maryjom: good point, it blurs the line
... for WCAG conformance, some countries exempt some SC
... it's fundamental to make sure we put the right stake in the ground
Wilco: I understand "accessibility requirement" as anything that someone says they want to meet
<shadi> +1
Wilco: it can be any WCAG SC, it can be techniques, external standard, anything that can be put in a requirements document
+1
Wilco: that's what I assumed in the current spec draft
<shadi> https://w3c.github.io/wcag-act/act-rules-format.html#accessibility-requirements
[current definition of a11y requirements: "Accessibility requirements are just that: A requirement that a particular web page conforms to for it to be considered accessible. A common example of accessibility requirements are the WCAG 2.1 success criteria. Some organizations have additional requirements that come from different sources, such as laws, internal standards, etc. These too are considered accessibility requirements which can be tested using the ACT
Rules Format."]
maryjom: the question I have is what if someone writes a rule for a WCAG technique that is advisory? if you meet the advisory technique you meet the criteria
Wilco: I don't think it's the case, that's a difference between sufficient technique and advisory techniques
shadi: actually there are advisory techniques that meet the requirements
<shadi> https://www.w3.org/TR/WCAG20-TECHS/G127.html
shadi: for example G127 is an advisory technique that also meets the requirement (and exceeds it)
... meeting the advisory technique sometimes imply meeting a requirement along the way
maryjom: I can see why shadi proposed "relates to" in his proposal
shadi: I remove the "relates to"; I agree with what Wilco means by accessibility requirements
... if we say this rule is for an advisory technique, or a best practice, what do we call this other category?
... in the latest proposal I called this "other reference"
... but not going into the prose, we should look at use cases
Wilco: I would argue these things *are* requirements, if you could put that in a document
shadi: OK, so you could write a rule for say G127 and refer to that accessibility technique as a requirement?
Wilco: yes
shadi: OK, so different rules could check for different kind of requirements but still appear to be at the same level
... e.g. you have a rule for a given SC "check that a title exists", then a rule that checks that if a page is in a set of pages it describes the location you're in; these rules have the same "standing" in our set of rules?
Wilco: what do you mean by "standing"?
shadi: one checs a WCAG SC, one checks a technique
Wilco: if your users need to know what is a WCAG requirement and what isn't, it's a usability issue. The UI should make that clear
... but doesn't look like something that should impact the rule
maryjom: what do the tool know which one to pick? if it's not clear in the rule format…
shadi: we can have multiple listings
... e.g. you write a rule for WCAG, but which is not a WCAG SC
Wilco: then it's not a rule for WCAG
shadi: right, but you still want to refer to the WCAG technique
... that's why I'm proposing to call that reference "other reference"
... then someone can come along and say "I call that a requirement"
... it's why we talked about "metadata" at TPAC
... you could have several such references, added by the author of the rule
... other people can come along, without changing the rule, only what it means to them in terms of being a requirement or not
Wilco: ok, so it seems you're meaning something different by "requirement"?
... anything that can pass/fail conforms/not-conforms can be an a11y requirement
... so why are a11y references not a11y requirement?
shadi: the difference is that when you're writing a rule, you write it for a purpose
... but someone can use it in a different context
... I think I'm trying to represent in the rule itself what you called a UI thing
... so that it is clear what it meant for the person who wrote the rule
Wilco: there are SC that are not legal requirements in some countries, so in some scenario these SC could be "references" instead of "requirements"?
shadi: yes, you could have line annotations to indicate that, this is a reference in this context, a requirement in this other context, etc
Wilco: it seems it limits reusability of the rule
... if someone from the BBC wrote a rule for having valid HTML, she would put the BBC guidelines as an a11y requirement
... if I was to write the same rule with a WCAG hat on, I would put that as an a11y reference
... so we end up with 2 rules, that only differ in their a11y mappings
shadi: one of the suggestions proposed by romain was to have the metadata outside of the rule itself
... have it in the rule itself because you know what you meant as a rule author, but be able to extend it without overriding the rule's content
Wilco: so you need to make a copy of the rule?
shadi: we don't know exactly, there can be different approaches
Wilco: ok
shadi: I think reuse is a common goal
... the idea for instance was to be able to reuse a rule written for WCAG in the context of EPUB, how would you accomplish that
Wilco: I don't think that needs to live inside the rule
... rules themselves define the requirements
shadi: OK, so that relates to something else romain was proposing, which is to have an external mapping table
maryjom: in the case of an advisory technique, which can be way to meet a requirement although it's not required, how to find a way to relate to the requirement?
Wilco: rules represent non-conformance
shadi: there are varying degrees of relationship
... the rules we have on ARIA checking are not directly conformance issues
... we may put a relationship to WCAG when there can be none
... similarly for advisory techniques, you can have some kind of "confidence" that you meet/fail conformance
... you have varying means of giving feedback to the evaluator
... how do you define this?
Wilco: through the assumptions
... you write the assumptions about the environment you're testing
... like assume that somebody isn't trying to hack ARIA but actually intends to impact the accessibility tree
shadi: what about things with no such confidence?
Wilco: same thing, you can express that as an assumption
... for instance even for valid HTML, you could write that you assume this would show up as a parsing error
shadi: right, but we want good quality rules as well
Wilco: do we? I mean seriously, what is "we" there?
... yes W3C is expecting good quality rules
... but the lecel of required accuracy depends on the users
... so we can't make a broad assumption about how much accurate the rule needs to be
maryjom: I would really like to see an example of how you'd write up something that tests an advisory technique
... that would help us visualize how it fits in that space
Wilco: how to write a best practice rule?
maryjom: yes, if you're testing an advisory technique
Wilco: let's stick to the HTML validation issue
... you can have an assumption that says that you expect a page valid to the HTML 5.2 standard
<Wilco> You can have an assumption that says if a page does not have valid HTML, it is assumed not to meet the parsing requirement
maryjom: you were saying you would document the related WCAG criteria in the expectations?
Wilco: no, I don't know; this is an example of a rule with an applicability that assumes that the HTML conforms to HTML 5.2
... that's a rule for a technique
... if you want to test for this specific validation technique, you would use expectations
... if you want to use the same rule for WCAG, then you could use an assumption
maryjom: do others on the call think it's a sufficient way to document rules that are for non-normative kind of things?
Wilco: I think we're struggling with a couple questions
... how do you define the normative status of your accessibility requirements?
... shadi proposed to have both a11y requirements and a11y references
... (not sure it's the best proposal)
... the other thing is how you want to include something that is *not* an accessibility requirement in a rule
... that's a second question I'm hearing
... my example shows how there's flexibility in rules
<shadi> [lost audio but want to ask, how would you reuse that rule for validation in another context where it is a requirement - do you then duplicate the rule?]
Wilco: a 3d question is how to deal with rules that are not direct failures but have some level of heuristics
anne_thyme: what was your proposal Wilco?
Wilco: I haven't made any yet
maryjom: the proposal came from shadi
... we've been so far looking to define the problem we wanted to solve
anne_thyme: how do we make sure that people understand what the mapping is?
... it's different whether it is an advisory technique, a sufficient technique, a full test, etc
Wilco: would you agree it's covered by the first question I had above?
anne_thyme: not sure what you meant by that
Wilco: I would say that SC are W3C normatives, techniques are W3C notes, RGAA are "french-governement-normative"(?), BBC are BBC-internal-normative
anne_thyme: if I write a rule, it could map to one SC when failing the rule means failing the SC
... but it can also relate to another thing that an advisory technique
... I think we need to create an explicit mapping for the rule to the requirement, not the other relations
Wilco: right, I think this is the second question I had above, how to you express related stuff
anne_thyme: I don't think it's enough to have one pile of requirements, one pile of relate sutff
... you need information like "if this rule passes, that's what it means / if this rule fails, that's what it means"
<Wilco> Have to go, bye everyone
maryjom: Silver had some prototypes they wanted us to look at
... I will send an email
... thank you everyone