W3C

Accessibility Conformance Testing Teleconference

18 Oct 2018

Attendees

Present
Romain, Jey, Wilco, Anne, MaryJo, Alistair, Shadi
Regrets

Chair
Wilco, MaryJo
Scribe
Jey

Contents


CFC: Spec and exit criteria can go to the AG WG to approve to publish

wilco: no objections on the CR, a few q's from Anne.

anne: Q1: a bullet called outcome definition, where outcome links to a definition list, but clarity on what it means should be worked on...

wilco: fair point, can this be worked on the CR.

shadi: was this discussed last week?

wilco: the point of adding that def, was to help with outcome resolution

anne: sounds good, still have no clue as to what it would mean in the rule

<Wilco> https://w3c.github.io/wcag-act/act-rules-format.html#outcome

wilco: means we have to add a definition to each rule (that would be in the act rules format).
... create an issue, to do some amendments, to add sections to outcome.

anne: next Q, "An ACT Rule SHOULD list an identifier and a reference for each accessibility requirement that it maps to.” à “An ACT Rule MUST list an identifier…"

wilco: SHOULD is better terminology than MUST (no action).

shadi: can this be fixed afterwards, as this is not a requirements

wilco: yes

anne: Section “7.2 Accessibility Requirements Mapping” From this new section it seems that I could list an accessibility requirement for an atomic rule, where it is actually the composite rule that is mapped to the success criterion, as long as I write something like this for the mapping: “failed: Further testing is necessary to determine if the success criteria are not satisfied”. Is it the intention to allow this?
... some people would like to link to SC, if we allow for this...

wilco: do not see a problem with it

shadi: Disagrees, having it open as suggested above sees an opportunity for misuse.

wilco: does that mean we need to change?
... Reason that this works, it gives the rule more flexibility, which is why we did this in the first place.

shadi: eg: create a keyboard trap check, that checks that there is no trap in a particular element (input box), and the rule passes, it passes the requirement except when the content is using other kinds of input elements. This example undermines the scope and it would only be a partial check...

wilco: still do not see if that is a problem..

shadi: may be we should say "I DO NOT KNOW" as against "PASSED".

agarrison: agrees with shadi
... if we list all the things that passed or failed, and has some edge case scenarios and cannot be well interpreted.

<shadi> Met-on-Pass: Accessibility requirement is met when the rule passes

<shadi> Met-on-Fail: ???

<shadi> Met-on-Inapplicable: ???

<shadi> Unmet-on-Fail: Accessibility requirement is not met when the rule fails

<shadi> Unmet-on-Fail: ???

<shadi> Unmet-on-Inapplicable: ???

<shadi> Unknown-on-Pass: @@@

<shadi> Unknown-on-Fail: @@@

<shadi> Unknown-on-Inapplicable: @@@

shadi: what other conditions would you like to allow

wilco: would like to allow for flexibility to the rules

shadi: i think we should not encode over arching logic into the rule...

agarrison: ultimately chaining multiple rules to get clean outcomes may be the solution

wilco: that takes us back significantly, as stacking composite rules is something new, and there are concerns.
... unsure what to do with this..

agarrison: surely, we should have thought of, stacking composite rules

wilco: we considered it, and dropped it.

shadi: can we say, as we cannot have an hierarchy of composite rules, we put that logic elsewhere as in, within the rule...

wilco: if we went with the list solution that we are discussing, how we would ever know if a SC is satisfied?

anne: eg - page has title

agarrison: we have diverged from the Q.
... provides a driving license test, point being there should not be extra caveats on PASS vs FAIL

shadi: 'further testing', does not convey the right message.
... more examples...

wilco: do not like the idea of using composite rules as mechanisms to aggregate
... what composite rules are for, is for looking at multiple solutions, that needs to be combined to test for a specific element...
... aggregating outcomes on a page level is not something that composite rule should cater to...
... why not use the accessibility requirement section, to have the same expressive logic?

shadi: still unclear about the concern, could we provide an example?

wilco: I do not see how putting it in expectation solves it?

shadi: because there is no clarity in expectation otherwise and it can be misleading..
... more examples...
... the expectation should be clear to express if it is a partial check...

agarrison: this exposes an interpretation problem

wilco: what is currently proposed, solves for that already...

everyone: voted on this issue, and still no consensus

anne's q: Section “9. Atomic Rules List (Composite rules only)” There was talk about allowing composite rules to list use other composite rules as input. Where did this end?

wilco: can you expand anne?

anne: there is a long email about the whole subject, that we just discussed, so stacking composite rules would be a good solution...

wilco: this links to previously discussed issue, and needs re-evaluation

anne: next Q|:

“Section “10.1 Applicability for Atomic Rules” We suggest (again) removing the following, that is a leftover from the old Auto-WCAG rules format: “When a formal syntax, such as a CSS selector [css3-selectors] or XML Path Language [XPATH], can be used, that (part of the) applicability MAY use that syntax in addition to the plain language description.” If we want to keep it, we suggest moving it to a note, since it’s not really part of the formal s[CUT]

agarrison: Question:

Applicability Section. We don’t account for the option to set applicability to being always true for a test. Certain things just don’t have an applicability test, or the test is the same as the applicability test e.g. test for marquee element.

wilco: clarifies above...

agarrison: agrees

everyone: wrapping up meeting...

Summary of Action Items

Summary of Resolutions

[End of minutes]

Minutes manually created (not a transcript), formatted by David Booth's scribe.perl version 1.154 (CVS log)
$Date: 2018/10/18 15:03:59 $