auto-WCAG Accessibility Conformance Testing (ACT) proposal discussion

ITEM: Review of ACT proposal

Judy: What is the status of the documents? There seem to be still some comments.

Wilco: Comments mostly editorial.

Judy: Ok, I can contact the WCAG WG.

Wilco: Had conversations with Jon Gunderson und Jesse Beach.
... would like to discuss their feedback today.
... We have a good idea what we want to do but we need to communicate better.

ITEM 1 Discussion: Developer focused

https://www.w3.org/community/auto-wcag/wiki/index.php?title=Talk:Accessibility_Conformance_Testing_for_W3C&oldid=2435

Wilco: Agree with Jon's comment that developers shouldn't be forced to choose between several options.
... Jesse would like to see tool that integrate with development tools (so that developers can use the evaluation during their work on a web site).
... Focus shouldn't only be on conformance testing.

John_HICKS: Clarifaction: developers = web developers (not tool developers).

Wilco: yes.

John_HICKS: There is a lot of complexity in web development. I don't see that ACT as adding another level of complexity.
... Developer should have a choice to select one of several options.

Wilco: Jesse's comment was that there shouldn't be several options.

shadi: The would be a simplification. There is only one way to meet a requirement.

shadi: Previously we talked about the use of warnings. Do we need both?

Wilco: yes, both are needed: warnings (something might cause an accessibility problem) and additional messages (this technique is not recommended or not allowed in this web site = rule defined by an organisation).
... The additional rules could be company policy or government stardards.

Judy: Comment on earlier comment (about complexity): WAI has gotten this comment before.
... For example during the work on WCAG 2.0.
... We need to find a balance between complexity and comprehensibility.
... The motivation for ACT was to clarify the interpretation of WCAG 2.0.
... This would be welcomed by evaluation tool developers.
... We need to make sure to avoid confusion (caused by test results).
... Maybe we need to decribe the expectation more clearly.
... Which level of complexity do we need?
... Can to ACT rules be used for other rules?

Wilco: Main focus is WCAG conformance testing.
... Company policy rules could be added (e.g. at BBC).
... Different kinds of rulesets (policy) can be stacked in the ACT framework.

Judy: What about the balance between complexity and simplicity? What it the audience?

charu: Two aspects of developer perspective: tool developers and web site developers.
... WCAG is very generic. The rule specification should support the checking of the requirements.
... Form the previous discussion it seems that we are trying to do more, e.g. by taking into account organisational requirements.

Wilco: The ACT framework it not limited to WCAG.
... Also EPUB accessibility testing goes beyond WCAG.
... Or ARIA best practice testing.
... The challenge is: How do we communicate this?

Charu: We are focussing on the tool developers?

Wilco: Yes.
... Web developers are the secondary audience.

John_HICKS: The comments by Jon and Jesse should be considered.
... It is up to the (web) developers how they build the web site. They need knowledge of WCAG.
... Complexity can't be avoided. It is not a negative thing.

emmaPR_bbc: Confirm that BCC does things a bit different from WCAG.
... BCC could use the ACT framework to build their own ruleset.
... E.g. BCC requires a hierarchical use of headings. That would be an additional requirements.

Wilco: The power of the framework is that BCC can write such a rule an contribute it to other organisations.

Judy: These issues should be addressed in the scope section of the proposal.

<Judy> https://www.w3.org/community/auto-wcag/wiki/Accessibility_Conformance_Testing_for_W3C#Scope

Judy: Clarify that main audience of the work are tool developers.
... State that complexity is necessary.
... Clarify to what extent the framework can be applied to other technology.
... What is the initial scope of the framework?

<Zakim> Judy, you wanted to suggest a way to communicate this

That other technologies like EPUB could be added.

Judy: 2nd point: Initial focus are HTML and ARIA and maybe EPUB.
... and other technologies might be added.

Wilco: good point.

<Judy> [jb: and i think css too]

Avneesh: I suggested to add EPUB because this technology will be important in the future.
... The tests have to cover a lot of cases. But the web developer needs to know only one technique to solve an accessibility issue.
... W3C has high focus on EPUB.
... In the framework EPUB should have the same priority as web.

shadi: Agreed with previous comments.
... Separate between framework and ruleset discussion.
... We need to clarify when and how rules (beyond WCAG) can be contributed back?
... The rules might even conflict with WCAG.
... This is a potential source of confusion and complexity.
... HTML, CSS, ARIA and EPUB are a big task already.

<emmaPR_bbc> I didn't suggest we contribute back (though not against helping) ... only that BBC could use auto-wcag as a basis for our own test suite

shadi: Processing non-WCAG rule would be too much effort.

Wilco: The focus of auto-WCAG is WCAG 2.0 AA.

<shadi> [[Not you Emma, I thought I heard Wilco say something like that]]

Wilco: Other groups might be working on other criteria.

Judy: Agreed with Shadi that we need to clarify that ACT does not supersede WCAG 2.0.
... We need to clarify the term framework. Suggest to rename deliverable 1 to framework.
... Agree with Avneesh on comment about EPUB. EPUB rules might be developed by a different pool of developers (but could be within this same group)..
... An extensible framework must also be extensible for later work in WCAG.
... This should also be mentioned.

<emmaPR_bbc> Agree Judy ... I'm hoping auto-wcag supports WCAG 2.0 and helps with clarification and understanding for conformance ... to help teams creating test tools

Judy: A huge concern is the scalability of the ACT work. This could be resolved by incorporation of existing rules. Where is this mentioned?

Wilco: yes, this is not covered yet.
... Question about terminology: Is "conformance" the right word?
... Or is there a better word?

Judy: Some organisations really want ACT to address conformance.

maryjom: Our concern is the different tools use different interpretations of conformance.
... So tools might yield different or contraticting results.
... The tools must be close to WCAG.

John_HICKS: For web developers "conformance" means "good enough".
... The word conformance is ok.

Judy: But "conformance" is used in WCAG with a specific meaning.

<emmaPR_bbc> I agree that for now conformance is probably the best word

Judy: That is a strong tie between ACT and WCAG.

Wilco: There seems to be agreement on the work conformance.

<John_HICKS> "conformance" ... is good enough

Charu: IBM uses the term "accessibility verification".

<emmaPR_bbc> Hoping ACT will help bring agreement to interpretations

<maryjom> Definition of conformance: Certification or confirmation that a good, service, or conduct meets the requirements of legislation, accepted practices, prescribed practices, prescribed rules and regulations, specified standards, or terms of a contract.

Wilco: Will email a summary of the comments by Jon and Jesse.

<emmaPR_bbc> meant auto-wcag, not ACT

<cpandhi> https://www.w3.org/community/auto-wcag/wiki/%28Proposed%29_Accessibility_Conformance_Testing_%28ACT%29_Task_Force_Work_Statement

Wilco: Can we forward the proposal to the WCAG WG now. Although some updates and clarifications are needed.

Judy: Yes, already done informally in IRC, and I'll follow up in email after..

Wilco: final thoughts?

Good discussion!

<emmaPR_bbc> Thanks annika ... great scribing

Judy: Wilco please update the documents and send a message to the mailing list.