Accessibility Conformance Testing Teleconference

26 Oct 2016

See also: IRC log


Romain, Shadi, Alistair, Wilco, Katie, MaryJo, Alan, Moe


ACT Framework Requirements, https://w3c.github.io/wcag-act/act-fr-reqs.html

Wilco: Formatted the framework.
... Last week we reviewed it with few comments.

Romain: URL on the document is not the one on the GITHUB.

Wilco: Will fix that.

<shadi> https://w3c.github.io/wcag-act/act-fr-reqs.html

Wilco: Last week lots of agreement. Alistair suggested editorial.

Alistair: I'll have a look at editorial over the coming week.

Shadi: Placed a pull request on md file.

<shadi> https://github.com/w3c/wcag-act/pull/2

Wilco: Minor changes from Shadi.

Shadi: Mainly changes to the intro part, then consistency so we use the term ACT rules in place of rules.
... Inline comments were placed on two parts - one of clarity; and one on readability.

<shadi> https://github.com/w3c/wcag-act/pull/2#discussion_r84838287

Shadi: We'll run through inline comments.
... It should be the one with three bullet points.
... We're setting a high bar, by saying tool developers and non-technical people.
... It could be that we just want errors to be readable by all.

Katie: Do we need readable, or better understandable.

Wilco: Maybe understandable is better.
... It is important to consider users of test tools too.
... If they don't know what the tools do, they might not know what still needs to be done manually.
... It would be everyone with a good understanding of accessibility, rather than lay-people.

Shadi: So you're saying that people should be able to read through the tests themselves; rather than just the errors or questions.

Wilco: I don't want to assume this is just for people making tools. It is useful as a description, and a process.
... Even without an implementation, the tests should be useful by themselves.

<Wilco> https://www.w3.org/WAI/GL/task-forces/conformance-testing/wiki/User_stories

Alan: Should we define who these rules are being developed for?

Wilco: This is done in the user stories.
... Reads "as a developer..." etc...

Shadi: Possibly say that people should be able to follow the steps in the rules by themselves.
... The rules are still very technical.
... We are still setting quite a high-bar.

Wilco: We have set the goal that rules should be able to replace test procedures in techniques.

Shadi: I thought that was off the table- more like a mapping to test procedures.
... It's still technical people reading them.

Wilco: We could try "tool developers and accessibility auditors"?
... Kathy - could be auditors and testers.

<Wilco> The ACT Framework ensures that ACT rules are readable by both tool developers, accessibility auditors and testers

Wilco: The sentence has been changed to the above.

<Wilco> The ACT Framework ensures that ACT rules are readable by both tool developers, accessibility auditors and testers

<MoeKraft> I would remove the word "both" since we have a list of 3

Shadi: We've talked about readable vs understandable.
... How do we know if we have met the requirement.
... Rules are written in such a way that they can be understood by the parties.

Alistair: A test for understanding will be if two teams can read through them and get the same results.

Wilco: Coming up with a solution.

<Wilco> The ACT Framework ensures that ACT rules are understandable by both tool developers, accessibility auditors and testers. Users of an ACT Rule should successfully be able to follow the steps of the rule and come to the same conclusion.

Wilco: The word both is redundant so is being removed.

Shadi: Remove successfully, but otherwise looks fine.

<shadi> https://github.com/w3c/wcag-act/pull/2#discussion_r84838463

Shadi: move to next comment. What do we mean by consistent?

Wilco: Is that something which would get confused?

Shadi: Ensure consistency.
... Are the results consistent across runs, content; or are they consistent across tools.

Wilco: Both.

<Wilco> The ACT Framework ensures that ACT Rules can be implemented in a way that produces results consistent in different tools and in different runs.

Alistair: Should be consistent.

<Wilco> The ACT Framework ensures that ACT Rules can be implemented in a way that produces consistent results in different tools and in different runs.

Shadi: Looks fine.

Wilco: We're ok for this.
... Call for consensus - so we can publish it.
... It will go to the WCAG working group.
... Shadi - how does that work.

Shadi - suggest a survey, which can go out today and run until next week.

Shadi - you have already asked for review.

Tool comparison table, https://w3c.github.io/wcag-act/ruleset-compared.html

Wilco: I've worked on a tool table.

<Wilco> https://w3c.github.io/wcag-act/ruleset-compared.html

Wilco: Wanted to set up a comparison between different tools.
... The table is a vague outline.
... Wanted to find out about other tools.
... This was a general request for information.
... There is a spreadsheet on GitHub too.

ACT Framework Skeleton, https://w3c.github.io/wcag-act/act-framework.html

Wilco: First draft of framework is set up.
... No comments in yet.
... So we'll just run through this, and get comments on this.

Katie: The output should be in an accessible format as a requirement.

Wilco: We'll have to be much more specific. Earl?
... We'll need an output that we can compare and agregate.

Alistair: ACT input template - you want to say element to whole page, rather than template.

Wilco: Have we gone through it?
... Common data format is needed. So we can compare tools; or use different test tools.
... You can aggregate results from different tools.
... Do you think we need to be more limited.

<shadi> +1 to Katie

Katie: The output format should allow accessible reports to be generated.
... You need to be able to access the data using a screen-reader.

Shadi: The error messages within the earl needs to be accessible.
... There could be outputs from the results that a user needs to view - which need to be accessible.
... Messages need to be accessible.

Katie: Could take language for output requirements from health email that was sent to another group this week.

Alistair: We should not expect users to be able to read raw test data. We should also not set requirements on tools.

Shadi: Those parts that are presented to the user should be accessible.

Wilco: If a tool needs to ask something of a user - the request needs to be accessible.

Alistair: Kathy could you give us an example.

Katie: We could keep this comment. The data format should be accessible.

Wilco: Agreed.
... Further thoughts on the draft.
... We need to start fleshing this out.
... Are there sections that are missing?

Open Actions Items, https://www.w3.org/WAI/GL/task-forces/conformance-testing/track/actions/open

Wilco: Regrets for Charu.
... Action items are done.
... Please fill out the meeting plan.

<shadi> https://www.w3.org/2002/09/wbs/93339/availability/

Shadi: Next week will be 1 hour earlier for US folks.
... For those in the states.

Summary of Action Items

Summary of Resolutions

[End of minutes]

Minutes formatted by David Booth's scribe.perl version 1.147 (CVS log)
$Date: 2016/10/26 15:31:24 $