Accessibility Conformance Testing Teleconference

Wilco: Last week we looked at adding a couple sections to the framework draft. Jemma and Wilco worked on these sections.

<Wilco> https://w3c.github.io/wcag-act/act-framework.html

<Wilco> https://w3c.github.io/wcag-act/act-framework.html#intro

Wilco: Want to get your thoughts on Introduction section
... Based on earlier work in About ACT

Katie: Overall is good. But last paragraph needs some updating
... "reproducible, understandable and consistent"
... Going for consistency along with how and why
... I think transparent is okay. But need consistency
... Transparent is fine but need why and how.

Shadi: Similar notes as Katie. I think the last paragraph is more describing the work than the rationale. This should be first. Make it a more positive approach.
... This work is to contribute to reproducibility, consistency, transparency. This will lead to a more common understanding by accessibility experts.

Katie: Points are correct but put in a positive light rather than negative

Allan: Would more :consistent validation" be good?
... Talking about consistent validation

How about interpretation?

Wilco: I could do how and why but there are certain things I'm not touching upon, e.g. consistency of output

Shadi: Add each of those buzz words. Make a list. "consistent results" "transparency of test methods"

Wilco: Nice suggestion
... Any other thoughts?
... Jemma

<Wilco> jemma: Reproducable implies consistent results, we should make it clear

Jemma: I think the word reproducible include the concept of consistency. Just need to make it clear.

<Wilco> https://github.com/w3c/wcag-act/pull/3/commits/ede7c5fe1bae932429fd5bdd6493e1ce9ea69559

<jemma> can you hear me?

<jemma> ok

<jemma> I don't hear anything

Wilco: This pull request is an update to the section on change management

Jemma, we can hear you

Shadi: I think Change Management is much broader. Wasn't this going to be Versioning?

Wilco: I took Change Management from IEEE language

Shadi: Is there also a procedure on to incorporate CM

Katie: That term in America is for when a company changes management process

Shadi: Could be for a product as well
... Envisioning more a format for versioning. Not sure if there are management procedures expected.

Wilco: I doubt it.
... Really don't see this more than tracking different versions

Shadi: date or data space, which contains a url for each version
... for example, https://www.w3.org/TR/WCAG20/
... But may also be tracking changes, a little more sophisticated

Wilco: So one of the things that I am wondering about is how do we cover assumptions, something a rule does not cover but may lead to a false positive
... Something that you could add to a rule without changing the test process
... What I think rule changes will look like is that we find out there is certain technology that is no longer supported or missing accessibility features, that would have to go into assumptions of a rule.
... If this is used, the results of the rule might be off
... Assumptions are kind of like issues.
... The way some things are tested

MoeKraft: We discovered one such issue where a test was being run against APIs instead of DOM

Charu: Agree. Browsers and agents act in a certain way and all of this should be tracked in assumptions.

Wilco: One example, links in a block of text should be clear of surrounding text. There are different things you can do to a link to distinguish links. There are other things that people could use that would allow them to pass this SC but rule wouldn't be able to detect
... for example background color, Some other technique that they are using that they are using say text shadow for distinguishing links, we could make a new version of rule that takes into account text shadow. Issue: text shadow isn't considered. Next version we can.

Shad: So the change to the rule can be in several different areas could be substantial or minimal, changing logic vs. broken sentence. What does this buy us?

Wilco: Not sure if the entire rule should be unchanging or document unchanging, or can a part be updated without changing the rule.
... When writing a rule, you may find there are more assumptions. Assumptions shouldn't change the rule but gives a list of things for the next version we may address.

Shadi: Do you want different kind of flags? change logic vs. not changing logic?

Wilco: Possibly

Shadi: This gets very dicey. A comma will not change impact but removing assumption changes rule or results. If you run a version of logic with one set of assumptions you get different set of results. I my view different versions.
... If you consider logic only that's one thing.

Wilco: Scope and assumptions tell you when you shouldn't use rule or rely upon its result.

Shadi: What if this rule assumes "shadow" is not used. Output doesn't apply. If someone adds that, then we get a result, pass/fail. Two different results.
... Need to indicate that something has changed
... What is the issue that we are trying to address?

Wilco: Auto WCAG writes rules. They are implemented. Someone uses a tool that implements those rules. The tool is dated. The WCAG community group found new ways to update rule. We know ...
... that changes were made because of issues to the rule. How will a user know that there are certain issues with the rules they are using?

Jemma: Change management includes how we are going to communicate in addition to the change. How are we going to communicate to developers and testers.
... Changes in rule and changes in test results. Need to dissect each point for each section and narrow down scope

Alan: Are we talking about Versioning Control. And communication of those changes?
... If we have a rule that form field should have a valid label. A lot of time we see strike through in sales price. We haven't checked that strikethrough is used but just checking valid label. Label is there but no usable.

Wilco: Good example
... If someone is using an older ruleset in their testing and look up documentation for that rule should be able to get to newer version of rule which includes list of improvements. Can I still use this? Are these critical?

Jemma: I think this is a product of change management. How do we get to the point of target of change management.
... Use public list. Get feedback, etc.

Wilco: I think this is beyond the scope for ACT. At least for the Framework
... Rules can be written by anybody. I think it is important that we have consistency in how changes to rule are managed.

Wilc: QA team will be able to use rules from different rule sets.

Jemma: What is the target of change management?

Wilco: The rule

Jemma: It's like a moving target
... Rules are very static based on description

Wilco: Rule would be a static document. Might be a new version of the rule and that would be its own document.

Charu: We are still discussing the English version of these rules and not the logic.
... What we have is an update log. So when we make changes to rule logic, it goes into update log. Since we have an English description, we also want to update the English description to reflect any changes.

Wilco: Assumptions can lead to changes.
... In this scenario we will have a failure. In future definition can address that assumption.

Charu: That will be two different things. Make updates and change log that describes changes

Wilco: I like the idea of Change Log
... Why did I go for a Change Management? Because Change Log is part of Change Management more than just versioning

Shadi: That should be clarified under different headings.

Wilco: Anyone else?
... I will take Jemma's work and input and come up with a new section for next week.

<Wilco> ACTION: Wilco to create a draft for Change management section [recorded in http://www.w3.org/2016/11/09-wcag-act-minutes.html#action01]

<trackbot> Created ACTION-16 - Create a draft for change management section [on Wilco Fiers - due 2016-11-16].

Drafting Framework sections

<Wilco> https://www.w3.org/WAI/GL/task-forces/conformance-testing/track/actions/open

Wilco: Would like to go through open action items
... Charu, have you had a chance?

Charu: I updated the audience and created a pull request.
... did try to take a stab at rule description. Not as easy as I thought.

<jemma> Wilco, could you kindly close my action item since I am done?

Charu: When I try to come up with requirement what the rule is supposed to test which is essentially SC. I thought we already had this covered in another section.

<cpandhi> https://w3c.github.io/wcag-act/act-framework.html

Charu: Wasn't sure exactly what to put in rule description
... accessibility requirement is Success Criteria

Wilco: Can be SC or any accessibility standard

Charu: I will take another stab at rule description

Wilco: Take requirements document to WCAG WG. Will do that today
... Change management done
... Intro section done.

<Wilco> ACTION: Wilco to update the intro section based on feedback [recorded in http://www.w3.org/2016/11/09-wcag-act-minutes.html#action02]

<trackbot> Created ACTION-17 - Update the intro section based on feedback [on Wilco Fiers - due 2016-11-16].

Wilco: I will create a new action item for Introduction section

Jemma: You're a hard worker. : )