Accessibility Conformance Testing Teleconference

12 Jun 2017

See also: IRC log


MaryJo, Wilco, Shadi, Chris, Romain, Moe, Manoj, Sujasree, Charu
Wilco, MaryJo


Issue 81 - First draft of publication requirements

shadi: just got a message from Stein Eric, asking me to take over. I'll try to bring sth later this week for next week's discussion

wilco: do you want anyone else to jump on that with you?

shadi: might be easier to just draft sth, and then people can jump in on that.

Versioning of rules

wilco: we had some discussions on the 1st draft of our spec

<Wilco> https://w3c.github.io/wcag-act/act-rules-format.html#quality-updates-version

wilco: we put in a little paragraph on version numbering.
... we're gonna use a version of an X.Y.Z scheme
... there was a few comments we had unaddressed, coming from Kathy and Alistair
... it came down to a concern about 1. wether this is too complex, too much overhead and 2. how to deal with many rules published all at once
... I think version numbers can be quite useful for rules, I like tracking the changes
... the WCAG techniques are currently updated every 6 months
... they just publish updates, and you just have to go back in history if you want to see what changed
... speaking for myself, for our customers, being able to track what things changed exactly, being able to identify where a specific issue come from, is particularly important
... I see it might be less useful for others

rdeltour: versioning sounds like a good idea to me as well

shadi: I guess there's another perspective. Not only a client wants to see the changes, but also how my own testing is impacted retroactively
... we need to separate between simple editorial changes, and more "major" change that impact the outcomes
... my understanding of Alistair's proposal is that would become a new test
... it's not a black and white clean line. what happens with regression testing needs to be addressed as well

wilco: I don't think the idea of making a new rule for a major change is gonna hold up
... we defined a major change as anything that can cause a PASS to become a failure
... it may be sufficient to track that in either a changelog or a version
... we could consider doing either one or the other

rdeltour: I think the recomendation should be the same for all rules, it shouldn't be optional
... otherwise it's less useful (not deterministic)
... a version number is either to parse than a changelog

maryjom: we use versions on rule sets, not on an individual rules
... we then let people know what change in a group a rules
... we don't update individual rules all that often

wilco: for Deque we do the same thing.
... clients only track the version of the tool. they don't always want the latest
... it seems to me that in practice nobody is tracking individual versions of rules
... at the moment we're not even tracking the history of rules
... is it sufficient if you bundle all your rules together and say "this is the latest version"
... for implementations would that be sufficient?

maryjom: I think it might
... if we're trying to make rule that go along with WCAG, we can make sure that bundle of rules is compatible with whatever the bunlde of technique is at

wilco: you'd lose 2 things
... if we track changes, you can look at a rule and see at what point the rule changed

maryjom: with github

wilco: you're supposed to include a changelog of all major or minor changes in an ACT rule, so you keep a history for an individual rule somewhere
... I guess the only thing you'd lose is that you can't say "I'm using rule X.Y.Z from the N release" as a tool developer, since we'd not be tracking individual versions

rdeltour: a very important use case is for a user, an a11y auditor, to lookup the information on how/why/when a rule's behaviour changed

<MoeKraft> +1

<MoeKraft> to Wilco

wilco: if you have a change log, you might not need a version number

shadi: I think we'll need both
... some people with say version X of the test rules, but some other want to know about individual rules

<Sujasree> <sujasree> +1 to wilco... it is sufficient to have change log and do not require version number on individual rule file

shadi: we might want to differentiate major and minor changes
... how much of a change can you still bear to be the same rule
... there might be a variety of changes, technology emerges, AT change, etc. all this can influence a rule
... and bugs, typos, etc

wilco: so you're saying we should still track versions of individual rules or a changelog would be sufficient?

shadi: I think a version w/b useful, but we also need a separate process for more major changes
... at w3c we use dates as version
... I don't mind whether it's a version number, or a date, etc.
... creating a changelog is quite simple

wilco: what do you tink of rolling the versions in the changelog itself?

shadi: it can work. I think it's enough if there's a date. we can also auto-generate a diff
... we can simplify the auhoring with an automated diff service

wilco: so it seems we're dropping the version number section, and make it a changelog
... we want to remove the version number section, bake it in the changelog section, don't necessarily want an X.Y.Z schema, dates may be sufficient.

Accessibility support discussion

<Wilco> https://w3c.github.io/wcag-act/act-rules-format.html#structure-acc-supp

wilco: we put an editor's note in the last draft
... Alistair had concerns regarding if we even needed to address accessibility support in rules
... questions whether rules should be about theoretical accessibility or not
... I'm very much of doing something with a11y support
... whether the proposed solution is the right answer I really don't know
... but it's such an important topic, and historically hasn't really be addressed by tools, so it's important to consider it
... we sort of agreed to put it in our scope. do you still think we need to start addressing this?

shadi: is there an example test rule we can use as a base, to avoid hypothetical concerns one way or the other?
... if we can show what we mean by consdering a11y support, not depending on a specific tool, this can make the discussion more tangible

wilco: in auto-wcag we said we'd look at things that are accessible according to the spec; we try to avoid the problem to move forward

charu: there are 3 different aspects: user agent / AT / markup technology
... we're covering a11y support for all 3 of these aspects
... if the user agent or AT is not up to a level of support we file a bug
... if we add ARIA 1.1 and it fails with Firefox, it's up to them to fix that

wilco: sure, it's a problem at their end if they don't support it, but it's also a problem for the user of the software
... developers should know the kind of problem so that they can avoid it

charu: right, but not sure the rules should cover that. the rules are designed for a certain markup technology

<Sujasree> +1 to charu ... also, it will be tough to maintain that traceability over time

charu: otherwise it make it complicated if we put all that info in the rule format
... I don't know if we should cover the UA and AT support as well

shadi: I think this is exactly the kind of confusion that w/b solved with a specific example
... my recollection is that it's more focused on the technology level
... "this rule requres aria-describedby" so that the tool user can have a selection process to include/exclude all the rules that require ARIA support
... basically, have explicit knowledge about it so that people can enable tests or not
... I think many people, when they hear a11y support, think about hard-coding versions of AT and UA
... then the rule w/b obsolete when the technology is updated

<MoeKraft> I have to drop. My apologies. I have intern meeting shortly.

sujasree: I was exactly thinking like Charu
... the matrix can't use all the versions of the tools

wilco: what about the scenario that shadi describes, where we list the features that may impact a11y support, and then the users look at this list to figure out which tool / combination is compatible
... e.g. ths rule requires this aria 1.1 feature

sujasree: It would definitely be useful to have this information, but updating the support info is difficult

wilco: at Deque we keep a rough list of what is supported, we don't add rules for things that are not well supported
... ideally we add rules as new technologies come out
... it seems to me that being able to adopt that and identify this thing is useful

<Charu> +1 to qualifying the rule with feature technology

rdeltour: I like putting the info of the feature in the rule
... it makes me think of having something like caniuse for a11y

charu: ???

wilco: now that we have more frequent updates to spec, one thing we could consider is to ref the version of the spec and name the feature in the spec
... maybe you'd point to "ARIA 1.0 alert"
... "this rule tests ARIA 1.0 alert"
... that would provide a leightweight way to declare a11y support

charu: I would agree to that!

+1 from me too

<maryjom> +1

shadi: I'm just wondering whether we should rename it from a11y support to something less scary

<maryjom> +1 to Shadi.

shadi: "descriptors of a11y feature" or something like that
... and an example would be really helpful :)

wilco: maybe "accessibility features" might work


chris: I like the renaming.
... whithin the rule set, pulling from the ARIA specification, baking that into the rule set... is that something we're looking into?

wilco: I think that w/b quite complicated
... might be an implementation feature more than a rule feature

Availability for ACT TF Teleconferences - https://www.w3.org/2002/09/wbs/93339/availability/

wilco: we had very low attendance last week :)
... please fill the survey so that we know who to expect in our next calls
... thank you everyone, talk to you next week!

Summary of Action Items

Summary of Resolutions

[End of minutes]

Minutes formatted by David Booth's scribe.perl version 1.147 (CVS log)
$Date: 2017/06/12 15:50:05 $