Accessibility Conformance Testing Teleconference

Accessibility Support

<Wilco> https://www.w3.org/TR/act-rules-format/#structure-acc-supp

wilco: my idea is to have. some sort of way to deal with a11y support in rules
... currently some orgs don't deal with a11y support at all, which is a downside
... we shouldn't miss the opportunity to miss that here
... some comments was received on the prose, it's time to open that discussion again
... my question for the group is: do we want to take this back and use it to push a11y support?

maryjom: it's a touchy subject
...
... you can't test with every screen readers, etc. how do you know you have full a11y support?

wilco: you can't. I don't know what full a11y support even means
... for whatever you're testing, you set a baseline of what you want to support
... say, for a website, you say you want to support the latest version of VO, NVDA, JAWS
... by setting a support level, you can decide what rule you want to use

maryjom: if you come up accross issues, you don't know if it's a bug in your code, in AT. It takes time to figure this out

wilco: it does put the burden on the developer
... you indicate that you support an AT and you may have to work around bugs in AT in your code

maryjom: it's hard to maintain, e.g. to follow screen reader changes. it's not always easy in practice

wilco: right. it seems to me nobody has a handle on a11y support at this point

agarrison: my commnent is that it's a very simplistic thing that replicates what you right for the test anyway
... on example 1, that's the selector you have in the test anyway. why replicate it in the a11y support section?
... same for example 2, it's redundant with the test
... a second comment: I don't think we should look at a11y support within our tests

<MoeKraft> https://w3c.github.io/wcag-act/act-rules-format.html#structure-acc-supp

agarrison: this is something you're interested in when selecting the technique
... then you use the tests to validate the technique
... the a11y support decision is all made in a higher level, in the technique selection
... keeping the technique up to date is another discussion
... I don't see how a11y support is important at the test level

wilco: I'm confused about what you mean by "at the technique level"

agarrison: [further describes with a detailed table example]

wilco: for tables, would you add 1 rule that addresses scope, 1 rule to address headers, 1 rule that address scope and headers?

agarrison: it's their choice wheather to use headers and IDs or scopes, it's the dev's choice how to build their stuff
... we don't care about their choices, we build the rules to test them all

wilco: say for our organisation, we use headers and IDs, not scopes
... a developer comes along and is only using scope, the rule would fail?

agarrison: what you do is provide a set of best practices, and then select some tests to support these best practices
... it's how it's done in all the organisations I've worked in

wilco: then you need rules for these best practices

agarrison: if you tell people you're only using headers and IDs, you write a test only for header and IDs

<MoeKraft> https://www.w3.org/TR/WCAG20/#accessibility-supporteddef

wilco: the problem is that there are things in Techniques which are not a11y supported

agarrison: [rephrases the comments]

wilco: so you're delegating that to people who write the techniques?

agarrison: or the people who select the techniques

rdeltour: I think I understand Alistair's point, but IMO the a11y support info is useful to select the test of filter the steps in the test?

moekraft: the rules themselves should be testing techniques that are a11y supported
... if you're using a technique, it's a11y supported
... the techniques themselves are supporting some AT
... there would be a level of redundancy if we add a11y support to the tests
... I think we need to call it out, but maybe not included it in every rule
... if someone relies on a11y support in test, it may not be implemented properly

<Wilco> Only accessibility-supported ways of using technologies are relied upon to satisfy the success criteria. Any information or functionality that is provided in a way that is not accessibility supported is also available in a way that is accessibility supported. (See Understanding accessibility support.)

moekraft: a screen reader can have the ability to override bad code

wilco: my understanding of the text abovie: you're only allowed to say you pass SC if the way you've done it works for AT
... I think we should provide information within the rule that allows people using the rule to figure out whether using the rule [???]
... the way I would write my table rule w/b: you write several steps
... 1. one uses scopes, 2. uses haeders+IDs
... some mechanism allows users to say "we have this context, so we can turn off step 1, or 2, or we need both"
... what is a11y supported depends on what your baseline is
... your baseline is different whether you're targetting macOS users only, or also mobile users with Android

agarrison: you would only need to include a support baseline thing in tests if the people writing the test wouldn't have already selected a technique already
... a user wrote a site without selecting the technique, then run the test, then the test can inform on the a11y support
... anyone else would be using best practices to follow for the target baseline
... who would write an entire product without selecting best practices first?

wilco: I'm trying to make the rules work in any situation

agarrison: they can select the technique they want
... I don't see us saying "you selected a technique we don't support"
... I don't see why this is so important at the test level

wilco: we want to be able to reuse the use accross different organizations and people
... let's take our table rule again
... if we (ACT TF) publish a rule on tables
... if an organization has to support Android an Talkback
... they have to tests other things than an organization that just supports Windows and macOS
... our options are: either we write multiple tests for these differenet organisation, or write 1 test that can be configured to fit the needs of these orgnanizations

agarrison: I think we should be going for the most atomic tests

moekraft: you may have something that VO doesn't support today but something that is a11y supported
... we need something to select the appropriate rule for a situation
... I see where you're coming from wilco, but sometimes it can be a11y supported and not on a particular AT
... we need to ensure that the techniques that we test for themselves are a11y supported
... and I do agree with Allistair that we should aim for smaller tests

wilco: what is the alternative to putting this complexity to the rules?
... Allistair you're talking about keeping the rules small. for the table example, you'd write several rules?

agarrison: it's not me, but every recommendation on writing tests recommends being atomic
... then you use a selector to select whether it's applicable or not

wilco: no disagreement from me! but what would you do for the table example?

agarrison: in our organisation, we have different tests

wilco: so you need to aggregate them?

agarrison: not when running them, but you could aggregate the results in a higher level
... you don't do the aggregation at the unit-test level, but for the whole test runner
... our tests should be just finding fail/pass, not an aggregator result for best practices

wilco: in a lot of rules you can use one of any number of techniques to pass.

agarrison: so if a rule fails, it doesn't necessarily means that a SC fails, since another rule can pass?
... what you want for test is to produce a result, and can be reused
... I'm a bit reticent about writing super-tests that run smaller tests, it's more a platform kind of thing

moekraft: we do have a separate mobile ruleset, which is a subset of our web ruleset
... some rules make sense on our platforms and not others
... if you have smaller rules, if one pass and the other fails, how do you come up with a summary of whether the SC passes?

wilco: the idea is that if all of them fail, the SC fails, and if any of them pass, the SC passes
... should we introduce the concept of a group?
... if a rule belongs to a group you need only one of them to pass

agarrison: if you only look at the failing tests, it doesn't matter whether you implement things in one way or another
... ... in LevelAccess (SSB), we test for all sort of things and just flag the things that fail

wilco: and if you have a group, you have a way to say that one of them passed so it's OK?

agarrison: one issue with WCAG is that it's difficult to say that something passed, but you can say when it fails

wilco: we're making progress. I'll take on myself to write a proposal with the idea of groups

moekraft: I was thinking labelling can be another example to look at

<Wilco> https://wilcofiers.github.io/act-rules/

wilco: I've been working on the rules to publish along our next draft [links not working currently]
... thanks everyone, let's talk next week!