– DRAFT –
Accessibility Conformance Testing Teleconference

<daniel-montalvo> Hi! Could someone please review act-rules/act-rules.github.io#2111 It is about removing an outdated accessibility support note about menuitem role

iFrames

<daniel-montalvo> Mark: I filed an issue about one of the examples of frame has non-empty accessible name

<daniel-montalvo> ... iFrames with role="presentation" the role="presentation" is still focusable in Firefox. It does not get ignore there

<Jean-Yves> github: act-rules/act-rules.github.io#2096

<daniel-montalvo> Mark: Originally Scott wanted to remove role="none" and role="presentation" from frames, but there was pushback from VoiceOVer. It is useful for VoiceOVer and Safari

<daniel-montalvo> ... In Chrome it ends up being treated as role="group"

<daniel-montalvo> ... The two that don't allow for scrolling if the frame is ignored

<daniel-montalvo> ... That is an accessibility issue

<daniel-montalvo> ... There are a lot of problems, not sure all of them are inapplicable

<daniel-montalvo> Wilco: We know there are browser specific cases.

<daniel-montalvo> Mark: One suggestion is leaving the applicability as-is but explaining it further

<daniel-montalvo> Jean-Yves: Currently the applicability has an exception that if the iframe is marked as decorative it has a role="presentation"

<daniel-montalvo> Mark: It is not clear that that exception in the applicability is because of these issues

<daniel-montalvo> JEan-Yves: We don't usually put the reasons in the applicability. Either we put them in the accessibility support or in the background

<daniel-montalvo> Mark: Accessibility support could say something to that effect. It is not very clear why the applicability has that exception

<daniel-montalvo> ... Is it because of accessibility support reasons or for other reasons?

<daniel-montalvo> Wilco: Is there an implementation issue too?

<daniel-montalvo> Mark: It came up when we tested it. Presentation role was not consistently ignored

<daniel-montalvo> ... There are problems with frames and it seems they will not get resolved anytime soon

<daniel-montalvo> Wilco: Resolitions -- We are going to add to the accessibility support section

<daniel-montalvo> Jean-Yves: We need to improve at linking accessibility support sections to applicability consideratoins

<daniel-montalvo> Wilco: We can make it an actual link

<daniel-montalvo> Jean-Yves: Yes. But even in the text when you read back the relation is not very clear

<daniel-montalvo> Mark: I'll open a PR with proposed solution

<daniel-montalvo> S/Resolitions/Resolution/

<Jean-Yves> Github: none

Deprecating Rules

<daniel-montalvo> Wilco: WCAG 2.2 is removing 4.1.1 as a whole. For WCAG 2.0 and 2.1 a Note is being added that explains that the SC automatically passes for HTML and XMLthis is not a failure

<daniel-montalvo> ... We could deprecate the rule and explain the reasons. Does that convince everybody?

<daniel-montalvo> Mark: The overall joy is to get rid of 4.1.1

<daniel-montalvo> Wilco: We are also thinking of deprecating the rule heading has non-empty accessible name

<daniel-montalvo> ... We tried secondary requirements. The Task Force considered that was not the right way to use secondary requirements

<daniel-montalvo> ... It's either a failure of 1.3.1 or, if it is not, it does not map to anything thus it may be better to deprecate the rule

<daniel-montalvo> Mark: 1.3.1 does not deal with visible pixels. I don't think this is affecting anything now

<daniel-montalvo> Jean-Yves: There is a conversation about the goal of secondary requirements

<daniel-montalvo> ... At the beginning it was OK to either fail or pass, now it seems in some cases they could pass and in other cases they should fail

<daniel-montalvo> ... And then we've also used them for stricter success criteria

<daniel-montalvo> ... Maybe goals a re not properly defined

<daniel-montalvo> Wilco: There be passed examples that would fail the stricter criteria

<daniel-montalvo> ... Color contrast enhanced has examples that pass contrast minumum

<daniel-montalvo> ... There are three categories: stricter SC, less stricter SC, or some overlap with other SCs

<daniel-montalvo> Wilco: Not making accessibility support a reason for moving things to secondary requirements has implications on audio and video rules. That is still a failure even in some cases you don't have the audio playing

<daniel-montalvo> Jean-Yves: At some point we should have a rule that form has a label to touch on 3.3.2 or 1.3.1

<daniel-montalvo> ... When we made the autoplay rule having only secondary requirements I opened an issue about that

<daniel-montalvo> ... If I don't have a label that is a failure of 3.3.2, if the label is not programatically then there is a 1.3.1 failure

<daniel-montalvo> Carlos: Shouldn't we have two rules?

<daniel-montalvo> Jean-Yes: The distinction for the 1.3.1 case is not that easy, you may not know if you have a label

<daniel-montalvo> Wilco: You'd expect at least one of these to be failed

<daniel-montalvo> ... I don't think secondary requirements lets us capture that

<daniel-montalvo> Jean-Yves: At present you need to at least report one SC -- sometimes it is not sure which one though

<daniel-montalvo> Carlos: If I have a methodology that tests only one of those, then I won't fail the examples that are failing the other one, or treat them as secondary requirements

<daniel-montalvo> Wilco: Yes

<daniel-montalvo> ... IF we want something like that it should be a different thing

<daniel-montalvo> ... We've had a similar discussion around 1.3.1 and 4.1.2

<daniel-montalvo> ... We ended up not going down that path at all, but we had that conversation

<daniel-montalvo> ... This opens up a lot of potential for inconsistency

<daniel-montalvo> ... I think I'd split this rule into two for more consistent testing of them

<daniel-montalvo> Jean-Yves: But then if we split which SC would we be failing?

<daniel-montalvo> Wilco: We just talked about undoing a decision we made in autoplaying audio. We will remove the SC that was under secondary requirements. Any concerns?

<daniel-montalvo> Mark: The autopaly one if you are using a screen reader you cannot find the content because of the noise

<daniel-montalvo> S/autopaly/autoplay/

<daniel-montalvo> ... This can do some harm, so failing seems reasonable

<daniel-montalvo> Daniel: Autoducking feature is a thing in screen reader. We may need to test for consistency of screen readers

BackWard Compatibility of ARIA

<Jean-Yves> w3c/aria#1990

<daniel-montalvo> Wilco: I raised this issue because the removal of aria-expanded being removed from a bunch of static roles

<daniel-montalvo> ... My expectation was that ARIA was deprecating uses of attributes (that was introduced in 1.2).

<daniel-montalvo> ... Now some uses were deprecated and others were removed

<daniel-montalvo> ... Backward compatibility and keeping clarity on how the spec is changing is very important, it gives us a place to reference

<daniel-montalvo> ... We can raise different issues for things that were recommended and are now removed and for things that were recommended but were not used at all

<Jem> w3c/aria#1990

<daniel-montalvo> JamesN: I understand there is a problem.

<daniel-montalvo> ... I'd like to know what you expect in terms of types of changes

<daniel-montalvo> ... Normative changes (mast to should) what should be doing? That's the same as removing an attribute from something

<daniel-montalvo> ... I propose to create a list of obsolete attributes in our spec, similarly to what HTML is doing

<daniel-montalvo> ... I propose to create a list of obsolete attributes in our spec

<daniel-montalvo> ... We previously had some attributes that we don't have a good replacement for

<daniel-montalvo> ... the 1.2 ones are because they were a specific use cases, and we needed to work around some technical issues in the implementations

<daniel-montalvo> ... If an attribute is not allowed it should not be exposed, whereas these global ones were always exposed

<Zakim> jcraig, you wanted to suggest there could also be aria issues to track changes (at least filing) of issues in implementations and maybe tools

<daniel-montalvo> ... It was heavy work to remove them when they were not needed

<daniel-montalvo> JamesC: Removal and deprecation is big enough of a step. We could work on an issue tracker that we could use to point to some implementation issues

<daniel-montalvo> ... That would help not implementing things that are either removed or obsoleted

<daniel-montalvo> ... That could happen even before we even deprecate or remove it from the specs

<daniel-montalvo> ... There are templates that Valerie has that we could use

<daniel-montalvo> ... Also, maybe we could have some author errors be flagged by browser tools

<daniel-montalvo> ... For example, images without an alt attribute

<daniel-montalvo> ... Some of these obsolete attributes can also flag these types of errors

<daniel-montalvo> Wilco: Having a maintained list out of the spec might do that. Preference would be for it to be in the spec though

<daniel-montalvo> ... We could then write ACT rules for things that are obsoleted or deprecated

<daniel-montalvo> ... We should have expectations for authors and user agents on what to od with obsoleted and deprecated roles

<daniel-montalvo> JamesN: When you say list, is it a list of attributes that are no longer supported or you are talking also about normative requirements about child roles for example

<daniel-montalvo> ... The first we could do, the second is just the changelog

<daniel-montalvo> Wilco: First one

<Zakim> jamesn, you wanted to react to Wilco

<daniel-montalvo> JamesN: If there was a label that we put in our GitHub that could automatically be pulled in the changes of this document, would that be acceptable?

<daniel-montalvo> wilco: Yes

<daniel-montalvo> JamesN: Seems reasonable, HTML does it. We should not get into the details as to why, just have the list

<daniel-montalvo> ... We should not go backwards retroactively, that is more for going forward

<daniel-montalvo> Wilco: I volunteer to go back to track previous changes

<daniel-montalvo> Jean-Yves: I think it's good to have these lists of changes

<daniel-montalvo> ... I just care about one ARIA version, but still understand rationale

<daniel-montalvo> ... I spend time looking for changes that I need to implement

Updates on ACT

<daniel-montalvo> Wilco: We have gone through your feedback on ARIA ACT, that is now completed

<daniel-montalvo> ... We need to reorganise some of the rules. We are almost at a point where we can go for a pass

<daniel-montalvo> JamesN: Ask for feedback, if you don't get any, that's your feedback

<Jem> is it template level?

<daniel-montalvo> Wilco: A lot of us open issues on ARIA. Any feedback on things we can do to collaborate better?

<daniel-montalvo> JamesN: I have not noticed any issues. If you aren't getting responses in the timelines you ned, reach back to us

<daniel-montalvo> ... Sometimes a draft PR can be helpful at least to move the issue forward. Not saying we'll accept it, but at least it gives us an idea on how the issue could be resolved

<daniel-montalvo> JamesC: If it is a giant PR, let's first discuss. Smaller PRs are easier

<daniel-montalvo> Jemma: What does reorganizationof ACT mean?

<Zakim> Jem, you wanted to ask the detail of "reorg"

<daniel-montalvo> Wilco: We split one of the rules up. We took all of the id referencing stuff and put it into its own rule, explicitly looking at what id reference requires the element to the part of the page that exists, for example references to error messages

<Zakim> Jean-Yves, you wanted to mention JSON data of ARIA

<daniel-montalvo> Jean-Yves: All require context information I use to need to access it programmatically. For example an ARIA JSON

<daniel-montalvo> JamesN: That's in our repo.

<daniel-montalvo> ... If you run any Respec version that will show it to you

<daniel-montalvo> ... We only regenerate when we publish a new version, it still is 1.2

<Wilco> https://www.w3.org/WAI/standards-guidelines/act/rules/

<Zakim> jamesn, you wanted to talk a little about required context role rule

End time on Friday

Wilco: How long do we want to go for tomorrow?
… I might need to leave before six

Jean-Yves: We can end the agenda at 6pm and if someone wants to stay in the room they can
… and perhaps we can start at 2pm

Wilco: I'll update the agenda

Subjective applicability (Trevor)

<trevor> Discussion: https://github.com/act-rules/act-rules.github.io/discussions/2061

<Jean-Yves> w3c/wcag-act#539

trevor: In the current rules format we have an objective applicability requirement
… with subjective applicability we can have situations where the applicability scope is too broad (e.g. non-text content)
… in the current rules we do have similar situations where parts of the applicability are moved to the expectation
… for example the text has minimum contrast rule which has an "except if the element is..."
… so we ended up hacking our rules format
… other issue is that we could not write some rules
… for example "this rules applies to elements styled as an heading" which we couldn't objectively define

<Wilco> https://github.com/w3c/wcag-act/pull/539/files

trevor: The first major change proposed in the PR is allowing subjective forms in the applicability
… these subjective forms are ways to capture subjectivity that we have used in our rules
… in each of the forms we start with an objective target and then we apply the subjectivity on top of it
… The second part of the changes is the requirement to define the subjectivity in a glossary
… this aims to have a group consensus preventing individuals to define subjectivity by themselves

Jean-Yves: In the discussion we converged to requiring subjectivity to be expressed in a standard format
… But if we need to update the formats of the subjectivity in the future we might not want to have this in the rules format for flexibility
… similar to what we have with the common input formats

trevor: In the TF we agreed that we should have it in a note or similar

daniel-montalvo: I agree with that approach
… otherwise this would need to go through AGWG at least

Wilco: This idea feels like extra paperwork to remind us that we should write good rules
… I'm not sure there is extra value in this

Jean-Yves: Compared to what? Not having a list of allowed formats?

Wilco: I'm not sure that would be the solution either.

trevor: If we could all collectively agree that we are going to write good rules, perhaps we don't need to express this in the rules format
… would we be comfortable with just updating best practices documents?

thbrunet: If we require subjective things to be defined we could get away without having the required subjective form

Jean-Yves: I like having some sort of gate keeping
… we do not always agree with how things need to be formulated
… what is too much or enough subjectivity?
… Do we want to discuss that every time we review a rule?

MarkRogers: What about how automated tools would deal with subjectivity?

Jean-Yves: That is something we might discuss later... but agree that objective is automated, and we should strive to have most objective rules

Helen_: Agree that is creates more overhead, but it also better supports traceability of the decisions we take when writing a rule

Wilco: Maybe we could put some constraints around what kind of subjectivity is appropriate
… I earlier advocated for the creation of a list of approved examples
… which we could update whenever we find another example that should be considered
… this way we could list aspects that a tester could examine

trevor: In the PR we have the requirement that when you use a subjective form you need to list what characteristics need to be matched
… We could use other alternatives, such as using logic operators to organise the characteristics
… another alternative is using scoring, where alternatives would have points and you need to reach a gola score
… yet other is specifying characteristics that the element needs to have without expressing a formula or score

thbrunet: The more I read this it feels like were trying to make something subjective into objective

trevor: My goal is we have something more concrete for implementors justifying why they are classifying something as applicable

thbrunet: We all have our nuances, as long it matches the fails and passes... I don't think we need to build classifiers

trevor: Would you be in favor of including concrete examples as part of the definitions?

thbrunet: yes

Jean-Yves: I also like the "getting warmer" method
… possibly with "getting colder" examples also

Jean-Yves: We should try to reduce the space for interpretation to be as small as possible

Jean-Yves: The examples are good for creating boundaries and promote discussions, as we already do in the rules

trevor: Do you still like the subjective forms or the definitions are enough?

Jean-Yves: They go together, and I feel we should have both

Jean-Yves: For the subjective definition we will need examples

Wilco: This impacts manual evaluation, so I would like to know from manual testers
… I like the simplicity of the getting warmer approach
… it closes the definition, anything that does not have the specified characteristics is not what the definition applies to
… I would like to have a new term such as "closed subjective" to distinguish from what we already have

kathy: I'm leading towards the logical approach
… when would a tester know they are getting hot?
… the logical operators method is mostly tied to how we define a concept
… in the getting warmer how do we know how many of the characteristics need to be present?

trevor: Would it help if we had examples stating what characteristics are met in each example?

kathy: That seems what is described in the logical approach

kathy: Nevertheless, examples will all be helpful

MarkRogers_: The logical approach could be biased towards headings and western characters
… the getting warmer is more extensible

Wilco: The alternatives need not be exclusive
… we could use different alternatives for different instances

Jean-Yves: We may need to have a format as a guideline to writing close subjective definitions

Jean-Yves: We can mix the logical and getting wormer saying something like "the concept usually has at least two of these characteristics"

<Wilco> +1 that feels like a great middle-ground

Jean-Yves: But it's good to have the ability to express something logically when needed, and be more flexible when not possible

Helen_: The more information I give testers the more they cannot make decisions for themselves
… a simpler definition seems to be better

<MarkRogers_> Headings work a bit differently in Japanese: https://www.w3.org/TR/jlreq/?lang=en#hanmen-design

suji: I like the combination of getting warmer and logical

CarlosD: The flexibility of having the possibility of having logical when possible and getting warmer when not is a positive for me

Helen_: If we are too prescriptive we will need to be exhaustive, otherwise the chances that people mess up when applying the definition will grow

Wilco: Passing a rule does not meaning meeting a success criteria, so having definitions that restrain what we test is not a problem per se, as long as we can keep expanding what we are able to test

enrico_: I like the getting warmer too
… but for each characteristic the text might not also be objective, so we should also explain the subjectivity in the characteristics listing
… giving examples would be very helpful

Jean-Yves: Agree that we should make as explicit as possible what defines each characteristic

enrico_: do we need to specify weights for the characteristics?

Jean-Yves: I don't think we will be able to find the correct weights

Jean-Yves: but nothing prevents implementers to define their own weights

Wilco: We have rules where we explicitly exclude things because they are edge cases and we are not sure how to handle them

Wilco: We are coming to an agreement, with Helen being an outlier
… I would like to talk to Helen later

MarkRogers_: In the CJK domain headings work differently
… so coming up with a set of conditions that work on everything might be difficult

Wilco: We don't need to make it work on everything

Jean-Yves: We can specify to what it applies

trevor: I want to revisit the subjective forms to see if they are still needed or how they can be improved
… and I have a list of points to work on
… like exploring the concept of closed subjectivity

Wilco: Any further comments?

Helen_: Having too many points makes it hard to review and hard to read

Refining secondary requirements (Kathy)

kathy: we started talking about secondary reauirements. Requirements that are related to a rule, but not specifically tested by id.
… they are often reported in implementation.
… Draft format to add secondary requirements to rules

<kathy> https://w3c.github.io/wcag-act/act-rules-format.html#outcome-mapping

<kathy> https://github.com/w3c/wcag-act/pull/542/files#diff-4f84b10f4fa5880bac20c3a8b3e31671a67e8bfce9b5d8e67519b1c513f1ed6bR265 - update for how to provide explanation

<kathy> https://github.com/w3c/wcag-act/pull/540/files - additional update for accessibility support

<daniel-montalvo> S/reauirements. Requirements/requirements. Requirements/

<kathy> https://github.com/act-rules/act-rules.github.io/discussions/2095 - discussion and decision to not include accessibility support

<kathy> https://w3c.github.io/wcag-act/act-rules-format.html#outcome-mapping

kathy: [show draft rule format, Section 4.4.1]
… this is where we have the categories for secondary requirements (4.4.1.1)
… there are 4 scenarios included for secondary requirements, with examples of rules that would use them.
… we require information has to why the requirement is secondary, currently in the Background section.

Wilco: we want implementers to report the "correct" SC to claim an implementation. But that is not always possible. With 2ndary, we can say "this must be reported, this may be, this may not be"

<kathy> https://github.com/w3c/wcag-act/pull/542/files#diff-4f84b10f4fa5880bac20c3a8b3e31671a67e8bfce9b5d8e67519b1c513f1ed6bR265

<kathy> act-rules/act-rules.github.io#2060

kathy: The PR adds some explanations. The reason for being secondary is moved to the Accessibility Mappings section, together with the requirement.
… two of the scenarios are merged (down to 3 scenarios).

<kathy> https://www.w3.org/WAI/standards-guidelines/act/rules/09o5cg/

kathy: Some rules alredy have that change, e.g. "text has enhanced contrast".

Jean-Yves: I like the fact that the scenarios are now named.

Wilco: I think this goes in the right direction.
… Do we really need to capitalize "Secondary"?

kathy: No real reason.

Wilco: This is a defined term, we can link the first used (in each paragraph) to the definition.
… We decided not to use secondary requirements for Accessibility Supports

kathy: that's in another PR.

Jean-Yves: I think we can keep the example of the removed scenario, because it it a fairly different use case of the same scenario.

kathy: the other PR is to explicit that 2ndary requirements are not meant to cover accessibility support differences.

<kathy> https://github.com/act-rules/act-rules.github.io/discussions/2095

<kathy> https://github.com/w3c/wcag-act/pull/540/files

[discussions around the precise wording of should/must]

Wilco: Do need to change something about where the explanation is?

kathy: That's part of the #542 PR.

Wilco: We'll merge as soon as this is done, and vote on the full draft.

Defining consistent / partially consistent (Wilco)

Wilco: we want to add a formal definition of (partial) consistency in the rule format

<Wilco> https://github.com/w3c/wcag-act/pull/541/files

Wilco: we had a concept of "minimally consistent", we have evolved from that.
… We don't need the definition to be in the format (can be just an internal TF def).
… This is a fairly complex definition. I tried to start from the simple case and expand from here.

trevor: I get a lot of confusion due to mixing "implementation" and "implementation procedure".
… my understanding is that a procedure only has one outcome; the implementation as a whole can be consistent depending on the union of outcomes of its procedures.
… i think it would help me to see how procedures are aggregated into an implementation from the start.

Wilco: I think this would put the complexity upfront and might not be that readable.
… I will try something, need to think how to do it.

thbrunet_: I think 'failed' should take precedence over 'untested'.

Wilco: Good point.

trevor: how can a procedure report several outcomes?

Wilco: e.g. an example with text both on good and bad contrast, a color contrast procedure could report both passed and failed.

CarlosD: I'm also confused by the difference between implementation and procedure

CarlosD: the first definition applies to implementation, the second one mention procedure.
… it may be easier to define what is an implementation, and how procedures results map to implementation result.
… I think it will be easier if we have the mapping to implementations first.

Wilco: this is defined with union of outcomes.

Wilco: we can make it more explicit

suji: do we also want Inapplicable test cases to satisfy the requirements?

Wilco: yes, because it is the same "not a problem" bucket.

Wilco: take e.g. a rule checking that HTML page has a language, an inapplicable example showing a PDF could have no lang, but then a procedure that also checks PDF would fail it and not be able to claim a consistent implementation.

Jean-Yves: it could help to have example of procedure/implementation/rules Might be to big, however

<Zakim> daniel-montalvo, you wanted to ask for definition of implementation #598

daniel-montalvo: I think the implementation could be the set of outcomes
… line 598

Wilco: implementation produces outcome for a page.

Wilco: "implementation procedure" = "rule inside my tool" != "ACT rule"

Jean-Yves: could be "check"

Wilco: this adds a new requirement to rules, not backward compatible. Do we want this?
… we can record the version (of the format, in the rule) to be backward compatible.
… all current rules actually meet that requirement
… we can easily record the rule format in the metadata.

kathy: failed cases do not need to fail all secondary requirements

first public working draft

Wilco: for subjective requirements, we are not ready to go public yet.
… secondary requirements looks ready-ish
… implementation is also pretty close.
… want to take that to AG and a first working draft

Jean-Yves: we can make a join TF/CG call to present it when ready.