Success Criteria vs. Checkpoints

Parent document: https://www.w3.org/WAI/EO/wiki/Role-Based_Decision_Tree

Success Criterion vs. Checkpoint Ownership

Issue ownership is very specific to the context involved. Depending on how specifically they're written checkpoints may be owned by roles that are not necessarily the same as the success criterion they address.

Simple Checkpoint - Domain Knowledge

For example a checkpoint for SC1.1.1 may be read "Confirm all <img>s have alt attributes." This is so specific to the implementation the only role that can own it is the developer since only that role could have made the omission.

The developer should have known this for several reasons:

HTML Standards: alt attributes are required by the HTML specification
Coding Standards: code validation should have revealed this as part of development best practices (especially to meet SC4.1.1 Parsing)
Accessibility Standards: alt attributes are critical to proper interpretation of <img>s by assistive technology

The first two points are role-specific domain knowledge that a professional HTML developer should already have. The last is important for accessibility but should be unnecessary given the first two points.

Simple Checkpoint - Domain Knowledge, Accessibility or Requirements

Another checkpoint might read "Confirm all decorative <img>s must has have alt="" so they are not announced." If the previous example does not already apply (and there is an alt attribute with text) then ownership of the checkpoint also probably belongs to the developer during testing. The context of testing is very important in this case.

The developer may have made implemented the code this way for different reasons:

Misinterpreting HTML Standards: thinking all alt attributes must have non-empty text values
Misunderstanding Accessibility Standards: not realizing that decorative images should not be announced by AT
Explicit specification for text to be included: incorrect requirements or specifications

The first point may come from not knowing HTML standards and that empty strings are valid. It could come from not knowing what decorative images are and how <img>s should implemented for them. But this is all based upon what a professional front-end developer should already know.

The second only extends and emphasizes this since it directly affects AT users. As developers become aware of the impact of alt text they should be more aware of the options and, if necessary, be able to identify decorative <img>s and implement them correctly.

Decisions from Non-Developers

It's the last point which points to other factors beyond the developer's role and responsibility. If the developer was explicitly told to include that text then there may be a requirement that the tester didn't not know - though this is unlikely. It suggests that one of the designer roles or content author does not understand the impact of adding unnecessary alt text and explicitly directed the developer to include. The developer was only following the requirements provided and did not (or could not) argue to change them.

In cases like this the issue is not implementation but design decisions. It's likely that the other roles are not aware of the impact of their choices on accessibility and need supplemental accessibility training. Or it may be that the tester is unaware of the importance of what appears to be decorative content. Whatever the case the other roles must be involved to determine next steps.

Shift Left - Early Accessibility Testing

Situations like this not only point to the need to educate other roles about their impact on accessibility but how these decisions occur early in the development lifecycle. Whenever the requirements (in a use case or style guide) were created was the time to test and evaluate their accessibility.

Once defined in an accessible way the specifications should not require further testing help ensure higher conformance of the resulting product. Designing-in accessibility at earlier stages in the lifecycle also reduces cost in testing and rework - especially the later it's found - whether during initial QA testing or once found in production.

Success Criteria and Design Decisions

The last example shows how design decisions can negatively impact accessibility. It underscores that all roles making feature, interaction, presentation, content or coding choices need to understand about accessibility and take responsibility for their decisions. It also leads to a need change context from simply testing for accessibility as an extension of QA testing but design in general.

In the decorative text example what should be done to reduce failures and design-in more accessible solutions is to recognize who is the ultimate owner of the issue. Decorative content, as a part of of SC1.1.1 Non-Text Content tends to focus on static images. As a result the primary owner would be the visual designer since images are one of their core presentation responsibilities. In the example decorative content might have been clearly marked in a style guide. The UX designer likely had significant input to the initial structures of the site and guide and other decisions but the final choices are not part of that role. Marking content decorative would tell the content author that there is no need to provide text for the developer to implement. And, as covered in the examples, the developer should already know appropriate methods to implement decorative images so they are ignored by AT.

Criterion Complexity and Ownership

SC1.1.1 is a very complex and lengthy success criterion. The many situations, options and sub-cases can shift some responsibilities. For example, in complex images (SC1.1.1, Situation B: Short Description Not Useful) the visual designer may put a lot of effort into designing a chart or illustration but the choice of techniques to communicate that to all users likely starts with the UX designer. Once the technique is chosen it may be up to the content author to provide the text that effectively describes it. With these fully defined in product requirements developers should have clear directions to implement a solution that conforms.

Situation Context and Roles

In case where the non-text content is a control or input widget (SC1.1.1, Situation C) the developer may more directly involved in the design process depending on how well the UX designers understands the ARIA requirements of their designs. In more unusual cases such as images used testing (SC1.1.1, Situation D) the crafting of the text by the content author could be far more important to avoid invalidating the result.

In each of these situations different roles have different levels of impact on the design decisions. At the criterion level they may not be enough to take impact the final say of the visual designer since even if the UX designer or content author are highly active in choosing approaches, the result is likely driven and conformance defined in the final presentation.

Checkpoint Ownership

As with situations and other sub-cases for criteria, how checkpoints are written to test them also play a factor in their ownership. Primary ownership for SC1.1.1 may rest with the visual designer but that doesn't mean the many checkpoints it generates are all owned by that role. A lot depends on how the checkpoint is written.

In the missing alt text example all issues were about implementation - a issue where the visual designer has little input. The same applies to the situation context and role examples where complex image checkpoints are more likely to be owned by the UX designer (if they describe the approach) or content author (if the text is not effective).

Success Criterion vs. Checkpoint Ownership - First Draft (for Reference)

The examples are derived from checkpoints and testing of existing sites. When looking at the ownership of success criteria the same process applies but, because the focus is much broader with more overlap extra consideration is required. Some criteria are more narrow in focus while others are broader with multiple situations, variations or other sub-cases.

Another facet is the level context of ownership through the development lifecycle. At higher levels, "ultimate owner" or "final decision maker" may differ from those at lower levels or earlier stages in design, development and testing.

Simple Criterion Ownership Cases

Main Actors: Developer and Author

Instances of simple ownership and context is SC3.1.1 Language of Page and SC3.1.2 Language of Parts. SC3.1.1 is very concise focusing on one specific element of a page. SC3.1.2 covers pieces of the page which may be comment elements (such as links between different versions of the content) or portions of content (such quotes and translations sections).

The focus of both is on identifying the "human language" of the page. In regard context the expert on human language is the content author. But the implementation is also critical as are the context for each so the developer role is also involved to provide the appropriate metadata.

Interested Parties: UX Designer

At the page level (SC3.1.1) there is little the UX designer impacts. If the site is multi-lingual it's up to the UX designer to define how visitors will navigate between them by providing links which may be presented in the destination site's language. The UX designer may define standards for how and where additional alternate language content will presented or if it's included at all. But these will be parts of the page (SC3.1.2) not at the page level which is a different context.

Dis-interested Parties: VX Designer, Business Owner

Language of page or parts are not a presentation issue so the visual designer is not involved at all other than to choose a different typeface. Other than an providing the original requirement to support for multiple languages (if there is one) the business owner will rely on the other roles to provide a solution with no additional input.

SC3.1.1 Basic Implementation Ownership

At the page level SC3.1.1 is easy to assign primary ownership to the developer. To satisfy the criterion requires adding the metadata. For mono-lingual sites the developer is usually fully aware of the language and probably the variation if applicable (such as "en-us" vs. "en-uk," or "fr-ca" vs "fr-fr"). Adding a lang attribute to a common page template is all that is required and solely in HTML code. There is rarely anything more to design or discuss. In these cases the content author has little to do add but the content itself.

SC3.1.1 Basic Implementation During Testing

During testing, checkpoints focus on whether there is an lang attribute in the <html> tag and that it has the appropriate value for the page content. The typical developer in the US would (or should have) default that to lang="en" or lang="en-us" to satisfy SC3.1.1. If there is no lang attribute this is clearly an implementation issue. If the lang attribute is present but its value is the same on all pages, even when they are in another language, because it's a not a variable matching that content it would likely still be the developer's responsibility to address it.

SC3.1.1 Complex Implementation Primary Ownership: Developer

On a multi-lingual site developers cannot use a single static value for lang. They must provide more infrastructure, probably by including a content management system, to select the content for the selected language of the page. Content authors use that system to enter the text and identify the language at the page level for standard elements and major sections of content.

If a checkpoints fails in this case (because the lang attribute doesn't match the page's content) it typically is the developer's responsibility on the server side due to non-support or error. If not, the author's error may have not specified the correct language code or entered the wrong content.

Overall though, at the page-level scope of SC3.1.1 it's much more likely to be fundamental issue implementation of page creation by the developer than the author. When everything is confirmed to be correctly implemented there will likely be few failures at all.

SC3.1.2 Ownership

With SC3.1.2 Language of Parts the context changes and so does ownership. For SC3.1.2 to even apply there has to be some change from the default language of the page. By definition these are exceptions to SC3.1.1 and apply to any and all instances. The causes of these will come from sources other than the developer.

As with SC3.1.1, checkpoints that solely focus on missing lang attribute to specifying language changes then it is (likely) the developer's responsibility if the infrastructure doesn't support it. But if it is supported, though developer may be able to identify the start and end of the transition but probably will need to consult with the author to get the code or edits needed to fully remediate any issues.

SC3.1.2 Primary Ownership: Content Author

Unlike SC3.1.1 the complexity of a multi-lingual page shifts from the developer to the content author. The reason is as mentioned above, the author role is the subject matter export on human language. Given input from the UX designer to support other languages the authorship responsibilities increase. They still provide the default language code and content.

The critical difference is identifying and specifying language transitions in body copy. Simple transitions (links between different language pages) require expertise in translation and providing the appropriate language code to developers who may not know them. These are usually very simple to address but the expertise of the author role become more important and cannot be assumed to be known by developers.

If sections of page content are in different languages are also implemented it's up to the authors and translators to identify the start and end of the transitions and provide the codes. Site infrastructure for this (such as a Content Management System or CMS) will generate the code automatically once properly configured. It's the responsibility of the author (translator) when creating that content to use those tools to mark them.

Since this is core to writing the body copy it cannot be delegated to the developer. Even if these cases are limited (such as a page of legal disclaimers in multiple languages) the author must provide these for coding. As a result the relatively simple implementation requirements are less significant than correctly translated text, presentation details (such as right-to-left changes), and language codes.

Secondary Owner: UX Designer

As previously discussed, UX designers are a secondary owner defining where different languages appear on the page as part of navigation and content. As a result this role has a significant impact specifying these changes for the developer and requesting appropriate translations from authors.

Secondary Owner: Developer

Unless the content variations are limited to a short section of content on one or just a few pages the solution required to support SC3.1.1 must support language changes in the page. As a result the developer's solution will be critical to support them. This maybe simple in the case of a few links between different language versions of a site. It can significantly more the intent is to include content sections in multiple languages on a single page.

The developer may nearly be primary owner because of the details of delivering code marking transitions but the expertise of having correct translations and language codes is more critical to meeting SC3.1.2 - especially when dealing with unfamiliar languages.

Criterion Ownership in Requirement and Design Stages

Much of the discussion above is about testing, checkpoints related to success criteria applied to an existing site or feature. When designing a new site or feature to add ownership can also change.

Shift Left Savings

Adding and reviewing WCAG success criteria review when defining requirements and designing pages can greatly reduce the chances of implementation issues. Adding and testing accessibility requirements early in the lifecycle can produce significant savings in time and resources that would lost having to remediate and re-implement issues identified during QA review (or in production).

In this case of these criteria specifying which languages will be supported, the correct language codes and providing them to developers should accelerate their work. As explicit requirements quality assurance, not accessibility testers should be able to more easily create test cases to validate the work.

SC3.1.1/SC3.1.2 Examples

In the discussion of these criteria the emphasis has been on content author and developer. Making decisions impacting accessibility earlier means they shift much of the ownership from developers and implementation to the other roles.

In simple cases like SC3.1.1 specifying the site will be (only in) English must include lang="en" at the start should ensure that developers implement it correctly the first time. If the site is multi-lingual, specifying the number of languages (and their codes) and the extent of the transitions in wireframes by the UX designer provides clear guidance for both the author and developer.

In the case of frequent language transitions (SC3.1.2), documenting the delivered site content in turn will help specify the requirements needed in the infrastructure to support it. If it restricted to a few links between single-language pages this may be easily handled in page templates and a few language codes. If multi-language body copy is common then the need for a CMS that supports them will be identified and factored into product planning.

Complex Criterion Ownership Cases

In comparison, SC1.1.1 Non-Text Content has several situations and sub-cases (such as complex, decorative and other image types). Depending on each sub-case the ownership may vary. If any sub-case is owned by a role it should be at least an secondary owner. Choice of primary owner should come from the secondary owner based upon "who ultimately decides" the majority of design questions. For SC1.1.1 this leans to the visual designer since the criterion is visual content and the assumption is that they have the final approval of the final design.

All the other roles may make key decisions and have critical input for SC1.1.1. But focus of this criterion is on still visual presentation. The visual designer makes the critical decisions that determine which sub-cases apply and those that do not. As a result issues rising with implementation, content, and operation of the visual ultimately come back to this role.