Evaluation tools

Planning

We are updating the Evaluation tools database as part of the WAI-ACT project. In addition to updating the content, we would like to improve the usability and usefulness of the information. We also want to design it for easier maintenance and longevity.

A first step is to analyze and document the users and use cases for the Eval tools database.

User stories

The following stories describe a range of potential tasks and users.
(These are for internal use so they don't need to be polished, just get the point across clearly.) Please have a look through these and comment if you can suggest any additional user stories or spot anything wrong with the ones we have.

Check templates on a local machine

A template developer needs to check several new site templates for accessibility and valid code,while they are stil on his own computer.

comment {name}

Check browser compatibility

A front-end developer with a new website design, needs to check it for browser compatibility across a wide range of browsers and versions to ensure that CSS is producing accessible visual output.

comment {name}

Check an entire website automatically

The web manager of an established website needs to have it's level of accessibility established, at least as far as automated evaluation can determine.

comment {name}

Learn about accessibility issues on a single page

Web technology students need tools to do automated single page checks and that provide links to WAI resources so that they can learn by doing web accessibility on their own web pages.

comment {name}

Illustrate accessibility issues on a single web page

A web accessibility trainer wants to give an on-screen illustration of the range of accessibility issues on a single web page (could be a prepared page or a page on corporate delegates' website).

comment {name}

Automatically monitor website

The manager of a website that has most of its content published via CMS, needs to monitor all new and updated pages to ensure that no automatically detectable accessibility issues have been introduced by content authors.

comment {name}

Perform spot-checks

As part of a full accessibility evaluation a web accessibility consultant needs to manually spot check specific features to ensure that they meet WCAG 2.0.

What is meant whith "spot checks"? Clarify. {Sylvie, 7 January 2014}
- Is this different from "specific accessibility aspects"? {Andrew, 11 Jan 2013}
  - I agree with Andrew, looks very similar. Eric
comment {name}

Check widget accessibility

A client-side script programmer has created a dynamic widget and needs to check that the dynamic content is accessible.

comment {name}

Check specific accessibility aspects

A developer needs to check specific aspects of accessibility, such as keyboard access or adequate color contrast.

Other specifics might include multimedia {Andrew, 11 Jan 2014}
comment {name}

[others]

...

comment {name}

Overall comments

I don't have new comments as I'm part of this group ;-) {Emmanuelle}
there are already many situations, I think it is enough. {Sylvie, 7 January 2014}
comment {name}

Needs

Checks the accessibility of
- AJAX
- WAI-ARIA
  - Landmarks
  - Roles
  - Live Regions
- Inline SVG
- Webfonts
  - Especially if icon fonts are used correctly, I guess it is impossible to detect the readability of a font.

Usability Observations

We have a few usability pitfalls. For example when using WCAG 1.0 and WCAG 2.0 as search criteria there are only results if a tool tests for both specs. The result is no result, but I’d expect Checkboxes to mean an OR search. This seems illogical to me and could be improved.

Also categories where no tools are available (Repair options -> Powerpoint into accessible HTML) should not be displayed.

It would be great to have the number of remaining options visible when selecting different categories. Example: Clicking MacOS should show (18) behind WCAG 1.0 in the current version, clicking Flash in addition to MacOS should bring the counter behind MacOS to 0, and vice-versa. Items with a number of zero should be disabled.

Technical Observations

As the searching is already built and we seem to need only a few more things to search for (see needs), we could basically leave the server side alone.
Or: If we want to have the auto updating, we could have the whole application run on the client side. We would parse the EARL file into JSON (Or use the JSON EARL format) and add that to the HTML (or load it from the server for better cacheability).
- Advantage: We could be much more responsive to user input, without reload. So Web 2.0.
- Disadvantage: It would not work without JavaScript. (Or we would have to recreate the code in PHP or use Node.js.) Also we might want to exclude older browsers (< IE9 or < IE10) as they are probably slow if we have hundreds of entries…
  [Note that many Govt agencies in Australia still rely in EI8 :(]
- Consideration - what are the mobile implications for client side? {Andrew, 2014.07.04}