Important note: This Wiki page is edited by participants of the EOWG. It does not necessarily represent consensus and it may have incorrect information or information that is not supported by other Working Group participants, WAI, or W3C. It may also have some very useful information.

Difference between revisions of "Eval Analysis"

From Education & Outreach
Jump to: navigation, search
(Title ideas)
(Title ideas)
Line 205: Line 205:
** <span style="color: #808080;"> or whatever number we come up with {Suzette}</span>
** <span style="color: #808080;"> or whatever number we come up with {Suzette}</span>
** <span style="color: #808080;">'''''no''''' - think we don't want "conformance" in the title of this since it won't get you to conformance, and the other doc focuses on conformance {Shawn}</span>
** <span style="color: #808080;">'''''no''''' - think we don't want "conformance" in the title of this since it won't get you to conformance, and the other doc focuses on conformance {Shawn}</span>
* Basic evaluation....

Revision as of 14:03, 11 January 2013

Nav: EOWG wiki main page
Related pages: Web Accessibility Preliminary Evaluation, Eval in process

Tasks & Use Cases

The following table lists tasks, example user cases with audience, and focus for each document. (Note that we haven't decided whether or not to do a new doc on evaluation in the process.)

task example use cases prelim conform
WCAG-EM process  
Get general idea of accessibility of a webpage (major problems, some issues, maybe OK)
  1. Contractor: I've developed a prototype and I've been asked "is it accessible?" What can I do quickly to check?
  2. Website manager: I got a complaint from a user and want to get an idea of the scope of the accessibility problems on my website.
  3. Website manager: I'm checking out design companies to commission to re-do my website and I want to see if their website is accessible.
  4. Website manager: I commissioned a website that was suppose to meet WCAG. I've just received the first prototype and I want to do a quick check to see how it is on accessibility.
  5. Advocate: I encountered an accessibility barrier trying to use a website and I want to know if there are many other problems or not.
  6. Web content writer, journalist, blogger: I want my pages to be accessible in general so that I can reach the widest audience.
mostly maybe some      
Get an overview of WCAG-EM
  1. Evaluation procurer: We are commissioning a WCAG conformance evaluation and want on overview of WCAG-EM, which we plan to reference in the Request for Tender/Proposals.
  2. @@
maybe lots maybe just a little    
Thorough, authoratative evaluation of WCAG conformance
  1. Web development agency: Our services include providing a full accessibility audit of the new templates (CMS) and all webpages prior to launch.
  2. Accessibility champion or Quality assurance (in a large organization): I need to regularly evaluate new sections and applications before launch.
  3. Independent certification: We need to show that our expert audit service and user testing by people with disabilities meet recognised international standards.
  4. Accessibility Maintenance: We periodically check the accessibility of sections of our website to monitor that we continue to meet our certification status.
  5. Developer: I want to certify that my business website conforms to WCAG 2.0.
  6. Multi-site benchmark study: Our accessibility consulting firm has been hired to evaluate and report on the conformance of 25 governmental websites.
  7. Researcher: I research accessibility, and investigate accessibility issues on multiple sites
  8. Education: I teach students to become web designers, developers or accredited accessibility assessors
  9. Advocate: I need to be able to direct people (eg businesses, information providers) to an authorative resource that they can use to evaluate the conformance of their website to WCAG 2.0

  maybe at first mostly    
Evaluate throughout design and development process
  1. Design specification: We want an independent assessment of high level objects during early stage design planning. We can provide sample use cases, early page mock-ups, wireframes and simple prototypes.
  2. Development Support: Our company provides independent evaluation including from disabled people. This supports development of specific content, new applications and mobile. Reference to WCAG 2.0 AA and agreed elements of AAA.
  3. Developer: I want my sites to be professionally designed and to meet the standards baseline, including WCAG 2.0 AA. To do that effectively I want to review new sections and applications as I go along.
  4. Project manager, designer, developer: I see all these new resources about evaluating websites after they're done — but I need something that applies to a complete redesign.
  5. Developer: I'm developing a new feature (or adding new content) and want to assess progress on specific functions

some     lots  


  • I have added some extra use cases. I wanted to have different types of websites: large and small websites, whole websites and sets of pages or new sections, and different types of developers/evaluators: inhouse and agency developers, third party evaluation, and also maintenance (I think its important!) I would like to discuss them and any other variations that anyone wants to add. To help discussion I turned the bullets into numbers. Also, I fixed a table format problem that I caused, but not the alignment issue. {Suzette}

Use case for preliminary evaluation

{@@dboudreau - 20120928} First draft - proposal

  • Vendor is commissioned to develop a WCAG 2.0 or Section 508 compliant website
  • Client wants to know if the vendor actually delivered an accessible website, but doesn't have the skill set to really measure it.
  • Client uses the Preliminary Evaluation process (simple tests) to check for basic things like alts, keyboard nav and headings on random pages.
  • Client determines if the website passes those tests or not, based on the tests results.
    • If the website passes those tests, then client or designated technical person digs further with additional informal tests.
    • If the website doesn't pass those tests, then client commissions a technical profile to integrally follow the WCAG-EM methodology.
  • Find and fix - developer, or content editor tests newly added content and fixes commonly occuring missed elements eg text alternatives, form elements, link labels, text resizing and colour contrast prior to release or more in depth evaluation. {Suzette}
  • From Website Accessibility Conformance Evaluation Methodology 1.0. Preliminary is mentioned three times including 1.3, 2.0 and this one: 4.1:"Note: It is recommended to carry out preliminary reviews before carrying out a conformance evaluation to identify obvious errors and to develop a rough understanding of the overall performance of the website."{Suzette}

Preliminary Evaluation Analysis

Approach for drafting: We talked about have a short, easy list at the top (5 min). Then having a longer list - maybe what you can check in 15 minutes. We're working on the longer list first, then will refine the short list. Currently there are a lot of draft checks in the long list and we want to cut that down.


Primary audience:

  • Non-technical people with low accessibility knowledge who want a general idea of the level of accessibility of a web page

Secondary audiences:

  • Newbie accessibility evaluators who want to dive in progressively
  • Knowledgeable evaluators who want to run very high level evaluation only


  • [open] For each check, how much explanation do we include on this page versus pointing to the relevant Accessibility Requirements section or Understanding WCAG page?
    • ...comment {name}
    • Reasons to have more information on this page: {Shawn}
      • Newbies can get the basics from this one page without having to follow links elsewhere.
      • Can have very basic, focused description, whereas other places might be more complex.
    • Reasons to have less information on this page: {Shawn}
      • Once people understand the basics, it will be clutter to have to wade through all the information each time they want to use this as a checklist.
      • Duplicating information increases maintenance.
      • @@ The title will probably contain "quick" so my gut feeling is to aim at being short, clear and easy. In this way, people will use this reference. Pointers to more complete information could be at the end under "Resources"{Vicki}
    • Perhaps if we had the expand-collapse functionality, it would work to have more explanation on this page? {Shawn}
  • One page — Focus on one web page, as opposed to broader issues of selecting representative pages, etc., which is addressed in WCAG-EM
  • Tool-specific info — For this draft we have some tool-specific guidance. However, there are potential issues with vendor-neutrality and we might need to address this a different way — for example, moving tool-specific guidance to Web Platform Docs or the WAI-Engage wiki where people can easily add tool-specific info.
  • Issue, quick versus thorough — Some things you can check partly in a quick check, but will need much more effort to do thorough check. Need to make this very clear so someone doesn't think passing the quick check means it's A-OK.
  • ...

Criteria for checks

These are draft ideas for now; they might change.

Criteria for longer list (15 min checks)

  • Common accessibility barriers (what we see often as mistakes in web pages)
  • Easy to understand
  • Not complicated issues (not if lots of debate on forums)
  • Pass-fail not complex; Not likely to give false positive or false negative
  • Checks that clearly related to specific WCAG success criteria
  •  ? easy of fixing
  •  ? has good example in BAD

Might give more weight to some checks that don't require seeing visual rendering of page.

Criteria for short list (5 min checks)

In addition to above:

  • Do-able with system limitations - does not require downloaded tools or a specific browser
  • ...

Title ideas

Previous page was titled Preliminary Review of Websites for Accessibility. Do we want to change it?
Should we keep "Preliminary" somehow in the title for old links (i.e., places that link using the old title)?

  • remove the word "preliminary" - it's a "stop-to-think" word, leave alone "cannot pronounce" ;) {Vicki}
  • I agree with Vicki, this word is really difficult to pronounce. I like the idea of replacing it with quick check. - Sylvie}
  • In other contexts (like exams) the word gets contracted to prelims. Semantically it is closer to the idea of doing some 'first attempt' checks, that are not final or complete but give a general idea of how well you are doing. It reminds me of hospital emeregency facilities in the UK using 'triage' to sort out who is critical and who can wait around for a couple of hours! Possibly alternatives could include 'initial', 'first pass',or 'pre-checks' Suzette}
  • Suggestion: Could a server-side redirect be used to prevent broken links and lost favorites? {Bim} -- Shawn replies: We will not break the URI! We most likely will put this new page at the same URI, even if the title is different.

Other title ideas: [comment template: summary - Comment {name}]

  • Quick Checks: Preliminary Evaluation of Web Accessibility
    • not - Difficult to pronounce, would like to find something that says the same thing but is easier. - Sylvie}
    • no - For the same "preliminary" reasons given above - Vicki}
  • Quick Checks: Preliminary Web Accessibility Evaluation
    • I'm not sure that 'Quick' is the right sales pitch for the document. Its more about doing some easy checks that indicate if there are possible problems that need further indepth investigation, or could be quickly fixed before organising a full inspection. - Suzette}
    • no - For the same "preliminary" reasons given above - Vicki}
  • Quick Checks for Accessibility (Preliminary Review of Web Accessibility)
    • part - I like this one but we could remove the brackets. What about: "quick and easy checks for accessibility" - Sylvie}
  • Quick and Easy Checks for Accessibility
    • like - {Sylvie}
    • like - {Vicki}
  • Quick Check to Evaluate Accessibility
    • maybe - "evaluate" might be good for SEO? {shawn}
    • like - like this one too - Sylvie}
    • like - {Vicki}
    • like - {Bim}
  • Quick Check to Test for Accessibility
    • maybe - "test" might be good for SEO? {shawn}
    • like - {Vicki}
  • Accessibility Quick Check
    • like - I can see people using this as a search term. {Bim}
  • Simple Evaluation
    • not - Maybe this one is too simple. - Sylvie}
    • Don't like - Evaluation of what? {Bim}
  • Evaluating accessibility quickly and efficiently
    • like - {Sylvie}
  • Checks to perform a quick web accessibility evaluation
    • like - {Sylvie}
  • Getting a quick idea of the accessibility level of a Website
    • like - {Sylvie}
  • 10 easy tests for accessibility conformance of your website
    • or whatever number we come up with {Suzette}
    • no - think we don't want "conformance" in the title of this since it won't get you to conformance, and the other doc focuses on conformance {Shawn}
  • Basic evaluation....