Important note: This Wiki page is edited by participants of the EOWG. It does not necessarily represent consensus and it may have incorrect information or information that is not supported by other Working Group participants, WAI, or W3C. It may also have some very useful information.

WCAG-EM review

From Education & Outreach
Revision as of 20:29, 22 March 2013 by Shawn (Talk | contribs)

Jump to: navigation, search

Nav: EOWG wiki main page


Reminder: EOWG will focus on the types of questions below under #Comments submitted on 22 October 2012. If you have comments on specific technical points in the document, you can submit them directly yourself -- that is, e-mail them to 22
If you're not sure, feel free to put your comments here and the Group will decide if they are ones you should send yourself.

March 2013 Comments on 26 February 2013 Working Draft


  • Location/Summary: Change.
    Rationale: {Name}

EOWG discussed comments:

  • Table of Contents: Change headings to something like: "Contents Summary" and "Detailed Contents". Include the Appendices in the Detailed Contents list, rather than under a separate heading.
    • initial comment: I wonder if it is necessary to list the main section in an "overview" heading, then to list all sections in a "sections heading" and have a list of appendices at last. Why not displaying a classical "table of contents" as in WCAG or other W3C documents? May be give the ability to expand/collapse the different sections (in the table of contents)? {Sylvie 22/March}
  • Step headings and "Methodology Requirement":
    1. Differentiate the Step headings from the statement that follows. Maybe make the headings even more terse, e.g.,
      "Define the Scope of the Website"->"Define the Scope",
      "Define the Goal of the Evaluation"->"Define the Goal",
      "Define the Conformance Target"->"Define the Conformance",
      "Define the Techniques and Failures to be Used (Optional)"->"Define the Techniques and Failures (optional)"
    2. Delete "Methodology Requirement" and the colon so it's like:
      <h4>Step 1.a. Define the Scope
      <p class="req">1.a. Define the scope of the website.
    • initial comment: I don't understand the difference between h4: "Step 1.a: Define the Scope of the Website" and the following sentence: "Methodology Requirement 1.a: Define the scope of the website." When you listen to it, it sounds redundant but there may be a reason for this redundancy. I think this question is repeated over all the section. {Sylvie 22/March}
  • Statement under Steps (e.g., "Methodology Requirement 1: Define the evaluation scope according to Methodology Requirement 1.a, Methodology Requirement 1.b, Methodology Requirement 1.c, and optionally Methodology Requirement 1.d."): remove the links to the sub-steps so it's:
    <p class="req">1. Define the evaluation scope
    <p class="req">2. Explore the website to be evaluated
    <p class="req">3. Select a representative sample of web pages from the website
    <p class="req">4. Audit the selected sample of web pages
    <p class="req">5. Document the evaluation findings
    • rationale: They are long, redundant, and the links are distracting.
  • Definition lists: Change the definition lists in some places where it would be better to use different markup. For specific places and rationale, see e-mail {Bim}
  • Summary sheet: Provide a one-page summary in multiple formats, e.g., HTML, RTF, spreadsheet, pretty PDF for printing. (EOWG can help with this.)

Introduction section

  • Location/Summary: Change.
    Rationale: {Name}
  • [Wayne - here's a placeholder for your comment]
    Purpose of the Document (and maybe elsewhere): @@Clarify that this is not required methodology for most developers.
    Rationale: @@ {Wayne}
  • Background Reading, linked headings: Unlink the headings and add those links in the list.
    Rationale: People are likely to miss that the headings are links and those are very important links, e.g., WCAG Overview. {Shawn}

EOWG discussed comments:

  • Purpose of the Document: Re-write the list to be more clear, and to focus on the most common uses first. fyi, see Who WCAG-EM is for in Overview page.
    Rationale: As written, the repetition of words makes it quite difficult to parse. Also, we are concerned about having "website developers" as the first item. See: Shadi (for discussion in EOWG f2f before CSUN).
      initial comments:
    • The readability of the "Purpose of this Document" section seems difficult. Because the bullet items are long, they seem too bunched together at their current spacing and too wide for easy reading of a list. Also, many of the lines begin with very similar wording: "web accessibility" begins 4 of the 8 lines; web or website begins every line. Lists with similar wording, especially at the beginning reduce readability since it's difficult to distinguish one line from another. Perhaps some lines could start with a distinct function instead of the type of user. Or set up the list as a table with the type of user on the left column and the function or purpose in the right column. For example:
      Function Example
      An evaluation tool Web developers who want to evaluate the accessibility conformance of their websites
      Analysis & reporting Consultants who want to report the accessibility conformance of websites to their owners
      A learning tool Accessibility researchers and disability advocates who want to learn accessibility conformance practices
      A teaching tool Trainers and educators who want to teach methods and approaches for web accessibility evaluation
      Accessibility Benchmarking Individuals who want to benchmark or compare the accessibility conformance of websites

      I'm not sure I know the best solution, only that the readability of this section is problematic.

      {Howard, 21/March }

    • The "who" refers back to a process, not a person, in the bullet item "Web accessibility monitoring activities who want to benchmark or compare the accessibility conformance of websites" under "Purpose of Document." One possible solution: change "monitoring activities" to "monitors."{Howard, 21/March }
  • Scope of this Document, first sentence: Format it as a list.
    initial comment: the first para under 'Scope' would be aesier to read as a list rather than a ';' separating parts of a long para {Andrew, 15/March}
  • Purpose of this document, last sentence: Delete the sentence starting with "Complementary resources on ..." and add those resources to the next section "Background Reading".
    initial comment: [EOWG agrees - @@ merge those into background reading] I agree with the readability difficulties. It could also be improved in the paragraph beginning with "Complementary resources on"
    Suggestion: Begin with: For further information, see complementary resources below." Then list those resources in a bulleted list. {Sylvie 22/March}

Using this Methodology section

  • Location/Summary: Change.
    Rationale: {Name}
  • Using this methodology, first paragraph: Reword to be more positive and be in-line with Easy Checks (which will replace the old Preliminary Eval page).
    Suggested rewording: [@@EOWG should probably suggest specific wording change for this! Will someone please take a pass at it.]
      initial comments:
    • In second sentence: "A more cursory approach for exploring the accessibility of a website", the term "cursory approach" is not clear to me. Is it jargony? Why not using a term that everyone can better understand? {Sylvie 22/March}
    • For better readability, it may better to split following senetence in two:
      Actual text: "In many cases it is advisable to carry out such preliminary reviews before applying this methodology, for example to identify obvious errors and to develop a rough understanding of the overall performance of the website."
      Suggestion: "In many cases it is advisable to carry out such preliminary reviews before applying this methodology. For example, identify obvious errors and develop a rough understanding of the overall performance of the website. {Sylvie 22/March}

EOWG discussed comments:

  • Particular Types of Websites, Web Applications section: Rewrite to be more clear.
    initial comment: In second bullet "Web Applications", the note of this part is very long due to the many links and few sentences. Is it possible to make it clearer? After I read it twice, I still try to understand it. {Sylvie 22/March}

Conformance Evaluation Procedure section

  • Location/Summary: Change.
    Rationale: {Name}

Step 1: Define the Evaluation Scope Section

  • Location/Summary: Change.
    Rationale: {Name}

Considerations for Particular Situations section

  • Location/Summary: Change.
    Rationale: {Name}

Application of this Methodology section

  • Location/Summary: Change.
    Rationale: {Name}

Copyedits & things that probably don't need discussion

  • Location: Problem: @@. Current wording: @@. Revised wording: @@. {Name}
  • In first sentence of second paragraph of Abstract, should be comma after "for example"{Howard}
  • Typo: (may be several times in document): technqiues instead of techniques. {Sylvie 22/March}
  • Consistency of section headings: The link text of sections (steps) that are referred to are not always the same as the sections titles. For example, at the end of section step 5.A, there is a link to "Step 5.b: Provide and Accessibility Statement)". First there need to be corrected in "an accessibility Statement". Second, the title of the heading is: "Step 5.b: Provide a Conformance Evaluation Statement (Optional)", whereas the Methodology Requirement 5.B is entitled: "Provide an accessibility conformance evaluation statement (Optional).". {Sylvie 22/March}
  • Typo: Sucess criteria instead of Success criteria (several occurences){Sylvie 22/March}


fyi, Comments that individuals sent directly (not through EOWG)

  • Excess use of 'per' especially in section 5. I'm not sure how to do an archive link but have sent the following direct: "I have read through WCAG EM and congratulate you on its progress. On a small point of language I noticed is an outbreak of the word 'per' - especially in section 5, which I would suggest could inhibit ease of understanding and is also unnecessary. I would propose removing all 'per' and using a plain language alternative, if necessary. FYI: I researched the opinions of others via Google, and here is one of the more interesting discussions on the use of per and as per: Per and 'as per' is a pseudo-latin construction. Opinions appear to divide between clarity, efficiency, and improper use. There are however alternatives which are more natural. The only examples that fit well in my head is taking medicine 'per day', or sharing pencils 'one per child' and 'per your instructions'. The use of 'Per website' and 'per web page' (WCAG EM methodological requirement 5c) seem particularly puzzling and drove me to Google to find out why. {Suzette}

Archived Notes (not for Eval TF)

  • [closed. updated Style with rationale for putting periods and commas outside of quoted terms.] This may need discussion, but I'm not sure where else to put it. According the Chicago Manual of Style Rule 6.8: "Periods and commas precede closing quotation marks, whether double or single." The way I did the previous sentence conforms to that. Things like "not "magnifying glass"." don't. We use this pattern a lot, but that means we're consistent. It just may be consistently wrong. OTOH, it would be an incorrect pattern in code, so coders may cringe if we adopt the Chicago rule. (3/20/13) {Anna Belle}

Comments submitted on 22 October 2012

Readability & Usability

What specific revisions will make the document more readable and usable?

  • linked terms distracting- Every time a term is mentioned, there's a link to the definition e.g. "web pages",websites, I find this quite distracting and I think it reduces the readability {Vicki}.
    Strongly agree. Some ideas: just link it the first time in the document, or a new section. OR provide an option for users to turn off these links. {shawn}
  • linked references in sentences - the linked reference (e.g. "3.5 Step 5: Report the Evaluation Findings") in the middle of a sentence is also a little distracting and breaks the reading. {Vicki}
  • examples matching use cases - readability, section 1.1 scope and 1.2 audience. I think this is a difficult read for the intended audience. To improve usability I would suggest there is a need for better tailored use cases that match 1.2 and are then addressed consistently. There seem to be some examples included about commerce but I think it needs to use a better defined range - that better capture the context of the audience be it large scale government sites, large commercial sites, banks and utilities, and big agencies through to smaller operators eg a local school, a local charity, small organic food retailer. {Suzette}
  • Delete text from Abstract - Abstract current wording: "This work is part of complementary W3C/WAI activities on Web Accessibility Evaluation and Testing that address different W3C/WAI guidelines and specifications."
    Comment: This leads reads off the page to 2 separate places. Suggest not having it in the Abstract. OK in the intro, but in the abstract, I think it takes the focus off this document too much. {Shawn}
  • Section numbering - Please consider not numbering every section. "3.4 Step 4:" etc adds complexity. It would be much simpler to just have "Step 4:" {shawn}
  • [minor] delete "W3C/WAI" where you can
  • [minor] add "(WCAG-EM)" to the title, abstract, and introduction


Is it clear that this set of methodologies is only for websites after they are built and does not address methodologies that might be used prior to and during development? If not, please suggest ways to clarify the scope.

  • Given that the title is about Compliance evaluation, I have no problem with clarifying that the scope is what educationalists call summative assessment (like the final exam!) rather that formative assessment (feedback you can learn from). I noted a number of mentions in the opening sections about the benefits or effects of iterative testing during the design process. My suggestion is to keep the two types of testing separate because they relate to substantially different scenarios. Developers probably should not do their own formal compliance testing. I would suggest that the EM has a short paragraph to discuss the difference so that the reader is aware of this and can chose whether the Compliance EM document is relevant or if they need to go to the less formal 'Design_and_ Evaluation' section. {Suzette}

1.1 Scope of this Document - second and third paragraphs, cited below:

"However, this methodology only addresses evaluation of website conformance to WCAG 2.0 after its development. Complementary resources from W3C/WAI on evaluation provide guidance on evaluation in other situations, and complementary W3C/WAI activities on Web Accessibility Evaluation and Testing address different W3C/WAI guidelines and specifications."

"Also, WCAG 2.0 conformance requirements apply to individual web pages. However, on most websites it is not feasible to evaluate every web page with reasonable effort. To ensure conformance of a website to WCAG 2.0 it is necessary to ensure that this is considered throughout the development process. Following this methodology helps assess the level of conformance with reasonable confidence where it is not feasible to evaluate every web page."

  • I prefer not to restrict the evaluation to "only" sites after development.
  • I find the second paragraph starting "Also, WCAG 2.0 conformance....etc." somewhat puzzling. In the first paragraph, there is mention that the evaluation methodology is only for sites after development but, on the other hand, in the second paragraph, there is mention of the importance of conformance during the development process and following "this methodology" (which in the first para is stated as only for after development) helps assess the conformance.

Suggestion (if the scope is fixed on already developed sies):

This methodology is best used to address evaluation of website conformance to WCAG 2.0 after development. WCAG 2.0 conformance requirements apply to individual web pages. On most websites, it can be challenging to evaluate every single web page. This methodology helps assess the level of conformance with reasonable confidence where it is not feasible to evaluate every web page.

Complementary resources from W3C/WAI on evaluation provide guidance on evaluation in other situations, and complementary W3C/WAI activities on Web Accessibility Evaluation and Testing address different W3C/WAI guidelines and specifications.

It is noted that conformance which is considered during the development process reduces significantly the costs of evaluation after development. {Vicki}

[important] The only one comment I do feel somewhat concerned about is the "Scope" and restricting the document to "only" sites after development. I look at it from the point of view of someone e.g. a developer coming across the document through a search and, with the patience (rather lack of it) that we have on the web, it might be disregarded by the restrictive focus. It is also a document which could still be useful to someone who might perform iterative evaluation through the development process. Therefore, I would recommend being more receptive to a wider audience by changing the wording somewhat. {Vicki via Sharron}

Evaluate throughout

How should we communicate the importance of evaluating throughout website development, not just at the end?

  • Abstract
    Abstract current wording: (in a paragraph) However, ensuring and maintaining accessibility requires that accessibility is considered throughout the development process. Complementary resources from W3C/WAI on evaluation provide guidance for other situations.
    Abstract suggested text: (separate paragraph) When developing or redesigning a website, accessibility should be considered early and throughout the design and development process. This is addressed in a new resource under development {which EOWG expects to have drafted for the next WCAG-EM draft so you can put the actual title of the page here}.
  • Introduction, Scope, middle paragraph:
    Current text: "However, this methodology only addresses evaluation of website conformance to WCAG 2.0 after its development. Complementary resources from W3C/WAI on evaluation provide guidance on evaluation in other situations, and complementary W3C/WAI activities on Web Accessibility Evaluation and Testing address different W3C/WAI guidelines and specifications."
    Suggested text: This methodology only addresses evaluating an existing website's conformance to WCAG 2.0. When developing or redesigning a website, accessibility should be considered early and throughout the design and development process. This is addressed in a new resource under development {which EOWG expects to have drafted for the next WCAG-EM draft so you can put the actual title of the page here}.
    (new paragraph) Related information outside the scope of this document is provided in Evaluating Websites for Accessibility, W3C WAI Web Accessibility Evaluation and Testing Activities, and WAI Guidelines and Techniques.


Are there any major considerations that we have overlooked or misrepresented?

  • Updates, maintenance, drift: An accessibility audit is static - a snapshot in time. By contrast, contemporary websites are organic - constantly evolving and growing. New material and novel features can be added by people who were not part of the original development team and these authors may have very limited editorial skills, and knowledge of development. As part of EM, I would suggest that that we should make a strong case for planned reviews to maintain standards and avoid the effect of drift that has been observed by researchers. {Suzette}

Related material

How much should we refer to and link to existing material as reference and how much should we include directly in the new material? Balance the inconvenience of jumping back and forth between existing documents against our reluctance to unnecessarily duplicate information.
How should we align the WCAG-EM work with the Preliminary Evaluation document now in process at EOWG?
What should the Conformance Evaluation page be in relation to WCAG-EM?
(see also Eval Analysis)


These comments were entered before we had the structure above, so might apply to above or not.

  • [Very Important] Methodology Requirement 1.d - This requirement appears to claim that the website owner and evaluator are free to choose which users may be excluded. Human rights do not work that way. What reasonable base technologies are required for conformance? This is particularly disturbing within the context of intranets where employers may choose to limit users to assistive technologies the employer can purchase cheaply. The note at the end is not sufficient.
    Example, many sites prescribe large print with high contrast to be a universal cure for low vision. This discounts the need for albinos to have lower luminosity. Without high luminosity, high contrast is not possible. {Wayne Dick}
  • [important] 3.1.5 Step d - appeared to invite employer determined prescription of assistive techonolgy and accessibility support. The result will be employer defined necessary employment discrimination. {Wayne Dick}
  • [minor] Relied Upon - Just hangs there without inclusion in the context of accessibility supported. {Wayne Dick}
  • [Med] The note to (Web Applications): It may not be feasible to exhaustively examine all possible settings, but sampling should apply just as any website. {Wayne Dick}
  • 3.1.5 Step 3. The evaluation commissioner, must be informed that if the self defined sufficient techniques are used then a future accessibility audit may ask for a proof of sufficiency. {Wayne Dick}
  • Throughout 3.1.5 PDF and Silver light are basic web technologies. They are no more technologie than docx or raw text. They as basic web technologies. The accessibility open question.
    This issue is particularly important if this document is going to be normative. I do specific reference to Adobe or Microsoft products as basic web technologies is serious vendor preference. {Wayne Dick}
  • 3.3 Step 3: Select a Representative Sample, (First sentence) "While ideally every web page of a web site is evaluated, usually this is not possible on most web sites."
    I feel that stating "usually this is not possible on most web sites" sounds somewhat contradictory (to our objective).
    Suggested text: "While ideally every web page of a website should be evaluated, this may not be possible for very large websites." {Vicki}
  • 3.3 Step 3: Select a Representative Sample - website with many heterogeneous sub-university example, the sample selection include users. Focus groups are an effective probably necessary.
    Large heterogenous sites behave more like a set of independent sites placed under one URL. As such it is important to organize focus groups within the different autonomous site owners to determine usage patterns and prioritize pages for evaluation. When organizing the CSU System, 23 Campuses, each having more than 50 academic departments, and more administrative departments, we found that the IT unit could not get a handle on usage patterns on its own. Individual site owners had to give direct input on priority and usage. Example: At CSU Long Beach the Computer Science lab invokes the Computer Science Department Website every time a student logs into a lab machine. That makes the Computer Science Department the most referenced site on the Campus. But it is not the most important. {Wayne Dick}
  • Responding to the term "with reasonable confidence": I have no problem with the term. Alternatively, if the above term is a point of discussion, you could simply modify the sentence to "that is representative of the entire target website". {Vicki}
  • 2.5 Involving Users (Optional)- Second last sentence. "While not required, it is strongly recommended to involve real people covering a wide range of abilities...."
    Suggest removing "real". {Vicki}
  • 3.1.4 Methodology Requirement 1.d: Third paragraph: second last sentence. "For example, ….., then the website is effectively not accessible for some users."
    Suggest adding something about conformance for emphasis e.g., "and therefore does not conform to the specified conformance level of WCAG 2.0. (3.1.3 Step 1.c.)" {Vicki}
  • Term "Common Functionality" Feedback was requested on the definition on "Common Functionality". Having read through the document, I don’t have a problem with the term "common functionality. However, it might pose a question mark to some. Therefore, I’ve put forth a few options below depending on what you mean and if you wish to be more specific:
    If it is the general functionality of the site, you could simply leave out the word "common" or use "basic functionality"
    If "common functionality" has a more technical connotation (referring to the style sheets, javascript, classes, i.e., the technical mandatory requirements for multiple device support and functionality), leave it as "common"
    Another suggestion in the line of "common" and "key" is “core”.
    Suggestion to modify the description (in line with the "Note"):
    Functionality of a website includes tasks that users of a website perform in order to achieve a goal. {Vicki}
  • 3.5.3 Step 5.c: Provide a Performance Score (optional) I think there are good reasons for offering a score in addition to or as an alternative to the absolute measure of pass/fail. I have read this section several times, and tried really hard to construct a formula to help me comprehend the differences but I am currently confused. Maybe my basic scenario is wrong, so perhaps a worked example would help? Eg what would the score show where some form elements are consistently incorrectly marked and are affecting 5 out of 10 pages tested. {Suzette}
  • Term "sub-site" : do you want to add the definition to the terms and definitions? {Vicki}
  • Section step 1.A, last paragraphis not clear to me. What is necessary there? May be it could be easier to understand with concrete examples. <quote>"The outcome of this step is an unambiguous definition for the target website, so that for each web page it can be determined whether it is within the scope of evaluation or not. Where possible, it is recommended to use formalizations such as regular expressions or listings of Universal Resource Identifiers that define the web pages that are within the scope of evaluation (part of the target website)."</quote> {Sylvie}
  • 1.3 Background Reading, Accessible Web Design
    This section doesn't feel right. For example, my first reaction was that some relevant documents are missing, such as Improving the Accessibility of Your Website – and that some included here are maybe not more important than others that weren't included, e.g., Developing Websites for Older People & Web Content Accessibility and Mobile Web seem less important in this context.

    Please reconsider the purpose of this list in the context of WCAG-EM. (Remember this section starts with "It is assumed that the reader of this document is familiar with the following related resources from W3C/WAI:") I can see that Essential Components of Web Accessibility and How People with Disabilities Use the Web are particularly important for evaluation — perhaps only list those 2 — and not under the heading "Accessible Web Design" but something more directly related to understanding evaluation in the bigger context.
    (minor point: This section is not parallel with the others above it — although I don't think that's a big deal. minor edit:link change: Web Content Accessibility and Mobile Web)
  • Background Reading.
    current wording: "It is assumed that the reader of this document is familiar with the following related resources from W3C/WAI:"
    Ideas for edited wording: "To effectively use this methodology, you should be familiar with the following related information."
    or "The information below related to WCAG 2.0 and evaluation is important background for using this methodology. " {Shawn}

Copyedits & things that probably don't need discussion

  • 3.1.2 Step 1.b. In-Depth Analysis. End of last sentence is missing a fullstop. {Vicki}
  • 3.2.3 "Note: This step is intended to identify groups of web page instances, including in web applications."
    Suggest drop the "in" as in "including web applications" or add "including those in web applications". {Vicki}

Comments sent directly (not through EOWG)