Disposition of Comments

Website Accessibility Conformance Evaluation Methodology (WCAG-EM),
Working Draft, 30 January 2014

This is a disposition of comments received on the Website Accessibility Conformance Evaluation Methodology (WCAG-EM), Working Draft of 30 January 2014. This page is intended for internal discussion by the WCAG 2.0 Evaluation Methodology Task Force (Eval TF).

Contents

  1. Overall Comments
  2. Abstract
  3. Introduction
  4. Purposes for this Methodology
  5. Relation to WCAG 2.0 Conformance Claims
  6. Background Reading
  7. Terms and Definitions
  8. Required Expertise
  9. Evaluation Tools (Optional)
  10. Review Teams (Optional)
  11. Involving Users (Optional)
  12. Principle of Website Enclosure
  13. Particular Types of Websites
  14. Particular Evaluation Contexts
  15. Step 1: Define the Evaluation Scope
  16. Step 2: Explore the Target Website
  17. Step 3: Select a Representative Sample
  18. Step 4: Audit the Selected Sample
  19. Step 5: Report the Evaluation Findings
  20. Typographical Comments

Overall Comments

ID Status Commenter Location Current text Suggested Change Rationale Resolution
03 Closed Sham Mitra Throughout Testing should also include the option to completely switch off layout and images, and have content linear to improve navigation Those with disabilities, particular visual impairments, care not for graphics or presentation.

Resolution: No change

Rationale: WCAG-EM does not provide details on point-for-point testing in WCAG-EM
04 Closed Sham Mitra Throughout Testing guidelines include testing across multiple desktop/mobile browsers. In STEP 3, I would recommend testing the pages across multiple browsers. Many corporations still run legacy versions of web browsers. Backwards compatibility with modern web technologies may impact meeting WCAG 2.0 guidelines. Additionally, cross-browser accessibility for a website can be problematic.

Resolution: No change

Rationale: WCAG-EM does not provide details on point-for-point testing in WCAG-EM
05 Closed Sham Mitra Throughout Through the evaluation process, there should be continuous feedback. The recommendation for 'AA' rating is a great baseline.

Resolution: No change

Rationale: WCAG-EM supports continuous feedback through the different steps
66 Closed Kerstin Probiesch Title "1.0" "Alpha Version" "1.0" could likely be misunderstood as Recommendation

Resolution: No change

Rationale: The document is clearly identified as a W3C Note. Versioning of WCAG-EM is important to identify evaluation reports made according to a particular version.
74 Closed Kerstin Probiesch WCAG-EM says nothing about the rating system: pass/fail or "anything" else? I think, is very critical because the results of different rating systems will not not be comparable. Provide pass/fail as rating system for evaluation.

Resolution: No change

Rationale: WCAG-EM does not introduce rating values beyond what is already defined by WCAG 2.0.

Abstract

No comment received on this section

Introduction

ID Status Commenter Location Current text Suggested Change Rationale Resolution
75 Closed Moe Kraft and Mary Jo Mueller First paragraph "...and the technologies used to create it (e.g. HTML, PDF, etc.)" I would rather see something like JavaScript or CSS there. PDF is not a technology used to create a website; it is content that could be contained on a website.

Resolution: Added WAI-ARIA to the examples.

Rationale: This list is not indended to be exhaustive. The idea is to give examples that people expects, like PDF. We added WAI-ARIA to indicate that also scripting is part of the technologies in scope.
92 Closed Raph de Rooij "The methodolody can be used in conjunction with techniques for meeting WCAG 2.0 success criteria, such as the Techniques for WCAG 2.0 documented by W3C/WAI, but does not require this or any other specific set of techniques" "It is recommendable to use the methodology in conjunction with techniques for meeting WCAG 2.0 success criteria, such as the Techniques for WCAG 2.0 documented by W3C/WAI. However, use of this or any other specific set of techniques is not mandatory." I agree with this sentence, but it has led to a bit of discussion here about the value and the status of the techniques for WCAG 2.0, which in practice is a very useful resource.

Resolution: No change

Rationale: Step 1.d has been edited and provides further clarification on the role of techniques.

Purposes for this Methodology

ID Status Commenter Location Current text Suggested Change Rationale Resolution
67 Closed Kerstin Probiesch "This methodology is designed for anyone who wants to follow a common procedure for evaluating the conformance of websites to WCAG 2.0." "This methodology is designed for anyone who is evaluating websites." "common procedure" is not defined in the document it is therefore a bit unclear to me what the term "common" means in this context.

Resolution: Changed "common procedure" to "common approach."

Proposed Rationale: Avoids the jargony wording "common procedure" but keeps the overall idea of "common" or "unified" approach. "Common approach" also fits better with the text above that says: "It supports common approaches and understanding for evaluating the extent of conformance of websites [...]".
93 Closed Raph de Rooij List of target audiences Add "Compliance and quality assurance managers who are responsible for meeting legal requirements and assurance aspects of information systems."

Resolution: Added "Web compliance and quality assurance managers who want to ensure that they met quality and policy requirements"

Rationale: Important audience and use-case not covered by the other items in the list.

Relation to WCAG 2.0 Conformance Claims

No comment received on this section

Background Reading

No comment received on this section

Terms and Definitions

ID Status Commenter Location Current text Suggested Change Rationale Resolution
76 Closed Moe Kraft and Mary Jo Mueller common web pages "Common web pages may also be web page states in web applications." It would be useful if the 'web page states' linked to the definition that comes later in the list. Clarify

Resolution: Reworded.

Rationale: Reworded to clarify this term, but at the same time limit the number of internal links in this phase of the document as much as possible.
91 Closed EOWG Web page states Perhaps "change of context" terminology from WCAG might work? Too vague, jargony, and technically too broad. ("state" is really general – in that definition is a drop down menu a new state as it changes the DOM?)

Resolution: Reworded.

Rationale: Reworded to clarify the meaning but keep the term "web page state".
94 Closed Raph de Rooij Add definition for "Audit": "A planned and documented activity performed by qualified personnel to determine by investigation, examination, or evaluation of objective evidence, the adequacy and compliance with established procedures, or applicable documents, and the effectiveness of implementation." (audit reference) Is 'detailed evaluation' the definition of the term 'audit'? It is used in Step 4

Resolution: No change

Rationale: Audit is a good word for detailed evaluation. The text in Step 4 says: "During this step the evaluator audits (detailed evaluation) the sample selected in Step 3".
95 Closed Raph de Rooij Website developer "content authors, designers, programmers, quality assurance testers, and project managers" "content authors, designers, front-end developers, back-end programmers, quality assurance testers, and project managers" The front-end developer plays a key role when it comes to web accessibility. It now is enclosed in the word programmers, which doesn't do it justice.

Resolution: Changed "programmers" to "front-end developers, back-end programmers"

Rationale: Added this although the aim of this list is not to be exhaustive (as indicated in the text).

Required Expertise

No comment received on this section

Evaluation Tools (Optional)

No comment received on this section

Review Teams (Optional)

ID Status Commenter Location Current text Suggested Change Rationale Resolution
68 Closed Kerstin Probiesch "using the combined expertise of review teams provides broader coverage of the required skills and helps identify accessibility barriers more effectively." Make clear that it is a thesis. Wether a review team will find more and is therefore effectively depends on various factors. I expect that a review team where each member of the team has long time experience in evaluating accessibiltiy may find more. And I believe that a single tester with a lot experience may find more than a review team with less experience.

Resolution: Reworded.

Rationale: Reworded to further clarify the role of review teams.

Involving Users (Optional)

ID Status Commenter Location Current text Suggested Change Rationale Resolution
69 Closed Kerstin Probiesch Give more explanation and definitions about expert evaluation, involving users and review teams. When we read the section "Review Teams" and the section "Involving Users" in relation one can get the impression that "Users" is a strictly different thing than "experts". If two evaluators are evaluating a website and one of the evaluators is disabled and the other one not is it then a Review Team or an individual evaluator who has involved a disabled user?

Resolution: Reworded.

Rationale: Reworded to further differentiate between teams of reviewers and users.

Principle of Website Enclosure

ID Status Commenter Location Current text Suggested Change Rationale Resolution
17 Closed Jason Kiss Throughout Document would flow better by dropping the Principle of Website Enclosure heading and the paragraph immediately below it and using the remainder of the content (including the diagram) in the section as an appendage to the section preceding it. The Principle of Website Enclosure is not referenced anywhere else in the methodology document and seems redundant given the remainder of context in this section. The remaining content illustrates the application of the description in the previous section anyway.

Resolution: No change.

Rationale: Keeping the heading helps to make the concept stand out more.

Particular Types of Websites

ID Status Commenter Location Current text Suggested Change Rationale Resolution
96 Closed Raph de Rooij Website Using Responsive Design Add a *note* on Responsive Design here, not a new type. This type of website doesn't belong here: all other types of websites described here can use Responsive Design.

Resolution: Added clarification.

Rationale: All types of websites are non-exclusive and can overlap. Added clarification at the beginning of the section to reflect that.

Particular Evaluation Contexts

ID Status Commenter Location Current text Suggested Change Rationale Resolution
14 Closed Chris Leighton Third-Party Assessment of Conformance "Third-party evaluators typically have little information about how a website was developed, its internal software, and all its areas and functionality. Therefore it is more difficult, often impossible, to make conformance claims for entire websites based on such an evaluation alone. However, such evaluation can be effective for validating conformance claims and statements about websites." The ability of third party reviewers seems unnecessarily diminished in the language used here. Why would insider knowledge of any website be needed to validate it. The content to user interface is being tested, not the "how a website was developed".

Resolution: Reworded.

Rationale: Reworded section to avoid any unintentional implications.
18 Closed Jason Kiss Re-Running Website Evaluation "Additional web pages" be replaced with "Different web pages" in alignment with the phrasing used in the third bullet point. It might also be useful to note explicitly that the pages in the sample should be updated/changed without the overall sample size needing to increase. The phrase "Additional web pages" in the second list item suggests that extra web pages need to be added to the previous sample, thus increasing the overall number of pages in the sample to be assessed. If the evaluation is run multiple times, the sample size could potentially get unwieldy or impractically large.

Resolution: Reworded.

Rationale: Reworded to better clarify the impact on the overall sample size.
77 Closed Moe Kraft and Mary Jo Mueller Evaluating third party content Keep in line with WCAG wording. This section is about evaluation considerations for this type of content, not how to apply conformance statements which is already clearly covered by WCAG.

Resolution: Reworded.

Rationale: Reworded to better reflect the corresponding tasks for the evaluator in such a context.
78 Closed Moe Kraft and Mary Jo Mueller Evaluating third party content Do you consider links on a web page that link to third party content to be in scope for the evaluation, or is it just embedded content that was created by a third party?

Resolution: No change.

Rationale: (1) does not matter linked vs embedded if still included in scope; (2) if part of a process then definitely in scope; (3) relates to concept of "principle of website enclosure" - if dependency then needs to be evaluated.
97 Closed Raph de Rooij Add one more evaluation context that is related to Evaluating During Development: "Evaluating During Authoring"
  1. This context is linked to ATAG 2.0 (draft) principle B.3: Authors are supported in improving the accessibility of existing content (ref: http://www.w3.org/TR/ATAG20/#principle_b3 )
  2. http://www.w3.org/TR/2013/WD-IMPLEMENTING-ATAG20-20131107/#sc_b315 describes how this particular context can contribute to the evaluation process and be integrated in the methodology.

Resolution: No change.

Rationale: Already sufficiently covered in "Evaluating During Development" which includes all aspects of development (design, authoring, programming, etc.).

Step 1: Define the Evaluation Scope

ID Status Commenter Location Current text Suggested Change Rationale Resolution
70 Closed Kerstin Probiesch Step 1.b: Define the Conformance Target "Note: It is often useful to evaluate beyond the conformance target " Change "often" to "always" ... "the SCs of the next highest level are always useful to evaluate", especially when only A is achieved. Especially when just level A is achieved by the website owner it don't take much time for evaluators to check SCs like...

Resolution: No change.

Rationale: WCAG-EM covers too many different situations to make this a general rule.
98 Closed Raph de Rooij Step 1.b: Define the Conformance Target "Part of initiating the evaluation process is to define the target WCAG 2.0 conformance level ("A", "AA", or "AAA") to evaluate for" A (reference to a) note should be added here that requiring Level AAA conformance as a general policy for an entire site is not recommended. Reference: http://www.w3.org/TR/WCAG20/#conformance-reqs, 1. Conformance Level

Resolution: No change.

Rationale: Setting the conformance target is not directly related to WCAG-EM; It is already covered in WCAG.
71 Closed Kerstin Probiesch Step 1.d: Define Evaluation Methods to be Used (Optional) "W3C/WAI provides a set of publicly documented (non-normative) Techniques for WCAG 2.0 that help evaluate conformance to WCAG 2.0 Success Criteria." Add a link to the document "What would be the negative consequences of allowing only W3C's published techniques to be used for conformance to WCAG 2.0?" (http://www.w3.org/WAI/WCAG20/wcag2faq.html#techsnot) and add a negative example for relying just on the techniques document while evaluating web content. Delete "Some evaluators might use other methods" and write "You are also free to use other methods". After the above cited sentence of WCAG-EM says: "Some evaluators might use other methods". This sentence implies that "most" evaluators are using the WCAG-Techniques. Despite of wether this is fact or not it could be understood in a way that if "just some evaluators" might use other methods it is better to use the Techniques, because "most" are using them.

Resolution: Reworded.

Rationale: Reworded to further clarify the optional use of techniques and failures.
99 Closed Raph de Rooij Step 1.d: Define Evaluation Methods to be Used (Optional) "However, it is good practice to specify the initial evaluation methods to be used during the evaluation" I assume that the evaluator is required to adequately document the methods used, in cases where they are not yet documented elsewhere (for example: in the Techniques for WCAG 2.0). Is this assumption true? If yes, where is it described?

Resolution: No change.

Rationale: In some cases evaluation protocols may be non-public. This is something to decide with the evaluation commissioner in Step 1.e. Documentation is carried out in Step 5.

Step 2: Explore the Target Website

ID Status Commenter Location Current text Suggested Change Rationale Resolution
02 Closed Sham Mitra Throughout this should also include a sample of the target audience Websites are built with an audience in mind, and it is my view that a sample of that audience should be invited to test the product continuously.

Resolution: No change

Rationale: Involving users during accessibility evaluation is not required but can be an additional requirement of the evaluation commission, if specified during Step 1.e.
19 Closed Jason Kiss Step 2c: Identify the variety of Web Page Types Change the word "Content" to "Pages" In the bulleted list items, the phrasing occasionally gets awkward, e.g., "Content with varying types of content", "Content that are", "Content with dynamic content".

Resolution: Reworded.

Rationale: Typos.
64 Closed Kiyochika Nakamura on behalf of Working Group 3 of Web Accessibility Infrastructure Committee in Japan Step 2.c: Identify the Variety of Web Page Types
  • Content that are created using different templates (if this is known to the evaluator);
  • Content that are authored by different people (if this is known to the evaluator);
  • Content that are created using different coding styles;
 Make the word "different" clear, and divide the types of web pages into two categories, such as "required" and "optional". We think that it is difficult to identify the types of web pages because these three sentences are ambiguous, Especially, "different coding styles" needs to be more explained here, or defined as a term.

Resolution: Reworded.

Rationale: Use "varying" instead of "different" to better clarify the intended meaning.
65 Closed Kiyochika Nakamura on behalf of Working Group 3 of Web Accessibility Infrastructure Committee in Japan Step 2.c: Identify the Variety of Web Page Types It seems too much pages to evaluate for the large site and a time-consuming step for the evaluators. Are the types of web page and web page states all required to choose as samples?

Resolution: Reworded.

Rationale: Reworded to better clarify that this stage is only explatory and that the selection is carried out in Step 3.
79 Closed Moe Kraft and Mary Jo Mueller Step 2.c: Identify the Variety of Web Page Types None of the bulleted examples cover what kinds of web page states the evaluator should look for.

Resolution: No change.

Rationale: Any of the bullets could equally apply to web pages as well as web page states.

Step 3: Select a Representative Sample

ID Status Commenter Location Current text Suggested Change Rationale Resolution
40 Closed Marijke van Grafhorst Introductory section Change order of steps 3.b and 3.c To match related steps 2.b and 2.c

Resolution: Changed order.

Rationale: Seems indeed more logical.
62 Closed Marijke van Grafhorst Introductory section What should be the minimum and maximum size of the sample? A quantification would make this section more clear

Resolution: No change.

Rationale: Size depends on website attributes.
21 Closed Jason Kiss Introductory section "The purpose ... the accessibility performance of the website with reasonable confidence." It should possibly refer to an acceptable level of confidence, with the definition of acceptable set by the evaluation commissioner. The methodology might benefit from a discourse on levels of confidence which balances out statistical levels of confidence (which are good for random samples) with the practical reality that sites contain many repeated patterns and practices. The word reasonable is highly subjective.

Resolution: No change.

Rationale: Currently there is no single, practical (to compute), and widely accepted way for defining discrete levels of confidence that applies to all types of websites, so that "acceptable" seems equally subjective as "reasonable".
72 Closed Kerstin Probiesch Introductory section "In cases where it is feasible to evaluate all web pages, this sampling procedure can be skipped and the selected sample is considered to be the entire website in the remaining steps of the conformance evaluation procedure." "feasible" is a critical term ... Please add also "(which is highly recommended)" after "evaluate all web pages". I believe that this pararaph and especially the term "feasible" needs further discussion. I am aware that something like "feasible" is needed - especially when it comes to very huge websites.

Resolution: Added.

Rationale: Increases emphasis.
100 Closed Raph de Rooij Introductory section - list of factors Add "Availability of evaluation findings and test results" If results of accessibility testing are available, it may very well be useful to assess whether such results can be reliably (re-)used in the evaluation. The usefulness depends on factors such as timeliness, completeness and correctness of the information provided, the format in which the results are available (example: EARL), the testing method used, persons that carried out the tests, etc.

Resolution: Added.

Rationale: Useful addition.
80 Closed Moe Kraft and Mary Jo Mueller Introductory section - Website development process Should address that design also plays a factor in the adherence to accessibility conformance. This set of 4 bullets doesn't address that design also plays a factor in the adherence to accessibility conformance.

Resolution: Added references to designers in several bullets.

Proposed Rationale: Useful addition.
101 Closed Raph de Rooij Introductory section - Required level of confidence Clarification is needed. No (reference to) information is available on this subject. Levels of confidence are not defined.

Resolution: No change.

Rationale: Currently there is no single, practical (to compute), and widely accepted way for defining discrete levels of confidence that applies to all types of websites.
87 Closed EOWG Step 3.b: Include Other Relevant Web Pages Consider making those dependencies more clear, and consider reducing such dependencies where possible. Not clear what the web pages are relevant in : "Include all other web pages and web page states that are relevant to people with disabilities and accessibility of the website into the selected sample."

Resolution: No change.

Rationale: The section already points to Step 2.e Identify Other Relevant Web Pages for explanation and more detail.
102 Closed Raph de Rooij Step 3.b: Include Other Relevant Web Pages "other web pages and web page states that are relevant to people with disabilities" It may be helpful here to briefly explain and/or give examples what is meant with "relevant to people with disabilities".

Resolution: No change.

Rationale: The section already points to Step 2.e Identify Other Relevant Web Pages for explanation and more detail.
41 Closed Marijke van Grafhorst Step 3.c: Include Exemplar Instances of Web Pages "The purpose of this step is to have sufficient representation of the different types of web pages and web page states." Please clarify 'sufficient' - a certain amount or a percentage?

Proposed Resolution: Removed "sufficient".

Proposed Rationale: Clarification of text.
42 Closed Marijke van Grafhorst Step 3.d: Include Complete Processes in the Sample Methodology Requirement Remind the reader of the note in step 3.c: "by including the complete progresses in the sample some listed criteria may be adequatly represented without having to include all the progresses identified in this step."

Resolution: Highlighted note in 3.b.

Rationale: To make more prominent.
44 Closed Marijke van Grafhorst Step 3.d: Include Complete Processes in the Sample "locate the starting point (web page or web page state) for the process and replace them in the selected sample" Please clarify 'them': Do you have to replace the chosen web page or web page state with the web page or web page state that contains the starting point for the process?

Resolution: No change.

Rationale: This is what the text says.
22 Closed Jason Kiss Step 3.e: Include a Randomly Selected Sample First sentence "A randomly selected sample of web pages and web page states acts as an indicator to verify that the structured sample selected through the previous steps is sufficiently representative of the content provided on the website." It seems clear from the methodology that the purpose of the randomly selected sample is to buttress and help ensure the representativeness of the structured sample, but this is not clearly expressed in Step 3.e, which confusingly suggests that the randomly selected sample helps to verify that the structured sample reflects the "accessibility performance of the site". But the "accessibility performance of the site" can't be assessed without having first performed the evaluation of the pages included in the total sample. This would make it clearer and more consistent with the language used in Step 4.e.

Resolution: Reworded.

Rationale: Good clarification.
23 Closed Jason Kiss Step 3.e: Include a Randomly Selected Sample If the randomly selected sample is to be 10% of the structured sample, something like 5% randomly drawn from the top 20%(?) of pages accessed, the other 5% selected from the remaining 80%. There may also be better value for money to be gained from skewing such a sample towards most visited pages. A recommendation to this effect could be useful.

Resolution: No change.

Rationale: This introduces a new thread of issues; random should remain random.
106 Closed Annika Nietzio Step 3.e: Include a Randomly Selected Sample "any web page and web page state on the website may be selected with similarly equivalent likelihood" "any web page and web page state on the website may be selected with a certain probability" What does "similarly equivalent likelihood" mean? Not all of the techniques described in this step will result in a uniform sample. The point is, that some randomness must be involved and that any page must have the chance to become part of the sample. Note that "certain" does not necessarily mean "with the same probability".

Resolution: Removed "with similarly equivalent likelihood".

Rationale: Good clarification.

Step 4: Audit the Selected Sample

ID Status Commenter Location Current text Suggested Change Rationale Resolution
01 Closed Karl Groves Methodology Requirement 4.a "Check that each full page (web page and web page state) in the selected sample satisfies each of the WCAG 2.0 Success Criteria of the target conformance level." While the goal is and should remain a full page that is accessible, the testing of the entire page is unnecessary This statement and supporting material make it appear as though an *entire* web page must be tested in its entirety in order to satisfy the requirements

Resolution: Reworded.

Rationale: Section has been rewritten to make it clearer.
47 Closed Marijke van Grafhorst Step 4.a: Check for Each Success Criterion "Check that each full page (web page and web page state) in the selected sample satisfies each of the WCAG 2.0 Success Criteria of the target conformance level." Give a margin of error and make a difference between incidental and structural failures. A year of practice showed that it is impossible to get a 100% result, so this methodology requires something that can't be realized.

Resolution: No change.

Rationale: This document inherits the absolute conformance from the WCAG model.
81 Closed Moe Kraft and Mary Jo Mueller Step 4.a: Check for Each Success Criterion "Check that all features are supported by the web browsers and assistive technologies defined in Step 1.c: Define an Accessibility Support Baseline;" How would you check that all features are supported by the web browser?

Resolution: Reworded.

Rationale: Section has been rewritten to make it clearer.
110 Closed Moe Kraft and Mary Jo Mueller Step 4.a: Check for Each Success Criterion Refine cross-linking between Step 4.a and 4.b The references are somewhat circular

Resolution: Reworded.

Rationale: Section has been rewritten to make it clearer.
25 Closed Jason Kiss Step 4.d: Check for Non-Interference This section would benefit from a brief description of the requirements for non-interference.

Resolution: Reworded.

Rationale: Section has been rewritten to make it clearer.
52 Closed Marijke van Grafhorst Step 4.e: Compare Structured and Random Samples "the randomly selected sample should not show new types of content not present in the structured sample" Shouldn't this be checked during step 3.e?

Resolution: No change.

Rationale: The check is for new things that are discovered in the random sample during the audit and that are not part of the structured sample.
54 Closed Marijke van Grafhorst Step 4.e: Compare Structured and Random Samples "This step is repeated until the structured sample is adequately representative of the content provided on the website" If you have to do this more than once, the conclusion could be that it is not possible for the evaluator to select a representative sample. A suggestion might be to go back to the web developer and ask which types/essential functions are included.

Resolution: No change.

Rationale: Evaluators may not always have access to the developers. It is up to the evaluator to determine what went wrong and how to avoid it.

Step 5: Report the Evaluation Findings

ID Status Commenter Location Current text Suggested Change Rationale Resolution
103 Closed Raph de Rooij Introduction section "to ensure reliable outcomes" "to ensure reliable and verifiable outcomes" For insight in reliability of outcomes, verifiability is a prerequisite.

Resolution: Replaced "reliable" by "verifiable"

Rationale: Better reflects the intended meaning.
06 Closed Sham Mitra Throughout The report should identify 'recommendations for improvement' on which the developer/s can re-work the solution.

Resolution: No change.

Rationale: Repair suggestions are out of scope. Evaluation commissioners could request it as additional requirements (Addressed in Step 1.e.
07 Closed Sham Mitra Throughout Document if any disabled testers were used to evaluate the product

Resolution: No change.

Rationale: Added this to Step 1.e as a possible wish by the evaluation commissioner, which would then be reflected in this step.
109 Closed Annika Nietzio Throughout Explain for each step in which context it is particularly useful. For example Step 5.d will be useful for large-scale evaluations. The reporting options presented in Step 5 b-e depend on the particular evaluation context. Not all steps are necessary in each context.

Resolution: No change.

Rationale: There are too many possible situations in which something could apply or not.
15 Closed Chris Leighton Step 5.a: Document the Outcomes of Each Step I find tabular data effective. Relational would be great too. There are enough criteria to test against for any one URI that recording and perhaps displaying pass, fail or commentary on in other manners could be wasteful.

Resolution: No change.

Rationale: Out of scope for this document. WAI provides Template for Accessibility Evaluation Reports which may be updated in a separate effort with such formatting.
26 Closed Jason Kiss Step 5.a: Document the Outcomes of Each Step The Note For the purposes of re-evaluation in particular, but also to identify persistent weaknesses in knowledge or process, it would be useful to recommend a preference that each repeated instance of a failure be reported back to the evaluation commissioner. Section allows for repeated failures of a Success Criterion to be reported once with one example.

Resolution: Added suggestion.

Rationale: Important clarification.
16 Closed Chris Leighton Step 5.c: Provide an Evaluation Statement (Optional) Optional, why? Without a summary of the test results "Step 1.b: Define the Conformance Target" which is not drafted as being optional, is an unanswered question.

Resolution: No change.

Rationale: The results are provided in Step 5.a, which is non-optional. A public evaluation statement is not a requirement for an evaluation.
104 Closed Raph de Rooij Step 5.c: Provide an Evaluation Statement (Optional) "Reason for not conforming to WCAG 2.0" "Reason for not fully conforming to WCAG 2.0"

Resolution: No change.

Rationale: WCAG 2.0 does not provide a graded conformance model.
13 Closed Miranda Mafe Step 5.d: Provide an Aggregated Score (Optional) Once a site fulfills all of the success criteria, there is an opportunity for a bonus point, to show that a website has gone beyond the sufficient requirements Once two websites pass all the success criteria, they both have the same (highest) score regardless of whether one is an extremely accessible website, while the other has only done the bare minimum of what is necessary.

Resolution: Calculation algorithm removed.

Rationale: Research is ongoing. There is currently no good proposal for a score.
27 Closed Jason Kiss Step 5.d: Provide an Aggregated Score This section should be heavily alarmed with warnings that reinforce the message that accessibility cannot be given a score, and that the real (only?) purpose of such a metric is to provide a unit-less coefficient that can be used to indicate improvement over time. There is a significant risk that 'scores' like this may get used further up management structures as hard metrics, which they are not.

Resolution: Calculation algorithm removed.

Rationale: Research is ongoing. There is currently no good proposal for a score.
28 Closed Jason Kiss Step 5.d: Provide an Aggregated Score "4. Mark the remaining WCAG 2.0 Success Criteria as [not applicable to] any web page or web page state in the sample selected in Step 3: Select a Representative Sample." Changing "not present in" to "not applicable to"

Resolution: Calculation algorithm removed.

Rationale: Research is ongoing. There is currently no good proposal for a score.
59 Closed Marijke van Grafhorst Step 5.d: Provide an Aggregated Score "This methodology proposes two simple scoring approaches to select from" To measure the progress it would be better to mark the failures and successes on each Success Criteria so you know which Success Criteria you have to focus on. Both approaches seem to be inadequate, because it is not retraceable which Success Criteria were met. It is possible to get a lower score with a more accessible website, because the Success Criteria that are not met are less essential.

Resolution: Calculation algorithm removed.

Rationale: Research is ongoing. There is currently no good proposal for a score.
73 Closed Kerstin Probiesch Step 5.d: Provide an Aggregated Score (Optional) "While aggregated scores provide a numerical indicator to help communicate progress over time, there is currently no single widely recognized metric that reflects the required reliability, accuracy, and practicality." Drop the whole "Scoring Section" except the sentence quoted. Because there is no scoring system which fulfill quality criteria like reliability (I'm missing also objectivity and validity) I think that 5d should not even be an optional step.

Resolution: Calculation algorithm removed.

Rationale: Research is ongoing. There is currently no good proposal for a score.
107 Closed Annika Nietzio Step 5.d: Provide an Aggregated Score (Optional) "Aggregated score per web page" The title is misleading. The title of the second procedure "Aggregated score per web page" gives the impression that the calculated number is related to a single web page. But following the procedure will result in a single number (score) for the entire sample (i.e. web site).

Resolution: Calculation algorithm removed.

Rationale: Research is ongoing. There is currently no good proposal for a score.
108 Closed Annika Nietzio Step 5.d: Provide an Aggregated Score (Optional) "A [Draft] W3C Research Report on Web Accessibility Metrics provides more background for different approaches and limitations of scoring metrics." "A [Draft] W3C Research Report on Web Accessibility Metrics provides more background on requirements and different approaches for scores and web metrics." The report is not only focused on limitations.

Resolution: Reworded.

Rationale: Better clarify the content of the document.

Typographical Comments

ID Status Commenter Location Current text Suggested Change Rationale Resolution
08 Closed Miranda Mafe Introduction The methodolody can be used in conjunction with techniques for meeting WCAG 2.0 success criteria, such as... Methodology instead of methodolody Spelling mistake Changed
105 Closed Raph de Rooij Introduction "This methodology, describes the steps" "This methodology describes the steps" The comma has no function and should be removed Changed
09 Closed Miranda Mafe Relation to WCAG 2.0 Conformance Claims Thus in the majority of situations, using this methodology alone does not result into WCAG 2.0 conformance claims for the target websites. Changing sentence to "...using this methodology alone does not result in WCAG 2.0 conformance claims for the target websites." Grammar suggestion Changed
10 Closed Miranda Mafe Step 3 - Methodology Requirement 3 "Website development process — lower adherence requires a larger sample to evaluate; consider the following:" Change "Website development process" to "Adherence to the Website Development Process" To fit with the rest of the titles, as well as clarify to what people are supposed to be adhere to. Changed
11 Closed Miranda Mafe, Jason Kiss, and Marijke van Grafhorst Step 5.d: Provide an Aggregated Score (Optional) "This particular score sensitive towards failures but insensitive towards their relative frequency." Changing sentence to "This particular score is sensitive..." Word missing Changed
12 Closed Miranda Mafe, Jason Kiss, Marijke van Grafhorst, and EOWG Step 5.d: Provide an Aggregated Score (Optional) "For example, a single failure will yield a high score even if that one failire is an inaccessible CAPTCHA that prevents access to the entire website." failure instead of failire Spelling mistake Changed
20 Closed Jason Kiss and EOWG Step 2.e Identify Other Relevant Web Pages Methodology Requirement shouldn't it be 2.e Typo Changed
24 Closed Jason Kiss Step 3.e: Include a Randomly Selected Sample Confidence in the overall evaluation outcome increases when the evaluation results from both selection [approaches] correlate. Typo Changed
29 Closed Marijke van Grafhorst, Kerstin Probiesch and EOWG Step 1.e: Define Additional Evaluation Requirements (Optional) "beyond what is needed to evalaute the extent of conformance" "beyond what is needed to evaluate the extent of conformance" Spelling mistake Changed
30 Closed Marijke van Grafhorst Step 2.c: Identify the Variety of Web Page Types "Content to look for to identify different types of web page and web page states include" "Content to look for to identify different types of web page and web page states includes" Grammar Changed
31 Closed Marijke van Grafhorst and EOWG Step 2.c: Identify the Variety of Web Page Types "Content that are created using different templates (if this is known to the evaluator)" "Content that is created using different templates (if this is known to the evaluator)" Grammar Changed
32 Closed Marijke van Grafhorst and EOWG Step 2.c: Identify the Variety of Web Page Types "Content that are authored by different people (if this is known to the evaluator)" "Content that is authored by different people (if this is known to the evaluator)" Grammar Changed
33 Closed Marijke van Grafhorst and EOWG Step 2.c: Identify the Variety of Web Page Types "Content that are created using different coding styles" "Content that is created using different coding styles" Grammar Changed
34 Closed Marijke van Grafhorst and EOWG Step 2.c: Identify the Variety of Web Page Types "Content that change appearance, behavior, and content depending on the user, device, browser, context, and settings" "Content that changes appearance, behavior, and content depending on the user, device, browser, context, and settings" Grammar Changed
35 Closed Marijke van Grafhorst Step 2.e: Identify Other Relevant Web Pages "Web pages explaining accessibility features of the websites" "Web pages explaining accessibility features of the website" Grammar Changed
36 Closed Marijke van Grafhorst Step 3: Select a Representative Sample "The actual size of sample web pages and web page states needed to evaluate a website depends on many factors including" "The actual size of sample web pages and web page states needed to evaluate a website depends on many factors, including" Grammar Changed to "The actual size of the sample of web pages and"
37 Closed Marijke van Grafhorst Step 3: Select a Representative Sample "How interactive the content is —websites" "How interactive the content is — websites" Typo No change
38 Closed Marijke van Grafhorst Step 3: Select a Representative Sample "or adapt to access devices require larger samples to cover these different situations;" "or adapt to access devices require larger samples to cover these different situations." Typo Changed
39 Closed Marijke van Grafhorst Step 3: Select a Representative Sample "Website development process — lower adherence to procedure requires a larger sample to evaluate;" "Website development process — higher adherence to procedure requires a smaller sample to evaluate;" Editorial Rewrite
43 Closed Marijke van Grafhorst Step 3.d: Include Complete Processes in the Sample "For any web page and web page state selected through Step 3.c: Include Exemplar Instances of Web Pages and that is part of a process..." "For any web page and web page state selected through Step 3.c: Include Exemplar Instances of Web Pages that is part of a process..." Editorial Changed
63 Closed Marijke van Grafhorst Step 3.e: Include a Randomly Selected Sample "Confidence in the overall evaluation outcome increases when the evaluation results from both selection approachs correlate." "Confidence in the overall evaluation outcome increases when the evaluation results from both selection approaches correlate." Spelling mistake Changed
45 Closed Marijke van Grafhorst Step 3.e: Include a Randomly Selected Sample "To perform this selection, randomly select unique instances of web pages and web page states from the target website and that are not already part of the structured sample selected through the previous steps." "To perform this selection, randomly select unique instances of web pages and web page states from the target website that are not already part of the structured sample selected through the previous steps." Grammar Changed
46 Closed Marijke van Grafhorst Step 3.e: Include a Randomly Selected Sample "Note: While the random sample need not be selected according to strictly scientific criteria" "Note: While the random sample needs not be selected according to strictly scientific criteria" Grammar No change
48 Closed Marijke van Grafhorst Step 4.a: Check for Each Success Criterion "check its conformance with each WCAG 2.0 Success Criterion within the target conformance level set inStep 1.b: Define the Conformance Target" "check its conformance with each WCAG 2.0 Success Criterion within the target conformance level set in Step 1.b: Define the Conformance Target" Typo Changed
49 Closed Marijke van Grafhorst Step 4.b: Check for Complete Processes "Behavior using different settings, preferences, devices, and interaction parameters;" "Behavior using different settings, preferences, devices, and interaction parameters." Typo Changed
50 Closed Marijke van Grafhorst Step 4.d: Check for Non-Interference "The WCAG 2.0 guidance on Understanding Requirement 5provides more background on non-interference" "The WCAG 2.0 guidance on Understanding Requirement 5 provides more background on non-interference" Typo Changed/rewrite
51 Closed Marijke van Grafhorst Step 4.e: Compare Structured and Random Samples "While the individual occurences of WCAG 2.0 Success Criteria" "While the individual occurrences of WCAG 2.0 Success Criteria" Spelling mistake Changed
53 Closed Marijke van Grafhorst Step 4.e: Compare Structured and Random Samples "In this case evaluators need to go back to Step 3: Select a Representative Sample) to select additional web pages and web page states" "In this case evaluators need to go back to Step 3: Select a Representative Sample to select additional web pages and web page states" Typo Changed
55 Closed Marijke van Grafhorst and EOWG Step 5.a: Document the Outcomes of Each Step "Evaluation outcomes from Step 4.b: Check for Complete Procceses" "Evaluation outcomes from Step 4.b: Check for Complete Processes" Typo Changed
56 Closed Marijke van Grafhorst Step 5.b: Record the Evaluation Specifics (Optional) "useids" "user-IDs" Typo Changed
57 Closed Marijke van Grafhorst and EOWG Step 5.c: Provide an Evaluation Statement (Optional) "This can be done when at least every non-optional methodology requirement is satisifed, the conformance target defined in Step 1.b. Define the Conformance Target is satisfied by all web pages and web page states audited (in Step 4: Audit the Selected Sample), and the website owner commits to ensuring the validty and maintaining the accuracy of the evaluation statement made." "This can be done when at least every non-optional methodology requirement is satisfied, the conformance target defined in Step 1.b. Define the Conformance Target is satisfied by all web pages and web page states audited (in Step 4: Audit the Selected Sample), and the website owner commits to ensuring the validity and maintaining the accuracy of the evaluation statement made." Spelling mistakes (twice) Changed
58 Closed Marijke van Grafhorst Step 5.c: Provide an Evaluation Statement (Optional) "8. Reason for not conforming to WCAG 2.0: "third-party content" or "lack of accessibility support for languages";" "8. Reason for not conforming to WCAG 2.0: "third-party content" or "lack of accessibility support for languages"." Typo Changed
60 Closed Marijke van Grafhorst Step 5.d: Provide an Aggregated Score (Optional) "For example, a single occurrence of a failure will have smaller effect on the score" "For example, a single occurence of a failure will have smaller effect on the score" Spelling mistakes (twice) Changed
61 Closed Marijke van Grafhorst and EOWG Step 5.d: Provide an Aggregated Score (Optional) "For example, a single failure will yield a high score" "For example, a single failure will yield a low score" none No change/rewrite
82 Closed EOWG Generally EOWG thinks you should *not* link all terms to their definitions; however, if you leave web page states or use another potentially unclear term that is not a well understood term, we recommend that you do link to the definition for the first use of the term in each section. none Revisited use of links.
83 Closed EOWG Particular Types of Websites - Web Applications "Web applications will typically require more time and effort to evaluate, and larger web page samples to reflect the different types of content, functionality, and processes." "Web applications will typically require more time and effort to evaluate, and they will need larger web page samples to reflect the different types of content, functionality, and processes." The sentence is not clear. Changed as suggested
84 Closed EOWG Step 2.d: Identify Web Technologies Relied Upon "During this step the web technologies relied upon (for conformance) to provide the website are identified." "During this step, the web technologies relied upon for conformance are identified." Also for that section, consider moving the WCAG link to the first sentence. Changed as suggested
85 Closed EOWG Step 3: Select a Representative Sample "The purpose of this selection is to ensure that the evaluation results reflect the accessibility performance of the website with reasonable confidence." "The purpose of this selection is to provide reasonable certainty that the evaluation results reflect the accessibility performance of the website." none No change (This would change the intended meaning that has been much discussed to reach this text)
86 Closed EOWG Step 3: Select a Representative Sample "In cases where it is feasible to evaluate all web pages, this sampling procedure can be skipped and the selected sample is considered to be the entire website in the remaining steps of the conformance evaluation procedure." "In cases where it is feasible to evaluate all web pages, this sampling procedure can be skipped and the "selected sample" in the remaining steps of the conformance evaluation procedure is the entire website." none Changed to: "In cases where it is feasible to evaluate all web pages (highly recommended), this sampling procedure can be skipped and the "selected sample" in the remaining steps of the conformance evaluation procedure is the entire website."
88 Closed EOWG Purpose of the Methodology "Periodic evaluation is also useful for monitoring the accessibility performance of websites over time." "Periodic evaluation is important for monitoring the accessibility performance of websites over time." none Changed
89 Closed EOWG Purpose of the Methodology "Web accessibility monitoring activities who want to benchmark or compare the accessibility conformance over time." Change it to be a person to match the others, e.g., "Web accessibility evaluators who want to..." none Changed
90 Closed EOWG Throughout Examples: "Include all common web pages and web page states that were identified in Step 2.a: Identify Common Web Pages of the Website into the selected sample for evaluation." and "Involving Users in Web Accessibility Evaluation provides further guidance beyond the scope of this document." "Include in the selected sample all common web pages and web page states that were identified in Step 2.a: Identify Common Web Pages of the Website." and For further guidance beyond the scope of this document, see Involving Users in Web Accessibility Evaluation. To improve readability, edit so that links to sections and documents are at the end of the sentences, wherever feasible. For many screen reader users, it is difficult to tell where a section title or document title ends when it is in the middle of the sentence. Also, for some sighted users in EOWG, links within sentences decreases readability. Revised use of links
111 Closed Moe Kraft and Mary Jo Mueller Purposes for this Methodology: Fifth bullet says 'Web accessibility monitoring activities who want to benchmark…' Grammar needs to be changed ‘activities’ aren’t a ‘who’. Suggest saying ‘activities used to benchmark ’. Grammatical Changed
112 Closed Moe Kraft and Mary Jo Mueller Relation to WCAG 2.0 Conformance Claims: 2nd paragraph, replace ‘into’ with ‘in’, ‘using this methodology alone does not result into WCAG’ ‘using this methodology alone does not result in WCAG...”. Grammatical Changed
113 Closed Moe Kraft and Mary Jo Mueller Terms and Definitions: Definition for common web pages says: ‘Common web pages may also be web page states in web applications.’ It would be useful if the ‘web page states’ linked to the definition that comes later in the list. Changed
114 Closed Moe Kraft and Mary Jo Mueller Terms and Definitions: Essential functionality definition first note says: ‘Note: Examples of functionality’. Should it be ‘Note: Examples of essential functionality’? On the same note should ‘filling and submitting’ be ‘filling in and submitting’ or instead ‘completing and submitting’? Changed
115 Closed Moe Kraft and Mary Jo Mueller Terms and Definitions: Evaluation commissioner definition: Should ‘monitoring survey’ be ‘monitoring survey owner’? Seems that a monitoring survey is not a person equivalent to a ‘commissioner’. Changed
116 Closed Moe Kraft and Mary Jo Mueller Terms and Definitions: Web page states note: Suggest changing ‘depending on users input’ to ‘depending on the user’s input’. Syntactical Changed
117 Closed Moe Kraft and Mary Jo Mueller Required expertise: Minor edit - Suggest changing ‘This includes understanding of relevant web…’ to be ‘This includes an understanding of relevant web…’ or ‘This includes understanding relevant web…’ Minor Changed
118 Closed Moe Kraft and Mary Jo Mueller Required expertise: Another minor edit since this sentence is so long and contains three ‘and’s – Suggest changing ‘…people with disabilities use, and evaluation techniques, tools, and… to …people with disabilities use, as well as evaluation techniques, tools, and…’ Minor Changed to semi-colon to better distinguish list
119 Closed Moe Kraft and Mary Jo Mueller Required expertise: Last sentence, suggest editing ‘…listed in section Background Reading.’ to say ‘…listed in the section Background Reading.’ Grammatical Reworded
120 Closed Moe Kraft and Mary Jo Mueller Scope of applicability: Examples of websites last paragraph – add ‘of’ to ‘…regardless whether or not…’ so it reads ‘…regardless of whether or not…’ Grammatical Changed
121 Closed Moe Kraft and Mary Jo Mueller Particular types of websites: Suggest simplifying the section title to ‘Types of websites’ Editorial No change
122 Closed Moe Kraft and Mary Jo Mueller Particular types of websites: Website using responsive design is missing its content. Needs replacement before publishing this as a note. Editorial Done
123 Closed Moe Kraft and Mary Jo Mueller Particular Evaluation Contexts: Suggest simplifying the section title to ‘Evaluation contexts’ Editorial No change
124 Closed Moe Kraft and Mary Jo Mueller Particular Evaluation Contexts: Evaluating composite websites - Suggest changing ‘When evaluating websites that are composed of separable areas,’ to be ‘When evaluating websites with separable areas,’ so the link text matches the definition. Editorial Changed
125 Closed Moe Kraft and Mary Jo Mueller Evaluation Procedure: First paragraph says: ‘Some of the activities overlap’. Should that say ‘Some of the activities can overlap’? Changed
126 Closed Moe Kraft and Mary Jo Mueller Step 1.d: Third sentence says: ‘However, it is not necessary to use these particular techniques…’ Suggest replacing ‘necessary’ with ‘required’ so the phrase reads: ‘However, it is not required to use these particular techniques…’ Changed
127 Closed Moe Kraft and Mary Jo Mueller Step 1.d: ‘inline’ Should be ‘in line’ Reworded
128 Closed Moe Kraft and Mary Jo Mueller Step 1.d: 2nd paragraph, first sentence reads easier if stated like this: ‘During this step, particular evaluation methods are defined, if any are to be used.” Reworded
129 Closed Moe Kraft and Mary Jo Mueller Step 2.b: Examples: minor edits. First bullet should say ‘…from the web shop;’ and the second bullet should say ‘Completing and submitting…” Changed
130 Closed Moe Kraft and Mary Jo Mueller Step 2.c: Minor edit to 2nd sentence: ‘They are also often…’ should read ‘They are often…’ Changed
131 Closed Moe Kraft and Mary Jo Mueller Step 2.c: 2nd paragraph, minor edit: rather than ‘…type of web page’ ‘…types of web pages…’ since the adjective is plural, the noun should be plural. Changed
132 Closed Moe Kraft and Mary Jo Mueller Step 2.c: All bullets starting with "Content with..." Suggestion for the bullets is to start each bullet with ‘Web pages…’ rather than ‘Content…’ because there are several instances of ‘Content with …content’ which is a little harder to understand. Reworded
133 Closed Moe Kraft and Mary Jo Mueller Step 2.c: 2nd bullet: Suggest splitting this bullet to two bullets, one talking about structural elements such as forms, tables, lists and headings. Scripting is already covered in the 4th bullet. The second bullet should cover things like embedded content such as multimedia and embedded documents (.pdf, etc.). No change
134 Closed Moe Kraft and Mary Jo Mueller Step 2.c: Bullets 6, 7, and 8 minor edit: In US English ‘content’ is a singular word, so you should use ‘is’ rather than ‘are’ so they read ‘Content that is authored…’ or ‘Content that is created…’ Grammatical Reworded
135 Closed Moe Kraft and Mary Jo Mueller Step 2.c: Next to last bullet minor edit: instead of ‘Content that change appearance’ ‘Content that changes the appearance…’ Grammatical Reworded
136 Closed Moe Kraft and Mary Jo Mueller Step 2.c: Last bullet minor edit: ‘Content with dynamic content…’ should read ‘web' Editorial Reworded
137 Closed Moe Kraft and Mary Jo Mueller Step 2.d: Minor edit in the Note: ‘jQuery’ has a lowercase j. Minor, typo Changed
138 Closed Moe Kraft and Mary Jo Mueller Step 2.e: Methodology requirement number says ‘3.c’ but this is the ‘2.e’ requirement. Typo Changed
139 Closed Moe Kraft and Mary Jo Mueller Step 2.e: First bullet minor edit: ‘…explaining accessibility features of the websites’ should read ‘…explaining the accessibility features of the website’ (singular ‘website’, add ‘the’) Grammatical Changed
140 Closed Moe Kraft and Mary Jo Mueller Step 3.d: Methodology blue box statement: ‘into’ should be ‘in’ to read ‘Include all web pages and web page states… …in the selected sample.’ Changed
141 Closed Moe Kraft and Mary Jo Mueller Step 3.d: Suggest 1st sentence after blue box state: ‘…that belong to a series of actions used as part of a complete process…’ for clarity. No change
142 Closed Moe Kraft and Mary Jo Mueller Step 3.d: For clarity, Suggest 2nd sentence after blue box state: ‘…part of a process, unless all other web pages and web page states that are part of that process are included in the selected sample. Changed
143 Closed Moe Kraft and Mary Jo Mueller Step 3.d: Next sentence before the numbered list suggest changing ‘into’ to ‘in’.: to read ‘in the sample’ Changed
144 Closed Moe Kraft and Mary Jo Mueller Step 3.d: Number 1 bullet suggest removing ‘and’ in ‘…and that is part of… to ‘…that is part of…’ Changed
145 Closed Moe Kraft and Mary Jo Mueller Step 3.d: I was confused by the word ‘replace’ because does this mean one web page was in there and you are replacing it with another? If not, suggest changing ‘process and replace them’ to read ‘process and include them' Changed
146 Closed Moe Kraft and Mary Jo Mueller Step 3.d: Number 2 bullet first sentence: change ‘into’ to ‘to’. Changed
147 Closed Moe Kraft and Mary Jo Mueller Step 3.d: Number 3 bullet note’s example: add ‘the’ to read ‘…without changing the contents of the shopping cart…’ Changed
148 Closed Moe Kraft and Mary Jo Mueller Step 3.d: Number 4 bullet 2nd sentence: Change ‘into’ to ‘to’. Changed
149 Closed Moe Kraft and Mary Jo Mueller Step 3.e: 2nd to last word in the first paragraph after the methodology box: Fix spelling of ‘approaches’. Typo Changed
150 Closed Moe Kraft and Mary Jo Mueller Step 4b Methodology statement suggested edit: Use ‘used in ‘ instead of ‘along’ to read ‘…web page state used in a complete process…' No change
151 Closed Moe Kraft and Mary Jo Mueller Step 4d Methodology statement suggested edit: add ‘for’ to the last phrase in the sentence to read ‘…requirement for non-interference.’ Same comment for first sentence of the next paragraph. Reworded
152 Closed Moe Kraft and Mary Jo Mueller Step 5 Recommend the title use ‘Document’ rather than ‘Record’ and the use of ‘recording’ and ‘recordings’ be changed to ‘document’ and ‘documentation’ to match terminology used in subsequent sections. Changed to "report"
153 Closed Moe Kraft and Mary Jo Mueller Step 5b 4th bullet minor edit: Suggest changing ‘arrive to’ to ‘navigate to’ and ‘useids’ to ‘userids’. Grammar/Typo Changed
154 Closed Moe Kraft and Mary Jo Mueller Step 5c Edit the reminder: change ‘into’ to ‘in’ to read ‘…does not result in WCAG2.0 conformance…’ Also, shouldn’t ‘websites’ be ‘website’ in the same statement? Grammatical Changed
155 Closed Moe Kraft and Mary Jo Mueller Step 5c Paragraph under reminder: Fix spelling of ‘validity’ from misspelled ‘validty’ Typo Changed
156 Closed Moe Kraft and Mary Jo Mueller Step 5c Sentence before number 7 bullet: Says ‘have to also’, so is this really a ‘must’ or a ‘should’. Prefer that type of WCAG language. Use WCAG wording. Reworded
157 Closed Moe Kraft and Mary Jo Mueller Step 5d Numbered bullet 4 should only be marked ‘not present’ if no web pages in the entire sample have content that applies to that success criteria. (e.g. multimedia success criteria is evaluated, but no web pages in the sample contain multimedia content.) I think it is clearer to state it that way. Reworded
158 Closed Moe Kraft and Mary Jo Mueller Step 5d Paragraph after numbered bullets: Add ‘is’ to ‘This particular score is sensitive…’ This particular score is sensitive…’ Grammatical Reworded
159 Closed Moe Kraft and Mary Jo Mueller Step 5d Aggregated score, last sentence: Correct spelling of ‘failure’ from ‘failire’. Correct spelling of ‘failure’ from ‘failire’. Typo Reworded
160 Closed Moe Kraft and Mary Jo Mueller Contributors section: Change ‘Mary-Jo’. to ‘Mary Jo’ – use a space instead of the hyphen. Typo Changed
161 Closed Moe Kraft and Mary Jo Mueller Introduction, 1st paragraph The activities carried out within these steps are influenced by many aspects such as the type of website (e.g. static, dynamic, responsive, mobile, etc.), its size, complexity, and the technologies used to create it (e.g. HTML, PDF, etc.), Replace the last "it" with "the website" Just a picky editorial comment. "It" is a little bit far from the last "website" so it takes the brain a little work to relate. Changed
163 Closed Moe Kraft and Mary Jo Mueller Introduction, 2nd paragraph It supports common approaches and understanding for evaluating the extent of conformance of websites to WCAG 2.0, though in the majority of use cases it does not directly result in conformance claims. Maybe add "This methodology" one more time. This methodology supports common approaches and understanding ... Editorial Changed
164 Closed Moe Kraft and Mary Jo Mueller Principles of Website Enclosure See also the definition for Common Web Pages. Provide a link to Common Web Pages. http://www.w3.org/TR/WCAG-EM/#common Missing link Changed
165 Closed Moe Kraft and Mary Jo Mueller Step 2: Explore the Target Website Under first Note, it would be good to refer to Easy Checks again. http://www.w3.org/WAI/eval/preliminary Informational No change
166 Closed Moe Kraft and Mary Jo Mueller Step 2.d: Identify Web Technologies Relied Upon Add WAI acronym to the Title for WAI-ARIA. Add Title for PDF Web Accessibility Initiative. Portable Document Format Editorial Added for PDF
167 Closed Moe Kraft and Mary Jo Mueller Steps 4 a and 4b 4a: For each web page and web page state in the sample selected in Step 3: Select a Representative Sample that is not within or the end of a complete process, check its conformance with each WCAG 2.0 Success Criterion within the target conformance level set ... 4b: For each complete process selected in Step 3.d: Include Complete Processes in the Sample, follow the identified default and branch sequences of web pages and web page states, and evaluate each according to Step 4.a: Check WCAG 2.0 Success Criteria. Something is bugging the programmer in me. In 4a, we say to skip pages that are parts of complete processes and in 4b for pages in complete processes we send you back to 4a. Maybe instead of saying "that is not within or the end of a complete process", we have a statement to the effect that "for web pages that are part of complete processes, see Step 4b to identify default and branch sequences of web pages and web page states..." Editorial Reworded
168 Closed Moe Kraft and Mary Jo Mueller Step 4e Methodology Requirement 4.e: Check that each web page and web page state in the randomly selected sample does not show types of content and outcomes that are not represented in the structured sample. Methodology Requirement 4.e: Check that each web page and each web page state in the randomly selected sample do not show types of content and outcomes that are not represented in the structured sample. Editorial Changed
169 Closed Moe Kraft and Mary Jo Mueller Step 2.e: Methodology requirement minor edit: Missing ‘the’ in ‘…and to accessibility of the website.’ It should read ‘…and to the accessibility of the website.’ Same comment in the next sentence outside of the blue box. Grammatical Changed