W3C

Site Navigation

Nearby

Results of Questionnaire Scripts for Evaluation Intro Videos

The results of this questionnaire are available to anybody. In addition, answers are sent to the following email address: shadi+EOsurvey@w3.org

This questionnaire was open from 2019-08-14 to 2019-09-03.

10 answers have been received.

Jump to results for question:

Introduction
Video 1: Evaluation Overview
Video 2: Preliminary Evaluation
Video 3: Selecting and Using Tools
Video 4: Comprehensive Evaluation
Video 5: Involving Users
Other Comments

1. Introduction

This survey is to collect input from EOWG on the scripts for the evaluation intro videos. Please provide your input in this form and do not edit the wiki directly to avoid confusion.

This is a detailed review, including of grammar, spelling, and wording preferences (especially for the spoken audio). For the visuals, please note further ideas you may have, which we will provide to the video production company as input.

Please enter in the comment field below by when you plan to complete this survey if you cannot complete it by the deadline of 1 September 2019.

Details

Responder
Laura Keen
Eric Eggert
Helen Burge
Hidde de Vries
Daniel Montalvo	As a general comment, I think we need to find a replacement for the word "cursory". It is strange to me and I think there are synonyms that people can understand more broadly. Maybe in some cases "quick", in some others "first" or "preliminary", "easy"...
Estella Oncins
Sharron Rush	Overall, this is a good start. In some cases however, I find the tone to be too academic, the language a bit too stiff and formal. There are different audiences for various resources and that does not seem to be reflected in the tone. For example, Easy Checks should be much more informal, assume nothing about the tech proficiency of the viewer. When the audience is meant to be developers, less formal languages is recommended as well. More practical, simple straight forward guidance. For the policy makers and managers the more formal language is probably appropriate.
Vicki Menezes Miller	September 3
Brent Bakken	Complete
Shawn Lawton Henry	Completed through review of narratives/"Audio". Didn't get to "Visual".

2. Video 1: Evaluation Overview

Please carefully review the script for Video 1: Evaluation Overview.

Please indicate any comments you have on:

The spoken audio
The illustrative visuals
The overall duration
Anything missing?
Anything to remove?
Any other observations?

Details

Responder	Video 1: Evaluation Overview
Laura Keen
Eric Eggert	Both, the Tools List and Easy Checks will change soon-ish (The former to be incorporated into the “new” design and the latter because it is in our update schedule). Having readable screenshots in the video will make the video outdated really quickly. We have to make sure to be able to replace those sequences easily with new material once we update the pages. I think this video is too long and introduces too many resources. Text in the illustrations is a i18n issue. (If we do it, we need clear instructions for translators to re-create videos in their language.) --- The narration has a lot of complicated words, especially for an introductory video there is not enough time to really understand one concept until a new one is introduced. There is also some repetition (accessibility, of course, but also evaluation and methodology in 16–19). Attempt to simplification as an example: Current: > Often you will need to carry out a more comprehensive evaluation of your products and services > WAI provides an evaluation methodology that you can follow, or ask your vendor to follow > Along with that WCAG-EM evaluation methodology you can also find an open source report tool > This allows you to record the findings as you follow the methodology, to provide a consistent report Simplified: > Often your products or services must be completely tested. > You, or your vendor, can follow the evaluation methodology called “WCAG-EM”. > A report generator is available, free to use, that helps recording the findings and provide a consistent report. --- The wording seems often unclear to me. Not having consistent periods at the end of a sentence makes reviewing it hard (because of the ambiguity: Does the sentence end at the end of the scene or carry over to a new scene). --- > Illustrations of "users"/"people" appear (do not need to be detailed or animated); no representations of "disability" or such - just of users (see sequence 21 of video 1) STRONGLY OBJECT to not representing disabilities when talking about users. We need not to point out disabilities specifics here, but whenever we refer to users we must make sure to be representative. When I think of “just users”, I cannot separate PwD from people without disabilities. They are all “just users”.
Helen Burge	Looks good to me
Hidde de Vries	* I think we could more explicitly bring across the point that accessibility is exponentially more expensive when added after the site is finished; maybe something like “Integrating accessibility from the start is likely to be much more cost effective than adding it later.” * I had to look up “cursory” in “cursory idea”, this word might be problematic more other non-native speakers, too?
Daniel Montalvo	Add back? It sounds strange to me. Add back at later stages -> add later
Estella Oncins	Great work! Not sure if the concept "functional requirements" will be understood. Maybe something more simple and easy to relate to accessibility: TEXT: "For example, is accessibility included in the functional requirements of your project?" SUGGESTION SOMETHING LIKE: "For example, are Accessibility Principles included in your project? Also for the visuals I would not use the word "Requirements" I would rather prefer "Accessibility principles" Not sure if the concept "comprehensive evaluation" is clear, users may not know what is mean and what it involves.
Sharron Rush	- Sequence 2: Seems to assume a level of knowledge of terms that may not be realistic. Suggest instead “Evaluation of web products and services means confirming that accessible design and development practices are followed. This is most effective when done early and throughout the process.” - Sequence 5: “to add back” is an awkward phrase and is not needed. Suggest omit it entirely so it says only “costly repairs at later stages.” - Sequence 6: Suggested revision, “Throughout design phases, good designers will continue to evaluate how well layout, colors, and interactions are meeting accessibility requirements.” - Sequence 7: Suggested, “…coding and mark-up of all programmed elements including forms, widgets, and applications.” - Sequence 8: Suggested: “Content authors will consider and verify the accessibility of text, images, and multi-media as they prepare these items for publication.” - Sequence 9: Suggested: “And so forth. When everyone involved throughout the design and development process evaluates accessibility early and throughout, outcomes are greatly improved.” - Sequence 10: Suggested “However, tools cannot automatically check everything. Some require human judgement.” - Sequence 22: Suggested “Accessibility evaluation often leans towards technical aspects and overlooks the actual experience of the end user.”
Vicki Menezes Miller	Seq.5: Not very clear. Try to re-word without "add back". Seq.6: Is "later on" necessary? Seq.10: Is "these" in "these different types of evaluations" necessary. I would remove it. Or, simplify to "There are tools which can help you evaluate accessibility." Seq.13: "without much prior technical skills" is a bit of a mouthful. Anyway to simplify it. Suggestion "Several checks can be carried out without much prior accessibility knowledge, technical skills or expertise". Seq 14: "follow such checks" - I'm not sure "follow" is the right word. Could it be replaced with "perform" (as carry out is used in the previous Seq.) Seq 16: Is it "Often" or "Eventually"? Seq 21: Why is there no representation of persons with disabilities? Seq 22: The referred to "users" - Are there persons with disabilities? Seq 25: For the visual, I think, in this instance, a variety of users would be good in different situations to show that accessibility impacts everyone. Not sure that the web page with errors will be sufficiently effective.
Brent Bakken	[ED] Seq. 2 - Audio: "Ensuring your products and services are accessible means evaluating them early and throughout" - Wondering if the word "throughout" will be understood. Some may think it means the entire site, product, or service instead of throughout design, development, and continued maintenance. Don't know how to say all that quickly though. Maybe not needed as the following content explains it anyway. [ED] Seq. 5 - Audio: Do the words "to add back" need to be in the script? Doesn't read well to me. Maybe, "If not, then accessibility may get overlooked and require costly [unexpected] repairs at later stages [ED] Seq. 21 - Audio: Replace "we" - "Throughout all these evaluation tasks, [one] should never forget about the actual end-users" [ED] Seq. 22 - Audio: Suggested edit - "Accessibility evaluation often leans towards technical aspects whereby [end-user experience] may be neglected"
Shawn Lawton Henry	Evaluating Web Accessibility: Overview. (capitalize) "Ensuring your products and services are accessible means evaluating them early and throughout" SLH: Maybe something along the lines of: "When developing or redesigning a website or web application, evaluate accessibility early and throughout the development process. This helps you identify potential accessibility problems early, when it is easier and less expensive to address them." --- "Evaluation starts from the on-set of a project, even before the design phase" "For example, is accessibility included in the functional requirements of your project?" "If not, then accessibility may get overlooked and require costly repairs to add back at later stages" SLH: I don't think this is 'evaluation'. That doesn't fit here. You might want to say something like: "Accessibility should be included from the on-set of a project, for example in the functional requirements. Evaluating accessibility can start early in the design phone." --- "Later on, designers need to continually evaluate how layout, colors, and interactions meet accessibility requirements" "Developers need to continually evaluate the coding and mark-up, for example of forms, widgets, and applications" "Content authors need to continually evaluate accessibility of the text, images, and multi-media they are publishing" "And so forth. Everyone involved throughout the design and development process needs to evaluate accessibility early and throughout" SLH: This doesn't feel right. It feels overwhelming and overstated. SLH: Also, these specifics – while interesting – go beyond what we provide in our evaluation resources, so I'm not sure about including them in this video. --- SLH: I think early in the video, say something like: "The W3C Web Accessibility Initiative, WAI, provides several free resources to help you with your evaluation." --- … "However, tools cannot automatically check everything and require human intervention" SLH: "However, tools cannot automatically check all aspects of accessibility. You'll need knowledgeable people to check everything." --- "The WAI website provides a section on "Evaluation Tools" with a list of different tools and guidance on selecting tools" SLH: "The WAI website provides a filterable list of Web Accessibility Evaluation Tools" and guidance on selecting tools." --- "Several checks can be carried out without much prior technical skills or accessibility expertise" "For example, you can follow such checks to get a cursory idea of the accessibility of your products and services" "You can find a collection of such checks under the menu item "Easy Checks: A First Review" on the WAI website" SLH: Need smoother transition, like: "Even if you don't have tools and accessibility knowledge, there are a few things you can check to help you get an idea if some basic accessibility has been addressed in the web page. Step-by-step guidance is provided in the resource: "Easy Checks: A First Review of Web Accessibility". --- "Often you will need to carry out a more comprehensive evaluation of your products and services" SLH: "Often you will need to do a comprehensive evaluation of products and services – yours other vendors." "WAI provides an evaluation methodology that you can follow, or ask your vendor to follow" "Along with the evaluation methodology you can also find an open source report tool" SLH: "Along with the evaluation methodology, WAI also provides an open source report tool" "This allows you to record the findings as you follow the methodology, to provide a consistent report" SLH: last phrase not needed "… to provide a consistent report " "Look for these free resources under the menu item "Conformance Evaluation" on the WAI website" SLH: "These WAI resrouces are called "WCAG-EM: Website Accessibility Conformance Evaluation Methodology" and "WCAG-EM Report Tool: Website Accessibility Evaluation Report Generator". --- "Throughout all these evaluation tasks, we should never forget about the actual end-users" "Accessibility evaluation often leans towards technical aspects whereby end-users may be neglected" "Find more guidance on "Involving Users" in the same area on "Evaluation" on the WAI website" SLH: "Evaluation often focuses on meeting standards and technical implementation. It's vital to keep in mind that the goal of accessibility is making products and services usable by people with disabilities. To help you evaluate the user experience, WAI provides guidance in the resource: "Involving Users in Evaluating Web Accessibility". To find these evaluation resources and other introductory videos, visit w3.org/WAI/eval "

3. Video 2: Preliminary Evaluation

Please carefully review the script for Video 2: Preliminary Evaluation.

Please indicate any comments you have on:

The spoken audio
The illustrative visuals
The overall duration
Anything missing?
Anything to remove?
Any other observations?

Details

Responder	Video 2: Preliminary Evaluation
Laura Keen
Eric Eggert	See video 1, also there seems to be a lot of repetition from video 1. I’d like to have this more stand alone.
Helen Burge	Looks good to me
Hidde de Vries	* In 5, should we show a real example of a browser plugin rather than just cogwheels?
Daniel Montalvo	Minor suggestion: You do not have to carry out all checks. Sometimes carrying out only a few checks and give you a general idea of how well accessibility was considered. -> You do not have to carry out all checks. Sometimes carrying out only a few checks gives you a general idea of how well accessibility was considered.
Estella Oncins	Great work! I am not sure if the constructions "cursory overview" or "cursory checks" is easy to understand specially for people like me non-native English speakers.
Sharron Rush	This approach is quite pedantic. Repetitive phrase "carry out" when applied to the Checks seems a bit awkward. Suggest a lighter, more high level approach.
Vicki Menezes Miller	Seq2: see comment on Seq. 13 above. Seq5: suggestion to remove "small". Suggested reword, change from "Other checks may require installing small plugins for your browser or using other browser functionality" to "Other checks may require installing plugins for your browser or using available browser functionality." Seq6: "Sometimes carrying out only a few checks and give you a general idea of how well accessibility was considered." Editorial: change "and" to "can"- Seq7: "Just remember that even if you carry out all cursory checks, that it is not a full conformance evaluation". Suggest to delete "that". Or, suggested modification "Just remember that even if you carry out all cursory checks, this does not correspond to a full conformance evaluation". Seq9: I would delete "Yet".
Brent Bakken	[ED] Seq. 4 - Visual: Is "My Homepage" supposed to represent an accessible page title or an inaccessible page title? Some would see that and think that is completely accessible while others will think it too generic. Maybe use something even more accessible (like "Home - XYZ Bank") or something even more inaccessible (like "HomeLJ3#B") as the example (depending on which you are trying to show). [!!] Seq. 6 - Audio: Typo... "You do not have to carry out all checks. Sometimes carrying out only a few checks [can] give you a general idea of how well accessibility was considered." [ED] Seq. 7 - Audio: What is spoken here is clear to me because I have heard this many times before. I am not sure that this will be clear to someone who is looking at this and doing checks for the first time. I think there is a better way to say it. Maybe something like, "Also keep in mind, using all of these cursory checks is not a full conformance evaluation. More testing should be formally conducted for full accessibility conformance"
Shawn Lawton Henry	Rather than commenting point-by-point on the draft, I took a pass at a somewhat different approach -- for discussion as you wish. Easy Checks for Evaluating Web Accessibility. Even if you don't know anything about web accessibility, you can figure out if a web page has some accessibility barriers. Even if you don't have evaluation tools, there are a few things you can check. The resource, "Easy Checks: A First Review of Web Accessibility" gives you step-by-step guidance for checking some aspects of accessibility. These checks cover only a few issues, and don't necessarily give you definite answers. Yet they definitely help you get an idea if some basic accessibility has been addressed in the web page. Most of the checks you can do with any web browser. Some of the checks are easier if you have an extension downloaded for your browser. You can use these checks as a first step to knowing how your web pages are doing on accessibility. Or other web pages, for example, of vendors you might want to work with. Make sure to read the "Next Steps" section of the resource for guidance on more robust evaluation, including with users. For the link to Easy Checks and other evaluation resources, visit w3.org/WAI/eval

4. Video 3: Selecting and Using Tools

Please carefully review the script for Video 3: Selecting and Using Tools.

Please indicate any comments you have on:

The spoken audio
The illustrative visuals
The overall duration
Anything missing?
Anything to remove?
Any other observations?

Details

Responder	Video 3: Selecting and Using Tools
Laura Keen
Eric Eggert	This video talks too much about anything else but checking tools before it talks about checking tools. Much of the “intro” is covered in video 1. I would at least suggest to put the latter part of the video earlier. --- The narration focuses on tool features and then on evaluation approaches before getting back to the features of tools. I think it is more coherent if we focus on features first and then go to approaches for testing. --- The narration could be more to the point. Don’t be afraid to use lists. Not everything needs to be a sentence. --- 27 seconds in is to late to mention the major caveat of automatic tools: “Yet such automated checks only address a smaller portion of the accessibility checks you need to carry out” Why not start the video with “Tools to assess website accessibility can save you a lot of time and effort but only cover a portion of what is testable.” You could then go on with “Here’s what’s important when choosing a tool and what the limitations can be.” --- The video is also too similar to video 1 – it would be good to see tools working, for example in your illustrated browser, clicking a button and highlight an image without an alt attribute, or having horizontal scrolling. Or a video without captions. While being more conceptual is ok for video 1, I think these need to be more practical.
Helen Burge	Looks good to me
Hidde de Vries	* For tools that automatically evaluate accessibility, I've gotten a lot of team members at clients excited when I mentioned they could integrate with CI/CD (continuous integration / continuous deployment), things like when a new Pull Request is created, axe (or something like it) runs and prevents merging as long as there are issues. This is great or awareness (as it is quite in your face, and, in fact, in the face of anyone trying to change code in a given codebase). Usually similar checks already exist for CSS/JS code quality * Maybe instead of “false results” we could speak of “false positives”?
Daniel Montalvo	Minor suggestion: For example, some tools can check certain accessibility aspects fully automatically -> For example, some tools can fully check certain accessibility aspects automatically
Estella Oncins	Great work! Not sure about the technical level that users of this video should have. Does the video addresses developers as well as content authors? Not sure if it will be good to provide an example for each group.
Sharron Rush	Shorter narration, plainer language for Editor's discretion. Evaluating web sites and applications for accessibility can seem overwhelming and time consuming, but it doesn't have to be that way. Tools are available to help make the job easier. Automated tools can check several web pages and summarize the outcome. This can be useful to maintain accessibility on a website and to prioritize what to repair first It's important to know, however that automated checks only address a small portion of the accessibility checks that are needed to ensure accessibility. Tools are available to support manual evaluation as well. For example, a tool could identify the page title and allow you to decide if the page is correctly named. Some tools help you collect results of automated and manual checks, by generating a bug list or evaluation report. Tools can also integrate differently into your work environment. For example, some tools integrate directly into your web browser. Other tools may support content authors within a content management system or support developers within a java-script framework. Some tools provide all these functions while others focus on specific evaluation tasks. (no edits to closing frames)
Vicki Menezes Miller	I feel that this part is overwhelming and the story doesn't tie up as neatly as the previous ones maybe simply because it seems quite long.
Brent Bakken	[ED] Seq.3 - Audio: "For example, some tools can check certain accessibility aspects fully automatically" does not read right to me. It has something to do with the word "fully" next to the word "automatically." Try one of the following... "For example, some tools can check certain accessibility aspects automatically" or "For example, some tools can fully check certain accessibility aspects fully automatically" [!!] Seq. 4 - Visual: Typo. Change "class" to "glass" [ED] Seq. 9 - Audio: Remove the word "these" [idea] Seq. 10 or 11 - Audio: Maybe the "environment" could be a developers work screens. Maybe a zoom into two monitors that a developer is doing some design coding on, then zooms into the detail of one of the screens where there is a browser add on tool being used, in her technical environment. [medium/strong] Seq. 14 - Visual: A hammer and screw driver remind me of tools that do the work, like HTML or CSS. Maybe instead of those, use tools like a level, tape measure, survey tools instead as these "construction" tools actually do testing to see if there is an issue. Kind of makes the analogy better I think.
Shawn Lawton Henry	Rather than commenting point-by-point on the draft, I took a pass at a somewhat different approach -- for discussion as you wish. Tools for Evaluating Web Accessibility. Web accessibility evaluation tools are software programs or online services that help determine if web content meets accessibility standards. Some tools are free online and there are more robust tools for purchase. The tools can save lots of time and effort figuring out accessibility barriers in a website – or even preventing some from being introduced. However, no tool can do it all. Some accessibility checks just cannot be automated. They require a knowledgeable human to do manual evaluation. Let's look at what tools can do. Most tools to automated checks and some help with manual evaluation. Some tools check one page, and some tools check an entire website. Some tools integrate into your content management system or JavaScript framework and evaluate during development. Tools produce different types of reports, for example, many tools prioritize the accessibility barriers they identify. Some tools you can customize for your environment. Two more caveats about tools: First, be aware that some tools provide inaccurate results. Second, remember that your goal is to provide accessible content that works well for users and meets standards. Be careful not to rely too much on evaluation tool results over real-life accessibility in practice. To learn more about what tools can do and which tools you might want to use, see the free WAI resource: "Selecting Web Accessibility Evaluation Tools". You can use WAI's database of over 100 tools to get information on specific tools available. For the links to the Web Accessibility Evaluation Tools List, the Selecting guidance, and other evaluation resources, visit w3.org/WAI/eval

5. Video 4: Comprehensive Evaluation

Please carefully review the script for Video 4: Comprehensive Evaluation.

Please indicate any comments you have on:

The spoken audio
The illustrative visuals
The overall duration
Anything missing?
Anything to remove?
Any other observations?

Details

Responder	Video 4: Comprehensive Evaluation
Laura Keen
Eric Eggert	8/9 start with the same, complicated words: > Conformance evaluations require more technical skills to understand the code and the accessibility requirements > Conformance evaluations also require understanding of assistive technologies and the impact of accessibility barriers on people with disabilities Why not just: > Conformance evaluations require technical skills to understand the code and the requirements as well as > an understanding of assistive technologies and the impact of accessibility barriers on people with disabilities (note: I removed “more” from “more technical” as a filler word. There are a lot of filler words in the texts.) --- > In some cases evaluation tasks are distributed across a team according to the skills of the individuals I’ve never seen that working out. If designers are involved, they usually either make an accessible design or they do not, the usually don’t come around and then evaluate the design. Because if they could do that they would make it accessible in the first place. --- > Conformance evaluations are most difficult the first time, when you are starting with accessibility Everything is difficult when you do it for the first time. Unnecessary obvious :-) --- > If you implement accessibility quality management, subsequent conformance evaluations will become easier This whole video lacks that a “Comprehensive Evaluation” is the result of not including accessibility in the day-to-day workflow. If you implement “accessibility quality management” (needs a definition or a link!), you would never need a “Comprehensive Evaluation” because your components would be intrinsically accessible. --- In general I am noticing that we are using the phrase “Web accessibility: essential for some, useful for all” again, which I like (New WAI tag line? Just teasing!) but 14 seconds sounds like a lot of time just for that sentence.
Helen Burge	Looks good to me
Hidde de Vries	I would expect something about repeating comprehensive evaluation, advice like “re-evaluate every month / year / decade”. Should we include that, and if so, might this video be the right place?
Daniel Montalvo
Estella Oncins	"Comprehensive Evaluation" for me it is a bit confusing because the first sequence is "Evaluating web accessibility: conformance evaluation" and then the script talks about "conformance evaluation" Seq 3: I would add "requirements linked to the accessibility principles" at the end of the "Conformance evaluations require more technical skills to understand the code and the accessibility requirements"
Sharron Rush	[[ Edit suggestions for clarity, brevity and plain language (editor's discretion) ]] A full conformance evaluation of a website or application is required in many cases. For example, before releasing or before procuring a digital product or service, to ensure accessibility requirements are met. Or when you begin to implement accessibility within a system, to help you prioritize what you need to repair first For when you need a comprehensive approach, WAI created WCAG-EM, an evaluation methodology for the Web Content Accessibility Guidelines. WCAG-EM defines a process with five steps: Define the scope of your evaluation; explore your website assets; select a representative sample of web pages; evaluate the selected sample; and report your evaluation findings WAI includes an open source report tool, so that outcomes from these steps are documented in a consistent report Conformance evaluations require technical skills to ensure understanding of the code and the accessibility requirements Conformance evaluations also require understanding of assistive technologies and the impact of accessibility barriers on people with disabilities In some cases evaluation tasks are distributed across a team according to the skills of the individuals For example, designers can evaluate how layout, colors, and interactions meet accessibility requirements Ideally someone in the evaluation team has background in usability, to understand how to evaluate the end-user experience by involving real end-users Conformance evaluations are most difficult the first time, when you are starting with accessibility If you implement accessibility quality management, subsequent conformance evaluations will become easier In all cases, make sure you include consideration of the end user experience rather than on technical requirements alone Web accessibility: essential for some, useful for all Visit w3.org/WAI for more information on web accessibility evaluation
Vicki Menezes Miller	Seq3: Keep it simple. Just provide one example, forget the "or.." Sequence: Maybe Seq4 could come before Seq3
Brent Bakken	[ED] Seq. 4 - Audio: Suggested edit, "Also if you are just starting to work on accessibility, to help you prioritize what you need to repair first" [Question] Seq. 5 - Audio: For the use of "WAI," is this going to be pronounced "way" or is it going to be said "web accessibility initiative? Please clarify in the script. [ED] Seq. 6 - Visual: I completely agree to not have the arrows in the 5 steps images. I do however think it is important to visual convey that each step follows the previous one. Maybe not with arrows but some kind of fading and transparency of the graphics. So that the viewer understands this is a stepped process workflow. [idea] Seq. 8 - Visual: Maybe behind the graduation cap there is some html code scrolling slowly in the back, kind of faded so that the hat takes focus. [idea] Seq. 10 - Visual: Continuing from #8 visual... Maybe behind the scroll/certificate there is some site with a pointer moving to different sections slowly in the back, kind of faded so that the scroll/certificate takes focus. [idea] Seq. 11 - Visual: What if each of those web pages floats on the screen to a separate colored person icon (example icon: https://www.bangory.org/staff/kaly-hutchinson/person-icon/). Maybe 4 or 5 of them, to show that each person is working on a different part.
Shawn Lawton Henry	Rather than commenting point-by-point on the draft, I took a pass at a somewhat different approach -- for discussion as you wish. Evaluating Web Accessibility Conformance. Conformance evaluation determines how well web pages or applications meet specific accessibility standards. Often conformance evaluations are done: * as a final check before releasing a product * in order to provide information to potential purchasers of your product * before procuring a product * when getting started on accessibility, to get a thorough list of accessibility issues that you need to address Accurate, effective conformance evaluation requires knowledge of the accessibility standard, accessible web design and development, assistive technologies, and how people with different disabilities use the Web. You'll probably also want to also use evaluation tools to make it more efficient. To help you evaluate conformance, WAI provides an evaluation methodology for the Web Content Accessibility Guidelines, called "WCAG-EM". WCAG-EM describes a process to: 1. define the scope of your evaluation 2. explore your website assets 3. select a representative sample of web pages from your website 4. evaluate the selected sample 5. report your evaluation findings WAI also provides an open source report tool that helps you follow the steps of WCAG-EM, record the outcomes, and get a generated report. One thing to note here: Conformance evaluation focuses on meeting standards. That should be one aspect of your accessibility efforts. Fundamentally, accessibility is about making your web content usable by people with disabilities. Other WAI resources help you address that aspect. For the links to "WCAG-EM: Website Accessibility Conformance Evaluation Methodology", "WCAG-EM Report Tool: Website Accessibility Evaluation Report Generator", and other evaluation resources, visit w3.org/WAI/eval

6. Video 5: Involving Users

Please carefully review the script for Video 5: Involving Users.

Please indicate any comments you have on:

The spoken audio
The illustrative visuals
The overall duration
Anything missing?
Anything to remove?
Any other observations?

Details

Responder	Video 5: Involving Users
Laura Keen
Eric Eggert	> Accessibility is about people; for example your customers, your clients, and your viewers Weird sentencing. “Accessibility is about people, like your customers, viewers, or readers” — customers and clients is too close in meaning, imho. --- > Yet unfortunately many people focus only on technical aspects of accessibility and forget about the end-users Could we not throw shade like that? There are all sorts of reasons for not doing user testing that don’t amount to “forgetting about the end-users”. “In addition to technical aspects, testing with users is very important.” (very is a filler word here but IMHO ok to emphasize the “important”) --- > For example, issues relating to the overall interaction design and user experience that are difficult to identify, even by seasoned experts This is throwing experts under the bus and might diminish the trust set to “experts” (which in itself has a deeply negative connotation). Some would even argue that “experts” are better to find such issues as they have a more general knowledge of different disabilities while the user will always have their unique perspective. “User tests can help you identify disability specific issues in interaction design and user experience” (not really happy with this phrasing but I hope it gives an idea) --- > Another benefit is that you get the results from usability testing without people with disabilities, in addition to identifying accessibility barriers Citation needed. If your accessibility is broken, you might get no data at all to advance your general usability along with your accessibility. There is a relationship, but framing it as inclusive might be a stretch. --- 10/11 > WAI provides guidance on Involving Users in Web Projects, throughout the design and development process > WAI also provides guidance on Involving Users in Evaluating Web Accessibility And. “WAI provides guidance on Involving Users in Web Projects, throughout the design and development process, and when evaluating the final product.”
Helen Burge	Looks good to me
Hidde de Vries	The sentence “Yet unfortunately many people focus only on technical aspects of accessibility and forget about the end-users ” is negative (although it is absolutely true), what about something like “It's essential to think beyond technical aspects of accessibility and include end-users in your evaluations.” ?
Daniel Montalvo	I need a verb here: Following these resources helps you ensure accessibility for your end-users rather than technical requirements only -> Following these resources helps you ensure accessibility for your end-users rather than focus on technical requirements only
Estella Oncins	Great work!
Sharron Rush	[[ Edit suggestions for focus and plain language - suggested alternatives to tedious repetition of "barriers" (editor's discretion) ]] Accessibility is about people; many of your customers, your clients, and your viewers have disabilities. Good design focuses on the needs of all users and not just on technical requirements. Let's explore that idea... Involving people with disabilities in the design and development process from the start has many benefits Reduce the cost of repair as you identify potential barriers early in the design process Finding and removing accessibility errors has been shown to improve the user experience for all users. People with disabilities reveal obstacles that may be difficult to evaluate from a technical perspective. For example, there may be problems in overall interaction design and user experience that are difficult to spot, even by seasoned experts (??? can't rewrite this one, don't know what it means???) Another benefit is that you get the results from usability testing without people with disabilities, in addition to identifying accessibility barriers Like any other usability testing, best results will come from working with a usability professional who can correctly interpret feedback from end-users with disabilities (no additional suggestions to the closing frames)
Vicki Menezes Miller	Seq8: not clear.
Brent Bakken	[ED] Seq. 7 - Visual: Maybe instead focus on one part of a page with a barrier that is typically not caught by testing, but identified by users usage over time. [idea] Seq. 9 - Visual: Just a fun idea. "User icons with text bubbles coming out across the screen to the usability testing professional (expert). They direct each bubble either to a "defect/issue" bin or to a "user preference" bin. Showing them correctly interpreting feedback from various users. Some are issues, some are just user preferences. [Question] Seq. 10 & 11 - Audio: For the use of "WAI," is this going to be pronounced "way" or is it going to be said "web accessibility initiative? Please clarify in the script.
Shawn Lawton Henry	Rather than commenting point-by-point on the draft, I took a pass at a somewhat different approach -- for discussion as you wish. Involve Users with Disabilities Throughout Your Projects. Accessibility is about making products and services usable by people with disabilities. That includes your customers, your clients, your employees, and others. Yet, web development projects often approach accessibility just as a checklist to meet standards and regulations. This approach risks focusing on technical implementation and missing the real purpose of accessibility – the user experience. It can lead to poor designs, wasted development efforts, and costly retrofitting. The solution: Involve people with disabilities throughout your project, so that designers and developers really know what accessibility is all about. Learn how people with disabilities use the web, and understand the assistive technologies and adaptive strategies that they use. Watch people using your website and applications, and others like it. See what works well and what doesn't. The result: Involving users early and throughout web projects not only leads to better products for users – it can also result in: * more efficient development * innovative solutions that make your product work better for more people in more situations (including people with out disabilities) * expanded market reach and higher customer satisfaction * a motivated, energized project team that is excited about accessibility How: Two WAI resources provide you guidance on how to involve users throughout your project, including getting a range of users, working with users, and notes for usability professionals. Find "Involving Users in Web Projects for Better, Easier Accessibility" and "Involving Users in Evaluating Web Accessibility" on w3.org/WAI/

7. Other Comments

Please provide any further thoughts or comments you may have. For example:

Any overall reflections on this set of 5 videos collectively?
Here and there, there are (few) repetitions in the videos (especially with the Overview video) - is this good or bad?
Is it good to reuse the phrase "essential for some, useful for all" (from the Perspectives videos) or does it not fit well with these videos on evaluation?
Should we close these videos by saying "for more information on web accessibility evaluation" or on "web accessibility" only (without "evaluation")?

Details

Responder	Comments
Laura Keen	I think these videos will be helpful for instructors, advocates, and for people who are new to accessibility. I think the repetition is good. I don't have strong feelings but I like the branding phrase "essential for some, useful for all". I think too many developers (and content contributors) still do not make the connection. It can only help to repeat the idea whenever possible. I like "for more information on web accessibility evaluation".
Eric Eggert	I would recommend to streamline the videos and make sure you have a better flow to them. Explain concepts more explicitly. Less repeats, at least in the inessential parts. Repeating “Conformance evaluation” in a video about that topic is not too useful. For some videos, enumerating could be a way to structure the videos: “Here are three features tools can have – one: … two: … three …” It would be good to have more of a narrative arc through the videos. Start by introducing what you will talk about and at the end, provide a list of important aspects. (This might be hard with those very short videos, but at the moment it feels more like an info dump, that you have to take in at video speed and I feel that watchers might not have enough time in the videos to really process the information.)
Helen Burge	For the links shown in the videos, will they be available as a list of resources under the video? I think mentioning links in videos is fine so long as the resources are available in a static format.
Hidde de Vries	I like these videos. I think the repetition is fine, it's not a lot and it probably helps if people only watch one or two individual videos, they would then not miss. Sounds good to me to repeat “essential for some, useful for all”. I have a slight preference for closing with “web accessibility” (without evaluation).
Daniel Montalvo	It looks very good to me. I think would be worth taking another pass to simplify some of the wording.
Estella Oncins	-Great work - I think that repetitions in the videos are good and reinforce the messages, they are also recommended in easy-to-read guidelines (https://easy-to-read.eu/wp-content/uploads/2014/12/EN_Information_for_all.pdf) - Reusing the sentence "essential for some, useful for all" is a really useful way to send a message, a kind of mantra or slogan. To me is like the UNCRPD "Nothing About Us Without Us" - If any "web accessibility" or maybe "web accessibility fundamentals" because you refer to the "Perspectives Videos". The word "Fundamentals" may help to reinforce the need to introduce "web accessibility" early in the project.
Sharron Rush	This is terrific, thanks team!
Vicki Menezes Miller	Overall, very good. However, I feel Video 3 may be a bit too long. If possible, I would prefer not to have repetitions in the videos. It could be boring. I think the phrase is positive - I like it. Maybe keep it simple, just say "web accessibility"
Brent Bakken	These are coming along very well. I am a little worried that some of the images my be very similar from video to video and maybe having them lose interest is some of them because of this. But I also understand that these are not completely intended to be a series that you watch one after another like you would with the perspectives videos. I do still think that if the visuals can be a little different to provide visual variety, that would be best.
Shawn Lawton Henry	Do we need to "write out" "WAI" on first reference in each video? --- Re: "Is it good to reuse the phrase "essential for some, useful for all" (from the Perspectives videos) or does it not fit well with these videos on evaluation?" My first reaction: No - <strong>strong No</strong>. That tagline phrase is awesome for the perspectives videos -- because that is what they are illustrating. I think it would dilute the phrase to use it with these evaluation videos. (We could decide that we want to use that phrase overall for accessibility promotion. Unless we do, I think we should not use it for these evaluation videos. And even if we do decide later to use it more broadly than the perspective videos, I don't think we'd want to use it for all 5 eval videos. It just doesn't fit.) /me should get her thinking cap on and come up with ideas for eval video tagline phrase(s), too. ;-) --- Re: "Should we close these videos by saying "for more information on web accessibility evaluation" or on "web accessibility" only (without "evaluation")?" I think we close each video with specific reference to the resource(s), e.g., "For the links to "WCAG-EM: Website Accessibility Conformance Evaluation Methodology", "WCAG-EM Report Tool: Website Accessibility Evaluation Report Generator", and other evaluation resources, visit w3.org/WAI/eval " --- All of the eval resources are listed on the main overview page: https://www.w3.org/WAI/test-evaluate/ (The secondary navigation has grouping just to keep that from being so long.) I think we want to point to the overview page, and not the nav grouping sections/pages. --- I think we want to give a URI directly to the eval overview page. This one works: w3.org/WAI/eval (and we could make w3.org/WAI/evaluation or w3.org/WAI/evaluate or other)

More details on responses

Laura Keen: last responded on 23, August 2019 at 17:55 (UTC)
Eric Eggert: last responded on 27, August 2019 at 08:46 (UTC)
Helen Burge: last responded on 27, August 2019 at 13:32 (UTC)
Hidde de Vries: last responded on 28, August 2019 at 09:26 (UTC)
Daniel Montalvo: last responded on 30, August 2019 at 11:26 (UTC)
Estella Oncins: last responded on 1, September 2019 at 09:16 (UTC)
Sharron Rush: last responded on 3, September 2019 at 18:15 (UTC)
Vicki Menezes Miller: last responded on 3, September 2019 at 20:07 (UTC)
Brent Bakken: last responded on 3, September 2019 at 21:01 (UTC)
Shawn Lawton Henry: last responded on 3, September 2019 at 21:51 (UTC)

Non-responders

The following persons have not answered the questionnaire:

Send an email to all the non-responders.

Compact view of the results / list of email addresses of the responders

WBS home / Questionnaires / WG questionnaires / Answer this questionnaire

Report issues on GitHub project w3c/wbs-design or by mail to sysreq.

Footer Navigation

Contact W3C

Copyright © 2024 World Wide Web Consortium.
W3C^® liability, trademark and permissive license rules apply.