W3C

Site Navigation

Nearby

Results of Questionnaire First Set of Video-Based WAI Resources (2nd Pass)

The results of this questionnaire are available to anybody. In addition, answers are sent to the following email address: shadi+EOsurvey@w3.org

This questionnaire was open from 2019-06-24 to 2019-07-10.

13 answers have been received.

Jump to results for question:

  1. Review level and timing
  2. Video 1.5: Involving Users (NEW!)
  3. Video 1: Evaluation Introduction (Unchanged)
  4. Other Comments
  5. ARCHIVED (DON'T RESPOND TO THIS QUESTION ANYMORE)
  6. ARCHIVED (DON'T RESPOND TO THIS QUESTION ANYMORE)
  7. ARCHIVED (DON'T RESPOND TO THIS QUESTION ANYMORE)
  8. ARCHIVED (DON'T RESPOND TO THIS QUESTION ANYMORE)
  9. ARCHIVED (DON'T RESPOND TO THIS QUESTION ANYMORE)

1. Review level and timing

Please indicate below the level of consideration you were able to provide for this review. If you were unable to get to it and would like more time, please indicate that as well. Thanks!

Summary

ChoiceAll responders
Results
I read the material carefully . 10
I skimmed the material 2
I need more time and have put a date below when I can get to it.
I am not going to be able to review this material and will defer to the decisions of the group.

Details

Responder Review level and timing
Kris Anne Kinney I skimmed the material 2nd time doing the survey, 1st time i forgot to submit it before closing the window.
Howard Kramer I read the material carefully .
Shawn Lawton Henry I reviewed and thought about the big picture thoroughly.
I skimmed the scripts, as I understood they are rough concept drafts just to get the gist -- and no one wants my editor hat on now! ;-)
Vicki Menezes Miller I read the material carefully .
Norah Sinclair I read the material carefully .
Brigitta Norton I read the material carefully .
Estella Oncins I read the material carefully .
Hidde de Vries I read the material carefully .
Laura Keen I read the material carefully . Overall, I love the idea of adding short specific videos to the WAI site. I think of it as increasing the accessibility of the site for people who have a hard time reading blocks of text. I specifically like the easy checks section. The outlines for videos 1-4 seem well thought. I don't see anything that I think is missed or doesn't belong.
Jennifer Chadwick I read the material carefully .
Lewis Phillips I skimmed the material
Brent Bakken I read the material carefully .
Sharron Rush I read the material carefully .

2. Video 1.5: Involving Users (NEW!)

Please review Involving Users. What do you think of this newly added video? (no need for detailed wording review, just the concept, approach, and overall key message for now please)

Details

Responder Comments
Kris Anne Kinney
Howard Kramer
Shawn Lawton Henry
Vicki Menezes Miller
Norah Sinclair I like the approach, tone, and key message. I would suggest changing the wording in one place: "yet be careful, just like any other user, also people with disabilities have their own preferences and personal biases"

change "yet be careful," to something to the effect: "however, do keep in mind...."
Brigitta Norton The message and flow makes sense, only addition would be to this line... "identifying design issues early on will save you a lot of time and effort" it would be worth mentioning that it will save money!
Estella Oncins Excellent approach
Hidde de Vries - One of the things I found hardest when I was tasked to involve users for clients, was to find users with disabilities. Should we give guidance on how to recruit users with disablitiies (in my case I ended up contacting user organisations for people with specific disabilities)?
- It may not be the right place, but perhaps it should give examples of users with disabilities?
- Overall it feels the content is quite abstract, one way to address that could be to talk about a specific thing a team is building, for example 'Say you are building this bank transfer screen.’, then applying everything else on that example (‘You may find usability issues in transferring this money’, ‘This heading makes it hard to understand that this is where the receiver's account number goes’, ‘John, who is blind won't be able to hear the “Transfer succesful” message, because it is not conveyed to AT’, etc). Perhaps this example is not great, but if we find a relatable example it could potentially make the whole video more practical?
Laura Keen I'm happy we're adding this video. This is an important reminder: "yet be careful, just like any other user, also people with disabilities have their own preferences and personal biases"
Jennifer Chadwick It's very important to make this a key factor in the success of the project. It can be difficult to find people but in the end, a project can be far more successful if there is involvement from real users. Is there any reason why would we shy away from making this a critical part of the process that should not be de-scoped from a project?

Perhaps the key is to have a solution if people say "how am I going to find people" … "I have no idea where to start"... "It costs too much to get people involved".

"ideally people with disabilities are involved throughout the design process"
"for example, when you are designing a new product or mock-up, do you involve people with disabilities in your test group? do you seek feedback on accessibility?"
"involving people with disabilities in usability testing has many benefits"
Lewis Phillips This will be a good video, the content and message are appropriate and useful.
Brent Bakken I think this is a great addition to the video series. I would almost debate putting this 3rd in the series of 5 videos. "Selecting and using tools," and "Comprehensive eval" are more technical and typically for developers, experts, or third party vendors. Involving users is not technical and can be implemented by any of the teams (content, design, dev) at any time. I just think it should be considered sooner in this list of video content.

I know you didn't want wording review, but I can't help it... ;)
Change "this is because people with disabilities are often more impacted by poor usability"
To "this is because people with disabilities are often more impacted by poor design"
Sharron Rush I think it is undoubtedly an important aspect but as people adopt this idea, there are cautions to be made. Not sure if this is the place to do this but we there is a a distinction to be made between expert users and standard users. For example, while the idea of including people with disabilities is becoming more wide spread, when people use the same users over and over, it becomes less useful and the feedback can be biased.

3. Video 1: Evaluation Introduction (Unchanged)

Please review Evaluation Introduction. Changes include:

  • "Preliminary Evaluation" is now a set of videos consisting of an overview video and a short video for each check provided in the Easy Checks resource
  • "Selecting and Using Tools" has been moved as a separate stand-alone video
  • Videos are now independent from each other (but may cross-reference as needed)
  • Example dialog provided to give better idea of the gist (this is not the final wording)
  • Videos are animated (eg. whiteboard or other illustrations) rather than moderated

What do you think of these changes? Do you have other thoughts or suggestions?

Details

Responder Comments
Kris Anne Kinney Definitely like the idea of animated videos with voice-overs for this, and not a person doing the actions while talking. I think it will make the videos more focused and less distraction of the person on screen.
Howard Kramer Dividing these into smaller videos is beneficial in that it avoids long videos which can cause users to drop off.
Will there be demos also of people evaluating pages using Easy Check as reference. I assume so but want to make sure.
Shawn Lawton Henry "* "Some checks can be carried out with much prior knowledge or tools""
Do you mean:
* "Some checks can be done without with much knowledge or tools"
And I think we want to make this point even clearer. :-)
/me forgot, not supposed to be looking at details of the scripts…

"Videos should relate to each other but not necessarily be dependent"
-> Videos are related and some might refer to others (and might be provided concatenated), yet each is fully independent.

"
Video 1: Evaluation Overview (~2-3 minutes) This video is intended to be integrated into the resource Evaluating Web Accessibility Overview
Video 2: Preliminary Evaluation (~10 minutes) This set of videos is intended to be integrated into the resource Easy Checks - A First Review of Web Accessibility.
Video 3: Selecting and Using Tools (~2-3 minutes) This video is intended to be integrated into the resource Evaluation Tools Overview
Video 4: Comprehensive Evaluation (~3-5 minutes) This video is intended to be integrated into the resource Conformance Evaluation Overview
"
That makes sense *if* people use the navigation along the left. However, some (maybe most?) people will probably just read the first page "Evaluating Web Accessibility Overview" and follow links from within the page content, and never even see the "Evaluation Tools Overview" and "Conformance Evaluation Overview" pages. (They just exist to keep the left nav from being too long.) Therefore, I think video #3 and #4 should also be on the main "Evaluating Web Accessibility Overview" page. And since they are there, I think #2.1 should be there as well. I think ideally #2.1 is written so the same video makes sense on both pages (rather than having different versions of that video).

/me looks now at https://www.w3.org/WAI/test-evaluate/ …
* Introduction – Video 1
* Initial Checks – Video 2.1
* Tools – Video 3
* Conformance Evaluation – Video 4
* Reports
* People
* Standards (just a pointer to another section)

… hummm.. I wonder if it makes it seem like the "people" part isn't important enough for a video? (obviously the opposite message that we want to send :-(

We should consider a video for involving users – probably one video that covers some of "Involving Users in Evaluating Web Accessibility" and "Involving Users in Web Projects for Better, Easier Accessibility" – and can be used here and there…

/me ducks at shadi throwing her wrench back at her ;-)

Vicki Menezes Miller Heading: "Evaluation Overview"
9th bullet point: "Some checks can be carried out with much prior knowledge or tools"
Question: Should it be rather "Some checks can be carried out without much prior knowledge or tools", i.e., change "with" to "without"

Heading: Video 3
Bullet point: "Example: tools can check for text alternatives but not their adequacy"
Do you mean "accuracy"?


Norah Sinclair Agree with this approach
Brigitta Norton These are good changes, animation will assist with visual impact.
Estella Oncins Excellent outline.
Hidde de Vries
Laura Keen In video 1 - it's great that we're specifically calling out:
"Involving users with disabilities, not only in evaluation, is essential"
Some more elaboration on involving users and the benefits of this

I do think at some point " Some more elaboration on involving users and the benefits of this" could be it's own short video. I'm not sure where it fits, including accessibility in the SDLC process vs tools for accessibility evaluation.

Video 2 - Easy Checks - Are we still updating the Easy Checks resource also?
Jennifer Chadwick - Videos are a great idea and we should have them! To me, having fresh, newly released videos in 2019 demonstrates how the W3C / WAI is active in the community, wanting to reach out and share its knowledge, guidance and authority on with all people to It's a strange thing to say but videos "humanize" a standards organization that is seen as being just the rule-makers that are 'enforcing' compliance.
- This preliminary evaluation is good.
- Evaluation is in the context of compliance with WCAG, but also walking through each of the end user experiences in order to remind the teams WHY they are doing this.
Lewis Phillips It is looking really good.
Brent Bakken I think all of these changes are good. I don't have any concerns about the changes listed hear.

I know that we are not yet working on specifics of the content / dialogue for each video. But, in Video 2 (easy checks), I was wondering if it would be worth anything to provide examples of what wouldn't/doesn't pass a check. I know we try to not focus on the barriers or what is wrong and just provide what it "should" look like, but if I was doing a check on page title, it would be nice to see some examples of inaccessible page titles too. Just an idea. I am okay with not including this.

I like the outlines so far. Looking forward to more on the dialogue and visuals. Great work!
Sharron Rush I approve of the changes but have some concerns about the example dialogue. So I am reserving the full review for later on and just taking it as the gist, which overall is OK. I would only suggest that even though it is introductory, we should be cautious about being repetitive and over simplifying.

4. Other Comments

Feel free to provide any further thoughts, reactions, and comments on any of these video suggestions.

Details

Responder Comments
Kris Anne Kinney Sorry for the brevity of my responses, I forgot to submit the first time so I'm trying to recreate my thoughts and that's nearly impossible right now.
Howard Kramer
Shawn Lawton Henry
Vicki Menezes Miller
Norah Sinclair
Brigitta Norton
Estella Oncins If you consider it relevant maybe provide an illustrative example of main problems found when a webpage is not accessible. I don't know how relevant would be also to classify errors as A, AA, AAA.
Hidde de Vries
Laura Keen
Jennifer Chadwick
Lewis Phillips
Brent Bakken One thing that has worried me in the past is that the WCAG EM report tool is very web site based. It is difficult to get developers or project managers to use it to create a report on a web application (especially a one page web app) because to tool language does not lend itself to evaluating such a system. I know this does not effect the production of the video, but if the EM report tool is ever changed to address the above concern, then the video could possibly be outdated.

I think most of these videos (#s 1, 3, 4, 5) will need to be on the longer side. probably closer to 2 minutes rather than 1 minute. I don't think we should try to limit these to be just at or over 60 seconds. They will need a bit more time to be thorough enough to be valuable.
Sharron Rush I guess video 2 is not ready for review and the others are on Hold for now?

5. ARCHIVED (DON'T RESPOND TO THIS QUESTION ANYMORE)

Please review Planning and Managing. Changes include:

  • Purpose, objectives, and primary audiences have been refined
  • Example dialog provided to give better idea of the gist (this is not the final wording)
  • Videos are animated (eg. whiteboard or other illustrations) rather than moderated

What do you think of these changes? Do you have other thoughts or suggestions?

Details

Responder Comments
Kris Anne Kinney I think the scope of people needs to be a little wider. Upper management needs to know about his - so they don't question the PM's Really like the sections of the resource that this video is "based on" and it should match those sections, initiate, plan, implement, and sustain.
Howard Kramer Changes are fine - maybe some example with the faces of different people at an organization and how they might be involved in this planning and managing.
Shawn Lawton Henry "This video is intended to be integrated into the resource Planning and Managing Web Accessibility"
Also https://www.w3.org/WAI/planning/ ?
Vicki Menezes Miller
Norah Sinclair
Brigitta Norton
Estella Oncins
Hidde de Vries
Laura Keen
Jennifer Chadwick - Be sure to include the roles and responsibilities involved in the planning (project manager, executive management, designer, developer, QA, accessibility specialist?).
- Examples are important. Would be helpful to show people reviewing a document where accessibility has been documented and scoped before the project starts.
- Helpful when demonstrating how it's being done, step by step through an automated check and manual inspection with code and testing.
- Part of the evaluation that I do is to Evaluations from the Usability Perspective: to review each design prototype and go through a process of checking how it meets the needs of all users: 1) how does this design/user flow work for people who are blind? 2) People with low vision? 3) Those using screen readers? Etc...
- For Video 3: Selecting and Using Tools - I know I'm biased :), but one of the key things some tools should do for your team is to make it easy to review issues on pages and then fix them - help you see issues, share them with your design and development teams through JIRA or your CMS directly.

Lewis Phillips I like the changes
Brent Bakken
Sharron Rush

6. ARCHIVED (DON'T RESPOND TO THIS QUESTION ANYMORE)

Please review Contacting Organizations. Changes include:

  • Purpose, objectives, and primary audiences have been refined
  • Example dialog provided to give better idea of the gist (this is not the final wording)
  • Videos are animated (eg. whiteboard or other illustrations) rather than moderated

What do you think of these changes? Do you have other thoughts or suggestions?

Details

Responder Comments
Kris Anne Kinney LOVE this resource. I think most of the problem with web accessibility, besides educating companies and developers is that people don't think its really something they need to do because no one complains. People need to know how to self-advocate for a better web.
Howard Kramer Changes look fine. I saw no mention for suggestions on how to find contact information on a site. That might be useful.
Shawn Lawton Henry "This video is intended to be integrated into the resource Contacting Organizations about Inaccessible Websites"
Also https://www.w3.org/WAI/teach-advocate/ ?
Vicki Menezes Miller
Norah Sinclair
Brigitta Norton
Estella Oncins
Hidde de Vries
Laura Keen
Jennifer Chadwick
Lewis Phillips I like the changes
Brent Bakken
Sharron Rush

7. ARCHIVED (DON'T RESPOND TO THIS QUESTION ANYMORE)

Please review Accessibility Statements. Changes include:

  • Purpose, objectives, and primary audiences have been refined
  • Example dialog provided to give better idea of the gist (this is not the final wording)
  • Videos are animated (eg. whiteboard or other illustrations) rather than moderated

What do you think of these changes? Do you have other thoughts or suggestions?

Details

Responder Comments
Kris Anne Kinney I know why we use the phrase "who care about accessibility" but I disagree with putting that out there. It gives people an out because they don't care about it or know how it affects their business. The audience should be owners of websites and mobile applications. We have to stop giving people an out when it comes to this.
Howard Kramer Changes seem fine.
Shawn Lawton Henry "This video is intended to be integrated into the resource Developing an Accessibility Statement"
Also https://www.w3.org/WAI/planning/ ?

"
Objectives:
* Educate organizations on the importance of accessibility statements
* Encourage organizations to adopt an open dialog with their customers
"
I'm not sure about the primary objective being to "Educate organizations on the importance". It's required in Europe, yes? In the big scheme of accessibility, are statements really that important?
… having said that, I do very much like the gist of the script! :-)
So maybe "Educate organizations on the importance of accessibility statements"
-> Educate organizations on the *benefits* of accessibility statements
Vicki Menezes Miller
Norah Sinclair
Brigitta Norton
Estella Oncins
Hidde de Vries
Laura Keen
Jennifer Chadwick
Lewis Phillips I like the changes
Brent Bakken
Sharron Rush

8. ARCHIVED (DON'T RESPOND TO THIS QUESTION ANYMORE)

Please review Business Case. Changes include:

  • Purpose, objectives, and primary audiences have been refined
  • Example dialog provided to give better idea of the gist (this is not the final wording)
  • Videos are animated (eg. whiteboard or other illustrations) rather than moderated

What do you think of these changes? Do you have other thoughts or suggestions?

Details

Responder Comments
Kris Anne Kinney
Howard Kramer Changes look fine. Would be a good place to mention a few organizations/businesses that have implement accessibility resulting in benefits beyond just accessibility. I.e. referencing some of the case studies.
Shawn Lawton Henry This video is intended to be integrated into the resource Business Case for Digital Accessibility
And https://www.w3.org/WAI/teach-advocate/ ?
Vicki Menezes Miller Not sure if this is still possible but somewhere - possibly in the area of social responsibility - I would love to see very brief examples of how accessibility makes a big difference when properly done, e.g. a blind student expressing how his learning experience is so much better when accessibility is properly implemented, someone hard of hearing, an elderly person benefitting from captions, a person with limited mobility etc. I know that there is a video of this nature on another site but it would be a good opportunity to include the human aspect here. The business case brings across extremely well various compelling reasons for accessibility but I'm still missing the humane nature of accessibility.
Norah Sinclair
Brigitta Norton
Estella Oncins I really like this page. I think that is really relevant.
Hidde de Vries
Laura Keen
Jennifer Chadwick
Lewis Phillips I like the changes
Brent Bakken
Sharron Rush

9. ARCHIVED (DON'T RESPOND TO THIS QUESTION ANYMORE)

We need to decide which videos we really want to develop in this first round - should we develop them all or just some of them (see other videos planned in the coming period)?

Please rate your priority for each video separately, with 1 being lowest score (not high-priority at this time) and 5 being highest (we desperately need this) - You can assign the same score to multiple videos, as needed! Please provide rationale for your rating and other thoughts you may have.

Summary

ChoiceAll responders
12345No opinion
Evaluation Introduction (all 4+10 videos) 3 4 6
Planning and Managing 1 5 7
Contacting Organizations 3 1 1 1 7
Accessibility Statements 2 2 1 8
Business Case 2 2 1 8

Averages:

Choices All responders:
Value
Evaluation Introduction (all 4+10 videos)4.57
Planning and Managing4.50
Contacting Organizations2.33
Accessibility Statements3.00
Business Case3.80

Details

Responder Evaluation Introduction (all 4+10 videos)Planning and ManagingContacting OrganizationsAccessibility StatementsBusiness CaseRationale
Kris Anne Kinney 4 5 5 5 5 Because the Evaluation introduction will be so large, I say to do the 4 smaller videos first, to get content rolling and interested and then show them how to use the evaluation tools and easy checks.
Howard Kramer 5 2 1 3 4
Shawn Lawton Henry 5 No opinion No opinion No opinion No opinion
Vicki Menezes Miller 4 5 1 2 3
Norah Sinclair No opinion No opinion No opinion No opinion No opinion
Brigitta Norton No opinion No opinion No opinion No opinion No opinion
Estella Oncins 5 5 2 3 4 I consider that Evaluation Introduction and Planning and Managing are essential in order to first raise awareness. After introduce a Business Use Case to show the benefits and then Accessibility Statements to show how to introduce accessibility practices in the company/organization. Once all principles and way to handle accessibility are clear then Contacting Organizations is the final step.
Hidde de Vries No opinion No opinion No opinion No opinion No opinion
Laura Keen No opinion No opinion No opinion No opinion No opinion
Jennifer Chadwick 4 5 1 2 3 How to..
followed by why it's important...
Lewis Phillips 5 5 4 No opinion No opinion
Brent Bakken No opinion No opinion No opinion No opinion No opinion
Sharron Rush No opinion No opinion No opinion No opinion No opinion

More details on responses

  • Kris Anne Kinney: last responded on 27, June 2019 at 22:05 (UTC)
  • Howard Kramer: last responded on 28, June 2019 at 01:58 (UTC)
  • Shawn Lawton Henry: last responded on 28, June 2019 at 12:54 (UTC)
  • Vicki Menezes Miller: last responded on 1, July 2019 at 16:31 (UTC)
  • Norah Sinclair: last responded on 3, July 2019 at 19:56 (UTC)
  • Brigitta Norton: last responded on 4, July 2019 at 02:30 (UTC)
  • Estella Oncins: last responded on 4, July 2019 at 10:37 (UTC)
  • Hidde de Vries: last responded on 5, July 2019 at 09:57 (UTC)
  • Laura Keen: last responded on 10, July 2019 at 16:39 (UTC)
  • Jennifer Chadwick: last responded on 10, July 2019 at 20:58 (UTC)
  • Lewis Phillips: last responded on 10, July 2019 at 21:42 (UTC)
  • Brent Bakken: last responded on 10, July 2019 at 21:56 (UTC)
  • Sharron Rush: last responded on 10, July 2019 at 22:16 (UTC)

Non-responders

The following persons have not answered the questionnaire:

  1. Eric Velleman
  2. Andrew Arch
  3. Shadi Abou-Zahra
  4. Sylvie Duchateau
  5. Kazuhito Kidachi
  6. Jedi Lin
  7. David Sloan
  8. Mary Jo Mueller
  9. Reinaldo Ferraz
  10. Bill Kasdorf
  11. Cristina Mussinelli
  12. Kevin White
  13. Kevin Rydberg
  14. Adina Halter
  15. Denis Boudreau
  16. Sarah Pulis
  17. Bill Tyler
  18. Gregorio Pellegrino
  19. Ruoxi Ran
  20. Sean Kelly
  21. Muhammad Saleem
  22. Sarah Lewthwaite
  23. Daniel Montalvo
  24. Mark Palmer
  25. Jade Matos Carew
  26. Sonsoles López Pernas
  27. Greta Krafsig
  28. Jason McKee
  29. Jayne Schurick
  30. Billie Johnston
  31. Michele Williams
  32. Shikha Nikhil Dwivedi
  33. Brian Elton
  34. Julianna Rowsell
  35. Tabitha Mahoney
  36. Fred Edora
  37. Rabab Gomaa
  38. Marcelo Paiva
  39. Eloisa Guerrero
  40. Leonard Beasley
  41. Frankie Wolf
  42. Supriya Makude
  43. Aleksandar Cindrikj
  44. Angela Young

Send an email to all the non-responders.

Compact view of the results / list of email addresses of the responders

WBS home / Questionnaires / WG questionnaires / Answer this questionnaire

Report issues on GitHub project w3c/wbs-design or by mail to sysreq.