W3C

Site Navigation

Nearby

Results of Questionnaire "Making Content Usable for People with Cognitive and Learning Disabilities" - EOWG Review

The results of this questionnaire are available to anybody. In addition, answers are sent to the following email address: shawn@w3.org

This questionnaire was open from 2020-07-31 to 2020-08-19.

12 answers have been received.

Jump to results for question:

  1. About the Survey and Review
  2. Review level
  3. First Impressions
  4. Abstract
  5. WCAG and Informative
  6. Summary (section 1)
  7. Introduction (section 2)
  8. User Stories (section 3)
  9. Design Guide (section 4)
  10. Usability Testing, Focus Groups and Feedback (section 5)
  11. Use Cases / Persona (section 6)
  12. A. Appendix: Mapping User Needs, Persona and Patterns
  13. B. Appendix: Considerations for Uptake in Different Contexts and Policies
  14. C. Appendix: Testable Statements for Each Pattern
  15. D. Appendix: Business Considerations
  16. Organization and Navigation
  17. Other comments?

1. About the Survey and Review

Making Content Usable for People with Cognitive and Learning Disabilities is a draft Working Group Note. This "wide review" is intended to be the last review before it is published. (So, speak now. :-)

You do not need to read the entire document. You can read only the abstract, summary, introduction, and the first text in each section (per the survey questions below), and briefly skim the rest. EOWG's role in this review is to focus on the understandability, approachability, ease-of-use, and integration with other WAI resources, per EOWG Technical Document Review guidance. This survey also has "Optional additional review" questions that the COGA TF is interested in. You are welcome to comment on those aspects in this survey, or directly yourself.

We will compile input from this survey and bring it to EOWG to review, before submitting it.

Please put substantive issues in this survey (as opposed to minor copy edits). Feel free to put questions for EOWG to discuss. As feasible, please indicate your comments as:

  • [high] very important
  • [med] medium-level importance
  • [low] low-level importance

If you have suggestions for minor copy edits, please submit them as a pull request (edit), GitHub issue, or e-mail public-cognitive-a11y-tf@w3.org.

Note:

  • This survey has separate questions for each section of the document — and some overall questions at the end.
  • Reminder that you can fill in some of this survey now, "Submit your answers" button at the bottom, and come back and do more later.
  • If you have comments that apply to more than one section, please put them in both relevant questions (unless they apply to all, then can put them in a general comment :-).

Related info:

If you have limited time, please focus on these survey questions:

  • 3. First Impressions
  • 5. WCAG and Informative
  • 7. Introduction (section 2)
  • 10. Usability Testing, Focus Groups and Feedback (section 5)
  • 16. Organization and Navigation

Details

Responder
Laura Keen
Sylvie Duchateau
Estella Oncins
Daniel Montalvo
Howard Kramer
Andrew Arch
Dónal Rice
Jennifer Chadwick
Vicki Menezes Miller
Kevin White The following are some general comments based on minutes of EO meeting of 14th August.

Target audience

I think I agree with the general points regarding who this material is for. There is a huge amount of information but it skirts between designer, developer, tester, user research, business analyst, product owner. I think a clear focus on the design side might be most sensible since the design patterns are not success criteria. This would suggest a rather different structure:

Personas
Objectives
Design patterns

User stories are something that can be confined to an appendix

Landing page

ABSOLUTELY! Converting it into a resource a la Older users or How people with disabilities use the web would be excellent. This would make it a much more manageable document. Would be great to work with them to make something like that.

Usability testing, focus groups and feedback

The more I look at this section the more I feel it is really problematic. At best there should be updates made to the Involving users resource and a link made to that. Although, I would see no problem in removing this section, as I noted there are many aspects of this that are going to be legislature specific and it is difficult to navigate that. I am also not sure it is WAI's job to train user researchers in how to conduct research sessions. Involving Users encourages early involvement and involving a range of users - there is the tiny wee section on Working with Users which is about right.

Potential sections to include in overview

Probably the sections I indicated above - with a bit of an introduction. Those would be the main things that I would point people too. The user needs are valuable to include into requirements work but that is much more focused and not sure that I would do it that often.
Shawn Lawton Henry Missing references to and relationship to existing WAI Resources -- especially:
* Involving Users in Web Projects for Better, Easier Accessibility https://www.w3.org/WAI/planning/involving-users/
* Involving Users in Evaluating Web Accessibility https://www.w3.org/WAI/test-evaluate/involving-users/
* Planning and Managing Web Accessibility https://www.w3.org/WAI/planning-and-managing/
and also:
* How People with Disabilities Use the Web (sub-pages) https://www.w3.org/WAI/people-use-web/
* relevant Perspectives Videos https://www.w3.org/WAI/perspective-videos/
* Accessibility, Usability, and Inclusion https://www.w3.org/WAI/fundamentals/accessibility-usability-inclusion/
* The Business Case for Digital Accessibility https://www.w3.org/WAI/business-case/
* Older Users and Web Accessibility: Meeting the Needs of Ageing Web Users https://www.w3.org/WAI/older-users/

---

I haven't fully thought this through, yet want to note it before I quit for the night:

We are introducing WCAG success criteria using persona quotes with "Problem" and "Works Well", e.g., in
https://www.w3.org/WAI/standards-guidelines/wcag/new-in-21/
(and we plan to add these to the WCAG Understanding documents)

This document also uses persona quotes with "Problem" and "Works Well" -- e.g., https://w3c.github.io/coga/content-usable/#alison-an-aging-user-with-mild-cognitive-impairment.
Are they used differently in this document than for the SC introductions? If so, is that potentially confusing? Or, is that OK and even good?

---

Personas should build on existing ones whether feasible https://www.w3.org/WAI/people-use-web/user-stories/
Sharron Rush

2. Review level

summary | by responder | by choice

Summary

ChoiceAll responders
Results
I reviewed it thoroughly.
I reviewed some sections thoroughly, and skimmed some sections. 11
I skimmed it. 1
I didn't get to it, and pass on this review.

Skip to view by choice.

View by responder

Details

Responder Review levelComments
Laura Keen
  • I reviewed some sections thoroughly, and skimmed some sections.
Sylvie Duchateau
  • I reviewed some sections thoroughly, and skimmed some sections.
Estella Oncins
  • I reviewed some sections thoroughly, and skimmed some sections.
Daniel Montalvo
  • I reviewed some sections thoroughly, and skimmed some sections.
Howard Kramer
  • I reviewed some sections thoroughly, and skimmed some sections.
Andrew Arch
  • I reviewed some sections thoroughly, and skimmed some sections.
 It is very long - 154 pages if I were to print it at standard size. have they considered breaking it up by section, or other groupings?
Dónal Rice
  • I skimmed it.
Jennifer Chadwick
  • I reviewed some sections thoroughly, and skimmed some sections.
 Firstly, I am so happy to have this resource. As an accessibility consultant, advocate and user experience designer, it is extremely beneficial to have such clearly detailed user stories and examples that pertain to high-relatable, real-life scenarios involving "real" people. In the past, as an advisor, it has been hard to describe how people with cognitive difficulties (a wide and varied area of user experience) 'use' the web and how to translate this into concrete design patterns for UX, visual and content designers can implement on a consistent basis. When I have more time, I will dedicate it to reviewing the research in detail.
Vicki Menezes Miller
  • I reviewed some sections thoroughly, and skimmed some sections.
 The document is very long.
Kevin White
  • I reviewed some sections thoroughly, and skimmed some sections.
Shawn Lawton Henry
  • I reviewed some sections thoroughly, and skimmed some sections.
Sharron Rush
  • I reviewed some sections thoroughly, and skimmed some sections.

View by choice

ChoiceResponders
I reviewed it thoroughly.
I reviewed some sections thoroughly, and skimmed some sections.
  • Laura Keen
  • Sylvie Duchateau
  • Estella Oncins
  • Daniel Montalvo
  • Howard Kramer
  • Andrew Arch
  • Jennifer Chadwick
  • Vicki Menezes Miller
  • Kevin White
  • Shawn Lawton Henry
  • Sharron Rush
I skimmed it.
  • Dónal Rice
I didn't get to it, and pass on this review.

3. First Impressions

Comment guidance:

  • What are your first impressions of the document?
  • How might we improve first impressions for target audience?

Details

Responder Comments
Laura Keen My first impression is the document is well written and comprehensive. I don't have anything to suggest. I like the user stories' bulleted lists up front.
Sylvie Duchateau My first impression is that the document is really dense, contains many information.
1. While reading the table of contents, I find it irritating to read in several sections, titles that have the same text such as:
2. In order to use consistent language, I would suggest to use either the user or users.
Example: "5.5.1 Does the User Understand What Things Are and How to Use Them?"
"5.5.2 Can Users Find What They Need?"
Rationale: first item talks about "the user", second talks about "users".
Sometimes the documents talks about "the users" and sometimes about "users".
Examples: "5.5.4 Can Users Avoid Mistakes and Easily Correct Them
5.5.5 Can the Users Maintain Focus?"
This should also be made consistent.
3. Following our discussion on gender specific documents, when I read user scenarios I have the impression that the users are more female names than male names and wonder if it is author's intention?
Estella Oncins I like the overall information provided in the document and also the style. It is clearly very informative.
I personally found it bit too long and too broad. Not clear how designers and developers will use it and/or apply it.

Daniel Montalvo I think this is very good material and should be kept. However, I don't feel comfortable having such guidance separate from other educational and informative EOWG resources
Howard Kramer I was impressed. It was very clear and easy to understand. Graphics / icons aided the understanding.
Andrew Arch First impressions:
- Seems very comprehensive
- Looks like it will be useful in helping address the needs of people with cognitive impairments
- The length is a little overwhelming

Suggestions:
- in the Introduction (or even in the summary), add a section that explains the remaining main sections and when/how to use them, ie what the reader will find in the reset of the document - Users stories / Design Guide / Testing / Personas
Dónal Rice Very impressive! I like how the objectives are presented first, then user stories and the design guide going into the objectives in more detail. That provides a good mix of top level things the developer/designer needs to know, then the motivation/context for why they need to know then, then the content in more detail.

The addition of the symbols is perhaps demonstrating what he design guidance states about supporting text with images. Perhaps larger and more relevant icons (although this is always a challenge) would help the initial reader like me to grasp some of the objectives on first reading.

I would have appreciated some references to literature on cognitive impairment. The high-level description provided in "2.2 Background about People with Learning and Cognitive Disabilities and the Web" is well written, and its a difficult thing to try to express and explain this area to people who know very little about cognitive impairment. However for people who know a little, or in my case very little, I would have appreciated some links to further reading as i'm always trying to improve my own understanding.
Jennifer Chadwick For first impressions on the document and resource overall, it appears to be structured in a way that reflects the practices it is preaching. There is a clear declaration of the purpose of the document, the purpose/why, how many sections and the purpose/topic of each section. It seems there has been consideration made for the length of the paragraphs and that only one idea or concept is presented in the paragraph at a time. The language is fairly plain.
Vicki Menezes Miller There is a lot of useful information but perhaps an overload of information.

Length: I feel the document is too long.
The length of the document combined with the amount of information may distract the reader from essential blocks of information. At a first glance, I would suggest moving the user stories to an appendix.

Target audience: In answer to the above question "first impressions for target audience?", who is the target audience? Under "Status of this document", it is written "This version has been enhanced from the previous version to focus on an audience of designers and developers, and provide comprehensive user stories and design patterns." I think that "content writers/authors" are definitely an essential part of the target audience (see Objective 3). It seems that there is sufficient guidance for content writers and designers but less for developers unless I simply can't find it (due to the length). If I were a developer, with such a large document, I think I would be overwhelmed and would have difficulty in finding the information I need. Maybe, if it is there, it might be worthwhile under "How to Use this Document" to add some pointers.
Kevin White Dauntingly large

Would be useful to make the Summary and Abstract sections work a bit more to direct the user. The summary is good but it isn't clear that these could be principles that could be followed to provide a better experience.
Shawn Lawton Henry Long. Very long. 178 printed pages.

Includes different types of information, for different users, doing different tasks. Which means the rest is clutter to users focused on specific tasks.

Hard to focus on and print out just the specific part(s) a user is interested in.
Sharron Rush First impression is that it is a very long document. Next is that it is filled with critically needed information about cognitive issues related to effective digital communication. The topics are broad and far-ranging so I am not sure who the target audience is. Although it identifies an audience of people who make web content or applications, much in this document is beyond content creators and would include designers, PMs, QA and even management. The document could be improved by segmentation and focused messaging for these specific audiences. The core message of "making content usable" could be applied here by really considering who will use it, how they will find it, and how easy it will be for them to apply this excellent guidance.

4. Abstract

Abstract review guidance:

  • Does the abstract provide a succinct introduction to the document?
  • Is there anything that should be added to or deleted from the Abstract?

Details

Responder Comments
Laura Keen Yes. I have nothing to add or delete.


Sylvie Duchateau Abstract sounds clear to me. I have nothing to comment.
Estella Oncins
Daniel Montalvo Yes, it is clear and provides a succinct introduction.
Howard Kramer Yes. I didn't see anything that needed changing or added.
Andrew Arch
Dónal Rice
Jennifer Chadwick Nothing to add to the Abstract. I like how it is presented with a short, succinct style with a bulleted list that clearly defines what the document is about.
Vicki Menezes Miller Nothing to add.
Kevin White Good although I think the term 'objectives' might be misleading or at least confusing until they are explored more.
Shawn Lawton Henry
Sharron Rush The abstract is fine

5. WCAG and Informative

Review guidance:

  • Is the relationship between this document and WCAG clear?
  • Is it clear that this document is informative, and not required to meet WCAG?

Summary

ChoiceAll responders
Results

Details

Responder WCAG and InformativeComments
Laura Keen Yes and Yes.
Sylvie Duchateau While reading the document, I cannot easily find where the relationship betwen WCAG and this document is written. For people knowing W3C documents, it may be clear that a working group note is informative, but his should be made clearer for users who do not know the difference between a working group note and a W3C standard. While skimming the document, I cannot rapidly find where this is clearly stated.
Estella Oncins Yes it is clear that the document is informative
Less clear is the relation to WCAG maybe including SC or relate objectives to SC as in Appendix C
(https://www.w3.org/WAI/GL/task-forces/coga/wiki/Testable_Statements_for_Each_Pattern) would make the relation more clear.
Daniel Montalvo Yes, they state clearly that this is beyond WCAG. It is not clear, though, why this specific set of best practices needs to in a TR and others are in other WAI resources.
Howard Kramer Somewhat - in the sense that the introduction lets you know that these guidelines go beyond WCAG.
Somewhat - it's implied in the last paragraph of 2.0, the introduction.
Andrew Arch it's pretty clear, but could also be mentioned in the section on Usability Testing - that usability testing is not testing for WCAG conformance, a technical WCAG test/audit is needed for that [medium]
Dónal Rice
Jennifer Chadwick Yes, nothing else to add.
Vicki Menezes Miller It is clear enough.
Kevin White Yes, although likely to be missed. Regular W3C document readers are likely to skip the abstract (ok, I did). Irregular readers may go straight to the summary. There is no visual salience provided to the message but I don't think there is anything in the publication design patterns that would provide that. It is touched on in other areas, but again it tends to blend in with the narrative.

This is [low] regardless.
Shawn Lawton Henry I thought it was clear -- but then someone told me that they didn't understand "provide supplemental guidance beyond the requirements of WCAG."
Sharron Rush Yes, it is stated as such in the abstract

6. Summary (section 1)

Summary review guidance:

  • Is there anything that should be added to or deleted from the Summary?
  • Do the listed Objectives provide a clear overview of the areas to pay attention to, without overlap?
    • ,/ul>

    Details

    Responder Comments
    Laura Keen Nothing to add or delete. I like the ordered list with icons and bold text. It makes the summary very readable.
    Sylvie Duchateau I think it is clear.
    Estella Oncins I really like the structure and the use of icons. It provide a very fresh look.
    Some objectives in this section are not the same as the ones described in the "User stories" and the "Design Guide".
    Daniel Montalvo I had difficulty understanding objectives 1 and 3. They seemed like mangled, but then I concluded objective one talks about conventions (consistent identifiers) and 3 talks more about clear language and signs.
    Howard Kramer I thought the summary, like other parts, was clear and well written.
    Andrew Arch Would be good here to get an overview of the big sections that follow, not just the identified objectives and quick actions that can be taken
    Dónal Rice
    Jennifer Chadwick Nothing to add. I like the use of a numbered set of items and the icons added at the same time are placed in a way that is clean and clear (at the end of the sentence or elsewhere would not flow so well. Here, there is a clear flow - from the concept of "idea #1, symbol of the concept, and a textual explanation no longer than two sentences."

    Vicki Menezes Miller 1. The introduction speaks only about "web content providers" - do you want to add other roles as earlier mentioned (i.e. developers and designers) or maybe slightly modify the sentence?
    2. I like the idea of the icons and they are generally very nice. The only ones I hesitate about are No. 1, 7 and 9, noting that No. 7 is very difficult to illustrate.
    3. The listed objectives and the ones in the left TOC do not always align. Please check.

    Kevin White [med] Summary is good although they are introduced as 'objectives'. Are they objectives of the document or objectives for me, the reader, to consider. Is there scope to present them as actions or principles or some such?
    [low] Icons, the perennial challenge! Not sure about 1, 3, 7 and 9. Sorry, no immediate suggestions.
    Shawn Lawton Henry Summary icons
    Generally, I find icons and images helpful. Examples where I think they help with comprehension:
    * https://www.w3.org/WAI/media/av/#how-to-make-audio-and-video-accessible
    * https://www.w3.org/WAI/tutorials/tables/
    * https://www.w3.org/WAI/media/av/planning/#checklist
    * https://user-images.githubusercontent.com/6991814/36610987-a17a20b6-1897-11e8-9741-926e190b0755.png
    * https://www.w3.org/WAI/business-case/

    For some of the icons in this Summary, I couldn’t figure out what they were trying to communicate or how they related to the topic. So, they were more of a distraction, and in some cases even uncomfortable. This could be frustrating for some readers.
    Specifically:
    * Help users understand… -- very confusing, seems like steps backwards.
    * Help users find… -- OK. strongly means “search” to me – which may be not really what you want to communicate?
    * Help users understand with clear text and images… -- cannot figure out what it is trying to communicate.
    * Provide support for different ways to understand content. -- complex and confusing – what does a clock have to do with this?
    * Help users avoid mistakes. -- OK.
    * Help users to maintain focus. -- looks like the scope of a shot gun is pointed at me:-(
    * Ensure processes do not rely on memory. -- no ideas what this is trying to communicate.
    * Make it easy to get human help and give feedback. -- very good!
    * Support adaptation and personalization. -- not clear and very uncomfortable (maybe still feeling like the shotgun target).
    * Test with real users! – OK. communicates to me older users.
    ---

    If you could come up with good icons, then I suggest that you use them throughout the document to help communicate the relationship of the information. For example, you include the dialog icon in 3.7, 4.8, A7

    ---

    Also note that I would find it much easier if the text did not wrap under the icon.
    Sharron Rush The Objective are useful framing and the repetition is self-reinforcing. The most useful presentation of the Objectives is in the Appendix A. Perhaps consider bringing that section forward as a more effective "Summary"

7. Introduction (section 2)

Introduction — including:
2.1 How to Use this Document
2.2 Background about People with Learning and Cognitive Disabilities and the Web
2.3 Building the User into the Development Process

Review guidance:

  • Does the introduction have the right type and amount of information?
  • Is there anything that should be added or deleted or clarified?
  • How does it relate to other information provided on the WAI website?
    (e.g., How People with Disabilities Use the Web and Involving Users and Planning and Managing)
    • Is some information duplicated or overlapping in a way that we (COGA TF, EOWG, and others) should address? For example, is there guidance in this document that applies more broadly than cognitive disabilities and maybe should be incorporated in the main resource instead?
    • Is related information on the WAI website appropriately pointed to?

Details

Responder Comments
Laura Keen I like the tone and structure of this document. It's comprehensive and readable at the same time. I think overlap is necessary. The bulleted lists add to the readability.
Sylvie Duchateau NO specific idea on that question.
Estella Oncins Information is very clear and well structured.
Not clear why at the beginning of the section mobile apps are included but at the end it states that is only for the website:
"If people with cognitive impairments are included in the usability testing and their feedback is accounted for, the website will be easier to use for everyone, including people who are experiencing stress or mental health issues".
Daniel Montalvo I think work could be done to make it more succinct. If this document is going to remain as is, more pointers to existent WAI resources are missing (imo).
Howard Kramer I thought the intro, like other parts, was clear and well written.
Andrew Arch 2.1 could do with an introduction to the remaining sections (Users stories / Design Guide / Testing / Personas) and when to use them [medium]
2.2 seems like good background - examples help (but example 6 is unclear [high])
2.3 is good BUT the order of the bullets doesn't match he order of the Sections [low]

A lot of existing WAI resources will have to cross reference this document - those references, plus the 'older people' docs.

Most of the material in Section 5 is general enough to be of use to anyone considering Usability Testing etc with any people with disability - it contains excellent advice that could be woven into 'Involving Users'. The entire section could even be lifted out and form a 'how to' alongside 'Involving Users' and the Note could cross reference the new document.
Dónal Rice
Jennifer Chadwick There is nothing else I would add, and the purpose of each sub-section is clear, i.e. it uses common phrases and concepts for the topics - "How to use this document", "Background" and "Building the user into the Development process." Perhaps this could also be called "How to build the user into the Web Development process" (or leave the "web" out if it's related to document and website accessibility). Also, I don't know whether this is generally helpful or harmful to those with cognitive needs, but I also tend to put key words or concepts in bold text. I feel as though it assists when people are skimming documents and may also help to impress the word into someone's memory.
Vicki Menezes Miller Clarification required:
The second paragraph, first sentence: "Traditionally, accessibility focused on making the interface usable for people with sensory and physical impairments (vision, hearing and/or mobility)."
Is this statement accurate? WAI resource "https://www.w3.org/WAI/fundamentals/accessibility-intro/ , clearly states that cognitive and neurological impairments have always been considered in addition to visual, hearing and other mobility disabilities.

Section on "How to Use This Document":
1) Needs some real information on how to use the document. It says "This document is divided into parts. Each part can be used when needed by different groups in the development team." Perhaps, provide some clearer guidance on the sections which follow.
2) Not sure about the usefulness of the bulleted list and how it relates to "How to Use this Document". That last sentence of 2.1 with the bulleted list might be better placed under "Introduction".

Section 2.2:
No comments.

Section 2.3:
There is overlap with other areas of WAI site and resources. References to existing WAI resources could be added.








Kevin White [low] Not sure what the 'how to use this document' does. It doesn't tell me how to use the document, and the list is simply a list of domains where accessible content is important. Is there maybe something that needs to identify each section and inform what it provides and how to use it? For example, the user stories section provides user needs that can be incorporated into requirements definitions? Or even simply that the user stories sections provide user needs
2.2 and 2.3 are good!
Shawn Lawton Henry EOWG has worked hard to avoid covering topics in multiple places on the W3C website. Instead, we seek to put the information in one place, and point to it there from other places where it is relevant.

Section 2.2 and 2.3 should not be in this document, and instead, this document can point to the information elsewhere.

Most of the topics in “2.2 Background about People with Learning and Cognitive Disabilities and the Web” are covered in https://www.w3.org/WAI/people-use-web/abilities-barriers/#cognitive
(and there is an open issue for more input from COGA on that https://github.com/w3c/wai-people-use-web/issues/48 )

I disagree with the usability accessibility image at the end of “2.2 Background about People with Learning and Cognitive Disabilities and the Web”. This is a very complex and debated issue. It is beyond the scope of this document. (It is touched on in https://www.w3.org/WAI/fundamentals/accessibility-usability-inclusion/ )

2.3 Building the User into the Development Process is already covered in <a href="https://www.w3.org/WAI/people-use-web/">How People with Disabilities Use the Web</a> and <a href="https://www.w3.org/WAI/planning/involving-users/">Involving Users</a> and <a href="https://www.w3.org/WAI/planning-and-managing/">Planning and Managing</a>

Sharron Rush I don't think the :How to Use..." section actually describes use of the document. It state that sections can be used by different roles within the development team/process but does not clearly identify how that would work. Might be useful to distribute to actual working teams and get feedback about if/how that segmentation was applied in a real world setting. I wonder if some of this excellent guidance and instruction might be more effectively distributed - in terms of being found and used - if integrated within existing WAI docs.

8. User Stories (section 3)

Review guidance:

  • Is it clear how this section relates to the other sections?
    • >
  • Do the first paragraphs provide a good introduction to this section, related to the rest of the document?
    • >

Optional additional review:

  • Do the user stories and needs do a good job of capturing and presenting user requirements?

Details

Responder Comments
Laura Keen Yes it is clear and well written.
Sylvie Duchateau No specific comment
Estella Oncins At the beginning of the section it is stated "They are divided into the same objectives as the design guide above", but the design guide is below.
In addition, it might be good the use of names from different cultures.
Daniel Montalvo Not much. Of course there is an overall relation to people with cognitive and learning disabilities, but one could take these sections and put them in different documents as well, or add some of these to existing WAI materials.
Howard Kramer I was thrown a bit at first with the user stories - because of moving in and out of the voice of the user to the voice of the report but figured it out after just a bit. I'm not sure if I have a suggestion to address this but if no one else comments on it it probably doesn't need addressing. Otherwise, I thought the section met the 3 criteria asked about above.
Andrew Arch the relationship between User Stories and Personas needs explanation.

For people not familiar with users stories as a design/development requirement tool, an introduction to their structure and use would be beneficial
Dónal Rice
Jennifer Chadwick The user stories are robust and present the content, the person and their needs in a relatable, casual way. The only thing I would suggest is provide a teaser text or explanation of exactly what story relates to each of the bullet points under the heading, i.e. "Clear Purpose". I found myself reading the bullet points which were very high-level as concepts, e.g. "actions" or "when I'm in the architecture of the website"... I found myself wanting concrete examples of what those words and concepts were referring to, immediately. Once I selected a name, i.e. "Carolyn" - I was then able to read someone's story, their problem, and understand those concepts much easier.
Vicki Menezes Miller First paragraph: the design guide is after this section not above.
Second paragraph: maybe some improvement.
Third paragraph: is it necessary? Just thinking along the lines of "tersification".

Optional review question:
I think they definitely capture the issues in the detailed user stories but I feel that they are sometimes quite long and maybe repetitive in the listing of user stories.

Kevin White
Shawn Lawton Henry
Sharron Rush

9. Design Guide (section 4)

Review guidance:

  • Is it clear how this section relates to the other sections?
  • Do the first paragraphs provide a good introduction to this section, related to the rest of the document?
  • Is it clear that this section includes specifics for designers and developers to do?

Optional additional review:

  • Is the content for each Objective useful and complete?
  • Is the Pattern Template suitable? Does it need more or less sections?
  • Is the quantity of content in the patterns about right and well focused or do some need more or less?
  • Are any user needs or patterns missing or incorrect?

Note: This section is intended to be aimed at developers/designers and to explain "what to do" in a concise way with just enough background to help understanding.

Details

Responder Comments
Laura Keen It is clear how this section relates to other sections. Yes and yes.
Sylvie Duchateau Will review later.
Estella Oncins There is some inconsistency in the use of wording and relation to the subsections. At the beginning of the section it is stated:
"This guide is divided into design themes. Each theme includes user stories, testing methodologies, and design checkpoints."
But design themes are called "patterns"? Not clear if "testing methodologies" are the sub sections "description", "how it helps" and "more detail" and design checkpoints are "examples". It is confusing maybe using the same wording along the section might help.
Daniel Montalvo Not much. Of course there is an overall relation to people with cognitive and learning disabilities, but one could take these sections and put them in different documents as well, or add some of these to existing WAI materials.
I saw a minor repetition and put a pull request for it at

https://github.com/w3c/coga/pull/146
Howard Kramer I thought this section addressed all the above criteria.
Andrew Arch
Dónal Rice
Jennifer Chadwick I wanted to keep my original comments intact because now, engaging with the Design section - I can see all of useful user stories being put into context of the design, and this is very helpful and solves the issue I had earlier in the document.
Vicki Menezes Miller First paragraph, first sentence: "This guide provides assistance making websites and applications friendly for people with cognitive and learning disabilities by providing guidance for designs and the design process"
- The word "for" is missing between "assistance" and "making"
- The guide also provides recommendations for content writers. I think this needs to be added somehow otherwise focus is on "design"

Second paragraph:
No comment.

Third paragraph:
A little confusing. Is it "themes" or "patterns"?
The ideas could be presented more clearly, e.g. "simply understanding" the themes and user stories may help some users versus "implementing these patterns is essential".

Note on the Note: This section is also for content producers not only developers/designers.
Kevin White
Shawn Lawton Henry
Sharron Rush

10. Usability Testing, Focus Groups and Feedback (section 5)

Review guidance:

Optional additional review:

  • Do the items in the Test Objectives section tie in well with the design guide?

Details

Responder Comments
Laura Keen I think that section 5 expands upon the existing WAI resources. The information is clear and easy to understand. The sections appear to me to have the right level of information. It may be a good idea for EOWG to review the existing WAI resources for minor edits but does not seem to me to need any substantial revision.
Sylvie Duchateau No time to review it yet, may be later.
Estella Oncins Maybe change: "Finding People to Include" to "Finding People to Involve"
Also some advice on how to interact with people with disabilities might be helpful as some usually communicate with an intermediary. So "ethics" is a delicate issue
Daniel Montalvo
Howard Kramer I think this section meets all the criteria mentioned above in a positive way and like the other sections it is clear, easy to follow, and offers clear detailed recommendations.
Andrew Arch This section provides a lot of detail, especially in the 5.3, 5.4 and 5.5. It could easily be removed from this document as it doesn't sit so neatly here like Sections 3, 4 and 6 do. As mentioned in Q7, it could form a very useful stand-alone document or be used to increase the advice in 'Involving users'.
Dónal Rice
Jennifer Chadwick It's a good section. My only comment is that it's important to provide some guidance on where to find people, and resources for this were included, so I am satisfied that I could share this page with a team and they would understand not only where and how to go about finding to find people (a practical resource) but also what to consider when working with real users - the sensitivity of understanding and respecting people.
Vicki Menezes Miller There is overlap with other WAI resources and I'm not sure how best to address this. Maybe add a list of related resources.
Secondly, there seems to be repetition in the paragraphs (and of earlier content) which becomes tiring when you reach this stage of the document.

I'm not sure it is necessary to provide such a very detailed guide on usability testing here. Maybe move it to a separate document in an Appendix?
Kevin White [med] Introduction seems overly long and touches on content that is covered elsewhere (https://www.w3.org/WAI/test-evaluate/involving-users/) or not necessarily adding anything (stuff on automated testing). Not sure what the key message is within it?
[med] Seeking informed consent may require consideration for local legislation. Does this need to be included?
[med] I wonder if it is worth having this all here? It goes into a reasonable amount of detail to explain usability testing, what it is and how to do it. The most important bit thought is the differences in having people with cognitive impairments participating (which is covered well).

I don't think this would cause confusion, but it does seem to be reiterating content that is presented elsewhere. It just seems a bit inefficient.

Having said that, it is good and I would be comfortable pointing people at it
Shawn Lawton Henry Summary: Most of Section 5 is either already covered in existing WAI resources or was deemed beyond the scope of what W3C/WAI resources should cover. The exception is the subsections of 5.5, which address specifics related to the Objectives in section 4.

Background:
Usability testing is a large topic – whole books are written on it. Including participants with disabilities in usability testing is a big topic – multi-page resources are written on it.

WAI previously decided that it was important to provide a resource on the WAI website that introduced including users, e.g., in usability testing -- and that we would briefly introduce the most important points, and *not cover other issues or the topic in general*. References:
* Analysis/Requirements and Changelog for "Involving Users in Web Accessibility Evaluation" https://www.w3.org/WAI/EO/changelogs/cl-eval-ut.html
* Requirements/Analysis and Changelog for Involving Users in Web Development https://www.w3.org/WAI/EO/changelogs/cl-impl-users.html#about

WAI provides the following resources specifically on including people with disabilities:
* Involving Users in Web Projects for Better, Easier Accessibility
https://www.w3.org/WAI/planning/involving-users/
* Involving Users in Evaluating Web Accessibility
https://www.w3.org/WAI/test-evaluate/involving-users/
And includes related information and pointers in several resources, including:
* Accessibility, Usability, and Inclusion
https://www.w3.org/WAI/fundamentals/accessibility-usability-inclusion/
* Planning and Managing Web Accessibility
https://www.w3.org/WAI/planning-and-managing/

EOWG has also worked hard to avoid covering topics in multiple places on the W3C website. Instead, we seek to put the information in one place, and point to it there from other places where it is relevant.

Comments:
This draft of Section 5 covers topics already covered in WAI resources (listed above), or elsewhere in the Content Usable document. For example, much of 5.2 Finding People to Include is essentially the information from 2.2 Background about People with Learning and Cognitive Disabilities and the Web.

The draft of section 5 goes beyond what WAI determined was in our scope. There would be several problems with extending the scope to include such information. For example, Section 5.3 Informed Consent does not use common terminology, does not cover important issues, and has information that some usability professionals might disagree with.

I suggest that this Content Usable document include in Section 5: 1. pointers to existing information on the WAI website and other section of this document, and 2. the specifics in the 5.5 subsections (and not the other information).
EOWG and the COGA Task Force might also have suggestions for updates to the existing WAI resources.

<draft of suggested text for section 5>

Section 5: Including people with cognitive and learning disabilities in design and evaluation

It is particularly important to include real people with cognitive and learning disabilities in projects. See the following resources on including people with disabilities (“users”):
* "Involving Users in Web Projects for Better, Easier Accessibility":
- describes how involving people with disabilities early in web projects helps
- provides guidance on how to involve users throughout your project, including in planning, designing, and developing
* "Involving Users in Evaluating Web Accessibility":
- encourages informal evaluation early
- provides guidance on including people with disabilities in a range of evaluation, including usability testing

These resources encourage you to get a range of people with different disabilities to participate in your project. To understand the range of cognitive and learning disabilities, see the “Background about People with Learning and Cognitive Disabilities and the Web” section.

For developing materials such as consent forms and usability test task descriptions, use the design patterns under Objective 3: Use Clear and Understandable Content.

The following sections provide specific questions to help evaluate if the design patterns under each Objective are achieved.
[content from 5.5.1 – 5.5.8]

</ END draft of suggested text for section 5>
Sharron Rush

11. Use Cases / Persona (section 6)

Review guidance:

  • Is it clear how this section relates to the other sections?
  • Do the first paragraphs provide a good introduction to this section, related to the rest of the document?

Optional additional review:

  • Does each persona make sense?
  • Are all key user requirements covered?

Details

Responder Comments
Laura Keen Section 6 is well written. I particularly like that each persona has multiple scenarios.
Sylvie Duchateau No time to review now, will review later
Estella Oncins This section is clear, very well structured and properly relates to the other sections. Maybe use different names from other cultures.
It will be great to have a more close link to SC as in appendix C
Daniel Montalvo
Howard Kramer Yes, yes, yes, yes on all questions above. Wondered if examples of good vs. bad design for each persona and scenario would help this section - or pointing to other parts of the document where these issues are addressed would help.
Andrew Arch All the persona's seem to have Anglo-Saxon or European names - we need more cultural diversity
Dónal Rice
Jennifer Chadwick As above, the user personas are robust and relatable.
Vicki Menezes Miller First sentence and first word of next sentence: "Any time there is a 'target audience', there will be people with learning and cognitive disabilities in that audience. However, "
I would delete the above as I find it unclear or rephrase the idea.

Which is the "Developers resources" page. Could that be linked up?

The personas and scenarios are fine.
Kevin White [med] Alison, scenario 1: The scenario starts talking about applications and OS interface elements but the goes on talk about web based interfaces. The latter wouldn't be subject to the change identified in the former. Which makes the request for standard form links less robust. Similarly the reference to decluttering the application menus. Not sure if there are other instances of blurring between web and OS.
Shawn Lawton Henry international names https://github.com/w3c/coga/issues/135
Sharron Rush

12. A. Appendix: Mapping User Needs, Persona and Patterns

Review guidance:

  • Given the size of this document and the target audience does this Appendix fit well?
    • >
  • Does it clearly show the relationship between the content in various sections? Does it help show there are no gaps?
  • Is anything missing?

Details

Responder Comments
Laura Keen I love that Appendix A maps the requirements to the personas and scenarios. It is a great tool for UX professionals to navigate this document and to ensure they've met the requirements.
Sylvie Duchateau No time to review this section for now, may be later.
Estella Oncins A four column with SC will be really helpful
Daniel Montalvo
Howard Kramer
Andrew Arch
Dónal Rice
Jennifer Chadwick Nothing to add but I am happy this is here - same content, presented in a different way to aid some users? This is excellent.
Vicki Menezes Miller The Appendix is very useful but I would really like to have a link to the Pattern in question so that the solution is easy to find.

Related WCAG SC would be very useful in a fourth column.
Kevin White
Shawn Lawton Henry
Sharron Rush

13. B. Appendix: Considerations for Uptake in Different Contexts and Policies

Review guidance:

  • Given the size of this document and the target audience does this Appendix fit well?
    • >
  • Does provide the right amount of information?
  • Is anything missing?

Details

Responder Comments
Laura Keen I think Appendix B is helpful for agencies and organizations that want to include support for people with cognitive disabilities. It gives organizations a focus and a place to start. I can think of anything to add.
Sylvie Duchateau No opinion, no time to review it for now, may be later.
Estella Oncins It would be great to have a mention to Easy-to-Read apart from plain language.
Include a policy for how to interact with users with cognitive disabilities might be helpful specially for testing purposes.
Daniel Montalvo
Howard Kramer
Andrew Arch
Dónal Rice
Jennifer Chadwick
Vicki Menezes Miller At a first glance, it seemed over ambitious given the mention of different geographical areas but, then, reading through, I'm not really sure that I have sufficient guidance... so...
Kevin White
Shawn Lawton Henry
Sharron Rush

14. C. Appendix: Testable Statements for Each Pattern

Review guidance:

  • Given the size of this document and the target audience does this Appendix fit well?
  • Does the content it links to fit the appendix?
  • Is anything missing?

    • Summary

    ChoiceAll responders
    Results

    Details

    Responder C. Appendix: Testable Statements for Each PatternComments
    Laura Keen https://www.w3.org/WAI/GL/task-forces/coga/wiki/Testable_Statements_for_Each_Pattern - mapping the objectives and patterns to WCAG SC is also a great tool for project teams to include support for cognitive disabilities in their SDLC.
    Sylvie Duchateau No opinion, no time to review it for now, may be later.
    Estella Oncins I think that this appendix is of relevant importance specially for user testing
    Daniel Montalvo
    Howard Kramer
    Andrew Arch
    Dónal Rice
    Jennifer Chadwick
    Vicki Menezes Miller The link to these testable statements are great. I feel that they are too hidden in an Appendix. Could they not be referenced in the Patterns section?
    Kevin White
    Shawn Lawton Henry
    Sharron Rush

15. D. Appendix: Business Considerations

Review guidance:

  • How does the content in this section relate to other information provided on the WAI website?
    (e.g., The Business Case for Digital Accessibility and Older Users and Web Accessibility: Meeting the Needs of Ageing Web Users)
    • Is some information duplicated or overlapping in a way that we (COGA TF, EOWG, and others) should address? For example, is there guidance in this document that applies more broadly than for cognitive disabilities and maybe should be incorporated in the main resource instead?
    • Is related information appropriately pointed to?
  • Given the size of this document and the target audience does this Appendix fit well?

Details

Responder Comments
Laura Keen I do think mentioning the Business case is useful in this document.
Sylvie Duchateau No opinion, no time to review it for now, may be later.
Estella Oncins Not clear if a Business Case Appendix is needed
Daniel Montalvo
Howard Kramer
Andrew Arch
Dónal Rice
Jennifer Chadwick
Vicki Menezes Miller Some duplication with WAI resources. I really don't think this section needs further expansion or even needs a specific section on the Business Considerations.
Kevin White
Shawn Lawton Henry Instead of this appendix, just point to existing WAI resources:
* The Business Case for Digital Accessibility https://www.w3.org/WAI/business-case/
* Older Users and Web Accessibility: Meeting the Needs of Ageing Web Users https://www.w3.org/WAI/older-users/
Sharron Rush

16. Organization and Navigation

Review guidance:

  • Is the organization of the document clear?
  • Is it easy to understand the purpose of the different sections, and the target audience and tasks that each section addresses?
  • How easy/difficult is it to navigate within the document?

Details

Responder Comments
Laura Keen I found this document to be well organized and easy to use. Well done!
Sylvie Duchateau No specific problem to navigate through the document.
Estella Oncins Yes, the organization and structure is clear even if it is a really long document.
Not clear if the audience of this document (designers and developers) will clearly understand how to approach the target audience.
Navigation is really well structured and easy to follow.
Daniel Montalvo
Howard Kramer
Andrew Arch
Dónal Rice
Jennifer Chadwick It is clear. There is a lot of information, but the way the document is structured is very helpful and makes me want to return to the document and spend time with it, learn more. It's not overwhelming, rather, it's inviting.
Vicki Menezes Miller 1. Organization: I find it also cumbersome that in Chapter 3, we have User Stories, then, much later in Chapter 6, Personas. Could the Personas be moved to an Appendix?
2. I'm not sure that the target audience is clear in each section. It seems to focus on developers and designers, but content is so important in this case and neither developers nor designers usually take decisions on content writing.
3. I personally find it somewhat difficult to navigate through the document, jumping between the personas, scenarios, patterns.
Kevin White [;ow] Repeated use of 'Pattern' and 'User Story' adds quite a bit of clutter
[low] Using the 'objectives' to structure the user stories and design guides sections makes sense but does make it a wee bit difficult finding where you are.
Shawn Lawton Henry Because the contents list is so long, I found it very difficult to get an overview of the document and to navigate between sections. Even when I turned off line spacing and zoomed out and used my bigger monitor, it was over 10 scrolls to get through the entire Table of Contents. It would be much easier if it the sub-sections were expandable and collapsible. If it all stays in a TR document and that is not feasible, maybe add a high-level contents list with only two levels?
Sharron Rush

17. Other comments?

Any other comments that didn't fit in above?

Summary

ChoiceAll responders
Results

Details

Responder Other comments?Comments
Laura Keen
Sylvie Duchateau I am going on vacation and will unfortunately have no time to review the document further, also it sounds really interesting. I will be available again on August 31st, if final comments/input is needed.
Estella Oncins
Daniel Montalvo
Howard Kramer
Andrew Arch Need to avoid metaphors as per the document's advice - D.1 has a paragraph starting with "On the other hand ...". 6.2.3 also use that metaphor.

Quite a lot of the writing could also be written more 'plainly', eg dropping various sections into the HemingwayApp suggests much is 'hard' or very hard' to read and there are many instances of 'passive voice'.
Dónal Rice Apologies I didn't spend more time reviewing. This looks quite a developed resource but I did not have sufficient time to go into it in detail.
Jennifer Chadwick No other comments in the sections where I have not put any comment. I'm excited to use this!
Vicki Menezes Miller There is a lot of great information but I feel the document would be more useful if much shorter. The length makes it simply tiring.
I also find that I have to jump around the document a lot in search of a solution to a problem... maybe you can find a way of getting around that by adding links as necessary.
Perhaps put the Personas in an Appendix?
Kevin White Amazing document with a huge amount of really useful information - thank you so much!!!
Shawn Lawton Henry "link rot" can be a real problem. I'm surprised at links that have gone bad, even on major governmental websites. Consider not linking outside W3C in this /TR/ doc, and instead linking to a wiki page where you can update the links much easier. (Although some might be uncomfortable wit a TR doc linking to a wiki page?)

[minor] exclamation marks in several places seem unprofessional and not appropriate for the content or type of document.

[minor] fix inconsistent capitalization and wording throughout, e.g., web page, Web page, webpage. e.g., in Abstract “Web content (Web pages) and Web applications” whereas the WAI style guide has lowercase “web” and it is lowercase in most place throughout this document.
Sharron Rush

More details on responses

  • Laura Keen: last responded on 5, August 2020 at 14:27 (UTC)
  • Sylvie Duchateau: last responded on 7, August 2020 at 15:05 (UTC)
  • Estella Oncins: last responded on 12, August 2020 at 14:20 (UTC)
  • Daniel Montalvo: last responded on 14, August 2020 at 11:27 (UTC)
  • Howard Kramer: last responded on 17, August 2020 at 00:14 (UTC)
  • Andrew Arch: last responded on 17, August 2020 at 12:27 (UTC)
  • Dónal Rice: last responded on 17, August 2020 at 16:31 (UTC)
  • Jennifer Chadwick: last responded on 17, August 2020 at 23:22 (UTC)
  • Vicki Menezes Miller: last responded on 18, August 2020 at 00:28 (UTC)
  • Kevin White: last responded on 19, August 2020 at 15:26 (UTC)
  • Shawn Lawton Henry: last responded on 22, August 2020 at 18:08 (UTC)
  • Sharron Rush: last responded on 24, August 2020 at 15:04 (UTC)

Non-responders

The following persons have not answered the questionnaire:

  1. Eric Velleman
  2. Shadi Abou-Zahra
  3. Kazuhito Kidachi
  4. Jedi Lin
  5. David Sloan
  6. Mary Jo Mueller
  7. Reinaldo Ferraz
  8. Bill Kasdorf
  9. Cristina Mussinelli
  10. Kevin White
  11. Kevin Rydberg
  12. Adina Halter
  13. Denis Boudreau
  14. Sarah Pulis
  15. Bill Tyler
  16. Gregorio Pellegrino
  17. Ruoxi Ran
  18. Sean Kelly
  19. Muhammad Saleem
  20. Sarah Lewthwaite
  21. Mark Palmer
  22. Jade Matos Carew
  23. Sonsoles López Pernas
  24. Greta Krafsig
  25. Jason McKee
  26. Jayne Schurick
  27. Billie Johnston
  28. Michele Williams
  29. Shikha Nikhil Dwivedi
  30. Brian Elton
  31. Julianna Rowsell
  32. Tabitha Mahoney
  33. Fred Edora
  34. Rabab Gomaa
  35. Marcelo Paiva
  36. Eloisa Guerrero
  37. Leonard Beasley
  38. Frankie Wolf
  39. Supriya Makude
  40. Aleksandar Cindrikj
  41. Angela Young

Send an email to all the non-responders.

Compact view of the results / list of email addresses of the responders

WBS home / Questionnaires / WG questionnaires / Answer this questionnaire

Report issues on GitHub project w3c/wbs-design or by mail to sysreq.