Requirements/Analysis and Changelog for Involving Users in Web Development
Page Contents
Changes Since Publication
Involving Users in Web Projects for Better, Easier Accessibility
Changes on 7 April 2010 version (from December 2009):
- misc minor edits. If you need a detailed list of changes, contact shawn@w3.org
Involving Users in Evaluating Web Accessibility
Changes on 3 August 2010 version:
- Added last phrase in next to laast bullet at the bottom: Many books, articles, conference presentations, and other resources cover usability evaluation techniques, including different types of usability testing; test design; developing test protocol including questionnaires, tasks, data collection; conducting pilot tests; and how many participants to include in usability testing. For example: Usability Testing Demystified , sample test plans and forms from The Handbook of Usability Testing , and Rocket Surgery Made Easy: The Do-It-Yourself Guide to Finding and Fixing Usability Problems with a sample usability test, scripts, and forms online.
Changes on 7 April 2010 version (from December 2009):
- misc minor edits. If you need a detailed list of changes, contact shawn@w3.org
This Changelog is an update of "Requirements and Changelog for 'Involving Users in Web Accessibility Evaluation", for the predecessor to these documents.
About "Involving Users"
This document will provide an overview of including people with disabilities and older people throughout Web design and development processes. It builds on the existing document “Involving Users in Web Accessibility Evaluation”, available at http://www.w3.org/WAI/eval/users.html
Scope
Will:
- be a broad introduction for including users throughout the process (with examples)
- motivate and encourage developers to get some users involved with a few simple steps
- explain why including users is so important (to help developers avoid providing well-meant but wrong solutions
- show that it can be done on a small, informal scale (but that there are professionals who can assist)
- identify where people are likely to mess up and provide some pointers to help them get it right (or examples)
- point to external resources for more details
- encourage adoption of a user-centered design process
Will not:
- cover fine detail on involving users in every aspect of the 'development lifecycle'
- be a step-by-step 'how to' for usability testing nor a comprehensive resource for usability testing with participants with disabilities
Note: The focus is real live users but should point to references that say more about general consideration of accessibility earlier.
Purpose, Goals, Objectives
Rationale:
- Many Web developers do not involve users in the design process. Even those who do involve users, rarely include people with disabilities or older users.
- There are many benefits of including users, especially older users and people with disabilities, in the development process from the beginning of the project.
- Bringing users into the process from the early stages helps designers and developers better understand accessibility issues and better implement effective accessibility solutions. It also encourages early incorporation of accessibility solutions, rather than trying to make adjustments at the end of the development process that are often more difficult to implement
- Understanding users' needs should also reduce user complaints
Objective:
- Provide basic information to help designers and developers get started with including people with disabilities and older users at all stages of Web design and development
- Encourage developers to involve users early and often
- Encourage developers to do whatever they can - reminding them that informal involvement of users is more useful than no involvement
Notes:
- Broaden the scope of the existing document from evaluation only to include/discuss all stages of the development process
- Ensure that older users are specifically included in the document
Audience
Primary audience:
- Web developers (designers, coders, etc.) who want to create accessible websites
- Website purchasers/procurers & project managers - want to know "how and when to bring in users"
- Standards developers and policy developers [maybe not specifically address, but make the beginning information generic enough that it clearly applies]
- Browser and user agents developers
Secondary audience:
- Anyone interested in better understanding the basics of involving users with disabilities in Web development - including educators, researchers, decision makers, accessibility professionals, professional evaluators
- Users who are interested to participate in the development process (and may want to approach an organization to ask to be involved)
- Content authors - understanding how people use their content will help write better, provide better alt-text, better links, etc
- Usability professionals who want to improve the accessibility of the websites they work on [action: leave this one out of consideration until we have a useful document for developers and others, and then we can see how best to address something to this group]
A couple of scenarios to keep in mind:
- small local business e.g., takeaway restaurant, developing a first site or first replacement site. Both developer and business owner may be relatively inexperienced at web development. This is a chance to review similar types of sites, to look for examples of good practice, look for different formats and contents. You need to consider what kind of tasks and activities, what information needs, and are there different strategies for providing the same information e.g., a static timetable vs an interactive query system, or different approaches to content management. Review of own site – might include any complaints and observing some real people doing typical tasks in a variety of contexts. Consider alternative use scenarios e.g., using a shared access machine in a library or senior citizen club, a person using zoomed text, a screen reader or non-mouse entry. If using multimedia what would be the experience of a person who is blind or hearing impaired.
- medium sized company or service provider – The previous site may have been running 2-3 years and evolved overtime. The IT department is an 'operator' not a developer and engages a professional web developer to upgrade existing system. This is a chance to review existing site, check out any record of complaints. Identify weaknesses in structural design and formats by observing invited users attempting typical tasks and less common tasks. Pay particular attention to issues identified in WCAG 2.0 guidelines that may result in barriers or reduced efficiency related to perception, operation and understanding. Seek out users with different profiles and accessibility needs who can help to specify critical requirements for the new site.
Notes
- Size:
- less than 3?4?5 printed pages
- Incorporate into Implementation Plan for Web Accessibility on completion
- see Archive below for previous wish-list and considerations (may need expanding with respect to older users)
- Perhaps point to other resources for how to work with users in general? Andrew want to research? (perhaps Why We Call Them Participants)
- which resources to point to?
References
WAI-AGE task force and EOWG Discussions:
- EOWG 5 March 2010
- EOWG 13 November 2009
- 12 November 2009 draft
- EOWG f2f 2 November 2009
- 31 October draft
- EOWG telecon 23 October 2009
- 23 October draft
- AA's 22 Oct draft
- WAI-AGE TF teleconference - 21 October 2009
- WAI-AGE TF telecon - 1 July 2009
- WAI-AGE TF telecon - 20 May 2009
- WAI-AGE TF telecon - 6 May 2009
- EO Meeting 23 October 2008
Related documents:
- Involving Users in Web Accessibility Evaluation - www.w3.org/WAI/eval/users.html
- Implementation Plan for Web Accessibility - www.w3.org/WAI/impl/Overview.html
- WAI-AGE project - www.w3.org/WAI/WAI-AGE/
- WAI-AGE deliverables - www.w3.org/WAI/WAI-AGE/deliverables.html#involve
- Notes on [Usability and] User Centered Design Process - www.w3.org/WAI/redesign/ucd
External references:
- Just Ask: Integrating Accessibility Throughout Design (Henry) - www.uiaccess.com/accessucd/
- Usability Testing Demystified (Dana Chisnell 2009) - good, short summary
Testing in the Wild, Seizing Opportunity (Chisnell) - not as good as Usability Testing Demystified - Bringing Accessibility into the Development Process - basic early not ucd
- The Lifecycle of Web Accessibility (Antonio Volpon evolt 2002) - some stuff outdated to the point of being wrong
- Accessibility testing - didn't see anything particularly important that's not in Just Ask
- Why We Call Them Participants (Chisnell) - ok, but a little too casual maybe for pointing to what we want to get across
- Discount Usability: 20 Years (Jacob Nielsen) - not comfortable linking to, doesn't seem to provide relevant info and includes things like "So yes, it does pay to take my courses :-)"
- User testing (NCBI/CFIT) - www.cfit.ie/user-testing - describes there service, so uncomfortable listing as a resources for additional info
- User testing for accessibility (RNIB) - disagree with some info
- Conducting user evaluations with people with disabilities (IBM) - good info, though nothing that's not in Just Ask -- so link to anyway, or not?
- User Testing: How to Involve Users in Technical Web Development (ICCHP) - not online
- Focus groups vs. usability testing (Webcredible) - too specific
- Getting the Right Participants (HFI) - maybe too specific?
- Accessibility basics - has some good info & perspectives, though not specificaly-related to these docs enough to list, I think
- and thousands more, including those listed at Usable Web, Usability and User Experience Resources, Web Design References: Usability, User Centered Design Resources, etc...
- Involving PWD in Standards work
- Involving people with disabilities in the standardisation process (John Gill & COST219ter) - good backgrouns, but not specific enough to justify inclusion
- USEM recommendations for involving users (USEM) - good recommendations from a European project which aims to facilitate, enhance and increase qualification and participation of disabled or elderly users and their respective organisations in the European standardisation process - include?
- Guides Related to Inclusion of Persons with Disabilties (Stand4All) - list of additional standards refs for consideration (including from CEN, ISO, etc)
Consider for next revision
1. "You could say that involving users with disabilities in your development project gives you improved usability for free."- add link to supporting studies in the business case appendix when that's done.
2. when it's done, consider adding to the More Info resource list: <a href="http://www.w3.org/WAI/EO/Drafts/access-use/accessibility-n-usability.html">Relationship between Web Accessibility and Usability@@</a>
3. Putting back in examples. Previous draft and changelog:
Examples of Benefits<h2>
@@changelog: add short "handle"/heading for each. maybe add "Appendix" to heading. reorder ending sections]]
@@changelog: include other scenarios (e.g., cool new app developers? specs developer!) policy &/or researcher developing ax &/or older user, w/o including both. (how make compelling ?)]]
Including users in the development process helps you more efficiently develop accessible websites that work well for people.
e.g.,
Understanding how people use your website, browser, or other tool, helps you focus your efforts on accessibility solutions that work well for real users in real situations.
e.g.,
Understanding users gives you a better perspective on standards and guidelines.
e.g.,
[@@ edit above if leave]]
Changelog of Revisions to Drafts
Note: See "Previous Changelog" for 2005 meeting minutes and e-mail comments.
9 March 2010 Edits (based on comments and EOWG teleconference 5 March)
Involving Users in Web Projects for Better, Easier Accessibility 9 March edit
- Rewrote Introduction. Moved first sentence under "How Involving Users Early Helps" to Introduction.
- Edited: "Below are the basics that you can do yourself to include users in developing websites and other web products." to "Below are the basics that you can do yourself to include users in your projects."
- Under "Including Users in Implementation" moved out of the bullet into its own paragraph: "For more in this, see Involving Users in Evaluating Web Accessibility, especially..."
Involving Users in Evaluating Web Accessibility 9 March edit
- In Introduction, last paragrpah, first sentence: changed from "different approaches for" to "different aspects of", resulting in: "This page is part of a multi-page Evaluating Web Accessibility resource suite that outlines different aspects of evaluating web accessibility."
- Edited first paragraph under "Initial Review'
- Edited Note for usability professionals section
- Changed "For any accessibility problems you identify, determine which components are responsible." to "Accessibility problems can be caused by one or more different components."
- Added middle sentence under Drawing Conclusions and Reporting: "Be careful drawing conclusions from limited evaluations or studies. Results from only a couple of people with disabilities cannot be generalized to apply to all people with similar disabilities or people with other disabilities. See the Caution above."
30 Nov New Doc
- Deleted Examples section
- Deleted Notes for usability professionals placeholder section
12 Nov New Doc
- deleted first paragraph: "A key aspect of designing for usability is including users throughout project development. [Similarly|Likewise], a key aspect of successfully designing for accessibility is including users with disabilities throughout project development."
26 October Eval Doc
- title "Involving Users in Web Accessibility Evaluation" to Web Accessibility Evaluation with Users
22 October 2009
- Added structure and notes to the draft document in preparation for EO discussion
- Updated Requirements in light of TF (21/Oct/09) discussion
20 October 2009
- Drafting of introduction
- Discussion from TF (20/May/09 & 01/Jul/09) incorporated for discussion
- External references noted above
25 June 2009
- Requirements document (this document) updated to become more structured
- Initial opportunities flagged in DRAFT - Involving Users in Web Development for TF discussion
11 May 2009
- Requirements document (this document) updated to reflect TF discussion at 6 May 2009 teleconference
28 January 2009
- Requirements document (this document) updated to reflect EOWG & TF discussion at October 2008 meeting
- Actions from October 2008 EOWG & TF meeting:
- amend 1st bullet from 'designing for Web accessibility' to more simply 'web design' -- also throughout, e.g., the title [recorded in http://www.w3.org/2008/10/23-eo-minutes.html#action02]
- change sub bullet 1 to 'introduce the concept of including users at all stages of your design and development process (from the start to the end)' [recorded in http://www.w3.org/2008/10/23-eo-minutes.html#action03]
- under the cautions bullet, mention that users aren't necessarily experts with solutions [recorded in http://www.w3.org/2008/10/23-eo-minutes.html#action04]
- consider (at some point) "usability testing" or "user testing" [recorded in http://www.w3.org/2008/10/23-eo-minutes.html#action05]
- make sure the requirements title and intro refers to this as an update/evolution of the existing 'involving users' [recorded in http://www.w3.org/2008/10/23-eo-minutes.html#action06]
- change 'usability testing professionals' to 'usability professionals' in audience note [recorded in http://www.w3.org/2008/10/23-eo-minutes.html#action07]
- consider adding a subsection on "Benefits" which would probably include the current 2nd, 3rd, 4th paragraphs [recorded in http://www.w3.org/2008/10/23-eo-minutes.html#action19]
- requirements:
- audience:
- include usability professionals in primary audience and see if we can write for both (but be willing to relegate them if it doesn't work) [recorded in http://www.w3.org/2008/10/23-eo-minutes.html#action08]
- include educators along with usability professionals [recorded in http://www.w3.org/2008/10/23-eo-minutes.html#action09]
- separate educators as it's own secondary audience (as they are teaching/guiding not doing) [recorded in http://www.w3.org/2008/10/23-eo-minutes.html#action10]
- secondary audience - include accessibility professionals [recorded in http://www.w3.org/2008/10/23-eo-minutes.html#action11]
- 'encourage' - change sub bullet to be more positive, and mention earlier involvement [recorded in http://www.w3.org/2008/10/23-eo-minutes.html#action12]
- 'encourage 2' - drop "in user testing" [recorded in http://www.w3.org/2008/10/23-eo-minutes.html#action13]
- 'cautions' bullet - add: try to introduce them in a positive language; subtract "most serious" [recorded in http://www.w3.org/2008/10/23-eo-minutes.html#action14]
- consider adding "time of reading" to constrain the length, e.g. 1 hour [recorded in http://www.w3.org/2008/10/23-eo-minutes.html#action15]
- ditch "more to encourage ..." [recorded in http://www.w3.org/2008/10/23-eo-minutes.html#action16]
- change "move resource from E to I ..." to a note [recorded in http://www.w3.org/2008/10/23-eo-minutes.html#action17]
- consider whether this applies to web content developers only, or broader to include authoring tools, browsers, etc. (further discussions on the document may clarify this) [recorded in http://www.w3.org/2008/10/23-eo-minutes.html#action18
- audience:
Archive
Format:
WAI resource; single Web pageStructure (draft):
- Introduction
- Involving users effectively
- Including diverse users
- Analyzing accessibility problems
- Drawing conclusions
- Related resources - including external resources
- Glossary (limited)
- @@ ethics and consent forms (where to fit in)
Tasks:
- Review the existing document for opportunities to more explicitly include older users, integrating information learned from the WAI-AGE Literature Review, for example, possibly expanding the glossary (Terminology and Notes) section
- Expand the existing document to cover including users early and throughout the design and development process (rather than just in evaluation), including older users
- Seek community review
Title brainstorms:
- Including Users for Better, Easier Accessibility
Involving Users for Better, Easier Accessibility - Involving Users in Web Development
- Involving Users in Web Accessibility Design and Evaluation
- Involving Users in Web Accessibility Development
- ...
Deleted from 2005 version
- a bunch of stuff specifically on evaluation...
- One of the benefits of including people with disabilities is that Web developers can learn how people with disabilities interact with the Web and with assistive technologies.
Consider for revision or related documents or other
Eval doc from 2009
- check what would be involved in broadening it to tools, etc. see what did in new companion doc
- Note at end of Introduction - maybe soften and integrate briefly earlier
- Note at end of Basics - see how might fit in intro
- under Drawing Conclusions and reporting, consider including this example: A study looking at the relationship between accessibility and usability of websites used only users who are blind compared with users who are sighted. The study results cannot be applied to all users with disabilities nor used to compare usability and accessibility issues broadly, but that was not made clear in the report.
Eval doc from 2005
- [done] Reduce spacing before top of list with previous paragraph
- Add image of green magnifying glass & look for other graphic opportunities
- consider change:
location: "Introduction" section, second paragraph
current wording: example of screen reader user
suggested revision: example of screen magnification
rationale: screen readers are mentioned elsewhere in the document (for example in "Involving Users Effectively") and often not an obvious AT to Web developers. - Link to when done:
- in For More Info section: [selecting consultants] with the blurb organizations that can help
- in For More Info section: How People with Disabilities Use the Web describes how different disabilities affect Web use and accessibility requirements for people with different kinds of disabilities, and includes scenarios of people with disabilities using the Web.
- in Terminology section: Basic Glossary (formerly known as (fka) Lexicon)
- document addressing issues of usability & accessibility & "basic accessibility" and "usable accessibility"
- consider if want to tweak to meet additional "requirements"
- side goal: creating evangelists
- audience: includes tool developers
- scope: including throughout design & development, not just eval [already cover that some...]
Important messages from 2005:
- one user not representative, don't do everything one user says
- distinguish between usability vs. accessibility issues
- distinguish between issues of user agent, website, assistive technology, user knowledge, ...
- more than just people who are blind
- Since this is part of the Evaluation Suite, focus on user involvement in accessibility testing; however, since ideally users are incorporated throughout the process, mention getting them involved from the start and throughout
- Carefully clarify that usability testing is not a requirement to ensure comply with WCAG
- level of user expertise (depends on target audience) - too advanced might know uncommon work-arounds, not advanced enough may not know thinks like links lists, headings nav, etc.) - people often mis-state their own level
- issues with non-PWDs using AT - false negatives, 'cause don't know how to use screen reader, think it's hard 'cause they don't know how to use them...
- stakeholders - live observation!!! (or at least highlight tapes), rather than dry report...
- issue: terminology. does "usability testing" conjure up formal testing through complete tasks too much? if so, should we use something else, such as "user involvement in accessibility testing" or "testing accessibility features with users" or ?? that is not too awkward?