View source for Review: Guidelines are only half of the story

[[Category:Review]]
<div id="intro" style="padding: 0em 1em 0em 0.5em; margin: 0em 0em 1em; width: 90%; border-left: .111em dotted #ccc;">
* '''Study Reviewed''': [http://dl.acm.org/citation.cfm?id=2207736 Guidelines are only half of the story: accessibility problems encountered by blind users on the web] by Christopher Power, Andri Pimenta Freire, Helen Petrie, and David Swallow, University of York, York, North Yorkshire, United Kingdom, 2012.
* '''Reviewed for''': [https://www.w3.org/WAI/GL/wiki/Post_WCAG_2_Issues_Sorted#Articles_to_review_and_pull_individual_items_from Post WCAG 2 Issues Sorted: Articles to review and pull individual items from, row 8]
* '''Reviewed by''': Laura Carlson
* '''Note''': This document is a Wiki-fied [http://www.d.umn.edu/~lcarlson/wcagwg/reviews/guidelines_only_half_the_story.html copy of the original].

</div>

<div id="summary" style="font-size: 1.1em; padding: 0.7em 1em 0.7em 0em;">

'''Summary''': In sum the usability study, [http://dl.acm.org/citation.cfm?id=2207736 Guidelines are only half of the story: accessibility problems encountered by blind users on the web] (henceforth referred to as "the study"), presents several usability issues as accessibility issues. In the short term [https://www.w3.org/WAI/GL/wiki/Review:_Guidelines_are_only_half_of_the_story#One_Issue_May_Warrant_Action_Short_Term one issue may warrant WCAG WG action.] The study expands the concept of accessibility to include usability. Expanding accessibility to include usability has previously occurred and is currently occurring in other venues including the United States Department of Education, [https://www.w3.org/WAI/GL/wiki/Review:_Guidelines_are_only_half_of_the_story#ocr Office for Civil Rights (OCR) resolution agreements]. This raises broader questions such as is a tighter integration of usability and accessibility needed? Should the World Wide Web Consortium (W3C) Web Accessibility Initiative (WAI) consider furthering efforts in the area of "Usable Accessibility" and User Centered Design (UCD)? Are broader design principles in order? These longer term questions may deserve contemplation if the Working Group has not already considered and come to consensus on them

</div>

== Literature Review ==

The study was discussed in an article by Detlev Fischer, on the WebAIM mailing discussion list, and on a SitePoint Form.

<table style="border-color: #d3d1c2; margin: 1.5em; max-width: 50em;
padding-left: 0.5em;">

<caption style="font-size: 1.2em; font-weight: 400;padding-left: 0.5em; text-align: left;">
Prior Discussion
</caption>

<tr>
<th scope="col" style="background-color: #d3d1c2; color: #000; font-weight: 600; padding : 0 0.5em; text-align: left vertical-align: bottom;">Source</th>
<th scope="col" style="background-color: #d3d1c2; color: #000; font-weight: 600; padding : 0 0.5em; text-align: left vertical-align: bottom;">Synopsis</th>
</tr>

<tr>
<td style="background-color: #f0e9d1; border-color: #d3d1c2; color: #000; padding: 0 0.5em; text-align: left; vertical-align :top;">Fischer, Detlev. [http://www.bitvtest.eu/articles/article/lesen/chi2012-wcag.html Methodological flaws put question mark on study of the impact of WCAG on user problems], 2012.</td>

<td style="background-color: #f0e9d1; border-color: #d3d1c2; color: #000; padding: 0 0.5em; text-align: left; vertical-align :top;">Discounts  most of  the study's data, metrics, analysis and conclusions, except the issue of clearly identifying links.</td>
</tr>

<tr>
<td style="background-color: #f0e9d1; border-color: #d3d1c2; color: #000; padding: 0 0.5em; text-align: left; vertical-align :top;">Gunnarsson, Birkir R. et. al. [http://webaim.org/discussion/mail_thread?thread=5314 Thread: Guidelines are only half of the story: accessibility problems encountered by blind users on the web], <cite>WebAIM mailing list   discussion</cite>, 2012.</td>

<td style="background-color: #f0e9d1; border-color: #d3d1c2; color: #000; padding: 0 0.5em; text-align: left; vertical-align :top;"> General agreement that the 6  issues, which the study called out as not covered in WCAG 2.0  were not  something that accessibility guidelines could or should  specifically address.</td>
</tr>

<tr>
<td style="background-color: #f0e9d1; border-color: #d3d1c2; color: #000; padding: 0 0.5em; text-align: left; vertical-align :top;">Mallory van Achterberg. [http://community.sitepoint.com/t/research-shows-adhering-to-wcag-doesnt-solve-blind-users-problems/15769 Research shows adhering to WCAG doesn't solve blind users' problem,] SitePoint forum, 2012.</td>

<td style="background-color: #f0e9d1; border-color: #d3d1c2; color: #000; padding: 0 0.5em; text-align: left; vertical-align :top;">Found that issues presented in the study were not specifically accessibility related but rather usability  related and outside of WCAG's scope. The author suggests that if  usability criterion  were to be increased in WCAG, WAI's definition of &quot;accessibility&quot; would require broadening.</td>
</tr>

</table>

== Confusing Usability Issues with Accessibility Issues ==

Accessibility and usability are closely related and often conflated, as they both can improve satisfaction, effectiveness, and efficiency.

In the WebAIM thread [http://webaim.org/discussion/mail_message?id=20370 Andrew Kirkpatrick illuminated], of the specific issues raised in this particular study, 6 are not covered in WCAG 2.0. In that same WebAIM thread Leonie Watson comments how it is difficult to map these issues to WCAG. It is difficult to map them because they appear to be usability issues.

ISO 9241 explains usability as the "extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction ([https://www.w3.org/WAI/GL/wiki/Review:_Guidelines_are_only_half_of_the_story#Definitions consult definitions] ) in a specified context of use." Specifically this means designing systems or products that have the following characteristics:

* Effective
* Efficient
* Engaging
* Error tolerant
* Easy to learn

=== Mapping Issues to Usability Characteristics ===

<table style="border-color: #d3d1c2; margin: 1.5em; max-width: 50em; padding-left: 0.5em;">
<caption style="font-size: 1.2em; font-weight: 400;padding-left: 0.5em; text-align: left;">
Reported Issues Mapped to Usability Characteristics
</caption>

<tr>
<th scope="col" style="background-color: #d3d1c2; color: #000; font-weight: 600; padding : 0 0.5em; text-align: left vertical-align: bottom;">Number</th>

<th scope="col" style="background-color: #d3d1c2; color: #000; font-weight: 600; padding : 0 0.5em; text-align: left vertical-align: bottom;">Issue</th>

<th scope="col" style="background-color: #d3d1c2; color: #000; font-weight: 600; padding : 0 0.5em; text-align: left vertical-align: bottom;">Usability Characteristic</th>
</tr>

<tr>
<td id="1" style="background-color: #f0e9d1; border-color: #d3d1c2; color: #000; padding: 0 0.5em; text-align: left; vertical-align: top;">1</td>

<td style="background-color: #f0e9d1; border-color: #d3d1c2; color: #000; padding: 0 0.5em; text-align: left; vertical-align :top;">Content found in pages where not expected by users</td>

<td style="background-color: #f0e9d1; border-color: #d3d1c2; color: #000; padding: 0 0.5em; text-align: left; vertical-align :top;">Efficiency</td>
</tr>

<tr>
<td id="2" style="background-color: #f0e9d1; border-color: #d3d1c2; color: #000; padding: 0 0.5em; text-align: left; vertical-align :top;">2</td>

<td style="background-color: #f0e9d1; border-color: #d3d1c2; color: #000; padding: 0 0.5em; text-align: left; vertical-align :top;">Content not found in pages where expected by users</td>

<td style="background-color: #f0e9d1; border-color: #d3d1c2; color: #000; padding: 0 0.5em; text-align: left; vertical-align :top;">Effectiveness</td>
</tr>

<tr>
<td id="3" style="background-color: #f0e9d1; border-color: #d3d1c2; color: #000; padding: 0 0.5em; text-align: left; vertical-align:top;">3</td>

<td style="background-color: #f0e9d1; border-color: #d3d1c2; color: #000; padding: 0 0.5em; text-align: left; vertical-align:top;">Pages too slow to load</td>

<td style="background-color: #f0e9d1; border-color: #d3d1c2; color: #000; padding: 0 0.5em; text-align: left; vertical-align:top;">Effectiveness</td>
</tr>

<tr>
<td id="4" style="background-color: #f0e9d1; border-color: #d3d1c2; color: #000; padding: 0 0.5em; text-align: left; vertical-align:top;">4</td>

<td style=
"background-color: #f0e9d1; border-color: #d3d1c2; color: #000; padding: 0 0.5em; text-align: left; vertical-align:top;">No alternative to document format (e.g. PDF)</td>

<td style="background-color: #f0e9d1; border-color: #d3d1c2; color: #000; padding: 0 0.5em; text-align: left; vertical-align:top;">Effectiveness</td></tr>

<tr>
<td id="5" style="background-color: #f0e9d1; border-color: #d3d1c2; color: #000; padding: 0 0.5em; text-align: left; vertical-align:top;">5</td>

<td style="background-color: #f0e9d1; border-color: #d3d1c2; color: #000; padding: 0 0.5em; text-align: left; vertical-align:top;">Information architecture too complex (e.g. too many steps to find pages)</td>

<td style="background-color: #f0e9d1; border-color: #d3d1c2; color: #000; padding: 0 0.5em; text-align: left; vertical-align:top;">Efficiency</td>
</tr>

<tr>
<td id="6" style="background-color: #f0e9d1; border-color: #d3d1c2; color: #000; padding: 0 0.5em; text-align: left; vertical-align:top;">6</td>

<td style="background-color: #f0e9d1; border-color: #d3d1c2; color: #000; padding: 0 0.5em; text-align: left; vertical-align:top;">Broken links</td>

<td style="background-color: #f0e9d1; border-color: #d3d1c2; color: #000; padding: 0 0.5em; text-align: left; vertical-align:top;">Error tolerance</td></tr>

</table>
         
== One Issue May Warrant Action Short Term ==

The issue of "Content found in pages where not expected by users" could possibly be addressed by reinstating WCAG 1 [http://www.w3.org/TR/WCAG10/#tech-meaningful-links Checkpoint 13.1]. As Detlev Fischer stated in [http://www.bitvtest.eu/articles/article/lesen/chi2012-wcag.html Methodological flaws put question mark on study of the impact of WCAG on user problems]<nowiki>:</nowiki>

<blockquote style="background-color: #f4f4f4; border: #dedede solid 0.1em; border-radius: 2em; color: #000; font-size: 1.03em; margin: 1em 3em; padding: 1em;">

The study's conclusion that the [http://www.w3.org/WAI/WCAG20/quickref/#navigation-mechanisms-refs-sufficient-head sufficient techniques for SC 2.4.4 Link Purpose] are in fact not sufficient at least for the group of blind users strikes me as valid. WCAG 1.0 recommended, in Checkpoint 13.1: "Clearly identify the target of each link. Link text should be meaningful enough to make sense when read out of context." WCAG 2.0 has relaxed this requirement by allowing authors to provide the purpose in the link's context. For blind users who rely on generated indices of headings, WCAG 2.0 is clearly a step backwards. As the lack of descriptive links will also contribute to other frequent problems such as "Content not found in pages where expected by users", it may indeed be worth rethinking SC 2.4.4 in future work on WCAG.

</blockquote>

== Usability Does Not Equal WCAG Conformance ==

A Web site can conform when evaluated against WCAG 2.0 yet be difficult to use for both people with disabilities and people without disabilities. Issues called out in the study (Content found in pages where not expected, content not found in pages where expected, pages that load slowly, no alternative to document formats, complex information architecture, and broken links) will likely affect all users not only people with disabilities.

Shawn Henry points out in [http://www.uiaccess.com/accessucd/ut.html Accessibility in User-Centered Design: Usability Testing] that usability testing,

<blockquote style="background-color: #f4f4f4;border: #dedede solid 0.1em; border-radius: 2em; color: #000; font-size: 1.03em; margin: 1em 3em; padding: 1em;">

...doesn't evaluate conformance to accessibility standards.

</blockquote>

== Others have Expanded Accessibility to Include Usability ==

The authors of the study are not alone in combining accessibility and usability. Brian Kelly introduced the [http://opus.bath.ac.uk/425/ concept of holistic approaches to accessibility] in 2004. This approach to accessibility takes into account factors such as usability.

<div id="ocr">Much more recently the definition of "accessible" has been expanded to include usability in [http://www2.ed.gov/about/offices/list/ocr/index.html United States Department of Education, Office for Civil Rights] (OCR) resolution agreements. The OCR and the U.S. Department of Justice (DOJ) share enforcement responsibility for academic and public accommodations under the Americans with Disabilities Act and its 2008 Amendments and Section 504 of the Rehabilitation Act. These agencies have the authority to conduct a compliance audit or to initiate an investigation in response to a complaint, which can be filed by anyone. These agencies will often seek to enter into a resolution agreement with the subject institution in lieu of conducting an investigation and seeking sanctions or bringing a lawsuit. The OCR has begun to use the following definition:

<blockquote style="background-color: #f4f4f4;border: #dedede solid 0.1em; border-radius: 2em; color: #000; font-size: 1.03em; margin: 1em 3em; padding: 1em;">

"Accessible" means a person with a disability is afforded the opportunity to acquire the same information, engage in the same interactions, and enjoy the same services as a person without a disability in an equally effective and equally integrated manner, with substantially equivalent ease of use. The person with a disability must be able to obtain the information as fully, equally and independently as a person without a disability. Although this might not result in identical ease of use compared to that of persons without disabilities, it still must ensure equal opportunity to the educational benefits and opportunities afforded by the technology and equal treatment in the use of such technology.

<cite>Sources: [https://www2.ed.gov/about/offices/list/ocr/docs/investigations/11116002-b.pdf South Carolina Technical College System Resolution Agreement (PDF)], [http://www2.ed.gov/documents/press-releases/university-cincinnati-agreement.pdf University of Cincinnati Resolution Agreement (PDF)], [http://www2.ed.gov/documents/press-releases/youngstown-state-university-agreement.pdf Youngstown State University Resolution Agreement (PDF)]</cite></blockquote>
</div>

== Usable Accessibility ==

The study is titled "Guidelines are only half of the story: accessibility problems encountered by blind users on the web." The first half of this title is significant as it expands accessibility to include usability, which the authors conclude is the other half of the story.

It is significant to recognize that usability is an important aspect for people with disabilities. Consciously addressing "Usable Accessibility" may help clarify the difference between what meets WCAG and what is actually usable by people with disabilities. The [http://www.w3.org/WAI/intro/usable#usable W3C Web Accessibility Initiative (WAI) explains]<nowiki>: </nowiki>

<blockquote style="background-color: #f4f4f4;border: #dedede solid 0.1em; border-radius: 2em; color: #000; font-size: 1.03em; margin: 1em 3em; padding: 1em;">

"Usable accessibility combines usability and accessibility to develop positive user experiences for people with disabilities. User-centered design processes (UCD) include both techniques for including users throughout design and evaluation, and using guidelines for design and evaluation. UCD helps make informed decisions about accessible design. Thus UCD is necessary to improve accessibility in websites and web tools...The goal of web accessibility is to make the Web work well for people, specifically people with disabilities. While technical standards are an essential tool for meeting that goal, marking off a checklist is not the end goal. People with disabilities effectively interacting with and contributing to the Web is the end goal. To make the Web work well for people with disabilities, designers and developers need to understand the basics of how people with disabilities use the Web. Following UCD to involve people with disabilities throughout design processes and involve users in web accessibility evaluation helps design solutions that are effective for users and for developers."

</blockquote>

The "Guidelines are only half of the story" study concludes by stating:

<blockquote style="background-color: #f4f4f4;border: #dedede solid 0.1em; border-radius: 2em; color: #000; font-size: 1.03em; margin: 1em 3em; padding: 1em;">

The results of this study indicate that it is time to move away from the problem-based paradigm for web accessibility, where our primary goal is to eliminate problems encountered by users...Taking a lesson from usability research, web accessibility research must define a much broader set of design principles, based on user data, that focuses on the use of the web by people with disabilities - not just on the problems they encounter. Once those design principles are clearly understood, only then can we look at proposing rules and heuristics that web developers can apply to evaluate their success in creating websites that people with disabilities can use well. This new paradigm will help us to discover the second half of the accessibility story.

</blockquote>

== Conclusion ==

Short term the Web Content Accessibility Guidelines Working Group (WCAG WG) may wish to consider the [https://www.w3.org/WAI/GL/wiki/Review:_Guidelines_are_only_half_of_the_story#One_Issue_May_Warrant_Action_Short_Term  one issue that may have a WCAG implication].

Longer term the WCAG WG may want to contemplate the broader questions such as is a tighter integration of usability and accessibility needed? Are broader design principles in order? And should WAI consider furthering efforts in the area of "Usable Accessibility" and UCD? What is in or out of scope?

== Definitions ==

; Effectiveness
: The accuracy and completeness with which users achieve specified goals. - [ISO 9241]
; Efficiency
: The resources expended in relation to the accuracy and completeness with which users achieve goals. - [ISO 9241]
; Satisfaction
: Freedom from discomfort, and positive attitudes towards the use of the product. - [ISO 9241]

== References ==

* Fischer, Detlev. <cite>[http://www.bitvtest.eu/articles/article/lesen/chi2012-wcag.html Methodological flaws put question mark on study of the impact of WCAG on user problems]</cite>, 2012.
* Gunnarsson, Birkir R. et. al. <cite>[http://webaim.org/discussion/mail_thread?thread=5314 Thread: Guidelines are only half of the story: accessibility problems encountered by blind users on the web]</cite>, <cite>WebAIM mailing list discussion</cite>, 2012.
* Henry, Shawn <cite>[http://www.uiaccess.com/accessucd/ut.html Accessibility in User-Centered Design: Usability Testing]</cite>, 2007.
* Kelly, Brian. <cite>[http://opus.bath.ac.uk/425/ Developing A Holistic Approach For E-Learning Accessibility]</cite>, 2004.
* United States Department of Education, Office for Civil Rights. <cite>[https://www2.ed.gov/about/offices/list/ocr/docs/investigations/11116002-b.pdf Resolution Agreement, South Carolina Technical College System, OCR Compliance Review No. 11-11-6002 (PDF]), 2009.</cite>
* United States Department of Education, Office for Civil Rights. <cite>[http://www2.ed.gov/documents/press-releases/university-cincinnati-agreement.pdf University of Cincinnati Resolution Agreement OCR Compliance Review #15-13-6001 (PDF)]</cite><cite>, 2014.</cite>
* United States Department of Education, Office for Civil Rights. <cite>[http://www2.ed.gov/documents/press-releases/youngstown-state-university-agreement.pdf Youngstown State University Resolution Agreement OCR Compliance Review #15-13-6002 (PDF)]</cite><cite>, 2014.</cite>
* Van Achterberg, Mallory. <cite>[http://webaim.org/discussion/mail_thread?thread=5314 Research shows adhering to WCAG doesn't solve blind users' problem,]</cite> SitePoint forum, 2012.
* W3C Web Accessibility Initiative. <cite>[http://www.w3.org/WAI/intro/usable#usable Usable Accessibility - Including Usability in Accessibility Research and Practice]</cite>, 2010.
* Web Content Accessibility Guidelines 1, W3C World Wide Web Consortium Recommendation. <cite>[http://www.w3.org/TR/WCAG10/#tech-meaningful-links Checkpoint 13.1 Clearly identify the target of each link. [Priority 2] ]</cite><cite>, 1999.</cite>
* Web Content Accessibility Guidelines 2.0, W3C World Wide Web Consortium Recommendation. <cite>[http://www.w3.org/WAI/WCAG20/quickref/#navigation-mechanisms-refs-sufficient-head Sufficient Techniques for 2.4.4 - Link Purpose (In Context)]</cite>, 2014.