This is a position paper for W3C Usability workshop in Washington DC, November 2002
Marja-Riitta Koivunen and Matt May, W3C
Usability can be defined in many ways. We see usability broadly according to the ISO 9241 definition: “the effectiveness, efficiency and satisfaction with which specified users achieve specified goals in particular environments”. Where effectiveness, efficiency and satisfaction are defined as:
While usability is bound to the examined user group and their goals the usability defects are often quite general. For instance, many sites lacking usability have problems with their
Definitions of accessibility commonly overlap with usability. In our view accessibility is seen as usability for users with various disabilities who may use alternative input modes or specialized assistive technologies, such as screen readers or braille devices. Typically, an accessible website:
Accessibility prioritizes usability defects in evaluating how much they prevent the use of a site for groups of disabled users and more standard access to data, irrespective of their chosen modes of interaction. Sites that are considered hard-to-use or cumbersome usually have not only generic defects in usability, but also accessibility problems that may totally prevent their use for users with cognitive, visual or other disabilities.
W3C develops technical recommendations that often seem to be quite far away from the end user reality, as well as guidelines that aim to enhance the user experience, such as the Web Content Accessibility Guidelines. Technical recommendations have an important effect to the user experience as they define the constraints within which the designers using these standards need to work.
The challenge here is to bring the user context to the people working on technical recommendations so that it is easy to see the consequences of technical decisions on the usability of products based on these recommendations.
It is easier to see the user context when developing guidelines for people designing Web pages or developing user agents or authoring tools. However, while challenging, it is important to analyze the general needs across different pages, tools, and agents so that they can be used to design technical recommendations that do not restrict the designers unnecessarily.
Here we have gathered a couple of scenarios of how usability methods could be used to help increase the usability of the W3C process. Some of these methods are already in use within some working groups.
There is a huge number of people who use W3C recommendations in various way or need to know what is happening in W3C. W3C develops recommendations in and documents the progress of over 30 working groups. Most of these users use the W3C site to find the information they need, so the usability of this site is greatly appreciated. Some work has been done on the home page but many pages under it are left to the creators and they could benefit from some design methodologies [Sano96, Mayhew99].
Evaluating the usability of this site can be done as any other site, by using usability heuristics, walkthroughs, usability testing with users with exercises such as thinking aloud, questionnaires, and interviews [Nielsen93, Preece93, Rubin94]. What should be taken into account here when changing the design is that there are many people and groups who are creating content for different purposes and often also want to demonstrate their own technologies.
Better consistency, especially in navigation, is high on our wishlist. An easy design template that can be nodified for different purposes could be of great help for new working groups. Free-form feedback from the users is collected in http://lists.w3.org/Archives/Public/site-comments/ list. We could also develop questionnaires and a toolbox that would allow tools to semiautomatically process the collected information. Site traffic metrics, such as the most popular pages accessed, could be used to design shortcuts to high-traffic pages or for checking for possible problems when pages do not get enough traffic. It might be interesting also to collect click trails within the site to see what kinds of routes the typical users take.
It is often hard to see the connection between technical decisions and their effect on end-user reality later. Scenario based design is commonly used to ease that [Carrol95]. Creating a set of user scenarios at the beginning of the process will help the designers of W3C technologies to remember the problems that the users will be dealing with. Some of these scenarios should take into account disabled users and international users so that those aspects are not forgotten.
The process should have some points for checking the scenarios and if changes have been made to them because of technical reasons. Recording these decisions as design rationale would be important too: much of our work is based on document trails such as this.
Finally, some scenarios can also be used as part of the recommendations to explain them to larger audiences.
The W3C recommendations are often very technical and require some background knowledge to be fully understandable. However, from an adaptability and deployment point of view it is extremely important to provide some information about the context and use in as understandable way as possible. Often user scenarios are used to provide this information.
W3C could recruit a pool of less technical users willing to read the recommendations and annotate the text or concepts they have problems understanding and then explain them in a glossary or the main text. This is would be most important later in the process, but often good and clear wording also helps the group to reach good decisions. Recommendations themselves are also Web sites describing different parts of the document. This design can be evaluated as well.
Finally, tools and educational materials can be developed to help users to use the recommendations. For instance, W3C has a validator for XHTML and CSS documents. To check a document, a user just writes a URI on a validator Web page and hits submit. The resulting page gives the defects and pointers to the suspect code. However, the user still needs to know how to correct the page. The next step would be a service that suggests a corrected page.
WAI has various education and outreach materials and tools that explain the accessibility guidelines and try to support accessibility evaluation. For instance, WAI has tutorials with concrete examples of how to use the WCAG guidelines [Letourneau2000]. WAI is also creating a design Gallery of sites tested for accessibility guidelines. In addition, the gallery sites could be tested against usability or the usability tests done for these sites could be explained as usability has a profound effect on accessibility.
Usability heuristics help the evaluators to remember to check the main causes of usability defects. For instance, a list of general user interface heuristics can look like the following [Nielsen93]:
There are several similar heuristics in short format to help memorize them [Nielsen, Nielsen96, Nielsen2002] or longer checklists [Tognazzini2001, Pierotti]. Usability professionals and domain professionals should develop W3C design heuristics from existing heuristic checklists for usability, accessibility, and internationalization and check the recommendations against them. The designs should be checked also against the previously suggested user scenarios. The Quality Assurance Activity also needs to be involved in this.
Designing and testing existing tools that can support a working group. For instance, tools supporting Web based collaboration are important and can include editing tools, annotations, problem tracking tools, telecommunication tools, tools for voting, storing design rationale, solving a problem in a group etc. Usability experts can attend the meetings, record the common tasks, and help design or select tools that enhance the process.
In addition to usability, there are four other current activities which have broad applicability across Web standards: the WAI Technical Activity; the Internationalization Activity; the Device Independence Activity, and the Quality Assurance Activity. In addition, work in the Multi-Modal Activity is is related to several other activities.
The goals of these activities support each other and can often overlap. For example, one of the central goals of usability precisely matches the accessibility needs of someone with cognitive disabilities: reduce cognitive friction. The relationship between the possible Usability interest group and WAI should be close and the guidelines should be integrated together to help the evaluators. It might even be possible to develop generic usability guidelines under the more accessibility-focused WAI work.
A cross-disciplinary group consisting of accessibility, internationalization, device independence and usability could streamline the process used by W3C working groups. It can gather different guidelines together for an integrated set of heuristics and automatic evaluation tools [Brajnik]. It could also help solicit feedback on documents, help evaluate W3C site, support design of tools and education material, and cultivate a unified vision of the necessary elements of future Web content, user agents, authoring tools and Web languages and protocols.
There is a tremendous amount of knowledge being cultivated in the field of usability. Much of the W3C standards work depends on a common frame of reference among the participants, but lacks a common usability approach due to the rapid development of both the Web and usability practices. The earlier we can utilize the usability knowledge in the design process, the better the end results. Increasing the usability of the W3C recommendations before the actual products are developed by including user scenarios to the process is a good example of that. It is worth looking at what other usability-related methods can be adopted from the design lifecycle [Mayhew99]. Also in a more basic level W3C and Technical Architecture Group (TAG) should be looking if there are some architectural level solutions that would have a positive effect on usability.
One of the most valuable elements of a usability interest group is getting together the people interested in usability to easily exchange knowledge and experiences. A central resource nominally responsible for creating best usability practices would not only be beneficial for W3C working groups, but those techniques could potentially have a greater effect on the Web at large on bringing user experiences toward a more common ground, and lowering the overall cognitive barriers involved in using the Web. These best practices can be integrated to the W3C process to be part of the W3C material, such as, the process document, W3C site design guidelines, and publishing rules.
It is possible to enhance usability within W3C on many levels and using various methods. However, we need to prioritize the most important places to start with as there is not sufficient resources to do everything at once. It is also important to think how the usability related work could be shared with the companies participating in the W3C working groups. Finally, some usability evaluations can be done while developing the final products among the member companies in lieu of the recommendations themselves.
[Brajnik] Brajnik, G. Quality Models based on Automatic Webtesting. http://www.dimi.uniud.it/~giorgio/papers/quality-models.html
[Carrol95] Carrol J. (Ed.) (1995) Scenario-Based Design: Envisioning Work and Technology in System Development. Wiley, USA.
[Letourneau2000] Letourneau, C. and Freed, G. (2000) Curriculum for Web Content Accessibility Guidelines 1.0. http://www.w3.org/WAI/wcag-curric/
[Nielsen93] Nielsen J. (1993) Usability Engineering, Academic press, USA.
[Nielsen2000] Nielsen J. (2000) Designing Web Usability, New Riders.
[Nielsen2002] Jakob Nielsen's Alertbox, May 12, 2002: Top Ten Guidelines for Homepage Usability, http://www.useit.com/alertbox/20020512.html
[Nielsen96] Jakob Nielsen's Alertbox for May 1996: Top Ten Mistakes in Web Design, http://www.useit.com/alertbox/9605.html
[Nielsen] Nielsen, J. Ten Usability Heuristics. http://www.useit.com/papers/heuristic/heuristic_list.html
[Mayhew99] Mayhew, D. (1999) The Usability Engineering Lifecycle - A Practitioner's Handbook for User Interface Design. Morgan Kaufmann Publishers.
[Pierotti] Pierotti, D. Usability Techniques Heuristic Evaluation - A System Checklist. http://www.stcsig.org/usability/topics/articles/he-checklist.html
[Preece93] Preece, J. (1993) A guide to usability. Addison-Wesley.
[Rubin94] Rubin, J. (1994) Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests. Wiley, USA.
[Sano96] Sano, D. (1996) Designing Large-Scale Web Sites: A Visual Design Methodology. Wiley, USA.
[Tognazzini2001] Tognazzini, B. (2001) AskTOG: First Principles. http://www.asktog.com/basics/firstPrinciples.html
[UsableNet] UsableNet Homepage http://www.usablenet.com/