Research Report on Mobile Web Accessibility

Status of this document

This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/.

This [dd month yyyy] [First Public] Working Draft of Research Report on Web Accessibility Metrics is intended to be published and maintained as a W3C Working Group Note after review and refinement. The report provides an initial consolidated view of the outcomes of the Mobile Web Accessibility Online Symposium held on 25 June 2012.

The Research and Development Working Group (RDWG) invites discussion and feedback on this draft document by research and practitioners interested in metrics for Web accessibility, in particular by participants of the online symposium. Specifically, RDWG is looking for feedback on:

• Summaries of the extended abstracts contributed to the online symposium;
• Discussion about the state-of-the-art and conclusions drawn in the document;
• Related resources that may be useful to the discussion within the document.

Please send comments on this Research Report on Mobile Web Accessibility document by [dd month yyyy] to public-wai-rd-comments@w3.org (publicly visible mailing list archive).

Publication as a Working Draft does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.

This document has been produced by the Research and Development Working Group (RDWG, as part of the Web Accessibility Initiative (WAI) International Program Office.

This document was produced by a group operating under the 5 February 2004 W3C Patent Policy. W3C maintains a public list of any patent disclosures made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) must disclose the information in accordance with section 6 of the W3C Patent Policy.

Table of Contents

1. Introduction
2. Related Work
   • 2.1 Mobile Technology and Accessibility
   • 2.2 Mobile Accessibility Guidelines
   • 2.3 Mobile Accessibility Evaluation
   • 2.4 Mobile Interaction Models
3. Symposium Papers
4. Emerging Themes
   • 4.1 Convergence
   • 4.2 Mobile Accessibility is Practical
   • 4.3 Harmonized Guidelines and Standards
5. Future Directions
6. Conclusions
7. References
8. Proceedings
9. Acknowledgements
10. Appendix - Paper Question and Answers

1. Introduction

Mobile devices are changing the way people access the Web and it is no longer safe to say that a specific device or set of devices will be used to access content. We scope mobile devices as those that are conventionally carried by the user and act as that user's personal computation and connection resource. Indeed, we subscribe to the Wikipedia definition which says "a mobile device (also known as a handheld device, handheld computer or simply handheld) is a small, hand-held computing device, typically having a display screen with touch input and/or a miniature keyboard and weighing less than 0.91 kgs". In this case, screen size and computational resources will vary including support for gestures, voice commands, and a variety of gadgets. Beyond the devices themselves, the software that allows people with disabilities to access web content is as varied as the devices themselves which include operating systems, browsers, Application Programming Interface (API)s, etc. are usually specific to a mobile device with each resulting in a unique user experience. Vendor implementation approaches to accessibility also vary with, for example, Android and iOS having different accessibility APIs. Standardization seems the only way to create a shared user experience and take much of the burden of learning new accessibility software and techniques from the user. Indeed, we note that directions for standards have already been published in this space, including the "Standards for Web Applications on Mobile: current state and roadmap" as well as those developed by the Web Accessibility Initiative (WAI) and oriented to the practitioner.

Initial work has been done by the World Wide Web Consortium (W3C) on creating some mappings between Web Content Accessibility Guidelines (WCAG) (Henry 2012) and Mobile Web Best Practice (MWBP) (Rabin and McCathieNevile 2005) but a gap exists for a more complete set of best practices for accessible mobile web content. Indeed as alluded to in the abstract, the mobile world is rapidly evolving with increasingly sophisticated devices, platforms, APIs and applications. While the mobile Web provides vast opportunities for everybody, especially to people with disabilities, the mobile Web also entails many accessibility challenges. There are many areas where further research needs to be conducted to understand what is the scale of accessibility issues on the mobile Web for people with disabilities; for instance, to address accessibility issues, mobile devices provide alternative interaction methods, using combinations such as voice and gesture interactions, but little is known about how these new interaction methods support accessibility. Furthermore, it is not clear how well existing standards such as WCAG (Henry 2012), User Agent Accessibility Guidelines (UAAG) (Jacobs et al. 2002), Accessible Rich Internet Applications (WAI-ARIA), MWBP (Rabin and McCathieNevile 2005) and others address accessibility in the mobile context. Despite some initial work on mapping between WCAG and MWBP (Yesilada et al. 2008, Chuter and Yesilada 2008) there is relatively little work bringing together cross-platform standards. Conversely there is a growing trend in the accessible and mobile communities for more flexible and generic device content solutions; to some extent addressed by the new WAI Indie UI Working Group. While a mobile Web practitioner might call this flexibility, responsive design or progressive enhancement with a focus on layout and bandwidth, an accessible practitioner might call this ubiquitous design or simply a complete design with a focus on content access. Both views aim at a generic client solution within a browser that can be reused across standards-supporting devices.

In real world use, mobile devices have popularized a number of interaction models such as gestures and voice commands. With these new interaction models come new accessibility challenges and opportunities. Gestures for example open a previously flat and "lifeless" touch screen to a blind user by allowing quick swipe scanning for input and output through auditory summaries and queues via a screen reader. Indeed, recent studies show the growing popularity of screen reader use on mobile devices (WebAIM 2012, WirelessRERC 2013), especially on iOS. The iOS default screen reader is VoiceOver, which is well known among the accessible community, and offers a number of accessibility features (Apple 2012). Though with multiple platforms and devices the mobile landscape is fragmenting, Android for example has different model and APIs for assistive technologies (Android 2012) than say iOS. Work is currently underway within the WAI Independent User Interface Task Force in an effort to begin standardizing alternatives for gestures to support accessibility provided by these devices, initial focusing on abstraction of the interaction events with Indie UI: Events 1.0.

While these new interaction forms are gaining popularity, more traditional forms of interaction - such as keyboard input - continue to be used in mobile devices and have changed little from the original qwerty keyboard layout. However, as with a small screen, a small keyboard poses many challenges for disabled users, especially users with motor and visual disabilities. A small keyboard may increase input errors by users accidentally hitting the wrong key, holding down a key too long, keying two keys at once, and so on. Additionally a visually impaired person using an onscreen keyboard will rely on scanning methods to determine the key to press; which will often take longer than a sighted user performing the same task, therefore applications that are time sensitive may be unusable by disabled users.

We can see that this novel mix of interaction paradigms create a very specific set of problems and opportunities, and implies that further research is needed to understand the challenges associated with the mobile Web.

In order to investigate these challenges, the Research and Development Working Group (RDWG) organized a symposium on mobile Web accessibility. The primary objective of this symposium was to gather, analyze, and discuss practical experience with mobile Web accessibility. For an overview of existing related standards, please visit the W3C WAI Guidelines and Techniques and the W3C Mobile Web Initiative The Web and Mobile Devices resource pages. In particular, the objective was to investigate:

• the technical challenges in terms of API, platform, browser, applications, and web content accessibility – What is the state (i.e., problems and successes) of mobile Web accessibility?
• the coverage of mobile accessibility by existing standards such as UAAG, WCAG, MWBP, and MWABP – How do W3C guidelines and best practices (UAAG, WCAG, MWBP, and MWABP) apply to Websites and Web applications accessed via mobile devices?
• new interaction models such as touch interfaces and their accessibility – How do existing interaction models such as touch interfaces and gesture interaction support accessibility? How can new interaction paradigms be leveraged for mobile Web accessibility?

This research report aims to present the findings of this symposium which constitutes the basis from which to further explore a research and development roadmap for mobile accessibility. This report presents an overview of the state of the art regarding mobile Web accessibility, introduces the papers that were presented at the symposium and provides input to future research on mobile Web accessibility.

2. Related Work

Mobile Web use can be challenging for everybody (Brewster 2002, Yang and Wang 2003, Chae and Kim 2004, Roto and Oulasvirta 2005, Oulasvirta et al. 2005, Wobbrock 2006, Cui and Roto 2008). Compared to accessing the Web on the desktop, mobile devices can have many limitations and contextual constraints. Mobile devices have much smaller screens which means the visual field is restricted (Reisel and Shneiderman 1987, Dillon et al. 1990, Chae and Kim 2004). They might be used in ‘off-desk’ environment (Carter et al. 2006), in poor lighting conditions - such as sunlight reflection on screen - which may affect a mobile user’s color perception and contrast sensitivity thereby increasing reading difficulty. They might also be used in noisy environments, which means one might have problems in hearing voice prompts or confirmatory/warning sounds. They might also have limited input capabilities which can cause input difficulties (James and Reischel 2001, MacKenzie and Soukoreff 2002, Brewster 2002). Mobile users who usually participate in multiple activities (e.g., talking, walking, way finding, sidestepping, etc. (Oulasvirta et al. 2005, Lin et al. 2007) while accessing the Web also have to distribute cognitive resources between multiple tasks. These are specific examples of situational disabilities that anybody can experience when accessing the mobile Web (Sears and Young 2003).

Disabled Web users experience many difficulties due to inappropriately designed content (WAI, 2005), and these barriers can persist on the desktop, mobile, or any other browsing platform. However, considering the context of use discussed above, the difficulty for people with disabilities might be much worse on the mobile Web. Even though there are user studies that investigate the difficulties experienced by mobile users without disabilities (Nielsen Norman Group 2011), there is little research that investigates the difficulties experienced by disabled mobile Web users (Trewin 2006). With this literature review, we first look at the state of the art in mobile Web technology including accessibility APIs, assistive technologies on mobile devices, etc. We then look at how accessibility guidelines address the mobile web and finally we review the new interaction paradigms on mobile devices.

2.1 Mobile Technology and Accessibility

One can simply define the mobile Web as being able to access the content on mobile devices. With existing mobile technology, this can be achieved by using Web applications, native applications or hybrid applications. Web applications are browser based applications built with technologies such as Hypertext Markup Language (HTML), Cascading Style Sheets (CSS) and Javascript (JS). Native applications are downloaded applications that take the advantage of phone capabilities and hybrid applications are also built with technologies such as HTML, CSS and JS, but they are downloadable and can also access phone capabilities. Considering these, when one talks about accessibility of the mobile Web, all of these application types have to be considered compared to Web accessibility on the desktop. Therefore, having all this variety in applications brings extra complexity. Mobile Web accessibility is not only about browsers and interaction through browsers, but needs to consider mobile devices and their support.

Accessibility APIs are very important components for achieving accessibility. They are used to communicate accessibility information about an application to assistive technologies (Faulkner et al. 2012). For example, screen readers access the content on a given application through these APIs. Compared to the desktop, accessibility APIs on mobile devices are not as developed. For instance, iOS (Apple 2012) and Android (Android 2012) provide APIs, but there are many other mobile platforms that do not provide accessibility APIs. However, most mobile platforms provide accessibility tools that in general supports alternative input and output. For example, Android has TalkBack which is a mobile screen reader (Android 2012), iOS has Voiceover and other accessibility tools (Apple 2012), Blackberry has a screen reader (Blackberry 2012), and Nokia has Mobile Speak (Nokia 2012). These mobile platforms also have accessibility settings, for example zooming, font resizing, custom gestures, color settings, haptics, etc. However, all these platforms show differences in their accessibility design and for developers it can be challenging to design a Web page or application that can be accessible in all platforms. Regarding the Web technologies, these platforms also support different technologies such as HTML, CSS, WAI-ARIA, Flash, etc. On the desktop, the platforms are not so divergent, however mobile platforms bring extra challenge to developers because of differences in supporting different technologies and platforms, and in some cases because of the limitations in the kind of applications allowed on a platform.

2.2 Mobile Accessibility Guidelines

There is no unified, universal set of mobile accessibility guidelines that include techniques and methods for developing accessible mobile Web pages and applications. The most related guidelines to mobile accessibility are jointly W3C’s MWBP (Rabin and McCathieNevile 2005), the Web Content Accessibility Guidelines (WCAG) (Henry 2012) and the overarching WAI Mobile Accessibility Guidelines. MWBP mainly provides techniques that can be used to design user friendly Web pages on mobile devices and WCAG includes guidelines for developing user friendly mobile pages (Harper and Yesilada 2008). There is also work that investigates the relationship between WCAG and MWBP, which shows that there are significant overlaps (Yesilada et al. 2008, Chuter and Yesilada 2008) between the two. Platform specific guidelines have also been developed, for example, Android (Android 2012), Blackberry (Blackberry 2012), iOS (Apple 2012), Nokia (Nokia 2012) and Windows Mobile (Microsoft 2012) have specific design guidelines for their platform. Other organizations have also published mobile accessibility guidelines (AB 2012). However, a gap exists for a more complete, unified, universal set of best practices for accessible mobile web content. A growing trend in the accessible and mobile communities is for more flexible and device generic content solutions. A mobile Web practitioner might call this responsive design or progressive enhancement (Marcotte 2011) with a focus on layout and bandwidth while an accessible practitioner might call this a ubiquitous design (Ubiquitous Web 2012). Both views aim at a generic client solution that can be reused across standards supporting devices. This raises several questions about the coverage of mobile accessibility by existing standards such as WCAG (Henry 2012), UAAG (Jacobs et al. 2002), WAI-ARIA (Craig and Cooper 2011), MWBP (Rabin and McCathieNevile 2005) and others.

2.3 Mobile Accessibility Evaluation

When we look at the evaluation of mobile Web accessibility, little research has been conducted. There are many inspection tools, emulators that can be used for testing but these are mainly vendor dependent tools, for example the Opera emulator (Opera 2012) or iOS (Apple 2012) based developer tools. W3C Mobile Web Initiative (MWI) also introduced machine test sets based on MWBP 1.0: W3C mobileOK Basic Tests 1.0 (Owen and Rabin 2008). Passing these tests means that the evaluated content provides a functional user experience for users of basic mobile devices whose capabilities at least match those of the Default Delivery Context (DDC). DDC can be considered as the minimum common denominator device profile. However, those tests are not based on specific requirements of disabled mobile Web users. There are a number of automated mobile evaluation tools that use the W3C mobileOK Basic Tests such as W3C mobileOK Basic checker (Marin and Rionda 2008), TAW mobileOK Basic checker (TAW MobileOK Checker 2010b) and ready.moby (ready.mobi 2010a). There are also some tools that are directly based on MWBP 1.0 including EvalAccessMOBILE (Arrue et al. 2007). MWI also developed an open-source library which can be downloaded and tested using its Web interface. There are also some automated evaluation tools that are based on this open-source library such as the MokE online evaluation tool (Garofalakis and Stefanis 2008). All the automated evaluation tools mentioned evaluate Web pages against the DDC, however there are also some tools that aim to consider different device specifications such as the tool described in (Vigo et al. 2009). However, none of these tools particularly focus on testing accessibility of mobile Web for disabled users. Mobile Web accessibility evaluation is also an area that further research needs to be conducted.

2.4 Mobile Interaction Models

Mobile devices such as the iPhone 5 and iPad, Google Nexus, BlackBerry Z10 and PlayBook, Nokia Lumia and so on, are usually smaller than 35 centimetres across the diagonal and as discussed above they are typically used in constrained environments. Traditional forms of interaction, such as keyboard or mouse interaction, do not exits on mobile devices or are almost impossible to use (Butts and Cockburn 2002); therefore, new interaction forms have started to emerge in answer to these difficulties. In some cases, techniques and technologies originally developed to improve accessibility for people with disabilities on the desktop have now migrated to mobile devices. Similarly, some techniques are developed for mobile devices and are now widely used by people with disabilities on desktop.

To address the limitations of keyboards, different input techniques have been proposed which include Soft Keyboard, joystick, voice input, tablet, touchscreen and multi-modal interface. The Soft keyboard is a program that presents keyboard icons in a graphical user interface and allows a user to input by tapping on-screen icons with fingers or stylus. Soft keyboards are very popular on mobile devices (Hinckley 2003) and they can also be used to enter text by tapping the touchscreen with a stylus (Myers et al. 2002). In addition, joysticks, tablets and touch-screens are used for text entry as alternative methods to keyboards (keypads) (Silfverberg et al. 2000, Mankoff et al. 2002, Wobbrock et al. 2004, Chau et al. 2006). Especially with iOS, MultiTouch (Apple 2012) and soft keyboards, technologies are very popular alternative methods to conventional keyboards/keypads. Voice control, referring to both speech recognition and non-speech vocalizations¹ is also used on mobile devices for speech dialing or editing text messages (Karpov et al. 2006). Multi-modal interface which combines a number of modalities such as head movement and speech for mobile users (Serrano et al. 2006) have also been suggested as possible input solutions, and are gaining popularity (Koslover et al. 2012). Auditory feedback and haptic feedback are also used to improve text entry performance for mobile users (Brewster et al. 2007). Recently, some work has also been proposed that combines audio and vibration to allow users explore graphical information on touchscreen interfaces (Giudice et al. 2012).

Regarding pointing which is the result of a lack of a mouse, a number of novel techniques and methods exist to improve the problems of target acquisition. Auditory feedback (Brewster 2002) can be used to improve pointing performance for mobile users. Screen magnification software assists visually impaired users with pointer manipulation in two ways: by improving the pointers visibility and also by tracking and locating the pointer on the fly. Many user studies on small devices show the benefits of haptic feedback (Pirhonen et al. 2002, Poupyrev et al. 2002, Brewster et al. 2007). Similarly, vibro-tactile display feedback benefits mobile device users (Williamson et al. 2007). Novel techniques have also been proposed to address target acquisition on mobile devices. For example, the barrier pointing method was introduced by (Froehlich et al. 2007) to improve pointing accuracy. This method uses the screen edges, corners, and the screen surface to support faster and more accurate touch screen target acquisition. However, the evaluation of this method shows that the overall target acquisition times were not statistically significantly different between the normal mode of interaction on most mobile device touch screens with stylus. Further, as the memory and processing capacity of current small devices are sufficient to support client-side speech recognition, command and control (C&C) technology has been applied (Paek and Chickering 2007).

3 Symposium Papers

4 Emerging Themes

From the abstracts, symposium discussion, and the authors answers to questions (see Appendix), it is clear that there are three main interconnecting themes running through both the mobile and accessible domains. These three themes are mentioned, or alluded to, across all abstracts and so we can see that there is a degree of independent agreement between experts that they are important for the future. Firstly, we see that accessibility is possible on mobile devices, not just from a situational perspective, but also from a conventional accessibility perspective. Secondly, we see the concept of device convergence being important across both domains. Finally, if this accessibility is possible and devices do start to converge, we see the need for a more inclusive and harmonized set of guidelines and standards.

4.1 Convergence of Fields

The overarching theme of the abstracts in this symposium was their discussion, both implicit and explicit, of the convergence of requirements, technologies, guidelines and best practice. We will deal with the harmonization of guidelines and best practice in the last section. However, it has become increasingly clear that the concerns of our community are beginning to run through that of the general mobile audience. As our phones, and the interfaces we use to control them, become increasingly complex; the situations of their use become increasingly extreme; and the technology becomes increasingly divergent; application developers are increasingly looking to harmonize. This desire for harmonization is leading to the decoupling of the program functionality from the lightweight interfaces which connect to it. This means that mobile convergence can be supported - and this convergence between accessibility and mobility is becoming increasingly real and attainable. Further, the requirements of use are also starting to mimic those faced in the accessibility domain, with outside environments requiring a degree of auditory interaction, with the increasing need for voice control to deal with more complex interfaces on small screen real-estates, and with multi-step interactions requiring increasing cognitive load; our frames of reference are becoming increasingly conjoined. But our converging domains benefit each other; accessible developments can be transferred and repurposed in the mobile domain, while the mobile domain forms the convergence business case for all devices, accessible ones included.

4.2 Accessibility in Use

Our participants shared the realization that there is a perception in the community of mobile practitioners and developers that practical mobile accessibility is not really possible. This is obviously not the case, with developments such as mobile VoiceOver leading the way in accessible mobile technology. Further, smartphone accessibility technology is also making general tasks such as note-taking more accessible with the development of Braille touch type technologies on multi touch screens. And these advances include novel gestural as well as more conventional ‘Perkins’ style interactions and inputs. Indeed, technologies for ‘mainstream’ interaction - in more properly to overcome some situational impairments - such as voice control (e.g. Apple’s Siri) are also having knock on benefits for disabled users and more general accessibility applications.

However, there has been a perception that keyboard interaction is not possible when soft keyboards are displayed on smartphone touch screen style devices. But, we feel that this is not the case. Certainly our participants have demonstrated that mobile keyboard access is both possible and effective. However, we also realize that while practically possible it may be necessary to a certain amount of guidance and best practice regarding keyboard accessibility in the mobile domain. In this case we feel that platform developers need to include explicit statements about keyboard interface accessibility being part of the platform’s intended look-and-feel, and for platform developers to include keyboard interface testing tools in their Quality Assurance suites. Indeed, from a purely marketing perspective it is in the platform developers best interests to maintain interoperability on their platforms. For instance Google might want to remind its developers that some Android phones continue to include physical D-pads. Apple might remind its developers that external Bluetooth keyboards are supported. Accessibility, is one way of maintaining good interoperability - because if it is accessible then it is also more likely to be interoperable and device independent.

4.3 Harmonized Standards

Our participants think that integrating findings from accessibility research to the mobile platform is timely, and there is an opportunity to transfer lessons learnt and experiences gained to the mobile domain which should not be missed. Indeed, there is great potential for reciprocity and interoperability between the mobile and accessible domains, and many of the lessons learnt from accessibility will be relevant to the mobile platform. Accessibility practitioners have been researching, innovating, and building device independent resources, in all but name, for many years; their expertise can, and should, be leveraged in efforts to develop the mobile platform. Conversely, the mobile platform is important from an accessibility standpoint, because it addresses many of the same issues, and is likely to be the focus of a significant development effort. Our aim should be to address these issues by identifying this reciprocity and the ways in which the mobility and accessibility can interoperate, developing a harmonized set of guidelines and standards to support it.

Work in the fields of accessibility and device independence (the interaction domain) is extensive. By gathering and summarizing relevant research within these fields, we can provide the basis for future integration. By finding the intersection of current work from these different, but related, fields we can start to investigate just how both mobility and accessibility can interoperate. Studying these commonalities will enable us to delineate the extent of the intersection, identify techniques which may be useful in one field but are missing in another field. Because the mobile platform is younger than accessibility, guidelines and best practice may at first flow into the mobile domain, thereby speeding its development. However, as the mobile platform is developed, and extended set will flow back into device and platform accessibility due to the greater research and development effort focused on the mobile domain.

6 Conclusions

Mobile devices provide many opportunities for everybody to access the Web, especially people with disabilities. Likewise, there are many accessibility barriers to mobile access. The RDWG organized a symposium to bring researchers and practitioners together to discuss the accessibility challenges associated with the mobile Web. The primary objective was to gather, analyze, and discuss practical experience (accessibility in use) and in particular to investigate:

• the technical challenges in terms of API, platform, browser, applications, and web content accessibility;
• the coverage of mobile accessibility by existing standards such as UAAG, WCAG, MWBP, and MWABP; and
• new interaction models such as touch interfaces and their accessibility.

This report first discussed the state of the art in mobile Web accessibility, and introduced the papers presented at the symposium. These five papers cover a broad range of topics including input issues, security limitations, alternative views of the Web - the Web as a platform and Web standards. Based on these papers and the discussions conducted in the symposium, three emerging themes came to light:

• Convergence of fields: it is becoming obvious that there are significant overlaps between the requirements of mobile and disabled Web users. Innovation for one can be a benefit the other. Understanding shared experiences can advance both fields.
• Accessibility in Use: By the nature of mobile devices and the variety of technologies available on mobile devices, mobile Web accessibility becomes very practical and requires a certain level of engineering skills. Innovation in this field needs to take this into account.
• Harmonized Standards: It is becoming obvious that all parties would benefit significantly more from a inclusive and harmonized set of guidelines and standards.

Based on these emerging themes, this report suggests that the research agenda needs to consider the future directions and adopt new approaches. In summary, the future directions need to consider the unified Web, convergence of technologies and migration of technologies between platforms. Finally, the research agenda needs to look ahead of future developments, don't lag behind and be agile.

7 References

8 Symposium Proceedings

Research Report on Mobile Web Accessibility

This document should be cited as follows:

S. Harper, P. Thiessen, Y. Yesilada, Research Report on Mobile Web Accessibility. 
W3C WAI Research and Development Working Group (RDWG) Notes. (2012)
Available at: http://www.w3.org/TR/mobile-accessibility-report

The latest version of this document is available at:

http://www.w3.org/TR/mobile-accessibility-report/

A permanent link to this version of the document is:

http://www.w3.org/TR/2012/WD-mobile-accessibility-report-{{date}}/

A BibTex file is provided containing:

@incollection {mobile-accessibility-report_FPWD,
  author = {W3C WAI Research and Development Working Group (RDWG)},
  title = {Research Report on Mobile Web Accessibility},
  booktitle = {W3C WAI Symposium on Mobile Web Accessibility},
  publisher = {W3C Web Accessibility Initiative (WAI)},
  year = {2012}, month = {December},
  editor = {Simon Harper and Peter Thiessen and Yeliz Yesilada},
  series = {W3C WAI Research and Development Working Group (RDWG) Notes},
  type = {Research Report},
  edition = {First Public Working Draft},
  url = {http://www.w3.org/TR/mobile-accessibility-report},
}

Contributed Extended Abstract Papers

The links provided in this section, including those in the BibTex files, are permanent; see also the W3C URI Persistence Policy.

@proceedings{mobile-accessibility-proceedings,
     title = {W3C WAI Symposium on Mobile Web Accessibility},
     year = {2012},
     editor = {W3C WAI Research and Development Working Group (RDWG)},
     series = {W3C WAI Research and Development Working Group (RDWG) Symposia},
     publisher = {W3C Web Accessibility Initiative (WAI)},
     url = {http://www.w3.org/WAI/RD/2012/mobile/},
}

• Elizabeth Woodward, Brian Cragun. Accessible Security For Mobile. BibTex file.

  @inproceedings{woodward2012,
       author = {Elizabeth Woodward and Brian Cragun},
       title = {Accessible Security For Mobile},
       booktitle = {W3C WAI RDWG Symposium on Mobile Web Accessibility},
       year = {2012},
       editor = {Simon Harper, Peter Thiessen, Yeliz Yesilada},
       pages = {article 1},
       url = {http://www.w3.org/WAI/RD/2012/mobile/paper1/},
  }

• Mike Wald, E.A. Draffan, Yunjia Li. Enabling Flexible User Interactions With Mobile Media. BibTex file.

  @inproceedings{wald2012,
       author = {Mike Wald and E.A. Draffan and Yunjia Li},
       title = {Enabling Flexible User Interactions With Mobile Media},
       booktitle = {W3C WAI RDWG Symposium on Mobile Web Accessibility},
       year = {2012},
       editor = {Simon Harper, Peter Thiessen, Yeliz Yesilada},
       pages = {article 2},
       url = {http://www.w3.org/WAI/RD/2012/mobile/paper2/},
  }

• Cristina Gonzalez, Javier Rodriguez, Caroline Jay. Accessibility In Multi-device Web Applications.

  @inproceedings{gonzalez2012,
       author = {Cristina Gonzalez and Javier Rodriguez and Caroline Jay},
       title = {Accessibility In Multi-device Web Applications},
       booktitle = {W3C WAI RDWG Symposium on Mobile Web Accessibility},
       year = {2012},
       editor = {Simon Harper, Peter Thiessen, Yeliz Yesilada},
       pages = {article 3},
       url = {http://www.w3.org/WAI/RD/2012/mobile/paper3/},
  }

• Jan Richards, Alison Benjamin, Jorge Silva, Greg Gay. Assessment Of Keyboard Interface Accessibility In Mobile Browsers. BibTex file

  @inproceedings{richards2012,
       author = {Jan Richards and Alison Benjamin and Jorge Silva and Greg Gay},
       title = {Assessment Of Keyboard Interface Accessibility In Mobile Browsers},
       booktitle = {W3C WAI RDWG Symposium on Mobile Accessibility},
       year = {2012},
       editor = {Simon Harper, Peter Thiessen, Yeliz Yesilada},
       pages = {article 4},
       url = {http://www.w3.org/WAI/RD/2012/mobile/paper4/},
  }

• Katja Forbes. Inclusive Mobile Experience: Beyond The Guidelines And Standards. BibTex file.

  @inproceedings{forbes2012,
       author = {Katja Forbes},
       title = {Inclusive Mobile Experience: Beyond The Guidelines And Standards},
       booktitle = {W3C WAI RDWG Symposium on Mobile Accessibility},
       year = {2012},
       editor = {Simon Harper, Peter Thiessen, Yeliz Yesilada},
       pages = {article 5},
       url = {http://www.w3.org/WAI/RD/2012/mobile/paper5/},
  }

9 Acknowledgements

Participants of the W3C WAI Research and Development Working Group (RDWG) involved in the development of this document include: [Alphabetical List of Contributors]

RDWG would also like to thank the chairs and scientific committee members as well as the paper authors of the RDWG online symposium on Mobile Web Accessibility.

10 Appendix

10.1 Question and Answers

Written answers to the main questions posed in the Symposium - therefore all responses are 'sic erat scriptum'.

PAPER 1: Accessible Security For Mobile

1. What particular security concern(s) are raised by the combination of the mobile Web and users with disabilities for social engineering vulnerabilities? What is the connection between accessibility in mobile devices and the security dangers and problems specifically faced by people with disabilities?

One of the big problems that we've encountered is that devices are made secure, but at the expense of someone who is disabled or situationally disabled being able to actually use their device. As an example, we evaluated a containerization solution--a solution that partitions a smartphone into a work partition and a personal partition so that users and the enterprise can separate work applications and data from personal apps and data. The first challenge we saw was that the container solution couldn't be installed by someone who was blind. If the solution would have been deployed at that stage, a user could not use the Eyes Free shell that they were used to in order to access the content in their work partition because they were given a choice between the Eyes Free Shell and the work partition. And, the navigation of the container system and applications provided in the work partition were not accessible. And when the assistive technology was running, the authentication to the container crashed, so that you could no longer authenticate to get into the container.   And, beyond that, assistive technologies had to be recompiled and installed as a separate namespace in each partition. So, the problem is more likely that the device can be made secure but is unusable by someone who has a disability. Consider biometrics. If I can't use fingerprints and the authentication solution is fingerprint authentication, the mobile solution is completely impractical for me and renders the device useless unless there's an alternative.

From the other perspective, we've seen assistive technologies that created security issues. Consider Siri...Siri is being blocked within IBM due to security concerns. First, IBM has issues with content being sent to the Apple cloud, but also, by default Siri, can (or could) be used even when the smartphone was locked with a passcode. I'm not sure if that’s still true. But it's an example of the impact.

2. The “..key areas that need to be addressed in order to create an accessible mobile security..” list mentioned, are these concerns specific to disabled users?

More and more, particularly with mobile, we see a blurring of the lines between serving those who have disabilities and those who don't. With mobile, everyone is experiencing situational disabilities. People who are driving their cars shouldn't be using their hands when they're trying to text and shouldn't be trying to look at their tiny screens and scroll around while driving. They can't use their hands, they can't use their eyes. They're experiencing situational impairments. Someone who is in a noisy airport can't hear their call very well. That's a situational hearing impairment. If the lighting isn't right, they may have challenges with seeing what's on their screen. If they're traveling, they may only have one hand free to use the device. The list goes on and on. Many of the techniques and concerns that we have in our focus on persons with disabilities are now impacting most people using mobile devices. We have already seen a case where password restriction was too difficult for persons who were blind or mobile. People stopped using the application which was in a limited pilot. So, I would say that the concerns we have identified are specific to disabled users and those with situational disabilities.

3. What steps towards including the needs of disabled users in cross-mobile-platform security standardization best practices would you recommend?

We tell everyone that the biggest step is to engage disabled users or engage people with the expertise to speak on behalf of disabled users. We came into one pilot that had performed a user needs study and had not included any people with disabilities. This is a problem for persons with disabilities, people who will experience situational disabilities and the IT staff, chief security officer and so on who won't get the input they need to make informed decisions about the deployment.

Secondly, build accessibility into the enterprise mobile security solution. Often we see accessibility treated as something to be done to a project after it is complete. We know that making changes earlier is less costly than trying to fix systems after they've been delivered.

We have several other recommended steps as mentioned in the paper which we can go over in more detail, but in the interest of time, we can summarize here.

1. Steps include:
2. Updating any local instructions, guidance and education related to accessible and secure mobile computing.
3. Determining which combinations (OS, device, carrier, assistive technologies) will be supported,
4. Making decisions about how you're going to support accessibility with ATs, which may require third party assistive technologies, evaluating those technologies
5. Building accessibility into any new security technologies (biometrics, containers, hypervisor solutions and so on)
6. and Determining how compliance monitoring, enablement and enforcement will be done.

If anyone is interested in more discussion on any of the steps listed, we can go into those in greater detail.

4. Would you like to see more of a security focus/emphasis in guidelines like WCAG? or Would following WAI (WCAG, ARIA) be sufficient as the guidelines/techniques to be applied?

At this time, we are not calling for any new, special mobile security focus in WCAG guidelines.  The guidelines for accessibility are developing and IBM has made input to the process.  Generally we see the WCAG and WAI-ARIA standards are the right direction.  The new Independent UI standard effort just started will provide another important piece to accessibility on mobile devices.

5. Do you feel there's a particular security concern for disabled users in respect to GEO location tracking?

Not really, but there is definitely a concern in general. Geolocation apps that identify a user's physical location are widely used in many industries--banking, retail, travel. They are even used by some law enforcement and to prevent fraud. They can be a good thing, but they also introduce new risks. If someone knows you, your gender, race, where you work, and so on and can combine that with geolocation, they can figure out where you are which can leave you open to burglary, stalking or worse. I think it's an issue for everyone these days and more so if you can't see who is stalking you or can't hear someone getting close or have a mobility impairment that prevents you from reacting the way you would like.  

6. What do you mean by "weakest links"? Which are they? [from Cristina Gonzalez Cachon]

This phrase "weakest links" comes from a quote by Kevin Mitnick who was the world’s most wanted computer hacker back in the 1990s. The quote reads “Companies spend millions of dollars on firewalls, encryption and secure access devices and it’s money wasted because none of these measures address the weakest link in the security chain.” That weakest link is us. It speaks to the human side of security. We can build the technology, but if we haven’t looked at how the user interacts with the system and accounted for and addressed their interactions with the technology, we still haven’t addressed the security challenges. It further implies that if the security doesn't work well, people find a way to circumvent it, which lessens the security.  The "yellow sticky" solution for secure passwords can make then less secure.

7. Which is the purpose of "remote monitoring" as commented in Open Research Avenue? [from Cristina Gonzalez Cachon]

There are many opportunities for research, including new accessible security authentication techniques, biometric authentications, remote monitoring and personalization of devices, and creation of frameworks for delivering accessible and security mobile applications. This area around remote monitoring focuses on how an enterprise can use mobile device monitoring capabilities, in combination with software inventory and distribution techniques to understand the configuration of the mobile device, the hardware, operating system, assistive technologies installed and what is in use, to be able to either provide guidance to the end user or to actually make that mobile device more accessible. It could be that someone's device is configured perfectly for their personal daily use, but when they try to use it for work, they discover that it doesn't provide the accessibility needed. So, the research here is in understanding the combinations, their interactions with work-related software, how mobile device management and monitoring tools that help with security can be adapted to better meet the needs of the users.

Additional Questions Asked

8.  General 1:  Samantha Bird - Are there any recommended tools to use with mobile accessibility testing ?

Automated tools for testing mobile accessibility are limited.    Apple has the Accessibility Inspector, but it is interactive, not automated.  Web-based apps can benefit from user agent switching to test.  This is an area that needs research and development.

9.  General 2. James Carter - What issues does responsive design generate for accessibility?

This is not something we have researched.  But our development teams are starting to use Responsive design.  Responsive design, in theory should be compatible and even help accessibility.  We assume you're talking about Web apps (we are unaware of responsive design for native apps).   Generally, the WAI-ARIA HTML5 tags for main content, navigation, search, need to be correct regardless of the display of the responsive design.  The user still needs to navigate and find headings and other content properly marked up.

10.  General 3. Shiv D - Importance of Mobile Accessibility and Effort to apply

One need only consider the tipping point of more than half of internet access is coming from mobile devices.  If mobile devices and apps are not accessible, we leave out a large user base.

11.  General 4 Marina Buzzi - I'm very interested in accessibility of dynamic apps on mobile devices for blind users  

When you say dynamic apps, do you mean responsive design?  See General 2 above.    Generally the issues on mobile devices for the blind are finding the data, especially data that updates on the screen.  In  a model where the user has to touch content to hear it, we need to consider the model of HTML5 / WAI-ARIA that surfaces alerts to the user based on a tiered level of importance.

PAPER 2: Enabling Flexible User Interactions With Mobile Media

1. Do you have any sense of the scale of benefits we may expect if annotation of the media is easy and able to occur in parallel to its rendering? and for what user group? especially for people with disabilities?

Streaming annotations synchronized with audio and video provides immediate access to key points of a subject and would not only help the 80% of those who are disabled who have specific learning difficulties by reducing the amount memory load but also provides vital access to the media for those with visual and hearing impairments and may reduce some complexities in content for individuals with other cognitive impairments. Realistically streaming any text with audio and video can help all users and supports the view that multimodal learning can enhance the uptake of knowledge.  This would be applicable to all those in a teaching and learning situation and many who are viewing and listening to items for pleasure.  Providing annotations that are synchronized with the media allows search engines to find information and thus researchers to find sections of media more speedily. The parallel rendering is implemented on some native apps, but on the Mobile Web it is still very difficult. Currently, it relies on the content provider to make the media file accessible on Mobile devices. For example, they need to provide the metadata, closed captioning and a suitable format that could be delivered via lower bandwidth. Integrating user generated content within this process will allow users to customize the media file and enhance the annotation making experience. User with different requirements, such as disabilities, can choose the best way of media delivering and presentation as they like.

2. What are the accessibility benefits in HTML5 that lead you to select it as a solution?

HTML5 is cross-platform Web standard that enables the use of descriptive and navigational elements and accessible media players that should not be browser dependent. So HTML5 to some degree solves the compatibility problems across mobile devices, which is part of the problem that accessibility on mobile platforms is trying to address. HTML5 along with other standards, such as Microdata, WebVTT, Media Fragment URI, etc, enrich the metadata about multimedia resources so that applications could use these metadata (or annotations) to make multimedia more accessible on mobile devices. It also opens possibilities of using semantic Web technology to improve the accessibility of multimedia apps on the Mobile platforms

3. What makes you think mobile media annotation is necessary, especially if mobile viewing is mostly of fragments or shorts?

The fact that you can search for the fragments via the annotations and thus connect with chosen content more easily. Viewing the whole video from the very beginning to the end, for example a 45 minute lecture, most of the time is not necessary, especially on mobile devices. It will be user friendly if the video is indexed by annotations linking to specific time points and users could choose which part they want to see. In addition, mobile access to multimedia resources are largely limited by the bandwidth, battery consuming, screen resolutions, etc. So only delivering the fragments also saves bandwidth and battery.

4. What kinds of annotations (speech, textual, bookmark, etc) are you interested in and how can these be enacted in a constrained - and possibly noisy - mobile setting?

Textual annotations and bookmarks are not impacted on by so called noisy mobile settings and are idea for mobile presentations when enacted in a bite sized way. We are interested in user generated annotations, such as comments, tags. We also need to add transcript to multimedia resources.

5. You say 'once the file is downloaded the media player mobile device is activated and no further interaction between the Web page or any other application can take place.' How does this apply to streaming media?

Mobile applications that include players tend to stream the media in isolation with no chance to pause and add items whilst the process is taking place. Here the "interaction" mainly means that the media player will take up the whole screen and users will not be able to do any other operations until they stop the player. Certainly, mobile device can play streaming media and the file wouldn't be fully downloaded. However, when you play it, the player will still take up the whole screen. One example is the BBC iPlayer. Currently, there is no easy way to get around that except developing native apps.

6. SMIL seemed to offer some solutions to this - why discount this, and why has SMIL fallen by the wayside - don't opportunities exist in the utilization of these technologies in this new and growing domain?

Yes, SMIL offers some solutions to this problem. But as far as we know, SMIL is not well supported by mobile devices, even though SMIL specification itself defines mobile profile extension in version 2.1 (see http://www.w3.org/TR/SMIL2/smil21-mobile-profile.html). One big problem about SMIL is that it is not supported natively by web browsers.

Additional Questions Asked

7. Which limitations exists in the implementation of the W3C Media Fragments Specification? As far as I understand you are proposing a mechanism to annotate media while playing it. [from Cristina Gonzalez Cachon]

The W3C Media Fragment Specification requires the cooperation of the client, proxy and server to deliver a certain byte range of the media file. There are many patterns that the byte range could be delivered. The core part of this process is the mapping from time, spatial and track to the correct byte range, which need to be done either on client or server side. Unfortunately, there is only few implementations currently, and they are limited to certain codecs and browsers. On client side, Firefox 10+ supports the temporal fragment. Ninsuna is a server-side implementation of Media Fragment Specification.

8. Is this use case out of the scope of the Media Fragments WG? Is it in the scope, but not yet implemented? [from Cristina Gonzalez Cachon ]

Media Fragment WG doesn't address the multimedia on mobile devices directly. But in the working group's requirement document (http://www.w3.org/TR/media-frags-reqs/#scenario3.5), it is obvious that one important usage of media fragment will be on the mobile devices. And another important statment given by the work group is that " data. There is often a lot of textual information available that relates to the media resource. Enabling the addressing of media fragments ultimately creates a means to attach annotations to media fragments" (see http://www.w3.org/TR/media-frags-reqs/#uc4)

---

PAPER 3: Accessibility In Multi-device Web Applications

1. How does your work relate to the symposium objectives and research questions?

Our work is focused on a specific technical challenge related to mobile accessibility: analyze techniques which help to solve/mitigate existing text input problems for mobile Web users. We take a broad view of accessibility - as theRIAM project showed, text entry problems affect all users on mobile devices - but people with motor impairment will be particularly affected, so a target group for our investigations will be older users who have trouble with text entry on mobile phones. We hypothesize that by understanding and helping this group, we will be helping users universally, who experience similar problems, to a lesser extent.This challenge is related to the “Guideline 3.3 Input Assistance: Help users avoid and correct mistakes” of the WCAG 2.0. More specifically, it is focused on section 3.3.3: “Error suggestion”. One of the main difficulties of this problem is to deal with the great diversity of hardware and software text input mechanisms. The appearance of new interaction models such as touch interfaces has led to the development of a great number of different keyboards, each of them presenting its own text input problems. Our work will contribute to know what is the state of mobile Web accessibility with regard to text input mechanisms, i.e. detect common problems in different families of devices and suggest possible techniques to solve them.

 

2. In your abstract you make an assertion that the user needs on the desktop can be directly transposed to the mobile context. Can you please elaborate on this? Isn't this a strong assertion and how do you know if it is a correct assertion?

We should rewrite the sentence saying that SOME of the desktop problems could also appear in the mobile context.  We state that similar problems apply to different user groups in different contexts. Specifically, situationally-impaired (small keys, distracting environment) mobile users experience similar problems as motor impaired desktop users.The correctness of the assertion has been proved as part of theRIAM project (Reciprocal Interoperability of Accessible and Mobile Webs) [1] following the methodology proposed by Trewin & Pain [3] to investigate pointing & typing errors of motor impaired users. It has been formally stated in Chen’s thesis [2]. They have proved it for a set of devices and we are analyzing if the same assertion applies for other family of devices.

 

3. How close are we to seeing results, and do you have any preliminary results or examples available?

At present, we don’t have any kind of results, since we are still analyzing the gathered data. The first step will be the creation of a complete table relating mobile keyboards types, common errors occurred when using them, and possible techniques required to correct them. The table will be done by analyzing logs of different mobile Web users while interacting with a basic prototype we have developed. Once the table has been completed, we will establish priorities to implement the correction techniques in different families of devices.

 

4. How does your work in MyMobileWeb relates to people with disabilities and accessibility in general?

MyMobileWeb is a generic framework to build multi-device Web applications. So far, it didn’t pay special attention to accessibility issues apart from basic techniques (e.g. alternative text for images). This work is a first step towards the consecution of a platform which facilitates the creation of accessible mobile Web applications by implementing the aforementioned text input correction techniques.

 

5. Expand on the way you intend to overcome the keyboard description problem.

We need a way to describe the different keyboards to be analyzed. The first approach inRIAM was to define the mapping between physical keys and characters by extension, as the prototype was based on a specific device model. In order to support a wide range of devices, we need to reduce the amount of information to express these mappings and also make them more comprehensible. The initial idea is to use a hierarchical model. For instance, we may describe a general key-to-character mapping for a specific device family (for instance, S40) and then describe model variations, language variations, etc. The idea is to include the keyboard mapping model in a Device Description Repository.

 

6. Do new smart phones already address some of the issues described?

Some devices incorporate text prediction techniques that help to reduce input errors when typing dictionary words (traditional T9, Swype, etc.) . The main problem is that, due to input mechanisms fragmentation, it is very difficult for developers to guess which features are supported in the keyboard of each device.

 

7. Over how many users will the MyMobileWeb data harvest be?

So far, we have collected data coming from a reduced number of older users (8 users). The intention of this study is not to cover a big amount of users but do it in depth. Moreover, while we analyze such data in order to obtain some preliminary results, we are still gathering new data from common users (not only older users).

 

8. Could you expand on the benefits you see for all mobile users.

The benefit is very clear: mitigate the problems of mobile Web users when entering text contents, thus improving the text input throughput and reducing user frustration.

 

References:

1. Web Ergonomics Laboratory – The University of Manchester. RIAM project: Reciprocal Interoperability of Accessible and Mobile Webs. http://wel.cs.manchester.ac.uk/research/riam

2. T. Chen (2011). Investigating retrospective interoperability between the accessible and Mobile Webs with regard to user input. PhD Thesis, The University of Manchester.

3. S. Trewin, H. Pain (1999). Keyboard and mouse errors due to motor disabilities. International Journal of Human Computer Studies 50:109-144.DOI:10.1006/ijhc.1998.0238

4. Fundación CTIC, Telefónica I+D et al. MyMobileWeb project. http://mymobileweb.morfeo-project.org

5. Media Informatics Group - University of Munich. UsaProxy project. http://fnuked.de/usaproxy/

6. Henze, N. and Rukzio, E. and Boll, S (2012). Observational and Experimental Investigation of Typing Behaviour using Virtual Keyboards on Mobile Devices. Proc. CHI, 2012.

---

PAPER 4: Assessment Of Keyboard Interface Accessibility In Mobile Browsers

Greg Gay, Inclusive Design Research Centre, OCAD University, Canada

1. In your abstract, you indicate that "mobile software developers will need to be aware of the important accessibility role played by keyboard interfaces and will need to test this functionality as a routine part of QA". In order to achieve this, what do you think can be (needs to be) done?
I think the best way for this to happen is for the platform developers to include keyboard interface testing tools in their QA suites. Frankly, it is in their best interest to maintain interoperability on their platforms. Android might want to remind its developers that some Android phones continue to include physical D-pads. Apple might remind its developers that external Bluetooth keyboards are supported.

2. What can be done to increase the mobile developers' accessibility awareness?
#1 platform developers need to include explicit statements about keyboard interface accessibility being part of the platform's intended look-and-feel #2 QA tools, as I already mentioned #3 a series of videos showing the same actions being accomplished in a variety of ways may help developers to better understand the issue.

3. In your abstract you cross reference requirements for keyboard accessibility on mobile devices with UAAG 2.0, how do these requirements relate to WCAG 2.0 from content perspective?
UAAG is a set of guidelines for user agents, but the line between user agents and operating systems can be easily blurred. E.g. Chrome OS. And UAAG already takes into account that a user agent may even be constructed from web content and run on another user agent (e.g.  online PDF viewers that render into HTML). Assuming that the platform supports keyboard interface control (which we show is generally the case), the responsibility then ships to content producers, not to break that keyboard accessibility with their content (whether regular HTML, Web apps, or mobile apps).

Additional Questions Asked

4. How do you perform the tests? would it be possible to automate them? To what extent? [from Cristina Gonzalez Cachon]
A. The tests were performed manually. I believe it would be possible to automate them both on devices and within the development environments. Probably 100% automation would not be possible, due to factors such as keyboard traps and the requirement to ensure that a focus indicator is actually visible.

PAPER 5: Inclusive Mobile Experience: Beyond The Guidelines And Standards

1. Do you think there is a tension between the increasing accessibility requirements resulting from the increasing complexity of mobile applications and development, and the need for less complex guidelines? and if there is a tension, what suggestions do you have to address such tension?

There is a great deal of tension between accessibility guidelines and complexity of mobile/touch based interaction design and development. This is not only due to the complexity of mobile applications but also due to the speed of that development and its mercurial nature as different companies and manufacturers apply their own user experience (UX) and inclusive design concepts. I’d like to answer the first part of this question with a current example and include tablets in my discussion as they are also mobile devices.

Having looked at Windows 8 “touch first” strategy and some Metro design patterns which will also be utilised in Windows Phone 8, user experience designers and touch developers are going to be confronted again with a new set of touch design constraints which will present accessibility challenges. Although assured by Microsoft that everything will be possible to do by keyboard that is designed to be done by touch, there are certain gestures (such as some wrist rotation based controls) that simply cannot be replicated in a keyboard/browser context. My statements are caveated with the fact that I have not done extensive accessibility testing on Win 8 Metro or Windows Phone 8 or Tablets (given its not publicly released yet) but have gone through the materials available online and some additional Microsoft design templates supplied during a sneak peek workshop. I have had the opportunity to have a play with a Win 8 tablet and do some app UX design for it.

What does this mean for Windows Phone 8 (Apollo)? Industry knowledge suggests that it will be tied closely to Windows 8 and will “share integrated ecosystems, components and user experiences” such as Internet Explorer 10. If that’s not a complex situation then I don’t know what is. That being the case, there should be an improvement in speech to text as Microsoft is introducing a redesigned version of Narrator, the text to speech solution, for Win 8 but the experience cannot focus on the visually impaired accessibility needs alone. Gesture based interactions, which much of the windows 8 and therefore Windows phone 8 ‘touch first’ approach relies on, present obstacles to anyone with motor impairment accessibility requirements. Additionally, how will this Narrator work with the ‘live tiles’ and ‘animated bubbles’ that are so integral to the Windows 8 and Windows Phone 8 experience?

The Microsoft Developer Network (MSDN) has many explanatory articles about how to build accessible Metro apps, most of which rely on implementations of HTML5-ARIA and JavaScript…or a combination of both in some way or another. One of the most heavily used screen readers, JAWS from at least versions 10 and above, even the latest version, JAWS 13, have trouble with HTM5-ARIA. I’m not a techie by any stretch of the imagination but even I can see many difficulties ahead here. The guidelines being referred to and added as references for this developer community are the W3C-WAI versions for HTML5 and nothing about mobile accessibility at all, not even the WCAG to MWBP mapping. They have a highly technical set of 11 ‘accessibility’ guidelines specific to their application development. There is nothing about the user experience. Microsoft is also assuring the disability community they will be working with the Assistive Technology (AT) companies to ensure everything is integrated but acknowledge the Win 8 & Windows Phone 8 will most likely require support to “ensure their ATs don’t break with each release” However a comment on the MSDN states “JAWS is once again broken now because XPDM mirror driver model was removed in Windows 8”5 which I don’t particularly understand what that means…and so on and so on…

I could actually go on and on with more Windows 8 and Win Phone 8 accessibility issues and challenges but hopefully I have answered the question with this example about whether there is tension between complexity of app development and a need for technology agnostic or less complex guidelines.

Now, as to what to suggest for an approach to these kinds of issues. This is really hard and I certainly don’t have all the answers.

As defined by Horst Rittel in 1973, I’d say we have symptoms of a pretty ‘wicked problem’. Some things that make up wicked problems include but are not limited to:

1. Have no definitive formulation
2. Have no criteria upon which to determine solving
3. Solutions to wicked problems can be good or bad
4. There are no complete lists of moves for a solution
5. There is always more than one explanation for a wicked problem
6. Every wicked problem is a symptom of another problem
7. No solution of a wicked problem has a definitive scientific test
8. Every wicked problem is unique.

Rittel also outlined some strategies to approach Wicked Problems, a collaborative approach; one which attempts, "…to make those people who are being affected into participants of the planning process . They are not merely asked but actively involved in the planning process…"

I would definitely advocate this as part of a strategy to solving or at least moving towards a solution of the wicked problem of trying to create standards in a mobile landscape made of quicksand. That being the case, user research to form the basis of these guidelines, added to the existing stable pillars of WCAG 2, which are relevant to the mobile and touch interactions.

The challenges with this approach are achieving a shared understanding and commitment to solving a wicked problem is a time-consuming process. That said, I believe it is time well spent.

I can make links to all of the Windows Phone 8 resources I have bookmarked online available to interested designers and developers.

2. What strategy would you recommend to help keep guidelines like WCAG up to date with mobile technology?

Continual engagement with users who have accessibility requirements is the key to maintaining living standards. In the first instance, a baseline needs to be established which is possible through detailed user research which I’ll talk about a bit later in answer to Question 4.

Cataloguing the new input mechanisms such as gesture, speech input and utilization of mobile device native functions such as the camera or geo-location and then identifying the accessibility challenges and subsequent requirements for design and development to meet those requirements. This first attempts to define the problem space which then provides the stage for solutions. Half of the issue we face is that the problem space isn’t well defined. We know it’s a huge issue but breaking it down into more manageable components would be a strategy for future standards development.

Standardizing gestures across platforms would go a long way towards assisting with good guideline creation. Luke Wroblewski has an excellent reference guide that covers all devices with gestural controls, such as Microsoft Surface and Wacom touch tablets, not just mobile devices. However, like everything in this dynamic area, it’s out of date already with Windows Phone 7 only being covered, not Windows Phone 7.5 and that will all get thrown out again when Windows Phone 8 turns up, which has a significantly reduced number of gestures and different concepts such as ‘semantic’ zoom which is activated using the ‘pinch’ gesture, currently identified as a gesture which will zoom in and out of a content area rather than ‘semantic’ zoom which displays all tiles in the Windows 8 screen landscape.

Using basic heuristics can also be of assistance with keeping guidelines up to date, as long as they are technology agnostic and specific to their sphere of use.

For example some gestural specific heuristics from Brian Pagan at UXMag:

1. “Gestures must be coarse-grained and forgiving - People's hand gestures are not precise, even on a screen's surface. In mid-air, precision is even worse. Expecting too         much precision leads to mistakes, so allow a wide margin and low cost for errors.         
2. “Users should not be required to perform gestures repeatedly, or for long periods of time, unless         that is the goal - Repetitive or prolonged gestures tire people out,         and as muscle strain increases precision decreases, affecting performance. If the idea is to make users exercise or practice movement though, this doesn’t apply.
3. Gestural and speech commands need to be socially appropriate for user's environmental contexts - The world can see and hear a user’s gestures and speech commands, and users will not adopt a system that makes them look stupid in public.         For example, an app for use in an office should not require users to make large, foolish-looking gestures or shout strange verbal commands. On the flipside, including silly gestures and speech into a children’s game could make it more interesting.

These heuristics can apply to general usability but can also easily be adapted to mobile accessibility such as not requiring speech from a user for passwords

Another idea would be to create a dedicated online area for users to input their challenges and requirements for mobile accessibility to have them considered by the W3C-WAI as the foundation for guidelines or changes to WCAG 2. There are forums online but they are usually technology specific and uncoordinated. For example AppleVis. This kind of ‘crowdsourcing’ could form part of the input for a working draft of these guidelines. People want to participate, they just don’t have a simple and well publicized method of doing so. This is not suggested as a free for all but would be utilised in a structured fashion, such as putting out a call for input around a specific guidelines area over a fixed period of time as not to required 365 days a year moderation.

3. Was the mentioned WCAG to MWPB mapping approach beneficial to help you get up to speed on related best practices and or for research and development?

Yes it did in demonstrating that there are many aspects of WCAG 2 that can apply to mobile devices. Color considerations for color blindness, contrast, use of understandable language that can be applied to accessibility hints that can be included in iOS apps. The gaping hole in this mapping was the lack of a full set of guidelines to cover gestural input which is required for us to manipulate what is effectively “pictures under glass”

Again standardizing gestures across platforms would be a great step forward to having something to baseline guidelines on.

4. A study is mentioned to help aid in the creation of new guidelines, could you describe your methodology in more detail?

The proposed study is a straightforward user research project, understanding what people do and why they do those tasks in that way. It will be looking for the challenges and pain point people with accessibility need face and from that we can possibly create some standards to alleviate the pain points.

Methods proposed to achieve this goal include

1. One on one semi-structured         interviews with users who have accessibility needs from their touch devices. This allows for the researcher to closely question how and         why a person uses their device. These will be used as inputs into the contextual inquiry.
2. A contextual inquiry research study which involves ‘shadowing’ users in the environment that they would use their mobile device. The intent is to observe the         usage without intruding and so ensuring the data gathered is as         naturalistic as possible. Questions are asked of the user only when absolutely necessary about things that have occurred.
3. Data analysis and synthesis to make meaning through abductive sense making and reframing the creation of personas (fictional representations of actual users), their user journey which describes their actions, emotions, thought processes and touchpoints.

This research will be pan-disability to cover a broad spectrum of user experiences.

5. What strategy would you recommend for testing guidelines’ effectiveness with disabled users?

Exactly that. Initially lab-based usability testing would be undertaken with real disabled users utilizing mobile apps or Websites that are deemed to be compliant. This would not just be ‘checkbox’ compliance testing (assuming there are agreed guidelines to test against) but also testing whether the product is usable and desirable for someone who has accessibility requirements.

The next stage is testing in the physical environment with users using observation techniques to see whether a design following particular guidelines is successful in delivering the right accessible experience in its actual context of use.

Additionally, if you as a designer or developer have a mobile in your pocket, you have a screen reader. Test it yourself first. There is no excuse.

Question 6: What is your WCAG X.0 wish list?

Full coverage of a consolidated set of gesture based interactions with guidelines in their design for people who have disabilities.

        

1 Paul Thurrott:         http://www.winsupersite.com/article/windows8/windows-phone-8-preview-142154

2 http://msdn.microsoft.com/en-us/library/windows/apps/hh452681.aspx                 

3 http://msdn.microsoft.com/en-us/library/windows/apps/hh700325.aspx                 

4 http://blogs.msdn.com/b/b8/archive/2012/02/14/enabling-accessibility.aspx                 

5 http://blogs.msdn.com/b/b8/archive/2012/02/14/enabling-accessibility.aspx - comment from user xpclient 15 Feb 2012 4:29 AM

6 Rittel 1972

7 http://en.wikipedia.org/wiki/Wicked_problem                 

8 Touch Gesture Reference Guide by Luke Wroblewski April 20, 2010         http://www.lukew.com/ff/entry.asp?1071         9 http://uxmag.com/articles/new-design-practices-for-touch-free-interactions                 

10 http://www.applevis.com/applevis-forum/accessibility-advocacy                 

11 Brett Victor, Apple.

12 John Kolko, Thinktiv

13 Henny Swan, BBC