WAI R&D Symposia » Mobile Home » Proceedings » This paper.
This paper is a contribution to the Mobile Accessibility Online Symposium. It was not developed by the W3C Web Accessibility Initiative (WAI) and does not necessarily represent the consensus view of W3C staff, participants, or members.
Accessibility in multi-device web applications
1. Problem Description
The ongoing research work described in this article aims to reduce accessibility problems in the Mobile Web environment, particularly those problems related to text entry. So far, most accessibility research has focused on solving problems for desktop Web users and it is necessary to extend these studies to the Mobile Web field.
2. Background
The RIAM (Reciprocal Interoperability of Accessible and Mobile Webs) project [1] probed the relationship between the Accessible Web and the Mobile Web [2], investigating the extent to which common accessibility problems were experienced by both disabled Desktop Web users and Mobile Web users, and thus the possibility of migrating existing solutions from the (desktop) Accessible Web to the Mobile Web.
The research was based on the assertion that small device users and motor-impaired desktop users make the same types of errors in text entry and cursor movements, and accessibility solutions from one user domain would therefore apply to the other. The work focused on previous research from Trewin and Pain [3], which showed that users with impaired hand and finger control due to stroke, radial palsy, muscle loss or wrist stiffness, make six types of typing error and three types of pointing error.
When users unintentionally press a key longer than the default key repeat delay, the key repeats itself, causing a long key press error. If users fail to press two keys simultaneously, they generate dropping errors. Other typing errors include additional key error (a key adjacent to the intended key is activated), bounce error (users unintentionally press the intended key more than once), missing key error (users fail to activate the intended key) and remote key error (users press a different key to the intended one). Users also have difficulties pointing, clicking and dragging the mouse. It transpires that these errors also affect Mobile Web users, who use small devices under various conditions.
3. Strategy
The aim of this ongoing research project is to improve and expand on the work of the RIAM project, which used the HP iPAQ HW 6515 device to prove the occurrence of the various errors and test techniques to solve some of them. Our work will analyse text input tasks in a rich set of diverse mobile devices in order to obtain a list of errors. Some of them will coincide with those defined in the RIAM project and new ones might also appear after analyzing the obtained results. Our aim will be twofold. On the one hand, we will try to find out if the techniques defined in RIAM could be applied in a multi-device environment. Notice that some of the techniques already implemented in RIAM, that covers long key press error, additional key error, bounce error and key ambiguity error, might need to be re-implemented to reach a great range of target devices. On the other hand, we will try to discover new techniques to mitigate the rest of the errors. In order to cover multi-device Web development, it will use MyMobileWeb [4], an existing software framework that generates multi-device user interfaces.
The research will investigate the feasibility of moving user input error correction technology from the desktop to the Mobile Web using the MyMobileWeb platform. In order to achieve this, two steps are necessary:
- To store what the user is typing in order to analyze and correct or suggest other options later. RIAM achieved this through a modified version of UsaProxy [5]. Integration in MyMobileWeb is implemented by means of a JavaEE Filter.
- The wide market of mobile devices includes different types of data input technologies: keyboards for text input and diverse pointing and navigation mechanisms. A formal description is required to model the different keyboards and establish the mappings between characters and keys. Keyboard technologies include QWERTY-like physical keyboards, 12-pad physical keypads, QWERTY-like on-screen keyboards, and newer on-screen keyboards like Swype and SwiftKey. Pointing and navigation technologies include trackballs, physical navigation keys, and tactile screens handled with a stylus or finger.
4. Major Difficulties
Current mobile phones use distinct types of keyboards with behaviors that are rather different so modeling the relationship between keys and characters is a major challenge.
- Some techniques described in the RIAM Project can only be used with certain types of keyboards, limiting them to a small group of devices. As modeling all types of keyboard and keeping the models up to date may be time consuming, our approach will be guided by device penetration among the population. Prioritization of which types of keyboards to model first may also be based on an analysis of device popularity among a specific population group (e.g., older people in England).
- Several techniques, including those addressing bounce errors for non-QWERTY keyboards, additional key errors and key ambiguity errors, need a model to express a mapping between characters and keys and no previous work has covered this.
5. Outcomes
We anticipate two outcomes. The primary outcome will be tools that help developers to create multi-device Mobile Web applications enabling users to interact with UIs more efficiently. This has important economic and social benefits, as it will assist existing users and potentially encourage older users to embrace mobile technology.
A secondary outcome will be the opportunity to generate additional industrial investment. If mobile applications generated using this novel software framework are successful, then future work would investigate how the input techniques can be migrated to native applications so that all applications can use them. Such an undertaking would be technically complex and require additional funding.
6. Open Research Avenues
So far, the integration of UsaProxy within MyMobileWeb platform has been performed. Moreover, a prototype has been developed in order to extract logs related to the text input actions. Nowadays, we are analyzing these logs to extract useful information about the user errors. While finishing this analysis, we have elaborated a draft report related to the applicability of the techniques that has been previously covered in RIAM towards a multidevice environment.
In what regards to future work, a first step will be to produce a table with all the existing keyboards and variants, the errors which may occur when using them, and the techniques required to correct them. Such a table must be considered alongside statistics about the popularity of operating systems and device models, to decide which errors/techniques to cover first. This would provide a prioritised list of errors/techniques to be covered in sequence, thus providing usability improvements which would iteratively benefit more and more Mobile Web users.
More work on the adaptation of existing JavaScript code to different OS and browsers is required. We are presently investigating the browsers in which the current RIAM code works, in order to estimate additional work.
In addition to investigating the application of the techniques used by the RIAM prototype to new software keyboards such as Swype and Swiftkey, we envisage in future studying the feasibility of other approaches outlined in the RIAM project, but not yet implemented: target expansion, steady clicks, sticky icons with adaptive gain control, and SUPPLE/SUPPLE++. Furthermore, the two techniques already developed in RIAM should be improved (dynamic keyboard and true keys).
Lastly, we will analyze in depth the data gathered by means of our prototype. This will allow us to discover other possible errors not considered in RIAM. Furthermore, we would like to compare our information with data coming from other experiments [6] and to further investigate if the techniques proposed in RIAM could be applied to solve/mitigate new types of errors detected. Finally, we will study if our work would be extensible to other fields rather than mobile Web, e.g native development, spell checkers, keyboards improvements, etc.
Acknowledgements
The authors want to thank previous researchers in this field, particularly Dr. Tianyi Chen, for his efforts in the application of desktop accessibility techniques to the mobile domain.
References
- Web Ergonomics Laboratory – The University of Manchester. RIAM project: Reciprocal Interoperability of Accessible and Mobile Webs. http://wel.cs.manchester.ac.uk/research/riam
- T. Chen (2011). Investigating retrospective interoperability between the accessible and Mobile Webs with regard to user input. PhD Thesis, The University of Manchester.
- S. Trewin, H. Pain (1999). Keyboard and mouse errors due to motor disabilities. International Journal of Human Computer Studies 50:109-144.DOI:10.1006/ijhc.1998.0238
- Fundación CTIC, Telefónica I+D et al. MyMobileWeb project. http://mymobileweb.morfeo-project.org
- Media Informatics Group - University of Munich. UsaProxy project. http://fnuked.de/usaproxy/
- Henze, N. and Rukzio, E. and Boll, S (2012). Observational and Experimental Investigation of Typing Behaviour using Virtual Keyboards on Mobile Devices. Proc. CHI, 2012.