MLWorkshop2020

Cognitive Accessibility and Machine Learning: Beyond rule based testing environments

This paper aims to capture some related questions and the perspective of the Accessible Platforms Architecture (APA) group and the Research Questions Task force (RQTF) with regard to supporting people with cognitive disabilities via the use of Machine Learning.

Overview

Accessibility best practices and inclusive design for people with learning and cognitive disabilities is often hard to test using traditional rule based accessibility testing environments. For example, Making Content Usable for People with Cognitive and Learning Disabilities has a pattern (4.2.5) Clearly Identify Controls and Their Use. This instructs designers to use a clear recognizable design for interactive controls. Simply make it clear what controls actually are, any related affordances and how to use them.

Clearly this is hard to test via current automated rule sets. However, machine learning could be trained on corpus from usability testing, what a user group with cognitive impairments may successfully recognize, and what designs fail and result in the users inability to use the interface.

That machine learned experience can be used to test more sites with improved accuracy. A further enhancement could introduce an acceptable certainty threshold, for example we could say "likely" any given component may have over 90% certainty that it is accessible. This could be a way to bridge the gap between the need for user testing and the related expense and time involved.

In some cases, the world of inclusion may help support machine learning performance and vice versa. For example, new personalization semantics, published by the APA, introduces standardized semantics that allows web applications to customize the presentation of that content to one that is familiar to individuals based on their specific needs and preferences. This is effective as these semantic tokens allow the association of user-preferred symbols with elements having those semantics, thereby allowing greater personalization.

These semantics could benefit any application trying to map particular concepts to language, and machine learning can support the creation of these symbols. For example, machine learning could populate the correct semantics for symbols. When the wrong symbol is chosen by the algorithm, adding the correct semantics, will be relatively easy for an author who can understand both the intent of the content and a related symbol.

On the other hand, when the semantics are in the code that enable loading the correct symbol in the first place, automated language translations may use this clarification to improve performance, effectively reducing where language translation in words may fail.

Finally, machine learning holds great potential for helping people with disabilities use the web, by adding tools and adapting content to make it more usable. This includes adapting the text, design and flow of the content to meet diverse user needs. This may come with NLP (Neuro-linguistic programming) and other technologies. However, until these technologies are reliable they run a risk of confusing users who are least likely to be able to understand and correct related errors.

APA look forward to discussing these topics and collaborating in order to better identify where there is a nexus between Machine Learning, and cognitive accessibility.

If we find that there are API requirements or formats that need to be developed to support accessibility, use cases, or clear related user needs and requirements an outcome could be to collaborate further on developing these after the workshop.

More info / contact

Please contact Joshue O Connor (W3C/WAI) joconnor@w3.org and Lisa Seeman lisa.seeman@zoho.com for more info.