Applying Trust, Scale, and Usability Factors to Standardize Voice Agent Web Integration

This page contains a video recording of the presentation made during Breakouts Day 2026, along with a transcript. Video captions and transcript were automatically generated and may not properly translate the speaker's speech. Please use GitHub to suggest corrections.

Table of contents

  1. Video
  2. Transcript

See also:

Video

Transcript

ehind: On the current slide, we'll discuss the draft module on voice systems and conversational interfaces. These systems encompass a broad spectrum of technology that includes traditional voice menu systems, like the automated phone lines we've all encountered. As well as modern voice user interfaces and AI assistance, like Alexa or Siri.

ehind: We often think of these as convenient features, but for many users with disabilities, they are primary access points and become insurmountable barriers.

ehind: The module has a list of 24 user needs, and we're seeking feedback not only on whether we're missing any user needs, but also feedback on how we're presenting them, is the best format, a long list, and so on.

ehind: So the first example challenge presented is memory and cognitive load. So, we've highlighted a core example challenge. Users with memory impairments may not be able to remember specific phrases or numbers used by the system. To understand why this is a barrier, we have to look at the cognitive mechanics.

ehind: Most standard voice menus are designed with a linear temporal constraint. Unlike a website, where a user can scan a page at their own pace, a voice system forces the user to process information in real time. This places a demand on working memory, executive function, and processing speed. When a system lists 4 or 5 options, it expects the user to hold the first option in their mind while actively evaluating the third and the fourth. This creates an immediate state of cognitive overload for individuals with memory impairments or neurodivergence. A user's ability to complete tasks like changing a phone plan or reaching a nurse shouldn't be dependent on passing a spontaneous memory test imposed by the interface.

ehind: Below the challenge, we've pulled 4 example needs directly from a research module to illustrate how we translate these barriers into actionable requirements.

ehind: First, I need to complete my task without remembering option values such as press 2 to speak with the pediatric nurse. In a typical system, the prompt might say, press 2 for the pediatric nurse. This forces the user to store the number 2 in their working memory, while still trying to decode the meaning of pediatric nurse. A cognitively accessible solution is to flip the phrasing to speak with the pediatric nurse, pause. Press 2. By providing the destination first, we allow the user to decide if the option is relevant before they ever have to worry about the specific input required. This maps to the WCAG3 goal of supporting functional needs, like usage with limited memory.

ehind: A second example would be, I need to complete my task without learning specific terms or phrases. Many modern AI assistants rely on directed dialogues where the system only understands specific, rigid activation phrases. For a user, this feels like having to learn a second language just to turn on the lights. Our module recommends the use of simple, jargon-free terms, and more importantly, the flexibility to accept alternative inputs. This mirrors the WCAG complete, clear language guideline, which emphasizes that the burden of understanding should be on the system, not the user.

ehind: Third, I need the system to recognize my speech, even if it isn't typical. This is a critical equality issue. Most speech recognition engines are trained on a fairly narrow set of typical voices. They often fail for users with non-typical speech patterns associated with conditions like aphasia, Down syndrome, or Parkinson's. Furthermore, these systems often employ strict timeouts that cut a user off if they pause to formulate a word. In WCAG 3, this is addressed under adequate time and speech not relied on, requiring that timeouts be extensible, and that speech should never be the only way to advance a workflow.

ehind: Finally, I need… with limited options that make sense to me, so I can identify options quickly and not struggle with multiple steps. Multi-level, deep-nested menus are not friendly to cognitive accessibility. If a user has to navigate through four branches to find a solution, the pathway becomes a cognitive hurdle in itself. We are advocating for flatter structures and limited options that align with the OK principle of unnecessary steps. These needs aren't just theoretical, they're the foundation for the design principles we explore in the next slide.

ehind: Having identified the core barriers, we now shift to possible solutions. These three strategies serve as actionable blueprints drawn from our research to bridge the gap between current voice system failures and improved cognitive accessibility.

ehind: Solution 1, Error Recovery and Human Connection. The first solution is to simplify error recovery and establish the standard, predictable way to reach a human. Automated voice systems will inevitably fail for some users, whether due to a speech recognition error, a complex query the AI isn't programmed for, or users simply becoming overwhelmed. Our research recommends an escape hatch, a universal, easy-to-reach path, such as pressing a reserve digit like 0 to bypass confusing loops. This ensures technology remains a tool for service rather than a wall. Furthermore, we must provide alternative interaction paths, such as keypad inputs, text-based chats, for users who find spontaneous speech too stressful or just impossible.

ehind: Solution 2, reducing cognitive load through prompt design. Second, we must use prompts to reduce cognitive load. This is a subtle but powerful linguistic shift. As discussed previously, the standard press 1 for the help desk forces users to memorize a digit while they're still trying to decode the destination.

ehind: Solution 3, information chunking. Finally, we must group lists of options into small, manageable chunks where each item represents a single, clear idea. While a typical user might hold 7 items in their short-term memory, individuals with impaired working memory or attention fatigue can often only handle significantly fewer. By breaking deep-mested menus into smaller chunks, and removing unnecessary fluff, we prevent the immediate cognitive overload that leads to task abandonment. Finally, alignment with other work. It's important to recognize these are anchored in current and upcoming standards work. There are links available within this deck. They include the protected guidelines for vocag 3.

ehind: The APA's Accessibility of Machine Learning and Generative AI draft. and the supplemental guidance on what Act 2 for how people with disabilities use the web. By combining all these solutions, and having an ability to call a human backup, we can create a resilient conversational interface.

Julie Rawe: This is Julie. I just want to jump in with one note, which is, this, we have several slides that have a box that says relevant to other inclusion work. For the WCAG 3, this is something where we are actively providing input, to help, ensure that WCAG 3 will, will address the user needs we are highlighting. And then we are also flagging some other W3C projects where either we see, an opportunity for collaborating, such as with the APA, AI accessibility work that's ongoing, and we're also flagging, in some cases, some recent work that is relevant to, what our modules are covering.

Becca Monteleone (she/her): Alright, so it looks like I'm taking over. Our, next research module we're discussing is on technology-assisted indoor navigation, or wayfinding. And so here, wayfinding means how people orient themselves to explore or navigate through buildings, such as museums, hospitals, airports, and public transportation stations. We separate wayfinding from outdoor navigation, primarily because GPS doesn't function inside of buildings.

Becca Monteleone (she/her): So some of the example use cases that we lay out here include, for example, a user navigating a layover at a busy airport using a combination of digital kiosks and the airport's app. Or a user making a grocery list on a supermarket app and then organizing their list to move through the store by aisle number and location of item.

Becca Monteleone (she/her): So we, in this research module, we conducted a literature review last year to update this research module, which included updates to the sections on challenges, user needs, and solutions, but we're looking for additional feedback on recent research or things we may have missed in that, that literature review, and as Eric said, also how we're presenting this information.

Becca Monteleone (she/her): So, drawing on recent research, we've identified a number of potential challenges that are faced by people with disabilities that require cognitive accessibility when wayfinding. So, for example, the example challenge we have here. People with disabilities that affect visuospatial function may have difficulty doing things like constructing a mental map. Or orienting, or reorienting themselves, and or wayfinding without additional information, like text, or landmark navigation, or other means. So some of the user needs that we've identified around this would be something like, one, I need to review a proposed route multiple times. So a user should be able to revisit both individual steps and the whole route at any time. Two, I need to reorient myself multiple times if the tool updates my routes. That would include things like real-time updates, such as the on-screen map rotating when you make a turn. Three, I need information presented in multiple ways, not just in written text. This might include images, audio, or other means. And then four, I need an uncluttered interface so I'm not confused or distracted by unnecessary details, which would include things like advertisements appearing or unnecessary graphics, these kinds of things.

Becca Monteleone (she/her): And so while we've provided an example here of visuospatial challenges, in the module, we highlight 7 different areas where users with cognitive accessibility needs might face challenges around wayfinding. And so to just give you an overview of what those are, those include memory, executive function, attention, language. Perception, processing, and interpretation, and knowledge acquisition, retention, or recall. Okay, now we can go on, Julie.

Becca Monteleone (she/her): And then, so, some possible solutions. So again, drawing from the recent academic research on wayfinding and disability, we also highlight a number of potential solutions for meeting user needs when wayfinding. There are currently 14 listed in the research module, but I'm just gonna highlight a few for you here.

Becca Monteleone (she/her): And so the first is to make sure to include physical, wayfinding aids, such as directional arrows, color-coded pathways, signs, integrate those into the digital aids, and so a user can be able to see on the digital aid what kind of physical wayfinding markers or landmarks to look for.

Becca Monteleone (she/her): Next, provide an option to present directions in the smallest steps possible. So while that's not ideal for all users, some of whom will benefit from being able to review the full route at once. Others will benefit from having step-by-step directions presented as a single action, right? Continue straight, turn left, etc. So, avoid providing too much information at one time.

Becca Monteleone (she/her): Thirdly, provide multiple methods for accessing directions. So, for example, provide step-by-step directions that are provided in text, in audio, or video, and a map with a directional overlay, such as an area… such as an arrow that also provides spoken directions.

Becca Monteleone (she/her): And then, additionally, provide methods to always access those directions. So, for example, instead of relying on static digital kiosks, have methods for users to be able to revisit directions at any time, especially while they're moving through the route. And then include ways to personalize the indication, right? So, yeah, and then finally, allow for, personalization of terms, such as directions and measurements, so that users can customize any of these, kind of, both key language, areas, as well as, like, those distance measurements. And then, as on the previous one, we've also linked to some other relevant inclusion work, both what we are, contributing to WCAG 3, as well as some other W3C projects that are relevant here. Thank you.

Rashmi RK: So… Hi, this is Rashmi. I will take over for the next module. And do… I will introduce you to the research module, that is Online Safety and Wellbeing Algorithms and Data. Internet use and apps can create a number of risks for people who need cognitive accessibility support. This module includes safety and well-being issues for these users, including cybercrime, mental health, privacy, algorithm, curated content, representation in data sets, artificial intelligence, etc. So, I will give you a glimpse of our module by sharing few key user challenges, user needs, and potential solutions to address those challenges.

Rashmi RK: Our first user deep draws attention towards challenge with social media and its effect on mental health. We all know how social media is affecting mental health. Social media platforms' algorithms is designed to keep users' attention, but often hurt well-being. Their algorithms often prioritize content that increases engagement, sometimes, but at the cost of user well-being. Spending prolonged time on these platforms has been linked to increased risks of anxiety, depression, and other overall reduce well-being for some user groups. Artificial intelligence may amplify this by learning individual user behaviors. For example, algorithms may continuously adopt to show content that keep users engaged for longer periods. Unfortunately, this is often related to negative emotions. This may have two main effects on mental health of the user. First, existing mental health issues may become worse, and second, people may develop mental health issues such as anxiety, depression, etc.

Rashmi RK: So, our users need better apps, and that do not have this effect of negatively impacting mental health, or lead to social exclusion. Now, let's come to second user need. Privacy and data protection options presented clearly and in an accessible language. Let's understand how this affects our users. People with cognitive and learning disabilities are at a higher risk and may be unable to take the recommended safety precautions. Safety of personal information or fear of data leak can prevent people from using apps and websites that may be valuable or essential to them.

Rashmi RK: For example, mental health apps are very useful intervention for providing healthcare, but research tells user leaves the app as they have concerns whether their mental health information will be kept private, or how their data will be used. So, clear and simple settings will help our users understanding the risk, and thus making decisions to use the essential apps.

Rashmi RK: People may also avoid apps due to complexity or cognitive load, and that leads to another negative side effect. This makes many groups invisible or underrepresented in data-driven decisions due to biases with algorithms.

Rashmi RK: The third user demands extra support to be safe and secure when using a website, especially if providing information or communicating with others. The underlying challenge with this user group is that they are vulnerable to financial risk. And may easily fall in trap of may be a victim of deception cybercrime, or scammers, etc. So the user needs extra support to be safe and secure when using a website, especially if providing or sharing the information or communicating with others.

Rashmi RK: Now, let's move to our focus to potential solutions. Next slide, please. Thank you. So, in our module, we have recommended potential solutions to promote safety and well-being of our users. This includes review and constantly improve algorithms, and other risks that may affect mental health and safety, including for diverse user groups and vulnerable groups. This may include, like, employ… employing server-side solutions, such as analytics to find cybercriminals or scammers. Further, our model recommends to establish a standard process for reviewing safety and well-being in vulnerable groups and responding appropriately. The review should promote to users to understand the nature of risks and benefit of treatments and alternatives. So that they can make proper decisions. Guarantee of the level of privacy of data and the ease of it, user can control their data. And when using data for design and decision making, ensure that all groups and subgroups are proportionately represented. Research and longitudinal studies should be performed to ensure that all groups and subgroups of people are proportionately represented in the dataset.

Rashmi RK: Safety should be a priority when making accessible… content accessible for people with cognitive and learning disabilities. All user information must be kept safe to the fullest extent possible. Any clues that the user has cognitive disability, such as request of even simplified version, should be protected information. Personalization system should be designed so that any information implying vulnerabilities are on the user device and are secured. And the most important thing, test privacy and data terms with different groups of users, requiring cognitive accessibility. To confirm that they understand what data may be collected, how it may be used, and who may have access to it, so that there should be no hesitation or fear, and they can use it easily.

Rashmi RK: And following these measures will not only help our user groups, but will help to promote safety and overall well-being of broader user groups. Today, I could cover a very small portion of the module, and I would request you all to go through the whole module and share your valuable inputs and insights to make it more effective and refine it further.

Rashmi RK: And as we see in the, this box, there are a few relevant work that is in WCAG 3 and APA, that is related with our module, that is related with authentication, risk, algorithms, and avoid deceptions in the WCAG 3, and in APA accessibility of machine learning and generative AI. So I… We would request you to please provide your feedback and make it better and effective. Thank you, over to Len.

Len Beasley (he / him): Hey there, this is Len Beasley. We're going to talk about supported decision-making online. Now, supportive decision-making is a person-centered approach that empowers individuals to make their own choices with the help of trusted supporters. Rather than distributing decision-making authority, this model emphasizes autonomy, dignity, and inclusion in everyday life. And this module includes a definition of the core principles of supportive decision-making, clarifies how it differs from legal guardianship. And outlines the specific roles of supporters. It also identifies the unique risks found in online settings, and offers practical strategies for implementation.

Len Beasley (he / him): So, a major challenge we see online is this tension between safety and autonomy. For many of us, managing spending or even managing digital assets can be difficult. However, the solution isn't to take away control. We have to ensure that online support tools act as empowerment, not digital parental controls. The goal is to provide a safety net that respects the user as an adult.

Len Beasley (he / him): Now, there are 16 core user needs covered in that research, with about 8 related user needs, but I'm going to highlight these 3 as they represent the foundation of a safe, autonomous experience. The first one here is kind of the support for those high-stake decisions. Users need help when a choice affects their financial or physical well-being. Protection from helpers is also not just about, like, a hacker thing or bad actors. We… user needs… users need systems that keep them safe from bad actors that they may actually know. And then the last one here on this list is just emotional regulation. We… we're looking for tools that regulate emotions to prevent impulsive or irreversible decisions.

Len Beasley (he / him): So now that I've identified those, let's actually look at the next slide and talk through, like, how we can, bring some of those together. Let's look at some possible solutions our research highlights. These aren't just features, they're structural changes to how we design digital interactions to support cognitive health.

Len Beasley (he / him): Our first recommendation is confirming users' understanding before reversible, irreversible actions. We've all seen that Are You Sure pop up. But for a lot of us, a simple yes click can become muscle memory, or even a mistake. So we're advocating for systems that ask users to demonstrate they understand the terms, perhaps by selecting the correct outcome from a list, rather than just clicking through.

Len Beasley (he / him): Next, we're proposing settings that allow for more thought when a risk is detected. Imagine a cooling off period for social media posts or unusual spending. Like, if an algorithm can detect a high-stress or high-risk situation, it could build in a delay. This gives the users time to regulate their emotions or consult a supporter before a post goes live or a large purchase is finalized.

Len Beasley (he / him): Finally, we're going to address the role of the supporter. Like, we suggest helper role definitions with audit trails. This allows a user to establish exactly what a helper can and cannot do in their settings. By providing a clear audit trail, we can then create accountability, ensuring that support remains helpful and doesn't cross the line into exploitation or over-control.

Len Beasley (he / him): It's important to note that this work isn't happening in a vacuum. These strategies align directly with the emerging WCAG3 guidelines, especially around help available and managing risk. We're also collaborating with groups like the Edge Technology Community Group to ensure surrogacy support is baked into the very architecture of the web. And all of these things, we would love feedback on what have we missed, what research should we really get into, how is the, how do some of these, support techniques, how can we improve them? How do we refine them, and how can we better work with a group? If there's a group missing on the relevant work area, please let us know. But ultimately, these solutions are about moving away from digital guardianship. And moving towards a true digital empowerment.

Len Beasley (he / him): And with that, I'm going to hand it over for questions and feedback.

Lisa Seeman: Cute. So, I think we're gonna stop recording now, hopefully? So some things to think about if… if you're aware of anything that we need to know about, such as additional user needs, or research, or technology we missed out. If there are other things you'd prefer changed, like, if you've looked at the document, perhaps the format of user needs don't work or make sense to you, we're very interested in groups that may want some kind of collaboration, integrating some of the user needs, and of course, any other questions or answers. And then we could start to explore practical solutions.