This is a DRAFT resource that supports Working Drafts of WCAG 3. Content in this resource is not mature and should not be considered authoritative. It may be changed, replaced or removed at any time.
🔙 WCAG 3.0 (Silver) Guidelines (Captions)
Captions
@@
Summary
Captioning in Mixed Reality relies on two related concepts:
- The ability to understand the purpose of a sound within an environment
- The ability to locate that sound within 3D space
The combination of these concepts soften the boundaries between captioning and audio description as both elements are required within XR captioning. Understanding sound purpose is most likely to be achieved through captioning. Locating sound within space is most likely to be achieved through audio description.
Why
Captioning provides an alternative method for users to understand auditory content. There are many reasons why people may prefer to use captions as an alternative to traditional audio or as a method to supplement audio content.
Who it helps
- Usage Without Vision - People that do not have access to visual information will require to have information presented in an alternative method. For XR captioning this should take into consideration the text that is being output and the orientation of sound location. In addition, any meta-information that is attached to textual content (e.g. the name of the person speaking) should be made available.
- Usage with Limited Vision - People with limited vision may have similar requirements to those without vision and all of the items mentioned previously should be considered. In addition to this, screen magnification users may need to carry out additional customization options that relate to the size of textual content and meta-information.
- Usage without Perception of Color - People that have atypical perceptions of colour may need to customise the presentation of captions within an XR environment. The presentation of captions in XR should take into account the real/virtual world that the user is interacting with and should make sure that text remains legible in regard to contrast with the background. Care should also be taken to ensure that any meta-information relating to the directionality of sound (e.g. radar plots, directional arrows) also takes color contrast issues into consideration.
- Usage without Hearing - People that use XR without hearing will require auditory information to be translated into an alternative format. Auditory information can include, but is not limited to, speech and key sound effects. In addition, the directionality of any sound will also have to be communicated to the user with this taking into consideration sound that takes place outside of the current view screen. The exact format that auditory information is translated into is not confined to captions. People may have a preference for signing of text alternatives or equivalents
- Usage with Limited Hearing - People with limited hearing may have some of the needs that are described when using XR captions without hearing. In addition to this alternative customisation options relating to sound direction may be required.
- Usage with Limited Manipulation or Strength - People with limited manipulation or strength may want to interact with content in an immersive environment that doesn't require particular bodily movement. These interactions can include captioning services where the timings for interactions may need to be modified or extended. In addition users of assistive technology may want to identify locations, objects, and interact with immersive environments.
- Usage with Limited Reach - People with limited reach may have similar user needs to people with limited manipulation or strength so these should be considered.
- Minimise Photosensitive Seizure Triggers - In order to minimise photosensitive seizure triggers, people may need to personalise the immersive environment in various ways. This can include personalisation of XR captions which should take into consideration methods that can reduce photosensitive seizures.
- Usage with Limited Cognition - People with limited cognition may require to change the speed at which they travel through an immersive environment. The timing of captioned content should take this into consideration. Personalisation of captioned services may be required in order to assist in creating an accessible immersive environment.
How
To be done:
- Different ways that the research has said to implement it.
- Explain the difference between what is done in the engine and what is done by the content developer.
- Research needed on how to mark caption file in order to take into consideration the location of sound source.
Planning responsibilities
For Content Development
The project team should consider the purpose of the content that is to be consumed using a given engine. Is this entertainment, informative, or educational? These aspects could affect which captioning features of the engine or type of action to be used. For example, an educational virtual environment would need to caption every character, with the ability to re-read captions or read a transcript. An entertainment virtual environment would want to limit excessive on-screen text that could detract from the users experience. A virtual reality game would need to inform the user of sounds behind where they currently are facing. The type of captioning selected is often an artistic decision.
For Engine Development
Ensure usability studies include a broad spectrum of captioning needs. Include:
- people who are deaf,
- people from the Deaf community (native sign language speakers),
- people who are hard of hearing,
- people with age-related hearing loss,
- people with auditory processing difficulty,
- people who prefer captioning, and
- People who have situational lack of sound.
Tips for collaboration
- Still to be developed. We will include this in a future working draft.
Planning for each stage
How to get started early
- Still to be developed. We will include this in a future working draft.
How to remediate
- Still to be developed. We will include this in a future working draft.
Design responsibilities
- Ensure that captions are in sync.
- Ensure the source of the caption, such as the speaker or other context is clear.
- Ensure sufficient colour contrast for captions.
- Captions can be resized without loss of legibility or clarity of context.
Designer tips
- When designing a UI, consider how subtitles might be incorporated. For example, is
chrome
at the bottom of the viewport tall enough that two lines of text could fit in that location? Could, when captioning is enabled, icons and statistics in a status bar be reduced in size to make room for subtitles? - Use color to show the differences between captions that relate to different speakers or captioned / audio description. NOTE: The use of color variation to associate captions with particular speakers is useful in this context. Generally color alone should not be used to denote important information to the user.
- Be aware of occlusion with captioned content.
User testing and meaningful involvement
Captions were originally developed for people with hearing loss. Today, studies show that 80% of people using captions do not have a hearing disability. Correctly captioning sound effects happening outside the user’s point of view is audio description, which is usually provided for people who are blind.It is important to involve people with a variety of disabilities and hearing ability in captioning. Involving people with disabilities in the early stage of a project can provide valuable input from experts on what will be the best way to provide a service to a broad spectrum of users.
Technical responsibilities
- Developers need to familiarize themselves with the native subtitling and captioning features of the
platform for which they are developing.
- As with other aspects of accessibility, compatibility is usually more important than providing features directly.
- However, the contrary position might be more relevant, as XR and VR remain on the leading edge of technology, it follows that captioning in these environments might also need to be ad hoc.
- In a similar vein, there are third-party APIs for streaming text. Compatibility with these is an important technical feature.
Technical tips
- Any non-player characters and avatars with speech dialog will have scripts. Have an option for exposing these scripts to the users/players. Speech-only dialog by voice actors will still have scripts, so retain the screenplay during development so that it does not have to be recreated post-production.
- Know that Automated Speech Recognition (ASR) is not a panacea.
- Learn what options for telephony based speech-to-text services are available in the geographic regions your products will be distributed. Provide features that facilitate use of these services.
W3 Immersive Caption Community Group - Subtitle Display
The Immersive caption community group, led by Christopher Patnoe, have created a number of demonstrations of captioning methods that can be used in XR.
BBC Research - Subtitle Display
BBC R&D have conducted exploratory work in how subtitles can be placed in 360 video, with this looking at a number of different options (evenly spaced, follow head, follow with lag, appear in front).
Oculus - Vacation Simulator - Subtitle Update
Vacation Simulator is a popular game that is available for the Oculus Quest VR headset. A recent update to this game brought in-game subtitles.
Very few resources exist to assist in creating captions for XR environments. This is, in part, due to the emerging nature of this field and the lack of a common ground between developers. We, therefore, leave this how-to resource section blank at this stage.
That said, the following resources serve as examples:
- The Immersive caption community group, led by Christopher Patnoe, have created a number of demonstrations of captioning methods that can be used in XR. Information on their work can be found at https://www.w3.org/community/immersive-captions/
- BBC R&D have conducted exploratory work in how subtitles can be placed in 360 video, with this looking at a number of different options (evenly spaced, follow head, follow with lag, appear in front). A summary of their work is available at http://downloads.bbc.co.uk/rd/pubs/whp/whp-pdf-files/WHP330.pdf
- Vacation Simulator is a popular game that is available for the Oculus Quest VR headset. A recent update to this game brought in-game subtitles. Information on the presentation of these can be seen at https://www.oculus.com/blog/see-what-we-said-vacation-simulator-subtitles-update-and-language-support/?locale=en_GB