Bug 12794 - <video> Add a non-normative note on how to provide text alternatives for media elements
Summary: <video> Add a non-normative note on how to provide text alternatives for medi...
Status: CLOSED LATER
Alias: None
Product: HTML WG
Classification: Unclassified
Component: HTML a11y Task Force (show other bugs)
Version: unspecified
Hardware: PC All
: P2 normal
Target Milestone: ---
Assignee: John Foliot
QA Contact: This bug has no owner yet - up for the taking
URL:
Whiteboard:
Keywords: a11y, a11ytf, a11y_text-alt, media, TrackerIssue
Depends on:
Blocks:
 
Reported: 2011-05-26 01:52 UTC by Silvia Pfeiffer
Modified: 2014-05-12 14:46 UTC (History)
12 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Silvia Pfeiffer 2011-05-26 01:52:54 UTC
People are unclear about the best way in which to provide non-timed text alternatives for accessibility needs to audio and video elements. Just to be clear: I'm not talking about captions and audio descriptions here, but about what a screenreader would say when a user tabs onto a media element on the page. What would a Web page author need to do.

The proposal is to use the aria attributes for this.

To clarify this, it would be good to add a non-normative example to the spec, maybe to the WAI-AIRA section or even to the media element section. Something like:

"Note: To provide non-timed text alternatives for media elements, use aria-* attributes. The following shows an example of how it can be used to provide short and long text alternatives to the initially loaded paused video, which only shows the placeholer image poster.png, and a summary for the video's content. The example provides this information both to accessibility users and users in legacy UAs that do not support the media elements.

<video src="file.mp4" poster="poster.png" aria-describedby="posteralt videosummary">
  <p id="videosummary">A Clockwork Orange Trailer (<a href="transcript.html">Transcript</a>)</p>
  <p id="posteralt">Poster frame is a clockwork orange movie poster (<a href="posterlongdesc.html">long description</a>)</p>
  <p><a href="file.mp4">Download the video file</a></p>
</video>
"
Comment 1 Simon Pieters 2011-05-26 10:13:25 UTC
I think the suggested example doesn't work since the paragraphs are descendants of <video> and thus not rendered (like display:none) and not present in the accessible tree. Correct me if I'm wrong.

Also, wouldn't the summary, transcript, and description be useful for all users?
Comment 2 Silvia Pfeiffer 2011-05-27 02:50:20 UTC
(In reply to comment #1)
> I think the suggested example doesn't work since the paragraphs are descendants
> of <video> and thus not rendered (like display:none) and not present in the
> accessible tree. Correct me if I'm wrong.

Excellent, this is what I want to find out: can we make it work.

IIUC the descendant paragraphs in <video> are in the DOM, but not rendered. Does that mean that the browsers mark them with display:none or @hidden and therefore they don't go through to the accessible tree, or are they marked differently? I'm asking because it would be a simple way to provide non-visible accessible text on the page, similar to what Jonas indicated may be possible to introduce for pointing aria-describedby to hidden elements.


> Also, wouldn't the summary, transcript, and description be useful for all
> users?

If they are, the author can, of course, put them on the page and the aria-describedby would point to the visible paragraph. However, with aria-describedby it is always the challenge how to describe to a blind user what is on the page without having to expose that text also to the sighted users. Therefore I picked this more challenging example.
Comment 3 Silvia Pfeiffer 2011-06-14 06:09:32 UTC
Also see some further discussions and examples at http://www.w3.org/WAI/PF/HTML/wiki/Media_Alt_Technologies
Comment 4 Silvia Pfeiffer 2011-07-13 22:20:15 UTC
This markup is still the simplest means of having non-visible text alternatives for the video and poster element.

Is it possible to expose the linked elements from aria-describedby through a shadow-dom approach to accessibility technology similar to what is described here http://glazkov.com/2011/01/14/what-the-heck-is-shadow-dom/ and similar to what is used for exposing the video controls to AT?
Comment 5 Silvia Pfeiffer 2011-07-13 22:21:35 UTC
Related bug: http://www.w3.org/Bugs/Public/show_bug.cgi?id=12228
Comment 6 Michael[tm] Smith 2011-08-04 05:03:58 UTC
mass-moved component to LC1
Comment 7 Ian 'Hixie' Hickson 2011-12-02 00:39:19 UTC
EDITOR'S RESPONSE: This is an Editor's Response to your comment. If you are satisfied with this response, please change the state of this bug to CLOSED. If you have additional information and would like the editor to reconsider, please reopen this bug. If you would like to escalate the issue to the full HTML Working Group, please add the TrackerRequest keyword to this bug, and suggest title and text for the tracker issue; or you may create a tracker issue yourself, if you are able to do so. For more details, see this document:
   http://dev.w3.org/html5/decision-policy/decision-policy.html

Status: Did Not Understand Request
Change Description: no spec change
Rationale:

Why do authors need anything other than just regular markup? To take your example:

  <h1>A Clockwork Orange Trailer</h1>
  <video src="file.mp4" poster="poster.png"></video>
  <p><a href="file.mp4">Download the video file</a>.
  <a href="transcript.html">Veiw the transcript</a>.</p>

(I don't see why anyone would care what the poster is.)

Can you point me to some existing sites to show how they handle this today? e.g. how does YouTube handle this?
Comment 8 Silvia Pfeiffer 2011-12-12 05:06:44 UTC
(In reply to comment #7)
> Why do authors need anything other than just regular markup? To take your
> example:
> 
>   <h1>A Clockwork Orange Trailer</h1>
>   <video src="file.mp4" poster="poster.png"></video>
>   <p><a href="file.mp4">Download the video file</a>.
>   <a href="transcript.html">Veiw the transcript</a>.</p>

That's certainly possible, and if @aria-describedby attribute was added to the video that linked to the transcript, that would allow screenreaders to announce it early, too. But that doesn't help in cases where the video is not supposed to have other markup about it around it, such as when it's "embedded" in another page through an iframe. This is why I suggested moving the extra content into the video element as fallback content. If that fallback content was available to accessibility APIs, then it would be the nicest way to have accessibility content that can be copied around with the video.


> Can you point me to some existing sites to show how they handle this today?
> e.g. how does YouTube handle this?

YouTube doesn't have a means to expose transcripts to embedded videos right now. This is actually a drawback with YouTube videos and users are trying to fix it by cutting and pasting he whole transcript into the "description" section. However, that is lost when YouTube videos are embedded.
Comment 9 John Foliot 2012-01-13 04:57:34 UTC
(In reply to comment #8)
> But that doesn't help in cases where the video is not supposed
> to have other markup about it around it, such as when it's "embedded" in
> another page through an iframe. This is why I suggested moving the extra
> content into the video element as fallback content. If that fallback content
> was available to accessibility APIs, then it would be the nicest way to have
> accessibility content that can be copied around with the video.

The requirement is as Silvia notes: authors require a means of adding both short and longer textual alternatives to media markup (<video> and <audio> elements) directly in the markup even when design considerations remove the ability to provide that text as visible on-screen text. 

Simon's comment (https://www.w3.org/Bugs/Public/show_bug.cgi?id=12794#c1) is correct in that adding this content as the fall back content will not be a viable solution, due to the fact that when <video> is supported the fallback content is not supposed to be exposed to users. The current specification states:
     "Note: In particular, this content is not intended to address accessibility concerns. To make video content accessible to the blind, deaf, and those with other physical or cognitive disabilities, authors are expected to provide alternative media streams and/or to embed accessibility aids (such as caption or subtitle tracks, audio description tracks, or sign-language overlays) into their media streams." http://www.w3.org/TR/html5/video.html#video

It is possible that aria-label could address the need for the short textual description of the video asset (but not the @poster asset), however the requirement for supporting a longer textual description has not been resolved. Related to Bug 12228 https://www.w3.org/Bugs/Public/show_bug.cgi?id=12228

Once this means is determined, author guidance on how to achieve this is required as non-normative text in the specification (http://www.w3.org/TR/html5/video.html#best-practices-for-authors-using-media-elements).


(In reply to comment #7)
> 
> (I don't see why anyone would care what the poster is.)

We have had numerous requests from non-sighted users via numerous email exchanges discussing this requirement. It is too bad that the Editor still does not understand this need, but it has previously been well articulated, and the requirement has not been addressed to date. 
Possibly related to Issue 142 (Status: Formal Objection) http://www.w3.org/html/wg/tracker/issues/142
Comment 10 Ian 'Hixie' Hickson 2012-01-13 20:37:48 UTC
EDITOR'S RESPONSE: This is an Editor's Response to your comment. If you are satisfied with this response, please change the state of this bug to CLOSED. If you have additional information and would like the editor to reconsider, please reopen this bug. If you would like to escalate the issue to the full HTML Working Group, please add the TrackerRequest keyword to this bug, and suggest title and text for the tracker issue; or you may create a tracker issue yourself, if you are able to do so. For more details, see this document:
   http://dev.w3.org/html5/decision-policy/decision-policy.html

Status: Rejected
Change Description: no spec change
Rationale: Just put the description in the embed. There's no reason this has to be complicated.

The issue relating to the poster hasn't been documented. Or at least, I've never seen any compelling description of why anyone would care. It's like asking for a description of an icon instead of asking what the icon means. The only times I've seen anyone asking for this is when misguided accessibility advocates have suggested it, in what I can only assume is a cargo-cult approach to making the Web accessible to people who can't see everything.
Comment 11 John Foliot 2012-01-14 01:16:28 UTC
The use of the term Cargo-cult Accessibility is offensive and should not be tolerated here, especially from someone who clearly hasn't got a clue about what accessibility and accommodation needs really mean. Stick to editing and stop trying to know everything.

TrackerRequest added.
Comment 12 Sam Ruby 2012-01-15 22:06:34 UTC
From the original resolution: if you would like to escalate the issue to the full HTML Working Group, please add the TrackerRequest keyword to this bug (done), and suggest title and text for the tracker issue (still needed).
Comment 13 Michael[tm] Smith 2012-01-18 03:42:05 UTC
John, please see Sam's comment #12:
> From the original resolution: if you would like to escalate the issue to the
> full HTML Working Group, please add the TrackerRequest keyword to this bug
> (done), and suggest title and text for the tracker issue (still needed).

As you were the person who escalated this one, can you please also supply the title and text needed?
Comment 15 Mark Sadecki 2014-02-13 14:58:15 UTC
The HTML WG concluded that that the TF could pursue this issue as an extension specification or a new feature of a future version of HTML.  

http://lists.w3.org/Archives/Public/public-html/2012Oct/0105.html

Closing this bug per discussion by the HTML Accessibility TF Bug Triage group:
http://www.w3.org/2014/02/12-a11y-bugs-minutes.html#item05