Introduce a new <firstframe> element as a child element of <video>
- Related to: ISSUE-142: No alternative text description for video key frame (poster)
- Submitted by: John Foliot
- Date: November 13, 2010. Last updated January 24, 2011.
- Please address feedback to the HTML Working Group mailing list (email@example.com)
In the current Draft Specification, an author can specify any image to be used as a placeholder for a video by using the @poster attribute. A bug has been filed which states that the image should also have an alternative text (by way of an @alt attribute). However, there is some debate and concern that adding @alt to the video element cannot meet the needs of those users who need or wish to know what the alternative to the specified image is, as any @alt value added to the video element should be related to the video content, and not the placeholder image - itself declared as an attribute of the <video> element - as attributes cannot take on further attributes.
As well, the @alt attribute as a solution to this problem suffers from some other known issues:
- Content of the alt attribute is not available to some users who may find it useful.
- If the alt attribute content is too long it may cause display issues in some browsers.
- If the alt attribute content is too long it may cause reading issues in some assistive technologies.
- Semantic structure cannot be added to alt attribute content.
This Change Proposal seeks to address this problem, and also looks to properly resolve Bug #10642, by introducing a new child element to the video element: <firstframe>.
- 1 Introduce a new <firstframe> element as a child element of <video>
- 1.1 Summary
- 1.2 Rationale
- 1.3 Details
- 1.4 Impact
- 1.5 References
The current system of providing an author selected placeholder image introduces a number of confusing and problematic issues, including:
- The current Draft Specification concerning Media Elements does not allow for @alt to be applied to the <video> element, a fact further confirmed by the DRAFT analysis of fallback mechanisms for embedded content and the corresponding chart at http://lists.w3.org/Archives/Public/public-html-a11y/2010Nov/att-0246/embeddedcontent.html.
Should the Draft Specification be modified to change this however, the value string for the @alt text should be directly related to the root video asset's content, and not a supporting image file, which may or may not be directly related to the video content: @alt (and it's value) represents an attribute of the <video> element, and not of another attribute of that element (@poster). In other words [video + alt] and not [video + (poster + alt)]
(NOTE: At this time, I believe that adding @alt to the video element is semantically weak and inappropriate: while I believe it is important if not critical to provide a textual summation of the actual video asset for accessibility considerations, attributes such as @aria-labelledby, @aria-describedby, or (@longdesc*) applied to <video>, or perhaps <summary> as a child element of <video>, would be more accurate and useful to non-visual users.)
- At this time, the @alt attribute is reserved for the IMG element and image only elements <area> and <input type=image> - no other embedded object (via <embed>, <object>, <iframe>, <video>, <audio>, etc.) takes this specialized attribute, and for good reason: @alt is defined as "The primary method for providing text alternatives for images" (http://dev.w3.org/html5/alt-techniques/#altmethod) and not for other types of richer embedded content such as VIDEO, because textual alternatives to the actual video content is already provided via longer text forms, such as transcripts, captions, etc., and referenced by the <track> element (a child element of video).
- The "name" of the current attribute - poster (@poster) - has already proven to be confusing to content authors. Numerous contributors to the HTML Accessibility Task Force mailing list, as well as comments associated to Bug 10642 have repeatedly alluded to @poster as the equivalent to traditional movie posters, as seen in brick-and-mortor movie theaters. While an image such as this *could* be used as a placeholder image, content authors are free to specify *any* image as a placeholder, including rich visual images such as those traditional posters, a 'key-frame' taken directly from the video, or a third party "splash screen" such as an MPAA green-screen.
- It has been (correctly) stated that the textual alternative to the placeholder image might also need to take on additional attributes, with internationalization being cited as one (for example, an English language splash screen preceding a foreign language video with English subtitles: does the video @lang attribute apply to the content of the video, or the splash screen? In the case of subtitle files associated to a video, the @srclang value is applied to the (child) <track> element, to allow for multiple language subtitles, as well as ensuring clear semantic association).
In the case of complex images (the Casablanca movie poster) a means of both a terse alt value as well as a longer textual description of the imagery is also needed - none of these additional attributes can be applied to the placeholder image defined by @poster, yet neither is appropriate to the <video> content either: they are directly related exclusively to the placeholder image.
- It has been argued that the image specified via @poster is in-fact the same as the video:
"The <video poster> is conceptually part of the media resource itself" http://www.w3.org/html/wg/wiki/ChangeProposals/NoPosterAlt
However the current Draft Specification states otherwise:
"The poster attribute gives the address of an image file that the user agent can show while no video data is available." http://dev.w3.org/html5/spec/Overview.html#video
However, even if you accept this argument, the ability to properly convey all of the relevant imagery and/or text in the placeholder image provided to the sighted user can exceed the capabilities of the @alt attribute, as the possible first frame image at http://popculturefan.com/a-clockwork-orange-poster.jpg suggests. The complete text of that actual movie poster reads:
Being the adventures of a young man who's principle interests are rape, ultra-violence, and Beethoven. Stanley Kubrick's Clockwork Orange. A Stanely Kubrick Production "A CLOCKWORK ORANGE" Starring Malcolm McDowell, Patrick Magee, Adrienne Corri and Miriam Karlin. Screenplay by Stanley Kubrick. Based on the novel of Anthony Burgess, Produced and Directed by Stanley Kubrick. Executive Producers Max L. Reab and Si Litvinoff. Warner Bros A Warner Communications Company.
... clearly too long for the @alt attribute alone, yet containing important textual information that must also be conveyed to non-sighted users somehow. Whether or not an individual agrees or disagrees that an image is conceptually part of a video, or a related but semantically unrelated stand-in image for the video, the @alt attribute alone is potentially too weak to address the possible complex imagery that often comes with a video 'poster'. Moving the image container from an attribute to an element allows for greater semantic attributes to be applied to the image: @lang, a "verbose as necessary" text description, etc.
For these reasons, this Change Proposal seeks to introduce a new child element to <video> (and potentially <audio> as well): <firstframe>. While this Change Proposal is not strongly wed to the actual name of the new element (outside of eliminating the poorly chosen "poster" name), I believe it is both accurate and useful nomenclature for the core concept: a child element of the media element(s) that specifically addresses both the author desire to specify an alternative placeholder image, as well as ensure that all supporting attributes - including important accessibility requirements - can be associated specifically to that image.
The precedent of children elements to the media elements of <video> and <audio> has already been well established, with the introduction of the <source> element ("...allows authors to specify multiple alternative media resources for media elements") and <track> element ("...allows authors to specify explicit external timed text tracks for media elements.") While this ability is extremely important for accessibility considerations (allowing for the direct association of captions and transcripts) it also serves all other users by allowing for the direct association of supplemental content to the video such as Chapters and MetaData via a child element. For these reasons, the current design pattern of:
<video> <source> <track> </video>
...is already becoming well understood and taken up by the mainstream authoring community. Adding additional child elements of <video> will thus be easy to understand and implement at the author level. A new <firstframe> element would "allow authors to specify an explicit external image file for media elements" in a similar way:
<video> <firstframe> <source> <track> </video>
However, as an actual HTML element, <firstframe> could now also take on the existing Global Attributes, some ARIA attribute models (such as aria-describedby), event handler content attributes (should authors seek to add scripting effects to the placeholder image), as well as other specific attributes - including @alt and (@longdesc*) - conceptually important in serving the accessibility needs of non-sighted users. It acknowledges the fact that the author chosen placeholder image may contain information important to the end user, and provides a robust means of ensuring that this meaning/intent and related attributes (such as @lang) can be correctly and properly applied to the image.
(* This change proposal acknowledges the current impasse (Formal Objection) surrounding @longdesc, and is agnostic on its outcome, but notes the potential usage if @longdesc is reinstated)
Modification to 4.8.6 The video element
- Remove @poster from Content attributes
- Remove the text:
"The poster attribute gives the address of an image file..." to "...Note: The image given by the poster attribute, the poster frame, is intended to be a representative frame of the video (typically one of the first non-blank frames) that gives the user an idea of what the video is like."
- Replace the removed text with the following:
The firstframe element gives the address of an image file that the user agent can show while no video data is available. The element, if present, must contain a valid non-empty URL potentially surrounded by spaces.
If the specified resource is to be used, then, when the element is created or when the firstframe element is set, changed, or removed, the user agent must run the following steps to determine the element's first frame:
- If there is an existing instance of this algorithm running for this video element, abort that instance of this algorithm without changing the first frame.
- If the firstframe element's value is the empty string, then there is no first frame; abort these steps.
- Resolve the firstframe element's value. If this fails, then there is no first frame; abort these steps.
- Fetch the resulting absolute URL, from the element's Document's origin. This must delay the load event of the element's document.
- If an image is thus obtained, the first frame is that image. Otherwise, there is no first frame.
- Change any subsequent references of poster frame to first frame
- Re-number "4.8.10 Media elements" to "4.8.11 Media elements"
- Add "4.8.10 The firstframe element"
Creation of 4.8.10 The firstframe element
(NOTE: I make no pretense of being a Technical Editor, and request assistance in ensuring that the following prose explanation be converted to workable technical text.)
- The firstframe element is a child element of video, and is conformant only as a child of video
- The firstframe element is an optional child element of video, and is not required for conformance
- (In practical terms firstframe is a special instance of <img>)
- The firstframe element can take the following attributes: Global Attributes, src, alt (others: ARIA, longdesc?)
- The firstframe element MUST contain alt, and MAY contain any of the other specified attributes, including src
- IF the author chooses to supply a specific image file as the first frame (via <firstframe>) then the firstframe element MUST contain a non-empty string for @alt and MUST contain a non-empty URI for the image file:
<video> <firstframe src="path to image" alt="short textual description"> ... </video>
- IF the author chooses to allow the first frame of the actual video asset to render onscreen, and that first frame contains visual information beyond a "blank" screen, then the author MUST use firstframe to supply alt text to that imagery:
<video> <firstframe alt="short textual description"> ... </video>
- IF the author chooses to allow the first frame of the actual video asset to render onscreen, and that first frame contains no visual information (blank screen), then the author MAY omit the inclusion of firstframe to the video element.
An example of usage
The following code example illustrates a best-practices example showing how firstframe can work with current aria attributes to provide rich textual descriptions for both the actual video as well as any placeholder image.
<h2 id="sono"><span lang="IT">Io Sono L’amore</span> (I Am Love)</h2> <video aria-labeledby="sono" aria-describedby="synopsis"> <source src="video/sono_amore.webm"> <firstframe src="images/ratings/mpaa_pg.png" alt="This film is rated PG by the Motion Picture Association of America"> <track src="subtitles/[subtitle file]" srclang="EN" kind="subtitle"> <track src="captions/[caption file]" srclang="IT" kind="caption"> </video> <p id="synopsis">I Am Love (Italian: <span lang="IT">Io sono l'amore</span>) is a 2009 Italian film directed by Luca Guadagnino set at the turn of the millennium in Milan. The film follows a haute bourgeoisie family through changing times and fortunes, and its disruption by the force of passion.<p>
- Builds on existing precedent of child elements of <video>
- As an actual element, allows for additional attributes to be assigned to the textual alternative of the first frame imagery (author specified or default), including accessibility attributes (@alt, @longdesc*, aria-attributes, etc.) and internationalization requirements (@lang)
- The proposed new name of firstframe is an unambiguous representation of what is being provided - the first frame of the embedded video - and attributes assigned to the element are unambiguously related to the visual representation that is the first frame seen by sighted users.
- The removal of the @poster attribute will cause current experimental examples of <video> in the wild to break once user-agents implement this change.
- Slightly more verbose code.
Conformance Classes Changes
Changes for UA's and validators.
Unknown, presumed none.
- HTML5: Techniques for providing useful text alternatives http://dev.w3.org/html5/alt-techniques/
- Verbose (As Long As Necessary) Descriptor Requirements http://www.w3.org/WAI/PF/HTML/wiki/Verbose_desc_reqs
- DRAFT analysis of fallback mechanisms for embedded content http://lists.w3.org/Archives/Public/public-html-a11y/2010Nov/0246.html
- Fallbacks for embedded content http://lists.w3.org/Archives/Public/public-html-a11y/2010Nov/att-0246/embeddedcontent.html
- 4.8.6 The video element http://dev.w3.org/html5/spec/Overview.html#video
- 4.8.8 The source element http://dev.w3.org/html5/spec/Overview.html#the-source-element
- 4.8.9 The track element http://dev.w3.org/html5/spec/Overview.html#the-track-element
- 3.2.3 Global attributes http://dev.w3.org/html5/spec/Overview.html#global-attributes
- @alt attribute http://dev.w3.org/html5/spec/Overview.html#attr-img-alt