previous   next   contents  

7. The SMIL Media Object Module

Editor
Rob Lanphier (robla@real.com), RealNetworks

Table of contents

7.1 Introduction

This section defines the SMIL media object module. This module contains elements and attributes used to describe media objects. Since these elements and attributes are defined in a module, designers of other markup languages can reuse the SMIL media module when they need to include media objects into their language.

Changes with respect to the media object elements in SMIL 1.0 provide additional functionality that was brought up as Requirements of the Working Group, and those differences are explained in Appendix A.

7.2 Definitions

Continuous Media
Audio file, video file or other media for which has a measurable and well-understood duration associated with it. For example, a five second audio clip is continuous media, because it has a well-understood duration of five seconds. Opposite of "discrete media".
Discrete Media
Image file, text file or other media which has no obvious duration. For example, a JPEG image is generally considered discrete media, because there's nothing in the file indicating how long the JPEG should be displayed. Opposite of "continuous media".
Intrinsic Duration
The duration of a referenced item without any explicit timing markup. Continuous media has an intrinsic duration defined by the media, and discrete media has no intrinsic duration. (In SMIL, discrete media is assigned an intrinsic duration of zero).

7.3 Overview of SMIL Level 0 Media Objects

This module defines the baseline functionality of a SMIL player. It is very close in functionality to the media object specification in SMIL 1.0.

7.3.1 The ref, animation, audio, img, video, text and textstream elements

The media object elements allow the inclusion of media objects into a SMIL presentation. Media objects are included by reference (using a URI).

Anchors and links can be attached to visual media objects, i.e. media objects rendered on a visual abstract rendering surface.

When playing back a media object, the player must not derive the exact type of the media object from the name of the media object element. Instead, it must rely solely on other sources about the type, such as type information contained in the type attribute, or the type information communicated by a server or the operating system.

Authors, however, should make sure that the group into which of the media object falls (animation, audio, img, video, text or textstream) is reflected in the element name. This is in order to increase the readability of the SMIL document. When in doubt about the group of a media object, authors should use the generic "ref" element.

7.3.2 Element Attributes for All Media Objects

Languages implementing the SMIL Media Object Module must define which attributes may be attached to media object elements. In all languages implementing the SMIL Media Object Module, media object elements can have the following attributes:

abstract

A brief description of the content contained in the element. Unlike alt, this attribute is generally not displayed as alternate content to the media object. Need to define relationship between abstract and alt.

alt

For user agents that cannot display a particular media-object, this attribute specifies alternate text. alt may be displayed in addition to the media, or instead of media when the user has configured the user-agent to not display the given media type.

It  is strongly recommended that all media object elements have an "alt" attribute with a brief, meaningful description. Authoring tools should ensure that no element can be introduced into a SMIL document without this attribute.

If the content of these attributes is read by a screen reader, the presentation should be paused while the text is read out, and resumed afterwards. We may want to remove the text on pausing the presentation, unless and until we have a complete story on how to schedule the presentation of the alt text.

The value of this attribute is a CDATA text string.

author

The name of the author of the content contained in the element.

The value of this attribute is a CDATA text string.

clipBegin (clip-begin)

The clipBegin attribute specifies the beginning of a sub-clip of a continuous media object as offset from the start of the media object. This offset is a real-time measurement of normal media playback.
Values in the clipBegin attribute have the following syntax:

Clip-value-level-0 ::= [ Metric "=" ] ( Clock-val | Smpte-val )
Metric            ::= Smpte-type | "npt" 
Smpte-type        ::= "smpte" | "smpte-30-drop" | "smpte-25"
Smpte-val         ::= Hours ":" Minutes ":" Seconds 
                      [ ":" Frames [ "." Subframes ]]
Hours             ::= Digit+ 
                  /* see XML 1.0 for a definition of ´Digit´*/
Minutes           ::= Digit Digit
Seconds           ::= Digit Digit
Frames            ::= Digit Digit
Subframes         ::= Digit Digit

This version changes hours from "Digit Digit" to "Digit+". We may wish to confirm that this aligns with SMPTE Note: additional BNF for level 1 extensions defined later

The value of this attribute consists of a metric specifier, followed by a time value whose syntax and semantics depend on the metric specifier. The following formats are allowed:

SMPTE Timestamp
SMPTE time codes [SMPTE] can be used for frame-level access accuracy. The metric specifier can have the following values:
smpte
smpte-30-drop
These values indicate the use of the "SMPTE 30 drop" format with 29.97 frames per second. The "frames" field in the time value can assume the values 0 through 29. The difference between 30 and 29.97 frames per second is handled by dropping the first two frame indices (values 00 and 01) of every minute, except every tenth minute.
smpte-25
The "frames" field in the time specification can assume the values 0 through 24.

The time value has the format hours:minutes:seconds:frames.subframes. If the frame value is zero, it may be omitted. Subframes are measured in one-hundredth of a frame.
Examples:
clipBegin="smpte=10:12:33:20"

We may want to expand on this wording by defining the relationship between the different timing types and NTSC and PAL

Normal Play Time
Normal Play Time expresses time in terms of SMIL clock values. The metric specifier is "npt", and the syntax of the time value is identical to the syntax of SMIL clock values.
Examples:
clipBegin="npt=123.45s"
clipBegin="npt=12:05:35.3
"
Marker
Not defined in Level 0. See clipBegin Media Marker attribute extension in Level 1 extensions.

If no metric specifier is given, then a default of "npt=" is presumed.

When used in conjunction with the timing attributes from the SMIL Timing Module, this attribute is applied before any SMIL Timing Module attributes.

clipBegin may also be expressed as clip-begin for compatibility with SMIL 1.0. Software supporting the SMIL Boston Language Profile must be able to handle both clipBegin and clip-begin, whereas software supporting only the SMIL media object module only needs to support clipBegin. If an element contains both a clipBegin and a clip-begin attribute, then clipBegin takes precedence over clip-begin.

Example:

<audio src="radio.wav" clip-begin="5s" clipBegin="10s" />

The clip begins at second 10 of the audio, and not at second 5, since the clip-begin attribute is ignored. A strict SMIL 1.0 implementation will start the clip at second 5 of the audio, since the clipBegin attribute will not be recognized by that implementation. See Changes to SMIL 1.0 Media Object Attributes for more discussion on this topic.

clipEnd (clip-end)

The clipEnd attribute specifies the end of a sub-clip of a continuous media object as offset from the start of the media object. This offset is a real-time measurement of normal media playback. It uses the same attribute value syntax as the clipBegin attribute.
If the value of the clipEnd attribute exceeds the duration of the media object, the value is ignored, and the clip end is set equal to the effective end of the media object. clipEnd may also be expressed as clip-end for compatibility with SMIL 1.0. Software supporting the SMIL Boston Language Profile must be able to handle both clipEnd and clip-end, whereas software supporting only the SMIL media object module only needs to support clipEnd. If an element contains both a clipEnd and a clip-end attribute, then clipEnd takes precedence over clip-end. When used in conjunction with the timing attributes from the SMIL Timing Module, this attribute is applied before any SMIL Timing Module attributes.

See Changes to SMIL 1.0 Media Object Attributes for more discussion on this topic.

copyright

The copyright notice of the content contained in the element.

The value of this attribute is a CDATA text string.

longdesc

This attribute specifies a link (URI) to a long description of a media object. This description should supplement the short description provided using the alt attribute or the abstract attribute. When the media object has associated hyperlinked content, this attribute should provide information about the hyperlinked content.

If the content of these attributes is read by a screen reader, the presentation should be paused while the text is read out, and resumed afterwards.

We may want to remove the text on pausing the presentation, unless and until we have a complete story on how to schedule the presentation of the alt text.

src

The value of the src attribute is the URI of the media object.

title

(Definition in the structure module - to be removed ?) The title attribute offers advisory information about the element for which it is set. Values of the title attribute may be rendered by user agents in a variety of ways. For instance, visual browsers frequently display the title as a "tool tip" (a short message that appears when the pointing device pauses over an object). It is strongly recommended that all media object elements have a title attribute with a brief, meaningful description. Authoring tools should ensure that no element can be introduced into a SMIL document without this attribute. Browsers typically display "alt" text in tool tip. Why is title favored here?

type

(Definition in the layout module - to be removed ?) Content type of the media object referenced by the src attribute. This value takes precedence over other possible sources of the media type (for instance, the "Content-type" field in an HTTP or RTSP exchange, or the file extension). When the content represented by a URL is available in many data formats, implementations MAY use the type value to influence which of the multiple formats is used. For instance, on a server implementing HTTP content negotiation, the client may use the type attribute to order the preferences in the negotiation.

We need to think through if this is what we really want to say; current SMIL implementations probably do not implement this behavior, though it's hard to imagine current implementations rely on any other behavior.).

xml:lang

Used to identify the natural or formal language for the element. For a complete description, see section 2.12 Language Identification of [XML10].

xml:lang differs from the system-language test attribute in one important respect. xml:lang provides information about the content's language independent of what implementations do with the information, whereas system-language is a test attribute with specific associated behavior (see system-language in SMIL Content Control Module for details)

7.3.3 Element Content

Languages utilizing the SMIL Media Object Module must define the complete set of elements which may act as children of media object elements. There are currently no required children of a media object defined in Level 0 of the Media Object Module, but languages utilizing the media object module may impose requirements beyond this specification.

7.4 Overview of SMIL Level 1 Media Object Module

This section defines the elements and attributes that make up the SMIL Level 1 Media Object Module definition. Languages implementing elements and attributes found in Level 1 must implement all elements and attributes defined in Level 0.

7.4.1 Media object initialization: the param element

param elements specify a set of values that may be required by a media object at run-time. Any number of param elements may appear in the content of a media object element, in any order, but must be placed at the start of the content of the enclosing media object element.

The syntax of names and values is assumed to be understood by the object's implementation. This document does not specify how user agents should retrieve name/value pairs nor how they should interpret parameter names that appear twice.

param Attribute Definitions

name

(CDATA) This attribute defines the name of a run-time parameter, assumed to be known by the inserted object. Whether the property name is case-sensitive depends on the specific object implementation.

value

(CDATA) This attribute specifies the value of a run-time parameter specified by name. Property values have no meaning to SMIL; their meaning is determined by the object in question.

valuetype

["data"|"ref"|"object"] This attribute specifies the type of the value attribute. Possible values:

type

This attribute specifies the content type of the resource designated by the value attribute only in the case where valuetype is set to "ref". This attribute thus specifies for the user agent, the type of values that will be found at the URI designated by value. See 6.7 Content Type in [HTML40] for more information.

Example

To illustrate the use of param: suppose that we have a facial animation plug-in that is able to accept different moods and accessories associated with characters. These could be defined in the following way:
<ref src="http://www.example.com/herbert.face">
  <param name="mood" value="surly" valuetype="data">
  <param name="accessories" value="baseball-cap,nose-ring" valuetype="data">
</ref>
In the following example, run-time data for the object's "Init_values" parameter is specified as an external resource (a GIF file). The value of the valuetype attribute is thus set to "ref" and the value is a URI designating the resource.
<ref classid="http://www.example.com/gifapplication">
  <param name="Init_values"
         value="./images/elvis.gif">
         valuetype="ref">
</ref>

7.4.2 The brush element

The brush element is a lightweight media object element which allows an author to paint a solid color or other pattern in place of a media object. Thus, all attributes associated with media objects may also be applied to brush. Since all information about the media object is specified in the attributes of the element itself, the src attribute is ignored, and thus is not required.

brush specific attributes

color
The use and definition of this attribute are identical to the "background-color" property in the CSS2 specification, except that the brush element does not require support for "system colors".

Considering adding support for blends, and removing the caveat about system colors

7.4.3 Element Attributes for All Media Objects

In addition to the element attributes defined in level 0, media object elements can have the attributes and attribute extensions defined below. The inclusion or exclusion of these elements is left as an option in the language profile.

clipBegin Media Marker attribute extension

Level 1 implementations may support the media marker syntax for clipBegin defined below.

Clip-value-level-1 ::= Clip-value-level-0 |
                      "marker" "=" name-val 
name-val          ::= ([^<&"] | [^<&´])*
                  /* Derived from BNF rule [10] in [XML10] 
                     Whether single or double quotes are 
                     allowed in a name value depends on which
                     type of quotes is used to quote the 
                     clip attribute value */

Used to define a clip using named time points in a media object, rather than using clock values or SMPTE values. The metric specifier is "marker", and the marker value is a string.

Example: Assume that a recorded radio transmission consists of a sequence of songs, which are separated by announcements by a disk jockey. The audio format supports marked time points, and the begin of each song or announcement with number X is marked as songX or djX respectively. To extract the first song using the "marker" metric, the following audio media element can be used:

<audio clipBegin="marker=song1" clipEnd="marker=dj1" />

clipEnd Media Marker attribute extension

Level 1 implementations may support the media marker syntax for clipEnd. clipEnd media markers use the same attribute value syntax as the clipBegin media marker attribute extension.

erase

Controls the behavior of the media object after the effects of any timing are complete. For example, when SMIL Timing is applied to a media element, erase controls the display of the media when the active duration of the element and when the freeze period defined by the fill attribute is complete (see SMIL Timing and Synchronization). Possible values for erase are never and whenDone.

erase="whenDone" is the default value. When this is specified (or implied) the media removal occurs at the end of any applied timing.

erase="never" is defined to keep the last state of the media displayed until the display area is reused. Any profile that integrates this element must define what is meant by "display area" and further define the interaction. Intrinsic hyperlinks (e.g., Flash, HTML) and explicit hyperlinks (e.g., area, a) stay active as long as the hyperlink is displayed. If timing is reapplied to an element, the effect of the erase=never is cleared. For example, when an element is restarted according to the SMIL Timing and Synchronization module, the element is cleared immediately before it restarts.

Need to figure out how this works when multiple media objects are played simultaneously to a single region.

Example:

<par>
  <seq>
        <par>
            <img src="image1.jpg" region="foo1" fill="freeze" erase="never" .../>
            <audio src="audio1.au"/>	
        </par>
        <par>
            <img src="image2.jpg" region="foo2" fill="freeze" erase="never" .../>
            <audio src="audio2.au"/>	
        </par>
         ...
        <par>
            <img src="imageN.jpg" region="fooN" fill="freeze" erase="never" .../>
            <audio src="audioN.au"/>	
        </par>
  </seq>
</par>

In this example, each image is successively displayed and remains displayed until the end of the presentation.

readIndex

This attribute specifies the position of the current element in the order in which longdesc and alt text are read out by a screen reader for the current document. This value must be a number between 0 and 32767. User agents should ignore leading zeros. The default value is 0. We may want to consider removing the limitation of readIndex

Elements that contain alt or longdesc attributes are read by a screen reader according to the following rules:

stripRepeat

Strip the intrinsic repeat value of the underlying media object. For example, many animated GIFs intrinsically repeat indefinitely. The stripRepeat attribute allows an author to remove the intrinsic repeat behavior of an animated GIF on a per-reference basis, causing the animation to display only once, regardless of the repeat value embedded in the GIF. When stripRepeat is used in conjunction with SMIL Timing Module attributes, this attribute is applied first, so that the repeat behavior can then be controlled with the SMIL Timing Module attributes such as repeatCount and repeatDur. Values: true or false. Default: false. Since we are trying to get away from boolean, we are considering changing the values to "override" and "none".

7.5 Media Player Interface

This section is not spelled out in sufficient detail to be in the final specification. We will either work toward fleshing this out, or postpone this work until a future version of SMIL.

A media object referenced by a media object element is often rendered by software modules referred to as media players that are separate from the software module providing the synchronization between different media objects in a presentation (referred to as synchronization engine).

Media players generally support varying levels of control, depending on the constraints of the underlying renderer as well as media delivery, streaming etc. This specification defines 4 levels of support, allowing for increasingly tight integration, and broader functionality. The details of the interface will be presented in a separate document.

Level 0
Must allow the synchronization engine to query for duration, and must support cue, start and stop on the player. To support reasonable resynchronization, the media player must provide pause/unpause controls with minimal latency. This is the minimum level of support defined.
Level 1
In addition to all Level 0 support, the media player can detect when sync has been broken, so that a resynchronization event can be fired. A media player that cannot support Level 1 functionality is responsible to maintain proper synchronization in all circumstances, and has no remedy if it cannot (Level 1 support is recommended).
Level 2
In addition to all Level 1 support, the media player supports a tick() method for advancing the timeline in strict sync with the document timeline. This is generally appropriate to animation renderers that are not tightly bound to media delivery constraints.
Level 3
In addition to all Level 2 support, the media player also supports a query interface to provide information about its time-related capabilities. Capabilities include things like canRepeat, canPlayBackwards, canPlayVariable, canHold, etc. This is mostly for future extension of the timing functionality and for optimization of media playback/rendering.

7.6 SMIL Media Object Document Type Definition (DTD)

<!-- ======================================================================= -->
<!-- SMIL Media Objects Module  ============================================ -->
<!-- file: SMIL-media.mod

     This is Smil-Boston.
     Copyright 2000 W3C (MIT, INRIA, Keio), All Rights Reserved.

     Author:     Rob Lanphier, Jacco van Ossenbruggen
     Revision:   $Id: extended-media-object.html,v 1.4 2000/06/22 17:41:41 hugo Exp $

     This DTD module is identified by the PUBLIC and SYSTEM identifiers:

     PUBLIC "-//W3C//ELEMENTS SMIL-Boston Media Objects//EN"
     SYSTEM "SMIL-media.mod"

     ======================================================================= -->

<!-- ================== Profiling Entities ================================= -->

<!ENTITY % media-object.content "EMPTY">
<!ENTITY % media-object.attrib "">

<!-- ================== Media Objects Entities ============================= -->

<!ENTITY % mo-attributes-level-0 "
        abstract        CDATA   #IMPLIED
        alt             CDATA   #IMPLIED
        author          CDATA   #IMPLIED
        clipBegin       CDATA   #IMPLIED
        clipEnd         CDATA   #IMPLIED
        copyright       CDATA   #IMPLIED
        longdesc        CDATA   #IMPLIED
        src             CDATA   #IMPLIED
        type            CDATA   #IMPLIED
">

<!ENTITY % mo-attributes-level-0-deprecated "
        clip-begin      CDATA   #IMPLIED
        clip-end        CDATA   #IMPLIED
">

<!ENTITY % mo-attributes-level-1 "
        erase        CDATA           #IMPLIED
        readIndex    CDATA           #IMPLIED
        stripRepeat  CDATA           #IMPLIED
">

<!ENTITY % mo-attributes "
        %mo-attributes-level-0;
        %media-object.attrib;
">

<!ENTITY % brush-attributes "
        color        CDATA           #IMPLIED
">

<!--
     Most info is in the attributes, media objects are empty or
     has children defined at the language integration level:
-->

<!ENTITY % mo-content "%media-object.content;">

<!-- ================== Media Objects Elements ============================= -->
<!-- Level 0 -->
<!ENTITY % ref.qname        "ref">
<!ENTITY % audio.qname      "audio">
<!ENTITY % img.qname        "img">
<!ENTITY % video.qname      "video">
<!ENTITY % text.qname       "text">
<!ENTITY % textstream.qname "textstream">
<!ENTITY % animation.qname  "animation">

<!ELEMENT %ref.qname;           %mo-content;>
<!ELEMENT %audio.qname;         %mo-content;>
<!ELEMENT %img.qname;           %mo-content;>
<!ELEMENT %video.qname;         %mo-content;>
<!ELEMENT %text.qname;          %mo-content;>
<!ELEMENT %textstream.qname;    %mo-content;>
<!ELEMENT %animation.qname;     %mo-content;>

<!ATTLIST %ref.qname;           %mo-attributes;>
<!ATTLIST %audio.qname;         %mo-attributes;>
<!ATTLIST %video.qname;         %mo-attributes;>
<!ATTLIST %animation.qname;     %mo-attributes;>
<!ATTLIST %textstream.qname;    %mo-attributes;>
<!ATTLIST %text.qname;          %mo-attributes;>
<!ATTLIST %img.qname;           %mo-attributes;>

<!-- Level 1 -->
<!ENTITY % media-level-1 "IGNORE">
<![%media-level-1;[
  <!ENTITY % brush.qname "brush">
  <!ELEMENT %brush.qname;         %mo-content;>
  <!ATTLIST %brush.qname;         %mo-attributes; %brush-attributes;>

  <!-- ================ Param (Level 1) =================== -->
  <!ENTITY % param.qname "param">
  <!ELEMENT %param.qname; EMPTY>

  <!ATTLIST %param.qname;
    %Id.attrib;
    name        CDATA          #IMPLIED
    value       CDATA          #IMPLIED
    valuetype   (data|ref|object) "data"
    type        %ContentType;  #IMPLIED
  >
]]>

<!-- end of SMIL-media.mod -->

7.6.1 Appendix A: Changes to SMIL 1.0 Media Object Attributes

clipBegin, clipEnd, clip-begin, clip-end

With regards to the clipBegin/clip-begin and clipEnd/clip-end elements, SMIL Boston defines the following changes to the syntax defined in SMIL 1.0:

Handling of new clipBegin/clipEnd syntax in SMIL 1.0 software

Using attribute names with hyphens such as clip-begin and clip-end is problematic when using a scripting language and the DOM to manipulate these attributes. Therefore, this specification adds the attribute names clipBegin and clipEnd as an equivalent alternative to the SMIL 1.0 clip-begin and clip-end attributes. The attribute names with hyphens are deprecated.

Authors can use two approaches for writing SMIL Boston presentations that use the new clipping syntax and functionality ("marker", default metric) defined in this specification, but can still can be handled by SMIL 1.0 software. First, authors can use non-hyphenated versions of the new attributes that use the new functionality, and add SMIL 1.0 conformant clipping attributes later in the text.

Example:

<audio src="radio.wav" clipBegin="marker=song1" clipEnd="marker=moderator1" 
       clip-begin="0s" clip-end="3:50" />

SMIL 1.0 players implementing the recommended extensibility rules of SMIL 1.0 [SMIL10] will ignore the clip attributes using the new functionality, since they are not part of SMIL 1.0. SMIL Boston players, in contrast, will ignore the clip attributes using SMIL 1.0 syntax, since they occur later in the text.

The second approach is to use the following steps:

  1. Add a "system-required" test attribute to media object elements using the new functionality. The value of the "system-required" attribute must be the URI of this specification.
  2. Add an alternative version of the media object element that conforms to SMIL 1.0
  3. Include these two elements in a "switch" element

Example:

<switch>
  <audio src="radio.wav" clipBegin="marker=song1" clipEnd="marker=moderator1"  
   system-required=
     "extended-media-object" />
  <audio src="radio.wav" clip-begin="0s" clip-end="3:50" />
</switch>

alt, longdesc

Added the recommendation that if the content of these attributes is read by a screen reader, the presentation should be paused while the text is read out, and resumed afterwards.

New Accessibility Attributes

readIndex
stripRepeat
The stripRepeat attribute was added to provide better timing control over media with intrinsic repeat behavior (such as animated GIFs).

7.6.2 Appendix B: Element Content

SMIL 1.0 only allowed anchor as a child element of a media element. In addition to anchor (now defined in the Linking module), the param is now allowed as children of a SMIL media object. Additionally, other new children may also be defined by the host language.

7.6.3 Appendix C: New sections

The param element

A new section describing the "param" element provides a generalized mechanism to attach media-specific attributes to media objects.

Support for media player extensions

SMIL Boston introduces a media object interface, and the concepts of levels of media object interface implementation.


previous   next   contents