W3CNOTE-HTMLplusTIME-19980918

Timed Interactive Multimedia Extensions for HTML

(HTML+TIME)

Extending SMIL into the Web Browser.

This version:
http://www.w3.org/TR/1998/NOTE-HTMLplusTIME-19980918
Latest version:
http://www.w3.org/TR/NOTE-HTMLplusTIME
Previous version:
--
Authors:
Patrick Schmitz (Microsoft)
Jin Yu (Compaq/DEC)
Peter Santangeli (Macromedia)

We would especially like to thank Rob Martell (Digital Renaissance) for his review and contributions to this specification.

Additional contributors:
Pablo Fernicola (Microsoft)
Kevin Gallo (Microsoft)
Richard Gardner (Microsoft)
Ajay Jindal (Microsoft)
Dave Massy (Microsoft)
Thomas Reardon (Microsoft)
Mark VanAntwerp (Microsoft)
Michael Winser (Microsoft)

Status of This Document

This document is a submission to the World Wide Web Consortium (see Submission Request, W3C Staff Comment). It is intended for review and comment by W3C members.

This document is a NOTE made available by the W3 Consortium for discussion only. This indicates no endorsement of its content, nor that the Consortium has, is, or will be allocating any resources to the issues addressed by the NOTE.

Abstract

This document presents Timed Interactive Multimedia Extensions for HTML (HTML+TIME). This is a proposal for adding timing and synchronization support to HTML. HTML+TIME builds upon the SMIL recommendation to extend SMIL concepts into HTML and web browsers. The current version is a result of collaboration and review among Microsoft, Macromedia, Compaq/Digital and Digital Renaissance.  It is currently only a proposal and subject to change.  It assumed that the reader is familiar with the ideas expressed in the W3C Recommendation: SMIL [SMIL].


Contents

Overview

The W3C has recently approved SMIL as a recommendation.  SMIL introduces many valuable ideas, but has some limitations. In particular, SMIL is a data interchange format for media authoring tools and players - it does not include a means to apply the ideas to HTML and web browsers. This document describes a means of extending SMIL functionality into HTML and Web browsers. The proposal includes timing and interactivity extensions for HTML, as well as the addition of several new tags to support specific features described in the SMIL 1.0 spec.  HTML+TIME also adds some extensions to the timing and synchronization model, appropriate to the Web browser domain.

HTML+TIME is not intended to supplant SMIL.  The SMIL 1.0 standard supports many applications that need to describe multimedia presentations, independent of HTML and traditional web browsers.  HTML+TIME is closely aligned with SMIL to make it simple for authors and tools to use both specifications.  HTML+TIME also introduces a number of extensions to SMIL that  are required for a reasonable level of flexibility and control.  These extensions could easily be worked into the SMIL specification as well (indeed, some of the ideas have been discussed by the SYMM WG in the context of SMIL).

The layout capabilities described in the SMIL specification are subsumed by the CSS functionality standard in current browsers. Several other minor features are also standard in HTML, and are not duplicated here (but are documented in Appendix B).

Finally, an Object Model is described for HTML+TIME.  The SMIL 1.0 specification did not include this, but given the tradition of HTML and the DOM, we feel this is a critical aspect of the specification.

Timed Interactive Multimedia Extensions (TIME)

A set of extensions are described to add additional timing, interaction and media delivery capabilities to HTML.  These are modeled closely along the lines of SMIL, and attempt to reuse terminology wherever feasible.  The timing and interaction support augment current script support for timers and DHTML. 

Using the timing extensions, any HTML element can be set to appear at a given time, to last for a specified duration, and to repeat (i.e. loop). Simple timing is supported with a very simple syntax, but more complex timing constructs can also be described.   Interactive timing is supported. The first section of this document describes how the timing support is designed, and how script writers use the timing extensions.

In order to easily integrate time-based media (movies, audio and animation content), a set of new media tags are introduced (again, based upon the SMIL 1.0 specification), and the associated integration with the timing model is documented.

Additional tags are described to support fine-grained control of synchronization and media-loading behavior. Also, the notion of temporal hyperlinks presented in SMIL is generalized to apply to HTML in general.

Conditional Content control

SMIL introduces some very powerful elements that support conditional delivery of content, specifically to support differing client platform multimedia capabilities and preference settings. These are included with minor changes in this proposal as well.

Object Model

Finally, the object model for the time support is documented.  Included is a discussion of support for media extensions and for extension behaviors that will take advantage of the timing support.

Why Support Timing?

With the advent of CSS, the DOM and dynamic properties, it is possible for HTML to be a much more powerful medium.  Designers can now begin to think of the web page not just as a static page of information, but as a dynamic, interactive presentation.  Various tools exist to author animation based upon custom runtimes, and at least one (Macromedia Dreamweaver) supports animation based upon script and timers.  Finally, the W3C Recommendation: SMIL [SMIL] specifies a means of describing media-rich timelines in a separate XML-based file.

None of these solutions provide a simple, standard means for HTML authors to easily add timing and interaction relationships to arbitrary HTML elements, and to coordinate these with time-based media. HTML+TIME will fill this need. It defines a simple and powerful standard for time containment within the document.

Timing and Interaction Support

Timing support is based upon a simple model of adding attributes to HTML elements. HTML elements can be set to have a begin time and a duration. Additionally, elements can be made to repeat.   We describe this as "decorating" the HTML with additional attributes.    The important point is that HTML authors need not learn an entirely new syntax or document structure to add timing to pages.  They simply add attributes to the elements that they want to be dynamic.

For more complex scenarios, authors can group timing into timelines. These timelines can then be controlled and timed as well.  The structure is in some ways analogous to the DOM structure (cf. the <div> tag in particular), in that it defines a local time region.  Two means are provided of defining local timelines: a new tag, and an attribute that can be applied to HTML container elements.

Note that the timing support augments the behavior of the elements only with respect to the time during which the element is active or visible. The rules for applying the timing are fairly simple:

For time-based media, a media player is controlled to start and stop the media when appropriate.

Basic Syntax

See the discussion of current Syntax Issues.

All HTML elements that are legal within the BODY and that represent content or style, can support timing. Appendix A presents the list of HTML elements, and the classification of elements for the purposes of timing support.

HTML Tag Attributes

For all HTML elements that support timing, the following simple timing attributes are supported:

begin
This defines the offset (i.e. the delay) in seconds at which the element should be displayed.  By default (i.e. in the absence of any time base specifier or timeline directives), this is an offset from the time the page is displayed.
Legal values are signed clock values.
beginWith
Defines the time base to be the start of the referenced element.  This supports relative timing between elements.  The current element will begin at the same time as the referenced element (plus or minus any begin value).
Legal values are element IDs within the current time scope (see Usage notes below). 
No more than one of beginWith, beginAfter or beginEvent should be specified.
beginAfter
Defines the time base to be the end of the referenced element.  This supports relative sequential timing between elements.  The current element will begin when the referenced element ends (plus or minus any begin value). If the referenced element does not have a specified duration (i.e. is active indefinitely), the current element may never be active/displayed (although this is not considered a script error).
Legal values are element IDs within the current time scope (see Usage notes below)
No more than one of beginWith, beginAfter or beginEvent should be specified.
beginEvent
Defines the time base to be relative to the referenced event.  Events can be timing events (e.g. element.onBegin) as well as any other event supported by the DOM (e.g. element.onClick, document.onLoad). This supports interactive timing within a document, and removes the need to add script to "wire" elements together.   Interactive  in this sense includes:
user-interactive
where the element begins in response to events like onclick
server-interactive
where the element begins in response to an event that will be raised by some media player (e.g. an .avi file renderer that supports event streams).
presentation-interactive
where the element begins in response to an event that will be raised by the presentation engine itself.  An example would be an element that appears only when a resync event is generated; an author can choose to present some indication that some media elements are resynchronizing.

The current element will begin when the referenced event is raised (plus any begin value). If the referenced event is never raised, the current element may never be active/displayed. If a negative begin (delay) value is used with this attribute, the element will start when the event is raised, but will start the local timeline at an offset from 0. See also the section on negative offsets in  Usage notes below.
If the named event is "none", this element will simply wait to be turned on (e.g. by script).
No more than one of beginWith, beginAfter or beginEvent should be specified.
Legal values include:

ElementID "." EventName
Any defined event name, referenced by the associated element ID. E.g. "button1.onclick"
Where timing events are used (i.e. onBegin and onEnd), element must be within the current time scope (see Usage notes below)
"none"
The string "none".
dur
This defines the duration in seconds for which the element should remain active or displayed.   When this attribute is absent (and no end attribute is specified), the duration for all normal HTML elements is indefinite. See also the discussion of media elements, below.
Legal values are clock values greater than 0.
end
This defines the end time for the element. When this attribute is absent (and no duration attribute is specified), the duration for all normal HTML elements is indefinite. See also the discussion of media elements, below.
Legal values are signed clock values.
endWith
Defines the end time to be the end of the referenced element.  This supports end-point sync between elements.  The current element will end at the same time as the referenced element (plus or minus any end value).
Legal values are element IDs within the current time scope (see Usage notes below).
endEvent
Defines the end-time to be relative to the referenced event.  Events can be timing events (e.g. element.onBegin) as well as any other event supported by the DOM (e.g. element.onClick, document.onLoad). See also the notes on beginEvent.
The current element will end when the referenced event is raised (plus any end value). If the referenced event is never raised, the current element may remain active. If a negative end (delay) value is used with this attribute, it will be ignored.
Note that if a duration is defined (with the end or dur attributes), the duration will override the endEvent.  This allows authors to specify a maximum duration in addition to the interactive mechanism.
If the named event is "none", this element will simply wait to be turned off (e.g. by script). This is not required, and is more easily accomplished by specifying no end or duration for an element; if will default to an indefinite duration, and can be turned off by script.
Legal values include:
ElementID "." EventName
Any defined event name, referenced by the associated element ID. E.g. "button1.onclick"
Where timing events are used (i.e. onBegin and onEnd), element must be within the current time scope (see Usage notes below)
"none"
The string "none".
repeat
This causes the element to play repeatedly (loop) for the specified number of times.   Each repeat iteration lasts for the duration defined by the dur or end attributes. This attribute has no affect if the duration is not defined or is indefinite (see also media elements, below). A repeat count of less than 1 will cut short the specified simple duration.  A repeat count of 1 is a no-op.
Legal values are integer or fractional iterations, greater than 0.
At most one of repeat or repeatDur should be specified.
In addition, the value "indefinite" may be specified to indicate that the element should repeat indefinitely (forever).
repeatDur
This causes the element to play repeatedly (loop) for the specified duration (in seconds).  Each repeat iteration lasts for the duration defined by the dur or end attributes. This attribute has no affect if the duration is not defined or is indefinite   (see also media elements, below). A repeat duration less than the simple duration specified in dur will cut short the duration.  A repeat duration equal to the simple duration is a no-op.
Legal values are clock values greater than 0.
No more than one of repeat or repeatDur should be specified.
In addition, "indefinite" may be specified to indicate that the element should repeat indefinitely (forever).
skip-content
This attribute is introduced for future extensibility of HTML+TIME (and SMIL). It is interpreted in the following two cases:

If the value of the "skip-content" attribute is "true", and one of the cases above apply, the content of the element is ignored. If the value is "false", the content of the element is processed.
The default value for  "skip-content" is "true".
Reviewers - is this really necessary in the HTML/CSS context?

timeAction
This defines the attribute to be controlled over time as the element begins and ends on the local timeline. For visible elements, the appropriate values are "display" and "visibility".  For HTML style elements (e.g. B, STRONG), this attribute is ignored and the action will be to control the intrinsic behavior of the element.   An alternative to this is to control an inline style on e.g. a SPAN element, using the "timeAction=style".
Note that XML extensions should support a "visibility" property for elements that are essentially content, and an "on" property for style elements. In this manner, extensions can be easily integrated with HTML+TIME.
This attribute only applies to element tags.
Legal values are:
display
This causes the style.display property to be used to make elements appear and disappear on the timeline. It is useful for things like slide shows in which the document should reflow as elements come and go on the timeline.
visibility
This causes the style.visibility property to be used to make elements appear and disappear on the timeline. It is useful for some presentations in which the document layout is determined independent of the timeline, and elements just appear and disappear in the document over time, without causing reflow actions.
This is the default.
style
This causes the inline style attribute to be removed and applied to the element over time (i.e. when the local time is not active, the implementation will call element.removeAttribute() for the inline style attribute, and replace the inline style attribute when the local time is active). If there is no inline style attribute defined on the element, this does nothing. This allows for easier control over complex time-based styles.
onOff
This causes an element "on" property to be toggled between true and false over time. If there is no "on" property defined for the element, this does nothing.  The "on" property enables and disables the intrinsic behavior of the element.  This allows for generic extension styles to be supported over time (e.g. the style elements of VML [VML] support this model).

Clock value syntax

Clock values have the following syntax:

Signed-Clock-value ::= ("+" | "-" )? Clock-value  ; default is "+"
Clock-value        ::= HMS-value | Timecount-value
HMS-value          ::= (Hours ":")? Minutes ":" Seconds ("." Fraction)?
Timecount-value    ::= Timecount ("." Fraction)? 
                        ("h" |"min" |"s" |"ms")?  ; default is "s"
Hours              ::= DIGIT+
Minutes            ::= 2DIGIT  ; range from 00 to 59
Seconds            ::= 2DIGIT  ; range from 00 to 59
Fraction           ::= DIGIT+
Timecount          ::= DIGIT+
2DIGIT             ::= DIGIT DIGIT
DIGIT              ::= [0-9]

The minutes and seconds fields in an HMS-value are constrained to the range 00 to 59; leading zeros must be specified for terms between ":" and "." delimiters.  Hours can be any integer value, and need not have leading zeros. The fractional seconds in both HMS-values and Seconds-values can have arbitrary precision, but nothing greater than millisecond accuracy is guaranteed.

The following are examples of legal clock values:

Simple cases

We can add attributes to any object in HTML to add timing. By default, all timed elements are relative to a document root context, and so exist in a single time scope for the page. I.e. no nested timing is required/defined in the examples cited. Advanced timing support allows for more powerful constructs when the authors needs them. Nevertheless, scripters just beginning to use TIME do not have to understand anything about a timing structure or hierarchy to do simple things.   Also, all the offsets are in the same timespace (in the simple, default case), making it easy to align elements in time that are laid out all over the page.

Example: Making paragraphs appear over time:

...
<p t:begin="1">
    This is a paragraph of text that appears after one second
</p>
<p t:begin="2">
    This is a paragraph of text that appears after two seconds
</p>
<p t:begin="3">
    This is a paragraph of text that appears after three seconds
</p>
...

Next Step -  Using durations and relative timing

In order to support greater flexibility in terms of relative timing, we make use of the duration and time base parameters.

Note to reviewers:
Many multimedia runtimes support complete hierarchical relative timing, tied to a scene graph or some equivalent. This makes sense, as the containment/lexical hierarchy is explicit with a scene graph.  However, most HTML authors do not work with a user model of HTML that includes a containment/lexical hierarchy based upon the actual DOM.   Tying time containment to DOM containment would be confusing at best.  As such, we will support default timing as described above, plus explicit time containers and referential time bases  (i.e. timing relative to another named element). HTML authors should easily accept referential timing, as IDs are used very commonly in scripting.

Example: Defining relative timing among elements:

...
<p id="P1" t:dur="5">
    This is some text that appears immediately and remains
    visible for 5 seconds.
</p>

<img src="image.gif" t:beginWith="P1" t:begin="0.5">
<!-- This is an image that appears just after the first 
    paragraph appears (i.e. at 0.5 seconds) and remains
    visible indefinitely. Note the support for fractional seconds.
-->

<p t:beginAfter="P1" t:begin="-0.5">
    This is some text that appears just before the first 
    paragraph goes away (i.e. at 4.5 seconds) and remains 
    visible indefinitely. Note the support for negative offsets.
</p>
...

Making content repeat

Users can make any given element or timeline play repeatedly (i.e. loop), using the repeat attributes. Authors can specify either a number of times to play the simple duration, or a total time for which to repeat the element timeline.

...
<t:audio t:src="intro.mid" 
    t:repeat="indefinite">
<t:animation id="anim1" t:src="intro.x" 
    t:repeatDur="2min">
<t:animation t:src="http:wakeup.x" t:beginAfter="anim1"
    t:repeat="indefinite">
...

The audio is set to repeat indefinitely.  The intro animation will repeat for 2 minutes, and then stop. The second animation will then begin, and will repeat indefinitely.  Note that the repeat controls can also be combined with the sequence element and timelines, described below.

Using the timeAction attribute

The timeAction attribute provides a flexible means of defining what it means for an element to be active and not on a timeline.  For the purposes of control in HTML+TIME, all HTML elements are grouped into categories (see also Appendix A). By default, all elements categorized as content will be controlled with the visibility property.  That is, before the element is active on the timeline (before its defined begin time), the element visibility will be set to "hidden".   While the element is active on the timeline (from the defined begin until the defined end), the visibility property will be set to "visible".  Again, after the end time, the visibility property will be set to "hidden".  All style elements will be controlled by removing the effect of the element intrinsic behavior.   It would be nice if all style elements supported an "on" property to control easily and in a well-document manner.

Examples of default timeAction usage:

...
<span t:begin="10" t:dur="20">
    This is some text that appears after ten seconds and remains
    visible for 20 seconds.
</span> 
...
<b t:begin="1" t:dur="10">
    This is some text will appear normally at first, then
    be displayed bold for 10 seconds, and then revert to normal
    display again.
</b>
...
<v:oval t:begin="2" t:dur="10" t:timeAction="visibility">
...

The above example shows the default behavior for timeActions.  The span will be hidden when inactive.  The bold element will be visible, but not bold, when inactive.   The VML extension element for an oval will be hidden, as it supports a visibility property.

Example using display value:

...
<span t:begin="10" t:dur="20" t:timeAction="display">
    This is some text that appears after ten seconds, and remains
    visible for 20 seconds. When it becomes visible and again when
    it is hidden, the document will reflow.
</span> 
...

The display value is useful when the author wants the document to reflow over time.  This is useful, e.g. for image sequences, where only the active image should affect the layout of the document.

Example using style value:

...
<span style="text-decoration:line-through; color:red" 
    t:begin="10" t:dur="20" t:timeAction="style">
    This is some text will appear normally at first, then
    be displayed in red strikethrough for 10 seconds, 
    and then revert to normal display again.
</span> 
...

The style value makes it easier to control a complex set of styles over time.  Any style control that can be defined using the inline style attribute can be animated over time using this timeAction setting.

Example using onOff value:

...
<v:oval t:begin="2" t:end="10" t:timeAction="visibility">
   <v:fill t:begin="5" t:end="10" t:timeAction="onOff">
   ...
</v:oval> 
...

This example shows the use of the onOff value to control the intrinsic behavior of an extension style - in this case a fill element for an oval in VML. The oval will appear unfilled for 3 seconds and then the fill will be applied until the oval is hidden at 10 seconds.

Advanced Timeline support

The simple time attributes provide a very easy to use mechanism to add simple timing to a page.  A good deal of animation is supported just via this simple syntax.    Nevertheless, there will be cases in which an author wants to build up more complex timing structures, and to easily manipulate them.  HTML+TIME provides a   new par attribute to structure timed elements. This introduces a local, nested timeline that can be manipulated independent of the document (or parent) timing. The naming comes from SMIL, and is short for "parallel". There may be a potential conflict with the notion of "paragraphs".  If need be, the token can be renamed "parallel" or perhaps "timeline".

An additional <par> tag could also be introduced, but this is syntactic sugar for <span t:par='true'>. Reviewers?

An example use might be to set up a block of paragraphs with declared timing, which the author wishes to manipulate as an independent segment of animation (i.e. a relative timeline) within the document. The par attribute defines a relative timeline which can be manipulated as a unit, moved in time, looped, cut and pasted, etc.

Timeline Attribute Syntax

par
Defines a new relative timeline context with the scope of the HTML container.  For the purposes of the timing model, this makes the container element equivalent to a timeline element.
Timed elements within the container scope will be relative (perhaps indirectly in the case of beginWith/beginAfter usage) to this new timing context, excepting elements defined with beginEvent. The timeline container itself is relative to the document root time scope (or to an enclosing timeline container).
If a begin delay and/or duration is defined for the sequence container, the entire container will be affected (e.g. hidden) before the begin time and after the duration completes.
Boolean attribute - if present and TRUE, makes the container a timeline container.

An alternate syntax may used, which is equivalent in terms of the time behavior. This uses a new element to define a local timeline.  Reviewers: is this necessary? It more closely follows SMIL syntax, but does not fit into HTML as cleanly.

<t:par>
Defines a new relative timeline context.  All timed elements within the timeline scope will be relative (perhaps indirectly in the case of beginWith/beginAfter usage) to this new timing context. The timeline element itself is relative to the document root time scope (or to an enclosing timeline element).

Timeline Element Attributes

Timeline containers can be timed in the same manner as any other element.  An offset value and/or a time base will offset the entire local context, and shift the time of everything within the timeline scope (except elements timed to events out of the timeline scope).

begin
Defined above.
beginWith
Defined above.
The referenced element must not be within the scope of (i.e. a descendent of) the timeline. If such a circular reference is detected at runtime, the attribute will be ignored, and the default time base will be used instead.
beginAfter
Defined above.
The referenced element must not be within the scope of (i.e. a descendent of) the timeline. If such a circular reference is detected at runtime, the attribute will be ignored, and the default time base will be used instead.
beginEvent
Defined above.
dur
Defined above.
If a duration is applied to the timeline, the entire block will be hidden before the begin time and after the duration completes.
end
Defined above.
endWith
Defined above.
The referenced element must not be within the scope of (i.e. a descendent of) the timeline. If such a circular reference is detected at runtime, the attribute will be ignored, and the default time base will be used instead.
endSync
Defines the end of the timeline container to be a function of one or more of the contained time-children.
This is particularly useful when the contained elements include media that has finite but unknown duration (e.g. mpeg movies).
The end will be computed depending on the value of the attribute.
Legal values are:
first
The container should end with the first (earliest) end of a contained, timed element. I.e. as soon as any contained, timed element ends, the container also ends.
last
The container should end with the last (latest) end of a contained, timed element. I.e. after all contained, timed elements have ended, the container also ends.
[id-ref]
The container should end with the end of the referenced element. The referenced element must be contained within the current timeline container.
Where the description refers to "contained" elements, it should be understood that this refers to time containment, and not simply HTML containment.  A div element that is a timeline may contain other divs that are not timelines; all HTML descendents of a timeline container are considered to be time-children (i.e. contained by the timeline div), if there is no intervening timeline container in the hierarchy.  The timing hierarchy need not be tied directly to the HTML hierarchy.

The default value of "endSync" is "last".
endEvent
Defined above.
repeat
Defined above.
Note that when a repeat control is applied to a local timeline, it does not apply the repeat to each child element, but rather repeats the series of actions as a timeline. Child elements that are not timed should not be affected by the repeat action.
repeatDur
Defined above.
Note that when a repeat control is applied to a local timeline, it does not apply the repeat to each child element, but rather repeats the series of actions as a timeline. Child elements that are not timed should not be affected by the repeat action.
timeAction
Defined above.

Examples

<span t:par="true" id="TL1" t:dur="10">
    <!-- This begins right away, and lasts for 10 seconds -->
    <p>
        This is some text that appears immediately
    </p>
    
    <p t:begin="2">
        This is some text that appears after two seconds
    </p>

    <p t:begin="3">
        This is some text that appears after three seconds
    </p>
</span>
<div t:par="true" t:beginAfter="TL1" t:begin="0.2">
    <!-- This begins slightly after the first chunk is done,
        at 10.2 seconds
     -->
    <p>
        This is some much more exciting text that appears 
        right away on the new timeline, which does not begin 
        until the first big timeline is done. 
        It should be 10.2 seconds into the page display 
        before you see this...
    </p>

    <p t:begin="1">
        This is some more exciting text that appears one 
        second into the second timeline, which should be 
        11.2 seconds after the page Hides up. Just imagine
        doing a slide show with this stuff...
    </p>
</div>

Sequence support

The case sometimes arises that authors want to have a series of (e.g.) images appear. This can be accomplished with the time attributes described above, but a very simple declarative syntax is indicated to support this specific case. The <t:seq> tag is provided for this purpose.  Note that this element can be used for other cases as well as the simple sequence, but this is not recommended. A sequence is not a good general purpose means of declaring timing structure when the document is being edited - changing from a sequence declared with the <t:seq> tag to another timing relationship requires much more work than changing timing attributes associated with individual elements.

Note to reviewers:  Should this instead be presented as an attribute to containers, a la "timeline"? If the goal is to support novices, an explicit element may be easier for them to use.

Sequence Element Syntax

<t:seq>
This defines a new relative timeline context. All timed elements within the sequence block are timed by default as though they had the beginAfter attribute set to the timed lexical predecessor.  Any time base attributes (beginWith or beginAfter) will be ignored. All timed elements within the sequence scope will be (at least indirectly) relative to this new timing context. The sequence element itself is relative to the document time scope (or to an enclosing timeline element).
Note that as with beginAfter timing, if any timed element does not have a specified duration, all following (i.e. dependent) elements may never begin.
Note that the definition refers to "all timed elements".  A timed element is an HTML element that has any TIME attribute specified (as well as the TIME elements described in this document).  Elements with no timing attributes will be ignored by the timing mechanism, and will function in a traditional static manner.
If a begin delay and/or duration is defined for the sequence container, the entire container will be affected (e.g. hidden) before the begin time and after the duration completes.

Sequence Element Attributes

The sequence element can be timed in the same manner as any other element.  An offset value and/or a time base will offset the entire local context, and shift the time of everything within the local timeline scope.

begin
Defined above.
beginWith
Defined above.
The referenced element must not be within the scope of (i.e. a descendent of) the timeline. If such a circular reference is detected at runtime, the attribute will be ignored, and the default time base will be used instead.
beginAfter
Defined above.
The referenced element must not be within the scope of (i.e. a descendent of) the timeline. If such a circular reference is detected at runtime, the attribute will be ignored, and the default time base will be used instead.
beginEvent
Defined above.
dur
Defined above.
If a duration is applied to the sequence container, the entire container will be affected (e.g. hidden) before the begin time and after the duration completes.
end
Defined above.
endWith
Defined above.
The referenced element must not be within the scope of (i.e. a descendent of) the timeline. If such a circular reference is detected at runtime, the attribute will be ignored, and the default time base will be used instead.
endEvent
Defined above.
repeat
Defined above.
Note that when a repeat control is applied to a local timeline, it does not apply the repeat to each child element, but rather repeats the entire sequence as a timeline. Child elements that are not timed should not be affected by the repeat action.
repeatDur
Defined above.
Note that when a repeat control is applied to a local timeline, it does not apply the repeat to each child element, but rather repeats the entire sequence as a timeline. Child elements that are not timed should not be affected by the repeat action.
timeAction
Defined above.

Contained (child) tag Attributes

Children of a sequence element can take most of the time attributes, excepting the time base. Reviewers: it appears that it is legal in SMIL to allow children of a seq element to repeat, and potentially to have indefinite duration.  How much flexibility should we allow?  E.g. should we allow endEvent specification? In addition, it may make sense for the default syncBehavior for all sequence children to be locked, so that sequences hold together.

begin
Defined above.
This will offset the element from the time base, which is always the end of the timed lexical predecessor.
dur
Defined above.
If no duration is defined for any child of the sequence block, following children may not display (see also media elements, below).
end
Defined above.
endEvent
Defined above.
repeat
Defined above.
Note that when a repeat control is applied to a local timeline, it does not apply the repeat to each child element, but rather repeats the entire sequence as a timeline. Child elements that are not timed should not be affected by the repeat action.
repeatDur
Defined above.
Note that when a repeat control is applied to a local timeline, it does not apply the repeat to each child element, but rather repeats the entire sequence as a timeline. Child elements that are not timed should not be affected by the repeat action.
timeAction
Defined above.

Example: A slide show of images:

...
<div width="200" height="200">
  <t:seq t:repeatDur="indefinite">
    <!-- This will sequence the three images, repeating indefinitely.
        Any timebase parameters will be ignored, as the time base is 
        implicit via the sequence block. Offsets are legal. 
        Durations are recommended, as the default is to remain 
        visible indefinitely, which means that nothing after that 
        will ever show up.
    -->

    <img src="image1.gif" t:dur="2" t:timeAction="display">
    <img src="image2.gif" t:dur="2" t:timeAction="display">
    <img src="image3.gif" t:dur="2" t:timeAction="display">

  </t:seq>
</div>
...

Note that this ignores all aspects of layout.  It is up to the HTML author to describe the desired layout (e.g. laid out left to right, stacked with absolute positions, etc.).  With the timeAction attribute set to "display", the document will reflow over time.

Example: A sequence of styles:

...
<p>
  <t:seq t:repeatDur="indefinite">
    <!-- This will sequence the three styles, repeating indefinitely.  -->

    <FONT color="red"   t:dur="2">
    <FONT color="green" t:dur="2">
    <FONT color="blue"  t:dur="2">
    Here is some text that will get a really gaudy color treatment.
    </FONT>
    </FONT>
    </FONT>

  </t:seq>
</p>
...

Interaction support

Multimedia without interaction is just a movie. It must be possible for the author to describe interactive responses to user actions, and to define timing variants that support interaction.  In this timing model, interactive timing is just a variant in which the begin time is indeterminate.  An element (or an entire timeline container) that should begin in response to some user input is simply defined with a beginEvent.  When the element is not tied to a specific event (e.g. a particular button click or a stream trigger), but rather will be started by script on the page, the element can be defined with  '...beginEvent="none"...' .   Such an element can be started from script using a simple, familiar syntax.

Exposed action methods

In order to support interactive control of a timed element, the following methods are exposed to script:

beginElement()
Starts the element on the timeline.  This applies the same action as finite timing does when the element begin time is reached on the local timeline. When the beginElement() method is called, the element is attached to the local timeline at the current time, plus or minus any defined delay value.  Any time dependents will be notified, and aligned correctly to the local timeline.
This method takes no arguments.
endElement()
Stops the element on the timeline.  This applies the same action as finite timing does when the element stop time is reached on the local timeline. Any time dependents will be notified, and realigned correctly to the local timeline.
Note that a sequence of elements can thus be defined with no defined duration, that will sequence correctly if each is stopped interactively using the endElement() method.  This can be combined with the use of durations to create a slideshow that can be advanced with clicks, but that will advance itself after some specified duration.
This method takes no arguments.

Example script syntax

Authors can simply use the familiar script events (methods) like onclick(), onmouseover(), onmouseout(), etc. to define actions on timed HTML elements.  The script method implementation simply references the timed element by id, and then calls one of the action methods exposed by the element. Note that several possible script solutions are described, just for documentation.

Note that the final image is set to begin when the slideshow is complete.  It is possible to set up a timeline of actions that chain off an interactive begin.     The timing for dependent elements is computed when the head of the timing dependency chain is turned on with a trigger.

<div height=200 width=300>

    <t:seq id="SLIDESHOW" t:beginEvent="none">
        <!-- This begins when a trigger is sent.
             This will sequence the three images.
             If the user clicks on an image before the assigned
             duration, it will advance to the next image.
        -->
        <img src="image1.gif" t:dur="5" t:timeAction="display"
             onclick="this.endElement()">
        <img src="image2.gif" t:dur="5" t:timeAction="display"
             onclick="this.endElement()">
        <!-- This uses endEvent syntax. Note that 'dur'
             will override if there is no click by 5 seconds -->
        <img src="image3.gif" t:dur="5" t:timeAction="display"
             t:endEvent="onclick">
    </t:seq>
    <img src="showOver.gif" t:beginAfter="SLIDESHOW"
             t:timeAction="display">

    <p align=center onclick="SLIDESHOW.beginElement()">
        Click here to begin the slideshow.
    </p>
    <p align=center>
        If it advances too slowly for you, just click on an image 
        to advance it interactively.
    </p>

</div>

Support for temporal hyperlinks

SMIL introduces the notion of temporal hyperlinks. Rather than create a new element to handle this, we add capabilities to the timing model to support the same functionality.  The implementation must catch navigation events (i.e. docReady events as well as changes to the hash property), and then advance the root timeline to the start time of the hash element.  If the hash element itself is not timed, the element parents are traversed up to the document body to find a timed element.  If the hash element is not contained in any timed block, then the document timeline begins normally. If the hash element is set to begin interactively (with beginEvent="none"), the element is turned on as though it had been triggered and the document timeline plays normally.

Timed Hyperlink control Attribute Syntax

In some instances, authors will want to preclude jumping into the middle of a timeline.   One example would be an advertisement before a presentation;  the author may not want the end-user to be able to skip the ad.  A new attribute is supported on the body tag to control this behavior, allowing the author to enable or disable the support:

useTimedHyperlinks="true"
Sets the current page to make all hyperlinks into the page be timed, advancing the timeline to the start time of the href property.  If there is no href property specified on a link into the page, this attribute has no effect.
Setting this to FALSE disables timed hyperlink support into the page, making all links behave as they traditionally would.
This is only legal on the BODY tag.  It will be ignored anywhere else.
Default value is TRUE.

This nice thing about this approach is that it requires no change to the link tag, and can cleanly work even if the link is coming from a page that has no timing defined.

Controlling the start of the Document timeline

The timing syntax primarily addresses relationships among elements on a page.   However, there also a need to define, and to be able to control the start of overall document time. In the simple case, it will be acceptable to start document time when the document is loaded.  However, in particular for long HTML documents it may be unacceptable to defer the document timeline until the document has completely loaded.  Rendering of the first screenful is performed as soon as possible in most browsers, and authors will require that time can be started to run animations, etc. near the top of the page.

The model presented here provides simple controls for authors to control the start of document time. By default, document time begins when the document is fully loaded.  This covers many cases, and simplifies the model for novice authors.  Additional settings cause document time to begin either immediately (as soon as possible), or when the document is complete (the document and all associated media and objects have been loaded).  Finally, there is an advanced option for authors to specify the point in the document at which time should begin.

Need some examples of usage, and in particular some warnings and examples of trouble they can cause themselves by starting time before document.onLoad.

Add attribute to doc root (on body tag) to specify alternatives:  immediate (default), onDocLoad, onDocComplete, onStartTag.

Document timeline start  control Syntax

A new attribute is supported to specify the rule for when the document root timeline starts. This attribute is only legal on the body element.

Note that if document time begins before the document has fully loaded, the author must define all timing relationships such that the timing relationships are legal when document time begins, and as the rest of the document is parsed.  This means that all timing references to other timed elements (e.g. using beginWith and beginAfter) must refer to elements that are defined earlier in the document.  I.e. authors may not use forward references if document time begins before the document is loaded.

timeStartRule
Attribute on body element that defines the desired start time for the root timeline in the document.
Legal values are:

immediate
Document timeline should start immediately upon instantiation of the document root within the DOM (i.e. as soon as the underlying code gets going).
This is the default setting.

onDocLoad
Document timeline should start when the document is fully loaded, but without waiting for any document associated media, etc. This actually ties the start of the document timeline to the window.onload event.

onDocComplete
Document timeline should start only when the document is fully loaded, and all associated media have been loaded. This actually ties the start of the document timeline to the document.readyState changing to "complete".

onStartTag
(naming?)
Document timeline should start when a startDocumentTimeline element has been parsed and instantiated.  Allows an author to specify how much of a document should be loaded before the document timeline should start.

A new (XML) tag is supported to control when document time starts.

<t:startDocumentTimeline/>
Defines the point at which document should start.  Only used if the body tag has the  timeStartRule attribute set to onStartTag.

HTML+TIME Usage notes

Time Scope

For time attributes that reference another timed element (e.g. beginWith, beginAfter), the referenced element must be timed (i.e. it must specify one of the TIME attributes or be a TIME element) and it must be in the current timing scope.  That is, the referenced element must be within the HTML subtree defined by the closest parent time container of the current tag, and must have the same parent time container (i.e. it must be a a time-sibling).

In the absence of timeline attributed containers (or   timeline tags), this will be the document root and all references will be in the same scope.  If a timed element using a reference is within the scope of a timeline container, the scope is the local timeline block.   It is illegal to reference any timed element outside of this scope.  This constraint is imposed to preclude ambiguous and potentially confusing time dependency graphs.

Negative Offsets

The model described by HTML+TIME explicitly allows negative offsets.  The common use of these is in something like a sequence, where one object should appear just before another completes.  In this common case, the final computed begin time is still positive.   Nevertheless, there are situations in which a negative computed begin time can obtain; this is not considered illegal.

When the computed start time for an element is negative relative to the timeline container, the element is started with the parent (it can never appear or have influence before the time container does).  However, the sync relationship of the local timeline to the parent is offset: the local timeline for the element is defined to begin before it actually appears, and so it effectively begins somewhere in the middle of its timeline.  This can be useful in situations where an element is set to repeat, and the author wants the first repeat iteration to begin in the middle (repeating motion-paths, scrolling, etc. are sometimes authored this way).

Invalid timing definition

It is possible to describe timing relationships between or among elements that are invalid.  Typically, the timing is invalid because it creates circular time-dependency references.  For example, if two elements are defined to begin with one another (using either beginWith or beginEvent syntax), this is invalid.  Invalid timing can result through chained combinations of begin and end timing specification.   Any combination that produces a circular reference is illegal.

When an invalid timing specification is detected by the implementation, an error will be generated, and one or all of the elements involved will revert to default timing.

If both a duration and any end value (including clip-end) are specified for any element, the effective duration will be the minimum of the specified duration attribute and the computed duration for the specified end attribute.

Media element extension

In order to coordinate HTML and time-based media elements on a common timeline, HTML+TIME introduces new tags to easily integrate time-based media. The new media tags will simplify the declaration of time-based media elements over the traditional methods of declaring various plug-ins or other embedded player objects.  It will also manage the exchange of simple timing and control information between HTML+TIME timing implementation and the media players.

The tags are defined as XML extensions in the new TIME namespace.  They will be ignored by down-level browsers.

Note that the definition of time-based media is not restricted to simple media like audio and video.  Support is also intended for animation media, including extension players for existing popular animation formats.  HTML+TIME would not render these, but would simply coordinate the associated players in the HTML time context.

SMIL introduced a set of new tags for a variety of media types.  Currently, the individual tags have no semantic significance over the catch-all <ref> tag - the semantics are really tied to the MIME type from the server or specified as an attribute.   However, the use of individual media type tags allows for future extensions such as specific attributes appropriate to individual media types (e.g. audio level).  We would prefer to use "media" in place of "ref", but we can adopt the SMIL naming if desired.

Associating media with players

The media tags take a src attribute to specify the source-media URL, and an optional specifier for the MIME/media type (the MIME type provided by the server is used by default).  The implementation will associate the type with an appropriate player, and manage the instantiation of a player.  Implementations may handle this e.g. by injecting HTML for an EMBED tag, or it may be handled more deeply. 

There is also a player attribute that supports a reference to EMBED or OBJECT element from the media tags. This allows authors to use the traditional means of declaring the player, in particular to support all the specific attributes and controls of the respective players. The referenced element must support the media player interface described in the object model below. This also supports integration of third-party media players with the HTML+TIME model.

Note that the declaration of individual media elements with associated time syntax does not preclude the implementation from associating a group of media elements with a single player instance.  It may be desirable from a performance standpoint to combine the media elements into a single grouped-element (e.g. creating a temporary playlist for the associated player).  However, this can have other drawbacks, including a loss of interactive control over individual elements.  The noCombine attribute provides author control over this.

HTML+TIME control of media players

Media players must support a basic set of controls to integrate with HTML+TIME.   The implementation or the media tags must present an interface to the media players (e.g. with a wrapper).  The details of the control are provided in the object model discussion  below.

Note that if no dur attribute is specified, the media wrapper will set the duration property of the element (it is legal to set the duration to indefinite). This will have the side-effect that end-time-dependents (other elements defined with beginAfter referencing this node) will have a defined begin time.

Media with event streams

Need description of integration of stream-based events with TIME model.  Should require minimal Object Model support, as events fit in like all other events.  Need to discuss recommended support.  Reviewers?

Stream-based events should be raised by the player object, and can be referenced as "object_id.event_name".  When the player object is implicit (e.g. using a video tag), the associated media element should raise the events.  This allows script and HTML+TIME event specifications to respond to all media and server generated events.

Media Element Syntax

The tags described here parallel the tags defined in SMIL 1.0 [SMIL].    The only difference is the media tag which replaces the generic ref tag in SMIL 1.0, and the omission of the SMIL text tag, which is subsumed by simple HTML tags for text. Reviewers: what was the intended use of ref, and the naming used?   Unlike SMIL, there is no need for the abstract region reference.  There is a need for reference to a player object, but this is supported as an attribute on all types.

<t:animation>
Defines an animation element to be incorporated into the document layout and timing.   This may need to be further refined to support specific animation types, allowing player-specific attributes.  For this version, the more generic media element can be used, and the attributes can be specified on the player object.
<t:audio>
Defines an audio element to be incorporated into the document timing. Depending on the implementation, the position of the audio element may be used to control pan/balance, etc.
<t:img>
Duplicates the HTML img tag, supporting more flexible declaration syntax, and some control over media loading.  The syntax supports containment of additional XML tags, for things like motion behaviors.  The implementation can also control the loading of the associated media based upon the defined start time of the element.
<t:media>
Defines a generic time-based media element to be incorporated into the document layout and timing.
<t:video>
Defines a video element to be incorporated into the document layout and timing. 
<t:textstream>
Defines a time-based text element to be incorporated into the document layout and timing.  Time based text is not well specified, but includes streaming captions, etc.

Media Element Attributes

The media element takes the base HTML attributes appropriate to any div tag.  The following are interpreted by the wrapper:

src
Defines the source URL of the media.  The extension defines the MIME type.
Support is planned for popular video and audio file types, including streaming variants.
Value must be a legal URL.
img
Defines the URL of an alternate image to be displayed if the MIME type is not supported.
This attribute is optional, and only makes sense for visual MIME types.  It is not legal for the img element.
Value must be a legal URL.
abstract
A brief description of the content contained in the element.
Value is a string.
author
The name of the author of the content contained in the element.
Value is a string.
begin
Defined above.
copyright
The copyright notice of the content contained in the element.
Value is a string.
beginWith
Defined above.
beginAfter
Defined above.
dur
Defined above.
If specified, this will override the natural media duration.
If not specified, the duration will be set from the media player, if it is defined.
end
Defined above.
endWith
Defined above.
endEvent
Defined above.
longdesc
This attribute specifies a link (URI) to a long description of a media object. This description should supplement the short description provided using the alt attribute. When the media-object has associated anchors, this attribute should provide information about the anchor's contents.
noCombine
If set to true, directs the implementation not to combine this media element into a bundled media stream.
Value is a boolean.  Default is false. 
player
Directs the implementation to use the referenced object to render the media for this element.  The object must support the media player interface described in the object model section below.
Value is an id, referring to an object element.
repeat
Defined above.
The wrapper will use the media player to make the media repeat, if the media player supports this.
Not all media will support this, and an error may be generated (but is not guaranteed).
repeatDur
Defined above.
The wrapper will use the media player to make the media repeat, if the media player supports this.
Not all media will support this, and an error may be generated (but is not guaranteed).
clip-begin
This is aligned closely to the SMIL specification, with minor changes.
Note: the name used in SMIL is non-standard for HTML/DOM identifiers.  The SMIL name is used here, but it should perhaps be changed to not use the hyphen.
The clip-begin attribute specifies the beginning of a sub-clip of a continuous media object as offset from the start of the media object.
Values in the clip-begin attribute have the following syntax:
Clip-time-value    ::= [Metric "="]( Clock-val | Index-val | Smpte-val 
                                     | timeID-val )
Metric             ::= Smpte-type | "clock" | "index"
Smpte-type         ::= "smpte" | "smpte-30-drop" | "smpte-25"
Smpte-val          ::= Hours ":" Minutes ":" Seconds 
                      [ ":" Frames [ "." Subframes ]]
Hours              ::= DIGIT+
Minutes            ::= 2DIGIT  ; range from 00 to 59
Seconds            ::= 2DIGIT  ; range from 00 to 59
Fraction           ::= DIGIT+
Frames             ::= 2DIGIT
Subframes          ::= 2DIGIT
Index-val          ::= DIGIT+
timeID-val         ::= [legal HTML id]
2DIGIT             ::= DIGIT DIGIT
DIGIT              ::= [0-9]

The value of this attribute consists of an optional metric specifier (defaults to "clock"), followed by a time value whose syntax and semantics depend on the metric specifier. The following formats are allowed:

SMPTE Timestamp
SMPTE time codes [SMPTE] can be used for frame-level access accuracy. The metric specifier can have the following values:
smpte
smpte-30-drop
These values indicate the use of the "SMPTE 30 drop" format with 29.97 frames per second. The "frames" field in the time value can assume the values 0 through 29. The difference between 30 and 29.97 frames per second is handled by dropping the first two frame indices (values 00 and 01) of every minute, except every tenth minute.
smpte-25
The "frames" field in the time specification can assume the values 0 through 24.

The time value has the format hours:minutes:seconds:frames.subframes. If the frame value is zero, it may be omitted. Subframes are measured in one-hundredth of a frame.
Unlike the strictest SMPTE format, TIME will allow for more than two digits in the hours field.  This is specifically to accommodate long format material.
Examples:
clip-begin="smpte=10:12:33:20"
clip-begin="smpte=102:12:33"

Clock Time
Clock Time (what SMIL calls "Normal Play Time") expresses time in terms of clock values. The metric specifier is "clock", but this is also the default metric, and so the metric specifier may be omitted. The Clock-value syntax is described above.
Examples:
clip-begin="123.45"
clip-begin="12:05:35.3
"

Index value
Index values (not expressed in SMIL) express a sub-clip start or end not in terms of time, but in terms of an index (e.g. into a character stream). They may only be used on naturally indexed media like textstreams. The metric specifier is "index". The argument is simply a positive integer. No sign is allowed on an Index value. If the clip-begin is specified as an Index-value, any clip-end must be as well.  Given the play rate of the textstream, a duration can be computed (the wrapper for the textstream renderer must support query of the duration for a given clip-begin and end).
This is may not seem a natural way of marking a sub-clip, but it is the only reasonable means of marking a character stream.  It is in some ways analogous to the frame specifier for Smpte-times, in that a "frame" in a textstream is a single character, and the frame rate is defined by the textstream itself.
SMIL does not define what a textstream is, and it may not be desirable to constrain this definition to match the current implementation within, e.g. RealNetworks' players.  If the definition of textstream is resolved to include specifics of time and/or logical named markers, this attributed may be deleted in favor of the time and timeID values.
Examples:
clip-begin="142"

timeID value
timeID values are simply named points in a media stream.  These are only supported by some players, and so are specific to certain MIME types (e.g. newer ASF). The strings are passed into the player, which can map the names to specific points defined as part of the authored media stream.
Examples:
clip-begin="Bill talks about NT"
clip-end="Bill talks about simplicity"
clip-end
The clip-end attribute specifies the end of a sub-clip of a continuous media object (such as audio, video or another presentation) that should be played. It uses the same attribute value syntax as the clip-begin attribute.
If the value of the "clip-end" attribute exceeds the duration of the media object, the value is ignored, and the clip end is set equal to the effective end of the media object.
Note that use of clip-end will define a duration. This may override the natural duration of the media itself.
In SMIL, no mention is made of how to handle a negative duration (i.e. clip-end < clip-begin).
The media tag will ignore a clip-end if it defines a negative or 0-length duration.
If a clip-end is specified without a clip-begin, the clip-begin should be computed from the defined duration.  If no duration is otherwise defined, the clip-end will be ignored.
If the clip-end defines a duration (with a clip-begin) and any other duration is defined, the minimum duration is used (see also Usage notes below).
Note: the name used in SMIL is non-standard for HTML/DOM identifiers.  The SMIL name is used here, but it should perhaps be changed to not use the hyphen.
title
This attribute offers advisory information about the element for which it is set. Values of the title attribute may be rendered by user agents in a variety of ways. For instance, visual browsers frequently display the title as a "tool tip" (a short message that appears when the pointing device pauses over an object).
Value is a string.
type
Optional MIME/media type of the media object referenced by the "src" attribute.
The default mechanism should be to get the MIME type from the server.
If this fails, the type attribute can specify the MIME type.
If this is missing, and if the src URL includes an extension, an attempt should be made to determine the type from the URL extension, if such a mapping is defined on the platform.
Value is a string.

Open issues

SMIL references a textstream media element, but does not in any way define or describe it.  Simple cases could be handled with HTML using timing markup. Nevertheless, having a separate media type has advantages when streaming a large amount of content - e.g. captions to a lengthy audio presentation. 

The specification of a "textstream" must be formalized before it can be widely supported by media players.

Media Loading and Synchronization

To integrate timeline media and animation into the page, authors must have control over the synchronization behavior of the page.  In addition, authors need to be able to define how time behaves relative to the initial loading or cueing of media.   HTML+TIME unifies support for asynchronous media loading and dynamic resynchronization of players into a simple concept of sync rules and scope. HTML+TIME defines additional attributes for media elements, as well as a general mechanism for managing dynamic synchronization of timeline elements.

This level of support requires a means to specify which media elements must be ready (i.e. loaded or cued) in order to play a portion of an animation timeline.  This allows the author to control when the page or any local timeline (e.g. a div) starts, relative to the media that is required.  The author can force the page to wait for all media to be ready.  Alternatively, the author can specify that a timeline (e.g. the main page timeline) can begin when certain specific media elements are ready, but before all media has been prepared.  The author can define and control the end-user experience.

Dynamic synchronization support provides an author the means to define which elements must remain in tight synchronization, and which elements (or local timelines) can slip if the players cannot keep up (e.g. due to network congestion).  This provides a balance between the requirements of coordinating an animation and the realities of network media delivery.

Functional model in HTML+TIME

The author's model will be very simple: they can describe the synchronization behavior of each media element and of each time container. By definition, synchronization includes the startup sync relationship; allowing an element to slip sync provides control over media loading. By describing the sync behavior of time containers, authors can control the scope of a synchronization context.  When the timing model must handle resync events (i.e. when a media player falls out of sync while playing), the sync rules of the time containers define how far-reaching the resync handling will be.

In addition, authors can describe how to fill in for elements that are not ready to play at the time they were originally authored to.

Synchronization Attribute syntax:

syncBehavior
Attribute on Media elements and time containers that defines the sync requirements for the associated timeline.
Legal values are:

canSlip
Allows the associated node to slip on the parent timeline. Setting the attribute to this value means that if the media is not ready when the timeline should begin, this node can pend (slip on the parent timeline) until the media is ready, and begin at that point. For a time container, "media" refers to the media for all contained elements. Setting the attribute to this value also means that if the player slips sync while playing, it is okay to let this element redefine the sync relationship to the parent timeline. For time containers, it means that if any element within the contained time scope falls out of sync that the time container can redefine sync to the parent timeline (this effectively defines the scope of the resync propagation).
This will probably be the default setting for all nodes. This lets a page have very loose sync and allows the most freedom to handle network problems, etc.

locked
Forces the associated node to maintain sync on the parent timeline. This can be eased with the use of the syncTolerance attribute. Setting the attribute to this value means that if the media is not ready when the timeline should start, the parent time container must pend (not start) until the media is ready. For a time container, "media" refers to the media for all contained elements. Setting the attribute to this value also means that if the player slips sync while playing, the element cannot slip against the parent timeline.  The parent must resolve the slippage to maintain sync on the timeline. This does not force a particular approach to maintaining sync. Suggested means include:
  • Pausing the parent timeline until the element that slipped can "catch up" with the parent timeline.
  • Pausing the element that slipped (only if it somehow got ahead of the parent timeline) until the parent timeline catches up.
  • Recueing (i.e. resetting the current position of) the element that slipped, jumping it ahead so that it "catches up" with the parent timeline.

For a time container, syncBehavior="locked" means that the local timeline must remain locked to the parent timeline. If elements within the time container are defined with syncBehavior="canSlip", the time container setting does not overrule the contained element setting. This only defines the sync relationship of the time container to the parent.
Note that the most common approach to resolving sync problems will be to pause the parent timeline.  If however the parent is also defined with locked sync, the resync must be propagated up the time tree until a parent is reached that has slip sync defined, or until the document body is reached.  In the case of a fully locked timing definition, the entire page timeline will be paused if any element falls out of sync and raises a resync event.


syncTolerance
Attribute on Media elements and time containers that defines the sync tolerance for the associated timeline. This is only used if the same element has syncBehavior=locked. This allows a locked sync relationship to ignore a little slew without forcing a resync event.
Legal values are seconds values as defined in the HTML+TIME spec (i.e. an integer or floating point count of seconds). There may be a non-zero default value for this attribute, to be determined through testing (I would guess at something between 0.1 seconds and 0.5 seconds). This should be documented in the object model.

fill
Attribute on any element that defines the behavior when the duration must be extended to account for a time-dependent that is not ready to start. This allows an element to play longer than the defined duration to "cover for" an element that is slipping on the timeline until the associated media is ready to play.
Legal values are:

extend
Causes the element to continue to play by extending the duration. This is appropriate for static elements, and time containers that have static and/or repeating elements within the timeline. If the element itself is set to repeat, this will have the same effect as the repeat value, below.

repeat
Causes the element to continue to play by repeating the duration. This is appropriate for elements that have a proper timeline to loop. If the element is already set to repeat, the current repeating behavior will be continued (e.g. a partial repeat iteration at the end will be completed, and not restarted).

freeze
Causes the element to continue to play by freezing (i.e. holding) the local timeline at the ending time. This is appropriate for static elements, and for elements which can neither loop nor continue to play.

remove
Causes the element to complete normally and not extend the duration at all.
This is the default.

Note that the behavior is only used when a direct exit-time dependent is not ready to play. If a dependent of a dependent is not ready, or if a dependent is defined relative to the start of the current element, no fill behavior is used.

Note that when the element has multiple time dependents, the fill behavior will be used if any one of the dependents cannot begin on-time (on-sync). There is no way to define the fill behavior for individual time-dependents. An example of a potential problem is an image that shows for a few seconds and is followed by a video and some audio. Assume the image is set to fill with the intention of covering for the video until it is ready. If both video and audio are defined to begin when the image ends (beginAfter="image_id"), and if the audio is not ready to start, the image will continue to show until the audio is ready. In this case, the author should define the audio as starting with the video, and make only the video directly dependent on the image.

Default settings Attribute syntax:

syncBehaviorDefault
Sets the default sync behavior for all elements on the page. This may only be applied to the body element.
Legal values are the same as for the syncBehavior attribute.
This could also be handled by making the syncBehavior inherit from the parent (if not specified).  Setting the attribute on the document body then has the same effect.
 
syncToleranceDefault
Sets the default sync tolerance value for all elements on the page. Only used if the syncBehavior for an element is "locked" (either by the default setting or by attribute on the element). This may only be applied to the body element.
Legal values are the same as for the syncTolerance attribute.
This could also be handled by making the syncBehavior inherit from the parent (if not specified).  Setting the attribute on the document body then has the same effect.

Example use cases:

Need examples of use of fill behavior.

By default, everything will have loose sync. This lets authors ignore the issues with maintaining sync among all the elements. This allows all the players the most leeway as well.

Example: syncing audio and video together, independent of the rest of the page:

    ...
    <span t:par="true" t:syncBehavior="canSlip">
    <t:media src="somevideo.avi" t:syncBehavior="locked">
    <t:media src="someaudio.wav" t:syncBehavior="locked">
    </span>
    ...

The sync of the video and audio is locked, meaning that the local timeline cannot start until the media for both elements is ready to roll. Also, if either player has problems during playback, the parent container must maintain sync.

The timeline sync is defined to slip, which means that the rest of the document timeline will not be held up when the video and audio media is loading/cueing. It also means that any resync event required by one of the video or audio players losing sync, will force the timeline to resolve the sync, but not to propagate the resync event to the parent timeline (which may be the document root timeline).

Example: streaming video relative to an animated page, and holding sync loosely:

    ...
    <body t:syncBehaviorDefault="locked" t:syncToleranceDefault="2.0">
    ...
       <t:media src="somevideo.asf" t:begin="5">
    ...

The video is set to begin playing 5 seconds after the document timeline begins, and to hold sync +/- 2 seconds to the document timeline. This allows other animations on the page to stay in sync with the video. If the video falls out of sync by more than 2 seconds, the time implementation (at the document level) must resync the video (e.g. by pausing the root timeline or the video timeline until they are more closely aligned).

Use of sync behavior and interactive timing

If an element is defined to start interactively (e.g. specifying beginEvent), the syncBehavior has a slightly modified interpretation. In this case, there is no original sync relationship defined between the element and the time container. As such, when the element is started (e.g. via script on a button), the sync relationship will be propagated to the time dependents, but no resync event is propagated to the time container. However, once the element has been started and a sync relationship has been established, the syncBehavior of the element will determine how the object will maintain sync. Thus an element can be defined to start interactively (with indeterminate sync) but to maintain sync once started. This is probably not a common authoring scenario.

Media player issues

It is assumed that media players and extensions provide at the basic level of control described below as the minimum for integration.

Integrating HTML+TIME and Television

An emerging application of HTML combines the web browser with television, either as a traditional broadcast or in digital forms like DVD (e.g. see the ATVEF spec [ATVEF].  HTML+TIME is an ideal tool for these applications, providing a means of controlling time and synchronization in a web page that accompanies the television content.  Several tools are provided for integrating HTML+TIME with applications like ATVEF:

Attribute syntax:

clockSource
Boolean attribute on media elements and time containers that forces the page clock to sync to the local timeline.
The default value is false.
The associated property is read-only, and cannot be set by script.
If set to true, the page clock will lock to the local timeline when it is active (even if it is paused).  When the local timeline is inactive (i.e. before it begins and after it ends), the page clock will not be affected.
If more than one element has the clockSource attribute set to true, and the elements are both active at any point in time, the element that appears later in the document will dominate.
 

Media/Content delivery control extensions

SMIL introduces the notion of conditional attributes, and the switch construct built upon the conditional attributes.  These are valuable constructs for many applications, and are included for support with HTML in web browsers.

The syntax described below is taken largely from the SMIL specification.  Minor extensions to SMIL are described to generalize the support for HTML and the browser environment.

Test Attributes

Test attributes provide a means of enabling or disabling an element based upon some predefined system parameters.  If the expression testing a particular parameter evaluates true, the element is rendered normally.  If the expression evaluates false, then the element is ignored (e.g. removed from the DOM tree), and will not be rendered.

In the context of SMIL,  the use cases describe conditional delivery of various content forms, based upon built-in conditionals related to network bandwidth, screen size and depth, system language and various other user preferences.  In the context of a some dedicated media players, the user preferences are associated with the SMIL renderer.  Certain attributes can be directly mapped to system settings or reasonable defaults on most platforms (e.g. language). Associating some user preferences (e.g. for typical speed of the network connection) is less direct for browsers.

The recommended solution is to support typical defaults for all the parameters, and allow specific preferences in the environment.  The preferences should be made available in the DOM (e.g. as attributes of the window or document).  Browser installers could set these values, and/or a simple form page or equivalent would allow the user to set the test-attribute values.

The attributes supported by SMIL include (taken from the SMIL spec, with the descriptions it provides, and additional notes in italics):

system-bitrate
This attribute specifies the approximate bandwidth, in bits per second available to the system. The measurement of bandwidth is application specific, meaning that applications may use sophisticated measurement of end-to-end connectivity, or a simple static setting controlled by the user. In the latter case, this could for instance be used to make a choice based on the users connection to the network. Typical values for modem users would be 14400, 28800, 56000 bit/s etc. Evaluates to "true" if the available system bitrate is equal to or greater than the given value. Evaluates to "false" if the available system bitrate is less than the given value.
The attribute can assume any integer value greater than 0. If the value exceeds an implementation-defined maximum bandwidth value, the attribute always evaluates to "false".
Implementors are recommended to support this with a static configuration value, rather than some dynamic measurement.
system-captions
This attribute allows authors to distinguish between a redundant text equivalent of the audio portion of the presentation (intended for a audiences such as those with hearing disabilities or those learning to read who want or need this information) and text intended for a wide audience. The attribute can has the value "on" if the user has indicated a desire to see closed-captioning information, and it has the value "off" if the user has indicated that they don't wish to see such information. Evaluates to "true" if the value is "on", and evaluates to "false" if the value is "off".
system-language
The attribute value is a comma-separated list of language names as defined in [RFC1766].

Evaluates to "true" if one of the languages indicated by user preferences exactly equals one of the languages given in the value of this parameter, or if one of the languages indicated by

user preferences exactly equals a prefix of one of the languages given in the value of this parameter such that the first tag character following the prefix is "-".

Evaluates to "false" otherwise.

Note: This use of a prefix matching rule does not imply that language tags are assigned to languages in such a way that it is always true that if a user understands a language with a certain tag, then this user will also understand all languages with tags for which this tag is a prefix.

The prefix rule simply allows the use of prefix tags if this is the case.

Implementation note: When making the choice of linguistic preference available to the user, implementers should take into account the fact that users are not familiar with the details of language matching as described above, and should provide appropriate guidance. As an example, users may assume that on selecting "en-gb", they will be served any kind of English document if British English is not available. The user interface for setting user preferences should guide the user to add "en" to get the best matching behavior.

Multiple languages MAY be listed for content that is intended for multiple audiences. For example, a rendition of the "Treaty of Waitangi", presented simultaneously in the original Maori and English versions, would call for:

            <t:audio t:src="foo.rm" t:system-language="mi, en"/>

However, just because multiple languages are present within the object on which the system-language test attribute is placed, this does not mean that it is intended for multiple linguistic audiences. An example would be a beginner's language primer, such as "A First Lesson in Latin," which is clearly intended to be used by an English-literate audience. In this case, the system-language test attribute should only include "en".

Authoring note: Authors should realize that if several alternative language objects are enclosed in a "switch", and none of them matches, this may lead to situations such as a video being shown without any audio track. It is thus recommended to include a "catch-all" choice at the end of such a switch which is acceptable in all cases.

system-overdub-or-caption
This attribute is a setting which determines if users prefer overdubbing or captioning when the option is available. The attribute can have the values "caption" and "overdub". Evaluates to "true" if the user preference matches this attribute value. Evaluates to "false" if they do not match.
system-required
This attribute specifies the name of an extension. Evaluates to "true" if the extension is supported by the implementation, otherwise, this evaluates to "false". In a future version of SMIL, this attribute value will be an XML namespace [NAMESPACES].
This could be very useful (e.g. to control support for 3-D content if an accelerator is installed), however this requires more work to describe an XML namespace mechanism for identifying hardware or software components.
system-screen-size
Attribute values have the following syntax:
screen-size-val ::= screen-height"X"screen-width
Each of these is a pixel value, and must be an integer value greater than 0.  Evaluates to "true" if the SMIL playback engine is capable of displaying a presentation of the given size. Evaluates to "false" if the SMIL playback engine is only capable of displaying a smaller presentation.
system-window-size
Attribute values have the following syntax:
window-size-val ::= window-height"X"window-width
Each of these is a pixel value, and must be an integer value greater than 0.  Evaluates to "true" if the current browser window (i.e. enclosing frame) is at least as large as the given size. Evaluates to "false" if the current browser window smaller.
system-screen-depth
This attribute specifies the depth of the screen color palette in bits required for displaying the element. The value must be greater than 0. Typical values are 1, 8, 24 .... Evaluates to "true" if the browser platform is capable of displaying images or video with the given color depth. Evaluates to "false" if the browser platform is only capable of displaying images or video with a smaller color depth.

The switch Element

The switch element allows an author to specify a set of alternative elements from which only one acceptable element should be chosen. An element is acceptable if the element is an HTML element, if any associated media-type can be decoded , and all of the test-attributes of the element evaluate to "true".  

An element is selected as follows: the parser evaluates the elements in the order in which they occur in the switch element. The first acceptable element is selected at the exclusion of all other elements within the switch.

Thus, authors should order the alternatives from the most desirable to the least desirable. Furthermore, authors should place a relatively fail-safe alternative as the last item in the <switch> so that at least one item within the switch is chosen (unless this is explicitly not desired). Implementations should NOT arbitrarily pick an object within a <switch> when test-attributes for all fail.

Note that http URIs provide for content-negotiation, which may be an alternative to using the "switch" element in some cases.

Element Content

The switch element should be able to contain any HTML content.

Examples

These examples are taken directly from the SMIL 1.0 specification.   

1) Choosing between content with different bitrate

In a common scenario, implementations may wish to allow for selection via a "system-bitrate" parameter on elements. The media player evaluates each of the "choices" (elements within the switch) one at a time, looking for an acceptable bitrate given the known characteristics of the link between the media player and media server.

...
  <t:switch>
    <img src="img_hires.gif"  t:system-bitrate="40000">
    <img src="img_midres.gif" t:system-bitrate="24000">
    <img src="img_lowres.gif" t:system-bitrate="10000">
  </t:switch>
...

2) Choosing between audio resources with different bitrate

The elements within the switch may be any combination of elements. For instance, one could merely be specifying an alternate audio track:

...
<t:switch>
   <t:audio src="joe-audio-better-quality" t:system-bitrate="16000" />
   <t:audio src="joe-audio" t:system-bitrate="8000" />
</t:switch>
...

3) Choosing between audio resources in different languages

In the following example, an audio resource is available both in French and in English. Based on the user's preferred language, the player can choose one of these audio resources.

...
<t:switch>
   <t:audio src="joe-audio-french.au"  t:system-language="fr"/>
   <t:audio src="joe-audio-english.au" t:system-language="en"/>
</t:switch>
...

4) Choosing between content written for different screens

In the following example, the presentation contains alternative parts designed for screens with different resolutions and bit-depths. Depending on the particular characteristics of the screen, the player can choose one of the alternatives.

...
  <t:switch>
    <div t:system-screen-size="1280X1024" t:system-screen-depth="16">
    ...
    </div>
    <div t:system-screen-size="640X480" t:system-screen-depth="32">
    ...
    </div>
    <div t:system-screen-size="640X480" t:system-screen-depth="16">
    ...
    </div>
  </t:switch>
...

Miscellaneous extensions

This is a list of miscellaneous SMIL elements which may need to be supported.   The relevant portions of the SMIL spec are included here.

Note: for a list of SMIL elements that will not be supported, and for specific differences between SMIL and TIME extensions, see Appendix B.

The meta element

The "meta" element can be used to define properties of a document (e.g., author, expiration date, a list of key words, etc.) and assign values to those properties. Each "meta" element specifies a single property/value pair.

Element Attributes

The "meta" element can have the following attributes:

content
This attribute specifies the value of the property defined in the meta element.
The "content" attribute is required for "meta" elements.
name
This attribute identifies the property defined in the meta element.
The "name" attribute is required for "meta" elements.
skip-content
This attribute is introduced for future extensibility of SMIL. It is interpreted in the following two cases:

If the value of the "skip-content" attribute is "true", and one of the cases above apply, the content of the element is ignored. If the value is "false", the content of the element is processed.
The default value for  "skip-content" is "true".

The list of properties is open-ended. This specification defines the following properties:

pics-label or PICS-Label
The value of this property specifies a valid rating label for the document as defined by PICS [PICS].

Element Content

"meta" is an empty element.

Syntax Issues

There was a general issue related to the parameter specification syntax in the HTML+TIME specification. This has been largely resolved, but the alternatives are described here for context.

The document currently presents one syntax to illustrate the model and provide simple examples in HTML.  As the chosen syntax is based upon a draft specification, it may change.   Nevertheless, the syntax changes, if any, will not materially affect the model of time containment in HTML documents.

In addition, this document describes the use of embedded XML elements in the HTML.   This has been referred to as "XML sprinkles".  While this is not part of a current standard, it has been discussed in a related note [XMLinHTML].

Parameter specification syntax

This version of the document presents a model of syntax that defines parameters using XML Namespace qualified attributes. The new XML Namespace proposal [XMLNS] allows for extension attributes that are qualified with a namespace id.  This provides the cleanest syntax, and is used with this document. Nevertheless, at this point the XML Namespace proposal is only a draft.

It might be argued that simple html "expando" attributes would be easiest for authors to use. Such a syntax imposes minimal changes to the HTML that they author today.  However, expandos are problematic in that they violate the HTML DTD (despite the fact that both major browsers parse them without problems). 

An alternative syntax was proposed that moves the parameters to a STYLE string.   These STYLE-expandos are HTML DTD compliant, and CSS specifies that unknown attributes be ignored, making this a reasonable syntax. This was rejected as less elegant than the XMLNS expando syntax.

E.g. where this document describes parameters as XMLNS expandos:

<p t:begin="1" t:dur="3">Some text...</p>

the syntax with STYLE-expandos would look like:

<p style="begin:1;dur:3">Some text...</p>

and the syntax with simple expando attributes would look like:

<p begin="1" dur="3">Some text...</p>

Handling SMIL documents

It should be possible to support SMIL 1.0 compliant documents using HTML+TIME. The layout mechanisms of SMIL can be translated to CSS2, and the timing constructs translate directly. A relatively simple extension could support such a translation mechanism.  This could even be placed within an alternate clause in a switch statement, if the browser supports HTML+TIME extensions.

Because of the asynchronous nature of loading the SMIL file, there may be some issues related to synchronization with the rest of the document.  In all likelihood, the start of the SMIL timeline would normally be deferred relative to the rest of the document, but this can be easily controlled with the synchronization control facilities in HTML+TIME.

Layout within the SMIL document will be achieved using standard CSS functionality.    The SMIL syntax describes a subset of CSS, and so no extensions should be required.  See also the related W3C note [SMIL-CSS]. The output HTML would likely be wrapped in a div (i.e. the SMIL:import tag should essentially subclass div).  The SMIL specification provides for a declaration of the dimensions of the presentation; these will define the default dimensions of the wrapping div. How should the declared dimensions interact with the use of the screen-size test-attribute?

Finally, note that any SMIL renderer that supports the defined interface for media players can be hosted as an object (or embed) on the page.  In this way, a pure SMIL 1.0 presentation can be placed in the web browser context for timing and synchronization with an HTML page.

Note that the translation is only for rendering, and not for editing.  As such, there is no requirement for retaining the full fidelity of SMIL information, as long as the media is correctly rendered. Certain SMIL attributes have no presentation function and these need not be preserved in the translation.

Object Model

Need to formally document the exposed methods, the events and the way the timing model will handle resync events. This is still rough, and needs further discussion and review. This is incomplete, but gives an idea of where we are going.

Methods:

beginElement()
Turns on the behavior, starting the local timeline. Fires an onBegin event and propagates time dependencies.
endElement()
Turns off the behavior, stopping the local timeline. Fires an onEnd event.  If the element is stopping prematurely (i.e. before the defined end time) this also raises an onResync event and propagates time dependencies. Note that turnOff is different from pause().  Unlike pause, turnOff is used to advance to the end of the local timeline, as though the element had played to the end.
cue()
Returns a boolean to indicate whether the element is ready to display (e.g. img media is ready).  Time containers return an aggregate of all descendents.  Todo: need to include notes on interaction of sync rules and cue state.
pause()
Pauses the local timeline, and raises an onResync event.
run()
Resumes a paused timeline.  If the element was not paused, this has no effect.
localToGlobalTime( int ltime, string elementid )
Convert from a local time to a global time in the referenced time space.  Returns an integer indicating the count of milliseconds that represents the local time ltime in the time space of the ancestor timeline specified by elementid.  If elementid is null, this converts from the local timeline to the document global timeline.  This is useful both for authoring applications as well as aligning or otherwise relating elements on disparate timelines.  Note that ltime need not be within the constraints of the local duration.   The conversion is always done relative to the simple local duration; to convert a simple local time in iteration n of a repeating timeline, the caller must account for the offset of n-1 simple durations.

Properties:

The following properties will be supported.  Some platforms may fire property change events, as an alternative means of wiring time-based script or other functions to the local timeline. The list below does not indicate legal elements for the different properties.

The properties for defining the basic timing are read-only.  The model supports run-time modification only through the methods defined and the currTime property.

For numeric timing properties, the values are the effective values.  Thus, if no dur or end attributes are specified, the dur and end properties will be infinite, and if no begin attribute is specified, the value will be 0.

abstract
string, read-only.
author
string, read-only.
begin
floating point number (seconds), read-only.
beginWith
string (element id), read-only.
beginAfter
string (element id), read-only.
beginEvent
string (element id "." event id), read-only.
clip-begin
string, read-only.
clip-end
string, read-only.
clockSource
boolean, read-only.
copyright
string, read-only.
dur
floating point number (seconds) or POSITIVE_INFINITY if indefinite, read-only.
end
floating point number (seconds) or POSITIVE_INFINITY if indefinite, read-only.
endWith
string (element id), read-only.
endEvent
string (element id "." event id), read-only.
endSync
string, read-only.
fill
string, read-only.
longdesc
string, read-only.
noCombine
boolean, read-only.
player
string (id), read-only.
par
boolean, read-only.
repeat
floating point number (seconds) or POSITIVE_INFINITY if indefinite, read-only.
repeatDur
floating point number (seconds) or POSITIVE_INFINITY if indefinite, read-only.
skip-content
boolean, read-only.
syncBehavior
string, read-only.
syncBehaviorDefault
string, read-only.
syncTolerance
floating point number (seconds), read-only.  Only valid if syncBehavior set to "locked"
syncToleranceDefault
floating point number (seconds), read-only.  Only valid if syncBehavior set to "locked"
timeAction
string, read-only.
timeStartRule
string, read-only.
title
string, read-only.
type
string, read-only.
useTimedHyperlinks
boolean, read-only.

Internal State properties

currTime
This provides read/write access to the local time for the element.  Reading this property provides the current time on the local timeline.  Note that when a local timeline repeats, this property presents the simple time ranging from 0 to the repeat duration.  Writing a value to this property will resync the element within the time container.  If the sync rules specify hard sync, the new value will be ignored.  The implementation could just raise a resync event and let the normal resync mechanisms deal with the new sync relationship, but this does not seem useful enough to justify the complexity or potential confusion in use.
isPaused
boolean that indicates whether the local element timeline is paused.
Value is "true" if the element timeline is currently paused.
Value is read-only (can be changed via the pause() and run() methods).

Test Attribute/Properties

Note that all properties/attributes associated with the test-attributes will be read-only. I.e. these values cannot be set other than in the original syntax. The test attributes and switch elements will only be evaluated once.  Any changes to attributes (e.g. via script) will have no effect upon the evaluation of a switch element.

Nevertheless, there should be a mechanism for users to control the system settings for captioning and overdub support, in accordance with the WAI accessibility guidelines [WAI].  This should probably take the form of a system control, or additional user agent (browser) support for indicating elements with the associated test attributes and allowing dynamic control of these.  Further work is required in this area.

system-bitrate
string, read-only.
system-captions
string ("on" or "off"), read-only
system-language
string, read-only.
system-overdub-or-caption
string ("caption" or "overdub"), read-only
system-required
string, read-only.
system-screen-size
string (widthXheight), read-only.
system-window-size
string (widthXheight), read-only.
system-screen-depth
integer, read-only.

Events:

This present the basic set of events that are associated with HTML+TIME implementations.  Further work may be done to identify additional support required or recommended.  In particular, the issues associated with media loading and bandwidth management should be addressed, to help ensure that a presentation plays back similarly on different implementations (browsers).

onBegin
Raised when the element starts for any reason - just because of the timing, or because of a turnOn call.
onEnd
Raised when the element stops for any reason - just because of the timing, or because of a turnOff call.
onRepeat
Raised when the local timeline repeats (i.e. on the first sample of each repeat iteration after the first one). This is not raised when the local timeline starts (i.e. on the first iteration).  If the local timeline does not repeat, this event will never be raised.
The event should include the iteration count (0 based, so the first event raised will have the value 1).
onResync
Raised when a media player has broken sync for some internal reason, and when resync is called. The default action of the timing model is to attempt to reestablish sync, depending on the media load rules. As part of this, it will propagate any time dependencies. The onResync event will generally bubble up to the first enclosing element that is set to slip sync.
The event should include information on the reason for the resync (e.g. communications, pause-request, etc.).  Not clear how to specify this without more formal specification of events in DOM.
onMediaComplete
Applies to media elements (especially streaming media).  Raised when media is done loading.  This should generally be before the media is done playing, and will support implementations in optimizing media loading/buffering.  Intended use is that when, e.g. a video is done loading, the next video in a sequence can begin to cue or buffer.

Support for media player extensions

It should be possible to integrate new media players into the HTML+TIME model, to support an open set of media types.  Media players generally support varying levels of control, depending on the constraints of the underlying renderer as well as media delivery, streaming etc. HTML+TIME defines 4 levels of support, allowing for increasingly tight integration, and broader functionality.  The details of the interface will be presented in a separate document. 

Level 0
must allow the media wrapper to query for duration, and must support cue, start and stop on the player. To support reasonable resync, the player must provide pause/unpause controls with minimal latency. This is the minimum level of support defined.
Level 1
In addition to all Level 0 support, can detect when sync has been broken, so that a resync event can be fired. A media player that cannot support Level 1 functionality is responsible to maintain proper synchronization in all circumstances, and has no remedy if it cannot (Level 1 support is recommended).
Level 2
In addition to all Level 1 support, supports a tick() method for advancing the timeline in strict sync with the document timeline. This is generally appropriate to animation renderers that are not tightly bound to media delivery constraints.
Level 3
In addition to all Level 2 support, also supports a query interface to provide information about the time-related capabilities of the associated player.   Capabilities include things like canRepeat, canPlayBackwards, canPlayVariable, canHold, etc. This is mostly for future extension of the timing functionality and for optimization of media playback/rendering.

HTML+TIME support for client behaviors

HTML+TIME provides the underpinnings for time-based animation and interaction.   HTML authors and animation tools vendors can build upon this basis to provide animation capabilities via simple script or DHTML behaviors [BEHAVIORS]. From the perspective of HTML+TIME, these additional behaviors are clients of the timing services.

Client behaviors must be able to leverage the timing and synchronization support provided by HTML+TIME.  The behaviors are considerably simpler than extension media players, and have a simple interface to HTML+TIME.  Time-varying behaviors should be modeled on a local timeline that can be arbitrarily sampled (i.e. no dependencies on the sample rate or the ordering of sample times).  Note that the implementation is not constrained by this requirement, and may be based upon interpolation, closest keyframe fit or random numbers. The details of the interface will be described in a separate document, but the general mechanism is described here.

A client behavior will attach to the local timeline by calling an addClient method on the element. As the local timeline advances, the HTML+TIME behavior will call back to a client behavior method: update().  Parameters include the current local time, which the client behavior will use to sample its respective timeline. Reviewers: an alternative mechanism would simply present a tick event, simplifying the model somewhat. This could make it hard to handle resync situations, as the time-children are not registered. Opinions?

A set of basic queries on the element will support information about the simple timeline duration, etc.

Note: if behaviors are developed that require a broader interface to HTML+TIME, they can be modeled as media players.

Issues:

The syntax for declaring client behaviors is currently open.  XML and scene graph descriptions would describe them within the block (i.e. element) that they modify (act upon).  This also makes sense from the TIME standpoint, as they appear underneath (i.e. within) the local timeline of the modified element.  However, traditional HTML marks up content by wrapping it in a span.  This conflicts with the user-models generally used in animation and time-based media authoring.

HTML integration

HTML Elements that are divided into two basic groups for the purposes of control by HTML+TIME behaviors:

  1. Displayed elements that support the style parameters display and visibility - e.g. H1, P, img.
  2. Style elements that modify the appearance of display elements - e.g. STYLE, FONT, B.

The first group are controlled by manipulating the display or visibility properties.    The second group must be controlled by removing the effect of the element when it is not active on the timeline.  Thus the HTML+TIME behavior for a bold tag must remove the bold effect before the element is defined to begin and after it ends.  It should not force the style off in a manner that would override an enclosing style, but rather should simply remove the effect of the style element outside the active duration.

There are some HTML tags that make no sense to integrate with a timeline, including those that occur within the HEAD, and some that affect neither display or content (e.g. COMMENT).

The classification of HTML elements is presented in Appendix A.

References

[ATVEF]
"Advanced Television Enhancement Forum Specification".
Available at http://www.atvef.com/atvef_spec/TVE-public.htm.
[BEHAVIORS]
"Element Behavior in CSS Proposal - June 10, 1998". Chris Wilson, editor.
Available at http://www.w3.org/Style/Group/1998/06/Behavior (W3C Members only).
[HTML40]
"HTML 4.0 Specification", D. Raggett, A. Le Hors, I. Jacobs, 24 April 1998.
Available at http://www.w3.org/TR/REC-html40/.
[ISO/IEC 10646]
ISO (International Organization for Standardization). ISO/IEC 10646-1993 (E). Information technology -- Universal Multiple-Octet Coded Character Set (UCS) -- Part 1: Architecture and Basic Multilingual Plane. [Geneva]: International Organization for Standardization, 1993 (plus amendments AM 1 through AM 7).
[NAMESPACES]
"Namespaces in XML", T. Bray, D. Hollander, A. Layman, 27 March 1998
W3C working draft. Available at http://www.w3.org/TR/WD-xml-names.
[PICS]
"PICS 1.1 Label Distribution -- Label Syntax and Communication Protocols", 31 October 1996, T. Krauskopf, J. Miller, P. Resnick, W. Trees
Available at http://www.w3.org/TR/REC-PICS-labels-961031
[RFC1766]
"Tags for the Identification of Languages", H. Alvestrand, March 1995.
Available at ftp://ftp.isi.edu/in-notes/rfc1766.txt.
[RFC2045]
"Multipurpose Internet Mail Extensions (MIME) Part One: Format of Internet Message Bodies", N. Freed and N. Borenstein, November 1996.
Available at ftp://ftp.isi.edu/in-notes/rfc2045.txt. Note that this RFC obsoletes RFC1521, RFC1522, and RFC1590.
[SMIL]
"Synchronized Multimedia Integration Language (SMIL) 1.0 Specification W3C Recommendation 15-June-1998 ".
Available at: http://www.w3.org/TR/REC-smil.
[SMIL-CSS]
"Displaying SMIL Basic Layout with a CSS2 Rendering Engine".
Available at: http://www.w3.org/TR/NOTE-CSS-smil.html.
[SMPTE]
"Time and Control Codes for 24, 25 or 30 Frame-Per-Second Motion-Picture Systems - RP 136-1995".  Society of Motion Picture & Television Engineers.
[VML]
"Vector Markup Language (VML)", World Wide Web Consortium Note 13-May-1998.
Available at http://www.w3.org/TR/NOTE-VML
[WAI]
"WAI Accessibility Guidelines: User Agent", W3C Working Draft 3-July-1998.
Available at http://www.w3.org/WAI/UA/WD-WAI-USERAGENT
[XML10]
"Extensible Markup Language (XML) 1.0", T. Bray, J. Paoli, C.M. Sperberg-McQueen, editors, 10 February 1998.
Available at http://www.w3.org/TR/REC-xml/
[XMLNS]
"Namespaces in XML World Wide Web Consortium Working Draft 2-August-1998 ", W3C XML Working Group.
Available at http://www.w3.org/TR/WD-xml-names
[XMLinHTML]
"XML in HTML Meeting Report ", W3C note  11 May 1998.
Available at http://www.w3.org/TR/NOTE-xh

Appendix A - HTML Elements classification

Each HTML element is classified according to the type (manner) of HTML+TIME control.  

It is important to note that certain elements can be considered either style or content.  A judgement call is made to classify them according to the more logical type.  Thus, BLOCKQUOTE, LISTING and PRE are classified as content (as they function more like a P element than a font style element), and CENTER (although a block element) is classified as a style.

Two elements are not accounted for in the sets below: COL and COLGROUP.  Some documentation states that the visibility property applies to these elements.  This could place them in the content group, although they do not really contain content the way most content elements do.  This needs further attention.

Note that it would be really nice to support timed SCRIPT content. Need to consider the model for delayed evaluation, repeated evaluation, etc.  Will it make sense to repeat script (with some declared pseudo-duration)? What constraints must be placed upon usage?

The defined types are:

content
An element that is used (primarily) to contain content.  Content elements are controlled by HTML+TIME using the visibility property, and where supported, the display property.
The set of content elements includes:

APPLET, BLOCKQUOTE, BODY, BUTTON, CAPTION, CITE, CODE, DD, DIR, DIV, DL, DT, FIELDSET, FORM, H1-6, HR, HTMLAREA, IFRAME, IMG, INPUT, LEGEND, LI, LISTING, MARQUEE, MENU, OL, OPTION, P, PRE, SAMP, SELECT, SPAN, TABLE, TBODY, TD, TEXTAREA, TFOOT, TH, THEAD, TR, UL.
style
An element that is used (primarily) to describe the style of content.  Style elements are controlled by HTML+TIME by removing the effect of the style when the element is not active on the timeline, and by applying the style when the element is active. The intended result is that "removing the effect" of a style element will not override inherited style.
Note that for A and AREA, removing the style means that the hyperlink is also disabled.
The set of style elements includes:

A, ACRONYM, ADDRESS, AREA, B, BIG, CENTER, DEL, EM, FONT, I, INS, KBD, PLAINTEXT, Q, S, SMALL, STRIKE, STRONG, SUB, SUP, TT, U, VAR, XMP.
timeline
These are the FRAME and FRAMESET elements.  Both must by definition define a new timeline.  They are content, but also define a timeline by default (just as the document.body element defines the root timeline).
head-only
An element that is only legal in the head of a document, and that will not be controlled by HTML+TIME.  These elements do not appear in the document body, and so cannot participate on the document timeline. 
The set of head-only elements includes:

BASE, BASEFONT, BGSOUND, DOCTYPE, LINK, META, STYLE, TITLE.
none
An element that will not be controlled by HTML+TIME.  These are elements that are orthogonal to the document display and/or which do not support any sort of display controls.  Most are logical elements.
The set of elements of type none includes:

BR, COMMENT, DFN, EMBED, HEAD, HTML, ISINDEX, LABEL, MAP, NOBR, NOFRAMES, NOSCRIPT, OBJECT, PARAM, SCRIPT, WBR.

Appendix B - Differences between SMIL and TIME

HTML+TIME is based in large part upon SMIL, and so the core syntax is very similar.   Because of the integration with HTML and CSS, the SMIL layout syntax and some SMIL boilerplate syntax is not needed.  Some differences in the timing syntax are introduced, however.  In some cases, these changes allow the syntax to match the HTML Document Object Model (DOM) in a more standard manner.  In other cases, HTML+TIME extends the SMIL functionality to support more control and integration with a document time model.  We have attempted to keep the extensions within the overall SMIL model, so that they could be worked into the SMIL specification as well.

Because layout is handled by existing HTML/CSS mechanisms, there is no layout section as there would be in SMIL documents. HTML+TIME marks up existing HTML with inline timing info, and so effectively integrates layout and timing.  This keeps the information relating to an element all together. This can make editing simpler and more straight-forward.

On a minor note, SMIL naming (using hyphens) seems to follow the CSS property naming practice, in conflict with the HTML attribute naming practice.  HTML+TIME preserves the SMIL names with hyphens for compatibility, but uses the HTML naming standard for new attributes.

One significant difference between HTML+TIME and SMIL is the description of an Object Model.  This is appropriate in the HTML environment, and should support platform implementers as well as authors.

This appendix details the differences between the two specifications.

SMIL elements not supported in HTML+TIME

Elements subsumed by HTML/CSS

These elements are not needed, as the equivalent functionality is already in HTML and/or CSS.  There is no functional loss in not supporting these.  The elements are presented in the order they appear in the SMIL 1.0 specification.

smil
No need for this within HTML.
head
HTML defines equivalent element.
layout
All layout handled inline by HTML/CSS.
region
All layout handled inline by HTML/CSS. SMIL defines an attribute "fit" that is not directly supported in CSS.  Quoting from the SMIL 1.0 spec:  "This attribute does not have a 1-1 mapping onto a CSS2 property, but can be simulated in CSS2".
root-layout
All layout handled inline by HTML/CSS.
meta
HTML defines equivalent element.
body
HTML defines equivalent element.
text
HTML defines a host of equivalent elements for static text content.  Note that textstream is still included in HTML+TIME.
a
HTML defines this tag already, with equivalent functionality.  SMIL defines an additional attribute "show", with three values.  Two of the values - "replace" and "new" - duplicate part of the functionality of the "target" property on the HTML <a> tag.   The third value - "pause" can be supported in most cases using simple script.  In certain cases, it may be impractical to support (e.g. pausing one browser instance to wait for another).
anchor
The equivalent functionality is supported by image maps, with area elements supporting the time and space restrictions on the link.

Element Attributes subsumed by HTML/CSS

id
Applies to content elements.  HTML supports this.
alt
Applies to media elements.  HTML supports this.

Element Attributes omitted

region
Applies to content elements.  As layout is handled inline, this no longer applies.

HTML+TIME Extensions to SMIL

SMIL Elements with extended syntax/semantics

These elements are supported in HTML+TIME, but have additional support or modified syntax.  The SMIL defined functionality is preserved by HTML+TIME.  The elements are presented in the order they appear in the SMIL 1.0 specification.

par
HTML+TIME presents equivalent as an attribute ("par") on HTML containers.   This simplifies the syntax, and preserves the functionality of par.
An open issue is support for an element equivalent to par.
media
This is functionally equivalent to the "ref" element in SMIL, but provides a more intuitive name.
switch
HTML+TIME supports this only in the body, where SMIL supports it in the head as well (but only for layout, which is handled inline in the body of HTML).
HTML+TIME does not restrict the children to be SMIL elements, but rather supports content elements. This may need further review to define restrictions on legal HTML elements.

SMIL Element Attributes with extended syntax/semantics

begin
HTML+TIME allows for signed clock-values.
end
HTML+TIME allows for signed clock-values.
The SMIL definition of implicit end appears to require that an element with a repeat="indefinite" attribute disappear immediately if no end or duration is specified.  This seems contra-indicated.
The SMIL definition of implicit end does not account for the use of clip-begin and clip-end.  It is not clear how these attributes are intended to interact with end and duration.  This is defined in HTML+TIME.
repeat
supports fractional repeat counts, in particular for animations and similar cases..
clip-begin
Clock value definition extended (described below).
clip-end
Clock value definition extended (described below).
Rules for error handling defined (missing in SMIL).
fill
Extended to support media load/resync gaps (in addition to defined timing gaps as SMIL describes).
Additional values defined:
extend - extends local timeline to fill gap (useful for many static and some animation elements)
repeat - repeats local timeline to fill gap.

SMIL Attribute Value definitions with extended syntax/semantics

clock value
Allows default metric of clock time, without metric specifier (simplifies common cases)
Replaces "npt" metric name with "clock", for ease of learning. 
Hours not limited to 2 digits, and not required to have leading 0.
clip-time value
Allows default metric of clock time, without metric specifier (simplifies common cases)
Replaces "npt" metric name with "clock", for ease of learning. 
Adds additional metrics Index-val and timeID-val.
Hours (in smpte val) not limited to 2 digits, and not required to have leading 0.
element event value
Syntax aligned to normal CSS/script.
Allowed events opened up to include all DOM-legal events. Note that time events ("begin" and "end") retain SMIL constraints.
SMIL defines "implementation-dependent" interpretation of clock-val events.   The definition seems to allow for widely varying timing depending upon the interpretation (allowing reference to media time), and seems contra-indicated.
SMIL makes it illegal to reference a time greater than the defined duration for an element.  This precludes reference to local timing that is extended to fill a gap or to cover a media load/resync problem.  This restriction is not included in HTML+TIME.

New Element Attributes introduced by HTML+TIME

beginWith
In general, HTML and DOM separate individual properties into individual attributes.   In keeping with this, beginWith presents the begin-time base represented by the SMIL timing relative to "element start event".
beginAfter
This presents the begin-time base represented by the SMIL timing relative to "element end event".
beginEvent
This presents the equivalent of "element event" timing in SMIL, and adds support for any DOM-legal event. It also allows the value "none" for pure indeterminate begin timing.
endWith
This presents the end-time base represented by the SMIL timing relative to "element end event".
endEvent
This presents the equivalent of "element event" timing in SMIL, and adds support for any DOM-legal event. It also allows the value "none" for pure indeterminate end timing.
repeatDur
This allows specification of repeat by total duration.
timeAction
Defines how timing affects HTML elements.
par
Defines an HTML container to be a local timeline (functional equivalent to "par" element in SMIL).
img
Applies to visual media elements (video, animation).  Defines an alternate image to display if the media cannot be rendered.
noCombine
Applies to all media elements. Hint to implementation to preclude potential optimization.
syncBehavior
Applies only to body element. Defines synchronization behavior for an element.
syncTolerance
Applies only to body element. Defines allowed tolerance for synchronization.
clockSource
Defines associated media element as synchronization master.
system-window-size
Test attribute.  Similar to system-screen-size, but for the browser window size.

Appendix C - XML DTD for new HMTL+TIME elements

This should present DTD for all new HTML+TIME elements.  It will not provide a means of verifying an entire HTML document with TIME extensions, but will support verification of the new elements.

Appendix D - Additional Issues to consider

Consider Accessibility issues.  See also http://www.w3.org/WAI/UA/WD-WAI-USERAGENT.html.  In particular, consider issues related to title, longdesc and controls for captioning.  Also, recommendations describe need for "a standard means of notifying third-party assistive technologies ... of the existence of audio description for a video object" (and related proxies).   Should this be incorporated somehow into the system-required test attribute, or should a new test attribute be created for this kind of thing?  Alternatively, should we have OM support for locating elements of interest to assistive technologies?  These issues may have to wait for a later version of SMIL to resolve.

Consider adding support for acceleration/deceleration.   This is very powerful for client scripts or behaviors, and hard for them to do otherwise.   It should just be a hint (it will never affect duration anyway), so that if applied to something like a movie player, it can be ignored without affecting the time graph.   Proposed support would be an attribute with a percentage value.  The value is the portion of the local duration over which acceleration or deceleration should be applied.  The effect will modify the rate at which the element is played such that the duration is preserved.  The sum of any acceleration and deceleration values for an element must be no greater than 100.

Consider adding support for transitions.  We should not go very far with this, or HTML+TIME will start overlapping with animation and video authoring tools.  Nevertheless, it would be nice to specify some behavior between videos in sequence, etc.  Simple fades, cross-fades, etc. If modeled as a hint, rather than a requirement, may be easier to handle. Getting into full effects and wipes makes less sense.  What about layering and opacity?  For Enhanced TV, transparency would be important (content screened over the BG video). 

Need to further document the Object Model, especially w.r.t. child behaviors and media player integration.  Need to describe the properties associated with the attributes.  Need to add more rigorous definition of what happens on repeat, and how this interacts with synchronization and resync.  Need to resolve how time children attach to time parents (e.g. update() call versus tick event).