previous   next   contents  

10. The SMIL Timing and Synchronization Module

Editors:
Patrick Schmitz (pschmitz@microsoft.com), (Microsoft)
Jeff Ayars (jeffa@real.com), (RealNetworks)
Bridie Saccocio (bridie@real.com), (RealNetworks)


Table of contents

10.1 Introduction

SMIL 1.0 solved fundamental media synchronization problems and defined a powerful way of choreographing multimedia content. SMIL Boston extends the timing and synchronization support, adding capabilities to the timing model and associated syntax. This section of the document specifies the Timing and Synchronization module.

There are two intended audiences for this module: implementers of SMIL Boston document viewers or authoring tools, and authors of other XML languages who wish to integrate timing and synchronization support. A language with which this module is integrated is referred to as a host language. A document containing SMIL Timing and Synchronization elements and attributes is referred to as a host document.

As this module is used in different profiles (i.e. host languages), the associated syntax requirements may vary.  Differences in syntax should be minimized as much as is practical.  The semantics of the timing model and of the associated markup must remain consistent across all profiles.  Any host language that includes SMIL Boston Timing and Synchronization markup (either via a hybrid DTD or schema, or via namespace qualified extensions) must preserve the semantics of the model defined in this specification.

Some SMIL 1.0 syntax has been changed or deprecated. Only SMIL document players must support the deprecated SMIL 1.0 attribute names as well as the new SMIL Boston names.  A SMIL document player is an application that supports playback of SMIL Language documents (i.e. documents with the associated MIME type "application/smil").

10.2 Overview of SMIL timing

This section is informative.

SMIL Timing defines elements and attributes to coordinate and synchronize the presentation of media over time. The term media covers a broad range, including discrete media types such as still images, text, and vector graphics, as well as continuous media types that are intrinsically time-based, such as video, audio and animation.

Three synchronization elements support common timing use-cases:

These elements are referred to as time containers. They group their contained children together into coordinated timelines.

SMIL Timing also provides attributes that can be used to specify an element's timing behavior. Elements have a begin, and a simple duration. The begin can be specified in various ways - for example, an element can begin at a time, based upon when another element begins, or when some event (such as a mouse click) happens. The simple duration defines the basic presentation duration of an element. Elements can be defined to repeat the simple duration, a number of times or for an amount of time. The simple duration and any effects of repeat are combined to define the active duration. When an element's active duration has ended, the element can either be removed from the presentation or frozen (held in its final state), e.g. to fill any gaps in the presentation.

Figure 1 illustrates the basic support of a repeating element within a simple par time container. The corresponding syntax is included with the diagram.

Basic strip illustration of timing

<par begin="0s" dur="33s">
   <video begin="1s" dur="10s" repeatCount="2.5" fill="freeze" .../>
</par>

Figure 1 - Strip diagram of basic timing support. The starred "Simple*" duration indicates that the simple duration is partial (i.e. it is cut off early).

The attributes that control these aspects of timing can be applied not only to media elements, but to the time containers as well. This allows, for example, an entire sequence to be repeated, and to be coordinated as a unit with other media and time containers. While authors can specify a particular simple duration for a time container, it is often easier to leave the duration unspecified, in which case the simple duration is defined by the contained child elements. When an element does not specify a simple duration, the time model defines an implicit simple duration for the element. For example, the implicit simple duration of a sequence is based upon the sum of the active durations of all the children.

Each time container also imposes certain defaults and constraints upon the contained children. For example in a <seq>, elements begin by default right after the previous element ends, and in all time containers, the active duration of child elements is constrained not to extend past the end of the time container's simple duration. Figure 2 illustrates the effects of a repeating <par> time container as it constrains a <video> child element.

More complex strip illustration of timing

<par begin="0s" dur="12s" repeatDur="33s" fill="freeze" >
   <video begin="1s" dur="5s" repeatCount="1.8" fill="freeze" .../>
</par>

Figure 2 - Strip diagram of time container constraints upon child elements. The starred "Simple*" durations indicate that the simple duration is partial (i.e. it is cut off early).

The SMIL Timing Model defines how the time container elements and timing attributes are interpreted to construct a time graph. The time graph is a model of the presentation schedule and synchronization relationships. In an ideal environment, the presentation would perform precisely as specified. However, various real-world limitations (such as network delays) can influence the actual playback of media. How the presentation application adapts and manages the presentation in response to media playback problems is termed runtime synchronization behavior. SMIL includes attributes that allow the author to control the runtime synchronization behavior for a presentation. 

The SMIL Timing and Synchronization syntax and precise semantics are described in the following section. A set of symbols are used in the semantic descriptions:

B
The time at which an element begins.
d
The simple duration of an element.
AD
The active duration of an element. This is the period during which time is actively advancing for the element. This includes any effect of repeating the simple duration, but does not include the time during which the element may be frozen.
AE
The active end. This is the end of the active duration of an element.

10.3 Language definition

The timing model is defined by building up from the simplest to the most complex concepts: first the basic timing and simple duration controls, followed by the attributes that control repeating and constraining the active duration.  Finally, the elements that define time containers are presented.

The time model depends upon several definitions for the host document: A host document is presented over a certain time interval. The start of the interval in which the document is presented is referred to as the document begin. The end of the interval in which the document is presented is referred to as the document end. The difference between the end and the begin is referred to as the document duration. The formal definitions of presentation and document begin and end are left to the host language designer (see also "Required host language definitions").

10.3.1 Shared timing support

This section defines the set of timing attributes that are common to all of the SMIL synchronization elements.

@@ Need to define "local time" or find a different term.

Basics - begin and dur

The basic timing for an element is described using the begin and dur attributes. Authors can specify the begin time of an element in a variety of ways, ranging from simple clock times to the time that an event (e.g. a mouse-click) happens. The simple duration of an element is specified as a simple time value. The length of the simple duration is specified using the dur attribute. The attribute syntax is described below. The normative syntax rules for each attribute value variant are described below (in Timing Attribute Values); a syntax summary is provided here as an aid to the reader.

begin : smil-1.0-syncbase-value* | begin-value-list | "indefinite"
Defines when the element should begin (i.e. become active). 
The attribute value is either a SMIL 1.0 syncbase declaration, a semi-colon separated list of values, or the special value "indefinite".
smil-1.0-syncbase-value* : "id(" id-ref ")" ( "(" ( "begin" | "end" | clock-value ) ")" )?
Describes a syncbase and an offset from that syncbase. The element begin is defined relative to the begin or active end of another element.
*Note: Only compliant SMIL document players are required to support the SMIL 1.0 syncbase-value syntax. Language designers integrating SMIL Boston Timing and Synchronization into other languages should not support this syntax.
begin-value-list : begin-value (";" begin-value-list )?
A semi-colon separated list of begin values. The interpretation of a list of begin times is detailed below.
"indefinite"
The begin of the element will be determined by a "beginElement()" method call or a hyperlink targeted to the element.
The SMIL Timing and Synchronization DOM methods are described in the Supported Methods section.
Hyperlink-based timing is described in the Hyperlinks and Timing section.
 
begin-value : ( offset-value | syncbase-value | syncToPrev-value | event-value | media-marker-value | wallclock-sync-value )
Describes the element begin.
 
offset-value : ( "+" | "-" )? clock-value
Describes the element begin as an offset from an implicit syncbase. The definition of the implicit syncbase depends upon the element's parent time container. The offset is measured in local time on the parent time container.
syncbase-value : ( id-ref "." ( "begin" | "end" ) ) ( ( "+" | "-" ) clock-value )?
Describes a syncbase and an offset from that syncbase. The element begin is defined relative to the begin or active end of another element.
syncToPrev-value : ( "prev.begin" | "prev.end" ) ( ( "+" | "-" ) clock-value )?
Describes a logical syncbase and an offset from that syncbase. The syncbase element is the previous timed sibling element, as reflected in the DOM (or the parent time container if there is no previous sibling). The element begin is defined relative to the begin or active end of the syncbase element. This is equivalent to the default syncbase for children of a <seq> time container.
event-value : ( id-ref "." )? ( event-ref  ) ( ( "+" | "-" ) clock-value )?
Describes an event and an optional offset that determine the element begin. The element begin is defined relative to the time that the event is raised. Events may be any event defined for the host language in accordance with [DOM2Events]. These may include user-interface events, event-triggers transmitted via a network, etc. Details of event-based timing are described in the section below on Unifying Event-based and Scheduled Timing.
media-marker-value : id-ref ".marker(" marker-name ")"
Describes the element begin as a named marker time defined by a media element.
wallclock-sync-value : "wallclock(" wallclock-value ")"
Describes the element begin as a real-world clock time. The wallclock time syntax is based upon syntax defined in [ISO8601].
dur
Specifies the simple duration.
The attribute value can be either of the following:
clock-value
Specifies the length of the simple duration, measured in local time.
Value must be greater than 0.
"indefinite"
Specifies the simple duration as indefinite.
Begin value semantics

If no begin is specified, the default timing is dependent upon the time container. Children of a <par> begin by default when the <par> begins (equivalent to begin="0"). Children of a <seq> begin by default when the previous child ends its active duration (equivalent to begin="0"). Children of an <excl> default to a begin value of "indefinite".

The begin value can specify a list of times. This can be used to specify multiple "ways" or "rules" to begin an element, e.g. if any one of several events is raised. A list of times can also define multiple begin times, allowing the element to play more than once (this behavior can be controlled, e.g. to only allow the earliest begin to actually be used - see also Restarting elements). 

In general, the earliest time in the list determines the begin time of the element. In the case where an element can begin multiple times, the next begin time is the earliest begin time after the current time. There are additional constraints upon the evaluation of the begin time list, detailed in Evaluation of begin and end time lists.

If there is an error in any individual value in the list of begin values, only the individual value will be ignored (as though it were not specified), but the rest of the list will not be invalidated. If no legal value is specified in the list of begin values, the default value for begin will be used.

When a begin time is specified as a syncbase variant, a marker value or a wallclock value, the defined time must be converted by the implementation to a time that is relative to the parent time container (i.e. to the equivalent of an offset value).  This is know as timespace conversion, and is detailed in the section Converting between local and global times.

Dur value semantics

If there is any error in the argument value syntax for dur , the attribute will be ignored (as though it were not specified).

If the element does not have a (valid) dur attribute, the simple duration for the element is defined to be the implicit duration of the element. The implicit duration depends upon the type of an element. The primary distinction is between different types of media elements and time containers. Note that if a media element has time children (e.g. animate or area elements), then it is also a <par> time container. If the media element has no time children, it is described as a simple media element.

If the author specifies a simple duration that is longer than the "intrinsic" defined duration for a continuous media element, the ending state of the media (e.g. the last frame of video) will be shown for the remainder of the simple duration. This only applies to visual media - aural media will simply stop playing.

Note that when the simple duration is "indefinite", some simple use cases can yield surprising results. See the related example #4.

Resolving times

Note that when the begin attribute refers to an event, or to the begin or active end of another element, it may not be possible to calculate when the begin will happen. For example, if an element is defined to begin on some event, the begin time will not be known until the event happens. When such a time becomes known (i.e. when it can be calculated as a presentation time), the time is said to be resolved (see also the discussion of Unifying scheduled and interactive timing).

Examples

The following example shows simple offset begin timing. The <audio> element begins 5 seconds after the <par> time container begins, and ends 4 seconds later.

<par>
   <audio src="song1.au" begin="5s" dur="4s" />
</par>

The following example shows syncbase begin timing. The <img> element begins 2 seconds after the <audio> element begins.

<par>
   <audio id="song1" src="song1.au" />
   <img src="img1.jpg" begin="song1.begin+2s" />
</par>

Elements can also be specified to begin in response to an event.  In this example, the image element begins (appears) when the user clicks on element "show". The image will end (disappear) 3 and a half seconds later.

<text id="show" ... />
<img begin="show.click" dur="3.5s" ... />

Timing attribute values

In the syntax specifications that follow, allowed white space is indicated as "S", defined as follows (taken from the [XML10] definition for "S"):

S ::= (#x20 | #x9 | #xD | #xA)*
Begin values

A begin-value-list is a semi-colon separated list of timing specifiers:

begin-value-list ::= begin-value (S ";" S begin-value-list )?
begin-value      ::= (offset-value | syncbase-value 
                      | syncToPrev-value | event-value
                      | media-marker-value | wallclock-sync-value)
End values

An end-value-list is a semi-colon separated list of timing specifiers:

end-value-list ::= end-value (S ";" S end-value-list )?
end-value      ::= (clock-value | syncbase-value 
                      | syncToPrev-value | event-value
                      | media-marker-value | wallclock-sync-value)
Parsing timing specifiers

Several of the timing specification values have a similar syntax. In addition, XML ID attributes are allowed to contain the dot '.' separator character. The backslash character '\' can be used to escape the dot separator within identifier and event-name references. To parse an individual item in a value-list, the following approach defines the correct interpretation. 

  1. If the value begins with a number or numeric sign indicator (i.e. '+' or '-'), the value should be parsed as an offset value.
  2. Else if the value begins with the token "prev", it should be parsed as a syncToPrev-value.
  3. Else if the value begins with the token "wallclock", it should be parsed as a wallclock-sync-value.
  4. Else: Build a token substring up to but not including any sign indicator (i.e. strip off any offset). In the following, ignore any '.' separator characters preceded by a backslash '\' escape character.
    1. If the token contains no '.' separator character, then the value should be parsed as an event-value with an unspecified (i.e. default) eventbase-element.
    2. Else if the token ends with the string ".begin" or ".end", then the value should be parsed as a syncbase-value.
    3. Else if the token contains the string ".marker(", then the value should be parsed as a media-marker-value.
    4. Else, the value should be parsed as an event-value (with a specified eventbase-element).

@@Note that this approach essentially reserves the following tokens: prev and wallclock for element IDs, and begin, end and marker for event names.

Clock values

Clock values have the following syntax:

Clock-val         ::= ( Full-clock-val | Partial-clock-val | Timecount-val )
Full-clock-val    ::= Hours ":" Minutes ":" Seconds ("." Fraction)?
Partial-clock-val ::= Minutes ":" Seconds ("." Fraction)?
Timecount-val     ::= Timecount ("." Fraction)? (Metric)?
Metric            ::= "h" | "min" | "s" | "ms"
Hours             ::= DIGIT+; any positive number
Minutes           ::= 2DIGIT; range from 00 to 59
Seconds           ::= 2DIGIT; range from 00 to 59
Fraction	  ::= DIGIT+
Timecount         ::= DIGIT+
2DIGIT		  ::= DIGIT DIGIT
DIGIT		  ::= [0-9]

For Timecount values, the default metric suffix is "s" (for seconds). No embedded white space is allowed in clock values, although leading and trailing white space characters will be ignored.

The following are examples of legal clock values:

Fractional values are just (base 10) floating point definitions of seconds. The number of digits allowed is unlimited (although actual precision may vary among implementations). 
For example:

00.5s = 500 milliseconds
00:00.005 = 5 milliseconds
Offset values

An offset value has the following syntax:

offset-value   ::= ( "+" | "-" )?( Clock-value )

An offset value allows an optional sign on a clock value, and is used to indicate a positive or negative offset. The offset is measured in local time on the parent time container.

The implicit syncbase for an offset value is dependent upon the time container: 

SMIL 1.0 begin and end values 

Note, only compliant SMIL document players are required to support the SMIL 1.0 syncbase-value syntax. Language designers integrating SMIL Boston Timing and Synchronization into other languages should not support this syntax.

smil-1-syncbase-value  ::= "id(" id-ref ")" 
                           ( "(" ( "begin" | "end" | clock-value) ")" )?
ID-Reference values

ID reference values are references to the value of an "id" attribute of another element in the document. 

Id-value   ::= IDREF

The IDREF is a legal XML identifier.

Syncbase values

A syncbase value has the following syntax:

 Syncbase-value   ::= ( Syncbase-element "." Time-symbol )
                      ( S ("+"|"-") S Clock-value )? 
 Syncbase-element ::= Id-value
 Time-symbol      ::= "begin" | "end"

A syncbase value starts with a Syncbase-element term defining the value of an "id" attribute of another element referred to as the syncbase element. The syncbase element must be another timed element contained in the host document. In addition, the syncbase element may not be a descendent of the current element. If the syncbase element specification refers to an illegal element, the syncbase-value description is ignored (although the entire time value list is not invalidated - only the particular syncbase value).

The syncbase element is qualified with one of the following time symbols:

begin
Specifies the begin time of the syncbase element.
end
Specifies the Active End of the syncbase element.

The time symbol can be followed by an offset value. The offset value specifies an offset from the time (i.e. the begin or active end) specified by the syncbase and time symbol. The offset is measured in local time on the parent time container. If the clock value is omitted, it defaults to "0".

No embedded white space is allowed between a syncbase element and a time-symbol. White space will be ignored before and after a "+" or "-" for a clock value. Leading and trailing white space characters (i.e. before and after the entire syncbase value) will be ignored.

Examples:

  begin="x.end-5s"      : Begin 5 seconds before "x" ends
  begin=" x.begin "     : Begin when "x" begins
  begin="x.begin + 1m"  : End 1 minute after "x" begins

Sync To Prev values

A sync-to-prev value has the following syntax:

 SyncToPrev-value   ::= ( "prev." Time-symbol )
                        ( S ("+"|"-") S Clock-value )? 

A sync-to-prev value is much like a syncbase value, except that the reserved token "prev" is used in place of the Syncbase-element term. The Time-symbol and optional Clock-value offset are as defined for syncbase values.

The previous element is the (timed) element that precedes this element within the parent time container (as reflected in the DOM). Note that the parent time container may not be the immediate parent of the current node, in some host documents. 

If there is no previous element (i.e. if the current element is the first timed child of the parent time container), then the begin of the parent time container is used as the syncbase (note that the Time-symbol is ignored in this case). The Clock-value offset is nevertheless added to the parent time container begin time, to yield the resulting time value.

@@This requires more complete examples, or we need to include them above somewhere. We need good examples of how this is used.

Examples:

  begin="prev.end-5s"      : Begin 5 seconds before the previous element ends
  begin=" prev.begin "     : Begin when the previous element begins
  begin="prev.begin + 1m"  : End 1 minute after the previous element begins

Event values

An event value has the following syntax:

  Event-value       ::= ( Eventbase-element "." )? Event-symbol 
                        ( S ("+"|"-") S Clock-value )? 
  Eventbase-element ::= Id-value

An Event value starts with an Eventbase-element term that specifies the event-base element. The event-base element is the element on which the event is observed. Given DOM event bubbling, the event-base element may be either the element that raised the event, or it may be an ancestor element on which the bubbled event can be observed. Refer to DOM-Level2-Events [DOM2Events] for details.
The "Id-value" is the value of an attribute declared to be of type ID (per the XML definition) in the host language, for the event-base element. This element must be another element contained in the host document.

If the Eventbase-element term is missing, the event-base element defaults to the element on which the attribute is specified (the current element). If this element has no associated layout (e.g. a time container in a SMIL document), then some UI events may not be defined (e.g. mouse events). Note that certain elements may specify a different default eventbase. E.g. the SMIL Animation elements (animate, animateMotion, etc.) specify that the default eventbase is the target element of the animation. See also [[SMIL Animation]].

The event value must specify an Event-symbol. This term specifies the name of the event that is raised on the Event-base element.  The host language designer must specify which types of events can be used. If an integrating language specifies no supported events, the event-base time value is effectively unsupported for that language.

The last term specifies an optional offset-value that is an offset from the time of the event. The offset is measured in local time on the parent time container. If this term is omitted, the offset is 0.

No embedded white space is allowed between an eventbase element and an event-symbol. White space will be ignored before and after a "+" or "-" for a clock value. Leading and trailing white space characters (i.e. before and after the entire eventbase value) will be ignored.

Note that it is not considered an error to specify an event that cannot be raised on the Event-base element (such as click for audio or other non-visual elements). Since the event will never be raised on the specified element, the event-base value is effectively ignored.  Similarly, if the host language allows dynamically created events (as supported by DOM-Level2-Events [DOM2Events]), all possible Event-symbol names cannot be specified, and so unrecognized names may not be considered errors. Host language specifications must include a description of legal event names, and/or allow any name to be used.

The semantics of event-based timing are detailed in the section Unifying Scheduling and Interactive Timing.

Examples:

 begin=" x.load "   : Begin when "load" is observed on "x"
 begin="x.focus+3s" : Begin 3 seconds after an "focus" event on "x"

Media marker values

Certain types of media can have associated marker values that associate a name with a particular point (i.e. a time) in the media. The media marker value provides a means of defining a begin or end time in terms of these marker values. Note that if the referenced id is not associated with a media element that supports markers, or if the specified marker name is not defined by the media element, the associated time may never be resolved.

  Media-Marker-value ::= Id-value ".marker(" S marker-symbol S ")" )

The marker symbol is a string that must conform to the definition of marker names for the media associated with the Id-value. 

Wallclock-sync values

Wallclock-sync values have the following syntax. The values allowed are based upon several of the "profiles" described in [DATETIME], which is based upon [ISO8601]. Exactly the components shown here must be present, with exactly this punctuation. Note that the "T" appears literally in the string, to indicate the beginning of the time element, as specified in [ISO8601].

wallclock-val  ::= "wallclock(" S (DateTime | WallTime)  S ")"
DateTime       ::= Date "T" WallTime
Date           ::= Years "-" Months "-" Days
WallTime       ::= (HHMM-Time | HHMMSS-Time)(TZD)?
HHMM-Time      ::= Hours24 ":" Minutes
HHMMSS-Time    ::= Hours24 ":" Minutes ":" Seconds ("." Fraction)?
Years          ::= 4DIGIT;
Months         ::= 2DIGIT; range from 01 to 12
Days           ::= 2DIGIT; range from 01 to 31
Hours24        ::= 2DIGIT; range from 00 to 23
4DIGIT         ::= DIGIT DIGIT DIGIT DIGIT
TZD            ::= "Z" | (("+" | "-") Hours24 ":" Minutes )
Complete date plus hours and minutes:

   YYYY-MM-DDThh:mmTZD (e.g. 1997-07-16T19:20+01:00)

Complete date plus hours, minutes and seconds:

   YYYY-MM-DDThh:mm:ssTZD (e.g. 1997-07-16T19:20:30+01:00)

Complete date plus hours, minutes, seconds and a decimal fraction of a second

   YYYY-MM-DDThh:mm:ss.sTZD (e.g. 1997-07-16T19:20:30.45+01:00)

Note that the Minutes, Seconds, Fraction, 2DIGIT and DIGIT syntax is as defined for Clock-values. Note that white space is not allowed within the date and time specification.

There are three ways of handling time zone offsets:

  1. Times are expressed in UTC (Coordinated Universal Time), with a special UTC designator ("Z").
  2. Times are expressed in local time, together with a time zone offset in hours and minutes. A time zone offset of "+hh:mm" indicates that the date/time uses a local time zone which is "hh" hours and "mm" minutes ahead of UTC. A time zone offset of "-hh:mm" indicates that the date/time uses a local time zone which is "hh" hours and "mm" minutes behind UTC.
  3. Times are expressed in local time, as defined for the presentation location. The local time zone of the end-user platform is used.

No embedded white space is allowed in wallclock values, although leading and trailing white space characters will be ignored.

The presentation engine must be able to convert wallclock-values to a time within the document. When the document begins, the current wallclock time must be noted - this is the document wallclock begin. Wallclock values are then converted to a document time by subtracting the document wallclock begin, and then converting the time to the element's parent time space as for any syncbase value, as though the syncbase were the document body. Note that the resulting begin or end time may be before the begin, or after end of the parent time container. This is not an error, but the time container constraints still apply. In any case, the semantics of the begin and end attribute govern the interpretation of the wallclock value.

Examples

The following examples all specify a begin at midnight on January 1st 2000, UTC

begin="wallclock(2000-01-01Z)"
begin="wallclock( 2000-01-01T00:00Z )"
begin="wallclock( 2000-01-01T00:00:00Z )"
begin="wallclock( 2000-01-01T00:00:00.0Z )"
begin="wallclock( 2000-01-01T00:00:00.0Z )"
begin="wallclock( 2000-01-01T00:00:00.0-00:00 )"

The following example specifies a begin at 3:30 in the afternoon on July 28th 1990, in the Pacific US time zone:

begin="wallclock( 1990-07-28T15:30-08:00 )"

The following example specifies a begin at 8 in the morning wherever the document is presented:

begin="wallclock( 08:00 )"

10.3.2 Time manipulations

New element controls for element time behavior are under discussion. Note that an Accessibility requirement for control of the playback speed is related to (but may end up with different syntax different from) the speed control.  In general, these time manipulations are suited to animation and non-linear or discrete media, rather than linear continuous media. Not all continuous media types will support time manipulations, e.g. streaming MPEG 1 video playing backwards. A fallback mechanism is described for these cases.

Three new attributes add support for timing manipulations to SMIL Timing, including control over the speed of an element, and support for acceleration and deceleration.  The impact on overall timing and synchronization is described. A definition is provided for reasonable fallback mechanisms for media players that cannot support the time manipulations.

Background

A common general application of timing supports animation. The recent integration of SMIL timing with SVG is a good example of the interest in this area.  Animation in the more general sense includes the time-based manipulation of basic transforms, applied to a presentation. Some of the effects supported include motion, scaling, rotation, color manipulation, as well as a host of presentation manipulations with a style framework like CSS.

Animation is often used to model basic mechanics. Many animation use-cases are difficult or nearly impossible to describe without a simple means to control pacing and to apply simple effects that emulate common mechanical phenomena. While it is possible to build these mechanisms into the animation behaviors themselves, this requires that every animation duplicate this support. This makes the framework more difficult to extend and customize. In addition, this model allows any animation behavior to introduce individual syntax and semantics for these mechanisms. This makes the authoring model much harder to learn, and complicates the job of any authoring tool designer as well. Finally, this model precludes the use of these mechanisms on structured animations (e.g. a time container with a series of synchronized animation behaviors).

A much simpler model for providing the necessary support centralizes the needed functionality in the timing framework. This allows all timed elements to support this functionality, and provides a consistent model for authors and tools designers. The most direct means to generalize pacing and related functionality is to transform the pacing of time for a given element. This is an extension of the transform that is implicitly performed to translate from the general document or presentation time space to the adjusted time space for the element (accounting for the begin time of the element, repeat functionality, etc.). Thus, to control the pacing of a motion animation, a transform is applied that adjusts the pacing of local time for the motion element. If time is scaled to advance faster than normal presentation time, the motion will appear to run faster. Similarly, if the pacing of time is dynamically adjusted, acceleration and deceleration effects are easily obtained. This model is detailed in the sections below.

Overview of support

Three basic time manipulations are proposed:

speed
Controls the pacing (or speed) of time. This is the basic manipulation upon which the others are built. The speed effectively scales the rate at which local time plays. As such, speed can modify the effective simple duration.
accelerate and decelerate
Dynamic manipulation of speed to simulate common mechanical motion. Acceleration and deceleration are crucial to motion, rotation, scaling and many other standard transforms. A simple model is presented to allow acceleration from rest at the beginning of the simple duration, and/or deceleration to rest at the end of the simple duration. This model has the advantage that it preserves the simple duration. The model is sometimes presented to authors as "Ease-In, Ease-Out".
autoReverse
Another very common mechanical phenomenon is that of a process that advances and reverses. Some examples include:

This support is often represented to authors as "Play Forwards, then Backwards". Because so many common use-cases apply repeat to the modified local time (as in the examples above), this function is modeled as modifying the simple duration. As such, autoReverse effectively doubles the simple duration. 

When the three features are combined, there is an inherent ordering that can be applied. The accelerate and decelerate features are applied locally on the simple duration, and have no side effects upon the active duration of the element. The autoReverse feature is applied to the simple duration, and doubles it. Thus, autoReverse wraps the effect of accelerate and decelerate. Speed has the broadest effect, scaling the progress of local time for the element. Taken from the perspective of a conversion from the document time-space to the local time-space, speed is applied earliest, autoReverse later and and then accelerate and decelerate are applied latest. See also Details of the time manipulations

Examples

The following motion animation will move the target twice as fast as normal:

<animateMotion dur="10s" repeatCount="2" speed="2.0" path= ... />

The target will move over the path in 5 seconds, and then repeat this motion. The active duration is thus 10 seconds.

The following rotation (a theoretical extension to the animation platform) will produce a simple pendulum swing on the target (assume that it is a pendulum shape with the transform origin at the top):

<animateRotate from="20deg" to="-20deg" dur="1s" repeatCount="indefinite"
        accelerate=".5" decelerate=".5" autoReverse="true" ... />

The pendulum swings through an arc in one second, and then back again in a second. It repeats indefinitely. The acceleration and deceleration are specified as a proportion of the simple duration (before autoReverse). As specified, the effect is to accelerate all the way through the downswing, and then decelerate all through the upswing. This produces a very realistic looking animation of real-world pendulum motion. The rotate element itself can be very simple, for example interpolating the rotation value in a transform matrix.

Attribute syntax

The speed attribute is supported on all timed elements. The argument value expresses a multiple of normal play speed that will be applied to the element and all time descendents. Thus 1.0 is normal speed, and speed="1" is a no-op, and speed="-1" means play backwards.

speed attribute

The speed attribute controls the local playback speed of an element, to speed up or slow down the effective rate of play. Note that the speed does not specify an absolute play speed, but rather is relative to the playback speed of the parent time container.  Thus if a <par> and one of its children both specify a speed of 50%, the child will play at 25% of normal playback speed .

speed
Defines the playback speed of element time. The value is specified as a multiple of normal (parent time container) play speed.

Legal values are signed floating point values. A value of 0 is not allowed.
The default is "1.0" (no modification of speed).

The details of the speed modification are described in Details of the time manipulations.

accelerate and decelerate attributes

These attributes define a simple acceleration and deceleration of element time, within the simple duration. This is useful for animation, motion paths, etc. The values are expressed as a proportion of the simple duration (i.e. between 0 and 1), and are defined such that the simple duration is not affected (although the normal play speed is increased to compensate for the periods of acceleration and deceleration). Note that these attributes apply to the simple duration; if these attributes are combined with repeating behavior, the acceleration and/or deceleration occurs within each repeat iteration.

The sum of accelerate and decelerate must not exceed 1. If it does, the deceleration value will be reduced to make the sum legal (i.e. the value of accelerate will be clamped to 1, and then the value of decelerate will be clamped to 1-accelerate ).

accelerate
Defines a simple acceleration of time for the element. Element time will accelerate from a rate of 0 at the beginning up to a run rate, over the course of the specified proportion of the simple duration. 
The default value is 0 (no acceleration).
Legal values are floating point values between 0 and 1 (inclusive).
decelerate
Defines a simple deceleration of time for the element. Element time will decelerate from a run rate down to 0 at the end of the simple duration, over the course of the specified proportion of the simple duration.
The default value is 0 (no deceleration).
Legal values are floating point values between 0 and 1 (inclusive).

The details of the accelerate and decelerate modifications are described in Details of the time manipulations.

Examples:

In this example, a motion path will accelerate up from a standstill over the first 2 seconds, run at a faster than normal rate for 4 seconds, and then decelerate smoothly to a stop during the last 2 seconds.  This makes an animation look more realistic. The animateMotion element is defined in the Animation section of SMIL Boston.

<img ...>
    <animateMotion dur="8s" accelerate=".25" decelerate=".25" .../>
</img>

In this example, the image will "fly in" from off-screen left , and then decelerate quickly during the last second to "ease in" to place. This assumes a layout model that supports positioning (a similar effect could be achieved by animation the position of a region in SMIL layout). The animate element is defined in the Animation section of SMIL Boston.

<img ...>
    <animate attributeName="left" dur="4s" decelerate=".25" 
             from="-1000" to="0" additive="sum" />
</img>
autoReverse attribute

This defines "play forwards then backwards" functionality. The use of autoReverse effectively doubles the simple duration. When combined with repeating behavior, each repeat iteration will play once forwards, and once backwards. This is useful for animation, especially for mechanical and pendulum motion.

autoReverse
Controls autoReverse playback mode.

Argument values are Booleans.
The default value is false (i.e. play normally).

The details of the autoReverse modification are described in Details of the time manipulations.

In this example, a motion path will animate normally for 5 seconds moving the element 20 pixels to the right, and then run backwards for 5 seconds (from 20 pixels to the right back to the original position), then forwards again and then backwards again, leaving the element at its original location. The active duration of the animation is 20 seconds. The animateMotion element is defined in the Animation section of SMIL Boston.

<img ...>
    <animateMotion by="20, 0" dur="5s" autoReverse="true" repeatCount="2"/>
</img>

Repeating elements

SMIL 1.0 introduced the repeat attribute, which is used to repeat a media element or an entire time container. SMIL Boston introduces two new controls for repeat functionality that supercede the SMIL 1.0 repeat attribute.  The new attributes, repeatCount and repeatDur, provide a semantic that more closely matches typical use-cases, and the new attributes provide more control over the duration of the repeating behavior.  The SMIL 1.0 repeat attribute is deprecated in SMIL Boston (it must be supported in SMIL document players for backwards compatibility).

Repeating an element causes the simple duration to be "played" several times in sequence. This will effectively copy or loop the contents of the element media (or an entire timeline in the case of a time container). The author can specify either how many times to repeat, using repeatCount, or how long to repeat, using repeatDur. Each repeat iteration is one instance of "playing" the simple duration.

If the simple duration is indefinite, the element cannot repeat. In this case, any repeatCount attribute is ignored, although a repeatDur attribute value can still constrain the active duration. See also Computing the Active Duration.

repeatCount and repeatDur attributes
repeatCount
Specifies the number of iterations of the simple duration. It can have the following attribute values:
numeric value
This is a (base 10) "floating point" numeric value that specifies the number of iterations. It can include partial iterations expressed as fraction values. A fractional value describes a portion of the simple duration. Values must be greater than 0.
"indefinite"
The element is defined to repeat indefinitely (subject to the constraints of the parent time container).
repeatDur
Specifies the total duration for repeat. It can have the following attribute values:
clock-value
Specifies the duration in parent local time to repeat the simple duration.
 
"indefinite"
The element is defined to repeat indefinitely (subject to the constraints of the parent time container).

At most one of repeatCount or repeatDur should be specified. If both are specified (and the simple duration is not indefinite), the active duration is defined as the minimum of the specified repeatDur, and the simple duration multiplied by repeatCount. For the purposes of this comparison, a defined value is considered to be "less than" a value of "indefinite". 

If the simple duration is indefinite, any repeatCount attribute will be ignored. Any repeatDur attribute value (other than "indefinite") will be used to constrain the indefinite simple duration. See also the examples below describing repeatDur and an indefinite simple duration.

If the simple duration is 0, any repeatCount attribute will be ignored. Any repeatDur attribute value will be used to define the active duration by showing the state of the element for the specified duration (this may be constrained by an end value - see Controlling active duration). See also the examples below describing repeatDur and a simple duration of 0).

@@ If simple duration is 0 and repeatCount is "indefinite" is the active duration 0 or indefinite?

If an element specifying audio media has a simple duration of 0 (e,g, because of clipBegin and clipEnd values), nothing should played even if the repeatDur specifies an active duration. The time model behaves according to the description, but no audio should be played.

These rules are included in the section Computing the Active Duration.

Examples

Need to create normative examples that demonstrate the new controls, and the interaction with implicit and explicit simple durations. Examples must also demonstrate the interaction of repeating behavior and time container constraints. 

@@ Need to add example of repeatCount < 1 and/or repeatDur < simple duration

In the following example, the 2.5 second simple duration will be repeated twice; the active duration will be 5 seconds.

<audio src="background.au" dur="2.5s" repeatCount="2" />

In the following example, the 3 second (implicit) simple duration will be repeated two full times and then the first half is repeated once more; the active duration will be 7.5 seconds.

<audio src="3second_sound.au" repeatCount="2.5" />

In the following example, the audio will repeat for a total of 7 seconds. It will play fully two times, followed by a fractional part of 2 seconds. This is equivalent to a repeatCount of 2.8.

<audio src="music.mp3" dur="2.5s" repeatDur="7s" />

Note that if the simple duration is zero (0) or indefinite, repeat behavior is not defined (but repeatDur still contributes to the active duration). In the following example the simple duration is 0 and indefinite respectively, and so the repeatCount is effectively ignored. Nevertheless, this is not considered an error. The active is equal to the simple duration: for the first element, the active duration is 0, and for the second element, the active duration is indefinite.

<img src="foo.jpg" repeatCount="2" />
<img src="bar.png" dur="indefinite" repeatCount="2" />

In the following example, the simple duration is 0, and so repeat behavior is not meaningful. However, the repeatDur determines the active duration. The effect is that the text is shown for 10 seconds.

<text src="intro.html" repeatDur="10s" />

In the following example, if the audio media is longer than the 5 second repeatDur, then the active duration will effectively cut short the simple duration.

<audio src="8second_sound.au" repeatDur="5s" />

The repeatCount and repeatDur attributes can also be used to repeat an entire timeline (i.e. a time container simple duration), as in the following example. The sequence has an implicit simple duration of 13 seconds.  It will begin to play after 5 seconds, and then will repeat the sequence of three images 3 times. The active duration is thus 39 seconds long.

<seq begin="5s" repeatCount="3" >
   <img src="img1.jpg" dur="5s" />
   <img src="img2.jpg" dur="4s" />
   <img src="img3.jpg" dur="4s" />
</seq>
SMIL 1.0 repeat (deprecated)

The SMIL 1.0 repeat attribute behaves in a manner similar to repeatCount, but it defines the functionality in terms of a sequence that contains the specified number of copies of the element without the repeat attribute. This definition has caused some confusion among authors and implementers. See also the SMIL 1.0 specification [SMIL10].

In particular, there has been confusion concerning the behavior of the SMIL 1.0 end attribute when used in conjunction with the repeat attribute. SMIL Boston complies with the common practice of having the end attribute define the element's simple duration when the deprecated repeat attribute is used. Only SMIL document players must support this semantic for the end attribute. Only a single SMIL 1.0 "end" value (i.e. an offset-value or a smil-1.0-syncbase-value, but none of the new SMIL Boston timing) is permitted when used with the deprecated repeat attribute. If repeat is used with repeatCount or repeatDur on an element, or if repeat is used with an illegal end value, the repeat value is ignored.

repeat Attribute
repeat
This attribute has been deprecated in SMIL Boston in favor of the new repeatCount and repeatDur attributes.
This causes the element to play repeatedly for the specified number of times. It is equivalent to a <seq> element with the stated number of copies of the element without the "repeat" attribute as children. All other attributes of the element, including any begin delay, are included in the copies.
Legal values are integer iterations, greater than 0, and "indefinite".

Note that elements that use the SMIL 1 repeat attribute with a value of "indefinite" are defined to end immediately after they begin. I.e. the active duration is effectively defined to be 0. This semantic is specific to the SMIL 1 repeat attribute, and does not apply to the new repeatCount and repeatDur attributes.

Controlling active duration

SMIL Boston provides an additional control over the active duration. The end attribute allows the author to constrain the active duration of the animation by specifying an end value using a simple offset, a time base, an event-base or DOM methods calls. The end attribute generally constrains the active duration that is otherwise defined by dur and any repeat behavior, although it will extend an implicit simple duration (see examples below). The rules for combining the attributes to compute the active duration are presented in the next section, Computing the active duration.

The normative syntax rules for each attribute value variant are described in the section Timing Attribute Values; a syntax summary is provided here as an aid to the reader.

end : smil-1.0-syncbase-value* | end-value-list | "indefinite"
Defines an end value for the animation that can constrain the active duration.
The attribute value is either a SMIL 1.0 syncbase declaration, a semi-colon separated list of values, or the special value "indefinite".
smil-1.0-syncbase-value* : "id(" id-ref ")" ( "(" ( "begin" | "end" | clock-value ) ")" )?
Describes a syncbase and an offset from that syncbase. The end value is defined relative to the begin or active end of another element.
*Note: Only compliant SMIL document players  are required to support the SMIL 1.0 syncbase-value syntax. Language designers integrating SMIL Boston Timing and Synchronization should not support this syntax.
end-value-list : end-value (";" end-value-list )?
A semi-colon separated list of end values. The interpretation of a list of end times is detailed below.
"indefinite"
The end value of the element will be determined by an endElement() method call.
The SMIL Timing and Synchronization DOM methods are described in the Supported Methods section.
 
end-value : ( offset-value | syncbase-value | syncToPrev-value | event-value | media-marker-value | wallclock-sync-value )
Describes the end value of the element.
 
offset-value : ( "+" | "-" )? clock-value
Describes the end value as an offset from an implicit syncbase. The definition of the implicit syncbase depends upon the element's parent time container. The offset is measured in local time on the parent time container.
syncbase-value : ( id-ref "." ( "begin" | "end" ) ) ( ( "+" | "-" ) clock-value )?
Describes a syncbase and an offset from that syncbase. The end value is defined relative to the begin or active end of another element.
syncToPrev-value : ( "prev.begin" | "prev.end" ) ( ( "+" | "-" ) clock-value )?
Specifies the previous timed sibling element, as reflected in the DOM, as the syncbase element, and describes the syncbase time and an offset from that syncbase. The end value is defined relative to the begin or active end of the previous sibling element.
event-value : ( id-ref "." )? ( event-ref  ) ( ( "+" | "-" ) clock-value )?
Describes an event and an optional offset that determine the end value. The end value is defined relative to the time that the event is raised. Events may be any event defined for the host language in accordance with [DOM2Events]. These may include user-interface events, event-triggers transmitted via a network, etc. Details of event-based timing are described in the section below on Unifying Event-based and Scheduled Timing.
media-marker-value : id-ref ".marker(" marker-name ")"
Describes the end value as a named marker time defined by a media element.
wallclock-sync-value : "wallclock(" wallclock-value ")"
Describes the end value as a real-world clock time. The wallclock time is based upon syntax defined in [ISO8601].

If end specifies an event-value or syncbase-value that is not resolved, the value of end is considered to be "indefinite" until resolved.

The end value can specify a list of times. This can be used to specify multiple "ways" or "rules" to end an element, e.g. if any one of several events is raised. A list of times can also define multiple end times that can correspond to multiple begin times, allowing the element to play more than once (this behavior can be controlled - see also Restarting elements). 

In general, the earliest time in the list determines the end value used in Computing the Active Duration. In the case where an element can begin multiple times, the end value used is the earliest end time after the current begin time. There are additional constraints upon the evaluation of the begin and end time lists, detailed in Evaluation of begin and end time lists.

The end value generally constrains all other values, and does not extend the active duration. However it will extend an implicit simple duration. In the following example, the dur attribute is not specified, and so the simple duration is defined to be the implicit media duration. In this case (and this case only) the value of end will extend the active duration if it specifies a duration greater than the implicit (media) duration.  For the difference between the implicit simple duration and the active duration, the ending state of the media (e.g. the last frame of video) will be shown. This only applies to visual media - aural media will simply stop playing, or will not play at all if the implicit simple duration is 0 (e,g, because of clipBegin and clipEnd values).

In the following example, the video will be shown for 8 seconds, and then the last frame will be shown for 2 seconds.

<video end="10s" src="8-SecondVideo.mpg" .../>

If the end value becomes resolved while the element is still active, and the resolved time is in the past, the element should end the active duration immediately. Time dependents defined relative to the end of this element should be resolved using the computed active end (which may be in the past), and not the observed active end. These cases arise from the use of negative offsets in the sync-base and event-base forms, and authors should be aware of the complexities this can introduce. See also Handling negative offsets.

In the following example, the active duration will end at the earlier of 10 seconds, or the end of the "foo" element. This is particularly useful if "foo" is defined to begin or end relative to an event.

<audio src="foo.au" dur="2s" repeatDur="10s" 
       end="foo.end" .../>

In the following example, the active duration will end at 10 seconds, and will cut short the simple duration defined to be 20 seconds. The effect is that only the first half of the element is actually played. For a simple media element, the author could just specify this using the dur attribute. However in other cases, it is sometimes important to specify the simple duration independent of the active duration.

<par>
   <audio src="music.au" dur="20s" end="10s" ... />
</par>

In the following example, the element begins when the user clicks on the "gobtn" element. The active duration will end 30 seconds after the parent time container begins. Note that if the user has not clicked on the target element before 30 seconds elapse, the element will never begin.

<par>
   <audio src="music.au" begin="gobtn.click" repeatDur="indefinite"
          end="30s" ... />
</par>

The defaults for the event syntax make it easy to define simple interactive behavior. The following example stops the image when the user clicks on the element.

<image src="image.jpg" end="click" />

Using end with an event value enables authors to end an element based on either an interactive event or a maximum active duration. This is sometimes known as lazy interaction.

In this example, a presentation describes factory processes. Each step is a video, and set to repeat 3 times to make the point clear. Each element can also be ended by clicking on the video, or on some element "next" that indicates to the user that the next step should be shown.

<seq>
  <video dur="5s" repeatCount="3" end="click; next.click" .../>
  <video dur="5s" repeatCount="3" end="click; next.click" .../>
  <video dur="5s" repeatCount="3" end="click; next.click" .../>
  <video dur="5s" repeatCount="3" end="click; next.click" .../>
  <video dur="5s" repeatCount="3" end="click; next.click" .../>
</seq>

In this case, the active end of each element is defined to be the earlier of 15 (5s dur * 3 repeats) seconds after it begins, or a click on "next". This lets the viewer sit back and watch, or advance the presentation at a faster pace.

Computing the active duration

This section still needs work - and will change in the next day or two. 

The table in Figure 3 defines a set of "forms" for the simple duration. These forms are used in the table in Figure 4 to delineate the possible combinations of attributes that can contribute to the active duration.
dur Implicit media duration Simple Duration Form
number * dur value explicit finite
"indefinite" * indefinite "indefinite"
unspecified 0 media dur implicit 0
unspecified number media dur implicit finite
unspecified continuous indefinite indefinite implicit indefinite
unspecified unresolved indefinite unresolved

Figure 3 - Describing the simple duration

@@There are two forms of table 4 presented. We are trying to decide which presents the information more effectively. You be the judge which makes more sense, which is clearer, and which we should include. Both use essentially the same terminology, and present the same semantics (i.e. there should be no discrepancy in the semantics described, but rather only differences in how the information is presented).

The table in Figure 4 shows the semantics of all possible combinations of simple duration, repeatCount and repeatDur, and end. The following conventions are used in the table:

Note in particular that where a value specifies "unresolved", that the table will be reevaluated (generally using a different row) if and when the associated value becomes resolved. For example if the element specifies:

<audio src="5-second.au" end="foo.click" />

The active duration is initially defined as equal to the (implicit finite) simple duration. If the user clicks on "foo" before 5 seconds, the end value becomes resolved and the active duration table is re-evaluated to be MIN( d, end-B) which causes the element to end at the time of the click. 

Some of the rules and results that are implicit in the table, and that should be noted in particular are:

Note that while the active duration is computed according to the rules in the table, the parent time container places constraints upon the active duration of all children. These constraints may cut short the active duration of any child, and so override the definition described here. For more information, see the section Time Container constraints on child durations.

The following symbols are used in the table as a shorthand:

B
The begin of an element.
d
The simple duration of an element.

@@This is a form of the table that has more rows, but may be clearer in delineating the different cases. It uses the term "unresolved indefinite" for end values, which is semantically equivalent to "unresolved" in the second table.

@@Note that in both table the handling of explicit 0 is a bit messy. This is only needed if we want to say that simple Dur of 0 and repeatCount of "indefinite" yields a 0 active duration, rather than an indefinite AD. The tables would simplify if we went with the latter.

Where the Active duration has a "+" suffix, the value may be reevaluated at some point. The footnotes within the row indicate when the value will be reevaluated. Note that when a value is re-evaluated, a different row in the table may apply.
Simple duration(d) repeatCount repeatDur end Active Duration
implicit 0 or
explicit 0
(ignored) unspecified unspecified 0
implicit 0 or
explicit 0
(ignored) "indefinite" unspecified indefinite
explicit finite or implicit finite unspecified unspecified unspecified d
explicit finite or implicit finite unspecified unspecified unresolved indefinite1 or "indefinite"2 d+
"indefinite" or implicit indefinite (ignored) "indefinite" or unspecified unspecified indefinite
"indefinite", implicit 0, or implicit indefinite (ignored) "indefinite" or unspecified unresolved indefinite1 or "indefinite"2 indefinite+
unresolved3 number or unspecified "indefinite" or unspecified unresolved indefinite1 or "indefinite"2 indefinite+
unresolved3 number "indefinite" unspecified indefinite+
unresolved3 (ignored) unspecified unspecified indefinite+
explicit finite or implicit finite number "indefinite" or unspecified unspecified repeatCount*d
explicit finite or implicit finite number "indefinite" or unspecified unresolved indefinite1 or "indefinite"2 repeatCount*d+
(ignored) "indefinite" or unspecified number unspecified repeatDur
(ignored) "indefinite" or unspecified number unresolved indefinite1 or "indefinite"2 repeatDur+
"indefinite", implicit 0, or implicit indefinite (ignored) number unspecified repeatDur
"indefinite", implicit 0, or implicit indefinite (ignored) number unresolved indefinite1 or "indefinite"2 repeatDur+
unresolved3 number number unspecified, unresolved indefinite1, or "indefinite"2 repeatDur+
explicit finite unspecified unspecified number MIN(d, end-B)
explicit finite unspecified "indefinite" number end-B
"indefinite", implicit 0, or implicit indefinite (ignored) "indefinite" or unspecified number end-B
unresolved3 number "indefinite" or unspecified number end-B+
(ignored) "indefinite" "indefinite" or unspecified number end-B
implicit finite or unresolved unspecified "indefinite" or unspecified number end-B
explicit finite or implicit finite "indefinite" "indefinite" or unspecified unspecified indefinite
(ignored) unspecified "indefinite" unresolved indefinite1 or "indefinite"2 indefinite+
(ignored) "indefinite" or unspecified "indefinite" unspecified indefinite
(ignored) "indefinite" "indefinite" or unspecified unresolved indefinite1 or "indefinite"2 indefinite+
explicit finite or implicit finite number number unspecified MIN(repeatDur, repeatCount*d)
explicit finite or implicit finite number number unresolved indefinite1 or "indefinite"2 MIN(repeatDur, repeatCount*d)+
explicit finite or implicit finite number number number MIN(end-B, repeatDur, repeatCount*d)
(ignored) "indefinite" or unspecified number number MIN(end-B, repeatDur)
explicit finite or implicit finite number "indefinite" or unspecified number MIN(end-B, repeatCount*d)
"indefinite", implicit 0, or implicit indefinite (ignored) number number MIN(end-B, repeatDur)
unresolved3 number number number MIN(end-B, repeatDur)+

Figure. 4a: Computing the active duration for different combinations of the simple duration, repeatDur and repeatCount, and end.

1 reevaluate if/when end becomes resolved by event
2 reevaluate if/when end becomes resolved by DOM
3 reevaluate when simple duration is resolved

@@This is a form of the table that has fewer rows, and does not call out the values that may be re-evaluated.

Note that any row that includes an "unresolved" value may be re-evaluated at some point (i.e. if the value becomes resolved). Note that when a value is re-evaluated, a different row in the table may apply.
Simple
duration d
repeatCount repeatDur end Active Duration
expl. finite unspecified unspecified unspecified, "indefinite" or unresolved d
implicit 0, or
implicit finite
unspecified unspecified unspecified d
"indefinite", 
impl. indefinite or unresolved
(ignored) unspecified or "indefinite" unspecified indefinite
implicit 0 or
explicit 0
"indefinite" unspecified unspecified 0
"indefinite", implicit finite, 
impl. indefinite or unresolved
unspecified or "indefinite" unspecified or "indefinite"  "indefinite" or unresolved indefinite
expl. finite, implicit 0, or
implicit finite
expl. finite unspecified or "indefinite" unspecified, "indefinite" or unresolved repeatCount*d
expl. finite or
implicit finite
unspecified or "indefinite" expl. finite unspecified, "indefinite" or unresolved repeatDur
implicit 0, "indefinite", 
impl. indefinite or unresolved
(ignored) expl. finite unspecified, "indefinite" or unresolved repeatDur
expl. finite unspecified unspecified expl. finite MIN( d, end-B )
expl. finite "indefinite" unspecified or "indefinite" expl. finite end-B
expl. finite unspecified or "indefinite" "indefinite" expl. finite end-B
implicit finite unspecified or "indefinite" unspecified or "indefinite" expl. finite end-B
implicit 0,
"indefinite", 
impl. indefinite or unresolved
(ignored) unspecified or "indefinite" expl. finite end-B
(anything except a 0 value) "indefinite" unspecified or "indefinite" unspecified, "indefinite" or unresolved indefinite
(anything) unspecified or "indefinite" "indefinite" unspecified, "indefinite" or unresolved indefinite
expl. finite or
implicit finite
expl. finite expl. finite unspecified, "indefinite" or unresolved MIN( repeatCount*d, repeatDur )
expl. finite or
implicit finite
expl. finite expl. finite expl. finite MIN( repeatCount*d, repeatDur, ( end-B ))
* unspecified or "indefinite" expl. finite expl. finite MIN( repeatDur, ( end-B ))
implicit 0,
"indefinite", 
impl. indefinite or unresolved
(ignored) expl. finite expl. finite MIN( repeatDur, ( end-B ))
expl. finite or
implicit finite
expl. finite unspecified or "indefinite" expl. finite MIN( repeatCount*d,  ( end-B ))

Figure. 4b: Computing the active duration for different combinations of the simple duration, repeatDur and repeatCount, and end.

It is possible to combine scheduled and interactive timing, e.g.:

<par dur="30s">
   <img id="mutebutton" src="mute.jpg"/>
   <text  src="description.html" />
   <audio src="audio.au" end="mutebutton.click"/>
</par>

The image and the text appear for the specified duration of the <par> (30 seconds). The audio will stop early if the image is clicked before the active end of the audio (which in this case is the duration of the actual audio media "audio.au").

It is possible to declare both a scheduled duration, as well as an event-based active end.  This facilitates what are sometimes called "lazy interaction" use-cases, such as a slideshow that will advance on its own, or in response to user clicks:

<seq>
   <img src="slide1.jpg" dur="10s" end="click" />
   <img src="slide2.jpg" dur="10s" end="click" />
   <img src="slide3.jpg" dur="10s" end="click" />
   <!-- etc., etc. -->
</seq>

In this case, the active end of each element is defined to be the earlier of the specified duration, or a click on the element. This lets the viewer sit back and watch, or advance the slides at a faster pace.

Freezing elements

By default when an element's active duration ends, it is no longer presented (or its effect is removed from the presentation, depending upon the type of element). Freezing an element extends it, using the last state presented in the active duration. This can be used to fill gaps in a presentation, or to extend an element as context in the presentation (e.g. with additive animation - see [SMIL-ANIMATION]).

The fill attribute allows an author to specify that an element should be extended beyond the active duration by freezing the final state of the element. For discrete media, the media is simply displayed as it would be during the active duration. For continuous media, the "frame" that corresponds to the end of the active duration is shown. For algorithmic media like animation, the value defined for the end of the active duration should be used.  The syntax of the fill attribute is the same as in SMIL 1.0, with two extensions:

fill : ( "remove" | "freeze" | "hold" | "transition" )
This attribute can have the following values:
remove
Specifies that the element will not extend past the end of the active duration.
freeze
Specifies that the element will extend past the end of the active duration by "freezing" the element state at the active end. The parent time container of the element determines how long the element is frozen (as described below).
hold
Setting this to "hold" has the same effect as setting to "freeze", except that the element is always frozen to extend to the end of the simple duration of the parent time container of the element (independent of the type of time container).
transition
Setting this to "transition" has the same effect as setting to "freeze", except that the element is removed at the end of the transition. The element is frozen to extend to its active duration + the time for transition within same time container. @@Need to refine this definition to be clearer where the transition duration comes from, etc. Need to specify that this only applies if there is a transition specified as per the Transitions module.

This attribute only has an effect on visual media elements. Non-visual media elements (audio) should ignore this.

Note that <a> and <area> elements are still sensitive to user activation (e.g. clicks) when frozen. See also the SMIL 1.0 specification [SMIL10].

The default value of the fill attribute depends on the element type, and whether the element specifies any of the attributes that define the simple or active duration. 

An element with fill="freeze" is extended according to the parent time container:

The fill attribute can be used to maintain the value of an media element after the active duration of the element ends:

<par endSync="last">
   <video src="intro.mpg" begin= "5s" dur="30s" fill="freeze" />
   <audio src="intro.au"  begin= "2s" dur="40s"/>
</par>

The video element ends 35 seconds after the parent time container began, but the video frame at 30 seconds into the media remains displayed until the audio element ends. The attribute "freezes" the last value of the element for the remainder of the time container's simple duration.

This functionality is also useful to keep prior elements on the screen while the next item of a <seq> time container prepares to display as in this example:

<seq>
   <video id="v1" fill="freeze" src.../>
   <video id="v2" begin="2s" src.../>
</seq>

The first video is displayed and then the last frame is frozen for 2 seconds, until the next element begins. Note that if it takes additional time to download or buffer video "v2" for playback, the first video "v1" will remain frozen until video "v2" actually begins.

@@Need a good example of freeze on a time container, showing both how it extends any frozen children, as well as how it cuts off and freezes any children that were active at the end.

Restarting elements

When an element is defined to begin at a simple offset (e.g. begin="5s" ), there is an unequivocal time when the element begins. However, if an element is defined to begin relative to an event (e.g. begin="foo.click" ), the event can happen at any time, and moreover can happen more than once (e.g. if the user clicks on "foo" several times). In some cases, it is desirable to restart an element if a second begin event is received. In other cases, an author may want to preclude this behavior.

In SMIL Boston, an element can have a list of begin values. In some cases, the intent is to begin at the earliest of the specified times (e.g. when the user clicks on any one of several images). In other cases, the intent is that the element restart when any of the begin times is encountered.

In addition, if an element is defined to begin relative to when another element begins (using the syncbase- value syntax), the syncbase element can restart. The restart attribute is used to control the restart behavior of an element.

restart = "always | whenNotActive | never"
always
The element can be restarted at any time. 
This is the default value.
whenNotActive
The element can only be restarted when it is not active (i.e. it can be restarted after the active end). Attempts to restart the element during its active duration are ignored.
never
The element cannot be restarted for the remainder of the current simple duration of the parent time container. 

The default value for the restart attribute is "always". This may not be a sensible default in all documents.  In particular SMIL Boston documents with streaming media may want restart="never" set on all of the elements. In order to not require restart="never" be added to every media element in the document, the WG is considering ways to override the default and set a new default for the document.

Note that there are several ways that an element may be restarted. The behavior (i.e. to restart or not) in all cases is controlled by the restart attribute. The different restart cases are:

When an element restarts, the primary semantic is that it behaves as though this were the first time the element had begun, independent of any earlier behavior. Any effect of an element playing earlier is no longer applied, and only the current begin "instance" of the element is reflected in the presentation.

The synchronization relationship between an element and its parent time container is re-established when the element restarts. A new synchronization relationship may be defined. See also Controlling runtime synchronization behavior.

As with any begin time, if an element is scheduled to restart after the end of the parent time container simple duration, the element will not restart.

Note that if the parent time container (or any ascendant time container) repeats or restarts, any state associated with restart="never" will be reset, and the element can begin again normally. See also Resetting element state.

The restart setting for an animation is evaluated when the syncbase element restarts, when the eventbase event happens, or when the DOM method call (e.g. beginElement()) happens. For example:

<img id="go_btn" dur="indefinite" .../>
<video id="foo" begin="go_btn.click" ... />
<audio id="bar" begin="foo.begin+2s" dur="10s" 
       restart="whenNotActive" ..." />

If the user clicks on the "go_btn" image at 5 seconds, element "foo" will begin, and element "bar" will be scheduled to begin at 7 seconds. If the user clicks the image again at 6 seconds, "foo" would restart and "bar" would be rescheduled to start at 8 seconds. If the user clicks again at 9 seconds, "foo" would restart but "bar" will not, as it is set to allow restart only when it is not active.

If an element is currently active when a restart is scheduled, the element should end immediately (at the time of the restart evaluation).  It should not continue playing until the rescheduled begin actually happens. For example:

<img id="go_btn" dur="indefinite" .../>
<video id="foo" begin="go_btn.click" .../>
<audio id="bar" begin="foo.begin+2s" dur="10s" />

If the user clicks the image once at 3 seconds, "foo" begins to play and 2 seconds later "bar" will play as well. If the user clicks again at 6 seconds, "foo" restarts immediately, "bar" is stopped, and "bar" will restart at 8 seconds.

Note that using restart can also allow the author to define a single UI event to both begin and end an element, as follows:

<img id="toggle" dur="indefinite" .../>
<audio id="foo" begin="toggle.click" end="toggle.click"
   repeatDur="indefinite" restart="whenNotActive" .../>

If "foo" were defined with the default restart behavior "always", a second click on the image would simply restart the audio. However, since the second click cannot restart the audio when restart is set to "whenNotActive", the click will just end the active duration and stop the audio. This is sometimes described as "toggle" activation. See also Unifying scheduling and interactive timing.

Note that in SMIL Language documents, a SMIL element cannot be visible before it begins so having a begin="click" means it won't ever begin. In languages with timeAction support, this may not be the case. For example, the following is reasonable:

<span begin="click" end="click" timeAction="class:highlight">
  Click here to highlight. Click again to remove highlight.
</span>

See also "The SMIL Integration Module" for details of language profiles.

Using restart for toggle activation

A common use-case requires that the same UI event is used begin an element and to end the active duration of the element. This is sometimes described as "toggle" activation, because the UI event toggles the element "on" and "off".  The restart attribute can be used to author this, as follows:

<img id="foo" begin="bar.click" end="bar.click"
              restart="whenNotActive" ... />

If "foo" were defined with the default restart behavior "always", a second click on the "bar" element would simply restart the element. However, since the second click cannot restart the element when restart is set to "whenNotActive", the element ignores the "begin" specification of the "click" event. The element can then use the "click" event to end the active duration and stop the element.

This is based upon the event sensitivity semantics described in Unifying Scheduling and Interactive Timing.

10.3.3 Time containers

SMIL Boston specifies three time containers: <par>, <seq>, and <excl>.

The par time container

<par>
A <par> container defines a simple parallel time grouping in which multiple elements can play back at the same time.

The default syncbase of the child elements of a <par> is the begin of the <par>. This is the same element introduced with SMIL 1.0.

The <par> element supports all element timing.

The seq time container

<seq>
A <seq> container defines a sequence of elements in which elements play one after the other.  

This is the same element introduced with SMIL 1.0, but the semantics (and allowed syntax) for child elements of a <seq> are clarified. The default syncbase of a child element is the active end of the previous element. Previous means the element that occurs before this element in the sequence time container. For the first child of a sequence (i.e. where no previous sibling exists), the default syncbase is the begin of the sequence time container. 

Child elements may define an offset from the syncbase, but may not define a different syncbase (i.e. they may not define a begin time relative to another element, or to an event). Thus, the syncbase of child elements of a sequence is always the same as the default syncbase. Note however that child elements may define an end that references other syncbases, event-bases, etc.

For children of a sequence, only the offset begin values are legal. None of the following begin values may be used:

disallowed begin-values:
( syncbase-value | syncToPrev-value | event-value | media-marker-value | wallclock-sync-value )

No constraints are placed upon the end argument values.

The <seq> element itself supports all element timing.

When a hyperlink traversal targets a child of a <seq>, and the target child is not currently active, part of the seek action must be to enforce the basic semantic of a <seq> that only one child may be active a given time. For details, see Hyperlinks and timing and specifically Implications of beginElement() and hyperlinking for seq and excl time containers.

The excl time container

SMIL Boston defines a new time container, <excl>.

<excl>
This defines a time container with semantics based upon par, but with the additional constraint that only one child element may play at any given time. If any element begins playing while another is already playing, the element that was playing is either paused or stopped. Elements in an <excl> are grouped into categories, and the pause/interruption behavior of each category can be controlled using the new grouping element <priorityClass>.

The default syncbase of the child elements of the <excl> is indefinite (i.e. equivalent to begin="indefinite").

The <excl> element itself supports all element timing.

With the <excl> time container, common use cases that were either difficult, or impossible, to author are now easier and possible to create. The <excl> time container is used to define a mutually exclusive set of clips, and to describe pausing and resuming behaviors among these clips. Examples include:

interactive playlist
A selection of media clips is available for the user to choose from, only one of which plays at a time. A new selection replaces the current selection.
audio descriptions
For visually impaired users, the current video is paused and audio descriptions of the current scene are played. The video resumes when the audio description completes.
interactive video sub-titles
Multiple language sub-titles are available for a video. Only one language version can be shown at a time with the most recent selection replacing the previous language choice, if any.

The interactive playlist use case above could be accomplished using a <par> whose sources have interactive begin times and end events for all other sources. This would require a prohibitively long list of values for end to maintain. The <excl> time container provides a convenient short hand for this - the element begin times are still interactive, but the end events do not need to be specified because the <excl>, by definition, only allows one child element to play at a time.

The audio descriptions use case is not possible without the pause/resume behavior provided by <excl> and <priorityClass>. This use case would be authored with a video and each audio description as children of the <excl>. The video element would be scheduled to begin when the <excl> begins and the audio descriptions, peers of the video element, would start at scheduled begin times or in response to stream events raised at specific times.

The dynamic video sub-titles use case requires the "play only one at a time" behavior of <excl>. In addition, the child elements are declared in such as way so to preserve the sync relationship to the video:

<par>
   <video id="vid" .../>
   <excl>
      <par begin="englishBtn.click" >
         <audio begin="vid1.begin" src="english.au" />
      </par>
      <par begin="frenchBtn.click" >
         <audio begin="vid1.begin" src="french.au" />
      </par>
      <par begin="swahiliBtn.click" >
         <audio begin="vid1.begin" src="swahili.au" />
      </par>
   </excl>
</par>

The three <par> elements are children of the <excl>, and so only one can play at a time. The audio child in each <par> is defined to begin when the video begins. Each  audio can only be active when the parent time container (<par>) is active, but the begin still specifies the synchronization relationship. This means that when each <par> begins, the audio will start playing at some point in the middle of the audio clip, and in sync with the video.

The <excl> time container is useful in many authoring scenarios by providing a declarative means of describing complex clip interactions.

Pause behavior

Using priority classes to control the pausing behavior of children of the <excl> allows the author to group content into categories of content, and then to describe rules for how each category will interrupt or be interrupted by other categories. Attributes of the new grouping element <priorityClass> describe the intended interactions. 

The <priorityClass> element is transparent to timing, and does not participate in or otherwise affect the normal timing behavior of its children (i.e. it only defines how elements interrupt one another). Child elements of the <priorityClass> elements are time-children of the <excl> element (i.e. the parent <excl> of the <priorityClass> elements).

Each <priorityClass> element describes a group of children, and the behavior of those children when interrupted by other time-children of the <excl>. The behavior is described in terms of peers, and higher and lower priority elements. Peers are those elements within the same <priorityClass> element. The priorityClass elements are assigned priority levels based upon the order in which they are declared within the excl. The first <priorityClass> element has highest priority, and the last has lowest priority.

When one element within the <excl> begins (or would normally begin) while another is already active, several behaviors may result. The active element may be paused or stopped, or the interrupting element may be deferred, or simply blocked from beginning. When elements are paused or deferred, they are added to a queue of pending elements. When an active element completes its active duration, the first element (if any) in the queue of pending elements is made active. The queue is ordered according to rules described in Pause queue semantics.

The careful choice of defaults makes common use cases very simple. See the examples below.

<priorityClass>
Defines a group of <excl> time-children, and the pause/interrupt behavior of the children. If a <priorityClass> element appears as the child of an <excl>, then the <excl> can only contain <priorityClass> elements (i.e. the author cannot mix timed children and <priorityClass> elements within an <excl>).

The <excl> element content model is thus (assume that container content is an updated version of the SMIL 1.0 DTD entity):

<!ENTITY % excl-content "priorityClass* | %container-content;">
<!ELEMENT excl (%excl-content;)*>

The <priorityClass> element supports a simple set of attributes to describe the behavior of its children:

<!ELEMENT priorityClass %container-content;>
<!ATTLIST priorityClass
  id     ID   #IMPLIED 
  peers  ( stop | pause | defer | never )  'stop'
  higher ( stop | pause )                  'pause'
  lower  ( defer | never )                 'defer'
>

If no <priorityClass> element is used, all the children of the <excl> are considered to be peers, with the default peers behavior "stop".

Note that the rules define the behavior of the currently active element and the interrupting element. Any elements in the pause queue are not affected (except that their position in the queue may be altered by new queue insertions).

peers = " stop | pause | defer | never "

Controls how child elements of this <priorityClass> will interrupt one another.
Legal values for the attribute are:
stop
If a child element begins while another child element is active, the active element is simply stopped.
This is the default for peers.
pause
If a child element begins while another child element is active, the active element is paused and will resume when the new (interrupting) element completes its active duration (subject to the constraints of the excl time container). The paused element is added to the pause queue.
defer
If a child element attempts to (i.e. would normally) begin while another child element is active, the new (interrupting) element is deferred until the active element completes its active duration. This can also be thought of as placing the new element in the pause queue, paused at its very beginning.
never
If a child element attempts to (i.e. would normally) begin while another child element is active, the new (interrupting) element is prevented from beginning. The begin of the new (interrupting) element is ignored.

higher = " stop | pause "

Controls how elements with higher priority will interrupt child elements of this <priorityClass>.
Legal values for the attribute are:
stop
If a higher priority element begins while a child element of this <priorityClass> is active, the active child element is simply stopped.
pause
If a higher priority element begins while a child element of this <priorityClass> is active, the active child element is paused and will resume when the new (interrupting) element completes its active duration (subject to the constraints of the excl time container). The paused element is added to the pause queue.
This is the default for the higher attribute.

lower = " defer | never "

Controls how elements defined with lower priority will interrupt child elements of this <priorityClass>.
Legal values for the attribute are:
defer
If a lower priority element attempts to (would normally) begin while a child element of this <priorityClass> is active, the new (interrupting) element is deferred until the active element completes its active duration. This can also be thought of as placing the new element in the pause queue, paused at its very beginning. The rules for adding the element to the queue are described below.
This is the default for the lower attribute.
never
If a lower priority element attempts to begin while a child element of this <priorityClass> is active, the new (interrupting) element is prevented from beginning. The begin of the new (interrupting) element is ignored, and it is not added to the queue.

When an element begin is blocked (ignored) because of the "never" attribute value, the blocked element does not begin in the time model. The time model should not propagate begin or end activations to time dependents, nor should it raise begin or end events.

excl and priorityClass Examples

Note that because of the defaults, the simple cases work without any additional syntax. In the basic case, all the elements default to be peers, and stop one another:

<excl dur="indefinite">
   <audio id="song1" .../>
   <audio id="song2" .../>
   <audio id="song3" .../>
   ...
   <audio id="songN" .../>
</excl>

is equivalent to the following with explicit settings:

<excl dur="indefinite">
   <priorityClass peers="stop">
     <audio id="song1" .../>
     <audio id="song2" .../>
     <audio id="song3" .../>
     ...
     <audio id="songN" .../>
   </priorityClass>
</excl>

If the author wants elements to pause rather than stop, the syntax is:

<excl dur="indefinite">
   <priorityClass peers="pause">
     <audio id="song1" .../>
     <audio id="song2" .../>
     <audio id="song3" .../>
     ...
     <audio id="songN" .../>
   </priorityClass>
</excl>

The audio description use case for visually impaired users would look very similiar to the previous example:

<excl dur="indefinite">
   <priorityClass peers="pause">
     <video id="main_video" .../>
     <audio id="scene1_description" begin="20s"  dur="30s".../>
     <audio id="scene2_description" begin="2min" dur="30s" .../>
     ...
     <audio id="sceneN_description" .../>
   </priorityClass>
</excl>

This example shows a more complex case of program material and several commercial insertions. The program videos will interrupt one another. The ads will pause the program, but will not interrupt one another.

<excl dur="indefinite">
   <priorityClass id="ads" peers="defer">
     <video id="advert1" .../>
     <video id="advert2" .../>
   </priorityClass>
   <priorityClass id="program" peers="stop" higher="pause">
     <video id="program1" .../>
     <video id="program2" .../>
     <video id="program3" .../>
     <video id="program4" .../>
   </priorityClass>
</excl>
Pause queue semantics

Elements that are paused or deferred are placed in a priority-sorted queue of waiting elements. When an active element ends its active duration and the queue is not empty, the first (i.e. highest priority) element in the queue is pulled from the queue and resumed or activated.

The queue semantics are described as a set of invariants and the rules for insertion and removal of elements. For the purposes of discussion, the child elements of a <priorityClass> element are considered to have the priority of that <priorityClass>, and to have the behavior described by the peers, higher and lower attributes on the <priorityClass> parent.

Queue invariants
  1. The queue is sorted by priority, with higher priority elements before lower priority elements.
  2. An element may not appear in the queue more than once.
  3. An element may not simultaneously be active and in the queue.
Element insertion and removal
  1. Elements are inserted into the queue sorted by priority (by invariant 1).
    1. Paused elements are inserted before elements with the same priority.
    2. Deferred elements are inserted after elements with the same priority.
  2. Where the semantics define that an active element must be paused, the element is paused at the current local time (position) when placed on the queue. When a paused element is pulled normally from the queue, it will resume from the point at which it was paused.
  3. Where the semantics define that an element must be deferred, the element is inserted in the queue, in an inactive (idle) state. When the element is pulled normally from the queue, it will begin (i.e. be activated).
  4. When an element is placed in the queue any previous instance of that element is removed from the queue (by invariant 2).
  5. When the active child (i.e. time-child) of an excl ends normally (i.e. not when it is stopped by another, interrupting element), the element on the front of the queue is pulled off the queue, and resumed or begun (according to rule 2 or 3).

Note that if an element is active and restarts (subject to the restart rule), it does not interrupt itself in the sense of a peer interrupting it. Rather, it simply restarts and the queue is unaffected.

Time dependency and pause/defer semantics

@@This section has more general impact that just in excl, and should perhaps be moved elsewhere. E.g. when an element is paused with the pause() DOM method, this semantic will apply as well.

When an element is paused, a resolved end time for the element may no longer be resolved (although it could be computed in some cases). This change in the end time must be propagated to any sync arc time dependents defined relative to the active end of the paused element. See also the "Propagating Times" section in the Timing draft.

When an element is deferred, sync-arc time-dependents of the element are resolved when the element actually begins, and not when it is placed in the queue. Similarly, the begin event is not raised until the element begins.

Scheduled begin times and <excl>

Although the default begin value for children of an <excl> is indefinite, scheduled begin times are permitted. Scheduled begin times on children of the <excl> cause the element to begin at the specified time, pausing or stopping other siblings depending on the priorityClass settings (and default values). 

To specify that a child of the <excl> should begin playing by default (i.e., when the <excl> begins), specify begin="0" on that child element. If children of an <excl> are scheduled to begin at the same time, the evaluation proceeds in document order. For each element in turn, the priorityClass semantics are considered, and elements may be paused, deferred or stopped. For example:

<excl>
   <img src="image1.jpg" begin="0s" dur="5s"/>
   <img src="image2.jpg" begin="0s" dur="5s"/>
   <img src="image3.jpg" begin="0s" dur="5s"/>
</excl>

Given the default semantics for excl, the first image will begin and then be immediately stopped by the second image, which will in turn be immediately stopped by the third image. The net result is that only the third image is seen, and it lasts for 5 seconds. Note that the begin and end events for the first two images are raised and propagated to all time dependents. If the behavior is set to "pause" as in this example, the declared order is effectively reversed:

<excl>
   <priorityClass peers="pause">
      <img src="image1.jpg" begin="0s" dur="5s"/>
      <img src="image2.jpg" begin="0s" dur="5s"/>
      <img src="image3.jpg" begin="0s" dur="5s"/>
   </priorityClass>
</excl>

In this case, the first image will begin and then be immediately paused by the second image, which will in turn be immediately paused by the third image. The net result is that the third image is seen for 5 seconds, followed by the second image for 5 seconds, followed by the 3rd image for 5 seconds. Note that the begin events for the first two images are raised and propagated to all time dependents when the excl begins.

In the following slideshow example, images begin at the earlier of their scheduled begin time or when activated by a user input event:

<excl>
   <img src="image1.jpg" begin="0s".../>
   <img src="image2.jpg" begin="10s; image1.click".../>
   <img src="image3.jpg" begin="20s; image2.click".../>
</excl>

Note, some surprising results may occur when combining scheduled and interactive timing within an <excl>. If in the above example, the user clicks on image1 and then on image2 before ten seconds have elapsed, image 2 will re-appear at the ten second mark. Image 3 will appear at twenty seconds. The likely intent of this particular use-case would be better represented with a seq time container.

Side effects of activation

Children of the <excl> can be activated by scheduled timing, hyperlinks, events or DOM methods calls. For all but hyperlink activation, the <excl> time container must be active for child elements of the <excl> to be activated. With hyperlink activation, the document may be seeked to force the parent <excl> to be active, and a seek may occur to the begin time target child if it has a resolved begin time. That is, the normal hyperlink seek semantics apply to a timed child of an <excl>.

With activation via a DOM method call (e.g. the beginElement() method), the element will be activated at the current time (subject to the priorityClass semantics), even if the element has a scheduled begin time. The exclusive semantics of the time container (allowing only one active element at a time) and all priorityClass semantics are respected nevertheless.

See also Hyperlinks and timing and specifically Implications of beginElement() and hyperlinking for seq and excl time containers.

Specifying the simple duration of par and excl with endSync

The endSync attribute is only valid for <par> and <excl> time containers, and media elements with timed children (e.g. animate or area elements). The endSync attribute controls the end of the simple duration of these containers, as a function of the children. This is particularly useful with children that have "unknown" duration, e.g. an mpeg movie, that must be played through to determine the duration.

@@ Add more info to the effect that: Note that paused children of an <excl> container have not ended their active duration. Moreover (something to explain the "Elements do not have to play to completion, but must have played at least once") (i try ...) elements with multiple activation due to multiple begin and end value has only to play once to be considered as having ended their active duration.

endSync = " first | last | all | id-ref "

Legal values for the attribute are:
first
The <par>, <excl>, or media element simple duration ends with the earliest active end of all the child elements. This does not refer to the lexical first child, or to the first child to start, but rather refers to the first child to end its active duration.
last
The <par>, <excl>, or media element simple duration ends with the last active end of the child elements. This does not refer to the lexical last child, or to the last child to start, but rather refers to the last active end of all children that have a resolved begin time.
This is the default value.
all
The <par>, <excl>, or media element simple duration ends when all of the child elements have ended their respective active durations. Elements with indefinite or unresolved begin times will keep the simple duration of the time container from ending.
id-ref
The <par>, <excl>, or media element simple duration ends with the specified child. The id must correspond to one of the immediate children of the <par> time container. 
Example: <par ... endSync="movie1" ...>

Semantics of endSync and indeterminate children:

@@ Do we need a note to call out that in some cases, endSync may define an indefinite simple duration for the time container. This would flow "through the computing the active duration" table accordingly, using "implicit indefinite" as the simple duration.

The following pseudo-code describes the endSync algorithm:

// 
// boolean timeContainerHasEnded()
//
// method on time containers called to evaluate whether
// time container has ended, according to the rules of endSync.
// Note: Only supported on par and excl
//
// A variant on this could be called when a child end is updated to
// create a scheduled (predicted) end time for the container.
//
// Note that we never check the end time of children - it don't matter.
//
// Assumes: 
//     child list is stable during evaluation
//     isActive state of children is up to date for current time.
//      [In practice, this means that the children must all be
//        pre-visited at the current time to see if they are done.
//        If the time container is done, and repeats, the children
//        may be resampled at the modified time.]
//
//   Uses interfaces: 
//   on TimedNode:
//     isActive()     	tests if node is currently active
//     hasStarted()   	tests if node has (ever) begun
//     begin and end  	begin and end TimeValues of node
//
//   on TimeValue 	(a list of times for begin or end)
//   isResolved(t)  	true if there is a resolved time
//    	                         at or after time t
//

boolean timeContainerHasEnded()
{

TimeInstant now = getCurrentTime(); // normalized for time container

boolean assumedResult;

// For first or ID, we assume a false result unless we find a child that has ended
// For last and all, we assume a true result unless we find a dis-qualifying child

if( ( endSyncRule == first ) || ( endSyncRule == ID ) )
   assumedResult = false;
else
   assumedResult = true;

// Our interpretation of endSync == all:
//  	we're done when all children have begun, and none is active
//

// loop on each child in collection of timed children,
//  and consider it in terms of the endSyncRule

 foreach ( child c in timed-children-collection )
{
   switch( endSyncRule ) {
      case first:
         // as soon as we find an ended child, return true.
         if( c.hasStarted() & !c.isActive() )
            return true;
         // else, keep looking (assumedResult is false)
         break;

      case ID:
         // if we find the matching child, just return result
         if( endSyncID == c.ID )
                 return( c.hasStarted() & !c.isActive() );
         // else, keep looking (we'll assume the ID is valid)
         break;

      case last:
         // we just test for disqualifying children
         // If the child is active, we're definitely not done.
         // If the child has not yet begun but has a resolved begin,
         // then we're not done. Note that if it has already begun,
         // then we do not care if it has more resolved begins.
         if( c.isActive()
             || ( !c.hasStarted() && c.begin.isResolved(now) ))
             return false;
         // else, keep checking (the assumed result is true)
         break;

      case all:
         // we just test for disqualifying children
        // all_means_last_done_after_all_begin

         // If the child is active, we're definitely not done.
         // If the child has not yet begun then we're not done. 
         // Note that if it has already begun,
         // then we do not care if it has more resolved begins.
         if( c.isActive() || !c.hasStarted() )
             return false;
         // else, keep checking (the assumed result is true)
         break;

   } // close switch

} // close foreach loop

return assumedResult;

} // close timeContainerHasEnded()

Time container duration

The implicit duration of a time container is defined in terms of the children of the container.  The children can be thought of as the "media" that is "played" by the time container element.  The semantics are specific to each of the defined time container variants.

Implicit duration of <par> containers

By default, the simple duration of a <par> is defined by the endSync=last semantics. The simple duration will end when all scheduled children have ended their respective active durations.

The simple duration of a <par> container can be controlled with the dur and endSync attributes. If the dur attribute is specified, the endSync attribute is ignored. Using endSync, the end of the simple duration can be tied to the active end of the first child that finishes, or to the active end of the last child to finish (the default), or to the active end of a particular child element.

Implicit duration of <seq> containers

By default, the simple duration of a <seq> ends with the active end of the last child of the <seq>.  If any child of a <seq> has an indefinite active duration, the simple duration of the <seq> is also indefinite. 

Implicit duration of <excl> containers

The implicit simple duration of an <excl> container is defined the same as for a <par> container, using the endSync=last semantics. However, since the default timing for children of <excl> is interactive, it will be common for <excl> time containers to have indefinite simple duration.

Implicit duration of media element time containers

For endSync={last or all}: The time children and the intrinsic media duration define the simple duration of the media element time container. If a continuous media duration is longer than the extent of all the time children, the continuous media duration defines the implicit simple duration for the media element time container. If the media is discrete, this is defined as for <par> elements.

For endSync={first}: The time children and the intrinsic media duration define the simple duration of the media element time container. The element ends when the first active duration ends, as defined above for endSync on a <par>, but no sooner than the end of the intrinsic media duration of continuous media. If the media is discrete, this is defined as for <par> elements.

For endSync={ID}: This is defined as for <par> elements.

If the calculated implicit simple duration is longer than the intrinsic duration for a continuous media element, the ending state of the media (e.g. the last frame of video) will be shown for the remainder of the simple duration. This only applies to visual media - aural media will simply stop playing.

This semantic is similar to the case in which the author specifies a simple duration that is longer than the intrinsic duration for a continuous media element. Note that for both cases, although the media element is effectively frozen for the remainder of the simple duration, the time container local time is not frozen during this period, and any children will run normally without being affected by the media intrinsic duration.

Time container constraints on child durations

Time containers place certain overriding constraints upon the child elements. These constraints can cut short the active duration of any child element.

All time containers share the basic overriding constraint:

While the child may define a sync relationship that places the begin before the parent begin, the child is not active until the parent begins. This is equivalent to the semantic described in Negative begin delays.

If the child defines an active duration (or by the same token a simple duration) that extends beyond the end of the parent simple duration, the active duration of the child will be cut short when the parent simple duration ends. Note that this does not imply that the child duration is automatically shortened, or that the parent simple duration is "inherited" by the child. 

For example:

<par dur="10s" repeatDur="25s">
   <video dur="6s" repeatCount="2" .../>
   <text begin="5s" dur="indefinite" .../>
   <audio begin="prev.end" .../>
</par>

The video will play once for 6 seconds, and then a second time but only for 4 seconds - the last 2 seconds will get cut short and will not be seen. The text shows up for the last 5 seconds of the <par>, and the indefinite duration is cut short at the end of the simple duration of the <par>. The audio will not show up at all, since it is defined to begin at the end of the active duration of the previous element (the <text> element). Since the text element ends when the time container ends, the audio would begin after the time container has ended, and so never is heard. When the <par> repeats the first time, everything has happens just as it did the first time. However the last repeat is only a partial repeat (5 seconds), and so on the video will be seen, but it will not be seen to repeat, and the last second of the video will be cut off.

Note the time container is itself subject to the same constraints, and so may be cut short by some ascendant time container. When this happens, the children of the time container are also cut off, in the same manner as for the last partial repeat in the example above.

In addition, <excl> time containers allow only one child to play at once. Subject to the priorityClass semantics, the active duration of an element may be cut short when another element in the time container begins.

Time container constraints on sync-arcs and events

We need a few good examples to illustrate these concepts.

SMIL 1.0 defined constraints on sync-arc definition (e.g., begin="image1.begin"), allowing references only to qualified siblings. SMIL Boston explicitly removes this constraint. SMIL Boston also adds event-based timing.  Both sync-arcs and event-timing are constrained by the parent time container of the associated element as described above.

Specifics for sync-arcs

While a sync-arc is explicitly defined relative to a particular element, if this element is not a sibling element, then the sync is resolved as a sync-relationship to the parent (i.e. to an offset from the parent begin). If the defined sync would place the resolved element begin before the parent time container begin, part of the element will simply be cut off when it first plays.  This is not unlike the behavior obtained using clipBegin.  However unlike with clipBegin, if the sync-arc defined child element also has repeat specified, only the first iteration will be cut off, and subsequent repeat iterations will play normally. See also Negative begin delays.

Note that in particular, an element defined with a sync-arc begin will not automatically force the parent or any ancestor time container to begin.

For the case that an element with a sync-arc is in a parent (or ancestor) time container that repeats: for each iteration of the parent or ancestor, the element is played as though it were the first time the parent timeline was playing. With each repeat of the parent, the sync-arc will be recalculated to yield a begin time relative to the parent time container. See also the section Resetting element state.

Specifics for event-based timing

@@ If we allowed events on begin in children of sequence, we would have to refine this language to say that an element is sensitive after the active end of the previous element, and until its own active end.

The parent time container must be active for the child element to receive events.

Sequence children are only sensitive to events during their active duration. In the following example, all children listen to the same end event and it works as expected:

<seq>
   <img src="img1.jpg" end="foo.click" />
   <img src="img2.jpg" end="foo.click" />
   <img src="img3.jpg" end="foo.click" />
</seq>

Negative begin delays

A begin time (ultimately) specifies a synchronization relationship between the element and the parent time container.  Syncbase variants, eventbase, marker and wallclock timing are implicitly converted to an offset on the parent time container, just as an offset value specifies this directly. For children of a <seq>, the result is always a positive offset from the begin of the <seq> time container. However, for children of <par> and <excl> time containers. the computed offset relative to the parent begin time may be negative.

If the computed begin offset is negative, the time container constraints specify that the element cannot actually begin until the parent time container begins. Nevertheless, the element behaves as though it had begun earlier. A negative begin offset can be thought of as defining a clipBegin value (with the same magnitude) for the first -- and only the first -- iteration of a repeated element. If no repeat behavior is specified, a negative begin offset is equivalent to a clipBegin specification with the same magnitude as the offset value.

10.3.4 State transition model

@@This must be updated to reflect the impact of parent time container constraints and the DOM methods (especially including the "paused" state, and possibly Active-to-Active transition for seek()). 

State paused: In the paused state, an animation continues to perform the transformation of the specified presentation values that were current at the moment of entering the pause state.

Pause transition: Active to Paused This transition may occur if an animation element has its pause() method called while in the active state, or as a result of <excl> stacking behavior and being interrupted by a sibling of the parent <excl>.

Unpause transition: Paused to Active This transition may occur if an animation element has its unpause() method called while in the pause state, or as a result of <excl> unstacking behavior.

At any moment in time, a timed element is in exactly one of the following states: idle, active, finished or frozen. The state transitions are caused by events called start, restart, freeze and stop. Figure 6 shows the legal transitions between the states of an element:

State Diagram

Figure 6: State diagram of an element

The following sections explain the semantics of the states and transitions of a timed element, and explain how to define the state transitions using timing attributes of the element.

Note that the states and transitions are part of the model, and do not imply a particular implementation. Note also that an element may transition through more than one state in a virtual instant (i.e. with no time spent in a given state).

The presentation effect of timed elements is generally to display media, or to play a timeline (e.g. for time containers). In some cases, the element may be an animation that manipulates the presentation, but does not directly display anything. In some integration scenarios, the presentation effect of the element may be to apply a stylesheet, or to otherwise modify the presentation. In these discussions, the common case of displaying media or playing a timeline is used to describe the states and transitions. The same semantics should be understood to apply to all defined actions or presentation effects, as specified in the language that integrates SMIL Timing and Synchronization. 

Initial state: Idle

When the document that contains a timed element is first presented, the element is created in the idle state. This is the common starting state for all timed elements. 

When a parent time container repeats or is restarted, all (timed) child elements of the time container will be reset. As part of the element reset, the element is re-initialized to the Idle state. See also Resetting element state.

In the idle state a timed element is inactive and does not affect the presentation of the document in any way. The element simply waits for the time or event specified in its begin attribute. Note that the element may transition immediately to the active state if the element begins immediately when the document begins.

Start transition: Idle to Active

For an element to become active, the element's parent time container must be active. Given this, a timed element  in the idle state transitions to the active state when the condition specified in the begin attribute becomes true. As described in the section on the begin attribute, this condition may depend upon one of several factors:

Additionally, an element may be started by a DOM beginElement() or  beginElementAt() method call, or as the result of being the target of an activated hyperlink.

An element may become active as soon as its parent time container becomes active, if the condition specified in the begin attribute is true at that point.

Note that the begin attribute can specify a condition that is a list of values. The specific semantics of evaluating the list of values is described in the section Basic - begin and dur.

Active state:

In the active state, a timed element  displays the associated media or performs the described timeline associated with the element. The active state includes the entire active duration of the element. The active duration of an element is specified by the interaction between the dur, end, repeatDur, and repeatCount attributes as detailed in the section Computing the Active Duration.

Freeze transition: Active to Frozen

If a timed element  has the fill attribute set to "freeze", "hold" or "transition", upon reaching the end of its active duration, the element will transition to the frozen state. 

Frozen state:

In the frozen state the element will continue to present the last defined state of visual media or the timeline state at the end of the active duration (aural media render nothing during the frozen state). The duration of the frozen state depends upon the value of the fill attribute (as described in Freezing elements) and on the parent time container (as described in Time Container constraints on child durations). 

The frozen state may have 0 duration, e.g. if the parent time container ends with the element.

Stop transition: Active to Finished

If a timed element  has the fill attribute set to remove (the default), upon reaching the end of its active duration, the element will transition to the finished state. Note that the active duration of a child element may end when an ascendant time container ends its simple duration.

Finished state:

In the finished state the timed element does not affect the presentation of the document. The duration of the finished state depends upon the parent time container. The finished state  lasts until the end of the current simple duration of the parent time container, or until the element is restarted (whichever comes first).

Restart transition: Frozen to Active

The ability of an element to make this transition depends upon the value of the restart attribute. If the restart attribute value is always or whenNotActive the element will transition to the active state in response to a DOM beginElement() or  beginElementAt() method call, or an additional begin event. The restart transition effectively resets the state of the animation element; the element's simple and active duration must be recomputed as if it were being started for the first time. See also Resetting element state.

Restart transition: Frozen to Idle

This transition happens the same way as the Frozen to Active transition (immediately above). If the element specifies an offset from the syncbase or eventbase, or if the DOM beginElementAt() method call specifies a non-0 offset, then the element is returned to the Idle state until it actually restarts.

Restart transition: Active to Active

An element may receive a DOM beginElement() or  beginElementAt() method call or may receive an additional begin event while in the active state. In this case, if the value of the restart attribute is "always" the element will re-transition to the active state and restart as described above. Any other value for the restart attribute will prevent this transition from occurring. 

Restart transition: Active to Idle

This transition happens the same way as the Active to Active transition (immediately above). If the element specifies an offset from the syncbase or eventbase, or if the DOM beginElementAt() method call specifies a non-0 offset, then the element is returned to the Idle state until it actually restarts.

Restart transition: Finished to Active

An element restart can result from a DOM call or an additional begin event, subject to the restrictions imposed by the restart attribute. When in the finished state, an element may re-transition to the active state if the value of the restart attribute is "always" or "whenNotActive". If  the restart attribute is set to "never", this transition can not occur.

Restart transition: Finished to Idle

This transition happens the same way as the Finished to Active transition (immediately above). If the element specifies an offset from the syncbase or eventbase, or if the DOM beginElementAt() method call specifies a non-0 offset, then the element is returned to the Idle state until it actually restarts.

10.3.5 Timing model details

Timing and real-world clock times

In this specification, elements are described as having local "time". In particular, many offsets are computed in the local time of a parent time container. However, simple durations can be repeated, and elements can begin and restart in many ways.  As such, there is no direct relationship between the local "time" for an element, and the real world concept of time as reflected on a clock.

When the time manipulation attributes are used to adjust the speed and/or pacing within the simple duration, the semantics can be thought of as changing the pace of time in the given interval. An equivalent model is these attributes simply change the pace at which the presentation progresses through the given interval. The two interpretations are equivalent mathematically, and the significant point is that the notion of "time" as defined for the simple duration and "local time" should not be construed as real world clock time. For the purposes of SMIL Timing and Synchronization, "time" can behave quite differently from real world clock time.

Interval timing

The SMIL timing model assumes the most common model for interval timing. This describes intervals of time (i.e. durations) in which the begin time of the interval is included in the interval, but the end time is excluded from the interval. This is also referred to as end-point exclusive timing. This model makes arithmetic for intervals work correctly, and provides sensible models for sequences of intervals.

Background rationale

In the real world, this is equivalent to the way that seconds add up to minutes, and minutes add up to hours.  Although a minute is described as 60 seconds, a digital clock never shows more than 59 seconds. Adding one more second to "00:59" does not yield "00:60" but rather "01:00", or 1 minute and 0 seconds. The theoretical end time of 60 seconds that describes a minute interval is excluded from the actual interval.

In the world of media and timelines, the same applies: Let a be a video, a clip of audio, or an animation. Assume "A" begins at 10 and runs until 15 (in any units - it does not matter). If "B" is defined to follow "A", then it begins at 15 (and not at 15 plus some minimum interval). When a runtime actually renders out frames (or samples for audio), and must render the time "15", it should not show both a frame of "A" and a frame of "B", but rather should only show the new element "B". This is the same for audio, or for any interval on a timeline. If the model does not use endpoint-exclusive timing, it will draw overlapping frames, or have overlapping samples of audio, of sequenced animations, etc.

Note that transitions from "A" to "B" also adhere to the interval timing model. They do require that "A" not actually end at 15, and that both elements actually overlap. Nevertheless, the "A" duration is simply extended by the transition duration (e.g. 1 second). This new duration for "A" is also endpoint exclusive - at the end of this new duration, the transition will be complete, and only "B" should be rendered - "A" is no longer needed.

Implications for the time model

For the time model, several results of this are important: the definition of repeat, and the value sampled during the "frozen" state.

When repeating an element simple duration, the arithmetic follows the end-point exclusive model. Consider the example:

  <video dur="4s" repeatCount="4" .../>

At time 0, the simple duration is also at 0, and the first frame of video is presented.  This is the inclusive begin of the interval. The simple duration proceeds normally up to 4 seconds. However, the appropriate way to map time on the active duration to time on the simple duration is to use the remainder of division by the simple duration:

  simpleTime = REMAINDER( t, d ) where t is within the active duration

Note: REMAINDER( t, d ) is defined as t - d*floor(t/d)

Using this, a time of 4 (or 8 or 12) maps to the time of 0 on the simple duration. The endpoint of the simple duration is excluded from (i.e. not actually sampled on) the simple duration. 

For most continuous media, this aligns to the internal media model, and so no frames (or audio samples) are ever excluded. However for sampled timeline media (like animation), the distinction is important, and requires a specific semantic for handling the frozen state. 

The effect of this semantic upon animation functions is detailed in the [SMIL-ANIMATION] module.

Unifying scheduling and interactive timing

A significant motivation for SMIL Boston is the desire to integrate declarative, determinate scheduling with interactive, indeterminate scheduling.  The goal is to provide a common, consistent model and a simple syntax.

Note that "interactive" content does not refer simply to hypermedia with support for linking between documents, but specifically to content within a presentation (i.e. a document) that is activated by some interactive mechanism (often user-input events, but including local hyperlinking as well).

SMIL Boston describes extensions to SMIL 1.0 to support interactive timing of elements. These extensions allow the author to specify that an element should begin or end in response to an event (such as a user-input event like "click"), or to a hyperlink activation, or to a DOM method call.

The syntax to describe this uses event-value specifications and the special argument value "indefinite" for the begin and end attribute values. Event values describe user interface and other events. If an element should only begin (or end) with a DOM method call, the begin and end attributes allow the special value "indefinite" to indicate this. Setting begin="indefinite" can also be used when a hyperlink will be used to begin the element. The element will begin when the hyperlink is actuated (usually by the user clicking on the anchor). It is not possible to control the active end of an element using hyperlinks.

Background

SMIL Boston represents an evolution from earlier multimedia runtimes. These were typically either pure, static schedulers or pure event-based systems.  Scheduler models present a linear timeline that integrates both discrete and continuous media.   Scheduler models tend to be good for storytelling, but have limited support for user-interaction. Event-based systems, on the other hand, model multimedia as a graph of event bindings.  Event-based systems provide flexible support for user-interaction, but generally have poor scheduling facilities; they are best applied to highly interactive and experiential multimedia.

The SMIL 1.0 model is primarily a scheduling model, but with some flexibility to support continuous media with unknown duration. User interaction is supported in the form of timed hyperlinking semantics, but there was no support for activating individual elements via interaction.

Modeling interactive, event-based content in SMIL

To integrate interactive content into SMIL timing, the SMIL 1.0 scheduler model is extended to support several new concepts: indeterminate timing and event-activation.

With indeterminate timing, an element has an undefined begin or active end time.  The element still exists within the constraints of the document, but the begin or active end time is determined by some external activation. Activation may be event-based (such as by a user-input event), hyperlink based (with a hyperlink targeted at the element), or DOM based (by a call to the beginElement() or beginElementAt() methods).  From a scheduling perspective, the time can be thought of as unresolved

The event-activation support provides a means of associating an event with the begin or active end time for an element.  When the event is raised (e.g. when the user clicks on something), the associated time is resolved to a determinate time.  The actual begin (or end) time is computed as the time the event is raised plus or minus any specified offset. 

The computed time defines the synchronization for the element relative to the parent time container. It is possible for the computed begin or end time to occur in the past, e.g. when a negative offset value is specified, or if there is any appreciable delay between the time the event is raised and when it is handled by the SMIL implementation. See also the section Handling negative offsets.

Note that an event based end will not be activated until the element has already begun. Any specified end event is ignored before the element begins.

The constraints imposed on an element by its time container are an important aspect of the event-activation model. In particular, when a time container is itself inactive (e.g. before it begins or after it ends), no events are handled by the children. If the time container is frozen, no events are handled by the children. No event-activation takes place unless the time container of an element is active. For example:

<par begin="10s" dur="5s">
   <audio src="song1.au" begin="btn1.click" />
</par>

If the user clicks on the "btn1" element before 10 seconds, or after 15 seconds, the audio element will not play.  In addition, if the audio element begins but would extend beyond the specified active end of the <par> container, it is effectively cut off by the active end of the <par> container.

Event sensitivity

The SMIL Boston timing model supports synchronization based upon DOM events. These can be user interface generated and other kinds of unpredictable events. The model for handling events is that the notification of the event is delivered to the timing element, and the timing element uses a set of rules to resolve any synchronization dependent upon the event.

The semantics of element sensitivity to events are described by the following set of rules:

  1. While a time container is not active (i.e. before the time container begin or after the time container active end), child elements do not respond to events (with respect to the Time model). Note that while a parent time container is frozen, it is not active, and so children do not handle begin or end event specifications.
    1. Note that an element will not receive an instance of an event that begins any ascendant time container. 
    2. @@The above semantic is hard to implement based upon a standard DOM event model. We may have to reconsider and say: Note that if the element and its parent time container are both specified to begin with the same event, the behavior is not defined. DOM Level 2 events does not provide a means to order the registered listeners for an event, and so implementations cannot guarantee that the parent will be activated before the child. Authors should avoid this construct in documents.

  2. If an element is not active (but the parent time container is), then events are only handled for begin specifications. Thus if an event is raised and begin specifies the event, the element begins and any specification of the event in end is ignored for this event instance.
  3. If an element is (already) active when an event is raised, and begin specifies the event, then the behavior depends upon the value of restart:
    1. If restart="always", then a new begin time is resolved for the element based on the event time. Any specification of the event in end is ignored for this event instance.
    2. If restart="never" or restart="whenNotActive", then any begin specification of the event is ignored for this instance of the event. If end specifies the event, an end value is resolved based upon the event time, and the active duration is re-evaluated (according to the rules in Computing the active duration).

It is important to notice that in no case is a single event occurrence used to resolve both a begin and end time on the same element.

These rules can be used with the restart attribute to describe "toggle" activation use cases, as described in the section: Using restart for toggle activation

Since the same event instance cannot be used to resolve both the begin and end time on a single element, uses like the following will have behavior that may seem non-intuitive to some people:

<audio src="bounce.wav" begin="foo.click" end="foo.click+3s" 
       restart="whenNotActive"/>

This example will begin repeating the audio clip when "foo" is clicked, and stop the audio clip 3 seconds after "foo" is clicked a second time. It is incorrect to interpret this example as playing the audio clip for 3 seconds after "foo" is clicked. For that behavior, the following markup should be used:

<audio src="bounce.wav" begin="foo.click" dur="3s"
       restart="whenNotActive"/>

Related to event-activation is link-activation.  Hyperlinking has defined semantics in SMIL 1.0 to seek a document to a point in time. When combined with indeterminate timing, hyperlinking yields a variant on interactive content. A hyperlink can be targeted at an element that does not have a scheduled begin time. When the link is traversed, the element begins. Note that unlike event activation, the hyperlink activation is not subject to the constraints of the parent time container. The details of when hyperlinks activate an element, and when they seek the document timeline are presented in the section Hyperlinks and timing.

Details of the time manipulations

Speed control

Speed modifies the pace of time for the element and its descendents, and so modifies the interpretation of the normal timing attributes with respect to the normal pace of (real-world) time. The attributes dur and repeatDur always specify a time in unmodified local time for the element. As a result, the observed simple duration and repeat duration for an element with a modified speed is not the same as the specified speed. This is important to making the model be consistent when the speed cascades in the time containment hierarchy, although it can make authoring somewhat more complex.

Note that a speed attribute on an element does not affect the element begin time. It may affect the element end time, if the end is defined only in terms of the simple duration or repeat duration. An end value (defined by the end attribute) is converted to element local time using the speed value. However, the result is that the active duration is not affected by the speed value, since the values (syncbase values, eventbase times, wallclocks times, etc.) are all defined in another timespace and converted to the local timespace. See also the examples below.

To compute the effect of speed on the simple duration or on the active duration if defined with repeat, the following function is used. This function is also used to convert a time in the parent local timespace to a time in the child local timespace that accounts for the speed attribute.

  Tpar is the time in the parent local timespace

  Tel is the time in the element local timespace

  Tel = (Tpar / speed)

When speed is applied to a time container, it  scales the rate of progress through the time container timeline. This effect cascades. When descendents also specify a speed value, the parent speed and the child speed are multiplied to yield the result. For example:

<par speed=2.0>
   <animation begin="2s" dur="9s" speed=0.75 .../>
</par>

The observed rate of play of the animation element is 1.5 times the normal play speed. The element begins 1 second after the par begins (the begin offset is scaled by the parent speed), and ends 6 seconds later (9/1.5). 

The following example shows how an event based end combines with time filters:

<par speed=2.0>
   <animation begin="2s" dur="9s" speed=0.75 
        repeatCount="4" end="click" .../>
</par>

This behaves as in the first example, but the animation element will repeat 4 times for a total of 24 seconds (in real time), unless a click happens before that. Whenever the click happens, the element ends. A variant on this demonstrates syncbase timing:

<par speed=2.0>
   <img id="foo" dur="30s" .../>
   <animation begin="2s" dur="9s" speed=0.75 
        repeatCount="4" end="click, foo.end" .../>
</par>

The image will display for 15 seconds. The animation element plays at an observed rate of 1.5 times play speed, but it will end after 15 seconds, when the image ends. The animation will have repeated 2.5 times at this point. Note that although the animation has a speed value, this does not impact the semantic of the syncbase timing. When the syncbase, eventbase, wallclock or media marker time is observed to happen, it will be applied anywhere it is used at that real time (although various timespace conversions are applied internally). 

Note that in the examples above, the default duration of the <par> container is defined as endSync="last". This behavior is not affected by the speed modifications, in the sense that the observed end of the elements will produce the correct simple duration on the parent time container. 

@@ Need to include notes about how begin offsets are also scaled by the parent timescale, but not the local element scale.

The following example illustrates an important effect of offset time scaling:

<par speed=2.0>
   <img id="foo" dur="30s" .../>
   <animation begin="2s" dur="9s" speed=0.75 
        repeatCount="4" end="foo.end+6s" .../>
</par>

The image will display for 15 seconds. The animation element plays at an observed rate of 1.5 times play speed, and it will end after 18 seconds. The offset added to the end of the image is scaled by the parent time container speed. The animation will have repeated 3 times at this point.

The following example illustrates the speed modifications along with the fallback for a video element that can only play normal forward speed.

<par speed=2.0>
   <video id="foo" dur="30s" accelerate="0.25" ...>
      <area begin="2s" dur="4s" .../>
   </video>
   <animation begin="2s" dur="9s" speed=0.75 
        repeatCount="4" end="foo.end+6s" .../>
</par>

The video ignores the acceleration and the speed value, and plays at normal speed for 15 seconds. The simple (and active) duration are still constrained by the speed. The area element reflects what the video is playing, and so the area becomes active after 2 seconds and remains active for 4 seconds. The animation element behaves just as it did in the previous example, and ends after 18 seconds and 3 repeats.

Issues with implicit duration and fallback speeds

@@ This should perhaps move to the Fallbacks section

Some media will use a fallback speed (e.g. because it cannot play at the requested speed - see the section Fallbacks for time filters on a media element). When this is the case, and the simple duration is defined by the implicit media duration, the results are sometimes less clean, as the following example illustrates. Assume that the video has an intrinsic duration of 30 seconds, and that this cannot be determined until the video plays through to the end (as is sometimes the case).

<par speed=1.5>
   <video id="foo" repeatCount="3" accelerate="0.25" ...>
      <area begin="2s" dur="4s" .../>
   </video>
</par>

The video ignores the acceleration and the speed value, and plays at normal speed for 30 seconds. At this point, the simple duration is resolved, and (accounting for the parent speed) is computed to be 20 seconds. At this point the video *should* be halfway into the second repeat iteration, moving 50% faster than normal playspeed. The video should continue playing for another 30 seconds. The fallback rules would define that each repeat iteration would play the first 20 seconds of the 30 second video. Given the linear behavior of the video element, the first 10 seconds will be shown for the (remainder of the) second repeat iteration, followed by the first 20 seconds of video for the third repeat iteration. Needless to say, this may not produce pleasing results. At the very least, authors should avoid mixing speed modifications with media elements that use the implicit simple duration, and cannot play at the specified speed. As stated elsewhere, the time modifications should be used where appropriate.

@@ Note sure if this next is useful:

Note that the timing of all durations, begin and end times and events, is based upon the computed values using the explicit speed and not the fallback speed.  If the media player is nominally playing at the explicit speed, but varies somewhat, then the effective speed, end and durations should be used in the time model (i.e. the behavior of the runtime in computing effective versus desired times, and managing synchronization, should be consistent with the behavior of the media player and timing model when the speed is 1.0, or normal play speed). Try: the rules and mechanisms that a runtime uses to compute effective times versus desired times should be consistently applied if the "attempted" speed is normal playspeed (1.0) or offspeed (e.g. 2.0). If the "attempted" speed is not the desired speed, then a strict interpretation of the desired times should be used for effective times.

@@ Proposal to help authors - is this useful?:

Define a new test attribute canPlaySpeed (?) to support alternate presentations depending upon the capabilities of the media players. Should be evaluated at runtime for the given media and associated player. Evaluates true if the media player will attempt to play the specified speed. Evaluates false otherwise. If applied to a time container, then should consider all the descendents of the time container, and only evaluate to true if all descendents will attempt to play the desired speed. In evaluating this, the cascaded desired speed must be used, and not the explicit speed on each element.

Acceleration and Deceleration

@@ Borrow pictures and description of manipulation from keySplines in SMIL Animation? Need to describe that time is actually manipulated and remapped, but can think of it as progress.

The speed or rate of progress through the simple duration must be increased to account for the acceleration/deceleration and preserve the simple duration.  The adjusted speed is described as the run rate. For an element with both acceleration and deceleration, the speed over the simple duration varies from 0 up to the run rate and then back down to 0.

To compute the run rate over the course of the simple duration, the following formula is used. Let a be the value of accelerate, and b be the value of decelerate. The run rate r is then:

r = 1 / ( 1 - a/2 - b/2 )

Thus, for example, if the value of accelerate is 1 (i.e. accelerate throughout the entire simple duration), the run rate is 2.

The speed s(t) at any point in time t  (within the simple duration d) is defined as a function of the run rate, as follows:

For: ( 0 <= t < (a*d) )          I.e. in the acceleration interval
    s(t) = r * ( t / ( a * d ))
For: ( (a*d) <= t <= (d-(b*d)) ) I.e. in the run-rate interval
   s(t) = r
For: ( (d-(b*d)) < t <= d )      I.e. in the deceleration interval
   s(t) = r * ( ( t - (d-(b*d)) ) / ( b*d ))

If in place of t we use p, the proportional progress through the simple duration, the equations simplify somewhat:

p = t/d
For: ( 0 <= p < a )          I.e. in the acceleration interval
    s(p) = r * ( p / a )
For: ( a <= p <= (1-b) )     I.e. in the run-rate interval
   s(p) = r
For: ( (1-b) < p <= 1 )      I.e. in the deceleration interval
   s(p) = r * ( ( p - (1-b) ) / b )
Examples:

In this example, a motion path will accelerate up from a standstill over the first 2 seconds, run at a faster than normal rate for 4 seconds, and then decelerate smoothly to a stop during the last 2 seconds.  This makes an animation look more realistic. The animateMotion element is defined in the Animation section of SMIL Boston.

<img ...>
    <animateMotion dur="8s" accelerate=".25" decelerate=".25" .../>
</img>

In this example, the image will "fly in" from offscreen left , and then decelerate quickly during the last second to "ease in" to place. This assumes a layout model that supports positioning (a similar effect could be achieved by animation the position of a region in SMIL layout). The animate element is defined in the Animation section of SMIL Boston.

<img ...>
    <animate attributeName="left" dur="4s" decelerate=".25" 
             from="-1000" to="0" additive="sum" />
</img>
Play Forwards then Backwards
Examples:

A simple example is provided in the syntax description above.

In the following example the motion path will behave as above, but will end at the earlier of 15 seconds or when the user clicks on the image. If the element ends at 15 seconds (if the user does not click), the motion path will leave the element at the end of the defined path, 20 pixels to the right. Note that repeatDur and end are not affected by the autoReverse attribute (although repeatCount is).

<img ...>
    <animateMotion  by="20, 0" dur="5s" autoReverse="true" 
         repeatDur="15" end="click" fill="freeze"/>
</img>

Accelerate and decelerate can be combined with autoReverse, and are applied to the unmodified simple duration. For example:

<img ...>
    <animateMotion by="20, 0" dur="4s" autoReverse="true" 
       accelerate=".25" decelerate=".25" />
</img>

This will produce a kind of elastic motion with the path accelerating for 1 second from the original position as it moves to the right, moving slightly faster than normal for 2 seconds, and then decelerating for 1 second as it nears the points 20 pixels to the right. It accelerates back towards the original position and decelerates to the end of the reversed motion path, at the original position.

Speed can also be combined with autoReverse, and modifies the entire effect. This example combines all three time manipulations:

<img ...>
    <animateMotion by="20, 0" dur="4s" autoReverse="true" 
       speed="0.5" accelerate=".25" decelerate=".25" />
</img>

This produces the same effect as in the previous example, except that everything moves half as quickly and takes twice as long. The active duration is 16 seconds.

Timing Model

A theoretical model can be described that assumes that all element local timelines (including any media elements) are completely non-linear and have unconstrained ballistics. This ideal model can be applied to many applications, including pure rendered graphics, text, etc. Nevertheless, many common applications also include media with linear behavior and other constraints on playback. When the timegraph includes media elements that have linear behavior, the model must adapt, and/or provide consistent semantics that accommodate these real world constraints.  This sections below include a discussion of these fallback semantics. 

Note that while the model does support timegraphs with a mix of linear and non-linear behavior, and defines specific semantics for timegraphs that cannot support the ideal non-linear model, it is not a goal to provide an ideal alternative presentation for all possible timegraphs with such a mix. It is left to authors and authoring tools to apply the time manipulations in appropriate situations. This section describes both the ideal model as well as the semantics associated with linear-media elements.

Ideal model

In the ideal model, the pace or speed of local time can be manipulated arbitrarily. The graph advances (or is sampled, depending upon your perspective) as the presentation time advances. A time container samples each of its children in turn, so that a graph traversal is performed for each render time.  Elements that are idle or stopped (i.e. neither active nor frozen) are pruned from the traversal as an optimization. As the traversal moves down the graph (from time containers to children), each local timeline simply transforms the current time from the parent time-space to the local time space, and then samples the local timeline at the transformed current time. Note that the speed and effects of the time filters effectively cascade down the time graph, since each element transforms local time for itself and all descendents.

When linear media are added to this model and the "current time" (sample) traversal encounters a media element, the media element is effectively told to "sample" at a particular position. Given that linear media can not sample arbitrarily, the semantic that is used is to verify the current position of the media (as observed on the player) against the current theoretical timeline position for the timegraph. Within certain  limits (e.g. defined by a syncTolerance attribute), divergence from the theoretical timeline position are ignored. So far, this is just a typical implementation of a synchronization manager.

When the speed (and even direction) of the local timeline can vary from normal forward playspeed, an additional parameter is added to the context for the "current time" traversal to indicates the speed of the local timeline. The speed is defined as a proportion of normal forward play speed. Thus a value of "1" is normal forward playspeed, a value of "2" is twice normal forward speed, and a value of "-1" is backwards, at the normal play speed. Note that a value of 0 is not allowed. The speed is often broken down logically into the rate and the direction. The rate is just the absolute value of the speed, and the direction is the "sign" of the speed.

Given the current computed position and speed for the timegraph, a media element with linear behavior can handle the case of non-normal playspeed with the use of fallback semantics. The fallback semantics depend upon how much or how little the media player is capable of. Some may play forwards and backwards but only at the normal rate of play, others may only handle normal forward play speed. The fallback semantics are detailed in the next sections. 

Fallbacks for time filters on a media element

@@Describe the properties that the fallback semantics are preserving so that the rationale is clear

When any of the time filters are applied directly to a linear media element, the element can directly examine the attributes and apply one set of fallback semantics. These include:

Note that the semantics of clipBegin and clipEnd are not affected (i.e. they are still respected).
The clipBegin and clipEnd semantics are always interpreted in terms of normal forward play speed. I.e. they are evaluated before any effects of time filters have been applied to the time model. This is consistent with the model that they can be evaluated by the media element handler, independent of the time model.

If a fallback semantic is applied to a media element and the media element has child elements (i.e. is a media time container), the local time as passed through to the children should reflect what the media actually performs. Thus if the accelerate and decelerate are ignored, the local time as passed to the child elements should not be filtered for acceleration and deceleration. This is important for cases like anchors that are timed to associate with points in the media.  If an author wishes to apply acceleration to the children of the media (e.g. to accelerate a set of animation children), a wrapping <par> with the time filters applied can be inserted under the media time container. See also the next section for a general discussion of time filters and time containers.

@@ Above semantic may be overly complex. It requires that the time runtime know what the media players can actually do so that the proper timing parameters can be passed down to the children. Is this restriction worth it, since playing with a fallback will lead to a different presentation than intended anyway. The presentation visually will act funny anyway, and the entire thing is constrained by the parent time container, so it may not be as bad as noted.

The effect of these fallbacks is illustrated by some examples. In these examples, the video player is assumed to be capable only of normal forward play speed.

In the following example, the acceleration and deceleration can be safely ignored without side effect. Since they are ignored, the area child should become active 1 second after the video begins (i.e. if the video ignores the acceleration, then the effect is not applied to the children either):

<video accelerate=0.5 decelerate=0.5 src...>
   <area begin=1s .../>
</video>

In the following example, the autoReverse attribute doubles the simple duration, but the video will just play the 3 second simple duration twice, in the forward direction both times. The area child should become active 1 second after the video begins each time (i.e. if the video ignores the reverse play, then the speed change is not applied to the children either):

<video autoReverse="true" dur=3s src...>
   <area begin=1s .../>
</video>

In the following example, the speed attribute decreases the simple duration by 33% (i.e. one third). The video will play the first two seconds at normal playspeed, rather than three seconds at 1.5 times playspeed. The area child should become active 1 second after the video begins each time (i.e. if the video ignores the speed manipulation, then the speed change is not applied to the children either):

<video speed="1.5" dur=3s src...>
   <area begin=1s .../>
</video>
Fallbacks for time filters on time containers

When any of the time filters are applied to a time container element, the implementation should generally respect the time filters and deal with the effects of modified time for each media player. Nevertheless, in some situations it may be desirable to apply a fallback semantic at the time container itself. 

@@Need more info on how time containers run backwards. The use of autoReverse is special, as it can generally use the end and begin times from the forward half. Note that time dependencies work backwards, but that event-based timing does not work correctly, and script events may not be fired. This needs more discussion. Need a reasonable means of simplifying the implementation and still allowing backwards play of fully scheduled timegraphs.

@@Examples!

More on the implementation

In the theoretical model, each element has a notion of local time that extends from 0 to some duration (the simple duration). To obtain the local time for an element in the simple case with no pacing control, an arithmetic transform is performed for the element and each ascendant time container up to the document root. This transform allows for the begin offset of the element relative to its parent time container, and for any repeat behavior of the element. The pseudo-code for this transform looks like:

@@Edit to make it produce both active and simple time? The only difference is in how the final element is handled. Also need to have a control over unconstrained versus constrained time. Sync arc calculation uses unconstrained time, but real sampling uses end-constrained time that respects freeze.

@@ Edit to use the most recent parent begin if it is active, rather than the constant repeatDuration

// globalToLocalTime()
//
// Recurses up to document root, and then transforms timeIn
// from parent to local time as the recursion unwinds
// Version 1: Ignores fill=freeze, assumes constant pace of time
//             assumes repeat duration is constant,
//             and assumes all times are resolved.
// Inputs:
//        timeIn - the parent simple time
// Outputs:
//        timeOut - the resulting local simple time
//
TimeInstant globalToLocalTime( TimeInstant timeIn )
{
   TimeInstant timeOut;
   Node parent = getParent();

   if( !parent.isDocumentTimeRoot() )
   {
      // We have a parent time container above us.
      // First convert timeIn to simple local time for it,
      // and then adjust for this node.
      timeOut = parent.globalToLocalTime( timeIn );
   }
   else // Our parent is the top time container
      timeOut = timeIn;

   // Adjust for begin offset. Assume "begin" has simplified begin time.
   timeOut = timeOut - this.begin; // Adjust for begin offset
   if( this.repeats() )
   {
       // Adjust for repeating simple duration.
       timeOut = timeOut % this.repeatDuration;
   }
   // timeOut is now converted to simple local time for this node
   return timeOut;
}

@@Next version shows how this is extended to support the time manipulations. For  autoReverse, remember the edge case for the endpoint. ?Talk about the imprecision around the reverse point: interval math introduces an epsilon inaccuracy at the end, where epsilon is the sample granularity (e.g. milliseconds in our implementation).

@@How to show the update dependencies of repeatDur, segmentDur? Could just show a method to recompute, that is called on changes. Ignore all optimization issues.

Converting between local and global times

@@@ Need to discuss how to convert a time specified as a syncbase-value (and by extension a wallclock-value or an event in document or system time ) to a time on the parent time container local timeline.  Especially given the wallclock stuff, we need to consider the name "local time". 

Define the notion of document global time, and note that it is the normalized timeline used to convert between different timespaces. Operations defined are GlobalToLocal, and LocalToGlobal. To convert from one timespace to another, simply convert the first time from local to global, and then from global to local for the second tiespace.

Basic mechanism for global to local conversion is iterate downward from the document body to the element converting the global time to a time on each time container encountered along the way. This is often implemented as a recursive algorithm, where the recursion moves from the local element up to the document body, and the work is done as the recursion "unwinds". For each time container, subtract the begin offset of the time container and then, use the remainder after dividing by the simple duration (which may vary over time - yuck!) or subtract the offset of the current repeat iteration from the begin time (better when working on begin resolution on a reset). Then, apply any filters for time manipulations (speed, acclerate/decelerate, autoReverse). 

Basic mechanism for localToGlobal reverses the above algorithm. If the element is active, then the effective begin time of the current iteration of all (repeating) ancestor time containers is used when adding the begin offset. If the element is not active, then for each ancestor time container that is not active, the earliest begin time is used. 

Note that the pure conversions do not take into account the clamping of active durations, nor the effects of fill (where time is frozen). Global to local time conversions used to translate between timespaces must ignore these issues, and so may yield a time in the destination local timespace that is well before or well after the simple duration of the element. 

An alternate form of the conversion is used when actually sampling the time graph. A time container is only sampled if it is active or frozen, and so no times will be produced that are before a time container begins. If the global to local time conversion for a time container yields a time during which the time container is frozen, the time is clamped to the value of the active end.

Evaluation of begin and end time lists

In evaluating the list of begin values, one of two questions is asked: 

The earliest time is used for example when an element is the target of a hyperlink activation (see also Hyperlinks and timing). The next begin time may be used by a scheduler when an element can begin more than once. 

By the same token, a list of end values is evaluated for two cases:

The earliest time is used for example to calculate the simple duration of a time container defined with endSync. The next end time may be used by a scheduler when an element can begin more than once.

Begin and end time lists are considered in tandem. Begin times can be constrained by an end specification, as well as by the parent time container. Note that if an end time is not resolved, or is specified as "indefinite", then it is considered to be "after" al begin times. The constraint rules are:

Thus, for a given point when an element is not active, the next begin and end time are derived from the begin and end lists as follows:

  1. Find the earliest resolved begin time after the current time
    1. Discard any unresolved times
    2. Discard any times that occur before the current time
    3. Discard any times that occur after the end of the parent simple duration
    4. Take the earliest time of the remaining times (if there are none, this step fails)
  2. If Step 1 fails, and if there are unresolved times, then the begin time is considered to be unresolved, and behaves as though "indefinite" had been specified. If Step 1 fails and there are no unresolved times, then this step fails.

If both Steps 1 and 2 fail, then the begin list only specifies resolved times in the past, or after the end of the parent time container. In this case, there is no (valid) begin time after the current time (although for the purposes of a hyperlink that targets the element, even an "invalid" time may be used with additional constraints - see Hyperlinks and timing).

If a begin time is found, then it must be checked against the list of end times. If no end attribute is specified, the end is "indefinite", and any begin found above is valid. If any end values are specified, the list is evaluated as follows:

  1. Find the earliest resolved end time after the candidate begin time
    1. Discard any unresolved times
    2. Discard any times that occur before the begin time
    3. Take the earliest time of the remaining times (if there are none, this step fails)
  2. If Step 1 fails, and if there are unresolved times, then the end time is considered to be unresolved, and behaves as though "indefinite" had been specified. If Step 1 fails and there are no unresolved times, then all end times are resolved and occur before the candidate begin time. In this case the candidate begin time is rejected, and the element does not begin.

If a valid begin and end value are found, the end value found with the rules above is the value used in Computing the Active Duration.

When it is specified that the element does not begin, begin and end events will not be raised in the DOM, and time dependents defined relative to the begin or end of this element will not be activated. 

In contrast to the rules above, if a beginElement() or a beginElementAt() call specifies a begin time after the last end time (with no unresolved end times), the active duration is indefinite, as though end had been defined as "indefinite". Note that if beginElement() or beginElementAt() is called when the parent time container is not active, the method call will have no effect.  If beginElementAt() is called and specifies a time after the end of the parent simple duration, the method call will have no effect. See also Supported methods.

Hyperlinks and timing

Hyperlinking semantics must be specifically defined within the time model in order to ensure predictable behavior. Earlier hyperlinking semantics, such as those defined by SMIL 1.0 are insufficient because they do not handle indeterminate and interactive timing, nor do they handle author-time restart restrictions. Here we extend SMIL 1.0 semantics for use in presentations using elements with indeterminate timing, interactive timing, and author-time restart restrictions.

A hyperlink may be targeted at an element by specifying the value of the id attribute of an element in the fragment part of the link locator. Traversing a hyperlink that refers to a timed element will behave according to the following rules:

  1. If the target element is active, seek the document time back to the (current) begin time of the element. If there are multiple begin times, use the begin time that corresponds to the current "begin instance".
  2. Else if the target element begin time is resolved (i.e. there is any resolved time in the list of begin times, or if the begin time was forced by an earlier hyperlink or a beginElement() method call), seek the document time (forward or back, as needed) to the earliest resolved begin time of the target element. Note that the begin time may be resolved as a result of an earlier hyperlink, DOM or event activation. Once the begin time is resolved (and until the element is reset, e.g. when the parent repeats), hyperlink traversal always seeks. For a discussion of "reset", see Resetting element state. Note also that for an element begin to be resolved, the begin time of all ancestor elements must also be resolved.
  3. Else (i.e. element begin time is unresolved), the target element begin time must be resolved. This may require seeking and/or resolving ancestor elements as well. This is done by recursing from the target element up to the closest ancestor element that has a resolved begin time (again noting that for an element to have a resolved begin time, all of its ancestors must have resolved begin times). Then, the recursion is "unwound", and for each ancestor in turn (beneath the resolved ancestor) as well as the target element, the following steps are performed:
    1. If the element is active, seek the document time back to the (current) begin time of the element. If there are multiple begin times, use the begin time that corresponds to the current "begin instance".
    2. Else if the begin time is resolved, seek the document time (forward or back, as needed) to the earliest resolved begin time of the target element.  
    3. Else (if the begin time is not resolved), just resolve the element begin time at the current time on its parent time container (given the current document position). Disregard the sync-base or event base of the element, and do not "back-propagate" any timing logic to resolve the element, but rather treat it as though it were defined with begin="indefinite" and just resolve begin time to the current parent time.

In the above rules, the following additional constraints must also be respected:

  1. If a begin time to be used as the seek target occurs before the beginning of the parent time container, the seek-to time is clamped to the begin time of the parent time container. This constraint is applied recursively for all ascendant time containers.
  2. If a begin time to be used as the seek target occurs after the end of the parent simple duration, then the seek-to time is clamped to the end time of the parent time container simple duration. This constraint is applied recursively for all ascendant time containers.

Note that the first constraint means that a hyperlink to a child of a time container will never seek to a time earlier than the beginning of the time container.  The second constraint implies that a hyperlink to a child that begins after the end of the parent simple duration will seek to the end of the parent, and proceed from there. While this may produce surprising results, it is the most reasonable fallback semantic for what is essentially an error in the presentation.

If a seek of the presentation time is required, it may be necessary to seek either forward or backward, depending upon the resolved begin time of the element and the presentation current time at the moment of hyperlink traversal. 

After seeking a document forward, the document should be in the same state as if the user had allowed the presentation to run normally from the current time until reaching the element begin time (but had otherwise not interacted with the document). In particular, seeking the presentation time forward should also begin any other elements that have resolved begin times between the current time and the seeked-to time. The elements that are begun in this manner may still be active, may be frozen, or may already have ended at the seeked-to time. If an element has ended, it logically begins and ends during the seek. The associated DOM events are raised, and all time dependents are updated. Also any elements currently active at the time of hyperlinking should "fast-forward" over the seek interval. These elements may also be active, frozen or already ended at the seeked-to time. The net effect is that seeking forward to a presentation time puts the document into a state identical to that as if the document presentation time advanced undisturbed to reach the seek time.

If the resolved activation time for an element that is the target of a hyperlink traversal occurs in the past, the presentation time must seek backwards. Seeking backwards will rewind any elements active during the seek interval and will turn off any elements that are resolved to begin at a time after the seeked-to time. Note that resolved begin times (e.g. a begin associated with an event) are not cleared or lost by seeking to an earlier time. Note further that seeking to a time before a resolved begin time does not affect the interpretation of a "restart=never" setting for an element; once the begin time is resolved, it cannot be changed or restarted. Subject to the rules above for hyperlinks that target timed elements, hyperlinking to elements with resolved begin times will function normally, advancing the presentation time forward to the previously resolved time. When the document seeks backwards before a resolved begin for an element time, this does not reset the element

These hyperlinking semantics assume that a record is kept of the resolved begin time for all elements, and this record is available to be used for determining the correct presentation time to seek to. Once resolved, begin times are not cleared by hyperlinking. However, they can be overwritten by subsequent resolutions driven by multiple occurrences of an event (i.e. by restarting). For example: 

<par begin="0">
   <img id="A" begin="10s" .../>
   <img id="B" begin="A.begin+5s" .../>
   <img id="C" begin="click" .../>
   <img id="D" begin="C.begin+5s" .../>
   ...
   <a href="#D">Click here!</a>
</par>

The begin time of elements "A" and "B" can be immediately resolved to be at 10 and 15 seconds respectively. The begin of elements "C" and "D" are unresolved when the document starts. Therefore activating the hyperlink will have no effect upon the presentation time or upon elements "C" and "D". Now, assume that "C" is clicked at 25 seconds into the presentation. The click on "C" in turn resolves "D" to begin at 30 seconds. From this point on, traversing the hyperlink will cause the presentation time to be seeked to 30 seconds.

If at 60 seconds into the presentation, the user again clicks on "C", "D" will become re-resolved to a presentation time of 65 seconds. Subsequent activation of the hyperlink will result in the seeking the presentation to 65 seconds.

If the time container were defined to repeat, or could restart, then all indeterminate times for children of the time container are cleared (reset to "indefinite") when the parent time container repeats or restarts. See also Resetting element state.

Implications of beginElement() and hyperlinking for seq and excl time containers

For a child of a sequence time container, if a hyperlink targeted to the child is traversed, this seeks the sequence to the beginning of the child. If the seek is forward in time and the child does not have a resolved begin time, the document time must seek past any scheduled active end on preceding elements, and then activate the referenced child. In such a seek, if the currently active element does not have a resolved active end, it should be ended at the current time. If there are other intervening siblings (between the currently playing element and the targeted element), the document time must seek past all scheduled times, and resolve any unresolved times as seek proceeds (time will resolve to intermediate values of "now" as this process proceeds). As times are resolved, all associated time dependents get notified as the intervening elements are activated and deactivated.

When beginElement() or beginElementAt() is called for the child of a sequence time container (subject to restart semantics), any currently active or frozen child is stopped and the new child is begun at the current time (even if the element has a scheduled begin time). Unlike hyperlinking, no seek is performed. The sequence will play normally following the child that is begun with the method call (i.e. as though the child had begun at its normal time).

@@The above semantic keeps things simple, although it may point up a need for a seekToElement() method in the DOM interface that mimics the hyperlink functionality.

Note that if a hyperlink targets (or if beginElement() or beginElementAt() is called for) an element A defined to begin when another element B ends, and the other element B has (e.g.) an event-base or syncbase end, the hyperlink or method call will not end element B. It will only active element A. If the two elements are siblings within a <seq> or <excl> time container, the parent time container enforces its semantics and stops (or pauses) the running element.

@@ What if the target element is in an excl and is in a priorityclass that is defined to be deferred if it tries to interrupt the current (higher priority) element? Do we overrule the priorityClass rules, and just stop or pause the running element?

Note that the presentation agent need not actually prepare any media for elements that are seeked over, but it does need to propagate the sync behavior to all time dependents so that the effect of the seek is correct. 

Propagating changes to times

There are several cases in which times may change as the document is presented. In particular, when an element time is defined relative to an event, the time (i.e. the element begin or active end) is resolved when the event occurs. Another case arises with restart behavior - both the begin and active end time of an element can change when it restarts. Since the begin and active end times of one element can be defined relative to the begin or active end of other elements, any changes to times must be propagated throughout the document.

When an element "foo" has a begin or active end time that specifies a syncbase element (e.g. "bar" as below):

<img id="foo" begin="bar.end" .../>

we say that "foo" is a time-dependent of "bar" - that is, the "foo" begin time depends upon the active end of "bar". Any changes to the active end time of "bar" must be propagated to the begin of "foo". The effect of the changes depends upon the state of "foo" when the change happens. The rest of the section describes the specific rules for propagating changes.

Note that it is possible for the syncbase element "bar" to end again, if it is restarted. When "bar" restarts, the a new end time is calculated and all time dependents are notified of the change.  For  example:

<img id="foo" begin="0" end="bar.end" .../>
<img id="bar" begin="btn.click" dur="5s" .../> 

Element "foo" will end when "bar" ends, however "bar" can restart on another click. When "bar" restarts, a new end is calculated, and "foo" is notified. However, as "foo" will not restart, the change is ignored. A variant on this illustrates a case when the time change does propagate through:

<img id="foo" begin="0" end="bar.end+10s" .../>  
<img id="bar" begin="btn.click" dur="5s" .../>

Element "foo" will end 10 seconds after "bar" ends. If "bar" is restarted within 10 seconds of when it first ended, "foo" will still be active, and the changed end time will propagate through.  Using example times, if the user clicks on the "btn" element 8 seconds after the parent time container begins, "bar" begins at 8 seconds and will end at 13 seconds. Element "foo" would then end at 23 seconds.  If the users clicks "btn" again 3 seconds after "bar" ends, (i.e. at 16 seconds), the end of "bar" now has the value of 21 seconds. This change propagates to "foo", and "pushes out" the end of "foo" until 31 seconds. 

The rule is that once an element has ended its active duration, changes that affect its end time are ignored (within the current simple duration of the parent time container).

When an element restarts (or when an ascendant time container repeats or restarts), all child times are recalculated, and may again become indefinite. For example:

<img id="foo" begin="btn.click" end="mouseout" .../> 
<img id="bar" begin="btn.click" end="foo.end"  .../> 

Both elements will start when the "btn" element is first clicked. Element "foo" will end when "mouseout" is raised on the img. At this point, the active duration of "bar" will become defined (resolved), and "bar" will end the active duration. If the user clicks on the target element again, both elements will restart, and "bar" will once again have an indefinite active duration. 

@@Add additional example with explanation. Note that if user clicks at 4s, image1 is never seen. The resolution of image end happens and sticks when image1 begin is evaluated.

<par>
   <img id="image3" dur = "24s" end="user.click"/>
   <img id="image1" begin ="10s" end="image3.end"/>
</par>

Handling negative offsets

The use of negative offsets to define begin times merely defines the synchronization relationship of the element. If does not in any way override the time container constraints upon the element, and it cannot override the constraints of presentation time. 

If an element has a begin time that resolves to a time before the parent time container begins, the parent time container constraint still applies. For example:

<par>
   <video begin="-5s" dur="30s" src="movie.mpg" />
</par>

The video element cannot begin before the par begins. The begin is simply defined to occur "in the past" when the par begins.

A begin or end time may be specified with a negative offset relative to an event or to a syncbase that is not initially resolved.  When the syncbase or eventbase time is (eventually) resolved, the dependent time that is computed with a negative offset may occur in the past. The computed time defines the scheduled synchronization relationship of the element, even if it is not possible to begin or end the element at the computed time.

When a begin time is defined to be in the past, the element begins immediately, but acts as though it had begun at the specified time (playing from an offset into the media). The behavior can be thought of as a clipBegin value applied to the element, that only applies to the first iteration of repeating elements. The media will actually begin at the time computed according to the following equation:

Let o be the offset value, d is the simple duration, AD is the active duration. 
If AD is indefinite, it compares greater than any value of o or ABS(o).
REM( x, y ) is defined as x - floor( x/y ). 
If y is indefinite, REM( x, y ) is just x.
If ABS(o) >= AD the element does not begin.
Else the element media begins at REM( ABS(o), d ).

If the element repeats, the iteration value of the repeat event has the calculated value based upon the computed begin, and not the observed number of repeats. Thus for example:

<ref begin="foo.click-8s" dur="3s" repeatCount="10" .../>

The element begins when the user clicks on the element "foo". It begins to play at 2 seconds into the 3 second simple duration. Any time dependents are activated relative to the computed begin time, and not the observed begin time. The begin event is raised  when the element begins, but has a timeStamp value that corresponds to the defined begin time, 8 seconds earlier. One second later, the element will repeat, and the associated repeat event will have the iteration value set to 3 (it is zero based). The element will end 22 seconds after the click.

Note: If script authors wish to distinguish between the computed repeat iterations and observed repeat iterations, they can count actual repeat events in the associated event handler.

Behavior of 0 duration elements

Media elements with an active duration of zero or with the same begin and end time trigger begin and end events, and propagate to time dependents. If an element's end time is before its begin time, no events are triggered (see also Evaluation of begin and end time lists).

Whether or not media is retrieved and/or rendered is implementation dependent.

Resetting element state

When a time container repeats or restarts, all descendent children are "reset" with respect to certain state:

  1. Any event-base times that were resolved are reset to unresolved (equivalent to "indefinite").
  2. Any syncbase times are reevaluated (i.e. the translation between timespaces must be recalculated - see Converting between local and global times).
  3. Any state associated with the interpretation of the restart semantics is reset. Thus, for example if an element specifies restart="never", the element can begin again after a reset. The restart="never" setting is only defined for the extent of the parent time container simple duration.

When an element restarts, the rules 1 and 2 are also applied to the element itself, although the rule 3 (controlling restart behavior) is not applied.

Note that when any time container ends its simple duration (including when it repeats), all timed children that are still active are ended. See also Time container constraints on child durations.

When an <excl> time container restarts or repeats, in addition to ending any active or paused children, the pause queue for the <excl> is cleared.

10.3.6 Controlling runtime synchronization behavior

Proposed new support in SMIL Boston introduces finer grained control over the runtime synchronization behavior of a document.  The syncBehavior attribute allows an author to describe for each element whether it must remain in a hard sync relationship to the parent time container, or whether it can be allowed slip with respect to the time container. Thus, if network congestion delays or interrupts the delivery of media for an element, the syncBehavior attribute controls whether the media element can slip while the rest of the document continues to play, or whether the time container must also wait until the media delivery catches up.

The syncBehavior attribute can also be applied to time containers. This controls the sync relationship of the entire timeline defined by the time container.  In this example, the audio and video elements are defined with hard or "locked" sync to maintain lip sync, but the "speech" <par> time container is allowed to slip:

<par>
   <animation src="..." />
   ...
   <par id="speech" syncBehavior="canSlip" >
      <video src="speech.mpg" syncBehavior="locked" />
      <audio src="speech.au"  syncBehavior="locked" />
   </par>
   ...
</par>

If either the video or audio must pause due to delivery problems, the entire "speech" par will pause, to keep the entire timeline in sync. However, the rest of the document, including the animation element will continue to play normally. Using the syncBehavior attribute on elements and time containers, the author can effectively describe the "scope" of runtime sync behavior, defining some portions of the document to play in hard sync without requiring that the entire document use hard synchronization.

This functionality also applies when an element first begins, and the media must begin to play. If the media is not yet ready (e.g. if an image file has not yet downloaded), the syncBehavior attribute controls whether the time container must wait until the element media is ready, or whether the element begin can slip until the media is downloaded.

The syncBehavior can affect the effective begin and effective end of an element, but the use of the syncBehavior attribute does not introduce any other semantics with respect to duration.

When the syncBehavior attribute is combined with interactive begin timing for an element, the syncBehavior only applies once the sync relationship of the element is resolved (e.g. when the specified event is raised). If at that point the media is not ready and syncBehavior is specified as "locked", then the parent time container must wait until the media is ready. Once an element with an interactive begin time has begun playing, the syncBehavior semantics described above apply as thought the element were defined with scheduled timing.

The syncBehavior attribute is subordinate to any sync relationships defined by time containers, sync arcs, event arcs, etc. The syncBehavior attribute has no bearing on the formation of the time graph, only the enforcement of it.

@@ Need to address how syncBehavior will interact with restart semantics.  In particular, do we re-establish the sync relationship when it restarts (this is my first guess, assuming that restart is allowed).  syncBehavior is not supposed to define all the behavior of the element, but rather just the behavior when there is some problem with sync, or when the user pauses or seeks an element.  E.g. we do not require that children of seq elements be locked, but we do require that the seq semantics be maintained. If restart is allowed, then that should be orthogonal to the syncBehavior. Note that all other aspects of timing (e.g. repeat, parent constraints and event-based timing override the syncBehavior, so I think we have a precedent.

Note that the semantics of syncBehavior do not describe or require a particular approach to maintaining sync; the approach will be implementation dependent. Possible means of resolving a sync conflict may include:

Additional control is provided over the hard sync model using the syncTolerance attribute. This specifies the amount of slip that can be ignored for an element.  Small variance in media playback (e.g. due to hardware inaccuracies) can often be ignored, and allow the overall performance to appear smoother.

@@ We need to move the general definition of the SMIL Default stuff elsewhere, and just specify the possible arg values here. Ideas where it should go?

The value of the syncBehavior and syncTolerance attributes are not inherited, but it is possible to set default behavior for elements or time containers using the SMILdefault syntax below.

The default value for syncTolerance is implementation dependent, but should be no greater than two seconds.

Sync behavior attributes

syncBehavior
Defines the runtime synchronization behavior for an element.
Legal values are:
canSlip
Allows the associated node to slip with respect to the parent time container.
When this value is used, any syncTolerance attribute is ignored.
locked
Forces the associated node to maintain sync with respect to the parent time container. This can be eased with the use of the syncTolerance attribute.
syncTolerance
This attribute on timed elements and time containers defines the synchronization tolerance for the associated element. It has an effect only if the element has syncBehavior="locked". This allows a locked sync relationship to ignore a given amount of slew without forcing resynchronization. resolution.
Legal values are Clock-values.
SMILdefault
Defines the default value for the runtime synchronization behavior for an element and all descendents, or the default synchronization tolerance for the associated element.
Legal values are:
syncBehavior:canSlip
Allows the associated node to slip with respect to the parent time container.
When this value is used, any syncTolerance attribute is ignored.
syncBehavior:locked
Forces the associated node to maintain sync with respect to the parent time container. This can be eased with the use of the syncTolerance attribute.
syncTolerance: Clock-value
Defines the default value for the synchronization tolerance for an element, and all descendents. 

Sync master support

An additional proposed extension allows the author to specify that a particular element should define or control the synchronization for a time container. This is similar to the default behavior of many players that "slave" video and other elements to audio, to accommodate the audio hardware inaccuracies and the sensitivity of listeners to interruptions in the audio playback. The syncMaster attribute allows an author to explicitly define that an element defines the playback "clock" for the time container, and all other elements should be held in sync relative to the syncMaster element.

In practice, linear media often need to be the syncMaster, where non-linear media can more easily be adjusted to maintain hard sync.  However, a player cannot always determine which media behaves in a linear fashion and which media behaves in a non-linear fashion. In addition, when there are multiple linear elements active at a given point in time, the player cannot always make the "right" decision to resolve sync conflicts. The syncMaster attribute allows the author to specify the element that has linear media, or that is "most important" and should not be compromised by the syncBehavior of other elements.

The syncMaster attribute interacts with the syncBehavior attribute. An element with syncMaster set to true will define sync for the "scope" of the time container's synchronization behavior. That is, if the syncMaster element's parent time container has syncBehavior="locked", the syncMaster will also define sync for the ancestor time container. The syncMaster will define sync for everything within the closest ancestor time container that is defined with syncBehavior="canSlip".

The syncMaster attribute only applies when an element is active. If more than one element within the syncBehavior scope has the syncMaster attribute set to true, and the elements are both active at any moment in time, the behavior will be implementation dependent.

syncMaster
Boolean attribute on media elements and time containers that forces the time container playback to sync to this element.


The default value is false.
The associated property is read-only, and cannot be set by script.

10.3.7 Common syntax DTD definitions

@@ Need to decide whether endSync belongs on media elements (with timed children) or not.

Timing attributes

<!ENTITY % timingAttrs
  begin          CDATA  #IMPLIED 
  dur            CDATA  #IMPLIED
  end            CDATA  #IMPLIED
  restart        (always | never | 
                   whenNotActive)  "always"
  repeatCount    CDATA  #IMPLIED 
  repeatDur      CDATA  #IMPLIED
  fill           (remove | freeze | hold) "remove"
>

Runtime sync behavior attributes

<!ENTITY % runtimeSyncBvrAttrs
  syncBehavior         (locked | canSlip) #IMPLIED
  defaultSyncBehavior  (locked | canSlip) "canSlip"
  syncTolerance        CDATA              #IMPLIED
  defaultSyncTolerance CDATA              #IMPLIED
  syncMaster           (true | false)     "false"
>

Time container elements

<!ELEMENT par ???>
<!ATTLIST par
  %timingAttrs
  %runtimeSyncBvrAttrs
  id             ID     #IMPLIED 
  endSync        CDATA  #IMPLIED 
>
<!ELEMENT seq ???>
<!ATTLIST seq
  %timingAttrs
  %runtimeSyncBvrAttrs
  id             ID     #IMPLIED 
>
<!ELEMENT excl ???>
<!ATTLIST excl
  %timingAttrs
  %runtimeSyncBvrAttrs
  id             ID     #IMPLIED 
  endSync        CDATA  #IMPLIED 
>

 

10.4 Integrating SMIL Timing and Synchronization into a host language

@@When this settles down, fill in subsections in TOC

This section describes what a language designer must actually do to specify the integration of SMIL Timing and Synchronization support into a host language. This includes basic definitions, constraints upon specification, and allowed/supported events.

10.4.1 Required host language definitions

The host language designer must define some basic concepts in the context of the particular host language.  These provide the basis for timing and presentation semantics.

The host language designer must define what "presenting a document" means. A typical example is that the document is displayed on a screen.

The host language designer must define the document begin. Possible definitions are that the document begins when the complete document has been received by a client over a network, or that the document begins when certain document parts have been received.

The host language designer must define the document end. This is typically when the associated application exits or switches context to another document.

@@ Check to see if we really have issues with this, e.g. for specifying floating point values. Can we use this in our definitions of offset value?

The host language must specify the formats supported for numeric attribute values. This includes integer values and especially floating point values for attributes such as keyTimes and keySplines. As a reasonable minimum, host language designers are encouraged to support the format described in [CSS2]. The specific reference within the CSS specification for these data types is 4.3.1 Integers and real numbers.

10.4.2 Required definitions and constraints on element timing

@@ Need to talk about specifying which elements can be timed, and what it means to time them.

The set of elements that may have timing includes ??? elements defined in host languages. 

Supported events for event-base timing

The host language must specify which event names are legal in event base values. If the host language defines no allowed event names, event-based timing is effectively precluded for the host language. 

Host languages may specify that dynamically created events (as per the [DOM2Events] specification) are legal as event names, and not explicitly list the allowed names.

10.4.3 Error handling semantics

The host language designer may impose stricter constraints upon the error handling semantics. That is, in the case of syntax errors, the host language may specify additional or stricter mechanisms to be used to indicate an error. An example would be to stop all processing of the document, or to halt all animation.

@@Broken link to Handling errors - Do we need a section on this?

Host language designers may not relax the error handling specifications, or the error handling response (as described in "Handling syntax errors"). For example, host language designers may not define error recovery semantics for missing or erroneous values in the begin or end attribute values.

10.4.4 SMIL Timing and Synchronization namespace

Language designers can choose to integrate SMIL Timing and Synchronization as an independent namespace, or can integrate SMIL Timing and Synchronization names into a new namespace defined as part of the host language. Language designers that wish to put the SMIL Timing and Synchronization functionality in an isolated namespace should use the following namespace:

@@ URI to be confirmed by W3C webmaster

http://www.w3.org/2000/TR/smil20

 

10.5 Document object model support

Any XML-based language that integrates SMIL Timing will inherit the basic interfaces defined in DOM [DOM2] (although not all languages may require a DOM implementation). SMIL Timing specifies the interaction of timing functionality and DOM. SMIL Timing also defines constraints upon the basic DOM interfaces, and specific DOM interfaces to support SMIL Timing.

Much of the related SMIL-DOM functionality is proposed in the [SMIL-DOM] section.  We may need to go into further detail on the specific semantics of the interfaces - the sections below are placeholders.

10.5.1 Element and attribute manipulation, mutation and constraints

Define rules on element and attribute access (inherit from and point to Core DOM docs for this). Define mutation constraints. This is currently covered in the [SMIL-DOM] section.

10.5.2 Event model

SMIL event-timing assumes that the host language supports events, and that the events can be bound in a declarative manner.  DOM Level 2 Events [DOM2Events] describes functionality to support this.

The specific events supported are defined by the host language. If no events are defined by a host language, event-timing is effectively omitted.

The [SMIL-DOM] section defines the initial set of time-related events that have been proposed.

10.5.3 Supported methods

SMIL Timing supports two methods for controlling the timing of elements: beginElement() and endElement(). These methods are used to begin and end the active duration of an element. Authors can (but are not required to) declare the timing to respond to the DOM using the following syntax:

<img begin="indefinite" end="indefinite" .../>

The beginElement(), beginElementAt() and endElement() methods are all subject to time container constraints in much the same way that event-based times are. If any of these methods are called when the parent time container is not active, the methods have no effect.

Calling beginElement() causes the element to begin in the same way that an element with event-based begin timing begins. The effective begin time is the current presentation time at the time of the DOM method call. Note that beginElement() is subject to the restart attribute in the same manner that event-based begin timing is. If an element is specified to disallow restarting at a given point, beginElement() methods calls must fail. Refer also to the section Restarting elements.

Calling beginElementAt() causes the element to begin in the same way that an element begins with event-based begin timing that includes an offset.

Calling endElement() causes an element to end the active duration, just as end does. Depending upon the value of the fill attribute, the element effect may no longer be applied, or it may be frozen at the current effect. Refer also to the section Freezing elements. If an element is not currently active (i.e. if it has not yet begun or if it is frozen), the endElement() method will fail.

Interfaces are currently defined in the [SMIL-DOM] section.

10.6 Glossary

10.6.1 General concepts

The following concepts are the basic terms used to describe the timing model.

Time graph

A time graph is used to represent the temporal relations of elements in a document with SMIL timing. Nodes of the time graph represent elements in the document. Parent nodes can "contain" children, and children have a single parent.  Siblings are elements that have a common parent.  The links or "arcs" of the time graph represent synchronization relationships between the nodes of the graph.

Note that this definition is preliminary.

Descriptive terms for times

The time model description uses a set of adjectives to describe particular concepts of timing:

implicit
This describes a time that is defined intrinsically by the element media (e.g. based upon the length of a movie), or by the time model semantics.
explicit
This describes a time that has been specified by the author, using the SMIL syntax.
desired
This is a time that the author intended - it is generally the explicit time if there is one, or the implicit time if there is no explicit time.
effective
This is a time that is actually observed at document playback. It reflects both the constraints of the timing model as well as real-world issues such as media delivery.

Scheduled timing

An element is considered to have scheduled timing if the element's start time is given relative to the begin or active end of another element. A scheduled element can be inserted directly into the time graph.

Events and interactive timing

Begin and active end times in SMIL Boston can be specified to be relative to events that are raised in the document playback environment.  This supports declarative, interactive timing. Interactive  in this sense includes user events such as mouse clicks, events raised by media players like a mediaComplete event, and events raised by the presentation engine itself such as a pause event.

More information on the supported events and the underlying mechanism is described in the DOM section of this draft [SMIL-DOM].

Syncbases

In scheduled timing, elements are timed relative to other elements. The syncbase for an element A is the other element B to which element A is relative. More precisely, it is the begin or active end of the other element. The syncbase is not simply a scheduled point in time, but rather a point in the time graph.

Note that this definition is preliminary.  The name may also change.

Sync arcs

"Sync-arc" is an abbreviation for "synchronization arc". Sync-arcs are used to relate nodes in the time graph, and define the timing relationship between the nodes.  A sync-arc relates an element to its syncbase. The sync-arc may be defined implicitly by context, explicitly by id-ref or event name, or logically with special syntax.

Note that this definition is preliminary.

Clocks

A Clock is a particular timeline reference that can be used for synchronization.  A common example that uses real-world local time is referred to as wall-clock timing (e.g. specifying 10:30 local time). Other clocks may also be supported by a given presentation environment.

Hyperlinking and timing

A hyperlink into or within a timed document may cause a seek of the current presentation time or may activate an element (if it is not in violation of any timing model rules).

Activation

During playback, an element may be activated automatically by the progression of time, via a hyperlink, or in response to an event. When an element is activated, playback of the element begins.

Discrete and continuous Media

SMIL includes support for declaring media, using element syntax defined in "The SMIL Media Object Module". The media that is described by these elements is described as either discrete or continuous:

discrete
The media does not have intrinsic timing, or intrinsic duration. These media are sometimes described as "rendered" or "synthetic" media. This includes images, text and some vector media.
continuous
The media is naturally time-based, and generally supports intrinsic timing and an intrinsic notion of duration (although the duration may be indefinite). These media are sometimes described as "time-based" or "played" media. This includes most audio, movies, and time-based animations.

10.6.2 Timing concepts

Time containers

Time containers group elements together in time.  They define common, simple synchronization relationships among the grouped child elements. In addition, time containers constrain the time that children may be active. Several containers are defined, each with specific semantics and constraints on its children.

Content/Media elements

SMIL timing and synchronization support ultimately controls a set of content or media elements.  The content includes things like video and audio, images and vector graphics, as well as text or HTML content.  SMIL documents use the SMIL media elements to reference this content. XML and HTML documents that integrate SMIL Boston functionality may use SMIL media elements and/or content described by the integrated language (e.g. paragraphs in HTML).

Basic markup

All elements - content/media as well as time containers - support timing markup to describe a begin time and a duration, as well as the ability to play repeatedly. There are several ways to define the begin time. The semantics vary somewhat depending upon an element's time container.

Simple and active durations

The time model defines two concepts of duration for each element - the simple duration and the active duration. These definitions are closely related to the concept of playing something repeatedly.

simple duration
This is the duration defined by the basic begin and duration markup.  It does not include any of the effects of playing repeatedly, or of fill.  The simple duration is defined by the explicit begin and duration, if one is specified.  If the explicit times are not specified, the simple duration is defined to be the implicit duration of the element.
active duration
This is the duration during which the element plays normally.  If no repeating behavior is specified, and end is not specified, the active duration is the same as the simple duration.  If the element is set to play repeatedly, the simple duration is repeated for the active duration, as defined by the repeat markup. The active duration does not include the effect of fill.

The constraints of a parent time container may override the duration of its children. In particular, a child element may not play beyond the simple end of the time container.

The terms for these durations can be modified with the Descriptive Terms for Times, to further distinguish aspects of the time graph.

Time manipulations

Time manipulations allow the element's time (within the simple duration) to be filtered or modified. For example the speed of time can be varied to make the element play faster or slower than normal. The filtered time affects all descendents of the element.  Several time manipulations are proposed for SMIL Boston. Time manipulation is primarily intended to be used with animation [SMIL-ANIMATION] (W3C members only).

Note that any time manipulation that changes the effective play speed of an element's time may conflict with the basic capabilities of some media players.  The use of these manipulations is not recommended with linear media players, or with time containers that contain linear media elements, such as streaming video.

There are a number of unresolved issues with this kind of time manipulation, including issues related to event-based timing and negative play speeds, as well as many media-related issues.

Determinate and indeterminate schedules

Using simple, scheduled timing, a time graph can be described in which all the times have a known, defined sync relationship to the document timeline.  We describe this as determinate timing.

When timing is specified relative to events or external clocks, the sync relationship is not initially defined. We describe this as indeterminate timing.

A time is resolved when the sync relationship is defined, and the time can actually be scheduled on the document time graph.

Indeterminate times that are event-based are resolved when the associated event occurs at runtime - this is described more completely in the section Unifying Scheduling and Interactive Timing. Indeterminate times that are defined relative to external clocks are usually resolved when the document playback begins, and the relationship of the document timeline to the external clock reference is defined.

A determinate time may initially be unresolved, e.g. if it is relative to an unknown time such as the end of a streaming MPEG movie (the duration of an MPEG movie is not known until the entire file is downloaded).  When the movie finishes, determinate times defined relative to the end of the movie are resolved.

Hard and soft sync

SMIL 1.0 introduced the notion of synchronization behavior, describing user agent behavior as implementing either "hard synchronization" or "soft synchronization".  Using hard sync, the entire presentation would be constrained to the strict description of sync relationships in the time graph. Soft sync allowed for a looser (implementation dependent) performance of the document.

While a document is playing, network congestion and other factors will sometimes interfere with normal playback of media. In a SMIL 1.0 hard sync environment, this will affect the behavior of the entire document. In order to provide greater control to authors, SMIL Boston extends the hard and soft sync model to individual elements. This support allows authors to define which elements and time containers must remain in strict or "hard" sync, and which elements and time containers can have a "soft" or slip sync relationship to the parent time container.

10.7 Appendix A: Annotated examples

10.7.1 Example 1: Simple timing within a Parallel time container

This section includes a set of examples that illustrate both the usage of the SMIL syntax, as well as the semantics of specific constructs.  This section is informative.

Note: In the examples below, the additional syntax related to layout and other issues specific to individual document types is omitted for simplicity.

All the children of a <par>  begin by default when the <par> begins.  For example:

<par>
   <img id="i1" dur="5s"  src="img.jpg" />
   <img id="i2" dur="10s" src="img2.jpg" />
   <img id="i3" begin="2s" dur="5s" src="img3.jpg" />
</par>

Elements "i1" and "i2" both begin immediately when the par begins, which is the default begin time. The active duration of "i1" ends at 5 seconds into the <par>. The active duration of "i2" ends at 10 seconds into the <par>.  The last element "i3" begins at 2 seconds since it has an explicit begin offset, and has a duration of 5 seconds which means its active duration ends 7 seconds after the <par> begins.

An image that illustrated the timeline might be useful here.

10.7.2 Example 2: Simple timing within a Sequence time container

Each child of a <seq>  begins by default when the previous element ends.  For example:

<seq>
   <img id="i1" begin="0s" dur="5s" src="img1.jpg" />
   <img id="i2" dur="10s" src="img2.jpg" />
   <img id="i3" begin="1s" dur="5s" src="img3.jpg" />
</seq>

The element "i1" begins immediately, with the start of the <seq>, and ends 5 seconds later.  Note: specifying a begin time of 0 seconds is optional since the default begin offset is always 0 seconds.  The second element "i2" begins, by default, 0 seconds after the previous element "i1" ends, which is 5 seconds into the <seq>. Element "i2" ends 10 seconds later, at 15 seconds into the <seq>.  The last element, "i3", has a begin offset of 1 second specified, so it begins 1 second after the previous element "i2" ends, and has a duration of 5 seconds, so it ends at 21 seconds into the <seq>.

Insert illustration.

10.7.3 Example 3: excl time container with child timing variants

  1. Exclusive element, children activated via link-based activation:
      <par>
        <excl>
          <par id="p1"> 
             ...  
          </par>
          <par id="p2">
             ...  
          </par>
        </excl>
        <a href="p1"><img src="Button1.jpg"/></a>
        <a href="p2"><img src="Button2.jpg"/></a>
      </par>
    

    This example models jukebox-like behavior. Clicking on the first image activates the media items of parallel container "p1". If the link on the second image is traversed, "p2" is started (thereby deactivating "p1" if it would still be active).

    Shouldn't we say, here, exactly where the elements of the selected par in the excl should begin when a click happens, e.g., if we are 10 seconds into the outer par and we click on button 2, does the MPG video in p2 start 10 seconds into its stream (in-sync), or does it start at its time 0?

  2. Exclusive element combined with event-based activation:

    Note that the specific syntax for beginEvent argument values is still under discussion.

      <par>
        <excl>
          <par begin="btn1.click"> 
             ...  
          </par>
          <par begin="btn2.click">
             ...  
          </par>
        </excl>
        <img id="btn1" src=... />
        <img id="btn2" src=... />
      </par>
    

    The same jukebox example, using event-based activation.

    In these two examples event-based and anchor-based activation look almost identical, maybe we should come up with examples showing the difference and the relative power of each.

  3. Exclusive element using determinate declarative timing:
      <excl>
        <ref id="a" begin="0s" ... />
        <ref id="b" begin="5s" ... />
      </excl>
    

    In the example above, the beginning of "b" deactivates "a" (assuming that a is still active after 5 seconds).  Note that this could also be modeled using a sequence with an explicit duration on the children.  While the determinate syntax is allowed, this is not expected to be a common use-case scenario.

    Issue - should we preclude the use of determinate timing on children of excl?  Other proposals would declare one child (possibly the first) to begin playing by default. Proposals include an attribute on the <excl> container that indicate one child to begin playing by default.

10.7.4 Example 4:  default duration of discrete media

For simple media elements (i.e. media elements that are not time containers) that reference discrete media, the implicit duration is defined to be indefinite. This can lead to surprising results, as in this example:

<seq>
   <img src="img1.jpg" />
   <video src="vid2.mpg" />
   <video src="vid3.mpg" />
</seq>

The default syncbase of a sequence is defined to be the effective active end of the previous element in the sequence, unless the active duration is indefinite in which case the default syncbase is the begin of the previous element. In the example, the implicit duration of the image is used to defined the simple and active durations. As a result, the default begin of the second element causes it to begin at the same time as the image. Thus, the image will not show at all!  Authors will generally specify an explicit duration for any discrete media elements.

10.7.5 Example 5:  end specifies end of active dur, not end of simple dur

There is an important difference between the semantics of end and dur.
The dur attribute, in conjunction with the begin time, specifies the simple duration for an element.

This is the duration that is repeated when the element also has a repeat specified.  The attribute end on the other hand overrides the active duration of the element.  If the element does not have repeat specified, the active duration is the same as the simple duration.  However, if the element has repeat specified, then the end will override the repeat, but will not affect the simple duration.  For example:

<seq repeat="10" end="stopBtn.click">
   <img src="img1.jpg" dur="2s" />
   <img src="img2.jpg" dur="2s" />
   <img src="img3.jpg" dur="2s" />
</seq>

The sequence will play for 6 seconds on each repeat iteration.  It will play through 10 times, unless the user clicks on a "stopBtn" element before 60 seconds have elapsed.

10.7.6 Example 6: SMIL-DOM-initiated timing

When an implementation supports the SMIL-DOM, it will be possible to make an element begin or end the active duration using script or some other browser extension. When an author wishes to describe an element as interactive in this manner, the following syntax can be used:

<audio src="song1.au" begin="indefinite" />

The element will not begin until the SMIL-DOM beginElement() method is called.

10.8 Appendix B: Authoring guidelines (to be added)

This is a placeholder for a set of authoring guidelines intended to help authors avoid potential mistakes and confusion, and to suggest best practices as intended by the authors.

10.9 Appendix C: Differences from SMIL 1.0

SMIL 1.0 defines the model for timing, including markup to define element timing, and elements to define parallel and sequence time containers.  This version introduces some syntax variations and additional functionality, including:

The complete syntax is described here, including syntax that is unchanged from SMIL 1.0.


previous   next   contents