L. SMIL Timing and Synchronization

Previous version:
http://www.w3.org/AudioVideo/Group/Timing/smil-timing-990727.html (W3C members only)
Editors:
Patrick Schmitz (Microsoft),
Warner ten Kate (Philips),
Ted Wugofski (Invited Expert, Gateway),
Jeff Ayars (RealNetworks), Bridie Saccocio (RealNetworks)

Abstract

This is a working draft specification of timing and synchronization functionality for SMIL and other XML documents that incorporate SMIL Timing and Synchronization. It is part of the work in the Synchronized Multimedia Working Group (SYMM) towards a next version of the SMIL language (SMIL Boston) and associated modules. This version extends the Timing and Synchronization support available in the SMIL 1.0 specification.


Table of Contents


1.0 Introduction

SMIL 1.0 solved fundamental media synchronization problems and defined a powerful way of choreographing multimedia content. SMIL Boston extends the timing and synchronization support, adding capabilities to the timing model and associated syntax. This section of the document specifies the Timing and Synchronization module.

There are two intended audiences for this module: implementers of SMIL Boston document viewers or authoring tools, and authors of other XML languages who wish to integrate timing and synchronization support.

In the process of extending SMIL 1.0 for modularization and use in other XML languages, some alternate syntaxes have been defined. If a document would otherwise be SMIL 1.0 compatible except for use of alternate syntax, the use of the SMIL 1.0 syntax is recommended so the document will be playable by SMIL 1.0 as well as later document players.

As this module is used in different profiles, the associated syntax requirements may vary.  Differences in syntax should be minimized as much as is practical.  The semantics of the timing model and of the associated markup must remain consistent across all profiles.  Any document type that includes SMIL Boston Timing and Synchronization markup (either via a hybrid DTD or via namespace qualified extensions) must preserve the semantics of the model defined in this specification.

The specification of timing and synchronization is organized in the following way. Time model concepts are introduced first, followed by a normative description of the time model and time graph construction. Clarification is provided for aspects of the model that were insufficiently documented in SMIL 1.0. The associated SMIL-DOM interfaces are described next. Open issues and examples are separated into appendices for readability purposes.

2.0 Background and Concepts

There are several important terms and concepts that must be introduced to describe the time model. This section first describes general terms and then defines basic timing concepts used throughout the document. This section ends with a discussion of how the SMIL timing model is being extended to support not only scheduled but also interactive presentations.

Note that certain areas of the model are still under discussion. A future draft will more precisely define the complete model, and interactions among the new functionality.

2.1. General concepts

The following concepts are the basic terms used to describe the timing model.

2.1.1. Time Graph

A time graph is used to represent the temporal relations of elements in a document with SMIL timing. Nodes of the time graph represent elements in the document. Parent nodes can "contain" children, and children have a single parent.  Siblings are elements that have a common parent.  The links or "arcs" of the time graph represent synchronization relationships between the nodes of the graph.

Note that this definition is preliminary.

2.1.2. Descriptive Terms for Times

The time model description uses a set of adjectives to describe particular concepts of timing:

implicit
This describes a time that is defined intrinsically by the element media (e.g. based upon the length of a movie), or by the time model semantics.
explicit
This describes a time that has been specified by the author, using the SMIL syntax.
desired
This is a time that the author intended - it is generally the explicit time if there is one, or the implicit time if there is no explicit time.
effective
This is a time that is actually observed at document playback. It reflects both the constraints of the timing model as well as real-world issues such as media delivery.

2.1.3. Scheduled Timing

An element is considered to have scheduled timing if the element's start time is given relative to the begin or end of another element. A scheduled element can be inserted directly into the time graph.

2.1.4. Events and Interactive Timing

Begin and end times in SMIL Boston can be specified to be relative to events that are raised in the document playback environment.  This supports declarative, interactive timing. Interactive  in this sense includes user events such as mouse clicks, events raised by media players like a mediaComplete event, and events raised by the presentation engine itself such as an onPause event.

More information on the supported events and the underlying mechanism is described in the DOM section of this draft [SMIL-DOM].

2.1.5. Timebases

In scheduled timing, elements are timed relative to other elements. The timebase for an element A is the other element B to which element A is relative. More precisely, it is the begin or end of the other element. The timebase is not simply a scheduled point in time, but rather a point in the time graph.

Note that this definition is preliminary.  The name may also change.

2.1.6. Sync Arcs

"Sync-arc" is an abbreviation for "synchronization arc". Sync-arcs are used to relate nodes in the time graph, and define the timing relationship between the nodes.  A sync-arc relates an element to its timebase. The sync-arc may be defined implicitly by context, explicitly by id-ref or event name, or logically with special syntax.

Note that this definition is preliminary.

2.1.7. Clocks

A Clock is a particular timeline reference that can be used for synchronization.  A common example that uses real-world local time is referred to as wall-clock timing (e.g. specifying 10:30 local time). Other clocks may also be supported by a given presentation environment.

2.1.8. Hyperlinking and Timing

A hyperlink into or within a timed document may cause a seek of the current presentation time or may activate an element (if it is not in violation of any timing model rules).

2.1.9. Activation

During playback, an element may be activated automatically by the progression of time, via a hyperlink, or in response to an event. When an element is activated, playback of the element begins.

2.1.10. Discrete and Continuous Media

SMIL includes support for declaring media, using element syntax defined in [SMIL-MEDIA]. The media that is described by these elements is described as either discrete or continuous:

discrete
The media does not have intrinsic timing, or intrinsic duration. These media are sometimes described as "rendered" or "synthetic" media. This includes images, text and some vector media.
continuous
The media is naturally time-based, and generally supports intrinsic timing and an intrinsic notion of duration (although the duration may be indefinite). These media are sometimes described as "time-based" or "played" media. This includes most audio, movies, and time-based animations.

2.2. Timing Concepts

2.2.1. Time Containers

Time containers group elements together in time.  They define common, simple synchronization relationships among the grouped child elements. In addition, time containers constrain the time that children may be active. Several containers are defined, each with specific semantics and constraints on its children.

2.2.2. Content/Media Elements

SMIL timing and synchronization support ultimately controls a set of content or media elements.  The content includes things like video and audio, images and vector graphics, as well as text or HTML content.  SMIL documents use the SMIL media elements to reference this content. XML and HTML documents that integrate SMIL Boston functionality may use SMIL media elements and/or content described by the integrated language (e.g. paragraphs in HTML).

2.2.3. Basic Markup

All elements - content/media as well as time containers - support timing markup to describe a begin time and a duration, as well as the ability to play repeatedly. There are several ways to define the begin time. The semantics vary somewhat depending upon an element's time container.

2.2.4. Simple, Active and Complete Durations

The time model defines three concepts of duration for each element - simple, active and complete durations. These definitions are closely related to two other concepts: playing something repeatedly (generally called "repeat"), and applying a "fill" - a presentational modifier that effectively extends the ending state of an element (e.g. the last frame of a movie) to fill out time within the time container.
simple duration
This is the duration defined by the basic begin and duration markup.  It does not include any of the effects of playing repeatedly, or of fill.  The simple duration is defined by the explicit begin and duration, if one is specified.  If the explicit times are not specified, the implicit duration of the element is used.
active duration
This is the duration during which the element plays normally.  If no repeating behavior is specified, the active duration is the same as the simple duration.  If the element is set to play repeatedly, the simple duration is repeated for the active duration, as defined by the repeat markup. The active duration does not include the effect of fill.
complete duration
This is the sum of the active duration and the duration of any fill. The fill may be specified explicitly, or defined implicitly by the time container. The duration of the fill is defined by the constraints of the time container.
The constraints of a parent time container may override the duration of its children. In particular, a child element may not play beyond the end of the time container.
The terms for these durations can be modified with the Descriptive Terms for Times, to further distinguish aspects of the time graph.

2.2.5 Time Manipulations

Time manipulations allow the element's time (within the simple duration) to be filtered or modified. For example the speed of time can be varied to make the element play faster or slower than normal. The filtered time affects all descendents of the element.  Several time manipulations are proposed for SMIL Boston. Time manipulation is primarily intended to be used with animation (W3C members only).
Note that any time manipulation that changes the effective play speed of an element's time may conflict with the basic capabilities of some media players.  The use of these manipulations is not recommended with linear media players, or with time containers that contain linear media elements, such as streaming video.
There are a number of unresolved issues with this kind of time manipulation, including issues related to event-based timing and negative play speeds, as well as many media-related issues.

2.2.6. Determinate and Indeterminate Schedules

Using simple, scheduled timing, a time graph can be described in which all the times have a known, defined sync relationship to the document timeline.  We describe this as determinate timing.

When timing is specified relative to events or external clocks, the sync relationship is not initially defined. We describe this as indeterminate timing.

A time is resolved when the sync relationship is defined, and the time can actually be scheduled on the document time graph.

Indeterminate times that are event-based are resolved when the associated event occurs at runtime - this is described more completely below. Indeterminate times that are defined relative to external clocks are usually resolved when the document playback begins, and the relationship of the document timeline to the external clock reference is defined.

A determinate time may initially be unresolved, e.g. if it is relative to an unknown end such as the end of a streaming MPEG movie (the duration of an MPEG movie is not known until the entire file is downloaded).  When the movie finishes, determinate times defined relative to the end of the movie are resolved.

2.2.7. Hard and Soft Sync

SMIL 1.0 introduced the notion of synchronization behavior, describing user agent behavior as implementing either "hard synchronization" or "soft synchronization".  Using hard sync, the entire presentation would be constrained to the strict description of sync relationships in the time graph. Soft sync allowed for a looser (implementation dependent) performance of the document.

While a document is playing, network congestion and other factors will sometimes interfere with normal playback of media. In a SMIL 1.0 hard sync environment, this will affect the behavior of the entire document. In order to provide greater control to authors, SMIL Boston extends the hard and soft sync model to individual elements. This support allows authors to define which elements and time containers must remain in strict or "hard" sync, and which elements and time containers can have a "soft" or slip sync relationship to the parent time container.

2.3. Unifying Scheduling and Interactive Timing

A significant motivation for SMIL Boston is the desire to integrate declarative, determinate scheduling with interactive, indeterminate scheduling.  The goal is to provide a common, consistent model and a simple syntax.

Note that "interactive" content does not refer simply to hypermedia with support for linking between documents, but specifically to content within a presentation (i.e. a document) that is activated by some interactive mechanism (often user-input events, but including local hyperlinking as well).

Background

SMIL Boston represents an evolution from earlier multimedia runtimes. These were typically either pure, static schedulers or pure event-based systems.  Scheduler models present a linear timeline that integrates both discrete and continuous media.   Scheduler models tend to be good for storytelling, but have limited support for user-interaction. Event-based systems, on the other hand, model multimedia as a graph of event bindings.  Event-based systems provide flexible support for user-interaction, but generally have poor scheduling facilities; they are best applied to highly interactive and experiential multimedia.

The SMIL 1.0 model is primarily a scheduling model, but with some flexibility to support continuous media with unknown duration. User interaction is supported in the form of timed hyperlinking semantics, but there was no support for activating individual elements via interaction.

Modeling interactive, event-based content in SMIL

To integrate interactive content into SMIL timing, the SMIL 1.0 scheduler model is extended to support several new concepts: indeterminate timing and event-activation.

With indeterminate timing, an element is described as a child of a time container, but with an undefined begin or end time.  The element still exists within the constraints of the time container, but the begin or end time is determined by some external activation (such as a user-input event).  From a scheduling perspective, the time can be thought of as unresolved.

The event-activation support provides a means of associating an event with the begin or end time for an element.  When the event is raised (e.g. when the user clicks on something), the associated time is resolved to a determinate time.  For event-based begin times, the element becomes active (begins to play) at the time that the event is raised. The element plays from the beginning of the media (subject to any explicit clipBegin).  For event-based end times, the element becomes inactive (stops playing) when the associated event is raised.

The constraints imposed on an element by its time container are an important aspect of the event-activation model. In particular, when a time container is itself inactive (e.g. before it begins or after it ends), no events are passed to the children. No event-activation takes place unless the time container of an element is active. For example:

<par begin="10s" end="15s">
   <audio src="song1.au" beginEvent="btn1.onClick" />
</par>

If the user clicks on the "btn1" element before 10 seconds, or after 15 seconds, the audio element will not play.  In addition, if the audio element begins but would extend beyond the specified end of the <par> container, it is effectively cut off by the end of the <par> container.  Finally, an endEvent cannot happen until an element has already begun (any specified endEvent is ignored before the element begins).

Related to event-activation is link-activation.  Hyperlinking has defined semantics in SMIL 1.0, and when combined with indeterminate timing, yields a variant on interactive content. In particular, hyperlinking is not constrained by the time container as event-activation is.  This is because a hyperlink will seek the document timeline as needed to ensure that the time container is active.

3.0 Language Definition

SMIL 1.0 defines the model for timing, including markup to define element timing, and elements to define parallel and sequence time containers.  This version introduces some syntax variations and additional functionality, including:

The complete syntax is described here, including syntax that is unchanged from SMIL 1.0.

3.1 Time Container

SMIL Boston specifies three time containers: <par>, <seq>, and <excl>.

3.1.1. par

<par>
A <par> container defines a simple parallel time grouping in which multiple elements can play back at the same timeThe timebase of the child elements of the <par> are the same as the timebase of the <par>. This is the same element introduced with SMIL 1.0.

The <par> element supports all element timing.

3.1.2 seq

<seq>
A <seq> container defines a sequence of elements in which elements play one after the other.  This is the same element introduced with SMIL 1.0, but the semantics (and allowed syntax) for child elements of a <seq> are clarified. The timebase of a child element is the end of the previous element. The timebase of the first child of the <seq> element is the timebase of the <seq> itself. Child elements may define an offset from the timebase, but may not define a different timebase (i.e. they may not define a begin time relative to another element, or to an event). Child elements may define a duration or end time, but may not define an end relative to another timebase (i.e. they may not end relative to any element).  Child elements may define an endEvent.

Reviewers - Note the proposed semantics above that constrain the child elements of a seq to not use timebase specification.  This simplifies the model for seq. This proposal is still under discussion.

The <seq> element supports all element timing.

3.1.3 excl

The new time container, <excl>, is defined here. A normative description is given first, followed by an informative discussion of the container's behavior.

<excl>
This defines a time container with semantics based upon par, but with the additional constraint that only one child element may play at a time. If any element begins playing while another is already playing, the element that was playing is stopped.
The default timebase of the child elements of the <excl> is indeterminate (i.e. equivalent to beginEvent="none").
A common request from SMIL 1.0 authors is the ability to play optional content without loading a new SMIL presentation. It should be possible to add or replace content without disrupting playback of the main presentation. The optional content should be allowed to be arbitrarily-complex, i.e., multiple, synchronized media elements. Examples of this are a hypermedia presentation with access to optional, more detailed information and a navigation bar or table of contents that provides for the selection of multiple content sections.

SMIL 1.0 does not allow optional content to be defined nor does it permit playback of a subsection of a presentation in isolation. All content defined in a SMIL 1.0 document is played back unless the current presentation is stopped or replaced with a different presentation. To simulate the above use cases, the author would have to hyperlink to another SMIL document containing the shared parts of the presentation plus the new content.

Interactive, event based timing allows authors to define optional content that can be shown in response to user (or other) events. The new <excl> time container makes use of this support, and supports the semantic that only one child is active at a given time. Thus, several playback options can be collected into an <excl>, instead of being placed in separate documents. In the following example, only the button graphics are shown initially. When the user clicks on one of the buttons, only then is the selection played. If the user selects "TopStory" then selects "Weather", "TopStory" is stopped and "Weather" is played. If nothing is selected, the optional content is never played.
<par>
    <excl>
        <par id="TopStory">
            <video src="video1.mpg" .../>
            <text src="captions.html" .../>
        </par>
        <par id="Weather">
            <img src="weather.jpg" .../>
            <audio src="weather_rpt.mp3" .../>
        </par>
    </excl>

    <a href="#TopStory">
        <img src="button1.jpg"  .../>
    </a>
    <a href="#Weather">
        <img src="button2.jpg"  .../>
    </a>
</par>

Because the optional content is played back within the context of the presentation, the same layout regions can be reused by the children of the <excl> container.

Children of the <excl> can be activated by hyperlinks or by events. When using events, the <excl> time container must be active for child elements of the <excl> to be activated by an event. When using hyperlinks, if the <excl> is not active, a seek will occur to the begin time of the <excl> and the children of the element will be activated. If the <excl> is currently active when the hyperlink is selected, a seek does not occur and playback of the child element begins. Playback of other active elements outside the scope of the <excl> is unaffected.

In the example above, an external link to the "Weather" child of the <excl> would activate the <par> containing the <excl>, as well as the target of the hyperlink. The two images would display, and weather.jpg and weather_rpt.mp3 would play back in parallel.

A pause functionality has been proposed as an option on exclusive time containers.  With this, instead of stopping an element when another begins, the first element would be paused while the new element plays. When the new element completes, the previously playing element would resume. Open issues are:

3.1.4 endSync

endSync

endSync is only valid for the <par> and <excl> time containers. It controls the end of these containers, as a function of the children. This is particularly useful with children that have "unknown" duration, e.g. an mpeg movie, that must be played through to determine the duration.
 
Legal values for the attribute are:
first
The <par> ends with the earliest end of all the child elements. This does not refer to the lexical first child, or to the first child to start, but rather refers to the first child to end. This value has no effect when used with <excl>.
last
The <par> ends with the last end of all the child elements. This does not refer to the lexical last child, or to the last child to start, but rather refers to the last child to end.


The end of the <excl> container occurs when no more child elements are playing. Used with the stacking pause, this means that at the point when no element is playing, the <excl> time container is defined to end.

id-ref
The <par> or <excl> ends with the specified child. The id must correspond to one of the immediate children of the <par> time container. Example: <par ... endSync="id(movie1)" ...>

3.1.5. Time Container duration

The implicit duration of a time container is defined in terms of the children of the container.  The children can be thought of as the "media" that is "played" by the time container element.  The semantics are specific to each of the defined time container variants.

Implicit duration of <par> containers

By default, a <par> will play until all the contained children have completed.  More formally, the implicit duration of a <par> element is defined by the maximum extent of the children.  The extent of an element is the sum of any begin offset and the active duration.  The begin offset is the computed offset relative to the begin of the <par> container (this may be different from the delay value, if the timebase for a child is not the par container).

If any child of a <par>is defined with an indefinite active duration (e.g. it repeats indefinitely), then the default duration of the <par> container is also indefinite.  If any child of a par container has an interactive (event-based) begin or end, the default duration of the <par> container is indefinite. Reviewers: This entire paragraph is under scrutiny for compliance with SMIL 1.0.

The duration of a <par> container can also be controlled using the endSync attribute.  The end of the <par> can be tied to the end of a particular child element, or to the end of the first child that finishes, or to the end of the last child to finish (which corresponds to the default behavior using maximum extent). Reviewers: What if the referenced child has an indefinite duration?

Implicit duration of <seq> containers

By default, a <seq> will play until the desired end of the last child of the <seq>.  If any child of a sequence has an indefinite desired end and the child refers to continuous media, the implicit end of the sequence is also indefinite.

Implicit duration of <excl> containers

The implicit duration of an <excl> container is defined much the same as for a <par> container.  Since the default timing for children is interactive, the typical case will define an indefinite implicit duration for <excl>. This is consistent with the common use-cases for interaction having open-ended durations.

3.1.6. Time Container constraints on child durations

A child element is subject to the overriding begin and end constraints of the parent time container.  No element can begin before the parent time container begins.  No element can remain active past the end of the simple duration of the parent time container (although it may be active in each repeat iteration of a parent time container).

3.1.7. Time Container constraints on sync-arcs and events

Probably need to define "active" and "inactive" more formally.
We need a few good examples to illustrate these concepts.

SMIL 1.0 defined constraints on sync-arc definition (e.g., begin="id(image1)(begin)"), allowing references only to qualified siblings. SMIL Boston explicitly removes this constraint. SMIL Boston also adds event-based timing.  Both sync-arcs and event-timing are constrained by the parent time container of the associated element as described above.

Specifics for sync-arcs

While a sync-arc is explicitly defined relative to a particular element, if this element is not a sibling element, then the sync is resolved as a sync-relationship to the parent. If the defined sync would place the element effective begin before the parent time container begin, part of the element will simply be cut off when it first plays.  This is not unlike the behavior obtained using clipBegin.  However unlike with clipBegin, if the sync-arc defined child element also has repeat specified, only the first iteration will be cut off, and subsequent repeat iterations will play normally.

Note that in particular, an element defined with a sync-arc begin will not automatically force the parent or any ancestor time container to begin.

For the case that an element with a sync-arc is in a parent (or ancestor) time container that repeats: for each iteration of the parent or ancestor, the element is played as though it were the first time the parent timeline was playing. This may require a reset of some sort in the implementation to ensure that the sync relationship to the parent time container is recalculated.

Specifics for event-based timing

The parent time container must be active for the child element to receive events.

3.1.8 Negative Begin delays

A negative begin (delay) value defines a clipBegin for the first -- and only the first -- iteration of a repeated element. Without specifying a repeat attribute, a negative begin value and clipBegin are synonymous.

3.2. Shared Timing support

This section defines the set of timing attributes that are common to all of the SMIL synchronization elements.

3.2.1. Basics - begin, end, dur

begin
Defines when the element should begin or become active. The begin time is relative to the timebase of the element.
The attribute value can be either of:
delay-value
Describes an offset from the timebase. The offset is measured in local time on the parent time container. Legal values are signed clock values.
timebase-value : "id(" id-ref ")"  (  "(begin)" | "(end)" )? ( signed-clock-val )?
Describes a timebase and an offset from the timebase. The offset is measured in local time on the timebase element.


The syntax follows the XPointer syntax, using "id()" to specify an element, followed by an optional time-point ("begin" or "end"), and/or an optional offset value.
If no time-point is specified, "begin" is assumed.
If no offset value is specified, "0" as assumed.
This is renamed from "event-value" to disambiguate from event timing which has a more general connotation.
Note that the syntax is extended to support offsets from either begin or end, and to support signed offsets.

wall-clock
The syntax for a wall-clock reference will likely follow ISO 8601, and might look like:  "wallClock(15:45)"


This defines the start time to be 3:45 in the afternoon, in local time.
 

logical-timebase-value:  "(prev)" (  "(begin)" | "(end)" ) ( signed-clock-val )?
Describes a logical timebase and an offset from the timebase. The offset is measured in local time on the timebase element.


The timebase is the lexical predecessor within the parent time container.  If there is no predecessor, the parent begin is used as a timebase.
If no offset value is specified, "0" as assumed.


end
This defines the end time for the simple duration of the element. The specified time is relative to the timebase of the element.
The attribute value can be either of:
delay-value
Describes an offset from the timebase. The offset is measured in local time on the parent time container. Legal values are signed clock valuesNote that this is signed, to end before another element ends. 
 
timebase-value : "id(" id-ref ")"  (  "(begin)" | "(end)" )? ( signed-clock-val )?
Describes a timebase and an offset from the timebase. The offset is measured in local time on the timebase element.


The syntax follows the XPointer syntax, using "id()" to specify an element, followed by an optional time-point ("begin" or "end"), and/or an optional offset value.
If no time-point is specified, "begin" is assumed.
If no offset value is specified, "0" as assumed.
This is renamed from "event-value" to disambiguate from event timing which has a more general connotation.
Note that the syntax is extended to support offsets from either begin or end, and to support signed offsets.

wall-clock
The syntax for a wall-clock reference will likely follow ISO 8601, and might look like:  "wallClock(15:45)"


This defines the start time to be 3:45 in the afternoon, in local time.

logical-timebase-value:  "(prev)" (  "(begin)" | "(end)" ) ( signed-clock-val )?
Describes a logical timebase and an offset from the timebase. The offset is measured in local time on the timebase element.


The timebase is the lexical predecessor within the parent time container.  If there is no predecessor, the parent begin is used as a timebase.
If no offset value is specified, "0" as assumed.

If the end specifies a time that is before the effective begin time of the element, the end specification is ignored.


Only one of dur and end should be specified.  If both are specified (and are otherwise legal), the simple duration is defined to be the minimum of the specified end, and the time computed from the desired begin and dur.

dur
This defines the simple duration for the element. It is an offset from the begin time of the element.


Only one of dur and end should be specified.  If both are specified (and are otherwise legal), the simple duration is defined to be the minimum of the specified end, and the time computed from the desired begin and dur.
Legal values are clock values greater than 0.

For continuous media, the implicit duration is typically a function of the media itself - e.g. video and audio files have a defined duration. For all discrete media, the implicit duration is defined to be 0. Note the related example.

If the author specifies an explicit duration (using either end or dur) that is longer than the intrinsic duration for a continuous media element, the ending state of the media (e.g. the last frame of video) will be shown for the remainder of the explicit duration. This only applies to visual media - aural media will simply stop playing. See also the discussion of the fill attribute, below.

3.2.1.1 Alternative begin/end syntax
An alternative syntax has been proposed for sync-arc specification.  This is under review. In particular, the names are under discussion, as some feel that they imply constraint-based behavior.
beginWith
Defines the timebase to be the start of the referenced element.  This supports relative timing between elements.  The current element will begin at the same time as the referenced element (plus or minus any offset value).


There are problems with separating the begin offset from the timebase. It is good because it more closely matches the SMIL-DOM that will be exposed, but it is bad because it requires two attributes.  We will also consider a syntax that specifies the offset after the ID:  beginWith="video1+5s".
Legal values are element IDs within the current document. We could also move this over to the XPointer syntax.

No more than one of beginWith, beginAfter or beginEvent should be specified.

 
beginAfter
Defines the timebase to be the end of the referenced element.  This supports relative sequential timing between elements.  The current element will begin when the referenced element ends (plus or minus any offset value). If the referenced timebase element does not have a specified determinate duration (e.g. is active indefinitely or has an event-timed end), the current element may never be active/displayed (although this is not an error).


Legal values are element IDs within the current document. We could also move this over to the XPointer syntax.
No more than one of beginWith, beginAfter or beginEvent should be specified.

 
endWith
Defines the end of the active duration to be the end of the referenced element.  This supports end-point sync between elements.  The current element active duration will end at the same time that the referenced element active duration ends (plus or minus any end offset).


Legal values are element IDs within the current document.  We could also move this over to the XPointer syntax. This should be defined to end the repeat, and not the simple duration (as for endEvent below). Ending with another element and then repeating seems less than useful. An open issue is whether the timebase is the simple end or the active end of the referenced element.

3.2.2. Interactive, Event-based Timing

Elements can be specified to begin and/or end in response to an event.  The event is specified with a new attribute, to clearly distinguish the form of timing being used. In this example, the audio element begins when the event is received (in this case, when the element "btn1" is clicked by the user):

<audio src="song1.au" beginEvent="btn1.onClick" />

It is possible to combine scheduled and interactive timing, e.g.:

<par dur="30s">
   <img id="mutebutton" src="mute.jpg"/>
   <text  src="description.html" />
   <audio src="audio.wav" endEvent="mutebutton.onClick"/>
</par>

The image and the text appear for the specified duration of the <par> (30 seconds). The audio will stop early if the image is clicked; otherwise it will play normally. Note that if the audio is stopped, the <par> still appears until the specified duration completes.

While an element can only have one defined begin (e.g. a defined time, or a beginEvent), it is possible to define both a determinate end or duration, as well as an endEvent.  This facilitates what are sometimes called "lazy interaction" use-cases, such as a slideshow that will advance on its own, or in response to user clicks:

<seq>
   <img src="slide1.jpg" dur="10s" endEvent="onClick" />
   <img src="slide2.jpg" dur="10s" endEvent="onClick" />
   <img src="slide3.jpg" dur="10s" endEvent="onClick" />
   <!-- etc., etc. -->
</seq>

In this case, the end of each element is defined to be the earlier of the specified duration, or a click on the element. This lets the viewer sit back and watch, or advance the slides at a faster pace.

If an event-timed element has begun and then receives a second begin event before it completes, it can either ignore the second event or restart, as though it had not received the first event. This behavior is controlled with the eventRestart attribute. Note that if an event timed element receives an event and then completes its duration, it can be restarted by another event (independent of eventRestart). This came up in many user scenarios where authors tied an element beginEvent to a button, but did not want to restart (e.g. music) on every click.  It is particularly useful when the beginEvent is mouseOver, which has a tendency to fire continuously...

The specific syntax used is still under review. Current proposals are:

The following attributes have been proposed to describe event-based (interactive) timing constructs.
beginEvent
Defines the timebase to be the referenced event. The current element will begin when the referenced event is raised (plus or minus any delay value). If the referenced event is never raised, the current element may never be activated.


More than one event may be specified. The element will begin when the first event among the set is raised.

If the argument value is "none", this element will simply wait to be turned on (e.g. by script or a local hyperlink).
Only one timebase may be specified for the begin of a given element (i.e. beginEvent cannot be combined with begin). If more than one timebase is specified, only the (lexical) first specification is used; all others are ignored.
Legal values are:

event-base-value-list : event-base-value (";" event-base-value-list )?
A semi-colon separated list of event base values.
event-base-value : ("id(" id-ref ")")? ( event-ref  ) ( signed-clock-val )?
Any defined event name, referenced by the associated element ID.


E.g. beginEvent="id(button1)(onclick)(3s)".
Note that if the id-ref is omitted, the current element is assumed.
The delay value may be omitted; it defaults to 0.
An alternative syntax would follow ECMAScript conventions:
[id-ref "." ]? event-ref [ "+"|"-" clock-val ]?
E.g. beginEvent="button1.onclick+3s" 

"none"
The string "none".
endEvent
Defines the end of the active duration to be relative to the referenced event. The active duration of the current element will end when the referenced event is raised (plus any delay value). If the referenced event is never raised, the current element may remain active indefinitely (subject to time container constraints).


Note that negative delay values may not be used.
More than one event may be specified. The element will end when the first event among the set is raised.
This attribute may be combined with a scheduled specification for the element end and/or a repeat specification.
If a finite active duration is defined for the element (with the end or dur attributes and/or repeatCount/repeatDur), the element will end at the earlier of the scheduled duration or the endEvent time.  This allows authors to specify an interactive end, but with a maximum active duration.
If the named event is "none", and no end time has been specified this element will simply wait to be turned off (e.g. by script or hyperlinking).
Legal values include:

event-base-value-list : event-base-value (";" event-base-value-list )?
A semi-colon separated list of event base values.
event-base-value : ("id(" id-ref ")")? ( event-ref  ) ( clock-val )?
Any defined event name, referenced by the associated element ID.


E.g.endEvent="id(button1)(onclick)(3s)".
Note that if the id-ref is omitted, the current element is assumed.
The delay value may be omitted; it defaults to 0.
endEvent uses the same arg-value name as beginEvent, but endEvent does not allow signed clock values.  We need to clean this up.
An alternative syntax would follow ECMAScript conventions:
[id-ref "." ]? event-ref [ "+"|"-" signed-clock-val ]? E.g. endEvent="button1.onclick+3s" 
 

"none"
The string "none".
eventRestart
Controls whether the element can be restarted while it is playing (i.e. during the active duration of the element).  If set to true, then a second beginEvent (or a method call via the SMIL-DOM interfaces) that occurs during the element's complete duration will restart the element.  If set to false, a second event will be ignored.


Argument values are Booleans.
The default value is true.

3.2.3. repeat, repeatCount and repeatDur

SMIL 1.0 introduced the repeat attribute, which is used to repeat a media element or an entire time container. SMIL Boston introduces two new controls for repeat functionality that supercede the SMIL 1.0 repeat attribute.  The new attributes, repeatCount and repeatDur, provide a semantic that more closely matches typical use-cases, and the new attributes provide more control over the duration of the repeating behavior.  The SMIL 1.0 repeat attribute is deprecated in SMIL Boston (it must be supported in SMIL document players for backwards compatibility).
The repeatCount and repeatDur attributes are used to cause an element to copy or loop the contents of the element media (or an entire timeline in the case of a time container).  Using repeatCount causes the element to repeat the simple duration of the element, effectively multiplying the simple duration by the value of the repeatCount attribute.  In this example, the first 3 seconds of the "snd1" audio will play three times in succession for a total of 9 seconds:
<audio src="snd1.au" dur="3s" repeatCount="3" />

The repeatDur attribute is used to specify that an element should repeat just as with repeatCount, but for a specified total duration. This can be useful when matching a repeat duration the duration of other elements. In this simple example, the "snd2" audio will repeat for a total of 10 seconds:

<audio src="snd2.au" dur="3.456s" repeatDur="10s" />

The repeatCount and repeatDur attributes can also be used to repeat an entire timeline, e.g.:

<seq begin="5s" repeat="indefinite" >
   <img src="img.jpg" end="5s" />
   <img src="img2.jpg" dur="4s" />
   <img src="img3.jpg" dur="4s" />
</seq>

The sequence has an implicit duration of 13 seconds.  It will begin to play after 5 seconds, and then will repeat indefinitely (i.e. subject to the constraints of the parent time container of the <seq>).

The repeatCount and repeatDur attributes modify the active duration of an element. If repeatCount is specified, the active duration is the simple duration multiplied by the repetition count. If a repeatDur is specified, the active duration is equal to the specified repeat duration.

Need to create normative examples that demonstrate new the controls, and the interaction with implicit and explicit simple durations. Examples must also demonstrate the interaction of repeating behavior and time container constraints. 

repeatCount
This causes the element to play repeatedly (loop) for the specified number of times.


A repeat count of less than 1 will cut short the specified simple duration.
A repeat count of 1 is a no-op.
Legal values are integer or fractional iterations, greater than 0.

repeatDur
This causes the element to play repeatedly (loop) for the specified duration.


A repeat duration less than the simple duration will cut short the duration.
A repeat duration equal to the simple duration is a no-op.
Legal values are clock values greater than 0.

For both repeatCount and repeatDur, the attributes have no affect if the element's simple duration is 0 or indefinite.


For both attributes, "indefinite" may be specified to indicate that the element should repeat indefinitely (subject to the time container semantics).
At most one of repeatCount or repeatDur should be specified (if both are specified, the repeat duration is defined as the minimum of the specified repeatDur, and the simple duration multiplied by repeatCount.

3.2.3.1. SMIL 1.0 repeat attribute (deprecated)
The SMIL 1.0 repeat attribute defined repeat in manner similar to repeatCount, but it specified that the semantic was equivalent to a sequence that contained the specified number of copies of the element without the repeat attribute.  Thus, if the element has a begin delay, the SMIL 1.0 repeat included the delay with each iteration. See also the Issues item Semantics of SMIL 1.0 repeat and sync arcs.
repeat
This attribute has been deprecated in SMIL Boston in favor of the new repeatCount and repeatDur attributes.


This causes the element (including any begin delay) to play repeatedly for the specified number of times.  It is equivalent to a seq element with the stated number of copies of the element without "repeat" attribute as children.
Legal values are integer iterations, greater than 0, and "indefinite".
See also the SMIL 1.0 specification.

3.2.4. fill

The fill attribute allows an author to specify whether the complete duration of an element should be extended beyond the active duration, especially to fill gaps in the parent time container. The semantics of the fill attribute are the same as in SMIL 1.0.
The fill attribute allows an author to specify that the effective duration of an element should be extended beyond the active duration, by "freezing" the ending state of the element and continuing to display the frozen state for the duration of the parent container. The semantics of the fill attribute are the same as in SMIL 1.0.
fill
Legal values are "remove" and "freeze".


The default value is generally "remove", but is dependent upon the parent time container and the use of indeterminate durations.
Setting this attribute to "remove" ensures that the complete duration will not exceed the active duration. Setting this to "freeze" will extend the complete duration by "freezing" the element state at the active end, and showing this for the remainder of the complete duration. The actual fill duration is subject to the constraints of the parent time container.
This attribute only has effect on visual media elements. Non-visual media elements (audio) should ignore this.
Links are still active during fill="freeze". See also the SMIL 1.0 specification.

3.3. Time Manipulations

New element controls for element time behavior are under discussion. Note that an Accessibility requirement for control of the playback speed is related to (but may end up with different syntax different from) the speed control.  In general, these time manipulations are suited to animation and non-linear or discrete media, rather that linear continuous media.

In the model for time manipulations, the element's local time can be filtered or modified.  The filtered time affects all descendents. Any filter that changes the effective play speed of element time may conflict with the basic capabilities of some media players.  The use of these filters is not recommended with linear media players, or with time containers that contain linear media elements.

The proposed extensions support use-cases commonly associated with graphic animation.

There are a number of unresolved issues with this kind of time manipulation, including issues related to event-based timing and negative play speeds, as well as many media-related issues.

3.3.1. speed

The speed attribute controls the local playspeed of an element, to speed up or slow down the effective rate of play. Note that the speed does not specify an absolute play speed, but rather modifies the playspeed of the parent time container.  Thus if a <par> and one of its children both specify a speed of 50%, the child will play at 25% of normal playspeed. Also note that a speed of -100% is equivalent to playing the media backwards.
Proposed syntax:
speed
Defines the playback speed of element time. The value is specified as a percent of normal (parent time container) play speed.


Legal values are signed percentage values.  The default is +100%.

3.3.2. accelerate and decelerate

These attributes define a simple acceleration and deceleration of element time. This is useful for animation, motion paths, etc. The values are expressed as a percentage of the simple durations, and are defined such that the simple duration is not affected (the normal playspeed is increased to compensate). If these are combined with repeating behavior, the acceleration and/or deceleration occurs within each repeat interation.
The sum of acceleration and deceleration must not exceed 100. If it does, the deceleration value will be reduced to make the sum legal.
Proposed syntax:
accelerate
Defines a simple acceleration of time for the element. Element time will accelerate from a rate of 0 at the beginning up to a run rate, over the course of the specified percentage of the simple duration. The default value is 0% (no acceleration).


Legal values are percentage values between 0 and 100.

decelerate
Defines a simple deceleration of time for the element. Element time will decelerate from a run rate down to 0 at the end of the simple duration, over the course of the specified percentage of the simple duration.


The default value is 0% (no deceleration).
Legal values are percentage values between 0 and 100.

3.3.3. autoReverse

This defines "play forwards then backwards" functionality. The use of autoReverse effectively doubles the simple duration. When combined with repeating behavior, each repeat iteration will play once forwards, and once backwards. This is useful for animation, especially for mechanical and pendulum motion.
Proposed syntax:
autoReverse
Controls autoReverse playback mode.


Argument values are Booleans.
The default value is false (i.e. play normally).

3.4. Controlling Runtime Synchronization Behavior

Proposed new support in SMIL Boston introduces finer grained control over the runtime synchronization behavior of a document.  The syncBehavior attribute allows an author to describe for each element whether it must remain in a hard sync relationship to the parent time container, or whether it can be allowed slip with respect to the time container. Thus, if network congestion delays or interrupts the delivery of media for an element, the syncBehavior attribute controls whether the media element can slip while the rest of the document continues to play, or whether the time container must also wait until the media delivery catches up.

The syncBehavior attribute can also be applied to time containers. This controls the sync relationship of the entire timeline defined by the time container.  In this example, the audio and video elements are defined with hard or "locked" sync to maintain lip sync, but the "speech" <par> time container is allowed to slip:

<par>
   <animation src="..." />
   ...
   <par id="speech" syncBehavior="canSlip" >
      <video src="speech.mpg" syncBehavior="locked" />
      <audio src="speech.au"  syncBehavior="locked" />
   </par>
   ...
</par>

If either the video or audio must pause due to delivery problems, the entire "speech" par will pause, to keep the entire timeline in sync. However, the rest of the document, including the animation element will continue to play normally. Using the syncBehavior attribute on elements and time containers, the author can effectively describe the "scope" of runtime sync behavior, defining some portions of the document to play in hard sync without requiring that the entire document use hard synchronization.

This functionality also applies when an element first begins, and the media must begin to play. If the media is not yet ready (e.g. if an image file has not yet downloaded), the syncBehavior attribute controls whether the time container must wait until the element media is ready, or whether the element begin can slip until the media is downloaded.

The syncBehavior can affect the effective begin and effective end of an element, but the use of the syncBehavior attribute does not introduce any other semantics with respect to duration.

When the syncBehavior attribute is combined with interactive begin timing for an element, the syncBehavior only applies once the sync relationship of the element is resolved (e.g. when the specified event is raised). If at that point the media is not ready and syncBehavior is specified as "locked", then the parent time container must wait until the media is ready. Once an element with an interactive begin time has begun playing, the syncBehavior semantics described above apply as thought the element were defined with scheduled timing.
Note that the semantics of syncBehavior do not describe or require a particular approach to maintaining sync; the approach will be implementation dependent. Possible means of resolving a sync conflict may include:

Additional control is provided over the hard sync model using the syncTolerance attribute. This specifes the amount of slip that can be ignored for an element.  Small variance in media playback (e.g. due to hardware innaccuracies) can often be ignored, and allow the overall performance to appear smoother.

3.4.1. Sync Behavior Attributes

syncBehavior
Defines the runtime synchronization behavior for an element.


Legal values are:

canSlip
Allows the associated node to slip with respect to the parent time container.


When this value is used, any syncTolerance attribute is ignored.
locked

Forces the associated node to maintain sync with respect to the parent time container. This can be eased with the use of the syncTolerance attribute.

syncTolerance

This attribute on timed elements and time containers defines the sync tolerance for the associated element. It has an effect only if the element has syncBehavior="locked". This allows a locked sync relationship to ignore a given amount of slew without forcing resynchronization. resolution.
 

Legal values are clock values.
There may be a non-zero default value for this attribute, to be determined through testing (a proposed guess is something between 0.1 seconds and 0.5 seconds). 

3.4.2. Sync Master Support

An additional proposed extension allows the author to specify that a particular element should define or control the synchronization for a time container. This is similar to the default behavior of many players that "slave" video and other elements to audio, to accommodate the audio hardware inaccuracies and the sensitivity of listeners to interruptions in the audio playback. The syncMaster attribute allows an author to explicitly define that an element defines the playback "clock" for the time container, and all other elements should be held in sync relative to the syncMaster element.
In practice, linear media often need to be the syncMaster, where non-linear media can more easily be adjusted to maintain hard sync.  However, a player cannot always determine which media behaves in a linear fashion and which media behaves in a non-linear fashion. In addition, when there are multiple linear elements active at a given point in time, the player cannot always make the "right" decision to resolve sync conflicts. The syncMaster attribute allows the author to specify the element that has linear media, or that is "most important" and should not be compromised by the syncBehavior of other elements.
The syncMaster attribute interacts with the syncBehavior attribute. An element with syncMaster set to true will define sync for the "scope" of the time container's synchronization behavior. That is, if the syncMaster element's parent time container has syncBehavior="locked", the syncMaster will also define sync for the ancestor time container. The syncMaster will define sync for everything within the closest ancestor time container that is defined with syncBehavior="canSlip".

The syncMaster attribute only applies when an element is active. If more than one element within the syncBehavior scope has the syncMaster attribute set to true, and the elements are both active at any moment in time, the behavior will be implementation dependent.

syncMaster
Boolean attribute on media elements and time containers that forces the time container playback to sync to this element.


The default value is false.
The associated property is read-only, and cannot be set by script.

3.5 Syntax Production Rules

clock value
Clock values have the following syntax. This is the same as in SMIL 1.0 except that the hours subtoken is not limited to 2 digits:
Clock-val             ::= Full-clock-val | Partial-clock-val | Timecount-val
Full-clock-val        ::= Hours ":" Minutes ":" Seconds ("." Fraction)?
Partial-clock-val     ::= Minutes ":" Seconds ("." Fraction)?
Timecount-val         ::= Timecount ("." Fraction)?
                          ("h" | "min" | "s" | "ms")? ; default is "s"
Hours                 ::= DIGIT+; any positive number
Minutes               ::= 2DIGIT; range from 00 to 59
Seconds               ::= 2DIGIT; range from 00 to 59
Fraction              ::= DIGIT+
Timecount             ::= DIGIT+
2DIGIT                ::= DIGIT DIGIT
DIGIT                 ::= [0-9]

The following are examples of legal clock values:


A fraction x with n digits represents the following value:

x * 1/10**n

Examples:

00.5s = 5 * 1/10 seconds = 500 milliseconds
00:00.005 = 5 * 1/1000 seconds = 5 milliseconds

4. Constructing the Time Graph

This is largely a placeholder for content that must be refined in later drafts.
Need to review all the stuff in SMIL that was not clearly enough described.  Need to nail down issues on cyclic graphs and other illegal constructs.
Graph structure, rules, implications (acyclic time graphs, etc.).  Need to describe what happens when have conflicting syntax (e.g. dur and end, repeat and repeatDur).  Need to describe what happens when have invalid specification (illegal argument values, etc.).

We may want to add an additional informative document that is more of an authoring guide.

Consider inclusion of algorithm to support normative description.

4.1 Graph Structure

A document with SMIL timing is parsed incrementally and during that process a graph structure is built.

More to come here, explaining precisely how syntax is translated to formal semantics and graph structure.

4.2 Conflicts and Conflict Resolution

When an element is parsed, it is introduced into the graph according to the temporal relation where it appears. In order to clarify the process of graph creation and graph consistency, we must consider the way the graph is built. When an element is parsed, the insertion into the graph follows these operations:

  1. Node creation. Create a node with begin and end references.
  2. Arc creation. Build the sync-arc and event-arc relationships for the node.

Arc creation is a source of potential inconsistencies. These must be detected and resolved according to a set of fixed rules, to ensure consistent performance across implementations.

More to come here.

5.0 Document Object Model Support

Much of the related SMIL-DOM functionality is proposed in the [SMIL-DOM] section.  We may need to go into further detail on the specific semantics of the interfaces - the sections below are placeholders.

5.1 Element and Attribute manipulation, mutation and constraints

Define rules on element and attribute access (inherit from and point to Core DOM docs for this). Define mutation constraints. This is currently covered in the [SMIL-DOM] section.

5.2 Supported Methods

Describe the methods and their semantics.  Interfaces are currently defined in the [SMIL-DOM] section.

5.3 Event Model

We need to describe how the Core DOM Level 2 event model, affects the timing model.  The [SMIL-DOM] section defines the initial set of time-related events that have been proposed.

6.0 References

[DOMReqts]
"Document Object Model Requirements for Synchronized Multimedia", P. Schmitz.
Available at http://www.w3.org/AudioVideo/Group/DOM/DOM_reqts. (W3C members only)
[HTML]
"HTML 4.0 Specification", D. Raggett, A. Le Hors, I. Jacobs, 24 April 1998.
Available at http://www.w3.org/TR/REC-html40.
[ISO-8601]
"Data elements and interchange formats - Information interchange - Representation of dates and times", International Organization for Standardization, 1998.
[SMIL1.0]
"Synchronized Multimedia Integration Language (SMIL) 1.0 Specification W3C Recommendation 15-June-1998 ".
Available at: http://www.w3.org/TR/REC-smil.
[SMIL-CSS2]
"Displaying SMIL Basic Layout with a CSS2 Rendering Engine".
Available at: http://www.w3.org/TR/NOTE-CSS-smil.html.
[SMIL-DOM]
"SMIL Document Object Model", Nabil Layaïda, Patrick Schmitz, Jin Yu.
Available at http://www.w3.org/1999/08/WD-smil-boston-19990803/DOM/smil-dom
[SMIL-MEDIA]
"The SMIL Media Object Module", Philipp Hoschka, Rob Lanphier.
Available at http://www.w3.org/1999/08/WD-smil-boston-19990803/Media/extended-media-object.
[SMIL-MOD]
"Synchronized Multimedia Modules based upon SMIL 1.0", Patrick Schmitz, Ted Wugofski, Warner ten Kate.
Available at http://www.w3.org/TR/NOTE-SYMM-modules.
[XML]
"Extensible Markup Language (XML) 1.0", T. Bray, J. Paoli, C.M. Sperberg-McQueen, editors, 10 February 1998.
Available at http://www.w3.org/TR/REC-xml

Appendix A: Annotated Examples

Example 1: Simple timing within a Parallel time container

This section includes a set of examples that illustrate both the usage of the SMIL syntax, as well as the semantics of specific constructs.  This section is informative.

Note: In the examples below, the additional syntax related to layout and other issues specific to individual document types is omitted for simplicity.

All the children of a <par>  begin by default when the <par> begins.  For example:

<par>
   <img id="i1" end="5s"  src="img.jpg" />
   <img id="i2" end="10s" src="img2.jpg" />
   <img id="i3" begin="2s" dur="5s" src="img3.jpg" />
</par>

Elements "i1" and "i2" both begin immediately when the par begins, which is the default begin time. "i1" ends at 5 seconds into the <par>.  "i2" ends at 10 seconds into the <par>.  The last element "i3" begins at 2 seconds since it has an explicit begin offset, and has a duration of 5 seconds which means it ends 7 seconds after the <par> begins.

An image that illustrated the timeline might be useful here.

Example 2: Simple timing within a Sequence time container

Each child of a <seq>  begins by default when the previous element ends.  For example:

<seq>
   <img id="i1" begin="0s" dur="5s" src="img1.jpg" />
   <img id="i2" end="10s" src="img2.jpg" />
   <img id="i3" begin="1s" dur="5s" src="img3.jpg" />
</seq>

The element "i1" begins immediately, with the start of the <seq>, and ends 5 seconds later.  Note: specifying a begin time of 0 seconds is optional since the default begin offset is always 0 seconds.  The second element "i2" begins, by default, 0 seconds after the previous element "i1" ends, which is 5 seconds into the <seq>. Element "i2" ends 10 seconds later, at 15 seconds into the <seq>.  The last element, "i3", has a begin offset of 1 second specified, so it begins 1 second after the previous element "i2" ends, and has a duration of 5 seconds, so it ends at 21 seconds into the <seq>.

Insert illustration.

Example 3: excl time container with child timing variants

  1. Exclusive element, children activated via link-based activation:
      <par>
        <excl>
          <par id="p1"> 
             ...  
          </par>
          <par id="p2">
             ...  
          </par>
        </excl>
        <a href="p1"><img src="Button1.jpg"/></a>
        <a href="p2"><img src="Button2.jpg"/></a>
      </par>
    

    This example models jukebox-like behavior. Clicking on the first image activates the media items of parallel container "p1". If the link on the second image is traversed, "p2" is started (thereby deactivating "p1" if it would still be active).

    Shouldn't we say, here, exactly where the elements of the selected par in the excl should begin when a click happens, e.g., if we are 10 seconds into the outer par and we click on button 2, does the MPG video in p2 start 10 seconds into its stream (in-sync), or does it start at its time 0?

  2. Exclusive element combined with event-based activation:
    Note that the specific syntax for beginEvent argument values is still under discussion.
      <par>
        <excl>
          <par beginEvent="btn1.onclick"> 
             ...  
          </par>
          <par beginEvent="btn2.onclick">
             ...  
          </par>
        </excl>
        <img id="btn1" src=... />
        <img id="btn2" src=... />
      </par>
    

    The same jukebox example, using event-based activation.

    In these two examples event-based and anchor-based activation look almost identical, maybe we should come up with examples showing the difference and the relative power of each.

  3. Exclusive element using determinate declarative timing:
      <excl>
        <ref id="a" begin="0s" ... />
        <ref id="b" begin="5s" ... />
      </excl>
    

    In the example above, the beginning of "b" deactivates "a" (assuming that a is still active after 5 seconds).  Note that this could also be modeled using a sequence with an explicit duration on the children.  While the determinate syntax is allowed, this is not expected to be a common use-case scenario.

    Issue - should we preclude the use of determinate timing on children of excl?  Other proposals would declare one child (possibly the first) to begin playing by default. Proposals include an attribute on the <excl> container that indicate one child to begin playing by default.

Example 4:  default duration of discrete media

For all discrete media, the implicit duration is defined to be 0. This can lead to surprising results, as in this example:

<seq>
   <img src="img1.jpg" />
   <video src="vid2.mpg" />
   <video src="vid3.mpg" />
</seq>

This will not show the image at all, as it defaults to a duration of 0, and so the second element will begin immediately.  Authors will generally specify an explicit duration for any discrete media elements.

Example 5:  endEvent specifies end of active dur, not end of simple dur

There is an important difference between the semantics of endEvent and end/dur.
The end and dur attributes, in conjunction with the begin time, specify the simple duration for an element.

This is the duration that is repeated when the element also has a repeat specified.  The attribute endEvent on the other hand overrides the active duration of the element.  If the element does not have repeat specified, the active duration is the same as the simple duration.  However, if the element has repeat specified, then the endEvent will override the repeat, but will not affect the simple duration.  For example:

<seq repeat="10" endEvent="stopBtn.onClick">
   <img src="img1.jpg" dur="2s" />
   <img src="img2.jpg" dur="2s" />
   <img src="img3.jpg" dur="2s" />
</seq>

The sequence will play for 6 seconds on each repeat iteration.  It will play through 10 times, unless the user clicks on a "stopBtn" element before 60 seconds have elapsed.

Example 6: SMIL-DOM-initiated timing

When an implementation supports the SMIL-DOM, it will be possible to make an element begin or end using script or some other browser extension. When an author wishes to describe an element as interactive in this manner, the following syntax can be used:

<audio src="song1.au" beginEvent="none" />

The element will not begin until the SMIL-DOM beginElement() method is called.

Appendix B: Open Issues

B.1. (rules for backwards compatibility - resolved).
B.2. Need to document the formal requirements.
B.3.  SMIL 1.0 "event" syntax

SMIL 1.0 supported a means of defining one element to begin relative to the begin or end of another element.  This was referred to as "event" timing in SMIL 1.0, however the syntax described a determinate synchronization relationship, and did not require an implementation to use events.  To reduce confusion with more traditional events such as user-input events, the SMIL 1.0 description and syntax for this has been deprecated (although conforming SMIL Boston players will support the SMIL 1.0 syntax).

We have not yet formally agreed upon this, but there seemed to be some growing consensus on the mailing list for the need to disambiguate event-based timing in the sense of interactive content, and SMIL 1.0 "event" based timing (which is really just a terminology for describing sync-arcs). 

B.4 - preface to examples - resolved

B.5 - click- on, click-off semantics
This is a proposal that needs discussion.

When the same event is specified for both the beginEvent and the endEvent, the semantics of the "active" state define the behavior. If the element is not active, then the beginEvent is sensitive, and responds to the event, making the element active.  Since the endEvent is not sensitive when the element is not active, it does not respond to this initial event. When the element is active, the endEvent is sensitive and responds to a second event, making the element inactive.  This supports the common use-case of clicking an element to turn it on, and clicking it again to turn it off.

B.7 - Semantics of <excl> with combined scheduled and interactive timing
Concerns have been expressed about the linking aspects and the combination with <excl>. It is not a problem if all the children have indeterminate begin times, as you could happily play one child instead of any other. If we allow scheduled timing with children of <excl>, you may be stopping a child that was temporally bound to its parent and in turn to other synchronized elements. So you could be stopping something that was part of a prescheduled presentation and continuing the others parts, while still assuming that the determined synchronization is taking place.


Linking to children of an <excl> actually violates the notion that the children of the <excl> can have any determined relationship with the parent.
What happens when child #2 has a determined start time of 15s (w.r.t. the parent) and then the user follows a link at time 14s (w.r.t.the parent) to child #3. Does child #3 get replaced by child #2 when 15s is reached? Highly annoying if you've just followed a link to it. Does child #3 continue playing, thus violating the determined start time of child #2.
We may require some extra information in the link specification (e.g. multiple source link ends) to allow the author to specify the wished-for behavior. It may only require that authors are aware that they may sometimes have to introduce an extra layer of structure to obtain the behavior they want.

Again, there is the choice of whether we implement pausing, and how we model it, that will influence this.

B.8 - Precise definition of timing relative to another node Resolved.

B.9 - Alternative syntax for begin and end - Described in section on Alternative begin/end syntax.

B.10 - Various issues with logical Timebases.
B.11 - Definition of syncBehavior and scope
Should the syncBehavior scope be defined with respect to the lexical parent (simple) or to the timebase?  HTML+TIME defined it using the simpler parent model.  As we experiment with it, and especially when using long sync-arcs, it may make more sense to enforce the sync relationship to the timebase rather than to the parent.
A separate but closely related comment:
Issue: need to resolve whether the sync relationship is resolved to be relative to the time container, or to the specified time-base.  In a hard sync world, this is moot.  However, if we ever add controls on the runtime sync management behavior, we have to decide if the element must stay in sync with the parent and siblings, or with the time-base.  The former is cleaner and may be simpler to support, but the latter may more accurately reflect the author's intent. This is tied to the section on "Controlling Runtime Synchronization Behavior", and so will get more attention in later drafts.
B.12 - Open issues on excl semantics
Open issues for excl time container:

The proposal partially described above is that we support the same notion that the linking introduces, but for children of the excl.  This is, if an element is playing when another begins (or is begun), that the playing element can optionally be made to stop and then resume when the new element completes.  This is useful for situations where you define a primary program and some potential interruptions (like commercials).  You want to pause the primary timeline and then resume it when the insertion completes.  This is useful for the same reasons that the linking behavior is useful, but supports the control over a specific timeline segment rather than on an entire document.  The requirement came from someone who wants to use the time syntax for another XML application, rather than in a SMIL-like scenario.  We are still discussing where this behavior should be described (on the element that is interrupted, or on the inserted element).  There are arguments and use-cases for both.

B.13 - Semantics of negative begin times
Issue: negative offsets for begin times, etc. Note that more generally it defines sync.  Given constraints of parent begin/end, begin times fall out. See related notes on sync-arcs and time container constraints.
B.14 - Semantics of SMIL 1.0 repeat and sync arcs.
An additional issue with the SMIL 1.0 repeat semantics: Given that the semantic describes repeat as a sequence of copies of the element (without the repeat attribute), and given the constraints placed upon sync-arc specifications for begin and end, it would seem that repeat cannot (in SMIL 1.0) be combined with sync-arc specifications (any sync-arc would necessarily reference an element that was out of scope in the sequence defined by the repeat). The SMIL Boston semantics for repeat would preclude this conflict (although the sync-arc restrictions are also removed in SMIL Boston).