This segment of the working draft specifies an architecture for applying timing information to XML documents. It specifies the syntax and semantics of the constructs that provide timing information. This approach builds on SMIL by preserving SMIL's timing model and maintaining the semantics of SMIL constructs.
This part of the working draft does not attempt to describe the exact syntax required to apply timing to XML documents as multiple options are still under consideration by the W3C SYMM Working Group. There are examples containing several possible syntaxes throughout this segment of the working draft, but these are for illustration purposes only and are likely to change.
Currently there exists no standardized method for adding timing to elements in any arbitrary XML document. This segment of the working draft defines the mechanisms for doing so.
Prior to SMIL 1.0 becoming a W3C recommendation, a significant number of W3C members expressed interest in integrating SMIL timing functionality with XHTML and other XML-based languages.
SMIL 1.0 describes timing relationships between objects, including complete XML documents. SMIL 1.0 can not control the timing of individual elements contained within these documents, e.g., the display of a single XHTML heading before the bulk body text appears, or the sequential display of the items in a list. When using SMIL 1.0 for this, a content author is forced to contain each temporal element set in a separate document, leading to very small documents in some cases. For example, consider the split up of text that must occur when creating closed captioning from a subtitle track using SMIL 1.0.
The SMIL 1.0 architecture assumes that SMIL documents will be played by a SMIL-based presentation environment. It does not treat the case where timing is an auxiliary component, and the presentation environment is defined by another language, like XHTML, a vector-graphics language, or any user-defined XML-based language and stylesheet.
This segment of the working draft specifies how SMIL timing can be used in other XML languages, providing a solution to the above cases. The work is driven by the following goals:
The following cases require the application of timing. These use cases are not listed in any particular order:
[*] Note: This assumes that the HTML document is a valid XML document.
<H1> element of an HTML document to schedule the
display of that header's text. <P>...</P> container. Such a document could be turned
into a textual performance of the play by adding the timing necessary to
sequentially present each of the child <P> elements of
the <BODY> of the document.
<SPAN> elements containing unique IDs. An external timing
document could then be used to apply unique timing to each of these
<SPAN> elements.
This section outlines the conceptual approach to adding timing to XML applications. The Specification section specifies the constructs used. There are three proposed methods of adding this timing:
How to ensure that in-line timing cooperates uniformly with CSS Timing or Timesheets is still under consideration.
In cases where SMIL timing is placed within an XML document, a hybrid DTD may be needed containing the DTD for the SMIL Timing and Synchronization module as well as the DTD for the XML language in which the original content document was written.
Reminder: the various syntaxes specified in this segment of the working draft are likely to change prior to the finalization of the working draft.
In some cases in-line timing will make authoring easier, especially in cases where the author wants the timing to flow with the structure of the content. In other cases, CSS Timing or Timesheets may be needed.
The semantics of in-line timing are the same as that of SMIL 1.0 timing, but the syntax is different. SYMM is currently considering two ways to add in-line timing to XML content.
<par>...</par> and
<seq>...</seq> elements to create time blocks that
apply timing to all child elements. For instance, an author could place a
seq element as a parent of a list of items and consequently make those list
items display one after the other.
par or
seq (or other time container types under consideration, e.g.,
<excl>), along with optional SMIL timing attributes like
duration, begin time (relative to that of any parent element), and end time,
to name a few. In order to declare that an element
should act as a time container, a new attribute is needed, possibly named
"timeLine" or "timeContainer". This attribute is only legal within grouping
elements in XML documents, and specifically cannot be applied to any of the
time container elements including par, seq and
excl. timeLine="t", or
timeContainer="t", where "t" is par,
seq, or some other time container under consideration. par
seq
excl
none
For example, to add timing to an XHTML <DIV> element so
that it acts as a "par" SMIL time container and has a duration of display
of 10 seconds, the syntax might be:
<DIV timeLine="par" dur="10s">.
Another attribute under consideration is the action
associated with the timing. This attribute would allow the author to specify
how the element's timing should be applied, e.g., to the display of its content
or to style attributes like the color of its content. One syntax currently
under consideration is "timeAction". In SMIL 1.0, the begin, end, duration,
and other times specified in elements are always used to place the element
on its parent element's timeline. A new attribute must be created to allow
alternate application of the specified time values, e.g., the begin time
could be applied to a style like the color of an element without affecting
the true begin time of the element.
The syntax for making the contents of a paragraph be red starting at 5 seconds
relative to the parent's timeline might look like the following. Note that
the begin time of the element is not specified and is assumed to be the begin
time of its parent (or the end time of its sibling if its parent is a
<seq> time container):
<P begin="5s" style="color: red" timeAction="style">
Open issue: using in-line timing, how do you set, say, the begin time of an element as well as the begin time of its style?
Here is an example of in-line timing being used to schedule the application of color style attributes as specified in the document's style sheet: Consider the playback of a music album where the audio track plays in concert with a list of the songs. Timing is added to the list so that the song that is currently playing is colored differently from the others. "timeAction" in this example means that the style of the class "playing" should be applied (only) to the text during the duration specified. Note that, in this example, "song 1", "song 2", and "song 3" all appear throughout the entire presentation; it is only their color that has been modified over time using (in-line) timing:
<head>
<style>
body { color: black; }
.playing { color: red; }
</style>
</head>
<body>
<audio ...>
<p dur="227s" timeAction="class:playing"> song 1 </p>
<p begin="228s" dur="210s" timeAction="class:playing"> song 2 </p>
<p begin="439s" dur="317s" timeAction="class:playing"> song 3 </p>
</body>
Reminder: the various syntaxes specified in this segment of the working draft are likely to change prior to the finalization of the working draft.
CSS Timing is the use of SMIL timing within a style sheet, where timing is a style attribute, just like, for example, color and font-weight in CSS, that is applied to elements in the content document. The resultant timing structure is based on and depends on the structure of the content document. In some cases, in-line timing may be inefficient, difficult, or impossible to add particular timing. In these cases, either CSS Timing or Timesheets may be needed. Some possible cases where CSS Timing will provide a better solution than in-line timing are:
The same two attributes mentioned in the In-Line Timing Framework section, above, will be needed. The first (possibly "timeContainer" or "timeline") is needed to be able to declare that an element should act as a time container. The second (possibly "timeAction") is needed to be able to specify how the timing should be applied, e.g., to the visibility of the object(s) or alternatively to a style applied to the object(s).
How to ensure that CSS timing and in-line timing cooperate uniformly is still under consideration.
Here is a simple example containing one possible syntax for integrating timing using CSS. In this example, the list will play in sequence as dictated by the style sheet in the HEAD section of the document. Note: the style sheet, like any CSS, could alternatively exist as a separate document.
</HEAD>
<STYLE>
UL { timeLine: seq; }
LI { font-weight: bold; duration: 5s; }
</STYLE>
</HEAD>
<BODY>
<UL>
<LI>This list item will appear at 0 seconds
and last until 5 seconds.
</LI>
<LI>This list item will appear after the prior
one ends and last until 10 seconds.
</LI>
<UL>
</BODY>
Timesheets refer to both the conceptual model along which timing, including the structure of the timing, is integrated into an XML document, as well as one possible syntax implementation. This approach provides a solution where time can be brought to any XML document regardless of its syntax and semantics.
A Timesheet uses SMIL timing within a separate document or separate section of the content document and imposes that timing onto elements within the content document. The resultant timing structure is not necessarily related to the structure of the content document. Some possible cases where a Timesheet will provide a better solution than in-line timing are a superset of such CSS Timing cases (which are included in the list below):
Timesheets assume an XML document conceptually composed of three presentation related sections:
The first section, content, relates to the particular XML document. It conforms to a DTD written for an XML language. The content part describes the media and its structure.
The second section, formatting, provides control of the properties of the elements in the content section. It conforms to a style language, which, for the purpose of this discussion, we assume to be CSS. The style section describes the style and (spatial) layout of presenting the content. "Formatting" might include matters like routing of audio signals to loudspeakers.
The third section, timing, provides control of the temporal relations between the elements in the content section. It conforms to SMIL's timing model. The time section describes the time at which content is presented as well as the time at which style is applied. The time section contains the information to prepare a presentation schedule.
Sections two and three provide presentation information to the content: the stylesheet on style and positional layout, the timesheet on temporal layout. The stylesheet and timesheet may influence each other, but there should be no circular dependencies.
The idea is that each section operates independent from and compliant with the others.
Here is a simple example where a timesheet exists, but in-line timing is also specified and overrides the timing imposed by the timesheet:
This example has a timesheet that specifies that each "li" element
will have a begin time of 10 seconds and a duration of 15 seconds. However,
the in-line timing in the second "li" element has presidence
over the timesheet and thus the second line item ends up having a begin time
of 0 seconds and a duration of 5 seconds. Note: this example could
have been done just as easily using CSS
Timing; the added power of Timesheets will be made clearer in the next
example.
<time>
<par>
li { begin=10s dur=15s }
</par>
</time>
<body>
<ul>
<li>This first line will begin at 10 sec and run for 15 sec.</li>
<li begin="0s" dur="5s">This second line's timing is dictated
by the in-line timing which overrides the timesheet timing
for each child "<li>" element. It will thus
begin at 0 seconds and last 5 seconds.</li>
</ul>
</body>
Following is an example showing some
HTML extended with timing via a Timesheet. As with the
CSS example, the Timesheet could just as
well have been contained in a separate document and applied externally.
CSS selector
syntax [CSS-selectors] has been used. The
use of CSS selectors here should not be confused with CSS Timing, proposed
in the prior section of this segment of the working draft.
The expected presentation of this would be to have the two Headings appear
together followed by the first list item in each list, namely Point A1 and
Point B1, appearing at 3 seconds followed thereafter by the second list item
in each list, namely Points A2 and B2, appearing at 6 seconds. All items
would disappear at 10 seconds which is the duration of the outer
<par>.
<html>
<head>
<time>
<par dur="10">
<par>
h1 {}
</par>
<par begin="3">
<!-- Selects the first LI in each list: -->
OL > LI:first-child { }
</par>
<par begin="6">
<!-- Selects the second LI in each list: -->
OL > LI:first-child + LI { }
</par>
</par>
</time>
</head>
<body>
<h1>Heading A</h1>
<ol>
<li id="PA1">Point A1</li>
<li id="PA2">Point A2</li>
</ol>
<h1>Heading B</h1>
<ol>
<li id="PB1">Point B1</li>
<li id="PB2">Point B2</li>
</ol>
</body>
</html>
Note: the property fields {.} could contain duration and syncarc relations if the author wished to add more complex timing.
Here is another example as mentioned in Use Case 2C. Assume a human body display language. In this example different parts appear and disappear in different combinations at different times regardless of the content structuring, i.e., regardless of the order of the data in the document body. The document DTD uses the human structure: human = { face, torso, 2 arms, 2 legs }. A leg has a thigh, knee, calf and foot. Etc. The document merely describes the structure of the human form. Here is an example of such a document:
<human>
<face id="face" ...>
<eye id="leftEye" color="green" .../>
<eye id="rightEye" color="blue" .../>
...
</face>
...
<torso>
...
</torso>
<arm id="leftArm" ...>
...
<hand id="leftHand" .../>
</arm>
...
<leg id="leftLeg" ...>
<thigh id="leftThigh" .../>
<knee id="leftKnee" .../>
<calf id="leftCalf" .../>
<foot id="leftFoot" .../>
</leg>
...
</human>
Both of the following examples are possible by applying a different timesheet in each case to the same XML document. For these examples, we use the XML "human" document, above. Note: these examples demonstrate the timesheet's ability to allow a content element to be displayed as if its parent were but with the parent not displayed, in other words the child element is displayed in the same place, spatially, as if the parent was displayed. "These examples presume that the XML language allows a content element to be displayed as if the full document was, but with some parents not displayed. In other words the child element is displayed in the same place, spatially, as if the entire document was displayed. Not all XML languages support this."
<time>
<par dur="60s">
<par>
#leftHand { }
#rightHand { }
</par>
<par begin="10s">
#leftFoot { }
#rightFoot { }
</par>
<par begin="20s">
#leftCalf { }
#rightCalf { }
#leftForearm { }
#righForearm { }
</par>
...
</par>
</time>
<time>
<par dur="60s">
<par>
#rightIndexFinger { }
#face { begin: 5s }
#rightThigh { begin: 10s }
</par>
<par>
#rightFoot { }
#rightCalf { begin: 5s }
#rightKnee { begin: 10s }
</seq>
</par>
</time>
Once SYMM has settled on the approach to integrating timing into XML-based documents, this section will precisely define the syntax and semantics of each. To reiterate: the exact syntax and the respective semantics are still being debated. The sample syntax in this part of the working draft currently serve as only a hint as to what is being considered as well as to what issues are in the process of being resolved.
In-line timing syntax has not been specified, but several possibilities are under consideration. The In-line Timing Framework section contains an example using SMIL timing.
CSS timing syntax has not been specified, but several possibilities are under consideration.
The exact specification of CSS Timing selectors is still being considered. Selector algebra will most likely be that defined by CSS2 [CSS Selectors].
The CSS Timing Framework section contains an example using SMIL timing.
Timesheet syntax has not been specified, but several possibilities are under consideration. The Timesheets Framework section contains several examples (1, 2) using SMIL timing.
The structure of the body may be used to impose temporal semantics, where a time property is assigned to an element. It is important to realize that time relations are imposed between the elements selected. For instance, when selecting a <ol> in a <seq> relation, it means that the ordered list is going to be displayed after or before some other element. It does not mean that the list items contained by the ordered list are to be presented in a sequence.
In order to provide a syntax for denoting temporal relations in line with the body structure, a new type of selectors is added to those already available from CSS.
CSS has the notion of class selectors. These selectors imply that the rule (time relation) they are part of should be applied for each element in the body that is a member of that class.
Timesheets add a new type of class selectors, henceforth to be called structure selectors. These selectors imply that the time relation they are part of applies to the result of expanding the structure selector into id selectors of all elements in the body that are members of that structure class. The id selectors have to appear in the order in which the elements lexically appear in the body. In this way, by selecting the class of descendants, the structure of the body section can be copied into the time section, such that the copied structure receives the temporal semantics required.
Another form of using the structure in the XML body is called ownership. Ownership dictates whether a temporal relationship imposed on an element applies to all of its descendants or only on the element itself. Ownership applies for example in the sequenced <ol> case when child <li> element(s) contain further markup. By specifying that ownership is on, the children of <li> element(s) will also take on the same temporal relationship as their parents.
As discussed earlier, in timesheets there are two ways to expand class selectors:
<seq> of
<li> without identifying all these <li> individually.
The exact specification of timesheet selectors is still being considered. Selector algebra will most likely be that defined by CSS2 [CSS Selectors] with some additional algebra defined as necessary.
In addition to selecting elements, style rules should be selectable. This enables changing style properties over time, just as we saw in the In-Line Timing color style example.
In the case where in-line timing and another method are active simultaneously, in-line timing always takes precedence if a conflict arises. This enables the creation of CSS Timing or Timesheets to be used as templates whose rules can be easily modified locally by in-line constructs.
This section provides the formal specification which has not yet been specified.
<imagelist timeLine="seq" end="28s"> <image dur="5s" src="image1.jpg" /> <image dur="3s" src="image2.jpg" /> <image dur="12s" src="image3.jpg" /> <image dur="10s" src="image4.jpg" /> </imagelist>
/* style sheet document "growlist.css": */
.seqtimecontainer { timeLine: seq; dur: 30s}
LI { dur: 10s; }
<!-- HTML document (which happens to be well-formed XML): -->
<HTML>
<HEAD>
<LINK rel="stylesheet" type="text/css" href="growlist.css" />>
</HEAD>
<BODY>
<UL class="seqtimecontainer">
<LI>This is item 1. It appears from 0 to 30 seconds.
</LI>
<LI>This is item 2. It appears from 10 to 30 seconds.
</LI>
<LI>This is item 3. It appears from 20 to 30 seconds.
</LI>
</UL>
</BODY>
</HTML>
<rectangle id="window" geometry="..." fill="...">
<square id="b1" ... >
<square id="s1" ... / >
</square>
<square id="b2" ... >
<square id="s2" ... / >
</square>
<square id="b3" ... >
<square id="s3" ... / >
</square>
</rectangle>
In order to time the presentation of the elements so that the big squares pop up one after the other, followed by the simultaneous appearance of the small ones, the timesheet might look like this:
<time>
<seq>
<par>
#b1 { dur: 2s }
#b2 { dur: 2s; begin: 2s; }
#b3 { dur: 2s; begin: 4s; }
</par>
<par>
#s1 { }
#s2 { }
#s3 { }
</seq>
</seq>
</time>
Note: the outer "window" rectangle has not been given any explicit timing.
for this example, we assume that the lack of timing implies a begin time
of zero and an infinite duration if the element does not have an implicit
duration.