TTML mapping to HTML 5

1 Timed Text Markup Language (TTML) applied to HTML5

The <a href="http://www.w3.org/TR/ttaf1-dfxp/">TTML format (Timed Text Markup Language)</a> is a W3C format intended for marking up external timed track resources.

1.1 Document format

A TTML file referenced by an HTML5 track element must consist of a <a href="http://www.w3.org/TR/ttaf1-dfxp/">TTML file body</a> and is labeled with the <a href="#mime-type">MIME type</a> <a href="http://www.w3.org/TR/ttaf1-dfxp#media-type-registration">application/ttml+xml</a>.

1.2 Display of TTML in HTML 5

Presenting TTML in HTML 5 consists of the following steps:

• Compute inline style sets for the source TTML document
• Construct the list of event times for the source TTML document
• For each event time:
  • Calculate the TTML cue object list
  • For each TTML cue object
    • construct the corresponding styled HTML document fragment.
• While media is playing:
• Generate the CSS blocks for the active cue objects at current media playback time.
• Insert CSS blocks into rendered output.

1.2.1 TTML Style resolution

TTTML offers three mechanisms for defining the equivalent of HTML 5 inline style. The nested and referential styles of TTML being used to avoid having large numbers of repeated attributes on each element, and allow groups of styles to be applied all at once; however this is merely a shorthand mechanism and is entirely equivalent to HTML 5 inline styles. TTML does not define an applicative mode of style application itself, but does not preclude the use of a mechanism such as CSS additionally being used for this purpose, where TTML style application would all have the same specificity as inline style.

The <a href="http://www.w3.org/TR/ttaf1-dfxp/semantics-style-resolution-processing-sss" name="semantics-style-resolution-processing-sss"> Specified Style Set</a> of properties is computed for each element:

• The initial specified style set for each element is set to empty
• Then <a id="semantics-style-association-referential" name="semantics-style-association-referential">style properties referenced by the affected element using the style attribute (referential styling) are processed in the following manner:</a> <a id="semantics-style-resolution-processing-referential" name="semantics-style-resolution-processing-referential"> For each style element referenced by a style attribute on the affected element and in the order specified in that style attribute; if the referenced style element is a descendant of a styling element, merge the specified style set of the referenced element into the specified style set of the affected element.
• Then <a id="semantics-style-association-nested" name="semantics-style-association-nested"> style properties referenced through child nodes of the affected element  (nested styling) are processed in the following manner:</a> <a id="semantics-style-resolution-processing-nested" name="semantics-style-resolution-processing-nested"> For each style element child of the affected element, and in the specified order of child elements, merge the specified style set of the child element, into the specified style set of the affected element.
• <a id="semantics-style-association-inline" name="semantics-style-association-inline"> Then style properties that are applies as attributes on the the affected element using the style attribute (inline styling) are processed in the following manner:</a> <a id="semantics-style-resolution-processing-inline" name="semantics-style-resolution-processing-inline"> For each style property expressed as a specified styling attribute of on the affected element merge that property into the specified style set of the affected element.

1.2.2 TTML cue object construction

A set of TTML cue objects are constructed from the referenced TTML file by evaluating the TTML document instance at the TTML cue event times, that is, the set of time coordinates where some element becomes temporally active or inactive. The TTML document instance is mapped once for each time coordinate in the <a href="#ttml-cue-event-times"> TTML cue event times</a> to a list of TTML cue objects as defined below, each TTML cue object is then converted into an HTML 5 cue object.

Each region active at the TTML cue event time in the source TTML will map to one <a href="#ttml-cue">TTML cue object</a> in the list. If there is no region specified in the TTML document instance, then the default region is used, and there will be at most one TTML cue object in the list.

1.2.3 Evaluating the TTML cue event times

Map the TTML source document to a set of event times by recursively walking the DOM tree annotating each node with its absolute begin and end times, based on the begin, end and dur attributes; and recursively for each of the nodes children. The initial time containement context is seq, and the initial reference start and end times are that of the media to which the timed track applies.

Compute time intervals for an element based on the time containment context, a reference start time and a reference end time in the following manner:

        set computed start time and computed end time to the zero time
            
       Compute the beginning of the current element interval:
          set begin to the value of the "begin" attribute if present or the zero time otherwise
          set computed start time to the reference start time + begin;

       Compute the simple duration of the interval:  
          (Note that par children have indefinite default duration, while seq children have 
          zero default duration. indefinite is truncated to the reference end time)
           
          if the "dur" attribute is set and the "end" attribute is not set and the time container context is seq
	      set referenceDur to the zero time          else
                if computed start time is less than the reference end time
                  set referenceDur to the reference end time - computed start time 
              else
	          set referenceDur to the zero time
              if the "dur" attribute is set
	          set dur to the "dur" attribute value.
              if dur is greater than referenceDur
                  set dur to referenceDur;
              else
                  set dur to referenceDur;
              
          set computed end time to computed start time + dur;
           
          (note end can truncate the simple duration)
          set offsetEnd to the zero time and add reference start time

          if attribute "end" is set
              set end to reference start time + value of "end" attribute
          else
              set end to reference end time;

          set computed end time to the min of end and computed end time 
    
    Compute the child nodes:
          if the time container context is par
              for each child element of the node:
                  Compute Time Intervals for the child with par context, start time 
                  as computed start time and end time as computed end time 
          else
              for each child element of the node:
                  set s to computedStartTime 
                  Compute Time Intervals for the child with seq context, start time 
                  as s and end time as computed end time 
                  set s to the computed end time of child;

An element is temporally active at time t, if the computed start time of the element is less than or equal to t, and t is less than the computed end time of the element. The TTML cue event times are those times where some element changes state from temporally inactive to temporally active or vice versa; that is, the Set of computed begin and end times in the annotated tree placed in order.

1.2.4 Evaluating the TTML document instance at event time

Map the TTML source document to a set of active regions at each <a href="#"> TTML cue event time</a> as follows:

1. <a id="semantics-region-layout-step-1" name="semantics-region-layout-step-1"> For each temporally active region element replicate the sub-tree of the source document headed by the body element;</a>
2. <a id="semantics-region-layout-step-2" name="semantics-region-layout-step-2"> Evaluating this sub-tree in a post-order traversal, prune elements if they are: not a content element, if they are temporally inactive, if they are empty, or if they aren't associated with the current active region;</a>
3. <a id="semantics-region-layout-step-3" name="semantics-region-layout-step-3"> If the pruned sub-tree is non-empty, then reparent it to the current active region element and add the current active region to the output list.</a>

A content element is associated with a region according to the following ordered rules, where the first rule satisfied is used and remaining rules are skipped:

1. if the element specifies a <a href="#layout-attribute-region"> region</a> attribute, then the element is associated with the region referenced by that attribute;

2. if some ancestor of that element specifies a <a href="#layout-attribute-region">region</a> attribute, then the element is associated with the region referenced by the most immediate ancestor that specifies this attribute;

3. if the element contains a descendant element that specifies a <a href="#layout-attribute-region">region</a> attribute, then the element is associated with the region referenced by that attribute;

4. if a default region was implied (due to the absence of any <a href="#layout-vocabulary-region">region</a> element), then the element is associated with the default region;

5. the element is not associated with any region.

<a id="semantics-region-layout-example" name="semantics-region-layout-example">Example</a>

An example of the processing steps described above is elaborated below, starting with <a href="#region-mapping-example-1-s">Example Sample Source Document</a>.

<tt tts:extent="640px 480px" xml:lang="en"
  xmlns="http://www.w3.org/ns/ttml"
  xmlns:tts="http://www.w3.org/ns/ttml#styling">
  <head>
    <layout>
      <region xml:id="r1">
        <style tts:origin="10px 100px"/>
        <style tts:extent="300px 96px"/>
      </region>
      <region xml:id="r2">
        <style tts:origin="10px 300px"/>
        <style tts:extent="300px 96px"/>
      </region>
    </layout>
  </head>
  <body xml:id="b1">
    <div xml:id="d1" begin="0s" dur="2s">
      <p xml:id="p1" region="r1">Text 1</p>
      <p xml:id="p2" region="r2">Text 2</p>
    </div>
    <div xml:id="d2" begin="1s" dur="2s">
      <p xml:id="p3" region="r2">Text 3</p>
      <p xml:id="p4" region="r1">Text 4</p>
    </div>
  </body>
</tt>

The event times for this document are 0s, 1s, 2s and 3s. The result of performing the processing described above for eah of these times will be an  intermediate document containing a sequence of region elements; for example at media time of 0s the following intermediate document would be produced:

          <region xml:id="r1" 
	       tts:origin="10px 100px" 
	       tts:extent="300px 96px" /> 
	         <body xml:id="b1"> 
	         <div xml:id="d1"> 
	         <p xml:id="p1">Text 1</p>
	     </div>
	    </body> 
	  </region>
	  <region xml:id="r2"
	        tts:origin="10px 300px" 
	        tts:extent="620px 96px" /> 
	          <body xml:id="b1"> 
	          <div xml:id="d1"> 
	          <p xml:id="p2">Text 2</p> 
	      </div>
	    </body> 
	  </region>
	  	  

1.2.5 TTML cue to HTML cue construction rules

To support the <a href="html5#timed-track"> timed track</a> model of HTML, each region element in the intermediate document is converted to a <a href="#timed-track-cue"> timed track cue</a> with the following assignments:

The <a href="#timed-track-cue-identifier"> timed track cue identifier</a>

Is set to the value of xml:id of the region used to construct the cue, or "" if the default region is used.

The <a href="html5#timed-track-cue-pause-on-exit-flag"> timed track cue pause-on-exit flag</a>

Is set to false unless the <a href="#pause-on-exit-flag"> html:pauseOnExit </a> attribute is set anywhere in the region markup.

The <a href="html5#timed-track-cue-writing-direction"> timed track cue writing direction</a>

Is set to the dominant writing direction used in the region markup if that is defined. "" otherwise.

The <a href="html5#timed-track-cue-snap-to-lines-flag"> timed track cue snap-to-lines flag</a>

Is set to false

The <a href="html5#timed-track-cue-line-position"> timed track cue line position</a>

Is made equivalent to the y part of the origin of the region if set, 0 otherwise.

The <a href="html5#timed-track-cue-text-position"> timed track cue text position</a>

Is made equivalent to the x part of the origin of the active region if set, 0 otherwise.

The <a href="html5#timed-track-cue-size"> timed track cue size</a>

Is made equivalent to the x part of the extent of the active region if set, 0 otherwise. (height should be set similarly)

The <a href="html5#timed-track-cue-alignment"> timed track cue alignment</a>

Set to zero.

The <a href="html5#timed-track-cue-voice-identifier"> timed track cue voice identifier</a>

Calculate the set of ttm:role attribute values used in the region markup. If it the set is a singleton set consisting of one of the following values, then the value of voice identifier is mapped as follows:

• If the role value is narration then set voice to narrator.
• If the role value is music then set voice to music.
• If the role value is lyrics then set voice to lyric.
• If the role value is sound then set voice to sound.
• If the role value is x-comment then set voice to comment.
• If the role value is x-credit then set voice to credit.

If value is not mapped above, or the set is multivalued, the value of voice is set to an integer; if the same set of roles is used in subsequent cues, then the same number shall be re-used, otherwise the number shall be unique over all cues in the track.

1.2.6 TTML cue text DOM construction rules

The body of the HTML5 cue is constructed from the markup of the region by converting the <a href="#ttml-intermediate-object">TTML Intermediate Document Object Tree</a> into a DOM tree for the <a href="html5/infrastructure.html#document">Document</a> owner. User agents must create a <a href="html5/infrastructure.html#documentfragment">DocumentFragment</a> node for each HTML 5 cue, and populate it with a tree of DOM nodes that is isomorphous to the tree of <a href="#ttml-intermediate-object" title="TTML Intermediate Document Object">TTML Intermediate Document Object Tree</a>, using the following mapping of <a href="#ttml-intermediate-object">TTML Intermediate Document Object</a> to DOM nodes:

 <a href="http://www.w3.org/TR/ttaf1-dfxp/semantics-style-resolution-processing-sss" name="semantics-style-resolution-processing-sss0">
 Specified Style Set</a> of properties of the element is merged into the The
 <a href="http://www.w3.org/TR/ttaf1-dfxp/semantics-style-resolution-processing-sss" name="semantics-style-resolution-processing-sss1">

The <a href="">ownerDocument</a> attribute of all nodes in the DOM tree must be set to the given document owner.

For each <a href="html5/elements.html#htmlelement"> HTMLElement</a> in the document fragment constructed above; if the <a href="http://www.w3.org/TR/ttaf1-dfxp/#semantics-style-resolution-processing">specified style set</a> computed for the corresponding TTML element is not empty, create a <a href="http://www.w3.org/TR/DOM-Level-2-Style/css.html#CSS-CSSStyleDeclaration"> CSSStyleDeclaration</a> and add to it the styles as defined by the ordered rules below, finally add the <a href="http://www.w3.org/TR/DOM-Level-2-Style/css.html#CSS-CSSStyleDeclaration"> CSSStyleDeclaration</a> to the <a href="html5/elements.html#dom-style"> style</a> attribute on the  <a href="html5/elements.html#htmlelement"> HTMLElement</a>.

1. if the specified set contains the property <a href="http://www.w3.org/TR/ttaf1-dfxp/#style-attribute-backgroundColor">backgroundColor</a> call <a href="">setProperty</a> with propertyName="background-color", value=<a href="#color-value"><color value></a>, priority="".
2. if the specified set contains the property <a href="">tts:color</a>, call <a href=""> setProperty</a> with propertyName="color", value=<color value>, priority="".
3. if the specified set contains the property <a href="">tts:direction,</a> call <a href=""> setProperty</a> with propertyName="direction", value=<direction value>, priority="".
4. if the specified set contains the property <a href="">tts:display</a>, call <a href=""> setProperty</a> with propertyName="display", value="", priority="".
5. if the specified set contains the property <a href="">tts:displayAlign</a>, call <a href=""> setProperty</a> with propertyName="", value="", priority="".

 (CSS3)

6. if the specified set contains the property <a href="">tts:extent</a> and the TTML element was region, call <a href=""> setProperty</a> with: propertyName="width", value=<width value>, priority="" and propertyName="height", value=<height value>, priority="". If extent is not set and the TTML element was region (e.g. the region is the default region), height and width of the div will be auto.
7. if the specified set contains the property <a href="">tts:fontFamily</a>, call <a href=""> setProperty</a> with propertyName="font-family", value=<font family value>, priority="".
8. if the specified set contains the property <a href="">tts:fontSize</a>, call <a href=""> setProperty</a> with propertyName="font-size", value=<font size value>, priority="".
9. if the specified set contains the property <a href="">tts:fontStyle</a>, call <a href=""> setProperty</a> with propertyName="font-style", value=<font style value>, priority="".
10. if the specified set contains the property <a href="">tts:fontWeight</a>, call <a href=""> setProperty</a> with propertyName="font-weight", value=<font weight value>, priority="".
11. if the specified set contains the property <a href="">tts:lineHeight</a>, call <a href=""> setProperty</a> with propertyName="line-height", value=<line height value>, priority="".
12. if the specified set contains the property <a href="">tts:opacity</a> and the TTML element was region, call <a href=""> setProperty</a> with propertyName="", value="", priority="" (CSS3).
13. if the specified set contains the property <a href="">tts:origin</a> and the TTML element was region, call <a href=""> setProperty</a> with: propertyName="position", value="absolute", priority="", propertyName="left", value=<left value>, priority="" and propertyName="top", value=<top value>, priority="".
14. if the specified set contains the property <a href="">tts:overflow</a> and the TTML element was region, call <a href=""> setProperty</a> with propertyName="", value="", priority="".
15. if the specified set contains the property <a href="">tts:padding</a> and the TTML element was region, call <a href=""> setProperty</a> with propertyName="padding", value=<padding value>, priority="".
16. if the specified set contains the property <a href="">tts:showBackground</a> and the TTML element was region, then if the div has no children call <a href=""> setProperty</a> with propertyName="background-color", value="transparent", priority="".
17. if the specified set contains the property <a href="">tts:textAlign</a>, call <a href=""> setProperty</a> with propertyName="text-align", value=<text align value>, priority="".
18. if the specified set contains the property <a href="">tts:textDecoration</a>, call <a href=""> setProperty</a> with propertyName="text-decoration", value=<text decoration value>, priority="".
19. if the specified set contains the property <a href="">tts:textOutline</a>,  call <a href=""> setProperty</a> with propertyName="", value="", priority="" (CSS3).
20. if the specified set contains the property <a href="">tts:unicodeBidi</a>,  call <a href=""> setProperty</a> with propertyName="unicode-bidi", value=<bidi value>, priority="".
21. if the specified set contains the property <a href="">tts:visibility</a>, call <a href=""> setProperty</a> with propertyName="visibility", value=<visibility value>, priority="".
22. if the specified set contains the property <a href="">tts:wrapOption</a> with value noWrap, call <a href=""> setProperty</a> with propertyName="whitespace", value="nowrap", priority="". (TBD)
23. if the specified set contains the property <a href="">tts:writingMode</a>, call <a href=""> setProperty</a> with propertyName="writing-mode", value=<writing mode value>, priority="" (CSS3).
24. if the specified set contains the property <a href="">tts:zIndex</a> and the TTML element was region, call <a href=""> setProperty</a> with propertyName="z-index", value=<z value>, priority="".

Map the following elements in the #metadata namespace to attributes on the parent <a href="html5/elements.html#htmlelement"> HTMLElement</a> as follows:

1. ttm:title : copy text content to the <a href="html5/elements.html#the-title-attribute"> title</a> attribute

Map attributes in the #metadata namespace on the TTML DOM element to attributes on the <a href="html5/elements.html#htmlelement"> HTMLElement</a> as follows:

1. ttm:agent : add the value of this attribute to the <a href="html5/elements.html#dom-classes"> class</a> attribute.
2. ttm:role : add the value of this attribute to the <a href="html5/elements.html#dom-classes"> class</a> attribute.

Copy xml:id, xml:lang attributes if present on the TTML DOM element to the <a href="html5/elements.html#htmlelement"> HTMLElement</a> as the <a href="html5/elements.html#dom-id"> id</a>, and <a href="html5/elements.html#dom-lang"> lang</a> attributes. <p>The xml:space attribute on an element, if the value is 'preserve'; then the content of the element should be contained within an  <a href="html5/elements.html#htmlelement">HTMLElement</a> element node with <a href="html5/infrastructure.html#dom-node-localname">localName</a> "<a href="html5/text-level-semantics.html#the-div-element">pre</a>" and the <a href="html5/infrastructure.html#dom-node-namespaceuri">namespaceURI</a> set to the <a href="html5/urls.html#html-namespace-0"> HTML namespace</a>.<p>All characteristics of the DOM nodes that are not described above or dependent on characteristics defined above must be left at their initial values.

Continuing the above example, the HTML fragment equivalents for the two HTML cue objects will be as follows:

          <div xml:id="r1" 
	       style="position:absolute; left:10px; top:100px;
	              width:300px; height:96px" /> 
	         <div xml:id="b1"> 
	            <div xml:id="d1"> 
	               <p xml:id="p1">Text 1</p>
	            </div>
	         </div> 
	  </div>

	  <div xml:id="r2"
	        style="position:absolute; left:10px; top:300px;
	        width:620px; height:96px" /> 
	        <div xml:id="b1"> 
	           <div xml:id="d1"> 
	              <p xml:id="p2">Text 2</p> 
	           </div>
	        </div> 
	  </div><br>
	  

Style values

The mapping from TTML style values into HTML 5 is as follows:

• <a id="color-value">color value</a> - Convert the TTML color to its RGBA equivalent, and set value to CSS rgba(R,G,B,A). [CSSCOLOR]
• <a id="direction-value">direction value</a> - map to like named CSS values.[CSS]
• <a id="display-value">display value</a> - map none to like named CSS value, map auto to CSS block if the HTML element is not span or text node, CSS inline otherwise.[CSS]
• <a id="display-align-value">display align value</a> - map .[CSS3]
• <a id="font-family-value">font family value</a> - copy value.[CSS]
• <a id="font-size-value">font size value</a> - cell based values must be calculated from the content rectangle of the video containing CSS box and denoted as CSS % metrics. Only square font sizes are converted (any second size is ignored), and to like named metrics from CSS.[CSS]
• <a id="font-style-value">font style value</a> - map to like named values from CSS.[CSS]
• <a id="font-weight-value">font weight value</a> - map to like named values from CSS.[CSS]
• <a id="line-height-value">line height value</a> - map to like named values from CSS.[CSS]
• <a id="opacity-value">opacity value</a> - map to like named values from CSS.[CSS3]
• <a id="overflow-value">overflow value</a> - map to like named values from CSS.[CSS]
• <a id="padding-value">padding value</a> - map to like named values from CSS.[CSS]
• <a id="text-align-value"> text align value</a> - map left, center and right to like named values from CSS. If direction is ltr, map start and end to left and right respectivley, if direction is rtl map start and end to right and left respectively. [CSS]
• <a id="text-decoration-value">text decoration value</a> - do not map noUnderline noLineThrough or noOverline; map lineThrough to line-through, otherwise map to like named values from CSS.[CSS]
• <a id="text-outline-value">text outline value</a> - map to like named values from CSS3.[CSS3]
• <a id="bidi-value">bidi value</a> - map bidiOverride to bidi-override, otherwise map to like named values from CSS.[CSS]
• <a id="visibility-value">visibility value</a> - map to like named values from CSS.[CSS]
• <a id="writing-mode-value">writing mode value - map to like named values from CSS3.[CSS3]
• <a id="z-value">z value</a> - The value is added to the z value of the media element that references the track in such a way that the media element rendering area (including any controls) will lie immediately behind the CSS boxes created for the cue elements, and the next immediately higher CSS box in the HTML page will lie in front of all CSS boxes created by cues.

1.3 Rendering Rules

Create a set of CSS boxes in relation to the rendering area of the <a href="html5/video.html#media-element"> media element</a> as follows:

1. If the <a href="html5/video.html#media-element"> media element</a> is a playback mechanism with no rendering area, abort these steps. There is nothing to render.

2. Let video be the <a href="html5/video.html#media-element"> media element</a> or other playback mechanism
3. let textArea be a CSS containing block whose containing block is the rendering area for video, set the writing mode (CSS3) for textArea to lr-bt.
4. Let tracks be the subset of video's <a href="html5/video.html#list-of-timed-tracks"> list of timed tracks</a> that have as their <a href="html5/video.html#rules-for-updating-the-timed-track-rendering"> rules for updating the timed track rendering</a> these rules, and whose <a href="html5/video.html#timed-track-mode"> timed track mode</a> is <a href="html5/video.html#timed-track-showing"> showing</a>.
5. Let cues be an empty list of <a href="html5/video.html#timed-track-cue"> timed track cues</a>.
6. For each track in tracks, append to cues all the <a href="html5/video.html#timed-track-cue"> cues</a> computed as above for each each <a href="#"> TTML cue event time</a>.
7. For each <a href=""> timed track cue</a> that is <a href="html5/video.html#timed-track-cue-active-flag">active</a>, run the following substeps:
   1. Let nodes be the HTML fragment computed for the cue.
   2. Apply the terms of the CSS specifications to nodes to obtain a set  CSS boxes relative to the CSS box created for the root div element in the HTML fragment, which is in turn relative to textArea: <a href="html5/references.html#refsCSS"> [CSS]</a>.
   3. Add the CSS boxes in boxes to display.
   .

1.4 HTML5 TTML Profile

1.4.1 Additional attributes

<a id="html-pauseOnExit">html:pauseOnExit</a> - this attribute in the <a href="html5/uri.html#namespaces">HTML namespace</a> if specified on a TTML element, is mapped to the HTML 5 cue attribute of the same name. It causes the media progress to halt when the media playback position is most nearly equal to the event time that the element containing the attribute becomes active. It takes any string value (or none?) the value is ignored, any number of such attributes may be present in a TTML cue. If no such attribute is present then the value mapped on the cue is false, otherwise it is true.