TextFormat Mapping to Requirements

It is possible to include such metadata in a first cue of 0 duration, which has been seen in SRT before, too:

00:00:00.000 --> 00:00:00.000
Kind=descriptions

Turning them on/off is a player issue, i.e will be done through the <track> element.

<video>
  <track id=”dv1” kind=”descriptions” srclang=”en” src=”dv1.wsrt”>
</video>

 <p begin="0s" end="10s" role="description"> This is a description cue. </p>
 <p begin="0s" end="10s" role="caption"> This is a caption cue. </p>
 <p begin="11s" end="12s" role="description"> This is another description cue. </p>

<video>
  <track id=”dv2” kind=”descriptions” srclang=”en” src=”dv2.wsrt”>
</video>

 00:00:00.000 --> 00:00:10.000
 This is a first description cue.

 00:00:10.000 --> 00:00:20.000
 This is a second description cue.

<video>
  <track label=”rough” kind=”descriptions” srclang=”en” src=”dv1.wsrt”>
  <track label=”detailed” kind=”descriptions” srclang=”en” src=”dv2.wsrt”>
</video>

<video>
  <track label=”description” kind=”descriptions” srclang=”en” src=”dv1.ttml”>
  <track label=”Beschreibung” kind=”descriptions” srclang=”de” src=”dv2.ttml”>
</video>

<video>
  <track label=”description” kind=”descriptions” srclang=”en” src=”dv1.wsrt”>
  <track label=”Beschreibung” kind=”descriptions” srclang=”de” src=”dv2.wsrt”>
</video>

<p dur="20s">
     <metadata>
        Title: The Prophet
        Binding: Paperback
        ISBN 10: 000100039X
        ISBN 13: 9780001000391
        Keywords: Prophet - Gibran
        Subjects: POETRY / Inspirational & Religious; 
     </metadata>
     And when the shadow fades and is no more, the light that lingers becomes a shadow to another light.<br/>
     And thus your freedom when it loses its fetters becomes itself the fetter of a greater freedom.
     <span><metadata>Author: Kahlil Gibran</metadata>
           Khalil Gibran <span lang="ar">جبران خليل جبران بن ميخائيل بن سعد,</span> 
     </span>

0:00:00.000 --> 00:00:00.000
Kind=descriptions
Language = de
OriginalProgramTitle = Of Apples and Oranges
OrigianlEpisodeTitle = Epilogue
SubtitleListRefCode = invoice123
CreationDate = 2010-11-24T11:15:52EST
RevisionDate = 2010-11-24T11:15:52EST
RevisionNumber = 99
MaxCharWidth = 30
MaxRows = 3
CountryOfOrigin = Australia
Publisher = Gingertech
Editor = Mr Bean
EditorContact = mrbean@gmail.com
Copyright = Gingertech
License = http://creativecommons.org/licenses/by-sa/2.5/

    <div region='descriptionArea'>
      <p xml:id='desc1' ttm:role='description' begin='18s' dur='1s'
        >Open on a man in sports jacket and tie in front of a plain white background waving. An email address sean@windows.com is overlaid</p>
      <p xml:id='desc2' ttm:role='description' begin='5.08s' dur='1s'
        >Woman in casual clothing stands in front of a white board covered in technical diagrams. An email address uche@windows.com is overlaid.</p>
     ...

<video>
  <track id=”tvd1” kind=”descriptions” srclang=”en” src=”tvd1.wsrt”>
</video>

 00:00:00.000 --> 00:00:10.000
 This is a first description cue.

 00:00:10.000 --> 00:00:20.000
 This is a second description cue.

<video>
  <track id=”tvd2” kind=”descriptions” srclang=”en” src=”tvd2.wsrt”>
</video>

 00:00:00.000 --> 00:00:10.000
 <1>This is a description cue by voice 1.

 00:00:10.000 --> 00:00:20.000
 <narrator>This is a description cue by the narrator.

 00:00:20.000 --> 00:00:30.000
 <music>This is some music.

Playback speed is a function of the screen reader software.

Example for dynamic end time is in TVD-4.

<p begin="30s" dur="5s">
   <metadata>
   <speak xmlns="http://www.w3.org/2001/10/synthesis">
   <s xml:lang="en-US">
      <voice name="David" gender="male" age="25">
      <prosody duration="5s">
        You shall be free indeed when your days are not without a care nor 
        your nights without a want and a grief</voice>
     </prosody>
   </s>
   </speak>
   </metadata>
   You shall be free indeed when your days are not without a care nor 
   your nights without a want and a grief
</p>

Or parts of SSML e.g. just the duration hint:

<p begin="30s" dur="5s" tts:duration="5s" xmlns:tts="http://www.w3.org/2001/10/synthesis">
   You shall be free indeed when your days are not without a care nor 
   your nights without a want and a grief
</p>

<video>
  <track id=”tvd2” kind=”metadata” srclang=”en” src=”tvd2.wsrt”>
</video>

 00:00:00.000 --> 00:00:00.000
 Kind=metadata
 Type=application/ssml+xml
 Version=1.0
 Language=en-US
 Lexicon=http://www.example.com/lexicon.file
 Lexicon=http://www.example.com/strange-words.file

 00:00:00.000 --> 00:00:10.000
 <voice gender="male">
   <s>We stand in front of the <prosody rate="-20%">capitol in Washington.</prosody></s>
 </voice>

 00:00:10.000 --> 00:00:20.000
 <voice gender="male">
   <s>There is a crowd gathered <break/> and it is raining.</s>
  </voice>

<tt xml:lang="en" xmlns="http://www.w3.org/ns/ttml" xmlns:ttm="http://www.w3.org/ns/ttml#metadata">
  <head>
    <ttm:agent xml:id="connery" type="person">
      <ttm:name type="family">Connery</ttm:name>
      <ttm:name type="given">Thomas Sean</ttm:name>
      <ttm:name type="alias">Sean</ttm:name>
      <ttm:name type="full">Sir Thomas Sean Connery</ttm:name>
    </ttm:agent>
    <ttm:agent xml:id="bond" type="character">
      <ttm:name type="family">Bond</ttm:name>
      <ttm:name type="given">James</ttm:name>
      <ttm:name type="alias">007</ttm:name>
      <ttm:actor agent="connery"/>
    </ttm:agent>
  </head>
  <body>
    <div>
      ...
      <p ttm:agent="bond">I travel, a sort of licensed troubleshooter.</p>
      ...
    </div>
  </body>
</tt>

 00:00:00.000 --> 00:00:10.000
 <1>This is a description cue by voice 1.

 00:00:10.000 --> 00:00:20.000
 <narrator>This is a spoken by <span class=fn>Charles de Gaulles</span>.

 00:00:20.000 --> 00:00:30.000
 <2 lang=fr>Bonjour mes amis.

<video>
  <track id=”tvd3” kind=”descriptions” srclang=”en” src=”tvd3.wsrt”>
</video>

would be given to the screen reader, which could route its output to a headphone or Braille device rather than the computer speaker.

<p begin="1s" dur="1t" role="x-extended-description>This description pauses the video"</p>

The video is paused, and so the text duration is minimal, since it is only its onset which is of interest. There can be no overflow here. If the descriptions are not extended, then in order to maintain synchronisation an explicit duration on the cue will need to be specified, the speech engine will need to fit the alloted duration, an author controls overflow by not having overlapping activve cues, and if the cues do overlap; its up to the TTS engine settings whether the audio is pre-empted, continues to completion or overlapped.

 00:00:00.000 --> next
 This is a first description cue.

 00:00:10.000 --> finish
 This is a second description cue.

 00:00:30.000 --> 00:00:40.000
 This is a third description cue.

"next" could signify to stop reading when the next cue gets active if not finished reading it, "finish" could signify to pause the video until the screen reader is finished. When an end time is given, it would also stop there because it can be assumed that something in the main audio track is happening there that is more important.

Example for dynamic end time is in TVD-4.

Example for dynamic end time is in TVD-4.

<?xml version="1.0" encoding="utf-8"?>
<tt xml:lang="en" ttp:timebase="clock"
xmlns="http://www.w3.org/ns/ttml"
xmlns:ttp="http://www.w3.org/ns/ttml#parameter">
  <body  role="x-nav-work" timeContainer='seq'>
    <div role="x-nav-section" timeContainer='seq'>
      <p role="x-nav-section" timeContainer='seq'>
        <span role="x-nav-section" dur="11.300s">Index point 1.1.1 </span>
        <span role="x-nav-section" dur="20.100s">Index point 1.1.2 </span>
        <span role="x-nav-section" dur="12.900s">Index point 1.1.3 </span>
        <span role="x-nav-section" dur="13.700s">Index point 1.1.4 </span>
      </p>
      <p role="x-nav-section" timeContainer='seq'>
        <span role="x-nav-section" dur="7.200s">Index point 1.2.1 </span>
        <span role="x-nav-section" dur="28.500s">Index point 1.2.2 </span>
        <span role="x-nav-section" dur="31.090s">Index point 1.2.3 </span>
        <span role="x-nav-section" dur="41.000s">Index point 1.2.4 </span>
      </p>
     </div>
    <div role="x-nav-section" timeContainer='seq'>
      <p role="x-nav-section" timeContainer='seq'>
        <span role="x-nav-section" dur="11.300s">Index point 2.1.1 </span>
        <span role="x-nav-section" dur="20.100s">Index point 2.1.2 </span>
        <span role="x-nav-section" dur="12.900s">Index point 2.1.3 </span>
        <span role="x-nav-section" dur="13.700s">Index point 2.1.4 </span>
      </p>
      <p role="x-nav-section" timeContainer='seq'>
        <span role="x-nav-section" dur="1.300s">Index point 2.2.1 </span>
        <span role="x-nav-section" dur="2.100s">Index point 2.2.2 </span>
        <span role="x-nav-section" dur="2.900s">Index point 2.2.3 </span>
        <span role="x-nav-section" dur="3.700s">Index point 2.2.4 </span>
      </p>
    </div>
  </body>
</tt>

<video>
  <track id=”cn1” kind=”chapters” srclang=”en” src=”cn1.wsrt”>
</video>

chapter-1
00:00:00,000 --> 00:00:18,000
Introductory Titles

chapter-2
00:00:18,001 --> 00:01:10,000
The Jack Plugs

chapter-3
00:01:10,001 --> 00:02:30,000
Robotic Birds

See example in CN-1.

chapter-1
00:00:07,000 --> 00:00:18,000
Digital Media

chapter-1.1
00:00:7,001 --> 00:00:10,000
<parent>chapter-1
Digital Video

chapter-1.2
<parent>chapter-1
00:00:10,001 --> 00:00:18,000
Digital Audio

It may also be possible for the accessibility API to navigate automatically on words and sentences in cues as minimal navigational granularity which does not need extra markup.

<video>
  <track id=”cn1” kind=”chapters” srclang=”en” src=”http://othersite.com/cn4.wsrt”>
</video>

e.g. http://www.example.com/navmenu.ttml#climax

<video>
  <track id=”cc1” kind=”captions” srclang=”en” src=”cc1.wsrt”>
</video>

 00:00:00.000 --> 00:00:10.000
 This is a first caption cue from 0 to 10 sec.

 00:00:10.000 --> 00:00:20.000
 This is a second caption cue from 10 to 20 sec.

<p begin='1s' dur='1s'>c1 </p> <p begin='3s' dur='1s'>c2 </p>

Regions have a property to determine whether the background should remain visible

 00:00:00.000 --> 00:00:10.000
 This is a first caption cue from 0 to 10 sec.

 00:00:15.000 --> 00:00:20.000
 This is a second caption cue from 15 to 20 sec.
There is no caption between 10 and 15 sec.

<div timeContainer='seq'>

<p dur='1s'>c1 </p>

<p dur='1s'>c2 </p>

</div>"

See example in CC-1.

Example positioning inside the video viewport:

 00:00:00.000 --> 00:00:15.000 A:start S:100% L:100%
 This first caption is positioned at top left corner.

 00:00:15.000 --> 00:00:20.000 A:end S:100% L:0%
 This second caption bottom right corner.

  <div region='Area1'>
    <p begin='1s' dur='2s'>text one</p>
    <p begin='3s' dur='2s'>text two, will append to text one for 1 second</p>
  </div>
  <div region='Area2'>
    <p dur='5s'>text in a completely other area which stays visible the whole time</p>
  </div>

Example positioning inside the video viewport:

 00:00:00.000 --> 00:00:20.000
 This is the first caption displayed bottom centered.

 00:00:15.000 --> 00:00:20.000
 This second caption pushed the first up by a line
 since it overlaps in time.

 00:00:15.000 --> 00:00:20.000 A:middle T:50% L:50%
 This third caption appears with the second one, but centered on screen.

 <div s:writingMode="tb-lr" >
    <p>
      頭を<span m:role="x-ruby">
        股<span m:role="x-rubytext" >また</span>
      </span>に突つ込んで祈るわ<span s:writingMode="lr">ruby</span>
    </p>
  </div>

Note the the use of Tate‑chuu‑yoko (also called kumimoji and renmoji) a block of horizontal type laid out within a vertical type line. Common in Asian texts.

Example vertical rendering:

 00:00:15,042 --> 00:00:18,042 A:start D:vertical L:98%
 <ruby>左<rt>ひだり</rt></ruby>に<ruby> 見<rt>み</rt></ruby>えるのは…

 00:00:18,750 --> 00:00:20,333 A:start D:vertical-lr L:98%
 <ruby>右<rt>みぎ</rt></ruby>に<ruby> 見<rt>み</rt></ruby>えるのは…

First cue renders vertically right-to-left, second left-to-right.

<p xml:space="preserve"
>Example of a cue
split on three lines
with some    extra    whitespace
</p>

 00:00:00.000 --> 00:00:10.000
 This is a first caption cue
 broken in two lines.

 00:00:10.000 --> 00:00:20.000
 This is a
 second caption cue
 broken in three lines.

This can be specified inline - making the caption file self contained and under the control of the caption author; inline style is optional

 <p xml:id="cc01" s:fontSize="14pt">Declared as fontsize 14pt</p>

But can be overridden by the page author or user stylesheet.

 <style>
  video::cue {
    font-size: larger!important;
  }
   video::cue#cc01 {
    color: red!important;
  }
</style>

 <style>
  video track#cc9 ::cue {
    font: 15px arial,sans-serif;
  }
  video track#cc9 ::cue-part(1) {
    font: italic 12px Georgia, serif;
  }
 </style>

<video>
 <track id=”cc9” label=”caption1” kind=”captions” srclang=”en” src=”cc9.wsrt”>
</video>

 00:00:00.000 --> 00:00:10.000
 This cue is in arial 15px sans-serif.

 00:00:10.000 --> 00:00:20.000
 <1>This cue part</1> is in 12px Georgia, serif

 <style>
  video track#cc10 ::cue#cue1 {
    background-color: red;
  }
  video track#cc10 ::cue#cue2 {
    background-color: rgba(255,255,0,0.5);
  }
 </style>

<video>
 <track id=”cc10” label=”caption1” kind=”captions” srclang=”en” src=”cc10.wsrt”>
</video>

 cue1
 00:00:00.000 --> 00:00:10.000
 This cue has a red background.

 cue2
 00:00:10.000 --> 00:00:20.000
 This cue has a semi-transparent yellow background.

 <style>
  video track#cc11 ::cue-part(1) {
    color: red;
  }
  video track#cc11 ::cue-part(2) {
    color: rgba(255,255,0,0.5);
  }
 </style>

<video>
 <track id=”cc11” label=”caption1” kind=”captions” srclang=”en” src=”cc11.wsrt”>
</video>

 00:00:00.000 --> 00:00:10.000
 <1>This cue text</1> has a red background.

 cue2
 00:00:10.000 --> 00:00:20.000
 <2>This cue text</2> has a semi-transparent yellow background.

 <style>
  video track#cc12 ::cue-part(1) {
    outline: yellow dotted thick;
  }
  video track#cc12 ::cue-part(2) {
    text-shadow: 2px 1px blue;
  }
 </style>

<video>
 <track id=”cc12” label=”caption1” kind=”captions” srclang=”en” src=”cc12.wsrt”>
</video>

 00:00:00.000 --> 00:00:10.000
 <1>This cue text</1> has a tick yellow dotted outline.

 cue2
 00:00:10.000 --> 00:00:20.000
 <2>This cue text</2> has a blue shadow.

 00:00:00.000 --> 00:00:10.000
 This is the first cue.

 00:00:10.000 --> 00:00:15.000
  

 00:00:15.000 --> 00:00:20.000
 This is the second cue with a blank cue in between.

The blank cue has a background, but it remains unclear how to set the cue background box to a fixed size.

<p dur="16.5s" s:wrapOption="noWrap">
  <span begin="0.5s">The </span>
  <span begin="1.3s">video </span>
  <span begin="2.1s">element </span>
  <span begin="2.9s">is </span>
  <span begin="3.7s">a </span>
  <span begin="4.5s">media </span>
  <span begin="5.3s">element </span>
  <span begin="6.1s">whose </span>
  <span begin="6.9s">media </span>
  <span begin="7.7s">data </span>
</p>

 <style>
  video track#cc14 ::cue-part(future) {
    visibility: none;
  }
 </style>

<video>
 <track id=”cc14” label=”caption1” kind=”captions” srclang=”en” src=”cc14.wsrt”>
</video>

 cue1
 00:00:00.000 --> 20.000
 A <00:00:01.000>word <00:00:02.000>a <00:00:03.000>time <00:00:04.000>is <00:00:05.000>displayed.

 cue2
 00:00:06.000 --> 10.000
 Now the line just appears.

 cue3
 00:00:10.000 --> 00:00:20.000
 This <00:00:11.000>is <00:00:12.000>more <00:00:13.000>word <00:00:14.000>by <00:00:15.000>word <00:00:16.000>text.

Cue1 is timed and will display one word at a time at the given timestamps, which makes it roll-up mode. Cue2 just appears and is thus pop-on mode. Cue3 is added below cue1, replaces cue2, and also displays word by word. The same can be achieved for individual characters to make it paint-on.

The number of lines shown depends on the duration of the caption cues and the number of lines in each.

 00:00:10.000 --> 00:00:20.000
 <narrator>This is a piece <span lang="fr>en anglais</span>.

 00:00:20.000 --> 00:00:30.000
 <2 lang=fr>Bonjour mes amis.

The semantics can be expressed using roles.

  <div region="r1" >
    <p>
      頭を<span ttm:role="x-ruby">
        股<span ttm:role="x-rubytext" tts:fontSize="50%">また</span>
      </span>に突つ込んで祈るわ
    </p>
  </div>

The presentation using CSS:

<style type="text/css">
  video::cue[m|role="x-ruby"]
  {
    display:ruby;
  }
  video::cue[m|role="x-rubytext"]
  {
    display:ruby-text;
  }
</style>

Example:

 00:00:15,042 --> 00:00:18,042 A:start D:vertical L:98%
 <ruby>左<rt>ひだり</rt></ruby>に<ruby> 見<rt>み</rt></ruby>えるのは…

 00:00:18,750 --> 00:00:20,333 A:start D:vertical L:98%
 <ruby>右<rt>みぎ</rt></ruby>に<ruby> 見<rt>み</rt></ruby>えるのは…

See for example CC-18.

Example:

 00:00:00.000 --> 00:00:10.000
 This is <b>bold</b>.

 00:00:15.000 --> 00:00:20.000
 This is <i>italics</i>.

Example:

 00:00:00.000 --> 00:00:10.000
 <1>This is speaker one.

 00:00:15.000 --> 00:00:20.000
 <2>This is speaker two.

In future <v name> markup is planned to make it more useful.

Example:

<video>
 <track id=”cc25” label=”caption” kind=”captions” srclang=”en” src=”cc25.wsrt”>
 <track id=”cc25-ext” label=”extended cc” kind=”metadata” srclang=”en” src=”cc25-ext.wsrt”>
</video>

Example extended caption:

 title-1
 00:00:00,000 --> 00:00:02,050
 <img src="http://people.xiph.org/~giles/2004/openweekend/theora-talk/xifish.png" alt="Xiph logo"/>
 <a href="http://www.xiph.org/about/" alt="Xiph about page">About <i>Xiph.org</i></a>

 title-2
 00:00:02,050 --> 00:00:05,450
 <img src="http://www.linuxcertified.com/images/redhat-logo.jpg" alt="RedHat logo"/>
 <a href="http://www.redhat.com/" alt="RedHat Website">Sponsored by <b>RedHat</b></a>

 title-3
 00:00:05,450 --> 00:00:07,450
 <a href="http://www.xiph.org/video/vid1.shtml" alt="Xiph video page">Original Publication</a>
 <a href="http://webchat.freenode.net/?channels=xiph" alt="Xiph irc">Chat with the creators of the video</a>

Example:

<video>
 <track id=”cc26-en” label=”Captions” kind=”captions” srclang=”en” src=”cc26-en.wsrt”>
 <track id=”cc26-de” label=”Gehoerlosen-Untertitel” kind=”captions” srclang=”de” src=”cc25-de.wsrt”>
</video>

See tvd-2 for an example.

See cc-25 for an example.

See CC-6 for an example.

See TVD-4 for an example.

See DV-14 for an example.

1. the alternative content's extent is specified in relation to the media viewport (e.g., picture-in-picture video, lower-third captions)
2. the alternative content has its own independent extent, but is positioned in relation to the media viewport (e.g., captions above the audio, sign-language video above the audio, navigation points below the controls)
3. the alternative content has its own independent extent and doesn't need to be rendered in any relation to the media viewport (e.g., text transcripts)

If alternative content has a different height or width than the media content, then the user agent will reflow the (HTML) viewport. (UAAG 2.0 3.1.4).

1. text scale (i.e., the general size of text) ,
2. font family, and
3. text color (i.e., foreground and background).

1. the bounding dimensions and coordinates of rendered graphical objects;
2. font family;
3. font size;
4. text foreground color;
5. text background color;
6. change state/value notifications.