Timed Text (TT) Authoring Format 1.0 Use Cases and Requirements

1.1 Motivation

A principal motivation for the development of a common authoring format for timed text is the lack of a standard content format that supports the representation and interchange of textual information which is synchronized with other media elements or which serves as a synchronization master itself.

Popular proprietary multimedia systems and their corresponding player components have defined distinct timed text formats for each proprietary use. As a consequence there is no common authoring format that serves as a portable interchange format between such systems. A goal of the present work is to define such a portable interchange format to ease the burden of authoring tool developers and users as well as enhance portability of timed text content.

A side effect of the development and deployment of a common timed text authoring format is that it simplifies the creation and distribution of synchronized text for use with a multitude of devices, such as multimedia players, caption, subtitle, and teletext encoders and decoders, character generators, LED displays, and other text display devices.

1.2 System Model

The uses cases and requirements specified in this document are based upon a system model, depicted in Figure 1 – System Model, wherein the timed text authoring format serves as a bidirectional interchange format among a heterogeneous collection of authoring systems, and as a unidirectional interchange format to a heterogeneous collection of distribution formats after undergoing transcoding to the target distribution formats as required.

Figure 1 – System Model

2.1 Acronyms

2.2 Terminology

Timed Text

Textual information that is intrinsically or extrinsically associated with timing information.

Timed Text Authoring Format

A content type that represents timed text media for the purpose of interchange among authoring systems.

Timed Text Authoring System

A content authoring system capable of importing and exporting timed text authoring format content.

2.3 Notations

The following notations are employed in this document:

• SXXX– use case scenario XXX

• RXXX– requirement XXX

• RX9X– solution space requirement X9X

• NXXX– non-requirement XXX

4.1 General

R100 – Specification Format

The TT AF specification(s) shall be authored using XML and XSL Stylesheets based on [XML Spec] and shall adhere to best current practices in the W3C for specification style and quality assurance.

R101 – Specification Modularity

The TT AF specification(s) shall be defined in a modular manner that logically separates significant areas of functionality to as great extent as is practical.

R102 – Specification Organization

The TT AF specification(s) shall be organized in such a manner as to separate the following aspects:

• TT Framework

• TT Core Vocabulary

• TT Core Document Types

• TT Extension Vocabulary(ies)

• TT Extension Document Type(s)

R103 – Core and Periphery

The TT AF specification(s) shall be defined in such a manner that core functionality is logically separated from peripheral functionality.

R104 – Evolution of Core

The TT AF specification(s) shall be defined in such a manner that core functionality can evolve over time, e.g., by the specification of multiple levels (or versions) of core functionality.

R105 – Ownership of Core

The TT AF specification(s) shall be defined in such a manner that core functionality be specified soley by the TT WG or, in the event that the TT WG is terminated, its successors within the W3C.

Note:

It is assumed that one or more appropriate namespace mechanisms will be used to segregate core functionality defined or adopted in the TT AF from peripheral functionality defined or adopted by clients of the TT AF.

R106 – Surjection of Core

The TT AF specification(s) shall be defined in such a manner that for every item in the TT AF core vocabulary, there shall be at least one TT AF core document type that makes use of that item, i.e., there exists a surjection from the set of TT AF core document types to the set of TT AF vocabulary items referenced by those document types.

Note:

The TT AF specification(s) may define standardized peripheral vocabulary that is not referenced by any TT AF core document type.

R107 – Evolution of Periphery

The TT AF specification(s) shall be defined in such a manner that peripheral functionality can evolve over time, e.g., by the future specification of one or more peripheral functionality modules.

R108 – Ownership of Periphery

The TT AF specification(s) shall be defined in such a manner that peripheral functionality need not be specified by the TT WG or the W3C, but may be specified by other W3C WGs as well as non-W3C clients of the TT AF.

R109 – Transformability

The TT AF shall be capable of being transformed, without undue complexity, into one or more legacy timed text content formats, e.g., [3GPP], [QText], [RealText], [SAMI], etc.

Note:

The above list of potential target timed text content formats is strictly informative, and is not intended to be exhuastive.

R110 – Streamable Tranformation

The TT AF shall be capable of being transformed into an idealized streamable representation format.

Note:

It is not required that an idealized streamable representation format be defined by the TT AF specification(s); however, the definition of such a format may be the subject of future activities by the TT WG.

Note:

It is intended that existing closed captioning and subtitle streaming formats used by analog and digital television services as well as timed text used in the context of streaming audio and motion video formats be potentially targeted by one or more transformations of the TT AF. These formats include [EIA-608B], [EIA-708B], [EN 300 706], [EN 300 743], etc.

R111 – Accessibility – Content

The TT AF shall include the following accessibility related features:

• Support for a mechanism to explicitly associate Equivalent Alternatives to the textual information in the TT presentation in accordance with [WAI XML AG] Guideline 1.

• Support for Content Rendering Adaption in accordance with [WAI SMIL AG], Section 5. See also Conditional Content.

• Use of a default text vocabulary that satisfies guideline 2 of [WAI XML AG] regarding structural and semantical stringency.

• Ability to extend or replace the default text vocabulary with other XML dialects to represent the textual information of the TT presentation. See also Intrinsic and Extrinsic Text Content.

• Support for explicit definition of a Navigational Structure associated with the TT presentation in accordance with [WAI SMIL AG], Section 4.3.

R112 – Accessibility – Authoring System

The TT AF specification(s) shall be defined in such a manner as to require a TT AS to adhere to all applicable aspects of [ATAG 1.0].

4.2 Content

R200 – Authorability

The TT AF shall be capable of being created and modified using a plain text editor, e.g., emacs, vi, etc.

R201 – Multiple Natural Languages

The TT AF shall be capable of representing content of different natural languages, where the content of distinct languages may be segregated into separate document instances or may be integrated into a single document instance.

R202 – Natural Language Coverage

The TT AF shall be capable of representing content of at least those specific natural languages that may be represented with [Unicode 3.2].

R203 – Natural Language Association Granularity

The TT AF shall be capable of associating natural language binding information with plain text information at the granularity of a single coded character.

R204 – Minimum Character Representability

The TT AF shall be capable of representing every coded character available in [Unicode 3.2] by using only those characters in [ASCII (ANSI X3.4)].

Note:

This requirement facilitates the entry and editing of characters in a TT AF document instance that would otherwise not be permitted due to lack of an appropriate character input method or lack of support for a non-ASCII character encoding system.

It is assumed that every TT AS will provide a means to enter and edit TT AF document instances represented in the ASCII character set.

R205 – Intrinsic and Extrinsic Text Content

The TT AF shall be capable of expressing text content intrinsically within a TT AF document instance, extrinsically by referencing from a TT AF document instance to text content in one or more external resources, or in any combination of these two modes.

R206 – Markup Association

The TT AF shall be capable of associating structural markup with intrinsic and extrinsic text content, where such markup may denote either or both semantic (functional) and presentational (formal) properties of the content.

Note:

In this context, presentational properties designate both stylistic and timing related presentation information.

R207 – Conditional Content

The TT AF shall be capable of expressing conditional content, where each alternative content choice is governed by one or more test expressions such that exactly one or zero content choice is selected when evaluating each choice in a predefined order.

R208 – Flowed Text

The TT AF shall be capable of expressing authorial intention to flow (layout) text content in an idealized, but unspecified user agent.

Note:

In this context, the concept of flow refers to an implied process by means of which textual information expressed in the character domain is mapped to a positioned glyph codomain.

Note:

It is not required that an idealized user agent or behavior of such a user agent be defined by the TT AF specification(s); however, the definition of such a user agent or user agent behavior may be the subject of future activities by the TT WG.

R209 – Logical Flowed Text Vocabulary

The TT AF shall be capable of expressing the following vocabulary as pertains to logical flowed text content:

• body

• division

• paragraph

• phrase

Note:

A generic role attribute is expected to be provided for use with these items in order to express distinct stylistic and semantic roles, e.g., a paragraph role may be associated with a division item to indicate that the logical division expressed corresponds to a stylistic and semantic paragraph.

Note:

One possible mapping for this vocabulary is to xhtml:body, xhtml:div, xhtml:p, and xhtml:span, respectively, as defined by [XHTML 1.0].

R210 – Presentational Flowed Text Vocabulary

The TT AF shall be capable of expressing the following vocabulary as pertains to presentational flowed text content:

• block

• block container

• character

• flow

• forced line break

• inline

• inline container

• region

• viewport

Note:

The items enumerated above are drawn in part from similarly named items defined by [XSL 1.0], Section 6, Formatting Objects.

Note:

The forced line break item may be expressed in terms of the character item, where the specified character has line separator or line break semantics.

The use of this item in combination with a wrap option style parameter with the value nowrap permits an intermediate mode of text representation with formatting semantics that fall between flowed text and non-flowed text.

Note:

The viewport and region items are intended to be analogous to the [XSL 1.0] vocabulary fo:simple-page-master and fo:region-*, respectively.

R211 – Flowed Text Vocabulary Relationship

The TT AF shall be defined in such a manner that a default relationship between logical and presentational flowed text vocabulary may be assumed as follows:

• body ⇔ flow

• division (display: block) ⇔ block container

• paragraph ⇔ block

• division (display: inline) ⇔ inline container

• phrase ⇔ inline

Note:

Parsed character data (#PCDATA) that appears in logical flowed text content should be assumed to map by default to parsed character data or character in presentational flowed text content.

R212 – Flowed Text Vocabulary Separation

The TT AF shall be defined in such a manner that use of logical flowed text vocabulary is separated from use of presentational flowed text vocabulary.

Note:

It is not required that the TT AF specification(s) define a document type that supports the simultaneous use of both logical and presentational flowed text vocabulary.

R213 – Non-Flowed Text

The TT AF shall be capable of expressing authorial intention to render non-flowed text content in an idealized, but unspecified user agent.

Note:

In this context, the concept of non-flowed text refers to textual information that is explicitly associated with positioned glyph information at authoring time; i.e., all bidirectional processing and character to glyph substitution processing and glyph position assignment has already occurred.

Note:

It is not required that an idealized user agent or behavior of such a user agent be defined by the TT AF specification(s); however, the definition of such a user agent or user agent behavior may be the subject of future activities by the TT WG.

R214 – Non-Flowed Text Vocabulary

The TT AF shall be capable of expressing the following vocabulary as pertains to non-flowed text content:

• area

• glyph

• glyph sequence

Note:

The glyph and glyph sequence vocabulary items are intended to make direct reference to specific glyphs in specific fonts, having already been transformed from the character domain to the glyph domain. References to glyphs would typically take the form of a glyph identifier or a glyph code. For further information on character to the glyph mapping process, see [CharMod], Section 3.1.3, Units of Visual Rendering.

R215 – Hybrid Flowed and Non-Flowed Text

The TT AF shall be capable of expressing authorial intention to create a hybrid of flowed and non-flowed text content; however, such an expression may require that these two types of content be segregated at a specific level of granularity.

R216 – Hyperlinking

The TT AF shall be capable of expressing simple hyperlinks, where the ending (destination) resource is either the starting (source) resource or an external resource.

R217 – Embedded Graphics

The TT AF shall be capable of expressing block and inline, embedded graphics of both bitmap and vector or outline formats.

If block or inline graphics are used for the purpose of expressing pre-rasterized text in a bitmap format, then the original text that produced such a pre-rasterized form shall also be required to be present in TT AF content.

R218 – Non-Embedded Graphics

The TT AF shall be capable of expressing block and inline, non-embedded graphics of both bitmap and vector or outline formats, where a graphic is represented by an external resource.

If block or inline graphics are used for the purpose of expressing pre-rasterized text in a bitmap format, then the original text that produced such a pre-rasterized form shall also be required to be present in TT AF content.

R219 – Embedded Fonts

The TT AF shall be capable of expressing embedded fonts of both bitmap and outline formats.

R220 – Non-Embedded Fonts

The TT AF shall be capable of expressing non-embedded fonts of both bitmap and outline formats, where a font is represented by an external resource.

R221 – Descriptive Vocabulary

The TT AF shall be capable of associating text content with descriptive information from an appropriate domain of discourse.

Note:

See [TEI], Chapter 10, Base Tag Set for Drama, and Chapter 11, Transcription of Speech, for examples of descriptive vocabulary pertaining to drama and the transcription of speech, respectively.

R222 – Embedded Audio

The TT AF shall be capable of expressing embedded audio that is intended to be played or rendered in sequence or in parallel with associated text content.

Note:

The use of embedded audio is intended to support aural cues that may accompany text.

R223 – Non-Embedded Audio

The TT AF shall be capable of expressing non-embedded audio that is intended to be played or rendered in sequence or in parallel with associated text content, where the audio content is represented by an external resource.

Note:

The use of non-embedded audio is intended to support aural cues that may accompany text.

R290 – Markup Format

The TT AF shall support the use of both [XML 1.0] and [XML 1.1] as serialized forms of a TT AF XML information set.

Note:

See [XML InfoSet] for further information on an XML information set.

R291 – Markup Format and Unicode Interaction

The TT AF shall require or recommend adherence to the practices recommended by [Unicode in XML].

R292 – Extrinsic Resource References

The TT AF shall support the use of [XLink] for the purpose of referencing external resources.

R293 – Schema Validity Specification

The TT AF specification(s) shall be defined in such a manner that the normative validity of markup content be specified in terms of either or both [RELAX NG] and [XML Schema Part 1] in combination with [XML Schema Part 2].

4.3 Styling

R300 – Inline Styling

The TT AF shall be capable of inline styling, where inline styling means the inclusion of stylistic presentation information in a TT AF document instance.

R301 – Inline Styling Form

The TT AF shall be capable of specifying inline styling by means of (1) distinct attributes, (2) a generic attribute, e.g., style, and (3) one or more inline stylesheets.

R302 – Out-of-Line Styling

The TT AF shall be capable of out-of-line styling, where out-of-line styling means the association of stylistic presentation information with TT AF content via some mechanism external to a TT AF document instance.

R303 – Out-Of-Line Styling Form

The TT AF shall be capable of specifying out-of-line styling by means of one or more external stylesheets.

R304 – Styling Prioritization

The TT AF shall be capable of associating priorities with stylistic presentation information in order to permit the resolution of multiple style specifications that apply to the same content.

R305 – Style Parameters – Aural

The TT AF shall be capable of associating the following aural style parameters with timed text content:

• azimuth

• cue before, during, after

• elevation

• pause before, after

• pitch

• pitch range

• richness

• speaking mode

• speech rate

• stress

• voice family

• volume

Note:

For further information on these style parameters, see [XSL 1.0], Section 7.6, Common Aural Properties and [CSS Level 2].

Note:

For further information on the speaking mode aural style parameter, see discussion of speak, speak-numeral, and speak-punctuation properties described in [XSL 1.0], Section 7.6.

R306 – Style Parameters – Visual

The TT AF shall be capable of associating the following visual style parameters with timed text content:

• absolute position

• background color

• baseline alignment point

• baseline alignment

• baseline dominance

• baseline shift

• bidirectional treatment

• block progression dimension

• block scroll amount

• border before, after, start, end

• break before, after

• clear

• color

• color profile name

• display none, block, inline

• display alignment

• float

• font family

• font size

• font style

• font weight

• height

• indent start, end

• inline progression dimension

• inline scroll amount

• intrusion-displace

• line feed treatment

• line height

• line stacking strategy

• line wrapping option

• opacity

• origin (top, left)

• overflow

• padding before, after, start, end

• reference orientation

• relative position

• space before, after, start, end

• text alignment

• text altitude (ascent)

• text decoration

• text depth (descent)

• text indent (first line)

• text shadow

• visibility

• white space collapse

• white space treatment

• width

• writing mode

• z-index

Note:

For further information on these style parameters, see [XSL 1.0], Section 7, Formatting Properties and [CSS Level 2].

Note:

The style parameters break before and break after are intended to model the like named style properties defined by [XSL 1.0], Section 7.19, Keeps and Breaks Properties, except that the possible values of these parameters are expected to be restricted to auto, line, and inherit. The new value line is intended to denote a line area context in order to express that a line break should precede or follow the element associated with this parameter.

Unlike XSL, these parameters apply only to inline vocabulary, and not block vocabulary.

Note:

A style parameter is intended to convey the notion of any type of style specification or declaration whether it is expressed as an attribute or a property.

R307 – Style Parameters – Visual – Temporal Fill Mode

The TT AF may be capable of associating the following visual style parameters that pertain to temporal filling (flowing) of timed text content:

• temporal fill mode

• temporal fill direction

• temporal block clear mode

• temporal fill interval

• temporal inter-fill interval

Note:

These parameters are intended to permit the expression of authorial intent with respect to the temporal filling (stacking) of glyph areas within a line area and line areas within a block area.

The temporal fill mode parameter permits specifying the granularity of temporal filling, e.g., line, word, character. The temporal fill direction parameter permits specifying the direction of fill (stacking) independently from the writing mode. The temporal block clear mode parameter permits specifying whether the containing block is cleared or is automatically scrolled, and by what extent, when the block is filled. The temporal fill interval parameter permits specifying the interval that a filled area should remain static before processing the next fill. The temporal inter-fill interval parameter permits specifying the interval between the end of a prior fill interval and the start of a subsequent fill interval.

R390 – Style Parameter Symmetry

The TT AF shall be defined in such a manner that if a stylistic presentation parameter may be specified as a style property, then that parameter shall also be specifiable as a style attribute, and vice-versa.

Note:

In this context, a style attribute refers to an attribute expressed in a markup language (e.g., an XML attribute), while a style property refers to a property expressed in a style language (e.g., a CSS property).

R391 – Style Parameter Definitions

The TT AF shall be defined in such a manner that if there is a conflict when adopting the name or value semantics of a style parameter specification, then the following order shall hold for resolving such a conflict:

1. XSL FO

2. SVG

3. SMIL

4. CSS Level 2

5. CSS Level 3

R392 – Style Element Type Shorthand Equivalence

The TT AF shall be defined in such a manner that to the extent that stylistic oriented markup element types are defined or adopted, then such element types shall be defined as shorthand equivalents of non-stylistic oriented element types in combination with specific style parameters.

4.4 Timing

R401 – Inline Timing

The TT AF shall be capable of inline timing, where inline timing means the inclusion of temporal presentation markup in a TT AF document instance.

R402 – Out-of-Line Timing

The TT AF shall be capable of out-of-line timing, where out-of-line timing means the association of temporal presentation information with TT AF content via some mechanism external to a TT AF document instance.

R403 – Synchronization Parameters

The TT AF shall be capable of expressing the following synchronization parameters in terms of any legal combination that expresses a possibly repeated or restarted, single, simple interval:

• begin

• dur

• end

• endsync

• fill

• fillDefault

• repeatCount

• repeatDur

• restart

• restartDefault

Note:

For further information on these synchronization parameters, see [SMIL 2.0], Section 10, The SMIL 2.0 Timing and Synchronization Module.

Note:

It is not required that the TT AF support the specification of multiple simple intervals, i.e., multiple start, duration, or begin values.

R404 – Synchronization Parameter Value Space Semantics

The TT AF shall be capable of expressing the following synchronization parameter value space semantics:

• Offset Values – a clock offset from an implied or explicit synchronization timebase;

• Event Values – a clock offset from a named event associated with an implied or explicit element node, including, at a minimum, named events that indicate the beginning or end of a timed element's active interval;

• Access Key Values – a clock offset from a specific key press event;

• Media Marker Values – a clock offset from a media marker, including, at a minimum, a media marker that denotes a SMPTE time code;

• Wallclock Values – a clock offset from an absolute wallclock time in an implied or explicit time zone.

Note:

It is not required that the TT AF support the specification of negative offset values.

R405 – Time Containment Semantics

The TT AF shall be capable of expressing sequential, parallel, and exclusive time containment semantics of consituent timed text content.

Note:

For further information on these time containment semantics, see [SMIL 2.0], Section 10, The SMIL 2.0 Timing and Syncrhonization Module.

4.5 Animation

R500 – Animation Modes

The TT AF shall be capable of expressing animation according to the following modes:

• continuous – linear

• continuous – non-linear

• discrete

Note:

By animation is meant the ability to alter some parameter or value over time.

R501 – Scroll Animation

The TT AF shall be capable of expressing animated scrolling of content, both in block and inline progression directions, with independent expression of scroll in, scroll out, and scroll repetition.

R502 – Highlight Animation

The TT AF shall be capable of expressing animated highlighting of content, with granularity at the level of individual characters or glyphs.

R503 – Fade Transition Animation

The TT AF shall be capable of expressing animated fade transitions of content, with granularity at the level of individual regions or areas.

Note:

See Presentational Flowed Text Vocabulary and Non-Flowed Text Vocabulary for information on region and area vocabulary items, respectively.

R504 – Animated Style Parameters – Aural

The TT AF shall be capable of animating the following aural style parameters:

• azimuth

• elevation

• speaking mode

• speech rate

• volume

R505 – Animated Style Parameters – Visual

The TT AF shall be capable of animating the following visual style parameters:

• background color

• block scroll amount

• border color

• color

• display

• inline scroll amount

• opacity

• origin

• visibility

Note:

It is possible to express fade-in and fade-out transitions by means of animating the opacity style parameter.

Note:

Animation of the display style parameter may cause reflow (reformatting) of content in a user agent at each animation step.

N506 – Animated Content

The TT AF is not required to support the animation of content.

Note:

By animation of content is meant the dynamic addition or changing of element content by means of animation functionality, e.g., replacing the text content of an element.

4.6 Metadata

R600 – Metadata Item Association

The TT AF shall be capable of associating arbitrary metadata, expressed as metadata items, with (1) a TT AF document instance and (2) any element contained within a TT AF document instance.

Note:

It is not required that metadata be able to be associated with an element's attributes or with any other child of an element other than a child that is characterized as an element itself.

R601 – Metadata Item Constituents

The TT AF shall be capable of expressing the following constituents of individual metadata items:

• name

• value type

• value

R602 – Metadata Item Value Representation

The TT AF shall give preference to the representation of metadata item values as element content as opposed to attribute content.

Note:

By element content is meant those children of an element information item that are characterized as elements or as character data. By attribute content is meant the normalized values of the attributes of an element information item.

R603 – Metadata Item Extensibility

The TT AF shall be capable of expressing metadata items whose names, value types, and semantics are defined externally to the TT AF specification(s).

R604 – Metadata Item Validation

The TT AF specification(s) shall be defined in such a manner as to permit and potentially require the ability to validate metadata.

R690 – Dublin Core Preference

The TT AF specification(s) shall be defined in such a manner as to give preference to those metadata items defined by [DCMES 1.1] in case that a conflict exists with another candidate metadata representation.