Distribution - OPEN

Owner: Nigel Megitt.
Started: 16/01/14

Issues Addressed

ISSUE-270 - OPEN
ISSUE-288 - OPEN
ISSUE-358 - RAISED

Summary

There are two scenarios in which TTML samples, i.e. short snippets bounded in time, are needed, which collectively relate to a single entity.

The first is for live authoring, in which an authoring station may need to create and emit a series of short documents for display with low latency. The second is for distribution, in which an encoder or packager may need to divide a longer document into a series of short documents for managed data rate distribution to an onward chain. An example of this would include use in MPEG DASH.

In the live case there is a follow-on use case, which is to permit an archival device to accumulate a set of TTML documents together to form a larger one, i.e. one whose temporal extent is the union of the temporal extents of the input documents.

In the distribution use case there is also a follow-on use case, which is to provide direction to presentation processors to enable them to display the series of documents with no visible join between them, i.e. as though they were received as a single document. There may be a need in this case to provide some guidance, either in the specification or in the contents of the provided documents, to minimise the implementation cost (at design time and at run time) associated with this use case.

At first glance TTML can already handle these scenarios by simply creating small documents. However there is no support for chained related documents. For example the use of xml:id expresses a uniqueness requirement that is per-document not per-set-of-documents.

This problem may be generalised where documents need to be combined for other purposes, for example rather than temporal sampling there could be spatial sampling, perhaps where content in different documents relates to different regions.

Additionally there is a use case for providers that wish to combine translation subtitles with hard of hearing captions, where the two entities may have been authored separately. In this case the TTML samples are not bounded in time but in intent.

A starter document for this (including errors) was posted to the reflector at 1.

Simplified use cases

In a live authoring context, a set of TTML documents needs to be combined efficiently into a single larger TTML document using an ‘accumulating’ processor.
In a distribution context, a TTML document needs to be segmented efficiently into a set of shorter TTML documents.
A distribution processor (i.e. a piece of consumer equipment) needs to be able to process a sequence of temporally ordered TTML documents that may overlap temporally with minimal processing required between documents.
In an advanced authoring context, a set of translation subtitles needs to be combined with a set of hard of hearing captions to form a single TTML document.

Proposal

Modify the scope of uniqueness of element identifiers to allow groups of documents to have elements that are identified as being the same, and to specify the rules for identity that apply.

This is at the syntactic level, but taking advantage of knowledge of the structure of TTML, i.e. it’s not something that could just be punted over to XML (which doesn’t deal with this scenario anyway).

Specify rules for documents to be combinable using a merge_documents process
Specify constraints on the merge_documents process and an algorithm for it

This proposal would allow documents from the same group to be combined. Segmenting documents is straightforward sub-selection from the originating document. Distribution processors can assume that any element in the same group with the same identifier, in the <head>, is the same, and potentially shortcut pre-processing of regions, styles etc.

Design considerations

Maintaining temporal equivalence

The computation of times must be taken into account. We assume that if two documents overlap temporally then the computed begin and end times for any content element need to be identical after those documents have been merged. Concepts such as nested time containers and par and seq semantics therefore may result in begin, end and dur attributes being modified as a result of the merge process.

Maintaining document order

The presentation of a TTML document is dependent on the document order of its contents. Therefore any merge process must have defined rules for maintaining document order such that the order of content from any given source document is maintained in the merged output document. Note that the merge process may result in some content being interleaved, within this constraint, so adjacency cannot also be maintained, in general.

Syntax of group identifier

The group identifier does not affect general processing of TTML documents, and indeed has no effect at all on processing of a single document. It is therefore considered to be metadata.

Metadata attribute ttm:documentGroup
Considered significant on the tt:tt element only
Optional
Type: NCName from [XML Schema Part 2]

ttm:documentGroup : <xsd:NCName>

Rules for combining documents in combinable groups

If all documents in a merge set conform to the following set of rules then the merge_documents process will generate a valid merged output document containing all of the contents of the input documents with equivalent begin and end times and preserved xml:ids and preserved document order.

The input documents SHALL be valid TTML documents.
Elements with the equivalent parent where that parent is a seq timeContainer SHALL NOT overlap temporally.
Elements that are intended to match across input documents SHALL be locally equal, where the test for local equality is defined below.
Elements that are permitted to occur at most once as a child of the same parent SHALL be locally equal within all of the input documents in which they are present.
Elements with the same xml:id SHALL have the same type, the same ancestry and be locally equal amongst all of the input documents in which they are present.
Any pair of sibling elements with xml:id attributes SHALL occur in the same relative document order in all of the input documents in which they are present.

Local equality test

The test for local equality is that the normalised attribute sets of the elements being compared are identical and the content excluding child elements is identical.

The normalised attribute set is the attribute set with the begin, end and dur attributes normalised into a synthesised continuous time base. This SHALL preserve begin and end markers when ttp:timeBase="smpte" and ttp:markerMode="discontinuous".

merge_documents process

The merge_documents process combines the input documents and generates a valid output document that contains all of the elements from each input document, maintaining document ordering and equivalent content element timing. This is a breadth-first traversal of the document tree, copying matched nodes once and merging their child elements recursively.

The merge_documents process is not commutative: that is, merge_documents(A, B) produces output that is in general different to merge_documents(B, A). This is because the process creates output whose content elements' document order is prioritised based on the order of input documents provided to the process. For example:

document A:

...
<body>
<div>
<p>First paragraph</p>
</div>
</body>
...

document B:

...
<body>
<div>
<p>Second paragraph</p>
</div>
</body>
...

merge_documents(A,B):

...
<body>
<div>
<p>First paragraph</p>
<p>Second paragraph</p>
<p>
</div>
</body>
...

merge_documents(B,A):

...
<body>
<div>
<p>Second paragraph</p>
<p>First paragraph</p>
</div>
</body>
...

However the merge_documents process is associative: merge_documents(A, merge_documents(B, C) ) is equivalent to merge_documents( merge_documents(A, B), C).

Constraints on the merge_documents process

Any elements in the merge output document SHALL have exactly the same ancestry as their equivalents in the source documents.
The document order of sibling elements in a source document SHALL be preserved in the merge output document. Note that there is no requirement that adjacent sibling elements remain adjacent, merely that an element A that occurs before its sibling element B in a source document will occur in the same relative order in the merge output document.
- Since it is possible for non-identified sibling elements to be identical, this constraint applies primarily to elements with xml:id attributes and secondarily to those that are not identified.
- If identified and non-identified sibling elements are interleaved with each other in the same input document, the identified elements act as hard ordering delimiters - see note below.
- Character data sections that are interleaved with child elements may be considered to be equivalent to anonymous spans with no attributes. The output shall omit those spans.
The merge process SHALL insert unmatched sibling elements into the output document in the order of provided input documents. See the example above.
Each element in each source document SHALL have exactly one equivalent element in the destination document.
Elements that are locally equal in the input documents (also known as matched elements) SHALL be copied into the merge output document once and the child elements from each input document merged in to this element as the parent.
The local equality comparison of timed elements SHALL use normalised values of the begin and end attributes. The normalisation process SHALL preserve begin and end attribute values used as markers when ttp:timeBase="smpte" and ttp:markerMode="discontinuous".
In order to maintain equivalent presentation time values the merge_documents process SHALL modify the begin, end and dur attributes if necessary for elements whose parent is a seq time container. The merge_documents process MAY express equivalent end times using either end or dur attributes but SHALL NOT use apply both end and dur attributes to the same element. When ttp:timeBase="smpte" and ttp:markerMode="discontinuous" the begin and end attribute values SHALL NOT be modified.
The merge process MAY normalise white space within documents to ensure that comparisons are equivalent. NOTE xml:space is an attribute that is included in the equality tests for all elements.
The merge process MAY use a deep equality test to establish equivalence across input documents of elements and their descendants to full depth using normalised times. If this test is true then the whole sub-tree may be copied once into the output document.
If the merge_documents process detects that the output will not be a valid document then it SHALL terminate with an error. The merge process SHALL check maximum cardinality of elements within their parents. For example if the merge process would result in more than one layout element as a child of the tt:head element then this will cause a terminating error. NOTE The elements whose maximum cardinality is 1 are tt:tt, tt:head, tt:body, layout, styling and animation.
The result of providing identical input documents to the merge process SHALL be equivalent but is not required to be identical to that input document.

Using id as a hard ordering delimiter in the case of interleaved identified and non-identified siblings

Consider the following example fragments:

Fragment C:

<div xml:id="d1">
<p>Fire!</p>
<p>Fire!</p>
<p xml:id="p1">What did you say?</p>
</div>

Fragment D:

<div xml:id="d1">
<p>Fire!</p>
<p xml:id="p1">What did you say?</p>
<p>Fire!</p>
</div>

In this example the identified p acts as a hard delimiter for deducing document ordering. This means that the merge_documents process must maintain the positions of "Fire!" paragraphs relative to the delimiter. The correct merged output for these fragments is:

<div xml:id="d1">
<p>Fire!</p>
<p>Fire!</p>
<p xml:id="p1">What did you say?</p>
<p>Fire!</p>
</div>

In the absence of this constraint a merge process might consider the two otherwise identical "Fire!" siblings as candidates for being matched as the same element, though maintaining the document ordering constraint here would be impossible.

Example - adding a line break to a span

Fragment E:

<div>
   <p>
      <span>This text would occupy two lines</span>
   </p>
</div>

Fragment F:

<div>
   <p>
      <span>This text would occupy
      <br>two lines</span>
   </p>
</div>

In this example the simple addition of a   interspersed with character data should result in the merged output being equivalent to Fragment F.

The character data segments in the  may for this purpose be merged as though they are two anonymous spans as follows:

Compare the character data by concatenating it:
- Local equality test: Is "This text would occupy two lines" equivalent to "This text would occupy" + " two lines"? TRUE
Merge  elements
- Content ordering: split into character data fragments wrapped in anonymous spans:
 - This text would occupy
 - two lines
- Merge with the   element maintaining ordering
- Remove anonymous span wrappers.

Algorithm for combining 2 documents with same group identifier

TODO: rewrite algorithm

Edits to be applied

New attribute on tt:tt groupIdentifier to identify a group - no need for document uniqueness since it's unique for a group of documents, so just an id e.g. an NCName.
New normative section expressing rules for combining documents in a group with the same identifier, as per rules, constraints and algorithm agreed (see proposal above).
?? Do we need a section on splitting documents into sub-documents? Not sure why this would be needed. If so, normative or informative?

Edits applied

Impact

These changes would not affect the structure or implementation of any single TTML2 document.
The only change is the addition of a single optional group identifier to the document which does not normatively affect presentation of that document.
This change helps to meet a currently unmet need for users who create a stream of small TTML documents and wish to combine them using normative rules.
This change would also be more generally applicable for other combinatorial requirements, for example combining subtitles or captions authored in different regions each in a separate document into a single document.

References

[1 Email including attached draft EBU document on segmentation]