TTML/changeProposal025
Distribution - OPEN
- Owner: Nigel Megitt.
- Started: 16/01/14
Issues Addressed
Summary
There are two scenarios in which TTML samples, i.e. short snippets bounded in time, are needed, which collectively relate to a single entity.
The first is for live authoring, in which an authoring station may need to create and emit a series of short documents for display with low latency. The second is for distribution, in which an encoder or packager may need to divide a longer document into a series of short documents for managed data rate distribution to an onward chain. An example of this would include use in MPEG DASH.
In the live case there is a follow-on use case, which is to permit an archival device to accumulate a set of TTML documents together to form a larger one, i.e. one whose temporal extent is the union of the temporal extents of the input documents.
In the distribution use case there is also a follow-on use case, which is to provide direction to presentation processors to enable them to display the series of documents with no visible join between them, i.e. as though they were received as a single document. There may be a need in this case to provide some guidance, either in the specification or in the contents of the provided documents, to minimise the implementation cost (at design time and at run time) associated with this use case.
At first glance TTML can already handle these scenarios by simply creating small documents. However there is no support for chained related documents. For example the use of xml:id expresses a uniqueness requirement that is per-document not per-set-of-documents.
This problem may be generalised where documents need to be combined for other purposes, for example rather than temporal sampling there could be spatial sampling, perhaps where content in different documents relates to different regions.
Additionally there is a use case for providers that wish to combine translation subtitles with hard of hearing captions, where the two entities may have been authored separately. In this case the TTML samples are not bounded in time but in intent.
A starter document for this (including errors) was posted to the reflector at 1.
Simplified use cases
- In a live authoring context, a set of TTML documents needs to be combined efficiently into a single larger TTML document using an ‘accumulating’ processor.
- In a distribution context, a TTML document needs to be segmented efficiently into a set of shorter TTML documents.
- A distribution processor (i.e. a piece of consumer equipment) needs to be able to process a sequence of temporally ordered TTML documents that may overlap temporally with minimal processing required between documents.
- In an advanced authoring context, a set of translation subtitles needs to be combined with a set of hard of hearing captions to form a single TTML document.
Proposal
- Modify the scope of uniqueness of element identifiers to allow groups of documents to have elements that are identified as being the same, and to specify the rules for identity that apply.
This is at the syntactic level, but taking advantage of knowledge of the structure of TTML, i.e. it’s not something that could just be punted over to XML (which doesn’t deal with this scenario anyway).
- Specify rules for documents to be combinable using a merge_documents process
- Specify constraints on the merge_documents process and an algorithm for it
This proposal would allow documents from the same group to be combined. Segmenting documents is straightforward sub-selection from the originating document. Distribution processors can assume that any element in the same group with the same identifier, in the <head>, is the same, and potentially shortcut pre-processing of regions, styles etc.
Design considerations
Maintaining temporal equivalence
The computation of times must be taken into account. We assume that if two documents overlap temporally then the computed begin and end times for any content element need to be identical after those documents have been merged. Concepts such as nested time containers and par and seq semantics therefore may result in begin
, end
and dur
attributes being modified as a result of the merge process.
Maintaining document order
The presentation of a TTML document is dependent on the document order of its contents. Therefore any merge process must have defined rules for maintaining document order such that the order of content from any given source document is maintained in the merged output document. Note that the merge process may result in some content being interleaved, within this constraint, so adjacency cannot also be maintained, in general.
Syntax of group identifier
The group identifier does not affect general processing of TTML documents, and indeed has no effect at all on processing of a single document. It is therefore considered to be metadata.
- Metadata attribute
ttm:documentGroup
- Considered significant on the
tt:tt
element only - Optional
- Type: NCName from [XML Schema Part 2]
ttm:documentGroup : <xsd:NCName>
Rules for combining documents in combinable groups
If all documents in a merge set conform to the following set of rules then the merge_documents process will generate a valid merged output document containing all of the contents of the input documents with equivalent begin and end times and preserved xml:id
s and preserved document order.
- The input documents SHALL be valid TTML documents.
- Elements with the equivalent parent where that parent is a
seq
timeContainer SHALL NOT overlap temporally. - Elements that are intended to match across input documents SHALL be locally equal, where the test for local equality is defined below.
- Elements that are permitted to occur at most once as a child of the same parent SHALL be locally equal within all of the input documents in which they are present.
- Elements with the same
xml:id
SHALL have the same type, the same ancestry and be locally equal amongst all of the input documents in which they are present. - Any pair of sibling elements with
xml:id
attributes SHALL occur in the same relative document order in all of the input documents in which they are present.
Local equality test
The test for local equality is that the normalised attribute sets of the elements being compared are identical and the content excluding child elements is identical.
The normalised attribute set is the attribute set with the begin
, end
and dur
attributes normalised into a synthesised continuous time base. This SHALL preserve begin
and end
markers when ttp:timeBase="smpte"
and ttp:markerMode="discontinuous"
.
merge_documents process
The merge_documents process combines the input documents and generates a valid output document that contains all of the elements from each input document, maintaining document ordering and equivalent content element timing. This is a breadth-first traversal of the document tree, copying matched nodes once and merging their child elements recursively.
The merge_documents process is not commutative: that is, merge_documents(A, B) produces output that is in general different to merge_documents(B, A). This is because the process creates output whose content elements' document order is prioritised based on the order of input documents provided to the process. For example:
document A:
... <body> <div> <p>First paragraph</p> </div> </body> ...
document B:
... <body> <div> <p>Second paragraph</p> </div> </body> ...
merge_documents(A,B):
... <body> <div> <p>First paragraph</p> <p>Second paragraph</p> <p> </div> </body> ...
merge_documents(B,A):
... <body> <div> <p>Second paragraph</p> <p>First paragraph</p> </div> </body> ...
However the merge_documents process is associative: merge_documents(A, merge_documents(B, C) ) is equivalent to merge_documents( merge_documents(A, B), C).
Constraints on the merge_documents process
- Any elements in the merge output document SHALL have exactly the same ancestry as their equivalents in the source documents.
- The document order of sibling elements in a source document SHALL be preserved in the merge output document. Note that there is no requirement that adjacent sibling elements remain adjacent, merely that an element A that occurs before its sibling element B in a source document will occur in the same relative order in the merge output document.
- Since it is possible for non-identified sibling elements to be identical, this constraint applies primarily to elements with
xml:id
attributes and secondarily to those that are not identified. - If identified and non-identified sibling elements are interleaved with each other in the same input document, the identified elements act as hard ordering delimiters - see note below.
- Character data sections that are interleaved with child elements may be considered to be equivalent to anonymous spans with no attributes. The output shall omit those spans.
- Since it is possible for non-identified sibling elements to be identical, this constraint applies primarily to elements with
- The merge process SHALL insert unmatched sibling elements into the output document in the order of provided input documents. See the example above.
- Each element in each source document SHALL have exactly one equivalent element in the destination document.
- Elements that are locally equal in the input documents (also known as matched elements) SHALL be copied into the merge output document once and the child elements from each input document merged in to this element as the parent.
- The local equality comparison of timed elements SHALL use normalised values of the
begin
andend
attributes. The normalisation process SHALL preservebegin
andend
attribute values used as markers whenttp:timeBase="smpte"
andttp:markerMode="discontinuous"
. - In order to maintain equivalent presentation time values the merge_documents process SHALL modify the
begin
,end
anddur
attributes if necessary for elements whose parent is aseq
time container. The merge_documents process MAY express equivalent end times using eitherend
ordur
attributes but SHALL NOT use apply bothend
anddur
attributes to the same element. Whenttp:timeBase="smpte"
andttp:markerMode="discontinuous"
thebegin
andend
attribute values SHALL NOT be modified. - The merge process MAY normalise white space within documents to ensure that comparisons are equivalent. NOTE
xml:space
is an attribute that is included in the equality tests for all elements. - The merge process MAY use a deep equality test to establish equivalence across input documents of elements and their descendants to full depth using normalised times. If this test is true then the whole sub-tree may be copied once into the output document.
- If the merge_documents process detects that the output will not be a valid document then it SHALL terminate with an error. The merge process SHALL check maximum cardinality of elements within their parents. For example if the merge process would result in more than one layout element as a child of the
tt:head
element then this will cause a terminating error. NOTE The elements whose maximum cardinality is 1 arett:tt
,tt:head
,tt:body
,layout
,styling
andanimation
. - The result of providing identical input documents to the merge process SHALL be equivalent but is not required to be identical to that input document.
Using id as a hard ordering delimiter in the case of interleaved identified and non-identified siblings
Consider the following example fragments:
Fragment C:
<div xml:id="d1"> <p>Fire!</p> <p>Fire!</p> <p xml:id="p1">What did you say?</p> </div>
Fragment D:
<div xml:id="d1"> <p>Fire!</p> <p xml:id="p1">What did you say?</p> <p>Fire!</p> </div>
In this example the identified p
acts as a hard delimiter for deducing document ordering. This means that the merge_documents process must maintain the positions of "Fire!" paragraphs relative to the delimiter. The correct merged output for these fragments is:
<div xml:id="d1"> <p>Fire!</p> <p>Fire!</p> <p xml:id="p1">What did you say?</p> <p>Fire!</p> </div>
In the absence of this constraint a merge process might consider the two otherwise identical "Fire!" siblings as candidates for being matched as the same element, though maintaining the document ordering constraint here would be impossible.
Example - adding a line break to a span
Fragment E:
<div> <p> <span>This text would occupy two lines</span> </p> </div>
Fragment F:
<div> <p> <span>This text would occupy <br>two lines</span> </p> </div>
In this example the simple addition of a <br>
interspersed with character data should result in the merged output being equivalent to Fragment F.
The character data segments in the <span>
may for this purpose be merged as though they are two anonymous spans as follows:
- Compare the character data by concatenating it:
- Local equality test: Is "This text would occupy two lines" equivalent to "This text would occupy" + " two lines"? TRUE
- Merge
<span>
elements- Content ordering: split into character data fragments wrapped in anonymous spans:
- <span>This text would occupy</span>
- <span> two lines</span>
- Merge with the
<br>
element maintaining ordering - Remove anonymous span wrappers.
- Content ordering: split into character data fragments wrapped in anonymous spans:
Algorithm for combining 2 documents with same group identifier
- TODO
- rewrite algorithm
Edits to be applied
- New attribute on
tt:tt
groupIdentifier
to identify a group - no need for document uniqueness since it's unique for a group of documents, so just an id e.g. an NCName. - New normative section expressing rules for combining documents in a group with the same identifier, as per rules, constraints and algorithm agreed (see proposal above).
- ?? Do we need a section on splitting documents into sub-documents? Not sure why this would be needed. If so, normative or informative?
Edits applied
Impact
- These changes would not affect the structure or implementation of any single TTML2 document.
- The only change is the addition of a single optional group identifier to the document which does not normatively affect presentation of that document.
- This change helps to meet a currently unmet need for users who create a stream of small TTML documents and wish to combine them using normative rules.
- This change would also be more generally applicable for other combinatorial requirements, for example combining subtitles or captions authored in different regions each in a separate document into a single document.
References
[1 Email including attached draft EBU document on segmentation]