View source for TimedText/IMSC-HRM-1ED-Implementation-Report

[[Category:TTML]]
[[Category:TimedText]]
[[Category:ImplementationReport]]
== Introduction ==

This is the implementation report for the [https://www.w3.org/TR/imsc-hrm IMSC Hypothetical Render Model] Recommendation. It demonstrates [https://www.w3.org/2021/Process-20211102/#transition-cr adequate implementation experience] and that the CR exit criteria have been met by subjecting a ''validating implementation'' to a test suite and 4 independent ''content producing implementations''.

== Validating implementations ==

The ''validating implementations'' are:

* [https://github.com/sandflow/imscHRM imscHRM], which is an open-source implementation of the Recommendation in pure Python.
* [https://github.com/bbc/ebu-tt-live-toolkit/pull/66 bbc/ebu-tt-live-toolkit#66], which is an open-source implementation of the Recommendation in Python, built as part of the BBC fork of the EBU-TT Live Interoperability Toolkit

[[Category:TTML]]
[[Category:TimedText]]
[[Category:IMSC]]

{{#css:
div.panel { padding:20px; border-radius:5px; }
p { margin-bottom: 1em; }

h1 { font-size: 2em; line-height: 1.25em; font-family: 'Helvetica Neue', Roboto, Calibri, 'Segoe UI', helvetica, sans-serif; }
h2 { margin-top: 3em; }
h3 { margin-top: 1.5em; }
#toc { width: 97%; margin-left: 0px; margin-right: 10px; margin-bottom: 15px;}
#firstHeading { display:none; }
body {
    font-family: 'Helvetica Neue', 'Segoe UI', sans-serif;
    font-size: 1.25em;
    font-size: 1.25rem;
    font-weight: 300;
    line-height: 1.7;
    -webkit-text-size-adjust: 100%;
    -moz-text-size-adjust: 100%;
    -ms-text-size-adjust: 100%;
    text-size-adjust: 100%;
	color: #000;
}
div#content { margin-left: 9em; }
.tocsection-1 > a { display: none; }
}}

<div style="margin: 0 0 15px 15px; padding: 10px; float: right; width: 40%;">
__TOC__

</div>

== Test Suite ==

The [https://github.com/w3c/imsc-hrm-tests test suite] consists of hand-crafted IMSC documents intended exercise the boundaries of the HRM algorithm. They are organized into two sets, each in its own directory:

* each IMSC document in the <code>pass</code> directory conforms to the Hypothetical Render Model
* each IMSC document in the <code>fail</code> directory does not conform to the Hypothetical Render Model

Conformance is defined at https://www.w3.org/TR/imsc-hrm/#conformance.

Each IMSC document contains a short description of the IMSC HRM feature being tested.

All IMSC documents conform to the [https://www.w3.org/TR/ttml-imsc1.2/#text-profile-constraints IMSC Text Profile]. They also conform to the [https://tech.ebu.ch/files/live/sites/tech/files/shared/tech/tech3380v1_0_1.pdf EBU-TT-D specification], unless the feature being tested cannot be exercised by a valid EBU-TT-D document, in which case the non-conformance is also identified within the test content itself.

The EBU-TT Live Toolkit implementation cannot process documents that do not conform to EBU-TT-D and therefore the tests that exercise features prohibited in EBU-TT-D are not applicable, and are marked as "n/a".

Note that those "n/a" tests are allowed for within the specification: they refer to style attributes that are included within the definition of a [https://www.w3.org/TR/imsc-hrm/#dfn-glyph glyph] tuple, which, in the case that they are not permitted by a profile, are a) the same across all glyphs and b) if it is necessary to assign a value, have whatever computed value TTML2 specifies.

The following records the result of subjecting the ''validating implementations'' to the test suite:

{| class="wikitable"
|-
! Test !! imscHRM result !! EBU-TT Live Toolkit result
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/fail/dur001-fail.ttml dur001-fail.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/fail/dur002-fail.ttml dur002-fail.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/fail/dur003-fail.ttml dur003-fail.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/fail/dur004-fail.ttml dur004-fail.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/fail/dur005-fail.ttml dur005-fail.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/fail/dur006-fail.ttml dur006-fail.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/fail/dur007-fail.ttml dur007-fail.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/fail/dur008-fail.ttml dur008-fail.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/fail/dur009-fail.ttml dur009-fail.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/fail/dur010-fail.ttml dur010-fail.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/fail/dur011-fail.ttml dur011-fail.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/fail/dur012-fail.ttml dur012-fail.ttml] || pass || n/a
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/fail/dur013-fail.ttml dur013-fail.ttml] || pass || n/a
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/fail/dur014-fail.ttml dur014-fail.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/fail/ipd001-fail.ttml ipd001-fail.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/fail/ipd002-fail.ttml ipd002-fail.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/fail/ipd003-fail.ttml ipd003-fail.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/fail/ipd004-fail.ttml ipd004-fail.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/fail/ipd005-fail.ttml ipd005-fail.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/fail/ipd006-fail.ttml ipd006-fail.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/fail/ngbs001-fail.ttml ngbs001-fail.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/fail/ngbs002-fail.ttml ngbs002-fail.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/fail/region001-fail.ttml region001-fail.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/pass/dur001-pass.ttml dur001-pass.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/pass/dur002-pass.ttml dur002-pass.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/pass/dur003-pass.ttml dur003-pass.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/pass/dur004-pass.ttml dur004-pass.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/pass/dur005-pass.ttml dur005-pass.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/pass/dur006-pass.ttml dur006-pass.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/pass/dur007-pass.ttml dur007-pass.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/pass/dur008-pass.ttml dur008-pass.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/pass/dur009-pass.ttml dur009-pass.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/pass/dur010-pass.ttml dur010-pass.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/pass/dur011-pass.ttml dur011-pass.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/pass/dur012-pass.ttml dur012-pass.ttml] || pass || n/a
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/pass/dur013-pass.ttml dur013-pass.ttml] || pass || n/a
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/pass/dur014-pass.ttml dur014-pass.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/pass/dur015-pass.ttml dur015-pass.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/pass/dur016-pass.ttml dur016-pass.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/pass/ipd001-pass.ttml ipd001-pass.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/pass/ipd002-pass.ttml ipd002-pass.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/pass/ipd003-pass.ttml ipd003-pass.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/pass/ipd004-pass.ttml ipd004-pass.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/pass/ipd005-pass.ttml ipd005-pass.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/pass/ipd006-pass.ttml ipd006-pass.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/pass/ngbs001-pass.ttml ngbs001-pass.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/pass/ngbs002-pass.ttml ngbs002-pass.ttml] || pass || pass
|-
| [https://github.com/w3c/imsc-hrm-tests/blob/main/pass/region001-pass.ttml region001-pass.ttml] || pass || pass
|}

== Content producing implementations ==

* The results of subjecting the imscHRM ''validating implementation'' to IMSC documents generating using two independent ''content producing implementations'' was reported at https://lists.w3.org/Archives/Public/public-tt/2023Oct/0007.html. One of the ''content producing implementations'' was expected to produce IMSC documents that conform to the HRM whereas the other was expected to produce IMSC documents that did not conform to the HRM. The imscHRM ''validating implementation'' returned the expected result in both cases.

* The results of subjecting the imscHRM ''validating implementation'' to ~3,500 IMSC documents generated using an independent ''content producing implementation'' were reported by Cyril Concolato (Netflix) via email to the TTWG chair. The set of documents covered 40+ different languages and different kinds of timed text (SDH, SUBS, or forced narratives). While all documents were initially expected to conform to the HRM, the imscHRM ''validating implementation'' reported that ~10 of them did not conform to the HRM. It was ultimately determined that these ~10 documents were constructed to specifically address a limitation of the ecosystem targeted by the ''content producing implementation'' and were not generally valid IMSC documents. 2 of the documents also identified errors in the imscHRM ''validating implementation', which were subsequently fixed. 

* The results of subjecting the imscHRM ''validating implementation'' to 25,000 IMSC documents generated using an independent ''content producing implementation'' were reported privately to the TTWG chair. While all documents were initially expected to conform to the HRM, the imscHRM ''validating implementation'' reported that 97 of them did not conform to the HRM. Out of these 97 documents, 60 turned out to contain syntax errors that made them non-conformant with the IMSC Recommendation. The remaining 37 documents were determined to contain authorial mistakes like duplicated or partially overlapping subtitles, which were caught by the imscHRM ''validating implementation''.

In all cases, all valid IMSC documents were reported as conforming to the Hypothetical Render Model (HRM).