TimedText/IMSC-HRM-1ED-Implementation-Report
Introduction
This is the implementation report for the IMSC Hypothetical Render Model Recommendation. It demonstrates adequate implementation experience and that the CR exit criteria have been met by subjecting a validating implementation to a test suite and 4 independent content producing implementations.
Validating implementations
The validating implementations are:
- imscHRM, which is an open-source implementation of the Recommendation in pure Python.
- bbc/ebu-tt-live-toolkit#66, which is an open-source implementation of the Recommendation in Python, built as part of the BBC fork of the EBU-TT Live Interoperability Toolkit
Test Suite
The test suite consists of hand-crafted IMSC documents intended exercise the boundaries of the HRM algorithm. They are organized into two sets, each in its own directory:
- each IMSC document in the
passdirectory conforms to the Hypothetical Render Model - each IMSC document in the
faildirectory does not conform to the Hypothetical Render Model
Conformance is defined at https://www.w3.org/TR/imsc-hrm/#conformance.
Each IMSC document contains a short description of the IMSC HRM feature being tested.
All IMSC documents conform to the IMSC Text Profile. They also conform to the EBU-TT-D specification, unless the feature being tested cannot be exercised by a valid EBU-TT-D document, in which case the non-conformance is also identified within the test content itself.
The EBU-TT Live Toolkit implementation cannot process documents that do not conform to EBU-TT-D and therefore the tests that exercise features prohibited in EBU-TT-D are not applicable, and are marked as "n/a".
Note that those "n/a" tests are allowed for within the specification: they refer to style attributes that are included within the definition of a glyph tuple, which, in the case that they are not permitted by a profile, are a) the same across all glyphs and b) if it is necessary to assign a value, have whatever computed value TTML2 specifies.
The following records the result of subjecting the validating implementations to the test suite:
Content producing implementations
- The results of subjecting the imscHRM validating implementation to IMSC documents generating using two independent content producing implementations was reported at https://lists.w3.org/Archives/Public/public-tt/2023Oct/0007.html. One of the content producing implementations was expected to produce IMSC documents that conform to the HRM whereas the other was expected to produce IMSC documents that did not conform to the HRM. The imscHRM validating implementation returned the expected result in both cases.
- The results of subjecting the imscHRM validating implementation to ~3,500 IMSC documents generated using an independent content producing implementation were reported by Cyril Concolato (Netflix) via email to the TTWG chair. The set of documents covered 40+ different languages and different kinds of timed text (SDH, SUBS, or forced narratives). While all documents were initially expected to conform to the HRM, the imscHRM validating implementation reported that ~10 of them did not conform to the HRM. It was ultimately determined that these ~10 documents were constructed to specifically address a limitation of the ecosystem targeted by the content producing implementation and were not generally valid IMSC documents. 2 of the documents also identified errors in the imscHRM validating implementation', which were subsequently fixed.
- The results of subjecting the imscHRM validating implementation to 25,000 IMSC documents generated using an independent content producing implementation were reported privately to the TTWG chair. While all documents were initially expected to conform to the HRM, the imscHRM validating implementation reported that 97 of them did not conform to the HRM. Out of these 97 documents, 60 turned out to contain syntax errors that made them non-conformant with the IMSC Recommendation. The remaining 37 documents were determined to contain authorial mistakes like duplicated or partially overlapping subtitles, which were caught by the imscHRM validating implementation.
In all cases, all valid IMSC documents were reported as conforming to the Hypothetical Render Model (HRM).