Warning:
This wiki has been archived and is now read-only.
User:Spfeiffe/TextTrackChange
From HTML WG Wiki
Contents
Text Track Cue API Changes
Recently, the Text Track Cue API has changed from:
enum AutoKeyword { "auto" }; [Constructor(double startTime, double endTime, DOMString text)] interface TextTrackCue : EventTarget { readonly attribute TextTrack? track; attribute DOMString id; attribute double startTime; attribute double endTime; attribute boolean pauseOnExit; attribute DOMString vertical; attribute boolean snapToLines; attribute (long or AutoKeyword) line; attribute long position; attribute long size; attribute DOMString align; attribute DOMString text; DocumentFragment getCueAsHTML(); attribute EventHandler onenter; attribute EventHandler onexit; };
to
interface TextTrackCue : EventTarget { readonly attribute TextTrack? track; attribute DOMString id; attribute double startTime; attribute double endTime; attribute boolean pauseOnExit; attribute EventHandler onenter; attribute EventHandler onexit; };
with the addition of a new WebVTT specific text track cue API:
enum AutoKeyword { "auto" }; [Constructor(double startTime, double endTime, DOMString text)] interface VTTCue : TextTrackCue { attribute DOMString vertical; attribute boolean snapToLines; attribute (long or AutoKeyword) line; attribute long position; attribute long size; attribute DOMString align; attribute DOMString text; DocumentFragment getCueAsHTML(); };
Relevant patches:
- Split TextTrackCue into an abstract TextTrackCue interface and a WebVTT-specific interface VTTCue.
- Split more WebVTT-specific things into the WebVTT spec.
The aim of these patches:
- Remove WebVTT-specifics from the generic TextTrackCue object
- Create an abstract TextTrackCue API that can be extended to other file formats such as TTML,
Related bugs:
- addTextTrack incorrectly assumes WebVTT content type
- TextTrackCue() constructor incorrectly assumes WebVTT content type
- revert removal of text and getCueAsHTML members; address constructor changes
Discussion issues
(1) Removal of .text from TextTrackCue
Argument to re-introduce:
- every TextTrackCue from every format that has been implemented by UAs has a content - this content should be exposed by the same IDL attribute in all TextTrackCue objects
- examples are the VTTCue API, TTML support in IE10, and MPEG-2 to HTML mapping
- even binary timed data has this far been delivered as text, e.g. data urls for timed thumbnails in WebVTT
- the @inBandMetadataTrackDispatchType on the TextTrack object interprets the content of a generic TextTrackCue - yet, there is now no attribute that contains that content
- we are not solving hypothetical use cases of potential binary cue content that nobody is currently intending to implement
Argument against:
- there may be timed cue formats that don't expose text, e.g. images, bitmaps or URLs
Proposal:
- re-introduce the .text attribute
(2) Removal of getCueAsHTML() from TextTrackCue
Argument to re-introduce:
- text track cues generally need to be rendered and getCueAsHTML() will provide a document fragment that the UA can render
- examples are WebVTT cue rendering, TTML rendering, and MPEG-2 to HTML making use of getCueAsHTML()
Argument against:
- there will be many timed cue formats with binary content that will not be converted to a HTML fragment for rendering, e.g. images, bitmaps, data URLs
- every new format that a UA supports needs an extensive algorithm to convert a cue's content to HTML - this needs to be specified and implemented in the UA and discoverable that it's available - own Object definitions make sense, e.g. TTMLCue API, CEA708CueAPI
- the MPEG-2 to HTML document defines getCueAsHTML() always as null except where the UA knows how to convert a caption cue to HTML - it makes sense that the UA returns a more appropriate object in these cases, such as VTTCue or TTMLCue, such that it is discoverable what type the UA identified
Proposal:
- no change
(3) Removal of TextTrackCue constructor
Argument to re-introduce:
- TextTrack.addCue() requires a cue to be constructed in HTML; by removing the constructor, we can only create new cues of a particular type (VTTCue only at this point in time)
- it makes more sense for JS-created, non-rendered kind=metadata cues that don't use most of the VTTCue attributes and not its rendering algorithm to be a TextTrackCue object
- previously it was possible to construct a cue of the WebVTT type and append it to a track with interpretation of cue content done by JavaScript, e.g.
cue = new TextTrackCue(0,2,'<region xml:id="r1" tts:origin="10px 100px" tts:extent="300px 96px" /> <body xml:id="b1"> <div xml:id="d1"> <p xml:id="p1">Text 1</p> </div> </body> </region>');
that's now possible only using new VTTCue and thus can break existing JavaScript implementations
- if the browser knows how to parse a text track (either from <track> or from an in-band track), but doesn't know how to parse the cue content, it has no way to expose them to JS right now
- spec has been changed before and looking at https://github.com/search?l=javascript&q=TextTrackCue&ref=searchresults&type=Code , there are already several incompatible uses of TextTrackCue constructor out there
Argument against:
- TextTrackCues that can be constructed but not rendered make sense only for @kind=metadata tracks
- breaks backwards compatibility without people noticing the change
Proposal:
- re-add constructor to TextTrackCue:
[Constructor(double startTime, double endTime, DOMString text)] interface TextTrackCue : EventTarget { ... };
Potential further improvement:
- introduce a cueFormatHint attribute for TextTrack, replacing the existing inBandMetadataTrackDispatchType attribute and providing an optional cue format hint for any TextTrack
- cueFormatHint is by default "plaintext"
- for WebVTT files parsed from <track>, the cueFormatHint would be "webvtt" for cues that are supposed to be parsed with the VTTCue, or "plaintext" otherwise
- however: browser should create VTTCue instead of TextTrackCue with "webvtt" as the hint and JS devs should use instanceof to find out what object type the browser created for in-band tracks