User:Spfeiffe/TextTrackChange

From HTML WG Wiki
Jump to: navigation, search

Text Track Cue API Changes

Recently, the Text Track Cue API has changed from:

enum AutoKeyword { "auto" };
[Constructor(double startTime, double endTime, DOMString text)]
interface TextTrackCue : EventTarget {
  readonly attribute TextTrack? track;

           attribute DOMString id;
           attribute double startTime;
           attribute double endTime;
           attribute boolean pauseOnExit;
           attribute DOMString vertical;
           attribute boolean snapToLines;
           attribute (long or AutoKeyword) line;
           attribute long position;
           attribute long size;
           attribute DOMString align;
           attribute DOMString text;
  DocumentFragment getCueAsHTML();

           attribute EventHandler onenter;
           attribute EventHandler onexit;
};

to

interface TextTrackCue : EventTarget {
  readonly attribute TextTrack? track;

           attribute DOMString id;
           attribute double startTime;
           attribute double endTime;
           attribute boolean pauseOnExit;

           attribute EventHandler onenter;
           attribute EventHandler onexit;
};

with the addition of a new WebVTT specific text track cue API:

enum AutoKeyword { "auto" };
[Constructor(double startTime, double endTime, DOMString text)]
interface VTTCue : TextTrackCue {
           attribute DOMString vertical;
           attribute boolean snapToLines;
           attribute (long or AutoKeyword) line;
           attribute long position;
           attribute long size;
           attribute DOMString align;
           attribute DOMString text;
  DocumentFragment getCueAsHTML();
};

Relevant patches:

The aim of these patches:

  • Remove WebVTT-specifics from the generic TextTrackCue object
  • Create an abstract TextTrackCue API that can be extended to other file formats such as TTML,


Related bugs:


Discussion issues

(1) Removal of .text from TextTrackCue

Argument to re-introduce:

  • every TextTrackCue from every format that has been implemented by UAs has a content - this content should be exposed by the same IDL attribute in all TextTrackCue objects
  • examples are the VTTCue API, TTML support in IE10, and MPEG-2 to HTML mapping
  • even binary timed data has this far been delivered as text, e.g. data urls for timed thumbnails in WebVTT
  • the @inBandMetadataTrackDispatchType on the TextTrack object interprets the content of a generic TextTrackCue - yet, there is now no attribute that contains that content
  • we are not solving hypothetical use cases of potential binary cue content that nobody is currently intending to implement

Argument against:

  • there may be timed cue formats that don't expose text, e.g. images, bitmaps or URLs

Proposal:

  • re-introduce the .text attribute

(2) Removal of getCueAsHTML() from TextTrackCue

Argument to re-introduce:

Argument against:

  • there will be many timed cue formats with binary content that will not be converted to a HTML fragment for rendering, e.g. images, bitmaps, data URLs
  • every new format that a UA supports needs an extensive algorithm to convert a cue's content to HTML - this needs to be specified and implemented in the UA and discoverable that it's available - own Object definitions make sense, e.g. TTMLCue API, CEA708CueAPI
  • the MPEG-2 to HTML document defines getCueAsHTML() always as null except where the UA knows how to convert a caption cue to HTML - it makes sense that the UA returns a more appropriate object in these cases, such as VTTCue or TTMLCue, such that it is discoverable what type the UA identified

Proposal:

  • no change


(3) Removal of TextTrackCue constructor

Argument to re-introduce:

  • TextTrack.addCue() requires a cue to be constructed in HTML; by removing the constructor, we can only create new cues of a particular type (VTTCue only at this point in time)
  • it makes more sense for JS-created, non-rendered kind=metadata cues that don't use most of the VTTCue attributes and not its rendering algorithm to be a TextTrackCue object
  • previously it was possible to construct a cue of the WebVTT type and append it to a track with interpretation of cue content done by JavaScript, e.g.

cue = new TextTrackCue(0,2,'<region xml:id="r1" tts:origin="10px 100px" tts:extent="300px 96px" /> <body xml:id="b1"> <div xml:id="d1"> <p xml:id="p1">Text 1</p> </div> </body> </region>'); that's now possible only using new VTTCue and thus can break existing JavaScript implementations


Argument against:

  • TextTrackCues that can be constructed but not rendered make sense only for @kind=metadata tracks
  • breaks backwards compatibility without people noticing the change

Proposal:

  • re-add constructor to TextTrackCue:
[Constructor(double startTime, double endTime, DOMString text)]
interface TextTrackCue : EventTarget {
 ...
};


Potential further improvement:

  • introduce a cueFormatHint attribute for TextTrack, replacing the existing inBandMetadataTrackDispatchType attribute and providing an optional cue format hint for any TextTrack
  • cueFormatHint is by default "plaintext"
  • for WebVTT files parsed from <track>, the cueFormatHint would be "webvtt" for cues that are supposed to be parsed with the VTTCue, or "plaintext" otherwise
  • however: browser should create VTTCue instead of TextTrackCue with "webvtt" as the hint and JS devs should use instanceof to find out what object type the browser created for in-band tracks