- Breakout: HTMLCue
- Proposer: Nigel Megitt
- Minutes: http://www.w3.org/2015/10/28-htmlcue-minutes.html
This well attended session discussed the idea of a new kind of Text Track Cue which would allow any fragment of HTML+CSS to be used to modify the display on a target element in synchronisation with a media element's timeline. The idea originated with Opera and has also been discussed in TTWG and WHATWG: to have a cue whose payload data is a fragment of HTML and whose default onenter() handler would attach that fragment to the target element, e.g. a video, for example to update the captions or subtitles, and whose onexit() would somehow remove or clear it. This approach is used for example in dash.js to present subtitles now, but the only browser supported cue type is a VTTCue that has to be overloaded to do this, and the extra VTT style attributes are ignored. A cleaner solution would be to have a supported generic cue type that doesn't include an initial list of VTT specific styles.
Previous discussions have suggested that this approach could have unintended consequences and would need to be sanitised; for example it would need to be clear that the changes do not create new navigation contexts, and there may be risks associated with running scripts embedded in the HTML, or loading external resources like images - those risks were not enumerated.
Different views were expressed, not all in agreement:
- use of "metadata" tracks subverts the intent behind the label "metadata" when the purpose of the tracks is to display content.
- a browser-native implementation not requiring use of js to attach the HTML payload to a target element would give browsers an opportunity to optimise for display rendering to achieve the required cue timing for example by pre-rendering.
- a native implementation could allow browsers to use default player controls: bespoke/custom subtitle handling is generally likely to go alongside custom player controls.
- to handle the risks a sandbox approach could be used similar to what is available for iframe (or indeed by simply using an iframe and its sandbox directly)
- browsers should not be expected to auto-load external resources like images from html attached in this way because that could create a privacy issue e.g. it would inform the image source server where in the related video the user is watching.
- but also in opposition to that: browsers should be expected to auto-load external resources attached in this way, since the origin of the source data can be considered 'trusted'.
- we should not try to subset html into a 'safe' set but rather build up a required and safe set from a blank starting point - we tried the former approach before and it caused problems.
- the idea of HTMLCue is a good one from an architectural perspective.
Two actions were noted:
- Work on a prototype using the TextTrack kind="metadata" and highlight any limitations or issues discovered
Other next steps include summarising the HTMLCue proposal in a clear document.