13625 2011-08-03 19:38:53 +0000 There is no way to pass audio and video content metadata to the user agent that is required in some cases for playback. 2014-02-26 18:24:30 +0000 1 1 1 Unclassified HTML WG HTML5 spec unspecified All All CLOSED WONTFIX a11y, a11ytf, media P2 normal --- 1 b.lund silviapfeiffer1 c.stevens cooper eoconnor ian mark.sadecki+w3c mark_vickers mike pcotton philipj public-html-admin public-html-wg-issue-tracking robin scherkus silviapfeiffer1 watsonm public-html-bugzilla oldest_to_newest 52220 0 b.lund 2011-08-03 19:38:53 +0000 Problem description: Commercial video providers want to use an HTML5 user agent to provide user interface for media playback on devices that also support the UPnP home networking protocol [1]. In this use case, home network content is stored in a content directory where each item in the content directory contains the URL of the content item along with content metadata. This metadata is called resource encoding characteristics properties [2]. Some of this data may be needed by the user agent for content playback, specifically: o protocolInfo - third and fourth fields [3] contain content type information o size - provides a hint of the content item size in the case of file based content o protection – contains additional information for content protection There is currently no way to represent this type of metadata in the video and audio element so it is available to the user agent. These metadata items may be relevant to types of content other than UPnP. Proposed solution: To address the above problem, it is proposed that three new attributes be added to the HTMLMediaElement for this metadata, e.g. contentInfo, size and protectionInfo. [1] http://upnp.org/specs/arch/UPnP-arch-DeviceArchitecture-v1.0.pdf, also ISO/IEC 29341-1:2008 [2] http://upnp.org/specs/av/UPnP-av-ContentDirectory-v4-Service.pdf, also ISO/IEC 29341-3-12:2008 [3] http://upnp.org/specs/av/UPnP-av-ConnectionManager-v3-Service.pdf, also ISO/IEC 29341-3-11:2008 section 2.5.2 53711 1 mike 2011-08-04 05:16:36 +0000 mass-move component to LC1 54157 2 philipj 2011-08-04 08:24:11 +0000 Would it be possible to treat UPnP devices much like a webcam so that you get access to it via getUserMedia? That way the UA is pretty much free to do whatever is necessary to get access to the content. 54176 3 b.lund 2011-08-04 16:29:00 +0000 (In reply to comment #2) > Would it be possible to treat UPnP devices much like a webcam so that you get > access to it via getUserMedia? That way the UA is pretty much free to do > whatever is necessary to get access to the content. I looked for getUserMedia in W3C HTML5 ed draft and WHATWG but didn't see anything? Can you provide a link? The problem is that the user agent might need this metadata for playback. If getUserMedia is something available to script then it wouldn't appear to solve this problem. Bob 54218 4 philipj 2011-08-05 06:27:40 +0000 http://www.whatwg.org/specs/web-apps/current-work/multipage/video-conferencing-and-peer-to-peer-communication.html#obtaining-local-multimedia-content getUserMedia has an options parameter which could probably be used to pass along whatever initialization data is needed. 54707 5 b.lund 2011-08-11 17:04:16 +0000 (In reply to comment #4) > http://www.whatwg.org/specs/web-apps/current-work/multipage/video-conferencing-and-peer-to-peer-communication.html#obtaining-local-multimedia-content > > getUserMedia has an options parameter which could probably be used to pass > along whatever initialization data is needed. Thanks for the link. It looks like getUserMedia is used to expose a local media source while the use case stimulating this bug was the client accessing content from a server. It seems however that the getUserMedia object could be applied to the remote content use case in the following way: - The options attribute would be used to convey the media URL that the local UA uses to fetch the media. - The options attribute would also be used to convey metadata specific to that content that the local UA needs to correctly decode the content. - Depending on the metadata, certain behavior of the MediaStream may be disabled, recording for example or forking of specific tracks. I am assuming that when using the MediaStream (returned by getUserMedia) as a child of <video>, multiple video, audio and text tracks in the MediaStream would be exposed per HTML5 support for multiple video/audio tracks and <track>. 57037 6 cooper 2011-09-20 15:31:17 +0000 Bug triage sub-team thinks this is an HTML A11Y TF priority for the media sub-team. 59801 7 c.stevens 2011-11-09 23:04:59 +0000 At the F2F meetings in Santa Clara, the HTML WG appeared to support this bug as a way to pass parameters to media elements for support of adaptive bit rate media as well as content protection schemes. However, the HTML WG would like a concise list of common parameters for these purposes. The Media Pipeline task force (MPTF) has agreed to review this and come up with a proposed list. Here is a link to the relevant MPTF requirements: http://www.w3.org/2011/webtv/wiki/MPTF/MPTF_Requirements#R7._Additional_Media_Parameters http://www.w3.org/2011/webtv/wiki/MPTF/MPTF_Requirements#R10._Content_Protection_Parameters 59806 8 mark_vickers 2011-11-09 23:14:50 +0000 (In reply to comment #7) > At the F2F meetings in Santa Clara, the HTML WG appeared to support this bug as > a way to pass parameters to media elements for support of adaptive bit rate > media as well as content protection schemes. However, the HTML WG would like a > concise list of common parameters for these purposes. > > The Media Pipeline task force (MPTF) has agreed to review this and come up with > a proposed list. The MPTF also took an action item to provide a document detailing the mapping from UPnP to HTML5, which is the example in the original bug report. This is currently in preparation. 60181 9 mike 2011-11-20 14:17:32 +0000 Flagging this with the needsinfo keyword to note that resolution is waiting on further information to be provided; see comment #7 and comment #8 60410 10 ian 2011-11-24 00:54:25 +0000 EDITOR'S RESPONSE: This is an Editor's Response to your comment. If you are satisfied with this response, please change the state of this bug to CLOSED. If you have additional information and would like the editor to reconsider, please reopen this bug. If you would like to escalate the issue to the full HTML Working Group, please add the TrackerRequest keyword to this bug, and suggest title and text for the tracker issue; or you may create a tracker issue yourself, if you are able to do so. For more details, see this document: http://dev.w3.org/html5/decision-policy/decision-policy.html Status: Did Not Understand Request Change Description: no spec change Rationale: see comment 9 60411 11 mike 2011-11-24 01:14:16 +0000 Bob, Mark, Clarke, Moving this to resolved=needsinfo is just a housekeeping way of indicating that this is waiting for additional information. It doesn't mean anybody considers it finally resolved ("resolved" is just the closest appropriate status that bugzilla provides). So please go ahead and reopen this when you have the info ready that you noted in comment #7 and comment #8. 60588 12 b.lund 2011-11-28 23:40:32 +0000 (In reply to comment #11) > Bob, Mark, Clarke, > > Moving this to resolved=needsinfo is just a housekeeping way of indicating that > this is waiting for additional information. It doesn't mean anybody considers > it finally resolved ("resolved" is just the closest appropriate status that > bugzilla provides). > > So please go ahead and reopen this when you have the info ready that you noted > in comment #7 and comment #8. In the HTML WG/Media Pipeline Task Force at TPAC CableLabs and Comcast agreed to document the how HTML5 would be used in a UPnP environment. This link http://html5.cablelabs.com/upnp/html5-upnp-integration.html#2-box-a provides additional details relevant to the UPnP use case behind this bug. Bob 60608 13 mike 2011-11-29 09:32:55 +0000 (In reply to comment #12) > In the HTML WG/Media Pipeline Task Force at TPAC CableLabs and Comcast agreed > to document the how HTML5 would be used in a UPnP environment. This link > http://html5.cablelabs.com/upnp/html5-upnp-integration.html#2-box-a provides > additional details relevant to the UPnP use case behind this bug. Excellent. I note that this bug is now just waiting on the list of parameters mentioned in comment #7 61644 14 c.stevens 2011-12-16 06:57:05 +0000 (In reply to comment #13) > (In reply to comment #12) > > In the HTML WG/Media Pipeline Task Force at TPAC CableLabs and Comcast agreed > > to document the how HTML5 would be used in a UPnP environment. This link > > http://html5.cablelabs.com/upnp/html5-upnp-integration.html#2-box-a provides > > additional details relevant to the UPnP use case behind this bug. > > Excellent. > > I note that this bug is now just waiting on the list of parameters mentioned in > comment #7 The Media Pipeline Task Force has submitted the following proposal in response to comment #7 http://www.w3.org/2011/webtv/wiki/MPTF/ADR_Minimal_Control_Model_Proposal 61645 15 c.stevens 2011-12-16 06:57:49 +0000 (In reply to comment #13) > (In reply to comment #12) > > In the HTML WG/Media Pipeline Task Force at TPAC CableLabs and Comcast agreed > > to document the how HTML5 would be used in a UPnP environment. This link > > http://html5.cablelabs.com/upnp/html5-upnp-integration.html#2-box-a provides > > additional details relevant to the UPnP use case behind this bug. > > Excellent. > > I note that this bug is now just waiting on the list of parameters mentioned in > comment #7 The Media Pipeline Task Force has submitted the following proposal in response to comment #7 http://www.w3.org/2011/webtv/wiki/MPTF/ADR_Minimal_Control_Model_Proposal 70404 16 contributor 2012-07-18 07:26:03 +0000 This bug was cloned to create bug 17971 as part of operation convergence. 74852 17 silviapfeiffer1 2012-09-28 23:58:06 +0000 If I understand you correctly, what you are asking for is to provide an API to influence decisions that the browser makes for HTTP adaptive streaming. [1] lists the following interfaces: * maximumBandwidth (input) - specifies the maximum bandwidth usage for downloading adaptive streaming content. The input is in bits/second. * minimumBandwidth(input) - specifies the minimum throughput above which the UA should optimize for continuous playback and below which it should optimize for quality, even if this implies buffering. The input is in bits/second. * startingPlaybackHint(input) - gives the UA a hint about the preferred startup optimization, whether it be for fast video startup or for high quality. IIUC this is proposing to have a middle ground between the situation where the Web Developer does everything related to adaptive streaming by hand (using [2]) and the situation where the Web Developer provides a list of resource descriptions (likely in a DASH file) and leaves the decisions completely to the browser which bytes to get from which file. == In addition, this proposal [1] includes a list of suggested statistics on videos: * downloadRate * representationID * droppedFrames * decodedFrames * bufferLength This second half reminds me a lot of bug 12399 and bug 14970 . It would be good to get involved with the Web Performance group and arrange for some of these statistics to be standardised there. Note that they have just asked for input on what next features they should work on. [3] == I'm also suggesting to move this to HTML.next, since I can't see this being resolved for HTML5. [1] http://www.w3.org/2011/webtv/wiki/MPTF/ADR_Minimal_Control_Model_Proposal [2] http://dev.w3.org/2011/webrtc/editor/getusermedia.html [3] http://lists.w3.org/Archives/Public/public-web-perf/2012Sep/0044.html 74942 18 ian 2012-10-01 18:38:08 +0000 Silvia, did you mean to assign this to me? If so, please mark it as a dupe of bug 17971. Thanks! 75005 19 silviapfeiffer1 2012-10-02 00:20:28 +0000 Ian, sorry, no I didn't (not even sure how that happened...) - forwarding your question, though, to stay in syc... Bob, Clarke: would your needs be satisfied by bug 14970, so can we close this as a duplicate of that bug? 78917 20 robin 2012-11-27 15:55:20 +0000 This is definitely interesting, but as Silvia suggests it won't make it in the 5.0 timeframe. Moving to HTML.next. 81736 21 robin 2013-01-21 15:59:46 +0000 Mass move to "HTML WG" 81854 22 robin 2013-01-21 16:02:33 +0000 Mass move to "HTML WG" 86848 23 silviapfeiffer1 2013-04-26 10:09:05 +0000 In the related bug 17971, Bob Lund repliad: The intent of 13625 was to request a media element API to pass media metadata to the player. The use cases are being addressed by EME and MSE work in the HTML-WG so this bug can be closed. I am closing this bug based on this information. If there is anything left, please re-open with a concrete proposal or add a new bug to the EME or MSE work.