Bugzilla – Bug 13625
There is no way to pass audio and video content metadata to the user agent that is required in some cases for playback.
Last modified: 2014-02-26 18:24:30 UTC
Commercial video providers want to use an HTML5 user agent to provide user interface for media playback on devices that also support the UPnP home networking protocol . In this use case, home network content is stored in a content directory where each item in the content directory contains the URL of the content item along with content metadata. This metadata is called resource encoding characteristics properties . Some of this data may be needed by the user agent for content playback, specifically:
o protocolInfo - third and fourth fields  contain content type information
o size - provides a hint of the content item size in the case of file based content
o protection contains additional information for content protection
There is currently no way to represent this type of metadata in the video and audio element so it is available to the user agent. These metadata items may be relevant to types of content other than UPnP.
To address the above problem, it is proposed that three new attributes be added to the HTMLMediaElement for this metadata, e.g. contentInfo, size and protectionInfo.
 http://upnp.org/specs/arch/UPnP-arch-DeviceArchitecture-v1.0.pdf, also ISO/IEC 29341-1:2008
 http://upnp.org/specs/av/UPnP-av-ContentDirectory-v4-Service.pdf, also ISO/IEC 29341-3-12:2008
 http://upnp.org/specs/av/UPnP-av-ConnectionManager-v3-Service.pdf, also ISO/IEC 29341-3-11:2008 section 2.5.2
mass-move component to LC1
Would it be possible to treat UPnP devices much like a webcam so that you get access to it via getUserMedia? That way the UA is pretty much free to do whatever is necessary to get access to the content.
(In reply to comment #2)
> Would it be possible to treat UPnP devices much like a webcam so that you get
> access to it via getUserMedia? That way the UA is pretty much free to do
> whatever is necessary to get access to the content.
I looked for getUserMedia in W3C HTML5 ed draft and WHATWG but didn't see anything? Can you provide a link?
The problem is that the user agent might need this metadata for playback. If getUserMedia is something available to script then it wouldn't appear to solve this problem.
getUserMedia has an options parameter which could probably be used to pass along whatever initialization data is needed.
(In reply to comment #4)
> getUserMedia has an options parameter which could probably be used to pass
> along whatever initialization data is needed.
Thanks for the link. It looks like getUserMedia is used to expose a local media source while the use case stimulating this bug was the client accessing content from a server. It seems however that the getUserMedia object could be applied to the remote content use case in the following way:
- The options attribute would be used to convey the media URL that the local UA uses to fetch the media.
- The options attribute would also be used to convey metadata specific to that content that the local UA needs to correctly decode the content.
- Depending on the metadata, certain behavior of the MediaStream may be disabled, recording for example or forking of specific tracks.
I am assuming that when using the MediaStream (returned by getUserMedia) as a child of <video>, multiple video, audio and text tracks in the MediaStream would be exposed per HTML5 support for multiple video/audio tracks and <track>.
Bug triage sub-team thinks this is an HTML A11Y TF priority for the media sub-team.
At the F2F meetings in Santa Clara, the HTML WG appeared to support this bug as
a way to pass parameters to media elements for support of adaptive bit rate media as well as content protection schemes. However, the HTML WG would like a concise list of common parameters for these purposes.
The Media Pipeline task force (MPTF) has agreed to review this and come up with a proposed list.
Here is a link to the relevant MPTF requirements:
(In reply to comment #7)
> At the F2F meetings in Santa Clara, the HTML WG appeared to support this bug as
> a way to pass parameters to media elements for support of adaptive bit rate
> media as well as content protection schemes. However, the HTML WG would like a
> concise list of common parameters for these purposes.
> The Media Pipeline task force (MPTF) has agreed to review this and come up with
> a proposed list.
The MPTF also took an action item to provide a document detailing the mapping from UPnP to HTML5, which is the example in the original bug report. This is currently in preparation.
Flagging this with the needsinfo keyword to note that resolution is waiting on further information to be provided; see comment #7 and comment #8
EDITOR'S RESPONSE: This is an Editor's Response to your comment. If you are satisfied with this response, please change the state of this bug to CLOSED. If you have additional information and would like the editor to reconsider, please reopen this bug. If you would like to escalate the issue to the full HTML Working Group, please add the TrackerRequest keyword to this bug, and suggest title and text for the tracker issue; or you may create a tracker issue yourself, if you are able to do so. For more details, see this document:
Status: Did Not Understand Request
Change Description: no spec change
Rationale: see comment 9
Bob, Mark, Clarke,
Moving this to resolved=needsinfo is just a housekeeping way of indicating that this is waiting for additional information. It doesn't mean anybody considers it finally resolved ("resolved" is just the closest appropriate status that bugzilla provides).
So please go ahead and reopen this when you have the info ready that you noted in comment #7 and comment #8.
(In reply to comment #11)
> Bob, Mark, Clarke,
> Moving this to resolved=needsinfo is just a housekeeping way of indicating that
> this is waiting for additional information. It doesn't mean anybody considers
> it finally resolved ("resolved" is just the closest appropriate status that
> bugzilla provides).
> So please go ahead and reopen this when you have the info ready that you noted
> in comment #7 and comment #8.
In the HTML WG/Media Pipeline Task Force at TPAC CableLabs and Comcast agreed to document the how HTML5 would be used in a UPnP environment. This link http://html5.cablelabs.com/upnp/html5-upnp-integration.html#2-box-a provides additional details relevant to the UPnP use case behind this bug.
(In reply to comment #12)
> In the HTML WG/Media Pipeline Task Force at TPAC CableLabs and Comcast agreed
> to document the how HTML5 would be used in a UPnP environment. This link
> http://html5.cablelabs.com/upnp/html5-upnp-integration.html#2-box-a provides
> additional details relevant to the UPnP use case behind this bug.
I note that this bug is now just waiting on the list of parameters mentioned in comment #7
(In reply to comment #13)
> (In reply to comment #12)
> > In the HTML WG/Media Pipeline Task Force at TPAC CableLabs and Comcast agreed
> > to document the how HTML5 would be used in a UPnP environment. This link
> > http://html5.cablelabs.com/upnp/html5-upnp-integration.html#2-box-a provides
> > additional details relevant to the UPnP use case behind this bug.
> I note that this bug is now just waiting on the list of parameters mentioned in
> comment #7
The Media Pipeline Task Force has submitted the following proposal in response to comment #7
This bug was cloned to create bug 17971 as part of operation convergence.
If I understand you correctly, what you are asking for is to provide an API to influence decisions that the browser makes for HTTP adaptive streaming.
 lists the following interfaces:
* maximumBandwidth (input) - specifies the maximum bandwidth usage for
downloading adaptive streaming content. The input is in bits/second.
* minimumBandwidth(input) - specifies the minimum throughput above which the UA
should optimize for continuous playback and below which it should optimize for
quality, even if this implies buffering. The input is in bits/second.
* startingPlaybackHint(input) - gives the UA a hint about the preferred startup
optimization, whether it be for fast video startup or for high quality.
IIUC this is proposing to have a middle ground between the situation where the
Web Developer does everything related to adaptive streaming by hand (using )
and the situation where the Web Developer provides a list of resource
descriptions (likely in a DASH file) and leaves the decisions completely to the browser which bytes to get from which file.
In addition, this proposal  includes a list of suggested statistics on videos:
This second half reminds me a lot of bug 12399 and bug 14970 . It would be good to get involved with the Web Performance group and arrange for some of these statistics to be standardised there. Note that they have just asked for input on what next features they should work on. 
I'm also suggesting to move this to HTML.next, since I can't see this being resolved for HTML5.
Silvia, did you mean to assign this to me? If so, please mark it as a dupe of bug 17971. Thanks!
Ian, sorry, no I didn't (not even sure how that happened...) - forwarding your question, though, to stay in syc...
Bob, Clarke: would your needs be satisfied by bug 14970, so can we close this as a duplicate of that bug?
This is definitely interesting, but as Silvia suggests it won't make it in the 5.0 timeframe. Moving to HTML.next.
Mass move to "HTML WG"
In the related bug 17971, Bob Lund repliad:
The intent of 13625 was to request a media element API to pass media metadata to the player. The use cases are being addressed by EME and MSE work in the HTML-WG so this bug can be closed.
I am closing this bug based on this information. If there is anything left, please re-open with a concrete proposal or add a new bug to the EME or MSE work.