Bug 19531 - simplify MIME type capability detection
simplify MIME type capability detection
Status: RESOLVED FIXED
Product: HTML WG
Classification: Unclassified
Component: Media Source Extensions
unspecified
All Windows 3.1
: P1 enhancement
: ---
Assigned To: Aaron Colwell (c)
HTML WG Bugzilla archive list
tpac2012
:
Depends on:
Blocks: 17000 20253
  Show dependency treegraph
 
Reported: 2012-10-14 01:57 UTC by Matt Ward
Modified: 2012-12-06 01:57 UTC (History)
6 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Matt Ward 2012-10-14 01:57:46 UTC
MIME type capability detection is really important for a lot of video player applications, and it has to be fast as to not introduce latency. It is currently really hard to tell what a MediaSource implementation is capable of playing. If you want to see if a user agent can play a certain MIME type via MediaSource you must currently: Create a MediaSource. Set it as the src on a <video> and add a callback to catch its transition to the "open" state. Once it is open, add a source buffer with the MIME that you want to play inside of a try catch statement. Catching a NOT_SUPPORTED_ERR is the current "no." Otherwise you can assume "yes."

We should keep in mind the scenario where a user has a transcode that they prefer and that transcode cannot be played back via MediaSource but can be played back via <video>.

Being able to get a synchronous answer here is probably most important. If there are strong feelings about reusing canPlayType from HTMLMediaElement that seems like a fine approach, otherwise we can define something specific for MediaSource.
Comment 1 Philip Jägenstedt 2012-10-15 07:59:03 UTC
http://dvcs.w3.org/hg/html-media/raw-file/tip/media-source/media-source.html#byte-stream-formats says

"Byte stream format specifications based on WebM and the ISO Base Media File Format are provided below. If these formats are supported then the byte stream formats described below must be supported."

In other words, the spec requires that if you support WebM in plain <video>, then you must also support it using MSE. This means that you can just use HTMLMediaElement.canPlayType(), which is synchronous.
Comment 2 Aaron Colwell (c) 2012-10-15 20:38:48 UTC
(In reply to comment #1)
> http://dvcs.w3.org/hg/html-media/raw-file/tip/media-source/media-source.
> html#byte-stream-formats says
> 
> "Byte stream format specifications based on WebM and the ISO Base Media File
> Format are provided below. If these formats are supported then the byte
> stream formats described below must be supported."
> 
> In other words, the spec requires that if you support WebM in plain <video>,
> then you must also support it using MSE. This means that you can just use
> HTMLMediaElement.canPlayType(), which is synchronous.

That wasn't my original intent when I wrote that. My original intent was "if you want to support WebM in MSE, then you must follow the rules that are outlined below". I didn't mean to imply that if canPlayType() indicates WebM support then the MSE implementation must support the associated bytestream as well. 

We can make that requirement explicit if people think that is the best path. I'm a little concerned that this would prevent UA's from incrementally adding support for different bytestream formats. If a UA supports both WebM & MP4 then they would have to add MediaSource support for both before exposing any MSE functionality. Chrome rolled out WebM-only support first and then eventually added MP4 later. I would assume that other UA's might want to follow a similar path.

I was planning on responding to this bug with something along the lines of the following:

partial interface MediaSource {
  static bool isTypeSupported(DOMString type);
};

True would be returned if the UA is able to create a SourceBuffer for the specified type. This seemed a little more explicit than what canPlayType() returns.
Comment 3 Philip Jägenstedt 2012-10-16 08:28:23 UTC
Oh, you may want to rephrase what I quoted, then :)
Comment 4 Adrian Bateman [MSFT] 2012-10-22 02:50:41 UTC
(In reply to comment #2)
> That wasn't my original intent when I wrote that. My original intent was "if
> you want to support WebM in MSE, then you must follow the rules that are
> outlined below". I didn't mean to imply that if canPlayType() indicates WebM
> support then the MSE implementation must support the associated bytestream
> as well. 

We should clarify this text. I think we definitely want to say that supporting a format in the HTML5 media element doesn't imply MSE support for it.

> I was planning on responding to this bug with something along the lines of
> the following:
> 
> partial interface MediaSource {
>   static bool isTypeSupported(DOMString type);
> };

Something like this seems reasonable.
Comment 5 Philip Jägenstedt 2012-10-22 08:29:47 UTC
Calling it canPlayType rather than isTypeSupported seems preferable to me, assuming that they have the same semantics.
Comment 6 Aaron Colwell (c) 2012-11-28 19:06:56 UTC
Changes committed.
http://dvcs.w3.org/hg/html-media/rev/0c638da9a67a

isTypeSupported() was added based on discussion at TPAC. This method simply indicates whether the MediaSource implementation will accept a particular format. It was decided to keep this separate from HTMLMediaElement.canPlayType() because the ability to play a type does not mean that the MediaSource implementation can accept that type.
Comment 7 Aaron Colwell (c) 2012-12-06 01:57:55 UTC
Changes committed
http://dvcs.w3.org/hg/html-media/rev/43be42e69533