Different user-agents may have certain limitations on the number of SourceIDs, number of tracks, and overlap semantics they support. We need a mechanism for Web Applications to be able to detect these limitations so they can adjust their behavior and select appropriate content.
Here is an initial list of possible limitations user-agents might have:
-Unable to change sample rate or number of audio channels during playback.
-Certain decoder config parameter changes may not be allowed or prevent seamless splices. (ie Vorbis codebook changes)
-Adding/removing SourceID's during playback may not be supported.
-Limited # of simultaneous audio & video tracks. This may limit # of SourceID's as well.
-Supporting instantaneous splices @ overlap point vs waiting until the next random access point.
This is an incomplete list, but outlines some of the implementation choices we've had to consider in the Chromium implementation.
Our initial suggestion is to create MIME parameter(s) similar to codecs and reuse the current canPlayType() infrastructure to determine what a UA supports.
I'd consider the things in the list of limitations as bugs, or are there hardware limitations that would make it impossible to support some of these things correctly?
http://lists.w3.org/Archives/Public/public-html/2011Apr/0213.html may be relevant here. I think that as far as possible, we should simply require that implementations follow the spec and file bugs when they don't.
(In reply to comment #1)
> I'd consider the things in the list of limitations as bugs, or are there
> hardware limitations that would make it impossible to support some of these
> things correctly?
Many of these concerns are related to hardware implementations. Hardware decoders may only allow particular types of parameter changes and still splice seamlessly. It also isn't clear what sort of gap insertion & overlap behaviors they support. I fully admit that this is a gray area right now and we need more implementation experience to determine what is really necessary. This bug is mainly a placeholder for a concern raised during the spec update process.
> http://lists.w3.org/Archives/Public/public-html/2011Apr/0213.html may be
> relevant here. I think that as far as possible, we should simply require that
> implementations follow the spec and file bugs when they don't.
I agree, but I am concerned that embedded implementations may not be able to implement the full flexibility the spec provides. It seems like some sort of profile or capability mechanism might help content authors detect this w/o simply relying on the UA header.
For now I'm lowering the priority of this bug since the requirements aren't fully fleshed out and we don't have direct experience with hardware limitations yet.
It sounds really complicated to specify how to expose all those limitations, and even more complicated (and unlikely) for authors to correctly make use of the information.
Instead, how about defining a fixed set of limitations that allow the use-cases you care about and then requiring all UAs to follow them. Later we can relax the limitations incrementally.
I agree that it may be complicated to define a way to expose all of the originally listed possible limitations, but I also think that one thing mentioned in this bug is important enough to break out to another bug: MIME type capability detection.
Bug 19531 defines the content type support mechanism. For now we think that we should wait for more implementation experience to see if we really need a programmatic mechanism for determining capabilities. Resolved LATER.