This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.
The specification is unclear how an app can change SourceBuffer at a particular time in a way that gives a good user experience. For example, changing SourceBuffer at the time when audio changes codec from Dolby to HE-AAC or vice-versa. Section 2.4.4 of MSE describes the situation as follows; “If buffered for at least one object in activeSourceBuffers contains a TimeRange that ends at the current playback position and does not have a range covering the time immediately after the current position” And concludes with Playback is suspended at this point since the media element doesn't have enough data to advance the media timeline. We believe this would generate a “waiting” event on the video element which an app could then use to modify the active source buffers to remove the SourceBuffer used for Dolby audio and add a SourceBuffer for the HE-AAC audio (or vice versa). This however is unlikely to give a good user experience, particularly as video playback would also be paused at this time. Please provide a means to change SourceBuffer at a particular time which will give a better user experience than relying on a “waiting” event being fired, a JS event handler being activated which then disables and enables AudioTracks. NOTE: This issue arises from joint discussions between the Open IPTV Forum, HbbTV and the UK DTG. These organizations originally sent a liaison statement to the W3C Web & TV IG which is archived here; https://lists.w3.org/Archives/Member/member-web-and-tv/2013Jan/0000.html (W3C member only link)
Marking all pre-Last Call bugs
Yes. I would expect a "waiting" event to fire in the situation you describe. I have my doubts about implementations being able to handle this case seamlessly especially in the mobile context. This is primarily why I made the spec not guarentee codec changes to be seemless and why different codecs must be handled by different SourceBuffers. The most natural solution is to ease the restriction on codec changes. If that is what the group decides, then I'll insist that fallback behavior be specified that allows implementations to play silence and/or display black frames when the codec changes if the UA is unable to handle such changes in the middle of playback.
This comment is submitted on behalf of the participants in the joint discussions between the Open IPTV Forum, HbbTV and the UK DTG. The combination of easing the restrictions on codec changes and defining the fallback behavior as you propose would address our issue for audio codec changes which violate the byte stream format rules. Changes in the number of channels that violate the byte stream format rules are addressed in #22137. Although we can see other instances where the byte stream format rules would be violated, we believe these are the most likely to be encountered in the real world. Please consider adding a NOTE either to the description of SourceBuffer or to the description of the byte stream format explaining that an incompatible change of byte stream format can be done by registering for the waiting event, waiting for the original format data to run out and then when the ‘waiting’ event handler fires, that handler disabling the SourceBuffer for the original format data and enabling the SourceBuffer for the new format data.
We have discussed this issue several times and concluded that we do not intend to support this scenario in v1. Once we get good implementation experience for the current spec, we should consider this issue for a v2 spec. Propose RESOLVE, WONTFIX.
This was discussed at the telcon on June 11. This bug is a feature request for HTML5 and is not MSE specific. We resolved that we would not add this functionality to MSE in this version. Resolving Won't Fix.