This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.

Bug 29188 - Require at least one block from each audio and video track in media segment definition
Summary: Require at least one block from each audio and video track in media segment d...
Status: RESOLVED MOVED
Alias: None
Product: HTML WG
Classification: Unclassified
Component: Media Source Extensions (show other bugs)
Version: unspecified
Hardware: PC Linux
: P2 normal
Target Milestone: ---
Assignee: Matt Wolenetz
QA Contact: HTML WG Bugzilla archive list
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-10-08 19:15 UTC by Matt Wolenetz
Modified: 2015-10-13 23:22 UTC (History)
3 users (show)

See Also:
wolenetz: needinfo? (jdsmith)


Attachments

Description Matt Wolenetz 2015-10-08 19:15:41 UTC
The coded frame processing algorithm assumes that a media segment consists of interleaved coded frames in DTS order from each track. However, only the WebM bytestream spec makes it clear that at least one coded frame from each audio and video track must exist in each media segment. Without similar requirement for other bytestreams, like ISO-BMFF, the coded frame processing algorithm could produce undesired output for scenarios like:

With Audio/Video SourceBuffer, append in order:
media segment containing only video for time [100.....200)
media segment containing only audio for time [0.....100)

Since these are the first appends ever (or after some previous parser reset or CFP discontinuity detection) to this SourceBuffer, the audio track's "last decode timestamp" is not set prior to the append of the second media segment, above. The result is there is no discontinuity detected during that append (so all tracks are not told to need a random access point, and their last decode timestamps are not reset.) Also, in sequence append mode, the timestampOffset would be -100 from the first media segment, making the buffers from the audio-only segment be clipped since its append doesn't trigger discontinuity and group_start_timestamp update.

While these do not appear on the surface to be absolute blocking concerns, users and implementors of the API would need to understand the behavior of CFP with respect to single-stream appends to muxed A/V SourceBuffers.

This bug tracks the possibility of enforcing the CFP's assumption that at least one coded frame from each audio and video track for the SourceBuffer be included in each media segment.

It is my understanding that MSE API users that append ISO-BMFF bytestreams currently are primarily dash.js or similar, for which single-track SourceBuffers are the common usage, so enforcing constraints in spec and implemnentations on multi-track SourceBuffers shouldn't regress those users.
Comment 1 Matt Wolenetz 2015-10-08 19:21:43 UTC
Jerry, are you aware of any MSE API users who might be regressed by adding this constraint to the MSE media segment definition?
Comment 2 Matt Wolenetz 2015-10-13 23:22:00 UTC
This bug has been migrated to the GitHub issue tracker. Please follow/update progress using the GitHub issue:
https://github.com/w3c/media-source/issues/31