Bug 18592 - How much is "enough data to ensure uninterrupted playback"
How much is "enough data to ensure uninterrupted playback"
Status: RESOLVED FIXED
Product: HTML WG
Classification: Unclassified
Component: Media Source Extensions
unspecified
All Windows 3.1
: P2 normal
: ---
Assigned To: Aaron Colwell (c)
HTML WG Bugzilla archive list
tpac2012
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2012-08-16 15:14 UTC by Philip Jägenstedt
Modified: 2013-01-30 21:21 UTC (History)
5 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Philip Jägenstedt 2012-08-16 15:14:33 UTC
http://dvcs.w3.org/hg/html-media/raw-file/tip/media-source/media-source.html#buffer-monitoring

The difference between HAVE_ENOUGH_DATA and HAVE_FUTURE_DATA is "enough data to ensure uninterrupted playback". Since it is the web application that decides the append rate, it seems difficult for the web browser to guess when it will be possible to play through to the end.

Is an implementation expected to extrapolate the append rate in a similar fashion as one would when estimating readyState for any other network resource, or should one go to HAVE_ENOUGH_DATA only when buffered covers all times from currentTime to duration?
Comment 1 Mark Watson 2012-08-16 15:38:36 UTC
You could ask this question of the media element in the general case. Does HAVE_ENOUGH mean the UA has enough to be *sure*, 100%, that playback will be uninterrupted? This is only possible if it has the entire content item. It would be better named HAVE_ALL... in that case.

If, on the other hand, the difference between these states is just a hint to the application, giving a UA-specific assessment of whether the rebuffer probability is low or high, then this is irrelevant in the Media Source case: the application knows better than the Media Element. In particular the application knows how much data it has downloaded and not yet passed to the media element.

I don't think the Media Element should make any measurements of the append rate, because it has no knowledge of what is causing the append rate to be what it is: there are many things the application could be doing and possibly actions it could take to avoid a rebuffer about which the media element has no idea. For example switching to a different bitrate which could dramatically change the append rate.

If we wanted to strictly follow the model that player code dealing with UI, controls etc. should not care whether media source is used or not, then the state change from FUTURE to ENOUGH should be triggered by the application through the Media Source API.

Alternatively, the application UI code will need to make its decision on when to start playback based on indications from the application download/append code, not on the Media Element state.
Comment 2 Philip Jägenstedt 2012-08-17 08:32:02 UTC
The question can indeed be asked for other kinds of resources as well, but for e.g. file and HTTP resources the browser is in a much better position to estimate when HAVE_FUTURE_DATA makes sense.

Unless there are use cases for the canplaythrough event with MSE, I think we should simply spec it to never go beyond HAVE_FUTURE_DATA, for simplicity.
Comment 3 Philip Jägenstedt 2012-08-17 12:26:46 UTC
http://dvcs.w3.org/hg/html-media/raw-file/tip/media-source/media-source.html#dom-append

Here's another similar requirement for append(data): "If the HTMLMediaElement.readyState attribute is HAVE_FUTURE_DATA and data causes all objects in activeSourceBuffers to have enough data to start playback."

In this part of the spec, the difference between HAVE_FUTURE_DATA and HAVE_ENOUGH_DATA isn't really clear.
Comment 4 Mark Watson 2012-08-17 15:35:03 UTC
(In reply to comment #2)
> The question can indeed be asked for other kinds of resources as well, but for
> e.g. file and HTTP resources the browser is in a much better position to
> estimate when HAVE_FUTURE_DATA makes sense.
> 

That's what I said, only I used more words ;-)

> Unless there are use cases for the canplaythrough event with MSE, I think we
> should simply spec it to never go beyond HAVE_FUTURE_DATA, for simplicity.

That would be fine for me.
Comment 5 Aaron Colwell (c) 2012-08-17 20:56:00 UTC
(In reply to comment #2)
> The question can indeed be asked for other kinds of resources as well, but for
> e.g. file and HTTP resources the browser is in a much better position to
> estimate when HAVE_FUTURE_DATA makes sense.
> 
> Unless there are use cases for the canplaythrough event with MSE, I think we
> should simply spec it to never go beyond HAVE_FUTURE_DATA, for simplicity.

I'm fine with this. Now I need to figure out a good place to put this information. :/
Comment 6 Aaron Colwell (c) 2012-08-17 21:01:49 UTC
(In reply to comment #3)
> http://dvcs.w3.org/hg/html-media/raw-file/tip/media-source/media-source.html#dom-append
> 
> Here's another similar requirement for append(data): "If the
> HTMLMediaElement.readyState attribute is HAVE_FUTURE_DATA and data causes all
> objects in activeSourceBuffers to have enough data to start playback."
> 
> In this part of the spec, the difference between HAVE_FUTURE_DATA and
> HAVE_ENOUGH_DATA isn't really clear.

When I was initially writing this I was thinking of HAVE_ENOUGH_DATA as roughly equivalent to reaching a prerolled state and taking HAVE_FUTURE_DATA as literally just having at least some data past the current position. I think I may accidentally be redefining their meaning here.

Perhaps this text should actually be for the HAVE_CURRENT_DATA -> HAVE_FUTURE_DATA transition and not have any way to transition to HAVE_ENOUGH_DATA.
Comment 7 Philip Jägenstedt 2012-08-20 08:48:10 UTC
I think that after going to HAVE_METADATA for the first time, readyState could be defined entirely in terms of currentTime and buffered:

* HAVE_METADATA: currentTime is not in a buffered range.
* HAVE_CURRENT_DATA: currentTime is exactly at the end of a buffered range.
* HAVE_FUTURE_DATA: currentTime is in a buffered range, but not at the end.
Comment 8 Philip Jägenstedt 2012-09-19 14:29:13 UTC
Note that a consequence of never going to HAVE_ENOUGH_DATA is that autoplay would do nothing, since that logic is in the transition to HAVE_ENOUGH_DATA in the HTML spec.
Comment 9 Adrian Bateman [MSFT] 2012-10-22 01:01:42 UTC
Triage: needs more thinking and discussion. Added tpac2012 whiteboard tag.
Comment 10 Adrian Bateman [MSFT] 2013-01-29 03:48:02 UTC
Having reviewed this in light of the recent discussions on the call, we think that we should keep support for HAVE_ENOUGH_DATA so that we can maintain the autoplay support.

One use case discussed is that of a library that creates a custom UI for a media element. Since MSE is designed simply to abstract away the network layer (i.e. how data gets into the buffer) it would be unfortunate if such a library needed to be substantially modified to work with MSE.

We think it should be left to implementations to determine how sophisticated they want to be with predicting when to switch to HAVE_ENOUGH_DATA. Some implementations may simply look at the amount of buffered content. Others may include the rate at which content is being appended by the application. This isn't very different from the way implementations can choose to consider the download rate for normal progressive download.
Comment 11 Aaron Colwell (c) 2013-01-29 18:17:08 UTC
(In reply to comment #10)

I agree with Adrian here. I don't think we should remove the transition to HAVE_ENOUGH_DATA and cause autoplay to no longer work.

Based on the discussion on the call today, I'll add a note to the existing text that clarifies that "enough data to ensure uninterrupted playback" is an implementation specific detail and the UA is free to use bytes buffered, time buffered, append rate, or any other metric it sees fit to determine when to transition to this state.

This transition is more about enabling autoplay than about "uninterrupted playback" since even in the non-MediaSource case, the UA can't actually ensure that uninterrupted playback will occur until it has received all of the data.
Comment 12 Aaron Colwell (c) 2013-01-30 21:21:34 UTC
Changes committed
https://dvcs.w3.org/hg/html-media/rev/aae26333e7d1

Added a definition for this term and non-normative notes to give guidance on what implementations might do to determine when to transition to HAVE_ENOUGH_DATA.