This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.

Bug 8731 - Consider expanding buffering control for media elements
Summary: Consider expanding buffering control for media elements
Status: CLOSED FIXED
Alias: None
Product: HTML WG
Classification: Unclassified
Component: pre-LC1 HTML5 spec (editor: Ian Hickson) (show other bugs)
Version: unspecified
Hardware: PC All
: P2 normal
Target Milestone: ---
Assignee: Ian 'Hixie' Hickson
QA Contact: HTML WG Bugzilla archive list
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-01-13 07:35 UTC by Maciej Stachowiak
Modified: 2010-10-04 14:31 UTC (History)
8 users (show)

See Also:


Attachments

Description Maciej Stachowiak 2010-01-13 07:35:06 UTC
Currently, the <audio> and <video> elements allow some limited hints of the authors intent on preloading the resource via the "autobuffer" attribute. The presence of the autobuffer attribute hints that the video is very likely to be played and the UA should consider buffering aggressively. Absence of the autobuffer attribute indicates either a hint *not* to buffer aggressively, or no hint at all. Per discussion on public-html, there are plausible use cases not addressed by this setup:

1) It seems valuable to distinguish no hint from explicit hint not to buffer, so that browsers can apply heuristics in the prior case without interfering with explicit author intent.

2) In some cases (very many videos with separate thumbnails provided via poster attirbute), it may be beneficial to request no preloading whatsoever, not even the metadata needed to get size and duration.

3) It may be useful to distinguish a hint to buffer truly as much as possible, from buffering just enough that playthrough is likely to succeed without further blocking.

Of the proposals made on the list, it seems the most thorough and best liked is:

* preload attribute with the following values:
  * preload=none - request to preload nothing, not even metadata
  * preload=metadata - request to preload only enough to get to the HAVE_METADATA state, and then stop
  * preload=all (or preload=full?) - request to buffer as much as possible - same as current autobuffer attribute
  * preload=playthrough - preload metadata plus however much of the media item seems enough to play through

Not all agree that the fourth value is worthwhile.

Other proposals involved a similar attribute named "buffering", with possibly fewer values, but on reflection playthrough seems like a better name.

Still other proposals suggested changing "autobuffer" to not be a strictly boolean attribute, such that "autobuffer=off" would reverse the meaning, but that seems to add unnecessary compatibility issues and is not particularly elegant syntax.
Comment 1 Philip Jägenstedt 2010-01-13 08:25:40 UTC
I agree with replacing autobuffer and the overall naming scheme of this suggestions.
Comment 2 Aryeh Gregor 2010-01-13 16:11:44 UTC
As an added comment, I think the spec should encourage browsers to use heuristics here if no attribute is specified.

(In reply to comment #0)
>   * preload=all (or preload=full?) - request to buffer as much as possible -
> same as current autobuffer attribute

The current autobuffer attribute doesn't actually say this is what should happen, it's just Firefox happens to implement it this way.

>   * preload=playthrough - preload metadata plus however much of the media item
> seems enough to play through
> 
> Not all agree that the fourth value is worthwhile.

Who said that?  As far as I can tell, the first one to suggest this value was Boris Zbarsky on January 9.  There were only three posts in the thread after that before you posted this bug, and all three implicitly endorsed the playthrough value.

If anything, I think the "none" value is the most dubious -- authors who don't read the spec might not realize it won't even fetch the duration, and will assume it behaves like "metadata".  Usually you'd only require one or two extra requests to get the metadata anyway, on par with an image or two, so it seems unlikely to be very useful, but might be confusing.  (Especially if Range: N,-M actually works on common servers, and/or X-Content-Duration or similar starts being used with Theora.)

(Maybe you meant "first" instead of "fourth"?)
Comment 3 Silvia Pfeiffer 2010-01-14 03:58:22 UTC
(In reply to comment #2)
> As an added comment, I think the spec should encourage browsers to use
> heuristics here if no attribute is specified.
> 
> (In reply to comment #0)
> >   * preload=all (or preload=full?) - request to buffer as much as possible -
> > same as current autobuffer attribute
> 
> The current autobuffer attribute doesn't actually say this is what should
> happen, it's just Firefox happens to implement it this way.

That's not strictly what Firefox does: it does progressive buffering. For that reason, "progressive" was also suggested as an alternative to "all" or "full" since it more accurately describes the means of buffering.
Comment 4 Silvia Pfeiffer 2010-01-27 02:04:37 UTC
Summarising how the current spec should change:

remove (or deprecate) "attribute boolean autobuffer;" from audio, video, media elements

introduce "attribute PreloadValues preload;" for audio, video, media elements with

interface PreloadValues {
  const unsigned short none=1;
  const unsigned short metadata=2;
  const unsigned short playthrough=3;
  const unsigned short all=4;
}

Comment 5 Silvia Pfeiffer 2010-01-27 03:13:21 UTC
(In reply to comment #4)
> Summarising how the current spec should change:
> 
> remove (or deprecate) "attribute boolean autobuffer;" from audio, video, media
> elements
> 
> introduce "attribute PreloadValues preload;" for audio, video, media elements
> with
> 
> interface PreloadValues {
>   const unsigned short none=1;
>   const unsigned short metadata=2;
>   const unsigned short playthrough=3;
>   const unsigned short all=4;
> }
> 

Alternatively, the preload could of course be a DOMString that can take on these values - not sure what the difference is and which would be preferable.
Comment 6 Maciej Stachowiak 2010-01-27 03:19:43 UTC
I think the string approach is more convenient and works better if new values are added in the future.
Comment 7 Philip Jägenstedt 2010-01-27 09:13:22 UTC
I don't think the distinction between playthrough and all is very useful. If you do in fact play through you will have loaded it all, after all...

preload=none I do support, and think it should suspend the resource fetch algorithm before step 2. This isn't only about bandwidth, setting up the decoding pipelines to get the duration and first frame is also resource intensive (details depends on implementation of course).
Comment 8 Aryeh Gregor 2010-01-27 14:29:05 UTC
(In reply to comment #7)
> I don't think the distinction between playthrough and all is very useful. If
> you do in fact play through you will have loaded it all, after all...

But if the user stops watching five minutes through a one-hour video, 'playthrough' would save a *lot* of bandwidth.  If anything, preload="all" should be dropped so there's only preload="playthrough".  But there's a difference; you can't freely seek if it's only loaded enough to playthrough, for instance (that might only be the first ten seconds).
Comment 9 Eric Carlson 2010-01-27 15:21:52 UTC
(In reply to comment #8)
> But if the user stops watching five minutes through a one-hour video,
> 'playthrough' would save a *lot* of bandwidth.  If anything, preload="all"
> should be dropped so there's only preload="playthrough".  
>
I also agree that preload="all" should be dropped, for the same reason we dropped the 'loaded' event: "loaded" is not a useful concept for media files that may be too large to fit on a client device, or may not be possible to load fully (eg. a live stream).

> But there's a
> difference; you can't freely seek if it's only loaded enough to playthrough,
> for instance (that might only be the first ten seconds).
> 
This is not necessarily true - some engines allow seeking to portions of a media resource that have not been loaded.
Comment 10 Aryeh Gregor 2010-01-27 15:43:29 UTC
(In reply to comment #9)
> I also agree that preload="all" should be dropped, for the same reason we
> dropped the 'loaded' event: "loaded" is not a useful concept for media files
> that may be too large to fit on a client device, or may not be possible to load
> fully (eg. a live stream).

Separate issue . . . "loaded" just won't do anything in many cases.  but preload="all" would mean to preload as much as is reasonable, which might be a lot more than preload="playthrough" in realistic scenarios.  E.g., a low-bitrate two-minute video might be loaded fully with no problem if preload="all" is set, but preload="playthrough" might only load the first five seconds.

> This is not necessarily true - some engines allow seeking to portions of a
> media resource that have not been loaded.

But if that part of the resource is not yet buffered, there will be lag when you seek that far.  I don't know if this is worth having the extra mode, but it was brought up.
Comment 11 Ian 'Hixie' Hickson 2010-02-14 04:36:33 UTC
Proposal based on reading the long mailing-list thread and all the comments on this bug:

We drop the autobuffer and autoplay attributes and replace them with one enumerated attribute called "loadhint", which takes the following values:

   loadhint=none*      -- load nothing
   loadhint=minimal*   -- load duration and size information only
   loadhint=autobuffer -- load at least enough to play through
   loadhint=autoplay   -- same as autoplay="" today

(* I don't especially like the values "none" and "minimal", if anyone has any clearly better suggestions please let me know)

Any objections?
Comment 12 Silvia Pfeiffer 2010-02-14 06:41:47 UTC
(In reply to comment #11)
> Proposal based on reading the long mailing-list thread and all the comments on
> this bug:
> 
> We drop the autobuffer and autoplay attributes and replace them with one
> enumerated attribute called "loadhint", which takes the following values:
> 
>    loadhint=none*      -- load nothing
>    loadhint=minimal*   -- load duration and size information only
>    loadhint=autobuffer -- load at least enough to play through
>    loadhint=autoplay   -- same as autoplay="" today
> 
> (* I don't especially like the values "none" and "minimal", if anyone has any
> clearly better suggestions please let me know)
> 
> Any objections?
> 


We could use "metadata" instead of "minimal", since that relates to the onmetadata event firing.

Does "autobuffer" mean: keep buffering until you reach the end of the file (which is what I am assuming)
or does it mean: keep buffering enough so you can play the next few seconds.

It would be great if we had a hint that stops a video from being buffered all the way through, even if the user hits pause after 3 seconds. I have this problem with YouTube all the time: once it starts playing, it will buffer everything, even if I only watch the first few seconds. That's an enormous waste of bandwidth. It would be better to just progressively buffer enough to be able to continue playing without getting stalled, but stop buffering on pause. Maybe a fifth value is needed - and that was what "playthrough" was meant to be.

Also, I wonder why you merged removal of the autoplay attribute with this. I think whether a video autoplays or not is a different dimension to whether it should buffer everything or not. Autoplay implies buffering beyond minimal, yes, but whether it implies a full buffering or a buffer of "playthrough" dimensions could be defined by the loadhint.

Comment 13 Maciej Stachowiak 2010-02-14 06:53:07 UTC
(In reply to comment #11)
> Proposal based on reading the long mailing-list thread and all the comments on
> this bug:
> 
> We drop the autobuffer and autoplay attributes and replace them with one
> enumerated attribute called "loadhint", which takes the following values:
> 
>    loadhint=none*      -- load nothing
>    loadhint=minimal*   -- load duration and size information only
>    loadhint=autobuffer -- load at least enough to play through
>    loadhint=autoplay   -- same as autoplay="" today
> 
> (* I don't especially like the values "none" and "minimal", if anyone has any
> clearly better suggestions please let me know)
> 

I'm not sure it makes sense to merge autoplay into the same attribute. autoplay affects behavior, it is not just a hint about how much to load. (It's true that strictly speaking UAs may ignore autoplay, but it seems to be at a lesser level of hint-ness and a separate concern from how much to preload).

I also do not like "autobuffer" as a value name. It's not any more or less automatic than the others. I think loadhint=playthrough would be better.

I also think "loadhint" as a name for the attribute is less clear than "preload".
Comment 14 Philip Jägenstedt 2010-02-14 08:07:40 UTC
I agree with Silvia about using "metadata" instead of "minimal".

I also agree with Maciej, we don't need to throw autoplay into the mix and preload is a better attribute name.

In summary, preload="" with 3 states:

none: stop at HAVE_NOTHING
metadata: stop at HAVE_METADATA (likely the browser default)
playthrough: stop at HAVE_ENOUGH_DATA

Possibly also play: stop at HAVE_CURRENT_DATA, but I don't think that's terribly useful.

I'm not sure what should happen if currentTime is > 0, should preload="playthrough" also try to buffer the data before currentTime? I'd say no.

The autoplay attribute implies, but it's possible to use preload="playthrough" without autoplay="" if playback will be started by script at a later time.
Comment 15 Ian 'Hixie' Hickson 2010-02-14 08:22:54 UTC
The idea of "autobuffer" is it would do whatever the browser thinks is best. If we split that into "download everything" and "download minimum needed for playthrough", then half the sites will use the wrong one (relative to what people want). If we only have the one option, then the user agent or the user can decide which to do, which seems to make more sense to me.

The idea of putting "autoplay" into this attribute is that it doesn't really make sense to autoplay without autobuffering. The autobuffer attribute is basically redundant if you want it to autoplay.
Comment 16 Silvia Pfeiffer 2010-02-14 09:35:46 UTC
(In reply to comment #15)
> The idea of "autobuffer" is it would do whatever the browser thinks is best. If
> we split that into "download everything" and "download minimum needed for
> playthrough", then half the sites will use the wrong one (relative to what
> people want). If we only have the one option, then the user agent or the user
> can decide which to do, which seems to make more sense to me.

So, the question really is: who knows best whether the best choice is to buffering "everything" or just "playthrough" - is it the author, the browser or the user? Maybe it is indeed the user.

How does the user get the choice? Would the browser vendor that cares about this offer such a difference as a browser preference? Or should "stop buffering" be an option in the video controls? Or the context menu? Is there something we can recommend?


 
> The idea of putting "autoplay" into this attribute is that it doesn't really
> make sense to autoplay without autobuffering. The autobuffer attribute is
> basically redundant if you want it to autoplay.

I still think that autoplay is a different dimension to buffering. It would need to be stated that if autoplay is used, loadhint is automatically set to autobuffer.
Comment 17 Ian 'Hixie' Hickson 2010-02-18 04:36:42 UTC
The idea of calling it loadhint="" was to emphasise that it's just a hint, but I guess many attributes are just hints and we haven't called any of them "hint". It's caused some confusion, but possibly no more than would be caused by having the attribute be confusingly named.

On the topic of autoplay vs this attribute, I really think the two aren't orthogonal. It really makes no sense to play without buffering, so the entire attribute is moot if we're autoplaying. Generally, that, to me, signals that it really should be the same attribute.

I'm not a fan of "playthrough"; it's likely to be the most-used value and I think it should have an easier name. I like "autobuffer" because it _is_ more "automatic" than the other values, in that it's telling the user agent to Do The Right Thing. I'm ok with just preload=buffer or something else simple like that if people think that "autobuffer" is that bad. (Similarly, preload=play might be ok if preload=autoplay strikes people as too wordy or confusing... personally I think autoplay makes sense since it makes it clear that something is going to happen. preload=play doesn't seem quite as obvious.)

So I thinking of the following proposal:

   preload=none
   preload=metadata
   preload=autobuffer
   preload=autoplay

(with the default being user-agent defined)

In reply to comment 16:
> How does the user get the choice? Would the browser vendor that cares about
> this offer such a difference as a browser preference? Or should "stop
> buffering" be an option in the video controls? Or the context menu?

I think the spec should mention all of these choices as valid implementation choices.
Comment 18 Maciej Stachowiak 2010-02-18 04:50:06 UTC
(In reply to comment #17)
> 
> On the topic of autoplay vs this attribute, I really think the two aren't
> orthogonal. It really makes no sense to play without buffering, so the entire
> attribute is moot if we're autoplaying. Generally, that, to me, signals that it
> really should be the same attribute.

I don't think that argument works in general. For example: in CSS, if you have "display: none" then your choice of font is moot, since you won't be rendering any text. But that doesn't mean that choice of font should be controlled by the "display" property, nor should display type be determined by font-face. To give another example, when the type attribute of the "input" element is "image", then the "src" attribute specifies the image to load. For other values of the type attribute it has no effect. Likewise the "multiple" attribute only applies for a subset of values of the "type" attribute. Yet it would not be the best design to make the choice of image, or the choice of multiple vs not, be part of the value of the type attribute.

It's not unexpected that attributes would sometimes interact, and that that not all possible combinations may make sense. I think it's only logical to combine things when they are truly affecting the same behavior. But "how much to load" and "whether to start playing automatically" are separate behaviors, even though they are contingent.
Comment 19 Eric Carlson 2010-02-18 04:54:26 UTC
(In reply to comment #18)
> It's not unexpected that attributes would sometimes interact, and that that not
> all possible combinations may make sense. I think it's only logical to combine
> things when they are truly affecting the same behavior. But "how much to load"
> and "whether to start playing automatically" are separate behaviors, even
> though they are contingent.
> 
I agree completely.
Comment 20 Philip Jägenstedt 2010-02-18 05:19:05 UTC
(In reply to comment #17)

> So I thinking of the following proposal:
> 
>    preload=none
>    preload=metadata
>    preload=autobuffer
>    preload=autoplay

I'd be OK with this but would also be OK with a solution that keeps autoplay as a separate attribute.
 
> (with the default being user-agent defined)

Surely the default can only be one of metadata and autobuffer?
Comment 21 Simon Pieters 2010-02-18 14:17:33 UTC
I think we should leave autoplay alone. autoplay is implemented and shipped in multiple browsers, and I see little value in changing the syntax for autoplay.
Comment 22 Aryeh Gregor 2010-02-18 18:51:20 UTC
(In reply to comment #18)
> It's not unexpected that attributes would sometimes interact, and that that not
> all possible combinations may make sense. I think it's only logical to combine
> things when they are truly affecting the same behavior. But "how much to load"
> and "whether to start playing automatically" are separate behaviors, even
> though they are contingent.

I agree with this.  It makes the most sense to me as a separate attribute.
Comment 23 Silvia Pfeiffer 2010-02-19 00:09:25 UTC
(In reply to comment #17)
> On the topic of autoplay vs this attribute, I really think the two aren't
> orthogonal. It really makes no sense to play without buffering, so the entire
> attribute is moot if we're autoplaying. Generally, that, to me, signals that it
> really should be the same attribute.

What if in future we decide that we want to versions of autoplay: one where we only buffer a certain amount ahead and one where we buffer the full file (something that I still think would be nice to have now, but we don't seem to be able to find consensus for here). There clearly the buffering strategy is independent of the choice to autoplay.


> I'm not a fan of "playthrough"; it's likely to be the most-used value and I
> think it should have an easier name. I like "autobuffer" because it _is_ more
> "automatic" than the other values, in that it's telling the user agent to Do
> The Right Thing. I'm ok with just preload=buffer or something else simple like
> that if people think that "autobuffer" is that bad. (Similarly, preload=play
> might be ok if preload=autoplay strikes people as too wordy or confusing...
> personally I think autoplay makes sense since it makes it clear that something
> is going to happen. preload=play doesn't seem quite as obvious.)

We could just say "preload=auto" since the buffering aspect is already in the word preload and auto says it's up to the UA to do what they like and find appropriate.


> So I thinking of the following proposal:
> 
>    preload=none
>    preload=metadata
>    preload=autobuffer
>    preload=autoplay
> 
> (with the default being user-agent defined)

How about:

    preload=none
    preload=metadata
    preload=auto
    preload=progressive
    preload=full

The last two are the distinction that I made above, which I think are values that - if we cannot agree on now - we should at least keep open as an option for the future.

Also, "auto" should be the default, IMO.
Comment 24 Ian 'Hixie' Hickson 2010-02-25 09:15:05 UTC
EDITOR'S RESPONSE: This is an Editor's Response to your comment. If you are satisfied with this response, please change the state of this bug to CLOSED. If you have additional information and would like the editor to reconsider, please reopen this bug. If you would like to escalate the issue to the full HTML Working Group, please add the TrackerRequest keyword to this bug, and suggest title and text for the tracker issue; or you may create a tracker issue yourself, if you are able to do so. For more details, see this document:
   http://dev.w3.org/html5/decision-policy/decision-policy.html

Status: Partially Accepted
Change Description: see diff given below
Rationale: The described problem is valid.

I went with:

    preload=none
    preload=metadata
    preload=auto

I'm not convinced there's a need for more than that — I don't see how an author can know whether prebuffering the whole thing or prebuffering the minimum required for smooth playback with just-in-time download is best, since that seems to be entirely a client-side thing.
Comment 25 Ian 'Hixie' Hickson 2010-02-25 09:19:20 UTC
http://html5.org/tools/web-apps-tracker?from=4810&to=4811
Comment 26 Philip Jägenstedt 2010-02-26 04:27:08 UTC
(In reply to comment #25)
> http://html5.org/tools/web-apps-tracker?from=4810&to=4811
> 

For the record, I am very satisfied with this solution. There are still bugs, which I will file separately.
Comment 27 Maciej Stachowiak 2010-03-14 18:17:22 UTC
(In reply to comment #24)

> 
> I went with:
> 
>     preload=none
>     preload=metadata
>     preload=auto
> 

Works for me.