Bug 15140 - The video element's poster attribute requires clarification in relation to precedence
The video element's poster attribute requires clarification in relation to pr...
Status: RESOLVED FIXED
Product: HTML WG
Classification: Unclassified
Component: HTML5 spec
unspecified
PC All
: P2 normal
: ---
Assigned To: Silvia Pfeiffer
HTML WG Bugzilla archive list
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2011-12-10 19:09 UTC by Ian Devlin
Modified: 2013-01-22 04:28 UTC (History)
7 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Ian Devlin 2011-12-10 19:09:23 UTC
The video element's poster attribute requires clarification as to whether, if defined, it should always take precedence over the available video data.

Currently Firefox, Google Chrome, Opera and Safari all behave in the following manner:

- if a valid URL is provided in a video element's poster attribute, that image is always displayed, regardless of whether video data is available (e.g. the first frame)

Internet Explorer 9 however will display the first frame of the video, if it's available, overwriting the specified poster image.

The specification definition as it is allows IE9's interpretation to be defined as correct, even though the other browser's interpretation would seem to be more logical.

I request that the specification be changed to indicate that if a valid image file is provided in the poster attribute of a video element, then it should take precedence over anything else (when deciding what to display).
Comment 1 Philip Jägenstedt 2011-12-11 10:51:09 UTC
This was supposedly fixed in bug 12465, by adding "unless a frame of video has already been shown" to the definition. This is IMO too vague, so I'll reiterate what I said in https://www.w3.org/Bugs/Public/show_bug.cgi?id=12465#c2 :

In Opera, the poster is shown if and only if (poster_attr && !poster_failed && !(played_or_seeked && frame_decoded), where the states mean:

poster_attr: poster attribute is set
poster_failed: decoding the poster as an image failed
played_or_seeked: a play or seek has been requested
frame_decoded: a frame has been decoded (any frame)

All state is reset at the beginning of the resource selection algorithm.
Comment 2 Philip Jägenstedt 2011-12-11 10:52:36 UTC
(In reply to comment #1)

> In Opera, the poster is shown if and only if (poster_attr && !poster_failed &&
> !(played_or_seeked && frame_decoded)

) http://xkcd.com/859/
Comment 3 Ian 'Hixie' Hickson 2011-12-14 00:44:26 UTC
I don't understand what's vague in the spec here. Can you walk me through the reading of the spec that leads you to conclude that IE9 is conforming here?
Comment 4 Philip Jägenstedt 2011-12-14 08:58:57 UTC
I'll let Ian answer for himself, but here are some nits:

"When a video element is paused and the current playback position is the first frame of video, the element represents the poster frame, unless a frame of video has already been shown, in which case the element represents the frame of video corresponding to the current playback position."

This doesn't cover the case where you are paused at the beginning of the video and seek to 0, which will cause the first video frame to be shown in both Opera and Firefox, as any seeking shows the video frame. (http://software.hixie.ch/utilities/js/live-dom-viewer/saved/1278)

It also isn't clearly defined that the "unless a frame of video has already been shown" state is reset when loading a new resource (in the resource selection algorithm).

"The intrinsic width of a video element's playback area is the intrinsic width of the video resource, if that is available; otherwise it is the intrinsic width of the poster frame, if that is available; otherwise it is 300 CSS pixels."

This isn't accurate, the intrinsic size should be the size of the poster if the poster is showing and the size of the video otherwise. The above definition would cause a video with a poster showing to take the size of the video as soon as the video as reach HAVE_CURRENT_DATA. Both Opera and Firefox use the size of the poster until the video is shown.
Comment 5 Silvia Pfeiffer 2011-12-14 09:21:24 UTC
 
> "The intrinsic width of a video element's playback area is the intrinsic width
> of the video resource, if that is available; otherwise it is the intrinsic
> width of the poster frame, if that is available; otherwise it is 300 CSS
> pixels."
> 
> This isn't accurate, the intrinsic size should be the size of the poster if the
> poster is showing and the size of the video otherwise. The above definition
> would cause a video with a poster showing to take the size of the video as soon
> as the video as reach HAVE_CURRENT_DATA. Both Opera and Firefox use the size of
> the poster until the video is shown.

In the interest of not suddenly changing size when you hit the play button, it would be nicer to have the poster rescaled to fit within the video's size (pillarboxed or letterboxed as necessary) when the video's size is available. In both cases you will, however, have to deal with a size jump: one at the time when the video reaches HAVE_CURRENT_DATA and the other when the user initiates play.

The safest approach for different video and poster sizes is explicit CSS settings, so it's not a big problem. But it would indeed be nice if all browsers showed the same behaviour.
Comment 6 Philip Jägenstedt 2011-12-14 09:42:57 UTC
(In reply to comment #5)
> > "The intrinsic width of a video element's playback area is the intrinsic width
> > of the video resource, if that is available; otherwise it is the intrinsic
> > width of the poster frame, if that is available; otherwise it is 300 CSS
> > pixels."
> > 
> > This isn't accurate, the intrinsic size should be the size of the poster if the
> > poster is showing and the size of the video otherwise. The above definition
> > would cause a video with a poster showing to take the size of the video as soon
> > as the video as reach HAVE_CURRENT_DATA. Both Opera and Firefox use the size of
> > the poster until the video is shown.
> 
> In the interest of not suddenly changing size when you hit the play button, it
> would be nicer to have the poster rescaled to fit within the video's size
> (pillarboxed or letterboxed as necessary) when the video's size is available.
> In both cases you will, however, have to deal with a size jump: one at the time
> when the video reaches HAVE_CURRENT_DATA and the other when the user initiates
> play.

Well, I disagree and was quite deliberate when implementing it the way I did in Opera. It just seems to make good sense to use the intrinsic size of the frame that is actually shown, regardless of where it comes from -- a video that changes resolution mid-stream would also change the intrinsic size. Also, using the poster's size means that we have consistency with the preload=none case where the video size is not known.

> The safest approach for different video and poster sizes is explicit CSS
> settings, so it's not a big problem. But it would indeed be nice if all
> browsers showed the same behaviour.

With this I don't disagree!
Comment 7 Ian Devlin 2011-12-15 14:33:31 UTC
(In reply to comment #3)
> I don't understand what's vague in the spec here. Can you walk me through the
> reading of the spec that leads you to conclude that IE9 is conforming here?

The specification says the following:
"The poster attribute gives the address of an image file that the user agent can show while no video data is available"

The problem bit here is "can show while no video data is available".

Taking an example where a valid poster attribute has been defined and the video's metadata has been loaded:

IE9 has video data available, therefore it shows it, overriding the poster attribute. The fact that the video data doesn't contain the first frame, or does and it's blank, is irrelevant to IE9 as it has "video data available" and there's no requirement for it to keep showing the poster image based on the definition in the spec.

To me, whilst I don't agree with the behaviour, IE9 can be said to be conforming to the spec.

Now if the spec. said something like:

"The poster attribute gives the address of an image file that the user agent must show if present (or regardless of having video data available)".

Then IE9 wouldn't conform to the spec.

That said, since you don't think that IE9 is conforming, is this because further down "not having video data available" is defined as either "having nothing" or only "having metadata" - then perhaps IE9 is incorrectly defining "having metadata" as "having video data available".
If this is the case, then you are correct. IE9 doesn't conform to the spec.

I suspect we need input from Microsoft here on what IE's definition of "have video data" actually is.

Note:
From brief tests with IE9, setting the preload attribute to "auto" or "metadata" causes the poster attribute to be hidden. A value of "none" for preload correctly causes the poster to be displayed.
Comment 8 Philip Jägenstedt 2011-12-15 15:05:09 UTC
IE9's behavior (if it is as you say) is clear not conforming:

"When a video element is paused and the current playback position is the first
frame of video, the element represents the poster frame, unless a frame of
video has already been shown, in which case the element represents the frame of
video corresponding to the current playback position."
Comment 9 Ian Devlin 2011-12-15 16:35:17 UTC
(In reply to comment #8)
> IE9's behavior (if it is as you say) is clear not conforming:
> 
> "When a video element is paused and the current playback position is the first
> frame of video, the element represents the poster frame, unless a frame of
> video has already been shown, in which case the element represents the frame of
> video corresponding to the current playback position."

Well then, it does indeed seem that IE9's behaviour is non-conforming, despite what Frank Olivier tweeted (https://twitter.com/#!/frankolivier/status/145215734006550528)

I've set up a quick test page here: http://www.iandevlin.com/html5/video-poster-test.html so you can see it for yourselves (should my word not be enough! ;-)
Comment 10 Ian 'Hixie' Hickson 2012-01-31 23:20:48 UTC
(In reply to comment #4)
> 
> This doesn't cover the case where you are paused at the beginning of the video
> and seek to 0, which will cause the first video frame to be shown in both Opera
> and Firefox, as any seeking shows the video frame.
> (http://software.hixie.ch/utilities/js/live-dom-viewer/saved/1278)
> 
> It also isn't clearly defined that the "unless a frame of video has already
> been shown" state is reset when loading a new resource (in the resource
> selection algorithm).

Ok, will fix these.


> This isn't accurate, the intrinsic size should be the size of the poster if the
> poster is showing and the size of the video otherwise.

I disagree. The video playback area should be the size of the video, not the poster, so that it doesn't change size when you start playing.


> Both Opera and Firefox use the size of the poster until the video is shown.

IMHO those are bugs.


(In reply to comment #7)
> 
> The specification says the following:
> "The poster attribute gives the address of an image file that the user agent
> can show while no video data is available"
> 
> The problem bit here is "can show while no video data is available".

That text is non-normative (it doesn't give any requirements) so it's just a vague statement to introduce the feature. I don't really know how else to briefly explain what poster="" is for without going into as much detail as the normative conformance criteria given later.
Comment 11 Philip Jägenstedt 2012-02-06 11:40:11 UTC
(In reply to comment #10)
> (In reply to comment #4)

> > This isn't accurate, the intrinsic size should be the size of the poster if the
> > poster is showing and the size of the video otherwise.
> 
> I disagree. The video playback area should be the size of the video, not the
> poster, so that it doesn't change size when you start playing.
> 
> 
> > Both Opera and Firefox use the size of the poster until the video is shown.
> 
> IMHO those are bugs.

AFAICT there are 2 options, implementation-wise:

A. Use the size of the frame that is being displayed regardless of origin, or 300x150.

B. Use the (current/initial/maximum/?) size of the video if it is known, the size of the poster image if it is known, or 300x150.

For page authors, I can think of 4 behaviors one might want:

1. Small thumbnail (probably with preload="none") that expands to video size on playback.

2. A variable size player matching the video size and scaling the poster if necessary.

3. A fixed size player when you don't know the size of the videos or poster images at all.

4. Irrelevant, because the poster (if it exists) and video size match anyway.

With implementation A, behavior 1 is the default and 2 requires setting the size with CSS (on the server or in the loadeddata event).

With implementation B, behavior 1 requires setting the poster size with CSS (on the server or client by using an <img> element to get the size of the poster) and then removing the style in the loadeddata event. Behavior 2 almost works as the default, except that preload break everything -- with preload="none" you'll get something closer to behavior 1, but exactly when the video size becomes known in each preload state is deliberately vague in the spec and could differ between implementations. In other words, to make it robust work you have to set the size of the video in CSS in the same way as for implementation A.

Behaviors 3 and 4 are the same in either implementation, and I would be surprised if they together are not the most common cases.

Hopefully this makes it more clear why I picked the behavior I did, and why I think using the video size (when known) is not a good option.
Comment 12 Ian 'Hixie' Hickson 2012-07-12 20:03:13 UTC
(note to self: see first part of comment 10 for some required fixes, and consider comment 11)
Comment 13 contributor 2012-07-18 07:27:50 UTC
This bug was cloned to create bug 17979 as part of operation convergence.
Comment 14 Silvia Pfeiffer 2012-12-07 10:11:12 UTC
Ian Hickson has applied two fixes for this. See:
https://github.com/w3c/html/commit/3545cd658bd717dd8473c8b87a9c69c51846843a
https://github.com/w3c/html/commit/308777caa577852fd0e463f071b45c7d58b4c4c3

Ian Devlin: do you want to see these required for HTML5 or is HTML5.1 sufficient?
Comment 15 Silvia Pfeiffer 2013-01-22 04:28:21 UTC
EDITOR'S RESPONSE: This is an Editor's Response to your comment. If
you are satisfied with this response, please change the state of
this bug to CLOSED. If you have additional information and would
like the Editor to reconsider, please reopen this bug. If you would
like to escalate the issue to the full HTML Working Group, please
add the TrackerRequest keyword to this bug, and suggest title and
text for the Tracker Issue; or you may create a Tracker Issue
yourself, if you are able to do so. For more details, see this
document:   http://dev.w3.org/html5/decision-policy/decision-policy-v2.html

Status: Partially Accepted

Change Description:
https://github.com/w3c/html/commit/3545cd658bd717dd8473c8b87a9c69c51846843a
https://github.com/w3c/html/commit/308777caa577852fd0e463f071b45c7d58b4c4c3

Rationale: Adopted the WHATWG change for HTML5.1 WD