Recent discussion in the MF WG:
http://lists.w3.org/Archives/Public/public-media-fragment/2014Feb/0000.html
http://lists.w3.org/Archives/Public/public-media-fragment/2014Feb/0002.html

The MF spec says:
http://www.w3.org/TR/media-frags/#media-fragment-display

"For a temporal URI fragment, it is recommended to start playback at a time offset that equals to the start of the fragment and pause at the end of the fragment."

It turns out that WebKit, Blink and Gecko do pause when playing past the end of the fragment, but at least Blink doesn't try to pause exactly. Also, it only pauses once and then forgets about the fragment end time. It also ignores the fragment end time if there's a media controller attached, but that seems weird.

It would be nice if the HTML spec were explicit about what to do here, since two independent implementations have happened in absence of such advice and they probably aren't completely interoperable. In particular any interaction with looping, MediaController and what happens when reaching the time a second time needs attention.

(I know that some of this has been discussed before, but I can't find the bug or list thread where it happened.)

It was my intention to not do anything with end times. If we want to do something with end times, I don't really know how to do it. Any suggestions?

I actually don't have a suggestion that I prefer over "do nothing at all", so I'm going to be a really bad ambassador for media fragments.

I've pointed Chris Double (Mozilla implementor) and people from public-media-fragment to this bug. (Eric Carlson is already in CC.)

(In reply to Ian 'Hixie' Hickson from comment #2)
> It was my intention to not do anything with end times. If we want to do
> something with end times, I don't really know how to do it. Any suggestions?

In discussions in the Media Fragment WG and from what I've heard from author requests over the years, pausing at the end of the fragment time (as recommended in the MF URI spec) is the preferred action.

This allows queueing up several media fragment URIs to create a playlist.

Accuracy of the pausing position is desirable but will be just like accurate seeking a quality of implementation issue.

(In reply to Philip Jägenstedt from comment #1)
> Recent discussion in the MF WG:
> http://lists.w3.org/Archives/Public/public-media-fragment/2014Feb/0000.html
> http://lists.w3.org/Archives/Public/public-media-fragment/2014Feb/0002.html
> 
> The MF spec says:
> http://www.w3.org/TR/media-frags/#media-fragment-display
> 
> "For a temporal URI fragment, it is recommended to start playback at a time
> offset that equals to the start of the fragment and pause at the end of the
> fragment."
> 
> It turns out that WebKit, Blink and Gecko do pause when playing past the end
> of the fragment, but at least Blink doesn't try to pause exactly.

Can you please clarify the last 2 sentences and what do you mean by "Blink doesn't try to pause exactly"? Do you have an example of a video and media fragment where the pause will be different between Blink (Chromium?) and WebKit/Gecko?

(In reply to Silvia Pfeiffer from comment #4)
> (In reply to Ian 'Hixie' Hickson from comment #2)
> > It was my intention to not do anything with end times. If we want to do
> > something with end times, I don't really know how to do it. Any suggestions?
> 
> In discussions in the Media Fragment WG and from what I've heard from author
> requests over the years, pausing at the end of the fragment time (as
> recommended in the MF URI spec) is the preferred action.
> 
> This allows queueing up several media fragment URIs to create a playlist.
> 
> Accuracy of the pausing position is desirable but will be just like accurate
> seeking a quality of implementation issue.

+1. And to come back to the original thread (http://lists.w3.org/Archives/Public/public-media-fragment/2014Feb/0000.html), there is also the question of what to do when the loop attribute is true and the src attribute of a audio/video/source element is a media fragment uri, i.e. should the player loop over the video fragment or not. Again, accuracy (like seeking) is less an issue.

Here's a quick test and the results I got:

<video autoplay onpause="alert(this.currentTime)">
<source src="https://archive.org/download/anita-leker-med-kameran/anita-leker-med-kameran.mp4#t=0,1" type="video/mp4">
<source src="https://archive.org/download/anita-leker-med-kameran/anita-leker-med-kameran.ogv#t=0,1" type="video/ogg">
</video>

Opera 20 (Blink): 1.146967 1.145042 1.235672
Safari 7 (WebKit): 1.0778980255126953 1.1833858489990234 1.1791207790374756
Firefox 30 (Gecko): 1.010068 1.010068 1.010068

I don't know how Gecko has implemented this, but clearly it's the best here.

For interop it should always pause at exactly 1, of course.

(In reply to Raphael Troncy from comment #5)
> Can you please clarify the last 2 sentences and what do you mean by "Blink
> doesn't try to pause exactly"? Do you have an example of a video and media
> fragment where the pause will be different between Blink (Chromium?) and
> WebKit/Gecko?

I was just being conservative because I had only tested Blink. See my previous comment for test results for all 3 engines that currently have support.

I think that conceptually a fragment is informative to the end system "this is the whole document and this fragment is what I want to bring your attention to".  So in HTML we jump to what the fragment points to, but the user can scroll away if they like.  Timed media has necessarily a concept of end-time, so I think that pause-on-end is the right behavior, and if a scroll-bar is shown it should offer all the content but start at the indicated start-time. I think any 'loop' behavior specified in the *media* would not be relevant (they wanted the media looped, not the fragment) but any loop behavior specified at the same level as the fragment should probably be observed.

(In reply to David Singer from comment #9)
> I think any 'loop' behavior specified in the *media*
> would not be relevant (they wanted the media looped, not the fragment) but
> any loop behavior specified at the same level as the fragment should
> probably be observed.

I didn't understand this bit. What is "at the same level as the fragment"? In concrete spec/implementation terms, how should looping interact with media fragments?

If we want to support this, I need a proposal for how exactly it should be specced, including rules for handling all the edge cases like seeking and media controllers with slaves with conflicting ranges and out-of-range ranges and interaction with pause-on-exit cues and interaction with user seeking and so on and so forth. No need to indicate support. :-)

Thanks Philip for the quick test, this is really useful. I will perform more systematic tests with other fragment ranges in order to have a feel of how divergent rendering engines are pausing when requested.

Ian, there are a number of test cases from the Media Fragments WG (yes, a spec which has been tested!) that cover what you're asking, not everything though. Feel free to re-use

Test cases are useful for implementors, but I'm talking more about a spec-level proposal for how it should integrate with the current loading mechanism.

(In reply to Philip Jägenstedt from comment #10)
> (In reply to David Singer from comment #9)
> > I think any 'loop' behavior specified in the *media*
> > would not be relevant (they wanted the media looped, not the fragment) but
> > any loop behavior specified at the same level as the fragment should
> > probably be observed.
> 
> I didn't understand this bit. What is "at the same level as the fragment"?
> In concrete spec/implementation terms, how should looping interact with
> media fragments?

I was confused, too.

What I would prefer is that if

<video src="media.webm#t=5,10" loop autoplay></video>

plays from second 5 as start time to second 10 as end time and then loops over this time fragment. It is also what users expect.

The fragment end time is only relevant as long as there is no user interaction. I.e. if the user decides to seek, it will seek to whatever position the user picked, even if it was outside the fragment.

If the browser has a loop button (Safari does), hitting that loop button could potentially reset the looping behaviour. Not sure - maybe somebody from Apple could chime in.

As for all the edge cases on media controllers etc, I would suggest to handle the end point like the end of a pause-on-exit cue.

> If the browser has a loop button (Safari does), hitting that loop button
> could potentially reset the looping behaviour. Not sure - maybe somebody
> from Apple could chime in.
> 
Safari doesn't have a loop button. Maybe you are thinking of the "seek back 30 seconds" button?

(In reply to Eric Carlson from comment #15)
> > If the browser has a loop button (Safari does), hitting that loop button
> > could potentially reset the looping behaviour. Not sure - maybe somebody
> > from Apple could chime in.
> > 
> Safari doesn't have a loop button. Maybe you are thinking of the "seek back
> 30 seconds" button?

Correct, thanks for clarifying my confusion!

(In reply to Silvia Pfeiffer from comment #14)
> (In reply to Philip Jägenstedt from comment #10)
> > (In reply to David Singer from comment #9)
> > > I think any 'loop' behavior specified in the *media*
> > > would not be relevant (they wanted the media looped, not the fragment) but
> > > any loop behavior specified at the same level as the fragment should
> > > probably be observed.
> > 
> > I didn't understand this bit. What is "at the same level as the fragment"?
> > In concrete spec/implementation terms, how should looping interact with
> > media fragments?
> 
> I was confused, too.
> 
> What I would prefer is that if
> 
> <video src="media.webm#t=5,10" loop autoplay></video>
> 
> plays from second 5 as start time to second 10 as end time and then loops
> over this time fragment. It is also what users expect.

Is there a use case for this that isn't already served well by Web Audio API?

Implementing it seems non-trivial, so I don't think not surprising Web authors is quite enough reason to do it.

(In reply to Philip Jägenstedt from comment #17)
> (In reply to Silvia Pfeiffer from comment #14)
> > What I would prefer is that if
> > 
> > <video src="media.webm#t=5,10" loop autoplay></video>
> > 
> > plays from second 5 as start time to second 10 as end time and then loops
> > over this time fragment. It is also what users expect.
> 
> Is there a use case for this that isn't already served well by Web Audio API?
> 
> Implementing it seems non-trivial, so I don't think not surprising Web
> authors is quite enough reason to do it.

What is the relationship between what the Web Audio API specifies and the <video> element? We are talking about looping over a video fragment (not over a audio fragment)

I mentioned Web Audio API because it can do gapless looping, and gapless is the hard bit.

For video, the question is the same: what are the use cases, and are they compelling enough to justify the implementation complexity?

(In reply to Philip Jägenstedt from comment #10)
> (In reply to David Singer from comment #9)
> > I think any 'loop' behavior specified in the *media*
> > would not be relevant (they wanted the media looped, not the fragment) but
> > any loop behavior specified at the same level as the fragment should
> > probably be observed.
> 
> I didn't understand this bit. What is "at the same level as the fragment"?
> In concrete spec/implementation terms, how should looping interact with
> media fragments?

Sorry.  Two cases:
1) the media file has an internal structure that says "loop me" or "loop this track".  Something outside the file (a media fragment) says "but only select seconds 5 to 10 of this media".

2) where the fragment is specified, there also is specified "and loop this" (e.g. in the markup)

In case (1) the person writing the 'loop me' was not aware of a later selection of a 5-second interval; the intent was to loop the entire resource.  Now we've selected a slice, it's not at all obvious it ought to be looped.

In case (2), it's at least plausible that the fragment and the loop instruction were written with mutual awareness that they were there.  Maybe looping is OK

(In reply to Philip Jägenstedt from comment #19)
> I mentioned Web Audio API because it can do gapless looping, and gapless is
> the hard bit.
> 
> For video, the question is the same: what are the use cases, and are they
> compelling enough to justify the implementation complexity?

We are sort of looping here (pun intended). Use cases have been expressed, voiced, written several times already. So I will repeat, in the case of a loop, a user wants to share a particularly fun and short moment of a video (a few seconds) that is best experiences when viewed multiple times in a loop (think of a Vine)

(In reply to David Singer from comment #20)
> (In reply to Philip Jägenstedt from comment #10)
> > (In reply to David Singer from comment #9)
> > > I think any 'loop' behavior specified in the *media*
> > > would not be relevant (they wanted the media looped, not the fragment) but
> > > any loop behavior specified at the same level as the fragment should
> > > probably be observed.
> > 
> > I didn't understand this bit. What is "at the same level as the fragment"?
> > In concrete spec/implementation terms, how should looping interact with
> > media fragments?
> 
> Sorry.  Two cases:
> 1) the media file has an internal structure that says "loop me" or "loop
> this track".  Something outside the file (a media fragment) says "but only
> select seconds 5 to 10 of this media".
> 
> 2) where the fragment is specified, there also is specified "and loop this"
> (e.g. in the markup)
> 
> In case (1) the person writing the 'loop me' was not aware of a later
> selection of a 5-second interval; the intent was to loop the entire
> resource.  Now we've selected a slice, it's not at all obvious it ought to
> be looped.
> 
> In case (2), it's at least plausible that the fragment and the loop
> instruction were written with mutual awareness that they were there.  Maybe
> looping is OK

How many (1) cases have you ever observed ... on the web ?

(In reply to Raphael Troncy from comment #21)
> (In reply to Philip Jägenstedt from comment #19)
> > I mentioned Web Audio API because it can do gapless looping, and gapless is
> > the hard bit.
> > 
> > For video, the question is the same: what are the use cases, and are they
> > compelling enough to justify the implementation complexity?
> 
> We are sort of looping here (pun intended). Use cases have been expressed,
> voiced, written several times already. So I will repeat, in the case of a
> loop, a user wants to share a particularly fun and short moment of a video
> (a few seconds) that is best experiences when viewed multiple times in a
> loop (think of a Vine)

OK. I don't think that's worth the added complexity, but let's stop going in circles now and leave it to Ian.

As far as I'm concerned, this bug is blocked pending a concrete proposal (see comment 11 and comment 13). (I'm also unclear on whether there's implementation interest here in the first place.)

I'm interested in removing the code in Blink, but am waiting on the use counters to reach the stable channel:
http://www.chromestatus.com/metrics/feature/timeline/popularity/281
http://www.chromestatus.com/metrics/feature/timeline/popularity/282

(In reply to Ian 'Hixie' Hickson from comment #11)
> If we want to support this, I need a proposal for how exactly it should be
> specced, including rules for handling all the edge cases like seeking and
> media controllers with slaves with conflicting ranges and out-of-range
> ranges and interaction with pause-on-exit cues and interaction with user
> seeking and so on and so forth. No need to indicate support. :-)

The temporal URI in a media element should be interpreted such that when the resource is loaded, it's playback region is limited to the media fragment interval, i.e. it starts playing from the media fragment start time and stops playing (i.e. pause) when the browser can tell that it has reached the end time (where "reached" probably means: reached it or just gone past it).

Once a seek occurs, the restriction of playing to the media fragment interval is lifted.

As for the case with a media controller, it would sure be nice if it was possible to play two media fragmented media resources in parallel - again as long as no seek occurs. Can we slave the media elements together at the start time of the media fragment? Could that become the "timeline offset", i.e. the 0-point? What are the implications?

(In reply to Philip Jägenstedt from comment #25)
> I'm interested in removing the code in Blink, but am waiting on the use
> counters to reach the stable channel:
> http://www.chromestatus.com/metrics/feature/timeline/popularity/281
> http://www.chromestatus.com/metrics/feature/timeline/popularity/282

These have been in Chrome's stable channel since May 20, and show that both the start and end time are essentially unused and removal would be safe.

If no proposal for how to handle the end time is forthcoming, I suggest WONTFIXing this, possibly with a note of non-support in the spec, and I'll (try to) remove the end time logic from Blink.

(In reply to Philip Jägenstedt from comment #27)
> 
> If no proposal for how to handle the end time is forthcoming, I suggest
> WONTFIXing this, possibly with a note of non-support in the spec, and I'll
> (try to) remove the end time logic from Blink.

Let's please not do that. I regard this as one of the features that is really valuable and Web developers haven't discovered yet to make the most of. Don't forget that we are only just this year seeing people move to using HTML5 video elements over Flash as their first choice. It's not surprising that they are not yet making use of all the features that HTML5 video offers.

As currently interoperably implemented, Firefox, Chrome and Safari all start at the time offset and pause and the end time. (I don't have access to IE, so can't test what it does.) Why is it a problem standardising on what is now interoperably implemented?

(In reply to Silvia Pfeiffer from comment #28)
> Why is it a problem standardising on what is now interoperably implemented?

We'd need something that can be implemented as pausing at the exact correct time while dealing with the edge cases in comment #11.

The existing implementation in Blink/WebKit is not very accurate at all, and fixing it doesn't seem to be worth the effort.

(In reply to Philip Jägenstedt from comment #29)
> (In reply to Silvia Pfeiffer from comment #28)
> > Why is it a problem standardising on what is now interoperably implemented?
> 
> We'd need something that can be implemented as pausing at the exact correct
> time while dealing with the edge cases in comment #11.

It's already implemented, so something already deals with at least some of these edge cases. We should be able to work out the rest.

 
> The existing implementation in Blink/WebKit is not very accurate at all, and
> fixing it doesn't seem to be worth the effort.

I believe the accuracy of Blink/WebKit is sufficient. Seeking is not accurate either and we don't have any plans to remove it for its inaccuracies. We just deal with them.

(In reply to Silvia Pfeiffer from comment #30)
> (In reply to Philip Jägenstedt from comment #29)
> > (In reply to Silvia Pfeiffer from comment #28)
> > > Why is it a problem standardising on what is now interoperably implemented?
> > 
> > We'd need something that can be implemented as pausing at the exact correct
> > time while dealing with the edge cases in comment #11.
> 
> It's already implemented, so something already deals with at least some of
> these edge cases. We should be able to work out the rest.

The WebKit/Blink implementation is equivalent to this:

var endTime = NaN;
video.onloadedmetadata = function() {
    endTime = parseEndTimeFromURL(video.currentSrc); // could be NaN
};
video.ontimeupdate = function() {
    if (!isNaN(endTime) && video.currentTime >= endTime && video.playbackRate >= 0) {
        endTime = NaN;
        if (!video.controller && !video.paused) {
            video.pause()
        }
    }
};

This can never be made accurate because it doesn't involve the media framework, so specifying something like this is not an option I think.

> > The existing implementation in Blink/WebKit is not very accurate at all, and
> > fixing it doesn't seem to be worth the effort.
> 
> I believe the accuracy of Blink/WebKit is sufficient. Seeking is not
> accurate either and we don't have any plans to remove it for its
> inaccuracies. We just deal with them.

Seeking is accurate to the frame level, not sure what you mean here...

(In reply to Philip Jägenstedt from comment #31)
>
> The WebKit/Blink implementation is equivalent to this:
> 
> var endTime = NaN;
> video.onloadedmetadata = function() {
>     endTime = parseEndTimeFromURL(video.currentSrc); // could be NaN
> };
> video.ontimeupdate = function() {
>     if (!isNaN(endTime) && video.currentTime >= endTime &&
> video.playbackRate >= 0) {
>         endTime = NaN;
>         if (!video.controller && !video.paused) {
>             video.pause()
>         }
>     }
> };

Excellent. That should completely define all your edge cases from comment #11.


> This can never be made accurate because it doesn't involve the media
> framework, so specifying something like this is not an option I think.

OK, so all you need is the ability to register a pause callback with the media framework to pause when normal playback reaches the endTime. That will improve accuracy and not change any of the behaviour.

(In reply to Silvia Pfeiffer from comment #32)
> (In reply to Philip Jägenstedt from comment #31)
> >
> > The WebKit/Blink implementation is equivalent to this:
> > 
> > var endTime = NaN;
> > video.onloadedmetadata = function() {
> >     endTime = parseEndTimeFromURL(video.currentSrc); // could be NaN
> > };
> > video.ontimeupdate = function() {
> >     if (!isNaN(endTime) && video.currentTime >= endTime &&
> > video.playbackRate >= 0) {
> >         endTime = NaN;
> >         if (!video.controller && !video.paused) {
> >             video.pause()
> >         }
> >     }
> > };
> 
> Excellent. That should completely define all your edge cases from comment
> #11.

Not really, because this check only happens every ~250 ms it's not going to be reliable what happens e.g. when endTime is very close to duration and the video is looping.

> > This can never be made accurate because it doesn't involve the media
> > framework, so specifying something like this is not an option I think.
> 
> OK, so all you need is the ability to register a pause callback with the
> media framework to pause when normal playback reaches the endTime. That will
> improve accuracy and not change any of the behaviour.

That's actually a fair bit of work, and not possible on all platforms, e.g. Android's MediaPlayer doesn't seem to have a way to pause at a given time. This, as well as the relative obscurity of the use case, is why it think it's time better spent to just drop the pause-at-end feature.

(In reply to Silvia Pfeiffer from comment #26)
> Once a seek occurs, the restriction of playing to the media fragment
> interval is lifted.

I doubt this is what you want, since specifying a start time causes a seek, as does synchronising with a media controller, as do some more obscure things like the earliest possible position being moved to past the current playback position (which could happen at any time). In effect, this is actually what the spec requires now (i.e., not supporting end times).

(In reply to Ian 'Hixie' Hickson from comment #34)
> (In reply to Silvia Pfeiffer from comment #26)
> > Once a seek occurs, the restriction of playing to the media fragment
> > interval is lifted.
> 
> I doubt this is what you want, since specifying a start time causes a seek,
> as does synchronising with a media controller, as do some more obscure
> things like the earliest possible position being moved to past the current
> playback position (which could happen at any time). In effect, this is
> actually what the spec requires now (i.e., not supporting end times).

Ah, no, I meant: a user-interaction initiated seek.

(In reply to Philip Jägenstedt from comment #33)
> (In reply to Silvia Pfeiffer from comment #32)
>
> > Excellent. That should completely define all your edge cases from comment
> > #11.
> 
> Not really, because this check only happens every ~250 ms it's not going to
> be reliable what happens e.g. when endTime is very close to duration and the
> video is looping.

It would need to deal with that edge case, too, even if it doesn't always hit it.


> > > This can never be made accurate because it doesn't involve the media
> > > framework, so specifying something like this is not an option I think.
> > 
> > OK, so all you need is the ability to register a pause callback with the
> > media framework to pause when normal playback reaches the endTime. That will
> > improve accuracy and not change any of the behaviour.
> 
> That's actually a fair bit of work, and not possible on all platforms, e.g.
> Android's MediaPlayer doesn't seem to have a way to pause at a given time.
> This, as well as the relative obscurity of the use case, is why it think
> it's time better spent to just drop the pause-at-end feature.

What about pause-on-exit of cues then? They have the same problem.

Essentially, we have to solve this problem anyway.

(In reply to Silvia Pfeiffer from comment #36)
> What about pause-on-exit of cues then? They have the same problem.
> 
> Essentially, we have to solve this problem anyway.

Sure, if end time handling is specified, then specifying it in a way that behaves like a pauseOnExit cue is probably a good idea.

Off-topically, I'm not a fan of pauseOnExit either:
http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2010-July/281552.html

Since I do not want this myself as a spec editor or implementor, I'm WONTFIXing this. This need not be the final say, of course, if you disagree, please prepare a spec proposal and submit a pull request for https://github.com/whatwg/html

In a few months (or years), if nothing has changed, I will attempt to remove the end time handling from Blink.