Bug 18587 - Define clearly what data is removed when setting an explicit duration
Define clearly what data is removed when setting an explicit duration
Status: RESOLVED FIXED
Product: HTML WG
Classification: Unclassified
Component: Media Source Extensions
unspecified
PC Linux
: P2 normal
: ---
Assigned To: Aaron Colwell (c)
HTML WG Bugzilla archive list
:
Depends on: 18709
Blocks:
  Show dependency treegraph
 
Reported: 2012-08-16 08:39 UTC by Philip Jägenstedt
Modified: 2012-11-28 18:31 UTC (History)
4 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Philip Jägenstedt 2012-08-16 08:39:27 UTC
http://dvcs.w3.org/hg/html-media/raw-file/tip/media-source/media-source.html#dom-duration

4. Remove all media data that is beyond the new duration from all SourceBuffer objects in sourceBuffers.

It's not clear to which granularity this should be done. Should any segments which extend beyond the new duration be removed, or should enough data be "removed" so that playback stops exactly at the new duration. I say "removed" because this is actually a case of attaching metadata to the stream to stop playing at a certain point, since a perfect cut can only be achieved after decoding.

Fulfilling this requirement well would make the implementation more complicated, so there would have to be strong use cases for it.
Comment 1 Aaron Colwell (c) 2012-08-17 22:23:31 UTC
(In reply to comment #0)
> http://dvcs.w3.org/hg/html-media/raw-file/tip/media-source/media-source.html#dom-duration
> 
> 4. Remove all media data that is beyond the new duration from all SourceBuffer
> objects in sourceBuffers.
> 
> It's not clear to which granularity this should be done. Should any segments
> which extend beyond the new duration be removed, or should enough data be
> "removed" so that playback stops exactly at the new duration. I say "removed"
> because this is actually a case of attaching metadata to the stream to stop
> playing at a certain point, since a perfect cut can only be achieved after
> decoding.
> 
> Fulfilling this requirement well would make the implementation more
> complicated, so there would have to be strong use cases for it.

The initial intent was to simply remove anything that spans the duration or
goes beyond it. I think this could be a quality of implementation issue with
respect to whether a full frame is dropped and silence is inserted or whether a
frame that spans the new duration is simply truncated.

How about I add a step after step 4 along these lines
"5. If the new duration occurs in the middle of an audio frame, the UA may drop
this frame and insert enough silence to achieve the desired duration or it may
keep the frame and truncate it to the desired duration."
Comment 2 Philip Jägenstedt 2012-08-20 09:10:57 UTC
Phrased like that, it sounds like an implementation would rewrite the overlapping segment to drop excess data before giving it to the decoding pipeline. Implementing TextTrackCue.pauseOnExit already requires pausing at the granularity of a single audio sample or video frame, so I think that if anyone were to implement this at all, that's a more likely implementation technique.

If there are implementors interested in handling this situation well, I think it should be spec'd in terms of pausing at duration and clamping buffered duration, leaving the technique open.
Comment 3 Aaron Colwell (c) 2012-09-14 18:00:45 UTC
(In reply to comment #2)
> Phrased like that, it sounds like an implementation would rewrite the
> overlapping segment to drop excess data before giving it to the decoding
> pipeline. Implementing TextTrackCue.pauseOnExit already requires pausing at the
> granularity of a single audio sample or video frame, so I think that if anyone
> were to implement this at all, that's a more likely implementation technique.
> 
> If there are implementors interested in handling this situation well, I think
> it should be spec'd in terms of pausing at duration and clamping buffered
> duration, leaving the technique open.

Some stuff in the spec has changed since I last commented on this bug but I believe this issue is still present. I think this bug is now referring to step 3 of the duration change algorithm (http://dvcs.w3.org/hg/html-media/raw-file/tip/media-source/media-source.html#duration-change-algorithm).

Initially I thought it made sense to add a step after step 3 to talk about this, but it doesn't seem appropriate since this information isn't about the duration change algorithm itself. Would adding a note below step 3 along these lines be ok? 

Note: This preserves audio frames that start before and end after the duration. The user agent must end playback at duration even if the audio frame extends beyond this time.
Comment 4 Philip Jägenstedt 2012-09-17 08:07:30 UTC
Adding a non-normative note cannot make a difference in what "Remove all media data with timestamps that are greater than new duration from all SourceBuffer objects in sourceBuffers" means. I don't see why removing data or the concept of audio frames need to be mentioned at all, if the desired effect is that playback should only proceed to duration. Certainly an implementation would be wise to throw away as much data as it can, which may be worth mentioning in a note.
Comment 5 Aaron Colwell (c) 2012-09-21 22:51:26 UTC
(In reply to comment #4)
> Adding a non-normative note cannot make a difference in what "Remove all media
> data with timestamps that are greater than new duration from all SourceBuffer
> objects in sourceBuffers" means. I don't see why removing data or the concept
> of audio frames need to be mentioned at all, if the desired effect is that
> playback should only proceed to duration. Certainly an implementation would be
> wise to throw away as much data as it can, which may be worth mentioning in a
> note.
I see your point. I forgot for a moment that the note implies non-normative.

Here is my main motivation for specifying that data is actually removed from the buffer. Say I have a SourceBuffer with data from [0-10) and a duration of 10. I decide to set the duration to 5. Then for some reason I change my mind and set it to 7. It seems to make more sense to me that the buffer contain [0,5) after this sequence. By not indicating that the data is removed from the buffer on duration change, some might believe that the buffer could contain [0,7) which I don't want to encourage.
Comment 6 Philip Jägenstedt 2012-09-24 08:33:45 UTC
Unless there's a keyframe at offset 5, that sounds like a rather complicated guarantee to make, one would have to annotate the segment containing offset 5 to stop playback at that offset and could end up with many such segments if one sets duration back and forth.

As for what data is removed, there would be some elegance in specifying that the equivalent of remove(new_duration, old_duration) (Bug 18709) to be run. If one wants playback to stop at some exact offset that's not all that is needed, but it would answer the original question of this bug report.
Comment 7 Adrian Bateman [MSFT] 2012-10-22 01:23:14 UTC
Next action: Update the duration change algorithm to use SourceBuffer.remove(old duration, new duration) once remove() has been specified by Bug 18709.
Comment 8 Aaron Colwell (c) 2012-11-28 18:31:48 UTC
Changes committed.
http://dvcs.w3.org/hg/html-media/rev/0c638da9a67a

Updated duration change algorithm to use remove().