This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.

Bug 27980 - [MSE] Ambiguous behaviour when running sourcebuffer.abort()
Summary: [MSE] Ambiguous behaviour when running sourcebuffer.abort()
Status: RESOLVED MOVED
Alias: None
Product: HTML WG
Classification: Unclassified
Component: Media Source Extensions (show other bugs)
Version: unspecified
Hardware: PC All
: P2 normal
Target Milestone: CR
Assignee: Matt Wolenetz
QA Contact: HTML WG Bugzilla archive list
URL:
Whiteboard:
Keywords:
Depends on: 27982
Blocks:
  Show dependency treegraph
 
Reported: 2015-02-08 10:07 UTC by Jean-Yves Avenard
Modified: 2015-10-13 23:01 UTC (History)
7 users (show)

See Also:


Attachments

Description Jean-Yves Avenard 2015-02-08 10:07:01 UTC
http://w3c.github.io/media-source/#widl-SourceBuffer-abort-void
as of 2015-02-08

3.2 Methods: abort() states

"If the updating attribute equals true, then run the following steps:

    Abort the buffer append and stream append loop algorithms if they are running.
    Set the updating attribute to false.
    Queue a task to fire a simple event named abort at this SourceBuffer object.
    Queue a task to fire a simple event named updateend at this SourceBuffer object.
"

So abort() should abort buffer append and stream append loop algorithm

However:
3.4 Event Summary
we have:
"The append or remove was aborted by an abort() call. updating transitions from true to false."

So here, abort() can abort the remove() operation.

Now, there's an ambiguous case.

What needs to be done when you do something like:
mediasource.duration = X
sourceBuffer.abort()

setting mediasource.duration is to run the range remove algorithm which will fire updatestart then later update/updateend

abort() is to fire abort/updateend

If we follow the spec to the letter, and abort doesn't abort then range removal algorithm.

We should have then:
updatestart
abort
updateend

and once range removal algorithm gets to run:
update
updateend

Chrome fires:
updatestart
abort
updateend

IE11:
updatestart
update
updateend

For both browsers, range removal algorithm has been run and the sourcebuffer buffered range is now { 0 , 5 }
Comment 1 Matt Wolenetz 2015-04-09 22:46:43 UTC
It seems the intent of abort() is to abort range removal algorithm as well as buffer and stream append algorithms, and is corroborated by the 'abort' event description. However, the exact behavior of the duration change algorithm in the context of an abort() aborting the range removal algorithm (on a change to a lower duration) needs clarification as well.

Ideally, the clarification would prevent the possibility of the abort()'ed duration change's range removal implementation retaining complete coded frames that start at or beyond MediaSource.duration or HTMLMediaElement.duration.

How does the following two part change sound?

1) In abort(), change step 3.1 from "Abort the buffer append and stream append loop algorithms if they are running." to "Abort the buffer append, stream append loop, and range removal algorithms if they are running."

2) In Duration Change, change from:
...
3. Update duration to new duration.
4. If the new duration is less than old duration, then run the range removal algorithm with new duration and old duration as the start and end of the removal range.
...

to

...
3. If the new duration is less than old duration, then do the following steps:
3.1. Run the range removal algorithm with new duration and old duration as the start and end of the removal range.
3.2. If the range removal algorithm in the previous step was aborted, then abort this algorithm.
4. Update duration to new duration.
...
Comment 2 Karl Tomlinson 2015-04-16 04:08:21 UTC
One awkward consequence of updating duration to new duration after the range
removal algorithm has completed is that the timing of the update depends on
whether or not the range removal algorithm is run.  If the setter is providing
a new duration less than old duration, then the effect will not be seen until
the update/updateend events have been queued, and so it would be prudent to
wait for these events.  However, if the new duration is not less than old
duration then those events are not dispatched.

Maybe there's a way to avoid that, and the problem already exists to some
extant with steps 5 and 6 of the duration change algorithm for alignment with
whole frames and updating the media controller as quoted in
https://www.w3.org/Bugs/Public/show_bug.cgi?id=27982#c0

I wonder whether these consequences are sufficient to rethink the decisions of
bug 18587 and instead aim to return to making the JS-visible effects of
setting duration synchronous and not abort()able.
Comment 3 Matt Wolenetz 2015-07-01 22:11:18 UTC
There is much to consider. Step 3.2 in my proposed solution in comment #1 is bad because step 3.2 can run prior to any possible abort of (or even start of the asynch range removal).

Further, there is this discussion (https://github.com/w3ctag/promises-guide/issues/31) about precise usage of "in parallel" instead of asynchronous. To date, I had understood all the steps in the spec were executed on the same thread (or at least serialized to that single thread in an implementation prior to the next step); this may not be the case for spec text going forward.

Jerry and I are working to find a better solution. For instance, I'm trying to understand why Remove() was made asynch/parallel in the first place, and can we refactor the DurationChange-triggered version of range removal to be synchronous, if not also making all of Remove() synchronous?
Comment 4 Jean-Yves Avenard 2015-07-01 23:52:14 UTC
In Firefox, we've understood asynchronous to mean just that, with no assumptions that all operations would run on a single thread.

We have undertaken a full rewrite of our MSE stack over the past few months, highly parallelizing the implementation.

Making any MSE calls synchronous now would be extremely difficult if not impossible without adding blocking calls which are inherently bad.

Making Remove synchronous would be a very unfortunate decision I believe.
Currently, all operations on the track buffers are asynchronous, that include the buffer append algorithm or the range removal algorithm. If one is to be async, its seems logical to have the other async too.
Certainly makes it easier in regards to a multi-threaded implementation.

I would much rather change the design of abort() instead. Including consideration for its removal.

In an asynchronous/multi-threaded world, you can't guarantee what state a call to abort() would leave the track buffers in. Some or all media segments may have already been processed. An init segment may have been processed which the JS would have no way of knowing.

The Reset Parser State algorithm, makes no consideration for the case where an init segment from the earlier call to appendBufer got processed.
Comment 5 Matt Wolenetz 2015-07-08 22:57:02 UTC
I agree that going back to a synchronous model is too big a hammer for this problem. I spoke with Aaron Colwell and two complementary ideas resulted:

1) Remove any text and logic in abort() spec and "abort" event spec involving aborting a Remove() (or aborting a range removal operation). Abort is really meant to reset the parser state. Remove() (or range-removal triggered by reducing duration) is already not allowed while processing appended media.

2) Disallow (and issue an exception) any reduction of duration that would remove buffered media. Remove() is available as a mechanism to explicitly remove buffered media, and once completed, a web app can inspect the buffered attribute prior to doing any explicit duration reduction operation.

I'm not sure how widespread is any usage of abort() to stop an in-progress Remove() (or duration-change-triggered range removal), though I doubt there are any interoperable expectations around HTMLMediaElement state for such a scenario.

The more concerning breaking change is #2: web apps would then be receiving exceptions for a duration change scenario that is more realistically used today than the scenario changed by #1.
Comment 6 Jerry Smith 2015-08-24 21:08:02 UTC
We agree that aborts of remove() and range removal algorithms don't have deterministic outcomes, and further believe that allowing aborts for those operations doesn't seem like a primary use case requirement.  We support disallowing abort for these cases.

The additional proposal to require remove() to make duration changes that remove buffered data also makes sense.  There might be some compat exposure to existing MSE sites, I guess.  That's probably manageable.  Duration changes that remove buffered data would require both a remove() and an explicit duration setting change.

Note this last use case also might not be a primary one.  Playlists or ad handling would more commonly remove everything from the buffer and start with new content.

I recommend making both changes proposed in this bug.
Comment 7 Karl Tomlinson 2015-08-24 21:48:38 UTC
(In reply to Matt Wolenetz from comment #5)
> 2) Disallow (and issue an exception) any reduction of duration that would
> remove buffered media.

Because duration is floating point, there may be potential for uncertainty in whether the new duration is less than the end of buffered media or not.

I wonder whether setting the new duration to the maximum of the new duration and the end of the buffered media may be better than having the setter potentially throw when aiming to reduce the duration to align with the end of the buffered media.
Comment 8 Jerry Smith 2015-10-13 01:31:16 UTC
It looks like the only remove reference for abort is in the event description.  That's a simple change.

The change to error on duration changes that intrude on the buffered range would affect the Coded Frame Removal Algorithm, which specifically handles removing buffered data outside of an updated duration end time.  The most direct change would be to remove that entire algorithm and update the Range Removal algorithm to replace step 6 with a check on the new duration overlapping buffered data.
Comment 9 Jerry Smith 2015-10-13 02:10:12 UTC
Correction:  The duration change must be handled separately in 2.4.6 Duration change algorithm.  The other algorithms are required to do normal Range Removal.
Comment 10 Matt Wolenetz 2015-10-13 23:01:51 UTC
This bug has been migrated to the GitHub issue tracker. Please follow/update progress using the GitHub issue:
https://github.com/w3c/media-source/issues/19