Aaron, can you elaborate on this? Wouldn't such a requirement be redundant with https://dvcs.w3.org/hg/html-media/raw-file/tip/media-source/media-source.html ?
(Specifically, the requirements in removeSourceBuffer().)

Ian, my memory is a little foggy on this, but I believe the original thinking was that the HTML spec should contain the steps around track removal so that the MSE spec doesn't have to explicitly specify them. I need to go back and look, but I don't believe the current spec has any text that describes what to do if a track is completely removed from the presentation. It seems like the two allowed conditions " an audio or video track has ended and all media data relating to that track corresponds to parts of the media timeline that are before the earliest possible position" don't really apply in this case. 

If there was an extra case along the lines of "all media data relating to that track are no longer available" and this section had an algorithm name that I could reference, then I could remove the associated MSE text.

You might want to also consider copying the firing of the 'change' event to the HTML spec if the selected/enabled track gets removed. I'm not sure how you feel about this, but I put it in there since it seemed reasonable to fire the event to notify the webapp that the selected/enabled track changed.

Ultimately, my main goal is to shrink the amount of HTMLMediaElement behavior that the MSE spec defines. I didn't really have good hooks to do that so I just put the behavior I needed in the MSE text. I appreciate you working with me on this though. :)

Ah, right. If the goal is to move the stuff from MSE to HTML that makes sense. I can work on that.

Looking at this more closely, I don't know how much it really makes sense to move this stuff. There's some stuff that might make sense to merge or drop — in particular, I don't know that we need to define anything for createObjectURL() URLs being set to src="", especially given the new srcObject API — but removeSourceBuffer() seems like it's pretty MediaSource-specific, I don't know that it would make sense to move to the HTML spec itself.

One thing I don't understand is all the fiddling with seekable and buffered. Why can't a browser implement seeking into a MediaSource if it wants to?

Aaron, what do you think of comment 4 (keeping most of this in MSE)? Is there anything specific you think definitely can't be in MSE (e.g. because there's no way to really define it properly without being in HTML proper)?

I'm actually a little confused by your responses in Comment 4. They don't appear to be related to what I wrote in Comment 2 except for maybe my last paragraph.

The first 3 paragraphs were my main concern. Basically I was just looking for text in the HTML spec that says what to do when a track's underlying data is no longer available so that I can map SourceBuffer removal behavior to that behavior in the HTML spec instead of defining it myself. If you don't want to do that, so be it. I'll just leave what I have in the MSE spec.


I don't expect the HTML spec to take on MSE's seekable and buffered logic. Those definitions are there just to indicate what time regions are actually buffered in the MediaSource object so that all the normal HTMLMediaElement algorithms work as expected. You shouldn't need to worry about that. Most of that is just defining what is exposed to JS and outline how SourceBuffer updates could effect HTMLMediaElement.readyState.

Aah, I see. Sorry, I misunderstood what you were looking for.

So, there is text that handles the case of a track becoming irrelevant due to it no longer being present in the cached media data:

"If at any time the user agent learns that an audio or video track has ended and all media data relating to that track corresponds to parts of the media timeline that are before the earliest possible position, the user agent may queue a task to first remove the track from the audioTracks attribute's AudioTrackList object or the videoTracks attribute's VideoTrackList object as appropriate and then fire a trusted event with the name removetrack, that does not bubble and is not cancelable, and that uses the TrackEvent interface, with the track attribute initialised to the AudioTrack or VideoTrack object representing the track, at the media element's aforementioned AudioTrackList or VideoTrackList object."

Is that enough for you to work with, or do you want something more?

Aaron?

(In reply to Anne from comment #8)
> Aaron?

Sorry this went unanswered for so long.

In general I think the text in comment #7 has what I want, but it needs to include a case in addition to "before the earliest possible position" that is something along the lines of "or the UA receives an indication that the track data will never be available again". 

In the case of MSE the SourceBuffer is destroyed so the underlying data for a particular track is GONE and won't ever return. Also in cases like DASH, where a URL in the manifest may be providing the data for a track, it might make sense for the UA to remove the track if a fetch ends up resulting in a 404 or 5XX. My primarey concern is more about the first case than the later.

Aaron, would you be willing to provide a PR for that change? We switched development model so everyone can contribute to the HTML Standard directly now: https://github.com/whatwg/html.

(If you don't want to, we'll still fix this, but you can expedite it, so to say.)

https://github.com/whatwg/html/issues/4491