"Whenever an audio track in an AudioTrackList is enabled or disabled, the user agent must queue a task to fire a simple event named change at the AudioTrackList object."

"Whenever a track in a VideoTrackList that was previously not selected is selected, the user agent must queue a task to fire a simple event named change at the VideoTrackList object."

Since newly added tracks previously didn't exist, it's not clear if these conditions apply.

I think it doesn't make sense to fire the event, FWIW.

This would be in addition to the "addtrack" event?

The VideoTrack case seems open and shut: if the track didn't exist, it can't possibly have been "previously" anything.

The AudioTrack case is less clear... I could make it more obvious that no event is fired.

This started in a Blink code review, so CC acolwell who was also involved.

To make it very explicit, you could hook it to AudioTrack.enabled or VideoTrack.selected being set to a new value.

It's not exactly clear how many change events should fire when changing a video track, since there are potentially two VideoTrack objects that change. However this is spec'd, it should be made clear that only one change event fires.

I think another key piece of information missing that may be related to my code review is what to do with tracks that are marked as enabled by default in the media file. I interpreted Step 11 in http://www.whatwg.org/specs/web-apps/current-work/multipage/the-video-element.html#getting-media-metadata to be equivalent to VideoTrack.selected or AudioTrack.enabled being set to true and therefore should trigger change events to fire. Is that what was intended?

(In reply to Aaron Colwell from comment #4)
> I think another key piece of information missing that may be related to my
> code review is what to do with tracks that are marked as enabled by default
> in the media file. I interpreted Step 11 in
> http://www.whatwg.org/specs/web-apps/current-work/multipage/the-video-
> element.html#getting-media-metadata to be equivalent to VideoTrack.selected
> or AudioTrack.enabled being set to true and therefore should trigger change
> events to fire. Is that what was intended?

Hmm, that's a good point. I hadn't looked closely at this and just assumed that tracks would pop into existence with the state they have in the media resource and that therefore no change events would fire.

Media Fragments complicates things a little bit. I don't think it makes sense to fire the change even in that case either, but don't know how to spec it.

My original goal was that 'change' wouldn't fire when 'addtrack' does, you'd just get an 'addtrack' event and then later if it was enabled or disabled you'd get a 'changed' event.

Maybe I should have some of the logic in the "Once enough of the media data has been fetched to determine the duration of the media resource, its dimensions, and other metadata" steps copied into the "If the media resource is found to have an audio/video track" steps, so that it's clear that the tracks begin their life in an enabled or disabled state that the UA thinks is appropriate, and only later change if necessary?

If change events are never supposed to be fired when tracks are added or as a result of media fragments, how about adding the steps for firing them to the .enabled and .selected setters?

However, some shuffling in the steps you cited would be good to clarify the order of the addtrack events relative to the duration event. As written it looks like these may be fired in any order. Maybe bake it all into "Once enough of the media data has been fetched..."?

It wouldn't be in the setters, since in theory the values could get changed for different reasons (e.g. via the browser UI, though I guess we do say that that should act like it's using the API).

Ok, how's this:

Checked in as WHATWG revision r8623.
Check-in comment: Try to clean up the mess around when 'change' events are fired for audio and video tracks
http://html5.org/tools/web-apps-tracker?from=8622&to=8623

(In reply to contributor from comment #10)
> Checked in as WHATWG revision r8623.
> Check-in comment: Try to clean up the mess around when 'change' events are
> fired for audio and video tracks
> http://html5.org/tools/web-apps-tracker?from=8622&to=8623

Overall I think this is good. The one area that I'm still trying to figure out is what situation the change event in step 6 would fire in http://www.whatwg.org/specs/web-apps/current-work/multipage/the-video-element.html#found-another-video-track . All the steps that appear to set enable to true before it appear to condition on not having a selected track. Is the idea that Step 4 could override a currently selected track? If so, I'm assuming the UA is expected to not invoke this power if JavaScript explicitly selects a track.

Suppose the UA has information that would let it select a track, but by the time you reach step 12 of "Once enough of the media data has been fetched to determine the duration of the media resource, its dimensions, and other metadata", you still haven't seen the appropriate track. Step 12 ("If there is no selected video track, then select a video track") enables the first video track. Then, later, you finally get the track you were looking for. Now you select it, and unselect the earlier track.

(In reply to Ian 'Hixie' Hickson from comment #12)
> Suppose the UA has information that would let it select a track, but by the
> time you reach step 12 of "Once enough of the media data has been fetched to
> determine the duration of the media resource, its dimensions, and other
> metadata", you still haven't seen the appropriate track. Step 12 ("If there
> is no selected video track, then select a video track") enables the first
> video track. Then, later, you finally get the track you were looking for.
> Now you select it, and unselect the earlier track.

Ok. I think I see now. This is to handle something like a track indicated in a media fragment URL not being present on the transition to HAVE_METADATA, but comes along later during playback? In this case, is the expected behavior to ALWAYS select that track when it becomes available? For examples, say I have the track that is selected by Step 12 and then later during playback the track for a fragment URL appears. My understanding is that this track will become selected the moment this new track gets added. The user then selects the track that was originally selected at Step 12. Once this is done, the user then seeks back to the beginning of the clip where the video track indicated by the media fragment no longer exists. I assume the VideoTrack is removed by this operation and reappears again when the playback position reaches the location it was added last time around. In this case, does the track indicated by the media fragment get selected again eventhough the user has explicitly selected a different track?

I realize this is an odd scenario, but I'm just trying to understand whether or not explicit user behavior is supposed to override these "auto selection" steps.

I think the spec says that if you seek to a place where there's no video data on the selected track, you get nothing. But that may not be well-defined.

The UA wouldn't autoselect the track again, though, because you wouldn't find out that the track was once again present. You already know the track exists.

(In reply to Ian 'Hixie' Hickson from comment #14)
> I think the spec says that if you seek to a place where there's no video
> data on the selected track, you get nothing. But that may not be
> well-defined.
This makes sense to me if you seek to a location where the track exists, but doesn't have any data in it. I'm talking about a case where the track does not exist at this location in the presentation. Chained Ogg files are one specific case where this can occur. You can have a sequence of segments in the file where the number of tracks is different depending on what part of the file you are in. I was under the assumption that the track lists would always reflect the current tracks that are available in the current portion of the timeline. For consistency, I thought this would mean that when you seek, you would always update the track lists to reflect the tracks that are actually available at that point in time. That is why I figured the tracks could go away and then come back. Was this not the intent?

> 
> The UA wouldn't autoselect the track again, though, because you wouldn't
> find out that the track was once again present. You already know the track
> exists.
This confuses me. Based on my assumptions above, the track only "exists" during certain portions of the presentation. I was expecting it to come and go during the appropriate portion of the timeline and so I was just trying to understand if selection could potentially happen again when this occurred.

As specced currently, the track list is the union of all the tracks that you can ever get to. "removetrack" only fires "If at any time the user agent learns that an audio or video track has ended and all media data relating to that track corresponds to parts of the media timeline that are before the earliest possible position". "addtrack" only fires when you first receive the bits telling you that the track exists. At no other time is the track list modified.

(In reply to Ian 'Hixie' Hickson from comment #16)
> As specced currently, the track list is the union of all the tracks that you
> can ever get to. "removetrack" only fires "If at any time the user agent
> learns that an audio or video track has ended and all media data relating to
> that track corresponds to parts of the media timeline that are before the
> earliest possible position". "addtrack" only fires when you first receive
> the bits telling you that the track exists. At no other time is the track
> list modified.

Hmmm... ok. So in the case of MediaSource there are a few issues that come up. 

1. A SourceBuffer can be removed, which means all data for the underlying tracks sourced from that object are removed. I specify that the tracks get removed and a removetrack event is fired. I don't think this is controversial behavior. Could we modify the existing WHATWG text to include something along the lines of "if all media data relating to that track is no longer available"? I think this would also make things more consistent with TextTrackList behavior when the <track> element is destroyed or moved to a new element. What I'm talking about here is roughly equivalent. The data is no longer available to the media element.

2. Based on previous discussions around language/kind changes it was said that a new track was supposed to be created when the language changes. This was to preserve the immutability of the xxxTrack object. I always assumed that this meant that the track object for the old language would be removed. I guess that is not the case. While I see how that could have some benefits, I wonder how the user/application is supposed to know which tracks actually have data at the current position. I realize that you are not a fan of this use case, but I'm trying to understand your thinking here.

What's the use case for removing media data that might be actively playing? That sounds like all kinds of hassle.

I don't really understand why there would be multiple tracks if there's only really one track and it just happens to change language. Just have one track with no particular specified language. But yeah, I don't understand the use case there either really.

You can create a chained Ogg file where the first part is just audio, the second part is just video and last part is audio+video, altogether four tracks with different attributes.

Even if there is not use case for this, we'd still need to define what happens when you encounter such a file. You can't just fail upfront because in a streaming scenario you what the rest of the file looks like.

Sure, such a situation would have four tracks, and when there's no data for a particular track (because it's not present at that time position), you render nothing for that track (this might need clarifying in the spec). But that doesn't involve _removing_ the track.

Removing the tracks when seeking would be weird, I agree. If I knew of a good reason to have media files like this, I would suggest having seekable and buffered TimeRanges on the AudioTrack and VideoTrack objects so that one could tell which ones actually have data for any given time.

(In reply to Philip Jägenstedt from comment #21)
> Removing the tracks when seeking would be weird, I agree. If I knew of a
> good reason to have media files like this, I would suggest having seekable
> and buffered TimeRanges on the AudioTrack and VideoTrack objects so that one
> could tell which ones actually have data for any given time.

I could give an example from existing broadcasts, but at this point is seems not worth the effort since Ian seems to just disregard them.

I don't disregard them, I just haven't seen a reason to believe that those are relevant to the Web. It's like pointing to the way newspapers do something, and saying that the Web therefore needs that also. That's probably true if what you're pointing at is the ability to change the font size. It's clearly not true if it's the ability to print register marks to align their different colour prints. It's arguable when it's something like columns or flowing one region to another.

Existing broadcasts aren't the Web. If the reasons they do something apply to the Web, then great, we should take that into account. If the reasons they do something are not relevant on the Web, then we should not.

Ok this bug seems to have diverged into multiple issues, and the original issue seems to be resolved.

  * Make it clearer that when a track is not defined for the current playback
    position, the user agent is supposed to act as if the track was present but
    had no data -> bug 26460

  * comment 17 -> bug 26461

  * having seekable and buffered TimeRanges on the AudioTrack and VideoTrack objects
    so one could tell which ones actually have data for any given time -> bug 26462

This the original bug here is resolved, I'm marking this FIXED.