The current API is a bit odd in that textTracks is an array of TextTrack where TextTrack has properties like kind/label/language, but audioTracks and videoTracks is are MultipleTrackList/ExclusiveTrackList where the same data is retrieved via getLabel and getLanguage getters.

I'm not sure what the motivation for this is, but it seems less confusing for script authors if the interfaces were more similar, possible sharing a common interface for what few properties are shared between text, audio and video tracks.

In particular, being able to use Array.prototype.forEach.call(...) on Array-like things (like NodeList) is very handy.

I've argued against this before, but I think I was wrong to do so. It feels unnatural when using the API currently.

I agree that consistency would be nice. Can we have this change applied really soon? Browsers are starting to implement the <track> element and JS polyfills, too.

Indeed, changing this before implementations are too far along would be very welcome...

pumping up importance for a decision (I know that browsers are starting to implement this, e.g. http://trac.webkit.org/changeset/89186 ).

Please don't change the severity field of bugs assigned to me. (Changing the severity is not going to make me look at things quicker — if you want me to look at things more quickly, please ping me on IRC or e-mail me. I use the severity field to categorise input for planning purposes.)

As it happens, I was just looking at this because I need to do something like this for bug 12412 and bug 12983. The problem with doing this is that it makes it hard to reuse the same API for MediaStream, because if we have a MediaStream constructor, we wouldn't want it to get the same VideoTrack and AudioTrack objects as HTMLMediaElements expose in their videoTracks and audioTracks lists, since that would make no sense (the latter aren't streaming tracks, they're buffered tracks, even if the input is actually a MediaStream).

I don't really want to duplicate all the interfaces here, and having a hierarchy like:

   AbstractTrack -> AbstractVideoTrack -> BufferedVideoTrack
               \                      `-> StreamingVideoTrack
                `-> AbstractAudioTrack -> BufferedAudioTrack
                                      `-> StreamingAudioTrack

...seems like overengineering, especially since we'd still need the Streaming* cases to implement another interface so they could be listed all together in the MediaStream constructor...

Maybe the MediaStream constructor should just take Streams, and it merges the enabled tracks of those streams, rather than taking Tracks? But then there's no good way to take a MediaStream with three video tracks and create from it a MediaStream with three video tracks and one audio track, or some such.

Going back to use cases, maybe we don't need to merge streams. We just need a way to enable and disable streams, and maybe twiddle the volumes on audio tracks.

So maybe the way to go is:

   AbstractTrack -> VideoTrack
                `-> AudioTrack

...and a MediaStream constructor that lets you chain from a previous MediaStream, so that if you want to mute a track on an outgoing MediaStream, you just create a new MediaStream from the GeneratedMediaStream, and that's the one you twiddle and pass on to PeerConnection, so the connections would look like:

                          1                 2                      :
   GeneratedMediaStream ----> MediaStream ----> PeerConnection ----:---> (peer)
     |                                                             :
    \|/                                                            :
   <video>                                                         :

...where 1 is "new MediaStream(generatedStream)", and 2 is addStream(stream).

Then, to disable outgoing video without affecting the local mirror, you just take the MediaStream object between you get form 1, and call something like: 

   stream.videoTracks[0].pause();

   stream.videoTracks[0].resume(); // later

...on it.

It's not clear to me what the difference is between a "buffered" track and a "streaming" track, and why they need to have different interfaces.

I don't think "streaming tracks" should have pause() or resume() methods on them as shown in comment #7. pause() implies that when you resume() later you'll carry on from where you left off, but we don't want tracks to get out of sync (I can't think of any use-cases for it, and I don't want to implement it). It does make sense to pause and resume entire streams (except for real-time ones, but we can just skip ahead the entire stream when resuming those), and it makes sense to enable and disable particular tracks, and I think that's what's wanted in comment #7.

Of course, an alternative way of solving this problem of inconsistency between textTracks and audioTracks/videoTracks would be to turn the textTracks list into a TextTrackList object.

I still would like to be able to create MediaStream objects from individual tracks. One use case that this would enable is taking the audio track from a stream that is being received from a remote peer and combining it with the audio track from a local stream so that MediaStreamRecorder can be used to record the conversation to a single blob (as opposed to recording the remote peer audio to one blob and the local audio to another blob). If the different interfaces are only to prevent a buffered track from being passed to a MediaStream constructor then I think this is better served by simply having a readonly boolean property "buffered" or "streaming" and letting the MediaStream constructor throw when a buffered track is present in the arguments. I would expect the MediaStream tracks to show up on the media elements when playing a stream: URL so maybe those tracks are not always buffered. The availability of individual tracks will have to be tracked in some way. I also think that the enabled/disabled/selected states should be enough to determine whether a track is sent with a PeerConnection or not, if a received track is not supposed to be played back I guess it could always be disabled/unselected, right?

EDITOR'S RESPONSE: This is an Editor's Response to your comment. If you are satisfied with this response, please change the state of this bug to CLOSED. If you have additional information and would like the editor to reconsider, please reopen this bug. If you would like to escalate the issue to the full HTML Working Group, please add the TrackerRequest keyword to this bug, and suggest title and text for the tracker issue; or you may create a tracker issue yourself, if you are able to do so. For more details, see this document:
   http://dev.w3.org/html5/decision-policy/decision-policy.html

Status: Accepted
Change Description: see diff given below
Rationale: Concurred with reporter's comments.

(In reply to comment #10)
> I still would like to be able to create MediaStream objects from individual
> tracks. One use case that this would enable is taking the audio track from a
> stream that is being received from a remote peer and combining it with the
> audio track from a local stream so that MediaStreamRecorder can be used to
> record the conversation to a single blob (as opposed to recording the remote
> peer audio to one blob and the local audio to another blob).

I haven't added this yet, but I agree that we should add it in due course. I've put in some plumbing to make adding this easy. (I just figured we should pace ourselves.) You wouldn't do it by adding separate tracks, you'd just do  new MediaStream([localStream, remoteStream])  and then disable any non-audio tracks.


(In reply to comment #9)
> Of course, an alternative way of solving this problem of inconsistency between
> textTracks and audioTracks/videoTracks would be to turn the textTracks list
> into a TextTrackList object.

I don't see why this is a problem or why it needs solving.

Checked in as WHATWG revision r6275.
Check-in comment: Revamp how audioTracks and videoTracks work on HTMLMediaElement and MediaStream.
http://html5.org/tools/web-apps-tracker?from=6274&to=6275

(In reply to comment #12)
> (In reply to comment #9)
> > Of course, an alternative way of solving this problem of inconsistency between
> > textTracks and audioTracks/videoTracks would be to turn the textTracks list
> > into a TextTrackList object.
> 
> I don't see why this is a problem or why it needs solving.

But you have resolved it nevertheless by adding:

typedef TextTrack[] TextTrackList;

That's what this bug was originally about before it became all tangled up with RTC. :-)

I don't think that line does what you think it does if you think it changes anything. It is a purely editorial change.

(In reply to comment #15)
> I don't think that line does what you think it does if you think it changes
> anything. It is a purely editorial change.

You are correct. It hides the fact that they are different and I don't think that helps. As an author, you should be able to do:

audioTracks(5) and videoTracks(5) and textTracks(5) and not have to do textTracks[5] and all sorts of other differences.

(In reply to comment #16)
> (In reply to comment #15)
> > I don't think that line does what you think it does if you think it changes
> > anything. It is a purely editorial change.
> 
> You are correct. It hides the fact that they are different and I don't think
> that helps. As an author, you should be able to do:
> 
> audioTracks(5) and videoTracks(5) and textTracks(5) and not have to do
> textTracks[5] and all sorts of other differences.

You should never be able to do audioTracks(5), and I'm pretty sure Gecko will never implement that.

(In reply to comment #17)
> (In reply to comment #16)
> > (In reply to comment #15)
> > > I don't think that line does what you think it does if you think it changes
> > > anything. It is a purely editorial change.
> > 
> > You are correct. It hides the fact that they are different and I don't think
> > that helps. As an author, you should be able to do:
> > 
> > audioTracks(5) and videoTracks(5) and textTracks(5) and not have to do
> > textTracks[5] and all sorts of other differences.
> 
> You should never be able to do audioTracks(5), and I'm pretty sure Gecko will
> never implement that.

Is that not what this definition means:
readonly attribute AudioTrackList audioTracks;

interface AudioTrackList {
  [..]
  getter AudioTrack (in unsigned long index);
}

Just checked and the examples actually say audioTracks[i] ... hmmm

I highly recommend reading up on WebIDL syntax. :-)

None of these interfaces use 'caller', they're all indexed getters.

(In reply to comment #19)
> I highly recommend reading up on WebIDL syntax. :-)
> 
> None of these interfaces use 'caller', they're all indexed getters.

OK, but we still lack the following two features on TextTrackList that we have on audio and video tracks:

  VideoTrack? getTrackById(in DOMString id);
           attribute Function? onchange;

mass-move component to LC1