1. (In reply to comment #0)
> This question has come up several times when I talk to people about the Media
> Source Extensions so I think it deserves some discussion.
> 
> Should the Media Source methods & attributes be in a separate object instead of
> added to the HTMLMediaElement?

I think so, yes.

> Here are a few other follow up questions to spark discussion:
> 1. What do we gain by moving this into a separate object?

One can have separate objects for separate streams instead of the id association mechanism. They can expose (or be) MediaStreams and thus integrate with WebRTC.

> 2. If they are placed in a separate object, can the object be associated with
> multiple tags?

Presumably, yes. Would that be a problem?

> 3. Will this prevent declarative syntax to enable Media Source and reuse of the
> <source> fallback mechanism?

No more than the current solution, surely? For comparison, see the mediagroup="" attribute on <video> which implicitly creates a MediaController object.

(In reply to comment #1)
> 1. (In reply to comment #0)
> > Here are a few other follow up questions to spark discussion:
> > 1. What do we gain by moving this into a separate object?
> 
> One can have separate objects for separate streams instead of the id
> association mechanism. They can expose (or be) MediaStreams and thus integrate
> with WebRTC.

Wouldn't you still need a single object to collect all the separate objects together into a single presentation?

Do we really want to integrate with WebRTC in this way? I always assumed that we would do this through an HTMLMediaElement. I figured that HTMLMediaElement would expose one or more MediaStreams and those would be connected to the WebRTC APIs.   I figured this would be a better path because it allows us to reuse a bunch of events, behavior, and attributes that already have well defined meanings.

> 
> > 2. If they are placed in a separate object, can the object be associated with
> > multiple tags?
> 
> Presumably, yes. Would that be a problem?

It could make garbage collection of segments in the source buffer a little tricky since you would have to keep track of multiple presentation time pointers and take them into account when trying to decide what segments to expire. I suppose this isn't too much different than the problem the HTTP cache has to deal with, but it is extra work. 


> 
> > 3. Will this prevent declarative syntax to enable Media Source and reuse of the
> > <source> fallback mechanism?
> 
> No more than the current solution, surely? For comparison, see the
> mediagroup="" attribute on <video> which implicitly creates a MediaController
> object.

Would the onsourceopen, onsourceclose, onsourceended event handler attributes also automatically get assigned to this object too? I'm not sure where the right place is to draw the line for where the declarative mechanism should end if most of the functionality is in a separate object. If we allow the MediaSource object to get automatically created, it seems like at a minimum we should also allow onsourceopen since that would be the easiest way to discover this object when it gets created.

I must admit to speaking before thinking things through. A full review of the spec is something I've been meaning to do, and one of the things I've been meaning to look hard at is why the id strings are used, since it looks rather out-of-place to me. Elsewhere in HTMLMediaElement where there are different tracks of some kind (textTracks, audioTracks, videoTracks) each of those are separate objects. That seems a lot more natural to me, but I can't answer your questions without reading the entire spec.

(In reply to comment #3)
> I must admit to speaking before thinking things through. A full review of the
> spec is something I've been meaning to do, and one of the things I've been
> meaning to look hard at is why the id strings are used, since it looks rather
> out-of-place to me. Elsewhere in HTMLMediaElement where there are different
> tracks of some kind (textTracks, audioTracks, videoTracks) each of those are
> separate objects. That seems a lot more natural to me, but I can't answer your
> questions without reading the entire spec.

I'd definitely appreciate your input.
The ID strings evolved out of the desire to be able to handle audio and video streams from separate files. We needed a way to identify which byte stream was being appended to. We could handle this with multiple source objects that get registered with some sort of source list. I think that would bring things in line with the xxxTracks lists.

As we've been updating our Media Source implementation, tests, & demos, I've started to become less convinced that an ID string is the right path. Just the simple fact that I have to pick an ID value to use for even the simplest muxed stream case feels wrong. It ususally ends up being something like 'SourceId1' which is arbitrary and doesn't provide any value. Using an object would make this go away.

I'll start working on some new IDL that removes the ID and introduces an object.

How about something along these lines? I've included the url based append signature from Bug 16998 and string enums suggested in Bug 16938. 

interface SourceBuffer : EventTarget {
  // Returns the time ranges buffered.
  readonly attribute TimeRanges buffered;

  // Append segment data.
  void append(DOMString url, 
              optional unsigned long long start, 
              optional unsigned long long length);

  // Abort the current segment append sequence.
  void abort();

  enum EndOfStreamError { "network", "decode" };
  void endOfStream(optional EndOfStreamError error);

  [TreatNonCallableAsNull] attribute Function? onappenddone;
  [TreatNonCallableAsNull] attribute Function? onappenderror;
};

interface SourceBufferList : EventTarget {
  readonly attribute unsigned long length;
  getter SourceBuffer (unsigned long index);
  void remove(SourceBuffer buffer);

  [TreatNonCallableAsNull] attribute Function? onaddsourcebuffer;
  [TreatNonCallableAsNull] attribute Function? onremovesourcebuffer;
};

[Constructor (DOMString type)]
interface MediaSource : EventTarget {
  readonly attribute SourceBufferList buffers;

  // Adds another source buffer.
  SourceBuffer addSourceBuffer(DOMString type);

  enum State { "closed", "open", "ended" };
  readonly attribute State readyState;

  [TreatNonCallableAsNull] attribute Function? onsourceclosed;
  [TreatNonCallableAsNull] attribute Function? onsourceopen;
  [TreatNonCallableAsNull] attribute Function? onsourceended;
};

Associating the MediaSource with a <video> would be similar to LocalMediaStream.
1. Create a MediaSource object.
2. Use createObjectURL() to get a blob URL for the MediaSource object.
3. Assign the blob URL to HTMLMediaElement.src.

(In reply to comment #0)
> This question has come up several times when I talk to people about the Media
> Source Extensions so I think it deserves some discussion.
> 
> Should the Media Source methods & attributes be in a separate object instead of
> added to the HTMLMediaElement?

I support it for clear architecture and inferface reason, but I have some concerns that we have to explicitly to contruct mediasource for each time usage.

> 
> 
> Here are a few other follow up questions to spark discussion:
> 1. What do we gain by moving this into a separate object?

We gain OOP's benefit, and make HTMLMediaElement clean,

> 2. If they are placed in a separate object, can the object be associated with
> multiple tags?

Yes it can be technological:MediaSource->MediaStream->Video/audio. but I do not think it make too much sense in real world, because, Media Source mainly used by online video SP, AD SP etc, so,the most frequent scenario is one MediaSouce one Video, MediaStream can be constructed from getUserMedia().

> 3. Will this prevent declarative syntax to enable Media Source and reuse of the
> <source> fallback mechanism?

Do you mean we may selectively create event handler for MediaSource when using a seperate object? And I think MediaSource way has no way to prepare alternative track using <source>, right?

(In reply to comment #6)
> (In reply to comment #0)
> > This question has come up several times when I talk to people about the Media
> > Source Extensions so I think it deserves some discussion.
> > 
> > Should the Media Source methods & attributes be in a separate object instead of
> > added to the HTMLMediaElement?
> 
> I support it for clear architecture and inferface reason, but I have some
> concerns that we have to explicitly to contruct mediasource for each time
> usage.

Why are you concerned about the explicit object construction? Are you suggesting that the objects get created implicitly by using a special URL or something like that? The current API requires that you set the src attribute to sourceMediaSourceURL. I figured that creating a MediaSource object is only slightly more work.

> > 3. Will this prevent declarative syntax to enable Media Source and reuse of the
> > <source> fallback mechanism?
> 
> Do you mean we may selectively create event handler for MediaSource when using
> a seperate object? And I think MediaSource way has no way to prepare
> alternative track using <source>, right?

No. I was trying to determine if an object oriented interface would prevent declarative enabling of the API. For example:

<video>
  <source src="x-media-source:asdf" type="video/webm; codecs=\"vp8, vorbis\"" />
  <source src="http://test.com/myvideo.mp4" type="video/mp4" />
</video>

The idea here is that for UAs that support the Media Source API, it would pick the first source, but for UAs that don't they would pick the second source. This may be a moot point since some initial experimentation with the x-media-source: scheme with current browsers show that they signal an error immediately if they encounter an URL scheme they don't recognize. If this is the general behavior then <source> can't be used to allow graceful fallback for browsers that don't support this API.

Discussed in telcon: http://www.w3.org/2012/06/19-html-media-minutes.html#item05

Just for the record. Here is the change that converts the old API into the object oriented one.
http://dvcs.w3.org/hg/html-media/rev/b499a199e427

Here is the change that replaces SourceBufferList.remove() with MediaSource.removeSourceBuffer()
http://dvcs.w3.org/hg/html-media/rev/32d0ee562168

The spec has been updated to an object-oriented API.