Bug 17002 - Specify a mechanism to determine which SourceBuffer an AudioTrack,VideoTrack, or TextTrack belong to.
Summary: Specify a mechanism to determine which SourceBuffer an AudioTrack,VideoTrack,...
Status: RESOLVED FIXED
Alias: None
Product: HTML WG
Classification: Unclassified
Component: Media Source Extensions (show other bugs)
Version: unspecified
Hardware: All All
: P1 normal
Target Milestone: ---
Assignee: Aaron Colwell (c)
QA Contact: HTML WG Bugzilla archive list
URL:
Whiteboard: tpac2012
Keywords:
Depends on:
Blocks: 20253
  Show dependency treegraph
 
Reported: 2012-05-08 20:38 UTC by Aaron Colwell (c)
Modified: 2013-01-15 22:25 UTC (History)
7 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Aaron Colwell (c) 2012-05-08 20:38:07 UTC
The current spec does not provide a way for a web application to determine which SourceID an AudioTrack or VideoTrack is associated with. This makes it difficult for the application to determine what the current Active ID set is.

A simple solution would be to place the SourceID in the AudioTrack.id & VideoTrack.id fields along with some sort of delimiter and the format native trackID.

Some id possibilities for SourceID 'source1' & format native TrackID 2:
source1:2
source1-2
source1,2

The choice of delimiter would require escaping rules or restrictions on what characters would be allowed in sourceID strings. Currently no restrictions exist in the text.

Adding the sourceID to the id field also helps keep the id for each track unique so that Audio|VideoTrack.getTrackById() works as expected.
Comment 1 Bob Lund 2012-05-11 19:46:42 UTC
(In reply to comment #0)
> The current spec does not provide a way for a web application to determine
> which SourceID an AudioTrack or VideoTrack is associated with. This makes it
> difficult for the application to determine what the current Active ID set is.
> 
> A simple solution would be to place the SourceID in the AudioTrack.id &
> VideoTrack.id fields

What about TextTrack?

Does using the SourceId guarantee uniqueness so it can be used by getElementById()? Aren't SourceIDs only unique with respect to an instance of HTMLMediaElement?

> along with some sort of delimiter and the format native
> trackID.

Having the format native trackID in the VideoTrack, AudioTrack and TextTrack for in-band tracks would also be useful so that metadata about the media resource could be associated with Video, Audio and Text tracks.

> 
> Some id possibilities for SourceID 'source1' & format native TrackID 2:
> source1:2
> source1-2
> source1,2
> 
> The choice of delimiter would require escaping rules or restrictions on what
> characters would be allowed in sourceID strings. Currently no restrictions
> exist in the text.
> 
> Adding the sourceID to the id field also helps keep the id for each track
> unique so that Audio|VideoTrack.getTrackById() works as expected.
Comment 2 Aaron Colwell (c) 2012-05-14 20:55:28 UTC
(In reply to comment #1)
> (In reply to comment #0)
> > The current spec does not provide a way for a web application to determine
> > which SourceID an AudioTrack or VideoTrack is associated with. This makes it
> > difficult for the application to determine what the current Active ID set is.
> > 
> > A simple solution would be to place the SourceID in the AudioTrack.id &
> > VideoTrack.id fields
> 
> What about TextTrack?

I accidentally forgot to mention TextTrack, but now that I'm looking at the HTML5 spec, it appears that TextTrack doesn't have an id attribute like AudioTrack and VideoTrack do. It isn't clear to me why this is. Perhaps an oversight?

I agree TextTrack should be included, but we should resolve the lack of id attribute before updating the Summary for this bug.

> 
> Does using the SourceId guarantee uniqueness so it can be used by
> getElementById()? Aren't SourceIDs only unique with respect to an instance of
> HTMLMediaElement?

I wasn't aware that you could fetch AudioTrack or VideoTrack objects by using getElementById(). Please point me to the relevant spec text.

You are right though that SourceIDs are only unique with within a specific HTMLMediaElement. I was under the assumption that AudioTrack.id and VideoTrack.id only needed to be unique within the HTMLMediaElement. Could you point me to text in the spec that indicates it must be globally unique within the document. I can't seem to find it.
Comment 3 Philip Jägenstedt 2012-05-15 09:03:11 UTC
getElementById should certainly not find AudioTrack, VideoTrack or TextTrack, as they are not HTML elements at all. AFAICT, the main purpose of exposing id on audio and video tracks if for integration with Media Fragments.
Comment 4 Aaron Colwell (c) 2012-05-15 17:06:16 UTC
(In reply to comment #3)
> getElementById should certainly not find AudioTrack, VideoTrack or TextTrack,
> as they are not HTML elements at all. AFAICT, the main purpose of exposing id
> on audio and video tracks if for integration with Media Fragments.

Ok good this matches my understanding as well. 

The Media Fragment spec and HTML5 spec appear to be out of sync. The HTML5 spec indicates that AudioTrack.id & VideoTrack.id contain the strings that should be used in the 'track dimension' of the fragment URI (http://dev.w3.org/html5/spec/media-elements.html#dom-audiotrack-id). The "Track Dimension" text and example in the Media Fragments spec seems to imply that the 'label' attributes on these objects should be use for the track dimension (http://www.w3.org/2008/WebVideo/Fragments/WD-media-fragments-spec/#naming-track). It seems to me that the Media Fragment spec is wrong here. 

Do you agree with this interpretation? If so I think it means 2 bugs need to be filed. One to update the text of Media Fragment spec to reference id instead of label. A second one would be filed to add an id attribute to TextTrack so it matches AudioTrack & VideoTrack.
Comment 5 Silvia Pfeiffer 2012-05-16 04:22:58 UTC
I've forwarded the bug to the Media Fragments WG
Comment 6 Silvia Pfeiffer 2012-05-16 05:06:05 UTC
I suggest resolving the addition of "id" to TextTrack as part of this bug, in particular if we require a strict string pattern on these identifiers.

I'd also suggest resolving what to extract from in-band tracks into the identifier as part of this bug. This will be different per encapsulation format.
Comment 7 Aaron Colwell (c) 2012-08-13 18:44:49 UTC
Now that the object-orient API changes are done, it's time to update the proposal for this bug. Here is the simplest think I can think of.

Add the following method to the MediaSource object.

partial interface MediaSource {
  SourceBuffer? getSoureBufferByTrackId(DOMString id);
}

- id can be a VideoTrack.id, AudioTrack.id, or TextTrack.id(when that is speced.)
- The MediaSource will ensure that the id values in all of the XXXTrack objects it provides data for are unique.
- This method returns null if the id is not associated with any tracks created by this MediaSource object.
- Throws an INVALID_ACCESS_ERR exception if id is null or empty.
- Throws an INVALID_STATE_ERR exception if called in the "closed" state.
Comment 8 Adrian Bateman [MSFT] 2012-09-25 15:23:51 UTC
From http://lists.w3.org/Archives/Public/public-html-media/2012Sep/0025.html:
> 6. Should I change the proposed fix to Bug 17002 to the following so the
> solution is independent of the id field?
> 
> partial interface MediaSource {
>   SourceBuffer? getSourceBuffer(VideoTrack videoTrack);
>   SourceBuffer? getSourceBuffer(AudioTrack audioTrack);
>   SourceBuffer? getSourceBuffer(TextTrack textTrack);
> }

As mentioned on the call today, I'd prefer the id string if we can make it work, but we (Microsoft) are in the middle of researching the multi-track aspects as will have a more thought out opinion soon.
Comment 9 Adrian Bateman [MSFT] 2012-12-04 15:54:47 UTC
(In reply to comment #8)
> From
> http://lists.w3.org/Archives/Public/public-html-media/2012Sep/0025.html:
>
> 6. Should I change the proposed fix to Bug 17002 to the following so the
>
> solution is independent of the id field?
> 
> partial interface MediaSource
> {
>   SourceBuffer? getSourceBuffer(VideoTrack videoTrack);
>  
> SourceBuffer? getSourceBuffer(AudioTrack audioTrack);
>   SourceBuffer?
> getSourceBuffer(TextTrack textTrack);
> }

Discussed at TPAC F2F. We will use the object based methods described here. This removes the dependency on bug 18960.
Comment 10 Aaron Colwell (c) 2012-12-08 01:30:51 UTC
Changes committed
http://dvcs.w3.org/hg/html-media/rev/ee6e8ae9337c
Comment 11 Aaron Colwell (c) 2013-01-15 19:26:30 UTC
Reopening to replace getSourceBuffer() with read-only sourceBuffer attribute proposed in 
http://lists.w3.org/Archives/Public/public-html-media/2013Jan/0007.html
Comment 12 Aaron Colwell (c) 2013-01-15 22:25:10 UTC
Changes committed.
https://dvcs.w3.org/hg/html-media/rev/fd2a58eec443