I was under the impression that inBandMetadataTrackDispatchType was intended to be like a MIME-type for track so that the web application could speculate about the type of data to expect from the track w/o having to receive any actual cues. I think this is a valuable thing to have and provides something we could create a registry around that maps inBandMetadataTrackDispatchType to expected cues exposed by the track.

I'm concerned about having type strings exposed here. What is the use case where the UA can determine the type, but not actually be able to parse the data it is returning? If it understands the type, it seems like it should be able to return a type specific cue for it. Something like this amplifies my concern that unspeced protocols will be allowed because this is starting to look like a "hash map" type interface.

(In reply to Aaron Colwell from comment #1)
> I was under the impression that inBandMetadataTrackDispatchType was intended
> to be like a MIME-type for track so that the web application could speculate
> about the type of data to expect from the track w/o having to receive any
> actual cues.

inBandTrack...Type contains fields from the track in the media resource that script can use to identify the type of metadata in that track.

> I think this is a valuable thing to have and provides something
> we could create a registry around that maps inBandMetadataTrackDispatchType
> to expected cues exposed by the track.
> 
> I'm concerned about having type strings exposed here. What is the use case
> where the UA can determine the type, but not actually be able to parse the
> data it is returning? If it understands the type, it seems like it should be
> able to return a type specific cue for it. Something like this amplifies my
> concern that unspeced protocols will be allowed because this is starting to
> look like a "hash map" type interface.

(In reply to Aaron Colwell from comment #1)
> I'm concerned about having type strings exposed here. What is the use case
> where the UA can determine the type, but not actually be able to parse the
> data it is returning? If it understands the type, it seems like it should be
> able to return a type specific cue for it. Something like this amplifies my
> concern that unspeced protocols will be allowed because this is starting to
> look like a "hash map" type interface.

As far as I know, every container format has metadata to say what type of data a stream contains (stream_type for MPEG-TS, CodecID for Matroska and WebM, Ogg Skeleton has a mime type). By exposing that data, we can tell JavaScript what the track contains without knowing how to parse it.

(In reply to Bob Lund from comment #2)
> (In reply to Aaron Colwell from comment #1)
> > I was under the impression that inBandMetadataTrackDispatchType was intended
> > to be like a MIME-type for track so that the web application could speculate
> > about the type of data to expect from the track w/o having to receive any
> > actual cues.
> 
> inBandTrack...Type contains fields from the track in the media resource that
> script can use to identify the type of metadata in that track.

OK. That is close enough assuming that there is some sort of registry to describe the valid values for this field and prevent conflicts.

My understanding though is that this bug is calling for this attribute to be removed from TextTrack and placed in the DataCue instead. Is this really what you want? I wouldn't expect the "track type" to change on a per cue basis just like the codec or encapsulation format doesn't typically change in the middle of a track.

(In reply to Brendan Long from comment #3)
> (In reply to Aaron Colwell from comment #1)
> > I'm concerned about having type strings exposed here. What is the use case
> > where the UA can determine the type, but not actually be able to parse the
> > data it is returning? If it understands the type, it seems like it should be
> > able to return a type specific cue for it. Something like this amplifies my
> > concern that unspeced protocols will be allowed because this is starting to
> > look like a "hash map" type interface.
> 
> As far as I know, every container format has metadata to say what type of
> data a stream contains (stream_type for MPEG-TS, CodecID for Matroska and
> WebM, Ogg Skeleton has a mime type). By exposing that data, we can tell
> JavaScript what the track contains without knowing how to parse it.

I don't have a problem with TextTrack.inBandMetadataTrackDispatchType. My assumption is that it would set that attribute based on the fields you've mentioned here, assuming name conflicts are avoided.

A DataCue.type value seems very different to me than this though. It implies to me that the UA is able to partially parse Blocks/Samples/Cues or whatever the container specific concept is called. It seems to me that if the UA is able to parse the data one level deeper, it should represent this as a format specific type instead of a generic DataCue. To me DataCue should indicate "I don't know how to parse this", whereas something like MP2TSPrivateDataCue would indicate "Here is a transport stream private data cue that has some fields and a block of data I don't know how to parse."

I feel like being explicit about the types expected in metadata tracks would greatly facilitate interoperability and make it clearer what the UA does and does not know.

Ted,

You are advocating for the removal of inBandMetadataTrackDispatchType here right?

Does this mean that all metadata should be multiplexed into a single "metadata" TextTrack? If not, how is the application expected to select the "right" metadata track for the specific metadata it is interested in? 

For example, I was thinking of metadata tracks as being useful for transporting auxiliary "data tracks" for things like the Breakdancing Robot Control Protocol(BDRCP) messages. Web applications connected to breakdancing robots would want to be able to identify these tracks and watch for messages so robot dancing could be initiated, but applications that just care about presentation level metadata like ID3 tags might not want to receive these cues. I viewed inBandMetadataTrackDispatchType as a way to determine the type of data conveyed by the particular track so that the web application could set event listeners appropriately. 

While breakdancing robots may not be popular ...yet... I'm sure there are other "data track" use cases that exist right now that may benefit from keeping inBandMetadataTrackDispatchType to allow grouping of metadata cues.

(In reply to Aaron Colwell from comment #6)
> Does this mean that all metadata should be multiplexed into a single
> "metadata" TextTrack? If not, how is the application expected to select the
> "right" metadata track for the specific metadata it is interested in?

I think this is what's being suggested. What I heard from people at Apple was that we don't necessarily know what type of metadata we're looking at, so maybe it would be better to have separate tracks for each type like we have now, and then have some sort of "we don't know what type this is" metadata track? I'm not sure that track would ever be useful to web authors though.

(In reply to Brendan Long from comment #7)
> (In reply to Aaron Colwell from comment #6)
> > Does this mean that all metadata should be multiplexed into a single
> > "metadata" TextTrack? If not, how is the application expected to select the
> > "right" metadata track for the specific metadata it is interested in?
> 
> I think this is what's being suggested. What I heard from people at Apple
> was that we don't necessarily know what type of metadata we're looking at,
> so maybe it would be better to have separate tracks for each type like we
> have now, and then have some sort of "we don't know what type this is"
> metadata track? I'm not sure that track would ever be useful to web authors
> though.

It seems to be a UA implementation, and perhaps a media container format, issue whether the type of data in a metadata track is known when the track is created. In the case where the type is known at track creation, having the type associated with each Cue in that track works but is sub-optimal.

It also seems that a UA, again perhaps based on media resource format, will either multiplex all Cue types into a generic metadata text track or it will create type specific metadata text tracks. I can't construct a use case why both would be done for a specific media resource instance.

(In reply to Edward O'Connor from comment #0)
> Some tracks may contain data cues of different types; exposing the type
> track-wide fails to take this into account.
> 
> Also, some media frameworks do not expose the original format of the
> metadata; thus, engines built on such media frameworks cannot populate this
> field.

Can you provide concrete examples, please?

The current approach to DataCue matches the use case example that Aaron provides about "Breakdancing Robot Control Protocol(BDRCP) messages" and not one where the cues within a track all belong semantically to different types of data.

EDITOR'S RESPONSE: This is an Editor's Response to your comment. If you are
satisfied with this response, please change the state of this bug to CLOSED. If
you have additional information and would like the Editor to reconsider, please
reopen this bug. If you would like to escalate the issue to the full HTML
Working Group, please add the TrackerRequest keyword to this bug, and suggest
title and text for the Tracker Issue; or you may create a Tracker Issue
yourself, if you are able to do so. For more details, see this document:

   http://dev.w3.org/html5/decision-policy/decision-policy.html

Status: Rejected
Rationale:
Text tracks are not supposed to have cues of more than one type in them, which is why inBandMetadataTrackDispatchType has to remain a property of TextTrack.