Warning:
This wiki has been archived and is now read-only.
OldSpecInput
Contents
Audio/Video/Text Track Creation
This section defines audio, video and text track, text track cue creation for each of the media resource types.
MPEG-2 TS
Track order
Order in the PMT. See ISO/IEC 13818-1:2013.
Track id attribute
Decimal representation of the PID in the PMT of the elementary stream containing the track data. Note in the case of CEA708 closed captions this will be the PID of the video elementary stream containing captions in the Picture User Data.
Track label attribute
Empty string.
Track language attribute
ISO_639_language_code in the ISO 639 Language Descriptor. See ISO/IEC 13818-1:2013.
Audio and Video Track Creation
The following table summarize how audio and video track attributes should be set.
Applies to | kind | Detection |
---|---|---|
video | main | First video elementary stream in the PMT |
audio | descriptions | AC-3: bsmod == 2 and full_svc == 0 |
main | First audio elementary stream in the PMT | |
main-desc | AC-3: bsmod == 2 and full_svc == 1 | |
translation | AC-3: Not first audio elementary stream in the PMT and bsmod == 0 |
TextTrack Creation
The following table summarize how text track kind should be set.
kind | Detection |
---|---|
captions | Presence of caption_service_descriptor or closed_caption_type in a video elementary stream |
subtitles | stream_type == 0x82 that is rendered by the UA |
metadata | stream_type == 0x05, 0x80 – 0xFF containing private sections (payload_unit_start_indicator == 1) |
Text Track Cue Creation
TextTrack.kind | Cue Generation |
---|---|
captions | Unspecified until a CEA708Cue subclass is defined |
subtitles | Unspecified until a SCTE27Subtitle subclass is defined |
metadata | The UA creates a DataCue for one complete private section table. Cue data is the entire private section table starting with table_id through CRC_32. The cue startTime and endTime is the time in the media resource timeline when the cue is created. Cue text is the empty string. Cue pauseOnExit is false. |
MPEG-2 TS Metadata Cue
Below is a proposal for an MPEG-2 TS metadata text track cue interface. There are three types of metadata exposed to script: transport stream program map table, transport stream description table, and private section. The tableId attribute identifies the specific type of metadata:
Value | Description |
---|---|
0x02 | Transport stream program map table |
0x03 | Transport stream description table |
0x40 - 0xFE | User Private |
interface MpegTsSection : TextTrackCue { //Generic MPEG-2 TS Section attribute octet tableId; attribute MpegTsSyntaxSection? syntaxSection; } interface MpegTsSyntaxSection { attribute unsigned short tableIdExtension; attribute octet versionNumber; attribute boolean currentNextIndicator; attribute octet sectionNumber; attribute octet lastSectionNumber; } interface MpegTsDescriptor { attribute octet tag; attribute ArrayBuffer data; } interface MpegTsElementaryStreamMetadata { attribute octet streamType; attribute unsigned short elementaryPID; attribute MpegTsDescriptor[] descriptors; } interface MpegTsPmt : MpegTsSection { //Transport Stream program map section (see Table 2-28) attribute unsigned short programNumber; attribute MpegTsDescriptor[] descriptors; attribute MpegTsElementaryStreamMetadata[] streams; } interface MpegTsDescriptionSection : MpegTsSection { // Transport Stream Description Table (see Table 2-30-1) attribute MpegTsDescriptor[] descriptors; } interface MpegTsPrivateSection : MpegTsSection { //Private section (see Table 2-30) attribute boolean privateIndicator; attribute MpegTsSyntaxSection? syntaxSection; ArrayBuffer privateData; }
ISOBMFF
Track order
Order of TrackBox (trak) boxes in the MovieBox (moov) container. See http://standards.iso.org/ittf/PubliclyAvailableStandards/c061988_ISO_IEC_14496-12_2012.zip.
Track id attribute
Decimal representation of the track_ID of a TrackHeaderBox (tkhd) in a TrackBox (trak) box. See http://standards.iso.org/ittf/PubliclyAvailableStandards/c061988_ISO_IEC_14496-12_2012.zip.
Track label attribute
The name field in the HandlerBox. See http://standards.iso.org/ittf/PubliclyAvailableStandards/c061988_ISO_IEC_14496-12_2012.zip.
Track language attribute
The language field in the MediaHeaderBox. See http://standards.iso.org/ittf/PubliclyAvailableStandards/c061988_ISO_IEC_14496-12_2012.zip.
The following table is based on the MP4 Registration Authority codec table and on the 2004 Edition of the ISO Base Media File Format standard (Download ISO/IEC 14496-12:2004).
WebM
Track order
Order in the EBML Initialization Segment. See http://www.w3.org/TR/media-source/#init-segment
Track id attribute
Decimal representation of the TrackUID of a track in the Segment info. See http://www.webmproject.org/code/specs/container/#ebml-basics and http://matroska.org/technical/specs/index.html#TrackUID
Track label attribute
The "name" field of a track in the Segment info. See http://matroska.org/technical/specs/index.html#Name
Track language attribute
The "language" field of a track in the Segment info. See http://matroska.org/technical/specs/index.html#Language
Ogg
Track order
Order in skeleton fisbone headers. https://wiki.xiph.org/Ogg_Skeleton_4
Track id attribute
The "name" field of the message header fields of a fisbone header in Ogg Skeleton. See https://wiki.xiph.org/Ogg_Skeleton_4
If no Skeleton header is available, use a decimal representation of the stream serialnumber. See http://xiph.org/ogg/doc/oggstream.html
Track label attribute
The contents of the "title" message header field of a fisbone header in Ogg Skeleton, if a Skeleton header is available. See https://wiki.xiph.org/Ogg_Skeleton_4
Empty otherwise.
Track language attribute
The "language" message header field of a fisbone header in Ogg Skeleton, if a Skeleton header is available. See https://wiki.xiph.org/Ogg_Skeleton_4
DASH
Track order
Order in the MPD.
Track id attribute
id attribute in the AdaptationSet element. Empty string if id attribute is not present. See ISO/IEC 23009-1.
Track label attribute
Empty string.
Track language attribute
lang attribute in the AdaptationSet element. See ISO/IEC 23009-1
Audio and Video Track Creation
The following table summarize how audio and video track kind attribute should be set.
Applies to | kind | Detection |
---|---|---|
video | main | Role schemeIdUri = urn:mpeg:dash:role:2011, value = main |
audio | descriptions | Role schemeIdUri = urn:mpeg:dash:role:2011, value = supplementary and Role schemeIdUri = urn:dlna:dash:role:2012, value = description |
main | Role schemeIdUri = urn:mpeg:dash:role:2011, value = main | |
main-desc | Role schemeIdUri = urn:mpeg:dash:role:2011, value = main and Role schemeIdUri = urn:dlna:dash:role:2012, value = description | |
translation | Role descriptor with URN = urn:mpeg:dash:role:2011, value = dub |
TextTrack Creation
The following table summarize how text track kind attribute should be set.
kind | Detection |
---|---|
captions | Role descriptor with URN = urn:mpeg:dash:role:2011, value = caption |
subtitles | Role descriptor with URN = urn:mpeg:dash:role:2011, value = subtitle |
metadata | AdaptationSet mimeType of 'application' or 'text' and no Role descriptor with value of caption or subtitle |
Exposing In-band Track Metadata
Three implementation alternatives are described for making media resource in-band track metadata available to Web applications:
- As a separate TextTrack, hereafter referred to as "In-band Metadata TextTrack". This alternative is based on the CableLabs Specification.
- As an attribute of the track object. This is based on a discussion on public-inbandtracks@w3.org.
- Use the existing inbandTrackMetadataDispatchType attribute. This only makes metadata available for inband text tracks of kind == 'metadata'.
In-band Metadata TextTrack Alternative
Media resource in-band track metadata is made available to the Web application as a TextTrack. The UA creates an in-band metadata TextTrack as appropriate for the media resource type.
The UA creates a DataCue that contains a representation of the media resource in-band track metadata is follows:
- startTime is set to the current time in the media resource timeline.
- endTime is set to Infinity.
- data and text attributes are set as follows, based on the type of the media resource:
- MPEG-2 TS:
- data - Let pmt length be the value of the "program_info_length" in the TS_program_map_section, interpreted as an integer as defined by the MPEG-2 specification. Set data to program descriptor bytes be the pmt length bytes following the "program_info_length" field.
- text is set to the empty string.
- pauseOnExit is set to false.
- MP4 ISOBMFF:
- Ogg:
- WebM:
- DASH:
- MPEG-2 TS:
In-band Metadata Attribute Alternative
Media resource in-band track metadata is made available to the Web application as a new attribute, e.g. trackMetadata, of the video, audio or text track. The inBandMetadataTrackDispatchType fulfills the function proposed for the trackMetadata attribute, but only for TextTracks where "kind" == metadata.
The UA sets the trackMetadata attribute for each video, audio and text track it creates, depending on the media resource type, as follows:
- MPEG-2 TS: Let stream type be the value of the "stream_type" field describing the track's type in the media resource's program map section, interpreted as an 8-bit unsigned integer. Let length be the value of the "ES_info_length" field for the track in the same part of the program map section, interpreted as an integer as defined by the MPEG-2 specification. Let descriptor bytes be the length bytes following the "ES_info_length" field. Set trackMetadata to the concatenation of: the string “video/mp2t”, the stream type byte and the descriptor bytes bytes, expressed in hexadecimal using uppercase ASCII hex digits.
- MP4 ISOBMFF:
- Ogg:
- WebM:
- DASH:
Another option would be to expose the descriptor data as an ArrayBuffer, and add an additional attribute to indicate that the stream is MPEG-TS:
interface TextTrack {
... readonly attribute ArrayBuffer metadata; // binary descriptor data
}
interface HTMLMediaElement {
... readonly attribute mediaType; // "video/mp2t"
}
inBandMetadataTrackDispatchType Alternative
Media resource in-band track metadata is made available to the Web application using the inbandMetadataTrackDispatchType attribute defined in HTML5. This attribute is only defined for TextTracks @kind == 'metatdata'; metadata will not be available to Web applications for video and audio tracks, and text tracks @kind != 'metadata'.
Alternative Pros and Cons
Alternative | Pros | Cons |
---|---|---|
In-band Metadata TextTrack |
|
|
In-band Metadata Attribute |
|
|
inBandMetadataTrackDispatchType |
|
|
Proposed HTML spec changes
Audio and video kind table
Additions to the existing table are green
Category | Definition | Applies to... | Examples |
---|---|---|---|
"alternative" | A possible alternative to the main track, e.g. a different take of a song (audio), or a different angle (video). | Audio and video. | Ogg: "audio/alternate" or "video/alternate"; DASH: "alternate" without "main" and "commentary" roles, and, for audio, without the "dub" role (other roles ignored). |
"captions" | A version of the main video track with captions burnt in. (For legacy content; new content would use text tracks.) | Video only. | DASH: "caption" and "main" roles together (other roles ignored). |
"descriptions" | An audio description of a video track. | Audio only. | Ogg: "audio/audiodesc".; AC3 audio in MPEG-2 TS: bsmod=2 and full_svc=0.; DASH: "supplementary" and urn:dlna:dash:role:2012 "description" roles together. |
"main" | The primary audio or video track. | Audio and video. | Ogg: "audio/main" or "video/main"; WebM: the "FlagDefault" element is set; DASH: "main" role without "caption", "subtitle", and "dub" roles (other roles ignored).; MPEG-2 TS: first audio (video) elementary stream in the PMT. |
"main-desc" | The primary audio track, mixed with audio descriptions. | Audio only. | AC3 audio in MPEG-2 TS: bsmod=2 and full_svc=1.; DASH: "main" and urn:dlna:dash:role:2012 "description" roles together. |
"sign" | A sign-language interpretation of an audio track. | Video only. | Ogg: "video/sign". |
"subtitles" | A version of the main video track with subtitles burnt in. (For legacy content; new content would use text tracks.) | Video only. | DASH: "subtitle" and "main" roles together (other roles ignored). |
"translation" | A translated version of the main audio track. | Audio only. | Ogg: "audio/dub". DASH: "dub" and "main" roles together (other roles ignored).; AC3 audio in MPEG-2 TS: not first audio elementary stream in the PMT and bsmod=0. |
"commentary" | Commentary on the primary audio or video track, e.g. a director's commentary. | Audio and video. | DASH: "commentary" role without "main" role (other roles ignored). |
"" (empty string) | No explicit kind, or the kind given by the track's metadata is not recognised by the user agent. | Audio and video. | Any other track type, track role, or combination of track roles not described above. |
Text kind table
Add this table in this section.
Category | Definition | Examples |
---|---|---|
"captions" | Text of the main dialogue. | MPEG-2 TS: Presence of caption_service_descriptor or closed_caption_type in a video elementary stream; DASH: "caption" role. |
"chapters" | ||
"descriptions" | A text description of a video track. | |
"subtitles" | Text translation of the main dialogue. | MPEG-2 TS: stream_type=0x82 that is rendered by the user agent; MP4 ISOBMFF: handler_type of 'subt'; DASH: "subtitle" role. |
"metadata" | Tracks to be interpreted by the Web application. | MPEG-2 TS: stream_type=0x05, 0x80 - 0xFF, 0x82 that is not rendered by the user agent; MP4 ISOBMFF: handler_type of 'meta'. |
Guidelines for creating metadata text track cues
Add this table in this section.
Media resource type | Cue creation guidelines |
---|---|
DASH | Cues are created as defined for the AdaptationSet mimeType. See ISO/IEC 23009-1. |
MPEG-4 BMFF | Cues are created as defined for the MetaDataSampleEntry namespace or mime_format. See ISO/IEC 14496-12:2012. |
MPEG-2 TS | Cue data is the entire private section table starting with table_id through CRC_32. See ISO/IEC 13818-1:2013. The cue startTime and endTime is the time in the media resource timeline when the cue is created. pauseOnExit is false. |
Ogg | TBD. |
WebM | TBD. |
Exposing a Media Resource Specific TextTrack
Add the following to Step 4.
If the media resource is a DASH MPD file
- The text track in-band metadata track dispatch type must be set to the concatenation of the AdaptationSet element attributes and all child Role descriptors.
Media Resource Formats
The following tables give a summary per media type of how the different stream types are exposed.
MPEG-2 TS
The following table is based on the 2013 Edition of the MPEG-2 TS standard (Download ITU-T H.222.0 | ISO/IEC 13818-1:2013).
id: The id for every track should be a string containing the track's PID formatted as a decimal number.
language: The language should be determined from the ISO_639_language_descriptor
NOTE: The list is probably too big, we could trim it and say that those not listed are not supported yet.
<BobLund131216>I updated this table.
- Unless there is a case where an audio/video elementary stream not rendered by the UA needs to be presented to the application, these streams should not cause creation of a text track.
- Stream types 0x05 and 0x80 - 0xFF are used for a variety of text tracks, e.g. 0x05 ETV signaling, 0x82 SCTE 27 subtitles. Others are user private but could still be used for Web applications, e.g. Nielsen rating signaling.
</BobLund131216>
NOTE: We could also extend it with DVB, ATSC, ... stream types, and/or using the SMPTE registration data.
MPEG-2 TS Stream Type | HTML 5 Track attributes | ||||||
---|---|---|---|---|---|---|---|
Stream type value | description | Track Interface | kind | inBandMetadataTrackDispatchType | cue payload | ||
0x00 | ITU-T - ISO/IEC Reserved | N/A | |||||
0x01 | ISO/IEC 11172-2 Video | VideoTrack if decoding is supported |
|||||
0x02 | ITU-T Rec. H.262 - ISO/IEC 13818-2 Video or ISO/IEC 11172-2 constrained parameter video stream | VideoTrack if decoding is supported + TextTrack if stream contains closed captions (as identified by a caption_service_descriptor in the PMT or a data_type == 0x03 in the Picture User Data |
|||||
0x03 | ISO/IEC 11172-3 Audio | AudioTrack if decoding is supported |
|||||
0x04 | ISO/IEC 13818-3 Audio | AudioTrack if decoding is supported |
|||||
0x05 | ITU-T Rec. H.222.0 - ISO/IEC 13818-1 private_sections | |
|||||
0x06 | ITU-T Rec. H.222.0 - ISO/IEC 13818-1 PES packets containing private data | N/A | |||||
0x07 | ISO/IEC 13522 MHEG | N/A | |||||
0x08 | ITU-T Rec. H.222.0 - ISO/IEC 13818-1 Annex A DSM-CC | N/A | |||||
0x09 | ITU-T Rec. H.222.1 | N/A | |||||
0x0A | ISO/IEC 13818-6 type A | N/A | |||||
0x0B | ISO/IEC 13818-6 type B | N/A | |||||
0x0C | ISO/IEC 13818-6 type C | N/A | |||||
0x0D | ISO/IEC 13818-6 type D | N/A | |||||
0x0E | ITU-T Rec. H.222.0 - ISO/IEC 13818-1 auxiliary | N/A | |||||
0x0F | ISO/IEC 13818-7 Audio with ADTS transport syntax | AudioTrack if decoding is supported |
|||||
0x10 | ISO/IEC 14496-2 Visual | VideoTrack if decoding is supported |
|||||
0x11 | ISO/IEC 14496-3 Audio with the LATM transport syntax as defined in ISO/IEC 14496-3/AMD-1 | AudioTrack if decoding is supported |
|||||
0x12 | ISO/IEC 14496-1 SL-packetized stream or FlexMux stream carried in PES packets | N/A | |||||
0x13 | ISO/IEC 14496-1 SL-packetized stream or FlexMux stream carried in ISO/IEC14496_sections | N/A | |||||
0x14 | ISO/IEC 13818-6 Synchronized Download Protocol | N/A | |||||
0x15 | Metadata carried in PES packets | N/A | |||||
0x16 | Metadata carried in metadata_sections | N/A | |||||
0x17 | Metadata carried in ISO/IEC 13818-6 Data Carousel | N/A | |||||
0x18 | Metadata carried in ISO/IEC 13818-6 Object Carousel | N/A | |||||
0x19 | Metadata carried in ISO/IEC 13818-6 Synchronized Download Protocol | N/A | |||||
0x1A | IPMP stream (defined in ISO/IEC 13818-11, MPEG-2 IPMP) | N/A | |||||
0x1B | AVC video stream conforming to one or more profiles defined in Annex A of Rec. ITU-T H.264 - ISO/IEC 14496-10 or AVC video sub-bitstream of SVC, or MVC base view sub-bitstream, or AVC video sub-bitstream of MVC | VideoTrack if decoding is supported |
|||||
0x1C | ISO/IEC 14496-3 Audio, without using any additional transport syntax, such as DST, ALS and SLS | AudioTrack if decoding is supported |
|||||
0x1D | ISO/IEC 14496-17 Text | TextTrack | |||||
0x1E | Auxiliary video stream as defined in ISO/IEC 23002-3 | VideoTrack if decoding is supported |
|||||
0x1F | SVC video sub-bitstream of an AVC video stream conforming to one or more profiles defined in Annex G of Rec. ITU-T H.264 - ISO/IEC 14496-10 | VideoTrack if decoding is supported |
|||||
0x20 | MVC video sub-bitstream of an AVC video stream conforming to one or more profiles defined in Annex H of Rec. ITU-T H.264 - ISO/IEC 14496-10 | VideoTrack if decoding is supported |
|||||
0x21 | Video stream conforming to one or more profiles as defined in Rec. ITU-T T.800 - ISO/IEC 15444-1 | VideoTrack if decoding is supported |
|||||
0x22 | Additional view Rec. ITU-T H.262 - ISO/IEC 13818-2 video stream for service-compatible stereoscopic 3D services | VideoTrack if decoding is supported |
|||||
0x23 | Additional view Rec. ITU-T H.264 - ISO/IEC 14496-10 video stream for service-compatible stereoscopic 3D services | VideoTrack if decoding is supported |
|||||
0x24-0x7E | ITU-T Rec. H.222.0 - ISO/IEC 13818-1 Reserved | N/A | |||||
0x7F | IPMP stream | N/A | |||||
0x80-0xFF | User Private | |
ISOBMFF
The following table is based on the MP4 Registration Authority codec table and on the 2004 Edition of the ISO Base Media File Format standard (Download ISO/IEC 14496-12:2004).
id: The id should be a string containing the track_ID in TrackHeaderBox, formatted as a decimal number.
label: The label should be the name in HandlerBox.
language: The language should be determined from the language in MediaHeaderBox.
ISOBMFF Types | HTML 5 Track attributes | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Track Sample Entry 4-Char-Code | Description | Stream type (Track handler) | Defined in/by | MPEG-4 Stream Type | MPEG-4 Object Type Indication | Track Interface | kind | inBandMetadataTrackDispatchType | cue payload | ||
3gvo | 3GPP Video Orientation | Metadata | 3GPP | TextTrack | |||||||
ac-3 | AC-3 audio | Audio | ETSI | 0xA5 | AudioTrack if decoding is supported, TextTrack otherwise | ||||||
alac | Apple lossless audio codec | Audio | Apple | AudioTrack if decoding is supported, TextTrack otherwise | |||||||
avc1 | Advanced Video Coding | Video | AVC | 0x20 | VideoTrack if decoding is supported, TextTrack otherwise | ||||||
avc2 | Advanced Video Coding | Video | AVC | 0x20 | VideoTrack if decoding is supported, TextTrack otherwise | ||||||
avcp | Advanced Video Coding Parameters | Video | AVC | 0x20 | VideoTrack if decoding is supported, TextTrack otherwise | ||||||
dra1 | DRA Audio | Audio | DRA | 0xA7 | AudioTrack if decoding is supported, TextTrack otherwise | ||||||
drac | Dirac Video Coder | Video | Dirac | 0xA4 | VideoTrack if decoding is supported, TextTrack otherwise | ||||||
dtsc | DTS Coherent Acoustics audio | Audio | DTS | 0xA9 | AudioTrack if decoding is supported, TextTrack otherwise | ||||||
dtse | DTS Express low bit rate audio, also known as DTS LBR | Audio | DTS | 0xAC | AudioTrack if decoding is supported, TextTrack otherwise | ||||||
dtsh | DTS-HD High Resolution Audio | Audio | DTS | 0xAA | AudioTrack if decoding is supported, TextTrack otherwise | ||||||
dtsl | DTS-HD Master Audio | Audio | DTS | 0xAB | AudioTrack if decoding is supported, TextTrack otherwise | ||||||
ec-3 | Enhanced AC-3 audio | Audio | ETSI | 0xA6 | AudioTrack if decoding is supported, TextTrack otherwise | ||||||
enca | Encrypted/Protected audio | Audio | ISMAc, ISO | ||||||||
encs | Encrypted Systems stream | (various) | ISO | ||||||||
enct | Encrypted Text | Text | ISO | ||||||||
encv | Encrypted/protected video | Video | ISMAc, ISO | ||||||||
fdp$20 | File delivery hints | Hint | ISO | ||||||||
g719 | ITU-T Recommendation G.719 (2008) | Audio | ITU G.719 | 0xA8 | AudioTrack if decoding is supported, TextTrack otherwise | ||||||
g726 | ITU-T Recommendation G.726 (1990) | Audio | SDV | AudioTrack if decoding is supported, TextTrack otherwise | |||||||
ixse | DVB Track Level Index Track | Metadata | DVB | TextTrack | |||||||
m2ts | MPEG-2 transport stream for DMB | Hint | DMB-MAF | ||||||||
m4ae | MPEG-4 Audio Enhancement | Audio | MP4v1 | AudioTrack if decoding is supported, TextTrack otherwise | |||||||
m4ae | MPEG-4 Audio Enhancement | Audio | MP4v2 | 0x40, others | AudioTrack if decoding is supported, TextTrack otherwise | ||||||
mett | Text timed metadata | Metadata | ISO | TextTrack | |||||||
metx | XML timed metadata | Metadata | ISO | TextTrack | |||||||
mjp2 | Motion JPEG 2000 | Video | MJ2 | VideoTrack if decoding is supported, TextTrack otherwise | |||||||
mlix | DVB Movie level index track | Metadata | DVB | TextTrack | |||||||
mlpa | MLP Audio | Audio | Dolby MLP | AudioTrack if decoding is supported, TextTrack otherwise | |||||||
mp4a | MPEG-4 Audio | Audio | MP4v1 | 0x40, others | AudioTrack if decoding is supported, TextTrack otherwise | ||||||
mp4s | MPEG-4 Systems | (various) | MP4V1 | various | TextTrack | ||||||
mp4v | MPEG-4 Visual | Video | MP4V1 | 0x20, others | VideoTrack if decoding is supported, TextTrack otherwise | ||||||
mvc1 | Multiview coding | Video | AVC | VideoTrack if decoding is supported, TextTrack otherwise | |||||||
mvc2 | Multiview coding | Video | AVC | VideoTrack if decoding is supported, TextTrack otherwise | |||||||
oksd | OMA Keys | Metadata | OMA DRM XBS | ||||||||
pm2t | Protected MPEG-2 Transport | Hint | ISO | ||||||||
prtp | Protected RTP Reception | Hint | ISO | ||||||||
raw$20 | Uncompressed audio | Audio | MJ2 | AudioTrack if decoding is supported, TextTrack otherwise | |||||||
resv | Restricted Video | Video | AVC | VideoTrack if decoding is supported, TextTrack otherwise | |||||||
rm2t | MPEG-2 Transport Reception | Hint | ISO | ||||||||
rrtp | RTP reception | Hint | ISO | ||||||||
rsrp | SRTP Reception | Hint | ISO | ||||||||
rtp$20 | RTP Hints | Hint | ISO | ||||||||
s263 | ITU H.263 video (3GPP format) | Video | 3GPP | VideoTrack if decoding is supported, TextTrack otherwise | |||||||
samr | Narrowband AMR voice | Audio | 3GPP | AudioTrack if decoding is supported, TextTrack otherwise | |||||||
sawb | Wideband AMR voice | Audio | 3GPP | AudioTrack if decoding is supported, TextTrack otherwise | |||||||
sawp | Extended AMR-WB (AMR-WB+) | Audio | 3GPP | AudioTrack if decoding is supported, TextTrack otherwise | |||||||
sevc | EVRC Voice | Audio | 3GPP2 | 0xA0 | AudioTrack if decoding is supported, TextTrack otherwise | ||||||
sm2t | MPEG-2 Transport Server | Hint | ISO | ||||||||
sqcp | 13K Voice | Audio | 3GPP2 | 0xE1 | AudioTrack if decoding is supported, TextTrack otherwise | ||||||
srtp | SRTP Hints | Hint | ISO | ||||||||
ssmv | SMV Voice | Audio | 3GPP2 | 0xA1 | AudioTrack if decoding is supported, TextTrack otherwise | ||||||
stpp | Sub-titles (Timed Text) | Text | DECE | TextTrack | |||||||
svc1 | Scalable Video Coding | Video | AVC | VideoTrack if decoding is supported, TextTrack otherwise | |||||||
svcM | SVC metadata | Metadata | AVC | TextTrack | |||||||
tc64 | 64 bit timecode samples | Timecode | Apple | TextTrack | |||||||
text | Textual meta-data with MIME type | Metadata | MPEG4 | TextTrack | |||||||
tmcd | 32 bit timecode samples | Timecode | Apple | TextTrack | |||||||
twos | Uncompressed 16-bit audio | Audio | MJ2 | AudioTrack if decoding is supported, TextTrack otherwise | |||||||
tx3g | Timed Text stream | Text | 3GPP | TextTrack | |||||||
urim | URI identified timed metadata | Metadata | ISO | TextTrack | |||||||
vc-1 | SMPTE VC-1 | Video | SMPTE | 0xA3 | VideoTrack if decoding is supported, TextTrack otherwise | ||||||
xml$20 | XML-formatted meta-data | Metadata | MPEG4 | TextTrack |
WebM / Matroska
See: The WebM spec and the Matroska spec.
id: The id should be a string containing the TrackUID formatted as a decimal number.
label: The label should be be the Name, decoded from UTF-8.
language: The language should be determined from the Language.
OGG
id: If the file contains a Skeleton section, the id should be the contents of the Name header, decoded from UTF-8. Otherwise, it should be a string containing the stream serial number, formatted as a decimal number.
label: If the file contains a Skeleton section, the label should be the contents of the Title header.
language: If the file contains a Skeleton section, the label should be determined from the Language header.
TODO: The Skeleton section itself should probably be exposed as a metadata track, and so should any unknown streams.