Adaptive Bit Rate Parameters, Error Codes and Feedback
This page specifies the proposed parameter, error codes and feedback values to support adaptive bit rate delivery of media.
- An application developer would like to enable an end-user to select only an "HD" level (or other high quality) stream.
- An application developer would like to enable an end-user to select a lower-quality stream to compensate for "hunting" in the heuristics.
- An application developer would like to enable an end-user to limit their overall bandwidth usage.
- An application developer would like to experiment with different adaptive heuristics.
- An application developer would like to report on CDN (Content Delivery Network) health and performance by getting playback statistics from widely deployed players.
- An application developer would like to provide appropriate responses to error conditions specific to adaptive playback.
- An application developer would like to report on the quality of a user's experience by monitoring playback statistics & script callbacks (for quality changes)
- An application developer would like to report which bitrates are performing best across different devices, networks, & delivery methods on widely deployed players
- An application developer would like to create a seamless video experience by appending segments or video sources without stopping playback
The current HTML5 model does support adaptive streaming, but in a very limited way.
- No Control (current spec) - In this model, the adaptive implementation is in the user agent and provides no APIs for application control.
There are three primary architectural models to consider for a more flexible and powerful HTML adaptive bit rate solution.
- Minimal Control - In this model, the manifest file and heuristics are handled by the user agent. A few parameters or interfaces may be implemented to have some effect on the management of the adaptive bit rate process, but the user agent is in control. This model has the advantage of a simple interface at the expense of limited ability to modify the design.
- Adaptation Control - This model provides additional script manageable features such as an ability to directly set the level (e.g. bitrate) of the media to use for each segment. The manifest file is still digested by the user agent, but the additional control gives some flexibility in how adaptation strategies are implemented.
- Full Media Control - This model extends the adaptation control model by allowing script to explicitly send the media segments. The manifest file is processed by script and any adaptation algorithm can be used to select the individual segments for playback.
- In addition to script access, parameters might be passed directly in the HTML media element tag.
- The models are not necessarily exclusive. For example, the WHATWG document suggests that setting a level directly from script would disable the heuristics of the UA implementation and setting the level to -1 would restore adaptation control to the UA.
Here is the list of proposed parameters and definition of those parameters.
- maxLevel (input, position) - This parameter specifies the maximum quality or bitrate level for media delivery and if present the position at which the level shall earliest be set. This is useful in the management of a limited bandwidth resource that is shared by multiple streams. This parameter can help ensure a uniform experience for multiple users rather than an inconsistent experience. This would fit into architectural model 1.
- minLevel (input, position) - Specify a minimum quality level or bitrate and if present the position at which the level shall earliest be set. This is provided in conjunction with maxLevel for the use-case where an application wishes to provide a facility for the end-user to lock their bitrate at a particular level to prevent "hunting". This would fit into architectural model 1.
- startingLevel (input) - Specify the quality level at which playback should begin. Some adaptive schemes do this implicitly based on the order of quality levels in the manifest. Explicit control would be beneficial to applications wishing to exercise greater control over the initial quality level displayed. This would fit into architectural model 1.
These codes are proposed as HTML error codes to be added to the current error code list.
Additional HTML error codes that are not specific to ADB are proposed here: HTML network Error Codes
This section describes how information communicted back from the user agent to the client application. This approach uses callbacks to send asynchronous feedback to the client application. These values would be used in architectural models 2 and 3. The suggested feedback values (obtained through <video> element accessors):
- downloadRate - The current server-client bandwidth
- videoBitRate - The current video bitrate
- droppedFrames - The total number of frames dropped for this playback session
- decodedFrames - The total number of frames decoded for this playback session
- bufferLength - The current length of buffered video (in media time)
Window and video dimensions may also be useful, but are already defined.
Duncan suggested adding the following.
- currentSegmentDuration - The duration of the current segment being downloaded (in media time)
- currentSegmentDownload - The duration of the segment already downloaded (in media time)
- nextSegmentDuration - The duration of the next segment to be downloaded (in media time)
- currentSegment - The segment number of the segment currently being downloaded
The following APIs allow adaptive streaming to be controlled from script without requiring script to actually send the media:
- setLevel(level) - Sets the stream level (would disable UA control of this). Using level = -1 would restore UA control. This would be part of architectural model 2.
- appendVideo(url, [range]) - Appends the next segment of video to the stream. We may also want to allow the segment to be defined by media time. This would be useful for real-time broadcast media and would be used in architectural model 3.
- segmentLoaded(segment) - The specified segment was successfully loaded. (Models 2 & 3)
- representationChanged(bitrate) - When the adaptive heuristic triggers a change to the specified bitrate (All Models)
- adaptationSetChanged(qualityLevels, position) - Notification that the set of available quality levels has changed, including an array of the available bitrates (Models 2 & 3). The position at which the new qualityLevels take place shall be indicated by the UA.