Bug 16479 - Video hardware resources management
Video hardware resources management
Status: NEW
Product: Web & TV
Classification: Unclassified
Component: Media Pipeline TF
unspecified
PC Windows 3.1
: P2 normal
: ---
Assigned To: Clarke Stevens
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2012-03-22 15:20 UTC by David Corvoysier
Modified: 2012-12-04 00:52 UTC (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description David Corvoysier 2012-03-22 15:20:53 UTC
**Description**:

On devices using hardware acceleration to render video, the number of video pipelines allowed to run in parallel is constrained (even high-end devices are typically not able to decode more than two full HD videos in parallel). 
This may lead to conflicts between several video elements competing for the same  hardware resources.

The HTML5 specification is currently silent about how such conflicts are resolved, and very few details are provided about the expected lifecycle of video elements:

Section §4.8.10.8 (normative) specifies that a video element that is detached from a Document goes into a pause state:
"...
When a media element is removed from a Document, the user agent must run the following steps:
...
- Pause the media element.
..."
Meaning that even when a page is destroyed, the hardware resources may still be preempted by a video element waiting for garbage collection (this is actually the case in at least two WebKit ports).

Section §4.8.10.18 however adds that resource constrained devices should release hardware resources.

"When a media element that is paused is removed from a document and not reinserted before the next time the event loop spins, implementations that are resource constrained are encouraged to take that opportunity to release all hardware resources (like video planes, networking resources, and data buffers) used by the media element. (User agents still have to keep track of the playback position and so forth, though, in case playback is later restarted.)"
... but it is informative !

To summarize, the UC is to provide means to an application to:
- detect video resources conflicts,
- explicitly release video resources it may have preempted.

**Need/justification**
Allow applications using video elements to coexist and/or degrade gracefully.

**Category**
This probably requires an evolution of the HTML5 specification

**PROPOSAL**:

- add more error codes to video playback (MEDIA_ERR_CONFLICT?)
- either move the language in 4.8.10.18 in a normative section, or add a specific procedure to release resources (video.gc() ?)
Comment 1 Philip Jägenstedt 2012-04-10 08:34:01 UTC
The way we (Opera) have dealt on this on one platform where it wasn't possible to create more than ~20 audio backends without crashing was to put the remaining backends on a "creation queue" to be started when the existing ones were destroyed. This makes it look to the script as if the network is just (very) slow and works well enough. To allow scripts to opt in to detecting this situation, I wouldn't object to a new event of the same caliber as stalled (waiting for network) and waiting (waiting for network, but tied to readyState for some reason), perhaps "yield" or something?