Bugzilla – Bug 16479
Video hardware resources management
Last modified: 2014-11-11 23:06:33 UTC
On devices using hardware acceleration to render video, the number of video pipelines allowed to run in parallel is constrained (even high-end devices are typically not able to decode more than two full HD videos in parallel).
This may lead to conflicts between several video elements competing for the same hardware resources.
The HTML5 specification is currently silent about how such conflicts are resolved, and very few details are provided about the expected lifecycle of video elements:
Section §220.127.116.11 (normative) specifies that a video element that is detached from a Document goes into a pause state:
When a media element is removed from a Document, the user agent must run the following steps:
- Pause the media element.
Meaning that even when a page is destroyed, the hardware resources may still be preempted by a video element waiting for garbage collection (this is actually the case in at least two WebKit ports).
Section §18.104.22.168 however adds that resource constrained devices should release hardware resources.
"When a media element that is paused is removed from a document and not reinserted before the next time the event loop spins, implementations that are resource constrained are encouraged to take that opportunity to release all hardware resources (like video planes, networking resources, and data buffers) used by the media element. (User agents still have to keep track of the playback position and so forth, though, in case playback is later restarted.)"
... but it is informative !
To summarize, the UC is to provide means to an application to:
- detect video resources conflicts,
- explicitly release video resources it may have preempted.
Allow applications using video elements to coexist and/or degrade gracefully.
This probably requires an evolution of the HTML5 specification
- add more error codes to video playback (MEDIA_ERR_CONFLICT?)
- either move the language in 22.214.171.124 in a normative section, or add a specific procedure to release resources (video.gc() ?)
The way we (Opera) have dealt on this on one platform where it wasn't possible to create more than ~20 audio backends without crashing was to put the remaining backends on a "creation queue" to be started when the existing ones were destroyed. This makes it look to the script as if the network is just (very) slow and works well enough. To allow scripts to opt in to detecting this situation, I wouldn't object to a new event of the same caliber as stalled (waiting for network) and waiting (waiting for network, but tied to readyState for some reason), perhaps "yield" or something?