This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.

Bug 26207 - Provide a way to check system capabilities required for UHD playback
Summary: Provide a way to check system capabilities required for UHD playback
Status: RESOLVED WORKSFORME
Alias: None
Product: HTML WG
Classification: Unclassified
Component: Encrypted Media Extensions (show other bugs)
Version: unspecified
Hardware: PC Windows NT
: P2 normal
Target Milestone: ---
Assignee: Jerry Smith
QA Contact: HTML WG Bugzilla archive list
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-06-26 19:57 UTC by Jerry Smith
Modified: 2014-08-12 15:50 UTC (History)
6 users (show)

See Also:


Attachments

Description Jerry Smith 2014-06-26 19:57:49 UTC
We need a way for sites to know aspects of the media pipeline to gauge whether SD, HD or UHD content may be streamed.  This is ultimately enforced by the media pipeline/DRM against license requirements.  The ability to check capabilities in advance might be used to improve the user experience.  Without it, sites must attempt UHD streaming and fallback if playback fails.

This bug continues a conversation at the end of https://www.w3.org/Bugs/Public/show_bug.cgi?id=24873.
Comment 1 Jerry Smith 2014-07-08 00:57:34 UTC
IsTypeSupported currently has an optional DOMString capability attribute that might be used to abstract some set of requirements for HD or UHD playback.  This could be a proprietary label per CDM or a label that represents some industry agreement among providers and content owners on a standard for playback.  These don't currently exist.

I see three options for dealing with this:

1. Limit EME to the DOMString capability attribute we currently have.
   a. This MAY be sufficient, but could depend on proprietary labels.
2. Provide some way to pre-check a license without the delay of downloading content, attempting playback and failing.
   a. This MIGHT be possible without a lot of EME change, but is TBD.
3. Provide a way to check against a set of criteria relevant to UHD playback.  
   a. This seems like the most direct approach, but has some negatives listed below.

A capabilities query has the advantage of letting content owners tune their playback bar when there isn't yet a consensus bar to meet.  A list of capabilities should include:

1. Hardware DRM
2. Hardware media pipeline
3. Display link protection
4. Media parameters needed to take advantage of UHD content

Returning capabilities data has at least a couple of negatives:

1. Returned data and even pass/fail verification on a specific set of capabilities might be used for fingerprinting.  This could require limiting the capabilities returned or adding a way for users to approve disclosing the data.
2. Any specific set of criteria could omit one that eventually becomes important to content owners.  The criteria needs to be reasonably specific to insure it is evaluated consistently on different browser/CDM implementations.

I propose modifying IsTypeSupported to return either the capability DOMString OR a specific mediaCapabilities dictionary that attempts to answer key playback questions without returning a large amount of fingerprinting data.  It would be queried as a set and return our equivalent of a pass/fail response ("maybe", "probably").  A draft WebIDL for discussion is included below:

static IsTypeSupportedResult isTypeSupported(DOMstring keySystem, optional DOMString initDataType, optional DOMString contentType, optional mediaCapabilities capability);

typedef (capabilities or DOMString) mediaCapabilities;

dictionary capabilities {
  bool hardwareMediaDecrypt;
  bool hardwareMediaPipeline;
  DOMString displayLinkProtection;
  bool timeBasedRestrictions
  short minDisplayLines
  short audioChannels;
  short colorBitDepth;
};
Comment 2 Mark Watson 2014-07-09 01:20:46 UTC
This is also related to Bug 25092.

A further capability which needs to be checked for UHD (and HDR and 5.1 audio and in principle for HD) is whether the output device is physically capable of rendering the content. In some cases devices will downscale / downmix but still there is no point in downloading and decoding quality higher than that which can be physically rendered. These things have nothing to do with DRM.

Output protection is not as simple capability that can be queried once: a device may only attempt to engage output protection when requested in a license. Even then, whether a failure to engage (a specific level of) output protection means the content cannot be rendered depends on the license policy and device capability.

I think we need a mechanism - perhaps based on media queries - which can query the physical resolution of the video element display area - accounting for output protection restrictions. Also notifications for when the state of this query changes.
Comment 3 David Dorwin 2014-07-11 18:59:16 UTC
(In reply to Jerry Smith from comment #1)
UHD might be a use case, but I don't think this feature request is more general and the summary should reflect this.

My initial thoughts:

#1 is simple and perhaps the short-term solution (it's already defined) while the other options are explored.

#2 may be the best way to provide answers to robustness questions. The server/application knows what types of licenses it supports, so why not let it ask the CDM directly. Anything else requires the application and UA to translate robustness requirements to text and text to robustness capabilities, respectively. This model is also consistent with getUsableKeyIds().

However, this may not always be sufficient since licenses don't normally specify the types of media (audio vs. video) or codec(s) to be used with each key. Without such information, applications might get false positives or negatives.

Also, it's unclear how this would work since licenses are usually issued after a license request and the license may depend on information from the device, which is not possible with such a simple query. There may be room for improvement here, but it's something to keep in mind.

#3 - There is a general lack of powerful capability detection for HTMLMediaElement, most of which is unrelated to robustness. I think that should be solved in general outside of EME. I believe this needs to be solved for unprotected content and there are a lot of stakeholders and interested parties that may not be participating in the EME specification.

Such a discussion will either solve the robustness problem or provide a model EME can build on. While such a querying API might be easily extensible, the robustness-media type mapping seems like something that may not fit well on a generic solution. Also, output protection might be the only capability for which detection may require a noticeable delay on subsequent calls. Such a delay could make the querying capability less useful since the goal is to start fetching the right media streams as quickly as possible.

I do not think standardizing labels for proprietary robustness mechanisms makes sense, at least without corresponding standardization of related aspects of the license. It seems difficult to come up with a consistent vocabulary that will work across key systems without excluding some valid scenarios. Content providers define their needs in the license, which is what makes #2 attractive.
Comment 4 Bob Lund 2014-07-22 15:10:39 UTC
(In reply to David Dorwin from comment #3)
> (In reply to Jerry Smith from comment #1)
> UHD might be a use case, but I don't think this feature request is more
> general and the summary should reflect this.

As you and Mark point out, this feature could be more generally applicable but gets into checking for media capabilities beyond content protection and output protection. General media capability checking wouldn't be defined by EME.

EME should focus on the content protection related checks.

> 
> My initial thoughts:
> 
> #1 is simple and perhaps the short-term solution (it's already defined)
> while the other options are explored.
> 
> #2 may be the best way to provide answers to robustness questions. The
> server/application knows what types of licenses it supports, so why not let
> it ask the CDM directly. Anything else requires the application and UA to
> translate robustness requirements to text and text to robustness
> capabilities, respectively. This model is also consistent with
> getUsableKeyIds().
> 
> However, this may not always be sufficient since licenses don't normally
> specify the types of media (audio vs. video) or codec(s) to be used with
> each key. Without such information, applications might get false positives
> or negatives.
> 
> Also, it's unclear how this would work since licenses are usually issued
> after a license request and the license may depend on information from the
> device, which is not possible with such a simple query. There may be room
> for improvement here, but it's something to keep in mind.
> 
> #3 - There is a general lack of powerful capability detection for
> HTMLMediaElement, most of which is unrelated to robustness. I think that
> should be solved in general outside of EME. I believe this needs to be
> solved for unprotected content and there are a lot of stakeholders and
> interested parties that may not be participating in the EME specification.
> 
> Such a discussion will either solve the robustness problem or provide a
> model EME can build on. While such a querying API might be easily
> extensible, the robustness-media type mapping seems like something that may
> not fit well on a generic solution. Also, output protection might be the
> only capability for which detection may require a noticeable delay on
> subsequent calls. Such a delay could make the querying capability less
> useful since the goal is to start fetching the right media streams as
> quickly as possible.
> 
> I do not think standardizing labels for proprietary robustness mechanisms
> makes sense, at least without corresponding standardization of related
> aspects of the license. It seems difficult to come up with a consistent
> vocabulary that will work across key systems without excluding some valid
> scenarios. Content providers define their needs in the license, which is
> what makes #2 attractive.
Comment 5 Jerry Smith 2014-08-12 15:21:17 UTC
We have agreed that the broader system capabilities topic is too broad to be fully addressed in the EME spec, and that the existing capabilities string looks useful for communicating feature combinations specified by the CDM.

I would like to continue experiments on pre-validating license terms (e.g. HDCP 2.2); but do not believe this bug should be held open while that is happening.  I recommend we close this bug.
Comment 6 Jerry Smith 2014-08-12 15:50:54 UTC
Changing resolution to works for me.