This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.

Bug 24216 - setMediaKeys() needs more specification
Summary: setMediaKeys() needs more specification
Status: RESOLVED FIXED
Alias: None
Product: HTML WG
Classification: Unclassified
Component: Encrypted Media Extensions (show other bugs)
Version: unspecified
Hardware: All All
: P3 normal
Target Milestone: ---
Assignee: David Dorwin
QA Contact: HTML WG Bugzilla archive list
URL:
Whiteboard:
Keywords:
Depends on: 25199
Blocks:
  Show dependency treegraph
 
Reported: 2014-01-07 00:49 UTC by David Dorwin
Modified: 2014-04-16 00:09 UTC (History)
5 users (show)

See Also:


Attachments

Description David Dorwin 2014-01-07 00:49:08 UTC
The setMediaKeys() algorithm currently says that the “keys” attribute is set, but there are other things that should happen and corner cases we should at least give guidance on.

tl;dr: I recommend adding:
  * A step related to Using the Newly Attached MediaKeys Object (first section)
  * Implementation of Option #2 related to Detaching (and re-attaching) a MediaKeys Object (second section).
    + That is, setMediaKeys() may throw an exception if a change involving either the HTMLMediaElement or MediaKeys is already in progress.
  * Adding a note that Detaching/Replacing a MediaKeys Object During Playback (third section) is a quality of implementation issue but likely to result in a bad user experience or exception.


*** Using the Newly Attached MediaKeys Object ***
First, it seems that there should be some attempt to use the keys in the MediaKeys object’s session(s) if appropriate. For example, createSession() says:
  “If the associated media element(s) are waiting for a key, queue a task to attempt to resume playback.
  “In other words, resume playback if the necessary key is provided.
  “The user agent may choose to skip this step if it knows resuming will fail (i.e. no usable key was added).”

Like createSession(), this should probably occur in a task, which means it is asynchronous and may not affect the very next frame.


*** Detaching a MediaKeys Object ***
The algorithm does not address the replacement/detaching of a previously set MediaKeys object.

== Detaching is Asynchronous ==
Like attaching, replacement/detaching is probably asynchronous since the media pipeline must be disrupted.

Because the setMediaKeys() method is synchronous, when replacing non-null MediaKeys A with non-null MediaKeys B:
I. An HTMLMediaElement implementation could have references to two CDM instances at the same time.
II. Immediately passing the disconnected MediaKeys A to setMediaKeys() on another HTMLMediaElement could be difficult to handle (the previous HTMLMediaElement is still disconnecting from it).

Scenario II above may be okay in situations where the MediaKeys object was NOT being used by the maximum number of HTMLMediaElements. However, with the maximum expected to be one in many cases, it could be a problem.

Note that even a null MediaKeys B could be a problem if a non-null MediaKeys C is specified before Media Keys A is detached.

== Questions ==
The following questions may be helpful for discussing and understanding the issues:
1. Is detaching expected to be supported or is it a quality of implementation issue?
2. Under what conditions is detaching/replacement allowed?
  2.1. What should happen in case II when a detach is pending and the maximum number of HTMLMediaElements are using the MediaKeys implementation?
3. What should happen if either is disallowed?
4. How can the application know that it is safe to call setMediaKeys() on an HTMLMediaElement or with a MediaKeys object?
5. What happens if a session is detached during the asynchronous attachment process?
6. Is it okay for implementations to only support A -> null -> B instead of A -> B? (This probably provides no value as explained earlier.)
7. Is it okay for implementations to only support detaching after setting src to null? (Even then, it’s not clear when it can be attached to a new media element.)
8. Is it okay for implementations to not allow detaching once attached?

Whether some of these things can be supported is probably a quality of implementation detail, but we know there will be implementations that don’t support them and we should define a consistent behavior for applications to expect.

== Options ==
Many of the issues relate to the window between when setMediaKeys() is called and when the underlying media stack starts using or stops using the CDM. We either need to make setMediaKeys() be asynchronous (#2 and #3 below) such that serially attaching and detaching among media elements is not expected to be supported or ensure that setMediaKeys() can run synchronously (#1 below).

I’m leaning toward #2 with the possibility of extending it to #3 in the future if there are use cases where events are useful. At the moment, #3 seems like more specification and implementation that is necessary, especially for such an unlikely use case. #2 provides a clear definition of what implementations should do in these corner cases.

#1 seems too limiting, though I can’t think of a good use case that it prevent. #2 certainly allows a user agent to implement #1.

For all options except maybe #1, we may need to note that *detaching* might be disallowed by some implementations in some or all cases. We should specify that such implementations might throw {pick one of NO_MODIFICATION_ALLOWED_ERR, INVALID_MODIFICATION_ERR, or INVALID_ACCESS_ERR}.  We may even want to be more explicit and add a step that says if the currently attached object cannot be *detached*, throw the exception. This 

+ Option #1: Only allow detaching when stopped (null .src?). Throw an exception in all other cases.
This is simplest for implementations since the CDM would not be in use and should allow detaching to be synchronous.
Are there realistic uses for detaching when not stopped?
Does this cover all cases?

+ Option #2: Note that setMediaKeys() may throw an exception if a change involving either the HTMLMediaElement or MediaKeys is already in progress.
Step 1 already says to throw a QUOTA_EXCEEDED_ERR exception when the MediaKeys is “in use and the user agent is unable to re-use it”. We could add text to note that this may include objects that have been detached and are not (yet) available for reuse. In other words, “in use” does not necessarily mean “currently attached” from the application’s point of view.

This is reasonably broad and should cover the asynchronous detaching cases. However, applications have no way (other than trial and catch) to know when they can attach to another HTMLMediaElement.

Depending on what other changes we make to the algorithm, we might define internal states to help in the definition of algorithms related to throwing the exception.

+ Option #3: Define event(s) on the MediaKeys object related to (dis)connection.
This is similar to #2 but provides a formal way for applications to know when the MediaKeys can be attached to another HTMLMediaElement and vice versa.

We could define a single event on MediaKeys for when detaching is complete. Applications need not wait for the event since an implementation could support attaching to multiple HTMLMediaElements. However, if setMediaKeys() throws QUOTA_EXCEEDED_ERR, the application could wait for this event.

(If we disallow A -> B (Option #4)): Since the event is always fired - regardless of whether more attachments are supported - applications could also use it to know when the HTMLMediaElement is ready to accept a new object.)

Note that the name of the event needs to focus on a detach occurring rather than the state of the MediaKeys because we need to allow implementations to support attaching a MediaKeys to more than one HTMLMediaElements.

My understanding is that such an event is similar to MSE, which asynchronously closes the MediaSource object when it is detached and does not allow it be attached to a new HTMLMediaElement until it is in the closed state. See https://dvcs.w3.org/hg/html-media/raw-file/default/media-source/media-source.html#mediasource-attach.

+ Option #4: Disallow A -> B transitions and require A -> null -> B.
It’s unclear that this actually solves anything since the two operations could be executed back-to-back.


*** Playback Experience When Detaching/Replacing a MediaKeys Object  During Playback ***
(This section specifically covers the playback experience and assumes the mechanics have been addressed in the previous section. It does not apply if Option #1 above is selected.)
Unless playing a large amount of clear content, the MediaKeys object is required for playback. The CDM it represents is also likely to be decoding content, which means detaching it could disconnect the decoder(s). (This is why it is recommended that applications attach a MediaKeys object before providing the content. [https://dvcs.w3.org/hg/html-media/raw-file/default/encrypted-media/encrypted-media.html#dom-setmediakeys]).

Questions:
1. Are applications allowed to attach/detach a MediaKeys object from an HTMLMediaElement while the latter is “potentially playing” (or various other states with a valid source)?
2. If so, what is the behavior when this happens?
  2.1. Should applications expect to be able to seamlessly switch from one MediaKeys to another?
  2.2. Is this a quality of implementation issue (playback may stop, briefly stall, etc.)?

To allow implementation flexibility, we probably want to allow implementations to throw an exception in this case. See the discussion of such an exception in the previous section.
Comment 1 Joe Steele 2014-02-18 08:05:00 UTC
I am not sure I understand the use case here. Wouldn't any use case requiring a mid-play MediaKeys switch also stall the decryption pipeline because the right keys are not present?

I would expect the CDM to send a key request at that point, not for the application to try independently to switch the keys needed.

Or are you thinking that the CDM would not have the right material to initiate the key request? I.e. a new createSession is required.  

In that case, wouldn't it be cleaner to have a recreateSession() call on the MediaKeySession object that takes the initData? Then you could reuse the existing object and push any concerns about which keys are present down into the CDM where they belong.
Comment 2 David Dorwin 2014-02-18 16:50:33 UTC
Discussed in the telecon today. There were no objections to option #2, though Jerry thought #1 was sufficient. I'll take another look at how simple it is to implement #1 within #2 and how much extra spec text #2 is vs. #1.
Comment 3 David Dorwin 2014-03-04 01:16:16 UTC
Hopefully addressing comment #1 and providing info discussed in the telecon, I don't have any such use cases in mind - and taking some of these actions would likely result in interrupted playback. However, the separation of HTMLMediaElement and MediaKeys makes these things possible so the spec should address them, even if that just means saying they are not allowed.
Comment 4 David Dorwin 2014-03-26 18:18:03 UTC
If we intend for the spec to allow setMediaKeys(null), we need to add a '?' to the parameter in the IDL.
Comment 5 David Dorwin 2014-04-16 00:09:02 UTC
https://dvcs.w3.org/hg/html-media/rev/a6c2ab4c2374 implements Option #2 and effectively provides the capabilty of Option #3 via the asynchronously resolved promise. All errors, including implementation limitations, are reported by rejecting the promise.

This changeset also includes a note about replacing a MediaKeys object during playback and adds '?' to the MediaKeys parameter to indicate that null is allowed.