Meeting minutes
New media actions for the Media Session API
Add new actions to support video conferencing websites (#264)
mounir: Jer, I believe you wanted to add "hang up". Same request internally a couple of weeks later.
tommy: Media session allows web sites to subscribe to actions, e.g. "next track" in a playlist. All actions have been geared to video playback.
… Considering actions for conersations and WebRTC.
… Ways to toggle the microphone and camera, and hang up.
… Also two actions to get the current state
ericc: "The browser wouldn't know the state the camera/microphone status", what does that mean?
tommy: If you're talking and muted on Hangout, it's still listening to your microphone to tell you you might be muted.
… hard to tell the difference.
ericc: You're not saying that the camera/microphone is actually active, rather whether the app is using the data or not?
tommy: Right, we couldn't tell without the app telling us.
ericc: As far as the actions go, with the existing media session actions, let's say there is a media session that registered play/pause and the user hits a hardware key which, if the session was not active would play/pause the media element
… When there's a media session and there's a play action registered, the action handler gets called, the UA no longer does anything, it's the responsibility of the page to play/pause the media element.
tommy: I believe that's correct for play/pause.
mounir: Play/pause are a special case where the spec allows UA to have a default implementation. Chrome will play/pause things. It is a bit hard however to do "next track".
tommy: We don't have any plan to provide default handlers for the 3 new actions.
mounir: We may be able to do something with "hang up", but we haven't researched that for now.
ericc: The thing that I'm concerned about is the user takes an action which they believe will stop the feed from the camera, and the UA is delegating responsibility for that to the page because there is a media session, and the page can either honor that or not, which seems like a bad thing.
… A nefarious site could make it look as though the camera had been turned off.
mounir: In my opinion, you already provided the media feed to that page.
… If you press mute on the app UI, you already need to trust the app.
… I believe that's the opposite, the app delegates the UI to the UA in this case
jernoble: It seems very easy for us, UA, to mute the mike, stop video frames from being delivered, etc.
… The problem here is that these actions may be interpreted to be part of the UA chrome, so there's a trust issue here.
… Different UAs may be able to treat this differently. We might just decide that when the mute button is clicked, we'll just mute the audio/video, to err on the side of users expectation rather than page expectation
ericc: To expand upon that a bit, I was thinking about the current affordances that we have in Safari browser chrome, and I was assuming that for play/pause, we would send an action to the media session with a registered handler. That's where the worry is coming from.
… As long as there is wording in the spec that the UA can have a default action for these actions, which it does even if there is a registered action handler, that should be fine.
tommy: We already do that for play/pause. It makes sense to me to extend it to these actions as well.
mounir: Would you want a default action for all of them? toggle camera/mike, hang up?
jernoble: I believe it will be difficult to hang up a WebRTC session automatically.
mounir: There is a very Chrome specific problem around that. I know it's doable, but it could be a nuclear bomb for the web site which may not be able to revover from it.
… If we are able to do a default action for hang up, most likely Safari should be able too.
… Will have to check with Mozilla folks (not around for this call).
… Conclusion: being able to handle toggle camera/microphone directly by the UA. You will shut down the camera/microphone regardless of what the app is doing with these actions.
jernoble: Correct.
New Media WG charter
tidoust: the current Media WG charter expires at the end of May so it needs to be renewed to continue this work
… technically, we have to have a new charter by tomorrow as it has to be reviewed
… I prepared a pull request starting from the current Media WG charter to refresh it
tidoust: there is also a refresh of deliverables list, the main change was to add WebCodecs
… which was adopted and was already a potential deliverable
… I dropped the EME feature around persistent usage record as it was dropped
tidoust: I prepared a pull request with these changes and I would like to hear your feedback
tidoust: any feedback that can be shared during the call?
tidoust: I will merge the pull request and kick off the W3C internal review process
tidoust: Please look at it and provide feedback as soon as possible
Synchronize /TR documents with Editor's Drafts
Synchronize /TR documents with Editor's Drafts (#27)
tidoust: I raised the issue and asked for feedback but didn't here much
… so I suggest that we adopt this unless we have comments
chcunningham: I'm supportive
tidoust: the group will then process forword with this
Move WebCodecs to the Media WG
Move WebCodecs to the Media WG
mounir: I sent an email this morning. WebCodecs adopted as a Media WG deliverable.
… For publication as FPWD, Google made some comments around registries. This holds publication as FPWD but should be merged soon.
Define MediaCapabilitiesDecodingInfo.codecSwitchingSupported
Define MediaCapabilitiesDecodingInfo.codecSwitchingSupported (#165)
chcunningham: API for transition: ability to switch from one codec to another via the MSE changeType API.
… It doesn't have any way to signal its capabilities about changes that are supported.
… Only way to discover that is the hard way by trying changes.
… Difference between the clear pipeline and eme playback is needed, addressed in the PR.
… Feedback I got from Shaka player on the initial design was that it was too complicated. A boolean "can I call changeType in the current context?" seems more useful.
… For the clear pipeline, that works smoothly. For the EME pipeline, there was never a moment where we were in a situation where a transition was supported and another transition was not.
… So proposal is true/false in the context of the pipeline, which is either clear/encrypted (key system + robustness)
… I didn't feel that we really have had feedback from other user agents. I'd like to call out for feedback.
cpn: Do you have input from Jon Piesing from an embedded browser perspective?
… They have rather more constrained pipelines that may warrant special recommendations.
jernoble: "Can I switch to this new pipeline" or "Can I switch to this new codec within this pipeline?"
chcunningham: The latter.
… Once you have the key system established, adding/removing media keys may not be possible anymore depending on the user agent.
… Not a required feature to go from one pipeline to another.
jernoble: Wondering about clear to encrypted scenarios.
chcunningham: I haven't thought about this. If you're doing clear and switching to encrypted today, is there a question there?
jernoble: I'm assuming web citizens may see this new API and be tempted to call it.
chcunningham: I'd be happy to add a clarification.
GregFreedman: We have clear to encrypted scenarios. Clear intro (a.k.a. bumper) and then we switch to encrypted playback.
… We're reusing the video element as much as possible. I can't say that we're using changeType today
… I could see us switching to a trailer at the end of a series in a different codec.
chcunningham: In Chrome, it's not very interesting. That second "changeType" call, if you haven't set up your key system yet, you'll end up doing a pipeline switch.
… The pipeline you're entering into will be exposed through media capabilities.
GregFreedman: Should we start playing our clear content within the encrypted pipeline?
chcunningham: That would work today. My point is that you're not switching codecs, you're switching pipelines entirely. That's a different thing.
… The question "Do you support codec switching smoothly when you switch pipelines?" is difficult to answer.
… Clear to encrypted is a very legitimate thing to do. I guess I wouldn't think this API is the right approach for that.
… I would do my ad with codec A in the clear, then set up the encrypted pipeline, then call changeType with the new codec and then start appending.
gkatsev: In video.js, if we're playing content that has clear and encrypted, we setup the encrypted pipeline before we start playing clear content.
chcunningham: There were definitely some implementation issues within Chrome going between clear and encrypted.
… The key system configuration in Media Capabilities describes the pipeline to be used.
gkatsev: Having explicitly things is great.
Announcement
mounir: I'm leaving Chrome at the end of this week, so will step down as co-chair for this group. Francois and Jer are looking for someone to replace me. I may or may not stick around for next call.
jer: Many thanks, Mounir, for the work you've been doing in the group in the past couple of years!
<cpn> +1