Meeting minutes
Agenda bashing
Matt_Wolenetz: I would like to give some update on MSE in workers
MSE in workers
Matt_Wolenetz: PR #306. MSE in workers, using a handle, some back and forth on transferable vs. movable.
… Implementing "SameObject" proposal right now in Chromium for the handle
… Just a tiny update. Origin trial. Twitch has noticed great improvements on key metrics. They're waiting for the handle piece.
… I appreciate everyone's feedback on this.
Mark_Watson: Sorry that I haven't been much active recently. Happy to take a look though. If volunteers are willing to take over as co-author, they would be welcome. In the absence of volunteers, I'll do it.
… I don't want to be the one delaying progress.
Jan-ivar: Thank you for being responsive to our feedback.
cpn: What would that entail for implementation milestones?
Matt_Wolenetz: The changes were pretty heavy. Transfer-only semantics are kind of weird. The serializable attempt was a way to workaround this and many of the underlying pieces that were worked out for Serializable in implementation and spec apply directly to the new Transferable handle spec/impl.
… The only thing remaining is SameObject and tests for that, and getting approval.
… As far as origin trial goes, the current trial ends in Chrome 103 and is for the object URL worker attachment and all of the rest of the stabilized usage of MSE-in-Worker after attachment has started. It doesn't include the more recent switch to transferable-only MediaSourceHandle for Dedicated Worker MediaSource attachment, nor the SameObject way of [CUT] Hopefully, no issues, but if there are, they should be fixed pretty fast, and plan to release after that.
Matt_Wolenetz: Chrome throws exception if retrieving handle from main-thread-owned MediaSource (only allowed on DedicatedWorker). Handle is transferable within same agent cluster (e.g., not via a SharedWorker or ServiceWorker) for use in attachment via srcObject on HTMLME.
Jan-Ivar: That sounds great, thanks.
Media Sesion and capture handle actions
cpn: For context, the WebRTC WG had discussions over how you do presentations over WebRTC-like presentations.
… Page that runs the conferencing session and the page that hosts the presentation, and users may want control over the presentation from the conferencing session.
… Media Session currently gives control of browser-initiated actions, such as play/pause. More recently, we added actions to toggle mike and camera.
… Question is now how to handle the WebRTC scenario.
… Should we look to use the same spec and infrastructure? Or are there more separatable things?
… We've had a couple of discussions on this already.
… I'd like us to move to some kind of decision that can help the capture handle work move forward.
Jan-Ivar: The way this came about was perhaps sideways. Some work for better integration of a page that is capturing another page. Google Meet is a great example.
… Driving use case is indeed conferencing with presentation window that I may want to share.
… It's pity Youenn is not there, he would mention a use case with Picture-in-Picture.
… Toggle camera, hang-up are not necessarily linear extensions to what Media Session was offering before.
… There's a question of how it fits in Media Session if it does.
Tommy: I'm not recalling what the Picture
… -in-Picture use case is about.
… We're working on a new Picture-in-Picture that would be open to any HTML element. That would seem to be a better fit for the use case.
… Having some sort of shared spec about actions might have some value. Media session is callback based. Presentations seems to be events-based. Not sure there's advantages to having a shared spec here.
… They are a couple of different ways actions can be routed. In Chrome, it does not have to be the active media session.
… You can send some actions to all media sessions.
… [Example of PiP windows]
… For the slide actions, I don't really know where we would expose those.
… At least in Chrome, we only send those to the active tab. For the keyboard, we would have to guess, at least in theory.
… I don't think we would want to take control of the user's media keys every time there's some ongoing presentation.
cpn: Raised in previous discussion. You may be listening to music while presenting.
Jan-Ivar: Correct but that conflict already exists with toggle microphone.
… Video conferencing tab and Youtube tab. Pause should go to Youtube tab. Toggle microphone should go to the conferencing tab.
Tommy: If PiP, action will go to it.
Jan-Ivar: I don't think the spec talks about that.
Tommy: I haven't looked deeply into this. If it doesn't, then we would need to add that. We're already doing it in Chrome when we send actions to media sessions that are not the current active one.
Jan-Ivar: The spec could formalize that further to mention list of media sessions. You mentioned a valid use case where action needs to be sent to all media sessions.
cpn: I think the spec leaves it quite implementation-defined.
Jan-Ivar: It talks about audio focus.
… One comment on capture handle. Still early and currently uses events. I don't see a problem with registering callbacks.
Tommy: In that case, it may make sense to have a shared registry of actions.
… I don't know that this means we would want to add capture handle actions to Media Session.
Jan-Ivar: The thinking was that it would simplify if there was one place to go for actions and one mechanism to subscribe handlers.
… It would help with adoption and visibility. Capture handle actions by itself may not get a lot of visibility.
cpn: If the two sets of actions become independent, my thought was that sharing would not be useful. You're suggesting that, depending on user-agent features and intents of the page, it may make sense to have a shared list.
Jan-Ivar: It seems to me appealing that a page can be explicit about the list of actions that it can receive, regardless of how these actions are triggered.
Tommy: Would it make sense for the shared actions to be set in the same but not necessarily set on the media session object?
… All through a same object?
Frank: The thing I keep going back to is that capture actions say: here are the things I'm ok receiving. Media sessions are more things that I can handle.
… The site does not really need to do anything with them.
… "It's cool if these sites do this to me" does not exist in Media Session.
… For me, that's the main difference between these two APIs.
… That said, it would be great to have a way for a site to list actions that it can handle and sites that can send the actions. That all thing is not in Media Session right now.
Jan-Ivar: Allowing other sites access?
Frank: Yes. Sometimes they're implicit. Sometimes they're explicit. There's no notion of who is allowed in Media Session. The user agent is allowed in the Media Session.
… The negotiation with the controlling site does not appear in Media Session.
Jan-Ivar: For capture handle actions, the goal is still to keep the user in control. The feature is explicitly behind getUserMedia permission.
… Transient activation is required, so that a web site cannot just randomly send actions.
… It's true that you have less of a guarantee that they represent user intent.
… There are limits to the actions that I would support adding in this way. Clearly, things that would allow scammers to e.g. move mouse would be problematic.
Frank: My question is: if a site registers for media session actions. Is it ok for capture handle actions? Does that mean that the UA need to provide hints? There's this whole question of intents.
… For capturing through media session, what is the intent? I'm OK with the UA sending these actions? I'm OK with anyone sending these actions?
Jan-Ivar: I don't see that bigger difference. Let's say that we had actions that don't opt in to capture handle.
… E.g. for next/previous slide. I don't think that this is one or the other. It could be opt-in. It may still make sense to have a common way to register these handlers.
… I'm hoping we can revisit the discussion once Youenn is around.
cpn: What do we do next? Do we want to setup a dedicated meeting that focuses on this issue with a core team?
… I'm keen on arriving at an agree-able solution.
Jan-Ivar: That would be great for me, especially if we can get Youenn in.
Frank: Likewise.
cpn: OK. Let's work on this offline then.
WebCodecs registrations
cpn: Two proposed registrations. CfC open on H.265 entry. Should close soon.
… For H.263, we have a contributor's proposal. There are a few questions around this. We had some initial discussion with Gary Sullivan which leads some of this work in ITU.
… If we need to go back to ITU-T with questions, I'd like to firm up the list of questions.
… Right now, we have a proposal but not clear whether it's acceptable.
… One of the questions is around packaging of capabilities. Should we try to define a codec string that allows those to be expressed?
… In terms of implementation, I understand that this is a port of ffmpeg.
… I'm worried that we may end up with something that is quite complex to specify.
… If we need external advice, what do we need to ask?
Bernard: Unfortunately, I think these questions will come back with other codecs. For instance, with SVC. And even for OPUS, we didn't include any capability.
… I think it needs to be part of the registration discussion. I think some registrations are not what they should be. There's an open issue on OPUS for instance.
cpn: My immediate concern is H.263 because that has been around for a while and we haven't made much progress.
Bernard: We're trying to enable a few features of ffmpeg, and not the whole H.263. Let's try to scope this to it.
… The second thing that is missing is an editor. I don't feel that we have authors and reviewers.
cpn: Correct, having some help to do this would be great. Suggestions welcome.
[side discussion on the fact that the document is informative, so no particular worries from a patent policy perspective]
Bernard: It might make sense to get a narrow definition of what precise capabilities the proposer needs. I think they were not interested in all the features of ffmpeg for H.263, right?
cpn: That's correct.
Bernard: If it's not going to ship in any browser, we don't need to be thorough on capabilities.
Matt_Wolenetz: For technical spec, we usually require two implementations. If there's only one implementation, I wonder whether it needs to be listed somewhere.
cpn: I think ChrisC was keen on adding the registration when we discussed that initially, even if it was for only one implementation.
… My main concern is, having said yes to add it, we're kind of stalled.
Bernard: It could be worth punting back to the original proposer and making them the owner.
… Between everyone, including Gary, I think we have the right set of people to agree on whatever needs to be agreed on.
cpn: I guess we can go back to the issue with the points that you made. I don't mind picking that up.
Bernard: That's a good resolution.
Jan-Ivar: Isn't there a PR already?
cpn: There is. We just need to firm that up.
TPAC agenda brainstorming
cpn: We have a dedicated session planned for the Friday afternoon of TPAC.
… The organizers asked chairs so that we know how much time we'll allocate for each topic to allow for efficient remote participation.
… In addition to our own meeting, we have a joint meeting with WebRTC.
… Can we figure out before that joined meeting what we'd like to cover there?
… Bernard, you were driving this agenda.
Bernard: Happy to do it this time as well. Focus may be slightly different.
cpn: Suggestions welcome. I'd like to have everything firmed up during August. This is fast approaching.
… I will probably email you directly.
Bernard: So many people are in vacation in August that September is really one month away.
Matt_Wolenetz: What is the expected audience?
… I'm asking because API users feedback would be pretty useful. It really helps prioritize work, e.g. too many features for MSE v2.
Bernard: Crucial indeed. Even if the feedback is not pleasant to hear!
cpn: Understood. That matches discussion in the Media & Entertainment IG from earlier today where we were wondering about gathering feedback from media player library developers.
Next meeting
cpn: August 9th. Possible Media Session meeting before that!