Meeting minutes
WebCodecs - Expose in VideoFrameMetadata some fields from VideoFrameCallbackMetadata
eugene: A bunch of timestamps are considered to be added to the VideoFrameMetadata registry.
… It's not clear to me if this VideoFrameMetadata is going to be that much useful.
… Unless we give all this metadata to encoders/decoders and over the network, all the timestamps will be lost very quickly.
… And that means that app developers will need to manage that metadata themselves (copy, send over the network)
… We're not helping them that much, burden is on them.
… At this point, I would like to hear the opinion of the WG if we even need this VideoFrameMetadata registry.
… Any user for it, given that metadata is very likely to be lost quickly anyway?
… One use case is that this could be used to surface some settings of the video pipeline.
… But I don't know if there's any real need for this.
… We could get rid of the registry altogether.
cpn: I think we need Youenn for this discussion.
… The original issue suggests a couple of RTC-related timestamps. I don't have more details than what he says there.
… I'm trying to recall what motivated us to introduce that in the first place.
… Face detection metadata in RTC scenarios, perhaps?
Eugene: You're right. I had forgotten about it. That's a valid use case.
cpn: The difficulty with that one is that we haven't progressed it. I don't think we have a defined registry entry for this.
Eugene: Exactly, it's been 2-3 years that WebCodecs has been implemented by a couple of implementers and there have been 0 entries to that registry, which suggests it is not really useful.
cpn: Specific question on captureTime and receiveTime and so on. I don't have strong opinion on this.
Eugene: I agree we need Youenn for this. I'll ping him on the GitHub issue.
cpn: Yes, maybe let's pick up on some of the WebRTC people, e.g., Bernard could perhaps talk about the face detection aspects.
… I agree with you otherwise that we can remove if it's unnecessary.
… You said something interesting here: WebCodecs is starting to stabilize.
… That makes me wonder about criteria to get to Candidate Recommendation so that we can secure IPR commitments from everybody.
… Do you still have editors meetings?
Eugene: Yes, we do, and there are outstanding updates, related to enabling people to implement their own SVC mode.
… That's more or less the state of affairs.
cpn: I would perhaps suggest that, once those have made progress, we can define that as the feature scope for transition to Candidate Recommendation.
… Goal is to progress the spec along the Recommendation track. A CR Snapshot transition gives everybody the possibility to exclude essential claims.
… The snapshot becomes a stable spec that can be referenced, while we continue to progress the spec, e.g., focusing on the tests.
… Also need horizontal reviews.
Eugene: We should do the reviews, indeed.
cpn: That's something I'd like to progress for WebCodecs. Whether we do that in a WG meeting or in an Editors call, I don't yet know. Anyway, my ask to the editors is: is the features scope ok or are there additional features that need to be added still?
Eugene: I'll create a GitHub issue so that we can discuss among editors.
Proposal: Pause iframe media when not rendered
Proposal: pause iframe media when not rendered
gabrielbrito: I work for Microsoft. We have this proposal for a new Permission policy to pause a media by iframes which are not currently rendered.
… The iframe could have some resources associated with it, destroying and re-creating them could take time and lead to subpar experiences.
… The interaction between the permission policy and the HTMLMediaElement makes the Media WG a good place to discuss.
… One scenario: the iframe is rendered on the screen, it plays media, and then it becomes non rendered. What should we do?
… Other scenario: the iframe has just been created but is still not rendered, the web site attempts to play media in the iframe (not rendered).
… Idea is to make things work as if the web page does not have autoplay permission for the second scenario.
… For the first one, the web site could treat this as if the user had paused the media. We would fire a "pause" event.
… When the media gets paused because of the permission policy, it does not resume automatically when the iframe gets rendered again.
… Either the website or the user would need to call "play" again.
… I wanted to get feedback from you regarding this proposal.
… non rendered means "display: none" for the time being.
Tommy: This would default to allowed?
gabrielbrito: It would default to "allowed" so that we don't change the current behavior.
… This proposal interfaces with a lot of other APIs. Also interaction with the Web Audio API which we've started to discuss in the context of the Audio WG.
cpn: From an end user perspective, what issue is this addressing?
gabrielbrito: We're aiming at scenario where performances would be compromised, e.g., if we had to destroy a whole subtree.
… Mostly related to performance from an end-user point of view.
… In the GitHub issue I created, someone suggested re-creating the iframe (or detaching/re-attaching) but we think it could lead to performance issue due to the need to re-create things from scratch.
Tommy: In the case where web developers depend on user agents help.
… Do you have web developers asking for this?
gabrielbrito: We do.
cpn: It would be interesting to have other perspectives from other browser vendors, unfortunately not around today.
… I see that Marcos commented on the issue. I don't know if it's a naming issue.
Tommy: I think this one is about the fact that the proposal is only talking about "display: none" whereas there are other ways to make iframes invisible.
gabrielbrito: I may raise issues on standards positions repositories to get feedback from Webkit and Gecko.
cpn: Yes, also happy to arrange a session with them.
gabrielbrito: Wondering where to join a meeting from the HTML group
tidoust: I don't think they have meetings in the way that we do here. More asynchronous mode through GitHub issues.
Encrypted Media Extensions (EME) layering
cpn: This is something I'm here to ask your help and advice.
… To give you some context, we're currently working on the EME spec and it's currently using an older version of ReSpec tooling.
… I'm trying to update things.
… This is throwing a number of errors, for example related to terms defined in HTML but not exported by HTML.
… There's an open pull request against the HTML spec.
… Anne asks how the layering works with EME.
… I'm not sure that's been attempted in practice.
… Part of landing this HTML change means that we need to go back to Anne with some thoughts on the layering.
… A couple of things, covered in separated issues: EME defines "valid MIME type" (both EME and Media Capabilities have their own definitions)
… Following issues in respective repositories, the idea would be to use the MIME Sniff algorithm and then use some additional algorithm steps to determine whether the MIME type is acceptable.
… There's a bit of work to update the specs and remove this concept of "Valid MIME type".
… Slide shows what EME says at the moment.
… Similar definition in Media Capabilities.
… Some slight differences, but the idea is to move from defining validity in this way to a more algorithmic approach.
… The other part of the issue is figuring the interaction points between the specs and the dependencies.
… In a few places, EME is extending HTML in multiple ways.
… Some examples listed here.
… e.g., to reset some internal state. Or patching of HTML behavior in the EME spec.
… I'd like to ask your advice on the recommended approach.
… I'm unsure what the guidance should be.
Tommy: This feels like monkey-patching at a distance. There are going to be steps in the HTML spec for you're going to say "also do that".
cpn: Do you know if there's a recommended approach? In a way, the EME is self-contained. Are there hooks that we can put in place?
Tommy: My main experience is working on Document Picture-in-Picture.
… Lots of places where it patches e.g., navigation specs. "after step 2 in navigation, do this and that..."
… Goal is to merge them upstream once the spec matures.
Bernard: I've heard EME is essentially bringing another codec. Do you need this deep integration or not if you consider that it's a new codec?
joey: That seems like a valid question.
… Not clear what HTML editors want here.
cpn: I'm not here to propose making a lot of changes if we don't need to.
joey: I think we're just missing some exports for the time being.
cpn: We may get back to say that we think that the way we've made it is clear enough.
joey: Yes, I'm happy to address something concrete.
Francois: This kind if issue also affects MSE. In Respec we can force referencing a non-exported term.
Example of "internal" terms referenced by the Presentation API
Bernard: I would separate the Respec issue, which I would consider editorial, from the layering discussion.
cpn: Once HDCP detection is merged, I can go and run some editorial updates on the way to First Public Working Draft.
… Thanks for the feedback. I'll continue to do the editorial bits and respond to the issue to ask for more detailed feedback.
Next meeting
cpn: Next meeting should be on May 14th