20:55:55 RRSAgent has joined #mediawg 20:56:00 logging to https://www.w3.org/2024/05/14-mediawg-irc 20:56:00 Zakim has joined #mediawg 20:56:07 Meeting: Media WG meeting 20:56:48 Agenda: https://github.com/w3c/media-wg/blob/main/meetings/2024-05-14-Media_Working_Group_Teleconference-agenda.md 21:00:53 present+ Chris_Needham, Bernard_Aboba 21:00:57 Chair: Chris_Needham 21:01:08 Present+ Joey_Parrish 21:01:58 Present+ Joey_Parrish, Erik Språng 21:02:26 eugene has joined #mediawg 21:03:02 present+ Eugene_Zemtsov, Mark_Foltz 21:03:26 present+ Marcos_Caceres 21:03:30 chair+ Marcos_Caceres 21:05:38 scribe+ cpn 21:06:35 Marcos has joined #mediawg 21:07:30 CN: we need decide on meeting time for TPAC 21:08:07 CN: I'd like to discuss EME next steps, and Web Codec topics getting the spec to CR 21:08:17 scribe: marcos 21:08:25 TOPIC: TPAC 2024 21:09:05 CN: We put together a plan for TPAC. We plan to meet for a whole day. Tentative it will be on the Thursday. 21:10:44 CN: However, we need to figure out how much time joint meeting time we need. Specifically with the RTC and Web Codecs. The Web Codec folks wanted to about integration points, but we should go to their meetings - possibly in coordination with Kronos. Based on that feedback, we might not meet with WebGPU folks at TPAC 21:10:54 CN: What do folks think about that? 21:11:27 BA: we did a 90 minute slot last time. That's the default? 21:11:59 CN: We would do a 90 minute session with WebRTC. That would be part of the same day. 21:12:12 CN: The same meeting day, that is. 21:12:53 CN: There the discussion with the related groups. And also things that are more for exploration. 21:13:37 CN: topic groupings would be Media. WG issues in progress , Collaboration with other WGs, and Explorations 21:13:51 CN: We will put in the meeting time requests based on this 21:14:02 CN: Anything else we want to discuss around TPAC? 21:15:43 Eugene: Topics that are important for RTC scenarios, Frame drop notifications , Advanced SVC, video encoder buffer controls, inter-frame dependencies 21:16:19 CN: We will get into those details as we pull in the agenda 21:16:29 TOPIC: EME next steps 21:17:09 CN: We have a pull request to add minHdcpVesion (issue #535). The PR is good to land. Got positive feedback 21:17:38 CN: We need guidance to from W3C Team for publication of FPWD 21:18:34 CN: We have a number of other issues labelled as V2. These might not block us from FPWD, but we should triage them. We don't need to look at them now, unless anyone wants to discuss them now 21:19:22 CN: There are a couple that I raised recently. After I did some fixes to the spec, I noticed some bugs so filed them. 21:20:20 JP: I'll take a look. I'll sync up with the others implementation to build consensus. 21:20:40 CN: It would be great to check if any of these are important if any are important for FPWD 21:21:39 JP: I'll go through and close any that need to be closed. I'll try to do some updates and make some kind of hot list or milestone. 21:22:57 CN: There are a number of other issues that we need to think about, #529, #521, things like a "valid mime type" that are defined elsewhere. Annevk raised some issues. 21:23:43 Marcos: I can also help with this 21:23:56 CN: What would layering involved? 21:25:06 MC: EME interacts with HTML media behaviours, conceptually, so even for "valid MIME type", which comes from mimesniff, passing through the parser. So where does EME sit and interface with other parts of the platform 21:25:16 ... and the eventing model, what task queue is used 21:25:49 MC: Happy to go through it in detail 21:26:34 CN: Perhaps we could do together 21:26:49 MC: It's worth doing, as the spec could end up smaller by reusing things from elsewhere 21:27:11 ... There's a structure we can use of inputs and outputs to hook into the wider platform 21:27:56 CN: we are making good progress on EME. Let's get the spec out soon 21:28:21 Topic: VideoFrame Metadata Registry 21:28:58 EZ: Things coming from the WebRTC capture pipeline. It's informational, it doesn't survive encoding or decoding 21:29:09 ... Useful for people working with VideoFrames 21:29:19 ... It's empty, but have suggestions for what to put there 21:30:05 ... One suggestion was to put RTC timestamps there. This is the time of the sender when the frame was created 21:30:16 ... Might be useful to measure lag or synchronize audio and video 21:30:33 ... But it's only exposed when the video comes from a WebRTC channel 21:30:42 ... WebRTC already does synchronisation 21:30:59 ... WebCodecs encoders don't do anything with it. The info is already available in the callback 21:31:11 ... So it looks useful, but don't know what to do with it exactly 21:31:39 ... Another thing proposed to be added is face detection. Certain webcams do face detection 21:31:52 ... This information can be obtained from the capture pipeline at no additional cost 21:32:05 ... Looks like it could be useful for encoders, but WebCodecs doesn't do anything with it 21:32:23 ... But it can be useful to web developers to do video processing on the CPU or GPU 21:32:39 ... It's only for certain cameras and certain circumstances. In most cases we don't have the data 21:32:48 ... If you're video encoding, these will be mostly empty 21:33:04 ... This is in the MediaCapture Extensions spec already 21:33:23 ... We discussed this issue for the WebCodecs spec, but at the same time it's in MCE 21:33:46 ... The info only comes from the capture pipeline, so makes sense there so not clear it should be part of WebCodecs at all 21:34:29 CN: We should check this, there's already a Shape Detection API. Does this overlap? Hope we don't end up with two specs doing the same thing 21:34:58 EZ: There's a subtle distinction here. Shape Detection runs algorithms on the video frames to detect shapes from scratch 21:35:19 ... But this comes from the camera 21:35:31 MC: But it should have the same structure? 21:35:43 BA: It doesn't have the same structure. It's being used for background blur etc 21:35:49 s/CN/MC/ 21:36:22 EZ: Next, there's another MCE extension to show where the background is. It's a background mask 21:36:35 ... It's not in the metadata registry, but we should move it there from VideoFrame 21:36:58 ... So there's a set of possible additions that sound useful 21:37:23 ... But WebCodecs encoders and decoders doesn't have use for the data, as it's not always available, very ad hoc 21:37:34 ... Not clear we need the registry. It's a burden for spec maintenance 21:37:43 ... Why couldn't it be done as part of other specs 21:38:02 ... So I'm suggesting to get rid of the registry and address these things in other specs, if we really need them 21:38:13 ... The registry has been empty for two years 21:39:54 CN: we introduced the registry for different application types, to avoid collisions between them, etc. the motivation was things like face detection metadata. So I'm a bit surprised that we haven't put anything in the registry. We didn't implement the process we had for updating the registry. So we should at least have added the face detection 21:39:54 metadata. 21:40:58 CN: The media working group doesn't take ownership of populating the entries, but we expect other WGs to send us pull requests. It seems unfortunate that no one has sent us PRs for, at least, face detection metadata. 21:42:18 BA: So there's some confusion about what the review should involve. But it doesn't affect WebCodecs 21:42:36 Bernard: The w3c are saying that because of WebIDL, the features can be detected automatically, so we might not need the registry. There is some confusion about why we would use the registry. Is it supposed to just copy the IDL into the registry or are we supposed to get other levels of scrutiny? 21:42:48 CN: happy to revise the process to keep things simple 21:43:37 Eugene: I'll reach out to Youen and Intel folks, who asked about adding those registry entries. We don't have their points of views, but it would be good the get them. 21:43:49 CN: I would like to hear to from Youen 21:44:46 https://github.com/w3c/webcodecs/pull/559 21:44:56 https://github.com/w3c/webcodecs/issues/607 21:45:53 BA: IDL tooling will detect if you have the same dictionary entry multiple times 21:46:14 MC: That depends how it's done 21:46:52 ... You'd have a partial interface in each spec, and combine them and run the parser over them, which should detect 21:47:03 BA: The key thing is to define what we want to achieve 21:47:34 ... What should the review consist of? Is there architectural view 21:48:29 CN: Worth having a review for architectural issues, overlaps or duplications 21:48:55 MC: Yeah, we did that with permissions. Same situation as here, we defined steps to introduce a permission and we'd check them in the registration process 21:49:16 BA: Are there guidelines to review against? 21:49:50 MC: Yes there is a set of criteria, is it specified properly, monkey patching, implementation commitments,. Assures oversight and gives quality 21:50:17 ... We'd lose that if it's purely automated. Do we want that level of scrutiny that needs the coordination? 21:50:47 EZ: We'd have to go to WebRTC WG and MCE spec editors and ask them to revert the change from their spec and make a PR for the registry 21:52:23 CN: We could point the registry entry to MCE? 21:52:38 EZ: That sounds viable, doesn't put much burden on us 21:53:15 CN: PA's concern about the monkey patching approach 21:53:29 MC: Ideally we should have extension specs at all 21:53:41 BA: This is extending a dictionary, not a monkey patch 21:53:47 MC: Even so 21:54:45 ... use of partials outside of the main spec, and it's valid WebIDL, even use of partial can be a concern 21:55:25 CN: But this would be an argument for folding into WebCodecs 21:55:41 MC: Perhaps you and I can look at some examples and figure it out from here 21:56:42 BA: Example of extending a VideoFrame with a VideoFrame, can be an architecture issue 21:56:59 CN: Examples to add? 21:57:12 marcos has joined #mediawg 21:57:49 EZ: RTC timestamps, but background is more speculative. Face detection is in between, mostly fine if webcams really provide it. But concern is if there's a Shape Detection API with its own object 21:59:02 ... But it has little to do with WebCodecs itself, so WC isn't a natural home 21:59:16 CN: So WebRTC WG should be the review group? 21:59:27 BA: That's asking them to review their own stuff 21:59:44 CN: We can provide an independent view in this WG 21:59:52 BA: Have review criteria and guidelines 22:00:02 MC: Yes, we could do that and refine 22:01:32 ... Part of the W3C registry process is to define guidance 22:02:42 https://docs.google.com/presentation/d/1ltl1ZV1KYV02wmFymBGe3q613bM5e3VsZxo-Lu5yn90/edit 22:03:24 Slides above cover the topics we discussed today 22:03:44 CN: So in summary, it sounds like we're not ready to drop the registry yet. We see a need to review entries being added and can add criteria to the registry definition based on what we learn 22:06:06 ... If Paul is OK with it, we could leave the face detection where it is and refer to it from the registry - moving the spec elsewhere wouldn't address the partial dictionary 22:06:35 ... Or move it to its own spec if leaving it in MCE is a problem 22:06:42 [adjourned] 22:06:46 rrsagent, draft minutes 22:06:48 I have made the request to generate https://www.w3.org/2024/05/14-mediawg-minutes.html cpn 22:06:54 rrsagent, make log public 22:18:17 s/scribe: marcos/scribe+ marcos/ 22:18:20 rrsagent, draft minutes 22:18:21 I have made the request to generate https://www.w3.org/2024/05/14-mediawg-minutes.html cpn