13:56:20 RRSAgent has joined #mediawg 13:56:20 logging to https://www.w3.org/2021/06/08-mediawg-irc 13:56:27 RRSAgent, make logs public 13:56:41 Meeting: Media WG monthly call 13:57:00 Agenda: https://github.com/w3c/media-wg/blob/main/meetings/2021-06-08-Media_Working_Group_Teleconference-agenda.md#agenda 13:59:00 Peng_Liu has joined #mediawg 14:00:25 jernoble has joined #mediawg 14:01:17 chcunningham has joined #mediawg 14:02:52 present+ Francois_Daoust, Chris_Needham, Dale_Curtis, Eric_Carlson, Jer_Noble, Peng_Liu, Tess_O_Connor, Youenn_Fablet 14:04:08 present+ Eugene, Dale_Curtis, Dan_Sanders, Gary_Katsevman, Mark_Watson, Nigel_Megitt 14:04:19 nigel has joined #mediawg 14:04:27 Present+ Nigel_Megitt 14:04:39 present+ Lei_Zhai, Chris_Cunningham 14:04:52 scribe: tidoust 14:05:16 present+ Paul_Adenot, Zachary_Cava 14:05:29 Topic: Rechartering update 14:05:45 scribenick: cpn 14:06:01 tidoust: AC review ongoing. Please review the draft charter and vote on it. The more support the better 14:06:13 scribe: tidoust 14:06:21 Topic: TPAC 2021 planning 14:07:01 cpn: TPAC is W3C's annual "F2F" meetings. Proposal is for TPAC to be organized over 2 weeks: one week for joint meetings, one week for breakout sessions. 14:07:13 ... One planned joint meeting with WebRTC WG, e.g. to discuss WebCodecs. 14:07:40 q+ 14:07:40 ... I suggest to focus TPAC on joint meetings. 14:08:40 Tess: If the group can do 1-2 day during that week, that's fine with me. I would recommend against scheduling a group meeting one week before that otherwise TPAC spans too many weeks in a row. 14:08:45 ack chcunningham 14:09:10 chcunningham: Happy to focus on joint meeting given the virtual nature of the meeting. 14:09:19 q+ 14:09:23 present+ 14:09:31 cpn: I can say that the Media & Entertainment IG will want to organize a joint meeting. That's me with another chair hat on. 14:09:38 ... Open for suggestions otherwise! 14:09:43 ack jernoble 14:09:51 Zakim has joined #mediawg 14:10:05 jernoble: If you want to meet, please say so! 14:10:41 Topic: Welcome Alastor Wu as editor for AutoPlay Policy Detection 14:11:13 jernoble: Alastor is not around today, but thank you for volunteering for this job. Looking forward to discussing Autoplay Policy Detection! 14:11:37 cpn: Alastor raised some questions on the spec. We'll need feedback from other implementers. 14:11:43 jernoble: Looking into it. 14:12:27 chcunningham: We'll discuss it internally and get back to you 14:12:51 dale: You can use me as a contact in the meantime. 14:13:51 topic: MSE 14:13:57 q+ 14:14:13 tidoust: The PR means we've added all the main features for MSE. Can we issue a CfC to publish FPWD? 14:14:23 ack jernoble 14:14:59 jernoble: what's the plan for MSE? 14:15:38 tidoust: If there are additional features you think should be in the FPWD, we can wait. FPWD starts a call for exclusions, so it's a good idea to have the features you're planning to have in there 14:16:21 jernoble: MSE in workers has already landed in Chrome. Also playback through gaps in unbuffered ranges. That work has already been done. If it's ready to go, we can start the publication process 14:16:35 tidoust: In that case I'd suggest postponing until those features are specified 14:17:04 dale: Matt's timeline is around June 30th, should have more to say next meeting. We're working towards an origin trial for MSE in workers 14:17:20 jernoble: A good opportunity to tie it all together for the FPWD 14:17:36 scribe+ 14:17:52 topic: WebCodecs issues 14:18:11 jernoble: 3 outstanding issues on WebCodecs 14:18:37 ... Design questions. Want to make sure that everyone gets heard and see whether we can come to consensus. 14:18:50 Subtopic: Should WebCodecs be exposed in Window environments? 14:18:50 https://github.com/w3c/webcodecs/issues/211 14:19:24 chcunningham: WebCodecs are currently exposed on Window and workers 14:19:38 ... WebCodecs does not do heavy lifting on the main thread. 14:20:25 ... It was intuitive to us that this got exposed to window. 14:20:57 ... The points raised by Youenn are about performance concerns: if main thread is with lots of contention, it becomes problematic to queue your inputs and outputs. 14:21:10 ... We think that's why workers are a good fit for many applications. 14:21:16 ... That's not the case for all applications though. 14:21:36 ... There are applications that are not latency-sensitive or that have to deal with main thread contention. 14:21:53 ... During last call, it was suggested that we polled out origin trial users. 14:22:14 ... Many of them didn't have an opinion. Those who expressed opinions favored exposure to window. 14:22:27 ... They reported on applications without performance constrains. 14:23:32 ... Additional points: worker adds complexity. Also integration with APIs that are not currently exposed to workers. This may be the case in the future but it is a pain for developers in the meantime to integrate these APIs together. 14:24:05 ... We did hear from the TAG yesterday that, following the dev survey, they found that it was good to expose to window after all. 14:24:26 Youenn: My perspective is that it is more on what we should prioritize. 14:24:50 ... FWIW, we had a very similar discussion in the WebRTC WG about MediaStreamTrack. 14:25:26 ... It's not finished. My understanding is that we are leaning towards "worker support is great, let's do that first. Window support is second target. We want to focus on worker and optimize for that." 14:25:51 ... With WebCodecs, I'm mostly focusing on video codecs. It's very similar to MediaStreamTrack. 14:25:57 ... We know that we won't regret exposing to workers. 14:26:28 ... We know that, initially, some people will have to create a JS shim with postMessage and the like, which is a bit sad, but it will push other APIs to workers down the line. 14:26:45 ... Moving more stuff out of the main thread is a good thing. 14:27:08 ... Some people are saying that there is no benefit so don't want to pay for the added complexity. 14:27:51 ... That's not entirely true. There will be repetitive tasks in the main thread. There will be GC calls. If you push to workers, you can better handler user events in the main thread. 14:28:32 ... UAs can also decide to lower the priority of transcoding workers that run in the background. More difficult to do if transcoding is running in the main thread. 14:28:46 ... In other words, end users may benefit from the additional work. 14:29:06 ... That's why I believe that we should push them towards that. We may not want to mandate that in the end. 14:29:42 q? 14:29:44 ... Avoid situations where devs say that "it works on my machine" and believe that it's going to work everywhere. 14:30:08 ... Start with things we won't regret. 14:30:26 ... [WebRTC example] 14:30:53 ... This is the world I want people to live in, and the reason I'm pushing for exposing WebCodecs to workers 14:31:39 q+ 14:32:25 ... Media capture track processing in web??.io. We got that feedback from an xCloud dev: "all your examples should be in workers". Main thread demos are going to be copied and pasted, leading to bad design. If you prototype things in window, you may remain in window, that's my fear. 14:32:30 ack chcunningham 14:33:36 chcunningham: Youenn wants to wait to see if this flies. My perspective is that the trial already demonstrates that, as evidenced by feedback received. They found no significant thread contention. 14:33:46 ... They tried workers as well and found no real improvement. 14:34:54 q+ 14:34:54 ... Example of game engine that does decode and render. Performance was good as well, and worried they might have to switch to workers. 14:35:04 ... Did not quite follow the point on GC. 14:35:32 ... Codec work could be de-prioritized too if main thread is contended. 14:35:56 ... Agree that the spec or doc should highlight API use in workers 14:36:45 Youenn: Re. de-prioritizing, you will need to keep the queue, so that's difficult. It could be done with changes. 14:37:06 ... I think we can quickly find consensus and move forward with workers. 14:37:14 ... Less sure about exposition to window. 14:38:04 ... Given where we're at with browsers stacks today, some things that could improve the situation are not there yet. 14:38:47 ... Re. GC, easier to optimize in a worker because you fully control the script. In the main thread, you might not own the entire JS, so more difficult to control when the GC is going to kick in. 14:38:59 q+ 14:40:10 ... [Mentions MediaRecorder as a way to do encoding in the main thread for people who do not want to use workers] 14:40:43 ... We already have some APIs for decoding audio on the main thread. Not perfect for sure. Instead of adding a new API, we could improve them. 14:41:18 ... WebCodecs is going to be successful in low-latency realtime use cases. It may shine in other use cases, but we should get it right in workers first. 14:42:41 eric has joined #mediawg 14:43:07 Eugene: About low-latency use cases, as I see it, having WebCodecs only on workers increases latency. It creates extra-hops between window and workers. Our colleagues from Stadia were concerned that input APIs are not available in workers, so they'd need to inject extra-jumps. 14:44:02 pengliu has joined #mediawg 14:44:21 Youenn: Google Stadia uses WebRTC right now, I believe. Not at a frame-by-frame level. Why the move to WebCodecs? 14:44:45 ... You have data channel in the worker. You pipe it to WebCodecs, then you get a frame, then piping into rendering. 14:44:56 ... If you need to go through network or something else, we're missing something. 14:45:15 Eugene: WebCodecs being a lower level, Google Stadia is exploring and prototyping. 14:45:41 ... I don't want to go into details for them, but that's kind of patronizing for us to assert right ways to do things for them and others. 14:45:49 ... Different designs could be working for them. 14:46:05 ... It's not only about decoding and outputing stuff. 14:46:12 ack jernoble 14:46:48 jernoble: It seems like progress could be made by having specific examples about pages that use WebCodecs, to promote move to workers. 14:47:13 ... There is specification experience in doing things in workers not to contend the main thread, typically for audio rendering. 14:47:38 ... We used to have bits in main thread, and had to backtrack as it was found to be bad design. 14:48:27 chcunningham: WebCodecs exposes decoded audio, but main use case will create a jitter buffer, etc. 14:49:12 jernoble: Re. "patronizing", we're trying to get the best end-user experiences. I don't believe that it is us patronizing. 14:49:39 ... We're not intending to patronize people and tell them how to do their job. We're trying to create best UX for end users. 14:49:54 q+ 14:50:09 ... We may be able to help with complexity by writing demonstration cases that show how to do things in workers. 14:50:42 ... If we are intending to use this API on the window, one thing that this API could improve is providing queueing hooks. 14:51:04 ... so that you can propose multiple frames to the decoder so that it always has something to chew on. 14:51:10 ack chcunningham 14:52:02 chcunningham: I don't follow the "queueing makes GC harder" argument. The queue is an internal detail at that point. 14:52:58 ... Re. GC on window not owning the full script, I feel we're talking past each other. The use cases I have in mind own the JS, they know exactly all the bytes that are going on there. 14:54:07 ... No objection to extending MediaRecorder. But extending it to do things that WebCodecs would do if it were exposed to window seems a bit strange. 14:54:21 ... WebCodecs does not have the same requirements in terms of timing for audio. 14:55:23 ... Re. priority of constituencies, I get that but sites also consider themselves as agents for end users and most are doing a good job at making sure performance and UX are good. 14:55:57 ... TAG was sympathetic to Youenn's points initially, but now we've heard from the users, and TAG is favouring window exposure. 14:56:08 q+ 14:56:14 ... That speaks to my previous points: we've heard that worker is not enough. 14:56:44 jernoble: Sites don't have the ability to control whatever happens in the main thread. Other pages may be running on the same process. 14:57:09 ... The user is the person using the computer. It is our job to be opiniated about this. 14:57:49 ... I don't think using the terms you and Eugene have been using is useful. 14:57:55 q+ 14:58:19 cpn: What do we do next? It seems some higher-bandwith conversation before next monthly call would be useful. 14:58:22 ack cpn 14:58:25 ack hober 14:58:50 hober: There are strong opinions. I'm sad I wasn't in the TAG breakout where this got discussed. 14:59:36 ... Re. "objections have been considered", in a WG like this that runs on consensus, we need to establish consensus first before we move on. 14:59:36 pengliu has joined #mediawg 14:59:36 eric has joined #mediawg 14:59:36 nigel has joined #mediawg 14:59:36 chcunningham has joined #mediawg 14:59:36 jernoble has joined #mediawg 14:59:36 cpn has joined #mediawg 14:59:36 gkatsev has joined #mediawg 14:59:36 QABaglan has joined #mediawg 14:59:36 padenot has joined #mediawg 14:59:36 Germain has joined #mediawg 14:59:36 BrandonCabral has joined #mediawg 14:59:36 hober has joined #mediawg 14:59:36 mounir has joined #mediawg 14:59:36 heejin_ has joined #mediawg 14:59:36 sangwhan has joined #mediawg 14:59:36 gregwhitworth has joined #mediawg 14:59:53 q? 15:00:10 chcunningham: I agree that we do not have consensus for now. 15:00:23 ack chcunningham 15:00:56 ... I would like to suggest that we issue a call for consensus on this issue. The points raised today were mostly a summary of the points raised in GitHub. I feel that we're going in circle. 15:01:11 ... We have heard from developers. We have heard from the TAG. 15:01:25 ... Would it be possible to issue a call for consensus on this? 15:02:03 cpn: My understanding is that a CfC will bring forth statements of positions. We'll establish what we already know that there is not yet a consensus. 15:02:15 ... The problem is how do we resolve that? 15:02:45 ... The W3C process talks about following the path that creates the least dissension. Perhaps that could be the path we follow. 15:03:18 ... If we expose the API to both window and workers, devs will be guided by examples. If they are good, that could be enough. 15:03:53 ... It seems to me that there is another WebRTC angle to this that we could perhaps discuss as this seems to originate from there. 15:04:06 Youenn: Yes, it is true that it follows from another WebRTC issue. 15:05:13 q+ 15:05:34 chcunningham: Re. CfC, I don't expect it to lead to consensus. The process would do two things: it would make positions explicit, and it forces the chairs to break a tie. 15:05:56 jernoble: I don't think that it is the role of the Chairs to break a tie. We're here to encourage consensus. 15:06:00 ack jernoble 15:06:40 jernoble: This conversation is a bit narrow, others should weigh in. 15:07:32 q? 15:07:37 ... It may be useful to find other solutions to concerns raised here, such as good practices, examples, other API features that could solve them without removing the API from window. 15:08:34 cpn: I would forward putting forward a proposed resolution along these lines: with supporting documentation, we would move forward with window and worker. We'd be guiding towards most appropriate usage. 15:08:58 ... Also supportive of running another meeting. Two other issues to discuss. 15:09:20 chcunningham: I personally would be fine with that proposal. Happy to have another meeting. 15:09:40 jernoble: OK, cpn and I, let's figure that out ASAP. 15:09:59 ... Let's document additional points that we discussed today in the GitHub API. 15:10:24 ... Apologies for running overtime! 15:10:36 ... Hope to see everyone in the next conference 15:10:45 RRSAgent, draft minutes 15:10:45 I have made the request to generate https://www.w3.org/2021/06/08-mediawg-minutes.html tidoust 15:16:36 RRSAgent, draft minutes 15:16:36 I have made the request to generate https://www.w3.org/2021/06/08-mediawg-minutes.html tidoust 15:20:26 i/tidoust: The PR means/scribe: cpn 15:20:42 RRSAgent, draft minutes 15:20:42 I have made the request to generate https://www.w3.org/2021/06/08-mediawg-minutes.html tidoust 17:12:25 Zakim has left #mediawg 17:36:36 RRSAgent, bye 17:36:36 I see no action items